From nem@REDACTED Fri May 1 00:09:13 2015 From: nem@REDACTED (Geoff Cant) Date: Thu, 30 Apr 2015 15:09:13 -0700 Subject: [erlang-questions] Measuring message queue delay In-Reply-To: References: Message-ID: <0F89F15B-644E-4467-82F5-40A38DAD5596@erlang.geek.nz> While I don?t have a canned solution for ?always on? monitoring of message queue delay, Erlang?s trace functionality would let you accurately determine the ?receive -> handle? message queue delay. It?s one of the most accurate ways to measure because the Erlang VM itself Timestamps the trace messages (you don?t have to worry about jitter/inaccuracy introduced by needing to capture the timestamp in erlang code). You?d need to build a process that handles messages like: * {trace, Pid, receive, {?$gen_call?, {_Sender,Ref},_Call}, Timestamp} (t0 for a given gen_server:call(Pid, _Call) from Sender at Timestamp) * {trace, Pid, call, {your_module, handle_call, [_Call, {_Sender, Ref}, _State], Timestamp} You?d create an ETS table and insert {{Pid, Ref}, Timestamp} when you see the receive message, then read that back out when you see a ?call? trace message to {Pid, Ref} (The Pid and Ref will be the same between trace messages - this lets you match up message queue inserts (receive) and removes (gen_server calling your callback)). You can extend this scheme to casts and generic messages but it?s harder to match up the receive and the callback. If you?re not monitoring too many processes this way, I?d be comfortable with running this long term on a production node. The process receiving trace messages will get one message for every message sent to the traced gen_server, and one for every function call into your callback module?s ?handle_call? function. With the right match spec on ?call? you can scope this tracing down to a very reasonable volume of trace messages. I don?t have a neat example of using the trace BIFs to do exactly what you want, but I do have a snippet of erlang shell code that shows a similar trace processor for function call/return latency: https://github.com/archaelus/eshellcode/blob/master/dbg/ets_time_tracer.erl Cheers, -Geoff > On Apr 29, 2015, at 2:12 AM, Roger Lipscombe wrote: > > For various reasons, I want a metric that measures how long messages > spend in a process message queue. The process is a gen_server, if that > has any bearing on the solution. Also, this is for production, so it > will be always-on and must be low-impact. > > I could do this by timestamping _every_ message sent to the process > and then reporting the deltas, but I think that's ugly. > > I thought of posting a message to self(), with a timestamp and then > measuring the delta. I could then post another one as soon as that > message is processed. Obviously, this leaves the process continually > looping, which is not good. > > So, instead, I could use erlang:send_after, but that requires two > messages: a delayed one triggered by erlang:send_after, and an > immediate one for measuring the delay. > > That's a bit complicated. > > Would it be sensible to send a message, with erlang:send_after, with > the _expected_ timestamp, and then compute the delta from that? > > Or, alternatively, what other ways could I measure how long a process > is taking to handle its message queue? > > Thanks, > Roger. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From pierrefenoll@REDACTED Fri May 1 01:45:30 2015 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Thu, 30 Apr 2015 16:45:30 -0700 Subject: [erlang-questions] Automatic currying In-Reply-To: <554293C7.7000301@khandkar.net> References: <554293C7.7000301@khandkar.net> Message-ID: I'd like to have a shorthand syntax: 1> Print = fun io:format("lookee here: ~p\n")/1. #Fun 2> Print([ stuff ]). lookee here: stuff ok 3> Print would actually be sugar for Print = fun (X) -> io:format("lookee here: ~p\n", X) end. with all the rules concerning funs applying in the same way. Cheers, -- Pierre Fenoll On 30 April 2015 at 13:42, Siraaj Khandkar wrote: > One of the things I often wish for, when using Erlang, is staged > application of any function without having to manually wrap it in stage > funs (which you get in languages that feature curried-by-default functions). > > The other day it occurred to me that the ability to get fun arity info and > to apply arguments as a list was all that was needed to get very, very > close to the behavior I wanted. Voila: > > -module(function). > -export([curry/1]). > > curry(F) -> > Info = erlang:fun_info(F), > {value, {arity, Arity}} = lists:keysearch(arity, 1, Info), > curry(F, [], Arity). > > curry(F, Args, 0) -> > apply(F, lists:reverse(Args)); > curry(F, Args, Arity) -> > fun (X) -> curry(F, [X | Args], Arity - 1) end. > > In action: > > $ erl > 1> > 1> Nums = lists:seq(1, 10). > [1,2,3,4,5,6,7,8,9,10] > 2> > 2> Plus = function:curry(fun (X, Y) -> X + Y end). > #Fun > 3> > 3> lists:map(Plus(1), Nums). > [2,3,4,5,6,7,8,9,10,11] > 4> > 4> lists:map(Plus(3), Nums). > [4,5,6,7,8,9,10,11,12,13] > 5> > 5> lists:map(Plus(5), Nums). > [6,7,8,9,10,11,12,13,14,15] > 6> > 6> Fold = function:curry(fun lists:foldl/3). > #Fun > 7> > 7> AddTo = Fold(fun (X, Y) -> X + Y end). > #Fun > 8> > 8> AddTo0 = AddTo(0). > #Fun > 9> > 9> AddTo100 = AddTo(100). > #Fun > 10> > 10> AddTo0(Nums). > 55 > 11> AddTo100(Nums). > 155 > 12> > 12> AddTo100(lists:seq(4, 78)). > 3175 > 13> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri May 1 02:01:56 2015 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 30 Apr 2015 17:01:56 -0700 Subject: [erlang-questions] Measuring message queue delay In-Reply-To: References: Message-ID: <5542C274.5080408@gmail.com> On 04/29/2015 02:12 AM, Roger Lipscombe wrote: > For various reasons, I want a metric that measures how long messages > spend in a process message queue. The process is a gen_server, if that > has any bearing on the solution. Also, this is for production, so it > will be always-on and must be low-impact. > > I could do this by timestamping _every_ message sent to the process > and then reporting the deltas, but I think that's ugly. > > I thought of posting a message to self(), with a timestamp and then > measuring the delta. I could then post another one as soon as that > message is processed. Obviously, this leaves the process continually > looping, which is not good. > > So, instead, I could use erlang:send_after, but that requires two > messages: a delayed one triggered by erlang:send_after, and an > immediate one for measuring the delay. If you used a CloudI service (http://cloudi.org) for the logic the gen_server process is currently doing (using the cloudi_service behaviour instead), the CloudI service request timeout would be decremented based on any delays the CloudI service request encounters, including queuing delays (queuing delays are always decremented, there is a service configuration option to include the delay of actually handling the service request and a separate option to include the delay of actually providing the response, request_timeout_adjustment and response_timeout_adjustment respectively). If you want to replicate just the Erlang source code that is handling the queuing delay, you can do that with a single erlang:send_after timer (you don't need two as you suggested) to fire with the timeout value of the request after taking the request from the process message queue (you need to not block the gen_server process and consume messages as quick as they arrive adding them to a queue in process heap memory). Once you are able to actually handle the request, you then take the request from the queue in the process heap memory, and cancel the timer which provides you with the time remaining. Comparing the time remaining to the timeout you initially provided to erlang:send_after gives you the time elapsed. However, I find it easier to use the logic already implemented for CloudI services. From ok@REDACTED Fri May 1 02:46:48 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 1 May 2015 12:46:48 +1200 Subject: [erlang-questions] Automatic currying In-Reply-To: <554293C7.7000301@khandkar.net> References: <554293C7.7000301@khandkar.net> Message-ID: <861F6F0F-E405-4E56-BC5B-FC24ED3EA9B5@cs.otago.ac.nz> On 1/05/2015, at 8:42 am, Siraaj Khandkar wrote: > > curry(F) -> > Info = erlang:fun_info(F), > {value, {arity, Arity}} = lists:keysearch(arity, 1, Info), > curry(F, [], Arity). You do know that you can write {arity, Arity} = erlang:fun_info(F, arity) ? > > curry(F, Args, 0) -> > apply(F, lists:reverse(Args)); > curry(F, Args, Arity) -> > fun (X) -> curry(F, [X | Args], Arity - 1) end. This is where it gets hairy. The thing is that in a language like Haskell or Clean, there is no difference between f and f {- applied to no arguments -} In Erlang, there is a big difference between F and F() What if I have F = fun (X) -> ... end and I want (curry(F))(X) to give me fun () -> ... X ... end? Having curry(F, Args, 0) -> Rev = lists:reverse(ARgs), fun () -> apply(F, Rev). would allow people to invoke the result or not, as they chose. I suggest *not* doing any of this. Haskell tends to involve stuff like (`f`y) for providing the second argument, which looks distinctly odd when f has more than 2, and don't let's mention `flip' and its relatives. Why do I suggest not doing this? First, as already mentioned, the fact that Erlang functions are strict and effectful means that the distinction between F and F() is vital. Second, this isn't really how Erlang is _meant_ to be used, so unlike Haskell, the compiler is not going to put much effort into making this less slow. Most important, other people aren't going to expect to read Erlang code written this way, so it's not going to be as maintainable as possibly longer code that's more direct. From siraaj@REDACTED Fri May 1 08:02:51 2015 From: siraaj@REDACTED (Siraaj Khandkar) Date: Fri, 01 May 2015 02:02:51 -0400 Subject: [erlang-questions] Automatic currying In-Reply-To: <861F6F0F-E405-4E56-BC5B-FC24ED3EA9B5@cs.otago.ac.nz> References: <554293C7.7000301@khandkar.net> <861F6F0F-E405-4E56-BC5B-FC24ED3EA9B5@cs.otago.ac.nz> Message-ID: <5543170B.2030602@khandkar.net> On 4/30/15 8:46 PM, Richard A. O'Keefe wrote: > > On 1/05/2015, at 8:42 am, Siraaj Khandkar wrote: > >> >> curry(F) -> >> Info = erlang:fun_info(F), >> {value, {arity, Arity}} = lists:keysearch(arity, 1, Info), >> curry(F, [], Arity). > > You do know that you can write > {arity, Arity} = erlang:fun_info(F, arity) ? Did not. Nice! >> curry(F, Args, 0) -> >> apply(F, lists:reverse(Args)); >> curry(F, Args, Arity) -> >> fun (X) -> curry(F, [X | Args], Arity - 1) end. > > This is where it gets hairy. The thing is that in a language > like Haskell or Clean, there is no difference between > > f > > and > > f {- applied to no arguments -} > > In Erlang, there is a big difference between > > F > > and > > F() This comparison conflates explicit staged application with implicit delayed evaluation. Semantically, the unit is still there even in Haskell, as the last argument of the thunk, it's just that it is implicit and when it gets (ostensibly) applied is not up to you (yes, I know there're ways to force it, but that is besides this point). Another strict language would be a better choice for comparison. Using SML, the equivalents would be: Value: SML: f Erlang: F Thunk activation: SML: f () Erlang: F() Very similar, with the unfortunate difference of () not being a value in Erlang. > What if I have > > F = fun (X) -> ... end > > and I want > > (curry(F))(X) > > to give me fun () -> ... X ... end? If you actually do want the final output to be a thunk, then that thunk has to be returned by the original, non-curried, function. i.e. instead of fun (X) -> .. end you can have fun (X) -> fun () -> .. end end or, alternatively, just use a value of your choice as an ostensible unit: fun (X, unit) -> .. end fun (X, {}) -> .. end So the last argument would have no meaning other than activation. Again, _if_ this is what you want - you ask for it explicitly. > Having > > curry(F, Args, 0) -> > Rev = lists:reverse(ARgs), > fun () -> apply(F, Rev). This breaks the contract of currying - it does something I did not ask it to do. > would allow people to invoke the result or not, as they chose. The opposite - forcing a thunk on them would take the choice away from them (besides just breaking the contract). > I suggest *not* doing any of this. > Haskell tends to involve stuff like (`f`y) for > providing the second argument, which looks distinctly > odd when f has more than 2, and don't let's mention > `flip' and its relatives. > > Why do I suggest not doing this? > First, as already mentioned, the fact that Erlang > functions are strict and effectful means that the > distinction between F and F() is vital. Addressed above. Currying has nothing to do with laziness. > Second, this isn't really how Erlang is _meant_ to > be used, Erlang was not "meant" to be used in any of the ways it is being used right now. lambdas were added relatively late (which means it was not meant for any of the staple FP list processing functions like map/filter/fold). > so unlike Haskell, the compiler is not > going to put much effort into making this less slow. Sure. We're no worse off here than with any other use of funs. If performance is an issue in some area of the code, than you optimize as needed. There's no conflict. Remember, I'm not advocating for anything radical, like overlaying all existing functions with curry/1 and only using the curried form (that would be a horrible user experience, given the Erlang syntax). I'm simply sharing a technique which can be used when you do want it, explicitly and judiciously. > Most important, other people aren't going to expect > to read Erlang code written this way, so it's not > going to be as maintainable as possibly longer code > that's more direct. As mentioned above - I'm not saying to curry all (or even most) functions this way - I'm saying that if/when one desires something like this - this is a convenient technique :) From dszoboszlay@REDACTED Fri May 1 10:01:13 2015 From: dszoboszlay@REDACTED (=?UTF-8?Q?D=C3=A1niel_Szoboszlay?=) Date: Fri, 1 May 2015 10:01:13 +0200 Subject: [erlang-questions] OTP in FIPS mode ? In-Reply-To: References: <276c51c38412b4daff5fc26285f48f2c@teksavvy.com> <55367BB8.2040800@tail-f.com> Message-ID: Hi, When I was working on the FIPS branch the first step was switching to the EVP-API. Interestingly, as Ingela pointed out, the EVP-API doesn't cover all the functionality in OTP's crypto module. IIRC the public key primitives were not available, but in that case even the low level OpenSSL API seems to work in FIPS mode (which means those calls do end up in the FIPS Object Module), so I left them as they are. Besides the API-change I also: - Blocked non-compliant calls in FIPS mode before they would reach OpenSSL (so you get an Erlang error instead of killing your VM). This is a must have for any FIPS fork, but it was quite trivial to implement. - Modified the ssl code so it won't offer the disabled ciphers in FIPS mode. Interestingly, it's always been possible to query the list of supported algorithms from crypto, but most users of this application simply assumed their algorithms were always available. This is not a big change and would be very useful in FIPS mode, and you can just cherry-pick my commit (there may be conflicts with other OTP versions though, I'm not sure). - I found a couple of places where week crypto algorithms (or keys) were used. Mostly tests, but there was a dict somewhere with md5 hashes as keys. These had to be switched to something FIPS compliant. I think you can directly cherry-pick my commits, conflicts are not likely. - I also made a huge refactoring in the crypto NIF code that is not really necessary for FIPS support. From R16B to R16B-01 the crypto.erl API was refactored: originally there were different functions for e.g. md5, sha1 and so on, from R16B-01 there are generic hash/encrypt/... functions that take the algorithm's name as an argument. However, the change was only in the Erlang layer: the C code was still using the old per-algorithm functions and the Erlang code was translating the new API to the old API of the NIFs. But the EVP-API of OpenSSL is essentially the same as the new Erlang API, so I switched over the C code to this new design as well. I think this commit looks very frightening, because it changes a good portion of the C module. But it is not strictly speaking necessary for the FIPS support at all. And I am personally also happy that I could throw away 800-1000 lines of C code. :) Anyway, if you need FIPS mode, I would recommend you to fork OTP, take the crypto application as it is from my branch and cherry-pick the commits to non-crypto code on top of the OTP version of your choice. Rebasing my changes of the crypto application to other OTP versions is not easy, you would get a lot of conflicts*. But since 17.0 I think only a couple of new algorithms were added to crypto, you can probably live without them in a FIPS environment. Cheers, Daniel * One of my colleagues at ESL ported the FIPS branch to a newer OTP release (17.4 maybe?), but to be honest I forgot who did it, and I couldn't find his repo between the many forks of OTP on github. 2015-04-23 15:52 GMT+02:00 Ingela Andin : > Hi! > > This question is in the OTP 19 scope. FIPS requires the use of the > EVP-API which sounds interesting regardless of > FIPS. We see this as a possible first step, then we can consider if and > what else we might want to do, to support or facilitate for others to > support FIPS. My current understanding is that using only the EVP-API > functions we can not make the crypto API completely functional, which > will require at least some additions to the crypto API and perhaps > OTP-techincal board considerations. > > Regards Ingela Erlang/OTP - Ericsson AB > > 2015-04-21 20:22 GMT+02:00 Drew Varner : > >> Here? s the discussion on a FIPS pull request that?s now closed: >> https://github.com/erlang/otp/pull/377 >> >> - Drew >> >> >> On Apr 21, 2015, at 12:32 PM, Niclas Eklund wrote: >> >> Hi! >> >> IMHO I think that it would be good if FIPS could supported by OTP, >> especially since the purpose of the FIPS standards are issued to ensure >> computer security and interoperability. I've seen a question about this at >> least once before on this list before - >> http://erlang.org/pipermail/erlang-questions/2012-April/065902.html But >> I don't know what became of it. >> >> Best regards, >> >> Nick >> >> >> On 04/21/2015 03:48 PM, jonetsu wrote: >> >> Hello, >> >> We are using an Erlang-based middleware using OTP, ConfD, which >> must now support FIPS mode. Briefly, FIPS is a U.S. standard >> that imposes a set of crypto parameters (ciphers, algorithms, >> etc...). FIPS-applications must use high-level OpenSSL >> methods (The EVP set of methods) since the low-level functions >> will make OpenSSL abort. The application must also call >> FIPS_mode_set(1) to enable this mode for a suitable OpenSSL build >> that supports FIPS. >> >> OTP uses low-level OpenSSL functions. >> >> Initially I considered replacing, for instance, the AES_* uses in >> crypto.c by their EVP equivalent, while keeping the interface to >> Erlang intact. >> >> Now, looking at the extent of the FIPS modifications to the OTP >> code done last year by D?niel Szoboszlay, who worked at Ericsson >> and Erlang Solutions, I wonder about my na?ve approach. >> >> Are anyone here familiar with this FIPS OTP port ? Any comments >> ? To anyone also familiar with ConfD: do you know of any effort >> done in using this FIPS-enabled OTP code ? >> >> Thanks for any comments and suggestions ! >> >> Regards. >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peppe@REDACTED Fri May 1 10:13:12 2015 From: peppe@REDACTED (Peter Andersson) Date: Fri, 1 May 2015 10:13:12 +0200 Subject: [erlang-questions] Error in Common Tests In-Reply-To: References: <5540B947.1050705@ninenines.eu> <5540DC58.6020307@erlang.org> Message-ID: <55433598.7010007@erlang.org> This is fixed now btw, and will be released soon. Best, Peter On 2015-04-30 15:47, Roberto Ostinelli wrote: > Thank you Peter! :) > > On Wed, Apr 29, 2015 at 3:27 PM, Peter Andersson > wrote: > > > Roberto, > > You won't get test case line info if it's the last line of > execution that causes the timetrap timeout. Because of tail call > elimination the {Suite,Testcase,Line} info is not saved on stack > unless there's a trailing expression. I.e. > > tc(_) -> func_that_may_not_return(). > > will not generate test case line info if it fails, but > > tc(_) -> func_that_may_not_return(), ok. > > will. > > That said, Common Test removes internal common_test and > test_server elements from the stack before printing the stack > trace to the log. This operation is too agressive and removes all > elements if no {Suite,Testcase,Line} tuple is found "to sync > with". Knowing it must be the last call in the test case causing > the timeout and knowing which internal function executed the test > case, I should be able to print relevant stack info to the log > even when a test case tuple is not present due to tail call > elimination. Fix is coming asap... > > Best, > Peter > > Ericsson AB, Erlang/OTP > > > > On 2015-04-29 13:15, Roberto Ostinelli wrote: >> Lo?c, >> Yes that did help me. It just stated {error,eoc} (i.e. no line of >> where this happened) but at least I could track it down to the issue. >> >> Thank you. >> r. >> >> On Wed, Apr 29, 2015 at 12:58 PM, Lo?c Hoguin > > wrote: >> >> Did you look inside the HTML logs that Common Test generates? >> They usually contain the complete error and stacktrace, while >> the console only gets partials. >> >> >> On 04/29/2015 01:54 PM, Roberto Ostinelli wrote: >> >> Dear list, >> When running CT I have a test where I get: >> >> - - - - - - - - - - - - - - - - - - - - - - - - - - >> Error detected: {'EXIT',{badmatch,{...}}} >> - - - - - - - - - - - - - - - - - - - - - - - - - - >> >> No additional information. My application logs do not >> show any errors. >> >> Weirdly enough, if I put a timer:sleep(10) somewhere in >> my code, this >> error does not show up anymore. >> >> Thing is: how can I get CT to tell me the module / line >> where this error >> happens? >> >> Thank you, >> r. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> -- >> Lo?c Hoguin >> http://ninenines.eu >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex@REDACTED Fri May 1 13:08:07 2015 From: alex@REDACTED (Alex Wilson) Date: Fri, 01 May 2015 21:08:07 +1000 Subject: [erlang-questions] Automatic currying In-Reply-To: <554293C7.7000301@khandkar.net> References: <554293C7.7000301@khandkar.net> Message-ID: <1430478487.2067.1.camel@cooperi.net> On Thu, 2015-04-30 at 16:42 -0400, Siraaj Khandkar wrote: > One of the things I often wish for, when using Erlang, is staged > application of any function without having to manually wrap it in stage > funs (which you get in languages that feature curried-by-default functions). Have you seen the Scheme-style "cut" parse_transform in erlando? https://github.com/rabbitmq/erlando#cut It's not currying, but you can often use it to meet a lot of the same needs. From Andras.Bekes@REDACTED Fri May 1 13:53:13 2015 From: Andras.Bekes@REDACTED (Bekes, Andras G) Date: Fri, 1 May 2015 11:53:13 +0000 Subject: [erlang-questions] {error,closed} vs. {error,econnreset} In-Reply-To: <54E4A07E.5010307@erlang.org> References: <54E4A07E.5010307@erlang.org> Message-ID: Hi Zandra, I tried tracing on all prim_inet module function calls, plus gen_tcp plus all messages, but did not find the econnreset or resv0 strings anywhere. I used the following code: dbg:tpl(prim_inet,[{'_',[],[{exception_trace}]}]). dbg:tpl(gen_tcp,[{'_',[],[{exception_trace}]}]). f(F),F=fun()->dbg:p(self(),[call,m,sos,sol]), {ok,Socket}=TCP_CONNECT("localhost",12345), ok=gen_tcp:send(Socket,<<"a:b",0>>),io:format("~p:~p\n",[self(),gen_tcp:recv(Socket,100)]) end. Please see the trace in the attached file. Do you have suggestions on what else shall I trace? Thanks, Andras From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Zandra Hird Sent: Wednesday, February 18, 2015 3:24 PM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] {error,closed} vs. {error,econnreset} Hi, yes this sounds confusing. We looked into it a little bit but couldn't find the reason for it right away. Have you tried tracing the resv0 in prim_inet? It might give you some more information about why this happens. We can look into it some more too when we get the time, and if you don't find out more before that :) / Zandra On 2015-02-10 14:39, Bekes, Andras G wrote: Hi All, Looks like I have a problem with the erroneous result of gen_tcp:recv/2, as it returns {error,closed} instead of my expectation of {error,econnreset}. I can reproduce my problem by: 1, starting a server application and forcing it into a busy state when it is listening but not accepting connections 2,starting 1000 clients (in Erlang) that connect to the server, filling the TCP listen backlog of the server. When the backlog is filled and after some time (~2 minutes), the Erlang connections return {error,closed}. This is what I do in Erlang: fun()->{ok,Socket}= TCPCONNECT(HOST,PORT), ok=gen_tcp:send(Socket,<<"SOMEDATA...">>), io:format("~p:~p\n",[self(),gen_tcp:recv(Socket,100)]) end. spawn this 1000 times. 3, a native client of the service however tells me that the connection was reset by peer. According to strace, it executes a recvfrom which returns ECONNRESET: recvfrom(...) = -1 ECONNRESET (Connection reset by peer) I also straced the Erlang beam process and it indeed gets an ECONNRESET result for the recvfrom call, but the Erlang return value is {error,closed}. Unfortunately it looks like I really need to separate these two results. I also tried gen_tcp in active mode, but no difference, the result is {error,closed} instead of {error,econnreset}. Can someone explain why the econnreset error is masked? Is there any way I can separate the two kinds of events? Thanks, Andras G. Bekes _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: econnreset_trace.txt URL: From Andras.Bekes@REDACTED Fri May 1 14:11:08 2015 From: Andras.Bekes@REDACTED (Bekes, Andras G) Date: Fri, 1 May 2015 12:11:08 +0000 Subject: [erlang-questions] {error,closed} vs. {error,econnreset} References: <54E4A07E.5010307@erlang.org> Message-ID: I forgot to mention that I was using Erlang R16B03 (erts-5.10.4) and the OS process indeed received the ECONNRESET result, according to strace. From: Bekes, Andras G (ICT) Sent: Friday, May 01, 2015 1:53 PM To: 'Zandra Hird'; erlang-questions@REDACTED Subject: RE: [erlang-questions] {error,closed} vs. {error,econnreset} Hi Zandra, I tried tracing on all prim_inet module function calls, plus gen_tcp plus all messages, but did not find the econnreset or resv0 strings anywhere. I used the following code: dbg:tpl(prim_inet,[{'_',[],[{exception_trace}]}]). dbg:tpl(gen_tcp,[{'_',[],[{exception_trace}]}]). f(F),F=fun()->dbg:p(self(),[call,m,sos,sol]), {ok,Socket}=TCP_CONNECT("localhost",12345), ok=gen_tcp:send(Socket,<<"a:b",0>>),io:format("~p:~p\n",[self(),gen_tcp:recv(Socket,100)]) end. Please see the trace in the attached file. Do you have suggestions on what else shall I trace? Thanks, Andras From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Zandra Hird Sent: Wednesday, February 18, 2015 3:24 PM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] {error,closed} vs. {error,econnreset} Hi, yes this sounds confusing. We looked into it a little bit but couldn't find the reason for it right away. Have you tried tracing the resv0 in prim_inet? It might give you some more information about why this happens. We can look into it some more too when we get the time, and if you don't find out more before that :) / Zandra On 2015-02-10 14:39, Bekes, Andras G wrote: Hi All, Looks like I have a problem with the erroneous result of gen_tcp:recv/2, as it returns {error,closed} instead of my expectation of {error,econnreset}. I can reproduce my problem by: 1, starting a server application and forcing it into a busy state when it is listening but not accepting connections 2,starting 1000 clients (in Erlang) that connect to the server, filling the TCP listen backlog of the server. When the backlog is filled and after some time (~2 minutes), the Erlang connections return {error,closed}. This is what I do in Erlang: fun()->{ok,Socket}= TCPCONNECT(HOST,PORT), ok=gen_tcp:send(Socket,<<"SOMEDATA...">>), io:format("~p:~p\n",[self(),gen_tcp:recv(Socket,100)]) end. spawn this 1000 times. 3, a native client of the service however tells me that the connection was reset by peer. According to strace, it executes a recvfrom which returns ECONNRESET: recvfrom(...) = -1 ECONNRESET (Connection reset by peer) I also straced the Erlang beam process and it indeed gets an ECONNRESET result for the recvfrom call, but the Erlang return value is {error,closed}. Unfortunately it looks like I really need to separate these two results. I also tried gen_tcp in active mode, but no difference, the result is {error,closed} instead of {error,econnreset}. Can someone explain why the econnreset error is masked? Is there any way I can separate the two kinds of events? Thanks, Andras G. Bekes _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonetsu@REDACTED Fri May 1 15:03:46 2015 From: jonetsu@REDACTED (jonetsu) Date: Fri, 01 May 2015 09:03:46 -0400 Subject: [erlang-questions] OTP in FIPS mode ? In-Reply-To: References: <276c51c38412b4daff5fc26285f48f2c@teksavvy.com> <55367BB8.2040800@tail-f.com> Message-ID: -----Original Message----- From: "D?niel Szoboszlay" Date: 05/01/15 04:01 Hi, Thanks for your comments, much appreciated ! > - Blocked non-compliant calls in FIPS mode before they would reach >? OpenSSL (so you get an Erlang error instead of killing your >? VM). This is a must have for any FIPS fork, but it was quite >? trivial to implement. That would be the CHECK_NO_FIPS_MODE in crypto.c.? Is there any at the Erlang level ? I'm asking beacause what I am considering at the moment is to only modify the crypto.c code.? No modification to Erlang code. For two reasons.? One is that I do not know Erlang, although by browsing the code lately I find it quite interesting :) But nowhere near being able to modify an application that is used in the field.? Let alone establishing test harnesses in the first place. Second is, the OTP that is used is already modified by tail-f AG as part of the ConfD product.? For instance, if I'm not mistaken, the SSL component is different, with crypto being the only part used from OpenSSL. I have the impression that the OTP base used in the product dates from some time.? It is possible to compile locally crypto.c, but when it comes to altering the company's Erlang code then all support is lost. This approach also means to keep the C <-> Erlang interface intact. Do you think it is at all possible to have a working FIPS mode without any modification to Erlang code ? Regards. -------------- next part -------------- An HTML attachment was scrubbed... URL: From noiddicle@REDACTED Fri May 1 16:35:49 2015 From: noiddicle@REDACTED (Red Davies) Date: Fri, 1 May 2015 10:35:49 -0400 Subject: [erlang-questions] SSL certificate examination (otp/lib/ssl and otp/lib/public_key questions) Message-ID: Greetings, I'm wanting to write code which makes an SSL connection to a remote server in order to download an analyze the certificate and the certificate chain. As OTP already has functions for doing certificate validation given a chunk of CA certificates I didn't want to re-invent that wheel with my own "personal brand" of bugs. The issue I'm having is that I'm failing SSL certificate validation for some websites for reasons that I don't seem to be able to ascertain using verify_peer and providing a cacertfile. Specifically, I get the following error: 10:06:19.142 [error] SSL: :certify: ssl_handshake.erl:1403:Fatal error: handshake failure and the following tuple: {:tls_alert, 'handshake failure'} When I see that kind of message it makes me think of protocol negotiation errors such as not being able to agree on a cipher. Certificate failures I would expect to come in the format found in ssl_alert.erl. My confusion lies in when I run the same request without any verification (empty options) and the connection negotiates flawlessly. When I watch the SSL negotiation on the wire using wireshark it all seems to be going well until the client (erlang) terminates the session with a handshake-failure. I'm still very VERY new to the erlang ecosystem so please forgive my questions if they lean too far on the elementary side but I'm looking for guidance on how to further debug this. Ideally, I would like to find out exactly where in ssl/public_key the negotiation is failing but I don't have sufficient experience in the language to know how to approach this. Looking at the sources and following along in my head I'm pretty sure it's failing in public_key.erl in one of the catch-all clauses but I have no idea how to approach tracing activity through such a module so I can either fix it or supply an additional untested failure-mode upstream. Can I step through otp library code execution and examine values at each step? Is this even a good idea? If the answer is a thousand lines of debugging is there a best-practice for making such (temporary) modifications for debugging? Am I going in completely the wrong direction? Any pointers to resources that could help me take this debugging process to the next level would be appreciated. If it make any difference, the certificate it's failing on is intermittently www.google.com and gmail.com (intermittent because the IPs keep moving, not because behaviour is changing in the application). The certificate for my website validates flawlessly consistently. Thanks, Red From sid5@REDACTED Fri May 1 21:54:30 2015 From: sid5@REDACTED (Sid Muller) Date: Fri, 1 May 2015 21:54:30 +0200 Subject: [erlang-questions] Server stops responding Message-ID: An HTML attachment was scrubbed... URL: From vinoski@REDACTED Fri May 1 23:35:55 2015 From: vinoski@REDACTED (Steve Vinoski) Date: Fri, 1 May 2015 17:35:55 -0400 Subject: [erlang-questions] Server stops responding In-Reply-To: References: Message-ID: On Fri, May 1, 2015 at 3:54 PM, Sid Muller wrote: > Hi, > > Can someone explain why the code from afile_server responds only once. If > I send the Pid a different message it just hangs the shell. > > Here is the code below: > -module(afile_server). > -export([start/1, loop/1]). > start(Dir) -> spawn(afile_server, loop, [Dir]). > loop(Dir) -> > receive > {Client, list_dir} -> > Client ! {self(), file:list_dir(Dir)}, > loop(Dir); > {Client, {get_file, File}} -> > Full = filename:join(Dir, File), > Client ! {self(), file:read_file(Full)}, > loop(Dir) > end, > loop(Dir). > > We compile it and run it: > 1> c(afile_server). > {ok,afile_server} > 2> Pid = afile_server:start("."). > <0.40.0> > > We send it a list_dir message: >

> 3> Pid ! {self(), list_dir}.
> {<0.33.0>,list_dir}
>
> And receive reponse just fine:
> 4> receive X -> X end.
> {<0.40.0>,
>  {ok,["afile_server.erl", "afile_server.beam"]}}
>

You've just received X...


> Send the same message again:
> 5> Pid ! {self(), list_dir}.
> {<0.33.0>,list_dir}
>
> And receive response, no problem:
> 6> receive X -> X end.
> {<0.40.0>,
>  {ok,["afile_server.erl", "afile_server.beam"]}}
>


...and now you've received the same X...


> Now we send a get_file message:
> 7> Pid ! {self(), {get_file, "afile_server.erl"}}.
> {<0.33.0>,{get_file,"afile_server.erl"}}
>
> But receive hangs:
> 8> receive X -> X end.
>

...and now you're trying to receive X, which is already bound to the
previous message, but that's not the message you're getting. If you were to
use some variable name other than X, a variable that's not yet bound, it
would work fine.

--steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From siraaj@REDACTED  Sat May  2 06:22:41 2015
From: siraaj@REDACTED (Siraaj Khandkar)
Date: Sat, 02 May 2015 00:22:41 -0400
Subject: [erlang-questions] Automatic currying
In-Reply-To: 
References: <554293C7.7000301@khandkar.net>
 
Message-ID: <55445111.40805@khandkar.net>

On 4/30/15 7:45 PM, Pierre Fenoll wrote:
> I'd like to have a shorthand syntax:
>
> 1> Print = fun io:format("lookee here: ~p\n")/1.
> #Fun
> 2> Print([ stuff ]).
> lookee here: stuff
> ok
> 3>
>
> Print would actually be sugar for
> Print = fun (X) -> io:format("lookee here: ~p\n", X) end.
> with all the rules concerning funs applying in the same way.

Maybe like this:

     1>
     1> Print = function:curry(fun (Prefix, Term) -> io:format("~s~p~n", 
[Prefix, Term]) end).
     #Fun
     2>
     2> PrintNaked = Print("").
     #Fun
     3> PrintLookee = Print("lookee here: ").
     #Fun
     4>
     4> PrintNaked(queue:new()).
     {[],[]}
     ok
     5>
     5> PrintLookee(queue:new()).
     lookee here: {[],[]}
     ok
     6>


From sid5@REDACTED  Sat May  2 00:20:54 2015
From: sid5@REDACTED (Sid Muller)
Date: Sat, 2 May 2015 00:20:54 +0200
Subject: [erlang-questions] Server stops responding
In-Reply-To: 
References: ,
 
Message-ID: 

An HTML attachment was scrubbed...
URL: 

From rory@REDACTED  Sat May  2 12:32:09 2015
From: rory@REDACTED (Rory Byrne)
Date: Sat, 2 May 2015 11:32:09 +0100
Subject: [erlang-questions] {error,closed} vs. {error,econnreset}
In-Reply-To: 
References: 
 <54E4A07E.5010307@erlang.org>
 
Message-ID: <20150502103209.GA19880@nybek.com>

Hi Andras,

On Fri, May 01, 2015 at 11:53:13AM +0000, Bekes, Andras G wrote:
> 
> Please see the trace in the attached file. Do you have suggestions on what else shall I trace?
> 
>>> On 2015-02-10 14:39, Bekes, Andras G wrote:
>>> 
>>> Looks like I have a problem with the erroneous result of gen_tcp:recv/2, as it returns {error,closed} instead of my expectation of {error,econnreset}.
>>> 
...
>>> 
>>> Unfortunately it looks like I really need to separate these two results.
>>> 
>>> I also tried gen_tcp in active mode, but no difference, the result is {error,closed} instead of {error,econnreset}.
>>> 
>>> Can someone explain why the econnreset error is masked? Is there any way I can separate the two kinds of events?
>>> 

It seems it was a design decision to mask both ECONNRESET recv errors and
all send errors. You can see where this happens for recv errors at around
line 9988 in erts/emulator/drivers/common/inet_drv.c. Open that file and
search for ECONNRESET and you will find the following section of code.

    n = sock_recv(desc->inet.s, desc->i_ptr, nread, 0);

    if (IS_SOCKET_ERROR(n)) {
        int err = sock_errno();
        if (err == ECONNRESET) {
            DEBUGF((" => detected close (connreset)\r\n"));
            return tcp_recv_closed(desc);
        }
        if (err == ERRNO_BLOCK) {
            DEBUGF((" => would block\r\n"));
            return 0;
        }
        else {
            DEBUGF((" => error: %d\r\n", err));
            return tcp_recv_error(desc, err);
        }
    }
    else if (n == 0) {
        DEBUGF(("  => detected close\r\n"));
        return tcp_recv_closed(desc);
    }

As you can see from this, ECONNRESET is being treated as though it's an EOF 
rather than an error: both end up calling tcp_recv_closed(). As a quick fix to
your problem, change the code to read:

        if (err == ECONNRESET) {
            DEBUGF((" => detected close (connreset)\r\n"));
            return tcp_recv_error(desc, err);
        }

However, this is an indiscriminate change which will effect all socket code,
including the networking code for distributed Erlang, and any third party
Erlang code you are using.

Also, be warned that this fix won't alert you to every incoming RST. For
example, if you have a large payload buffered in the socket driver in Erlang 
(i.e. a payload that is too large for the kernel socket send buffer) and you 
receive an RST, then your next call to gen_tcp:recv/2 will return 
{error, closed} rather than {error, econnreset}. See below for example code 
which shows this. The reason this happens is as follows:

 1. When there is outbound data buffered in the socket driver queue, 
    gen_tcp:send/2 becomes an asynchronous call. Which is to say, the new data
    is added to the driver queue and the call returns 'ok'. Then the actual 
    send syscall takes place at a later time, according to epoll, select or 
    whatever.

 2. Then, when you are in passive mode and an error is detected on the send 
    syscall there is no caller to return it to, so it is marked as a 
    "delayed close error". This has two consequences: 

      (a) it is masked to a generic {error, closed}; and
      (b) it is returned on the next call to gen_tcp:recv/2 or gen_tcp:send/2.

So, the send error is ultimately returned on a gen_tcp:recv/2 call, and all
send errors are masked as {error, closed}.

In active mode the problems are similar.

I've got a patch for the recv errors and I'm working on a patch for the send 
errors. Both patches will allow the user to set a socket option that shows all 
econnreset errors (changing some epipe errors to econnreset in the case of send
errors). With any luck, I'll release these patches over the next week or
two as part of a larger set of patches. No guarantee they'll be accepted 
though.

Rory

%%--- Server Code ---
-module(server).
-export([run/0]).

-define(PORT, 7777).
-define(RECBUF, (4 * 1024)).

run() ->
    SockOpts = [{recbuf, ?RECBUF}, {reuseaddr, true}, {active, false}],
    {ok, LSock} = gen_tcp:listen(?PORT, SockOpts),
    {ok, Sock} = gen_tcp:accept(LSock),
    ok = gen_tcp:close(LSock),
    ok = inet:setopts(Sock, [{linger, {true, 0}}]),
    ok = gen_tcp:send(Sock, "Any payload"),
    timer:sleep(1000),
    ok = gen_tcp:close(Sock).

%%--- Client Code ---
-module(client).
-export([run/0]).

-define(PORT, 7777).
-define(SNDBUF, (4 * 1024)).
-define(PAYLOAD_SIZE, (30 * 1024)).

run() ->
    SockOpts = [{sndbuf, ?SNDBUF}, {active, false}],
    {ok, Sock} = gen_tcp:connect("localhost", ?PORT, SockOpts),
    Payload = lists:duplicate(?PAYLOAD_SIZE, $.),
    ok = gen_tcp:send(Sock, Payload),
    ok = timer:sleep(2000),
    Res = gen_tcp:recv(Sock, 0),
    io:format("Result: ~p~n", [Res]),
    gen_tcp:close(Sock).


From dszoboszlay@REDACTED  Sat May  2 18:13:43 2015
From: dszoboszlay@REDACTED (=?UTF-8?Q?D=C3=A1niel_Szoboszlay?=)
Date: Sat, 2 May 2015 18:13:43 +0200
Subject: [erlang-questions] OTP in FIPS mode ?
In-Reply-To: 
References: <276c51c38412b4daff5fc26285f48f2c@teksavvy.com>
 <55367BB8.2040800@tail-f.com>
 
 
 
 
Message-ID: 

Hi,

I'm afraid it's not possible to get away by modifying the C code only, but
you can minimise the Erlang code changes easily.

This commit implements the basics of FIPS support and it's mostly about C
code:
https://github.com/esl/otp/commit/00b3a04d17a653b4abddeebd6dd8a2c38df532d0#diff-1315c36aca592e79ef0073fcf3019e5f
The crypto.erl modifications are mostly for two reason: properly reporting
the list of supported ciphers and tweaking a bit with error handling. The
first is not too important to have (as I said, other parts of OTP do not
respect this list), the second is mostly a cosmetic change: you can always
cause a badarg exception in your NIFs which may be not very informative,
but would crash the Erlang process calling the forbidden algorithm
nevertheless. The only algorithm I had to ban in the Erlang code was SRP,
as some parts of it were implemented in Erlang. But I think banning the C
parts would be enough.

You will need to modify the Erlang code at the following places however:

   - Replacing the hash in the PKIX DB (see parts of
   https://github.com/esl/otp/commit/675ee6860d2c273bcc6c6a0536634a107e2a3d9f#diff-98fe4d9262b05740af899ce038ac0c0d
   dealing with ssl_internal.h, ssl_manager.erl and ssl_pkix_db.erl)
   - Removing forbiden algorithms from the list of ciphers offered by SSL.
   You can do it without tweaking with the ssl application too, by changing
   the options passed to ssl from your client code when setting up a server
   socket/client connection.

By the way, for quite some time now Erlang/OTP only uses libcrypto from
OpenSSL. The ssl protocol is implemented in Erlang using the crypto
primitives from libcrypto.

Hope it helps,
Daniel


2015-05-01 15:03 GMT+02:00 jonetsu :

>
> -----Original Message-----
> From: "D?niel Szoboszlay" 
> Date: 05/01/15 04:01
>
> Hi,
>
> Thanks for your comments, much appreciated !
>
> > - Blocked non-compliant calls in FIPS mode before they would reach
> >  OpenSSL (so you get an Erlang error instead of killing your
> >  VM). This is a must have for any FIPS fork, but it was quite
> >  trivial to implement.
>
> That would be the CHECK_NO_FIPS_MODE in crypto.c.  Is there any at the
> Erlang level ?
>
> I'm asking beacause what I am considering at the moment is to only
> modify the crypto.c code.  No modification to Erlang code. For two
> reasons.  One is that I do not know Erlang, although by browsing the
> code lately I find it quite interesting :) But nowhere near being able
> to modify an application that is used in the field.  Let alone
> establishing test harnesses in the first place. Second is, the OTP
> that is used is already modified by tail-f AG as part of the ConfD
> product.  For instance, if I'm not mistaken, the SSL component is
> different, with crypto being the only part used from OpenSSL. I have
> the impression that the OTP base used in the product dates from some
> time.  It is possible to compile locally crypto.c, but when it comes
> to altering the company's Erlang code then all support is lost.
>
> This approach also means to keep the C <-> Erlang interface intact.
>
> Do you think it is at all possible to have a working FIPS mode without
> any modification to Erlang code ?
>
> Regards.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From e@REDACTED  Sat May  2 22:42:17 2015
From: e@REDACTED (e@REDACTED)
Date: Sat, 02 May 2015 22:42:17 +0200
Subject: [erlang-questions] Automatic currying
In-Reply-To: <861F6F0F-E405-4E56-BC5B-FC24ED3EA9B5@cs.otago.ac.nz>
References: <554293C7.7000301@khandkar.net>
 <861F6F0F-E405-4E56-BC5B-FC24ED3EA9B5@cs.otago.ac.nz>
Message-ID: <554536A9.7090105@bestmx.net>

> I suggest *not* doing any of this.
> Haskell tends to involve stuff like (`f`y) for
> providing the second argument, which looks distinctly
> odd when f has more than 2, and don't let's mention
> `flip' and its relatives.
>
> Why do I suggest not doing this?
> First, as already mentioned, the fact that Erlang
> functions are strict and effectful means that the
> distinction between F and F() is vital.
> Second, this isn't really how Erlang is _meant_ to
> be used, so unlike Haskell, the compiler is not
> going to put much effort into making this less slow.
> Most important, other people aren't going to expect
> to read Erlang code written this way, so it's not
> going to be as maintainable as possibly longer code
> that's more direct.

i want to add,
according to my experience,
currying tends to make code really (i mean _really_) unreadable.

i can not formally explain this effect,
on the contrary,
i love currying, and i would expect it to work nice,
but in real life:
it is very hard to _DETECT_ a partial application
during code reading.

so i consider "massive", wordy, explicit erlang's style
of partial application a huge advantage.

also i vote +1 to the Richard's observation regarding argument order
in the light of currying.




From juangamella@REDACTED  Sun May  3 02:52:16 2015
From: juangamella@REDACTED (=?UTF-8?Q?Juan_Gamella_Mart=C3=ADn?=)
Date: Sun, 3 May 2015 02:52:16 +0200
Subject: [erlang-questions] User defined types: Floats
Message-ID: 

Hello everyone,


I've noticed it's posible to define types specifying subsets of
integers, for example,

-type digits() :: 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9.

However, it doesn't seem to apply to floats. For example,

-type my_floats() :: 1.0 | 2.0.

does not compile.

What are the reasons behind this? Am I making a mistake?

Thank you in advance,

Juan Gamella


From roberto@REDACTED  Sun May  3 18:12:30 2015
From: roberto@REDACTED (Roberto Ostinelli)
Date: Sun, 3 May 2015 18:12:30 +0200
Subject: [erlang-questions] mnesia dirty writes & race conditions
Message-ID: 

Dear list,
What is (or are) the expected outcome(s) of a race condition during mnesia
dirty writes on a table of type `set`? What in case of distributed mnesia,
does this behavior changes if the writes are called from different nodes?

I'm wondering if I should expect a net split, one of the two writes
randomly overriding the other one, or something different?

Thank you,
r.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jesper.louis.andersen@REDACTED  Sun May  3 20:21:26 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Sun, 3 May 2015 20:21:26 +0200
Subject: [erlang-questions] mnesia dirty writes & race conditions
In-Reply-To: 
References: 
Message-ID: 

On Sun, May 3, 2015 at 6:12 PM, Roberto Ostinelli 
wrote:

> Dear list,
> What is (or are) the expected outcome(s) of a race condition during mnesia
> dirty writes on a table of type `set`?
>

The lower bound is "whatever ETS gives you" which means that writes to a
row is atomic, on a single node. The last writer wins. If two nodes writes
to the same record at the same time, I'm not sure you have any guarantees
about what you read on other nodes.

In other words, I wouldn't expect any guarantee, in particular writing {K,
V1} on node1 and {K, V2} on node2 could result in write order {K, V1} ->
{K, V2} on one node and {K, V2} -> {K, V1} on another node. If you want
that guarantee, you should run a transaction.

The problem can often be avoided because if you know a priori that your key
K is under your sole responsibility, then a dirty_write is safe insofar
noone else can change it, and you control the writing order, which means
everyone eventually ends up with the same value.

-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roberto.ostinelli@REDACTED  Sun May  3 21:23:53 2015
From: roberto.ostinelli@REDACTED (Roberto Ostinelli)
Date: Sun, 3 May 2015 21:23:53 +0200
Subject: [erlang-questions] mnesia dirty writes & race conditions
In-Reply-To: 
References: 
 
Message-ID: 


> On 03/mag/2015, at 20:21, Jesper Louis Andersen  wrote:
> 
> In other words, I wouldn't expect any guarantee, in particular writing {K, V1} on node1 and {K, V2} on node2 could result in write order {K, V1} -> {K, V2} on one node and {K, V2} -> {K, V1} on another node. If you want that guarantee, you should run a transaction.

Wouldn't mnesia:async_dirty/1,2 solve this problem, while keeping dirty writes?

From roberto.ostinelli@REDACTED  Sun May  3 21:25:20 2015
From: roberto.ostinelli@REDACTED (Roberto Ostinelli)
Date: Sun, 3 May 2015 21:25:20 +0200
Subject: [erlang-questions] mnesia dirty writes & race conditions
In-Reply-To: 
References: 
 
 
Message-ID: <51EA4EEC-7550-41F3-AC4B-BD6385A49AE9@widetag.com>


> Wouldn't mnesia:async_dirty/1,2 solve this problem, while keeping dirty writes?

Sorry I meant mnesia:sync_dirty

From ok@REDACTED  Mon May  4 06:22:54 2015
From: ok@REDACTED (Richard A. O'Keefe)
Date: Mon, 4 May 2015 16:22:54 +1200
Subject: [erlang-questions] Server stops responding
In-Reply-To: 
References: ,
 
 
Message-ID: <0A9C517D-CC5D-479B-83DF-9E8659FBEEA8@cs.otago.ac.nz>


On 2/05/2015, at 10:20 am, Sid Muller  wrote:

> I figured it must be something so banal that I'm missing.
>  
> Thank you for your patience, for some reason I had made wrong assumptions about how the shell works.

Tip: in the shell, f(X) forgets the current binding for X.
f() forgets all bindings.




From ok@REDACTED  Mon May  4 06:47:56 2015
From: ok@REDACTED (Richard A. O'Keefe)
Date: Mon, 4 May 2015 16:47:56 +1200
Subject: [erlang-questions] Automatic currying
In-Reply-To: <554536A9.7090105@bestmx.net>
References: <554293C7.7000301@khandkar.net>
 <861F6F0F-E405-4E56-BC5B-FC24ED3EA9B5@cs.otago.ac.nz>
 <554536A9.7090105@bestmx.net>
Message-ID: <061A66B1-20F8-4ADA-BEB8-4FAB8514EDCB@cs.otago.ac.nz>

The advantage of the proposed currying function is that
it is just library code.

Let me show you what Eiffel does these days.

 ::= 'agent' [ '.' ]
                 ['('  {',' }* ')']

That is, an "agent expression" looks pretty much like a normal
method call with the keyword 'agent' in front of it.

If there is no agent receiver, the Current object is the
receiver.

 ::=  | '{'  '}'.

In the first case, a call of the agent will use the
specified object; in the second case, the receiver will
be a (the first) parameter of the agent and {C} gives its
type.

 ::=  | '?' | '{'  '}'

Agent parameters that are expressions are evaluated at the
time the agent expression is evaluated.  Their values are
remembered as part of the closure.  Agent parameters that
are question marks are parameters of the agent.  The
compiler can figure out the type of such parameters, but if
you want to make it clear to humans reading the code, you
can use the {C} format if you want to.

So given a method foo of three arguments, you can write

  agent r.foo(x, y, z)          -- 0 arguments
  agent r.foo(x, y, ?)          -- \
  agent r.foo(x, ?, z)          -- these four have
  agent r.foo(?, y, z)          -- 1 argument
  agent {C}.foo(x, y, z)        -- /
  agent r.foo(x, ?, ?)
  agent r.foo(?, y, ?)
  agent r.foo(?, ?, z)
  agent {C}.foo(x, y, ?)
  ....
  agent r.foo(?, ?, ?)          -- 3 arguments
  agent {C}.foo(?, ?, ?)        -- 4
  
You can't *re-order* arguments, but you can leave any
arguments you want open, not just trailing ones, as
currying does.

Could this syntax be adapted to Erlang?
Sure.  'agent' -> 'fun', receiver -> module, '.' -> ':'.
fun f(X, ?) and fun f(?, Y) would make perfect sense,
and fun f(X, Y) would be clearly distinct from f(X, Y)
just as agent r.f(x, y) is a parameterless procedure
different from f(x, y) in Eiffel.

Currying fits nicely into languages where every function
has EXACTLY ONE argument, like ML, Miranda, F#, Clean, ...
Erlang is not such a language, and neither is Eiffel,
which is why if we *did* want a currying construct for
Erlang, Eiffel's agent expressions might be a better fit.

For what it's worth, I've got some 'currying' methods
in my Smalltalk, so that
(aBlock bindFirst: x) value: y
=
aBlock value: x value: y.
Having written them, I've found them remarkably useless.



From dangud@REDACTED  Mon May  4 08:18:34 2015
From: dangud@REDACTED (Dan Gudmundsson)
Date: Mon, 04 May 2015 06:18:34 +0000
Subject: [erlang-questions] mnesia dirty writes & race conditions
In-Reply-To: <51EA4EEC-7550-41F3-AC4B-BD6385A49AE9@widetag.com>
References: 
 
 
 <51EA4EEC-7550-41F3-AC4B-BD6385A49AE9@widetag.com>
Message-ID: 

On Sun, May 3, 2015 at 9:25 PM Roberto Ostinelli <
roberto.ostinelli@REDACTED> wrote:

>
> > Wouldn't mnesia:async_dirty/1,2 solve this problem, while keeping dirty
> writes?
>
> Sorry I meant mnesia:sync_dirty
>

No, that gives you flow control but nothing else.


> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roberto.ostinelli@REDACTED  Mon May  4 09:40:21 2015
From: roberto.ostinelli@REDACTED (Roberto Ostinelli)
Date: Mon, 4 May 2015 09:40:21 +0200
Subject: [erlang-questions] mnesia dirty writes & race conditions
In-Reply-To: 
References: 
 
 
 <51EA4EEC-7550-41F3-AC4B-BD6385A49AE9@widetag.com>
 
Message-ID: <6513A4EB-AD96-4D91-953C-1C62539D2133@widetag.com>


> No, that gives you flow control but nothing else.

I see. Here's why I am asking this question.

My spec requirements of writes / sec into a mnesia database replicated on a 3 to 4 nodes cluster are higher than what a transaction can give me. 

Therefore, I am currently using mnesia dirty writes, because as J. Louis put it I have loose control on the input and I do not anticipate a situation with conflicting writes. 

However, I'd feel better in finding a way to guarantee the atomicity of the operation. 

What are my options? The only thing I can see right now is to resolve using some kind of leader election, and have the leader dirty write all the messages to mnesia. 

Any other ideas (besides fragmentation)?



From petergi@REDACTED  Mon May  4 11:04:54 2015
From: petergi@REDACTED (Peter J Etheridge)
Date: Mon, 04 May 2015 19:04:54 +1000
Subject: [erlang-questions] my first crash.dump
Message-ID: 

dear list,in learning Erlang i am trialling Intellij IDEA.
as a test, i entered from p.165 of joe's book, his module
fac1.although no errors were evident in what i had transcribed,when i
ran debugger i received my first;erl_crash.dump3,258 typos found.from
just 9 lines of joe's code?
any clues?best regards to all,peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From tuncer.ayaz@REDACTED  Mon May  4 12:34:40 2015
From: tuncer.ayaz@REDACTED (Tuncer Ayaz)
Date: Mon, 4 May 2015 12:34:40 +0200
Subject: [erlang-questions] my first crash.dump
In-Reply-To: 
References: 
Message-ID: 

On Mon, May 4, 2015 at 11:04 AM, Peter J Etheridge wrote:
> dear list,
> in learning Erlang i am trialling Intellij IDEA.
> as a test, i entered from p.165 of joe's book, his module fac1.
> although no errors were evident in what i had transcribed,
> when i ran debugger i received my first;
> erl_crash.dump
> 3,258 typos found.
> from just 9 lines of joe's code?
> any clues?

For reference, can you post the code here?

Also, what Erlang version did you use?


From jesper.louis.andersen@REDACTED  Mon May  4 16:07:29 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Mon, 4 May 2015 16:07:29 +0200
Subject: [erlang-questions] mnesia dirty writes & race conditions
In-Reply-To: <6513A4EB-AD96-4D91-953C-1C62539D2133@widetag.com>
References: 
 
 
 <51EA4EEC-7550-41F3-AC4B-BD6385A49AE9@widetag.com>
 
 <6513A4EB-AD96-4D91-953C-1C62539D2133@widetag.com>
Message-ID: 

On Mon, May 4, 2015 at 9:40 AM, Roberto Ostinelli <
roberto.ostinelli@REDACTED> wrote:

> However, I'd feel better in finding a way to guarantee the atomicity of
> the operation.


When you request atomicity of the operation, you are asking for
linearizability, which is a quite specific guarantee. You request that
there is some point in time, and once you go beyond that point, any read
will always see the new value and can't be stale (This is an informal
description, but the real linearizability guarantee has a fully formal
notion). Reads which overlap the point in time sees either the new or the
old value, but reads that come after the point in time will *always* see
the new value. This is the essential guarantee of atomicity/linearizability.

In the sense of CAP, you are now in CP land, as linearizability is strictly
in CP. This means you need coordination of some kind in order to solve it.
Your method of electing a leader is one, if done correctly (i.e., hi
consensus algorithms or distributed locking). The other one is to run a
mnesia-transaction, which almost gets you there[0].

But chances are you don't need linearizability for the operation in the
first place. And then, you can avoid having to coordinate as you may be
able to put yourself in AP instead. AP is typically much faster due to the
lack of coordination, but do see the work of e.g., Neha Narula (et.al.) for
counterexamples to this.

[0] I've argued before that mnesia is neither CP nor AP, and since you
can't be CA, mnesia isn't really a distributed system in the typical notion
of the word.


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From mononcqc@REDACTED  Mon May  4 17:30:42 2015
From: mononcqc@REDACTED (Fred Hebert)
Date: Mon, 4 May 2015 11:30:42 -0400
Subject: [erlang-questions] mnesia dirty writes & race conditions
In-Reply-To: 
References: 
 
 
 <51EA4EEC-7550-41F3-AC4B-BD6385A49AE9@widetag.com>
 
 <6513A4EB-AD96-4D91-953C-1C62539D2133@widetag.com>
 
Message-ID: <20150504153041.GA30039@ferdair.local>

On 05/04, Jesper Louis Andersen wrote:
>But chances are you don't need linearizability for the operation in the
>first place. And then, you can avoid having to coordinate as you may be
>able to put yourself in AP instead. AP is typically much faster due to the
>lack of coordination, but do see the work of e.g., Neha Narula (et.al.) for
>counterexamples to this.

This is an interesting point made in "Highly Available Transactions: 
Virtues and Limitations" by Peter Bailis et. al.  
(http://www.bailis.org/papers/hat-vldb2014.pdf) (see section 3 on page 
3):

> Accordingly, to increase concurrency, database systems offer a range 
> of ACID properties weaker than serializability: the host of so-called 
> weak isolation models describe varying restrictions on the space of 
> schedules that are allowable by the system. None of these weak 
> isolation models guarantees serializability, but, as we see below, 
> their benefits are often considered to outweigh costs of possible 
> consistency anomalies that might arise from their use.

Specifically, table 2 (http://i.imgur.com/7Lw9lBd.png) shows databases 
such as MySQL, Postgres, and Oracle all possibly supporting 
serializability, but by default would allow much lower guarantees 
(repeatable reads or read committed), which are high-availability 
transactions.

Repeatable Reads (RR) are defined as follows, which I believe is pretty 
much what MVCC stands for:

> the ANSI standardized implementation-agnostic definition
> is achievable and directly captures the spirit of the term: if a 
> transaction reads the same data more than once, it sees the same value 
> each time (preventing ?Fuzzy Read?).

Read Committed (RC) is defined as:

> Under Read Committed, transactions should not access uncommitted or  
> intermediate  versions  of  data  items. This prohibits both ?Dirty 
> Writes? [...] and also ?Dirty Reads?

And that's about it. This tells you multiple transactions could happen 
at the same time and result in a non-linearizable history. Two RC 
transactions could both operate at once, and through some interleavings 
of read and write locks across transactions, give you results that would 
not make sense without the specific concurrent interleaving they have 
seen. They would not be linearizable or serializable.

An important note here is that a highly available transaction (HAT) is 
defined as a transaction that eventually commits if it can contact at 
least one replica for each of the data items it attempts to touch; This 
is slightly different from the ususal "can I write to this row on any 
given node", but does mean multiple levels of failure (even a majority 
of them) could allow some transactions to still work under RR or RCs.

> As shown in Table 2, only three out of 18 databases provided 
> serializability by default, and eight did not provide serializability 
> as an option at all. [...] Given that these weak transactional models 
> are frequently used, our inability to provide serializability in 
> arbitrary HATs appears non-fatal for practical applications.

It is not explained outright, but I'm guessing the reason why many of 
these transactions are *not* made highly-available via common RDBMs is 
that they're more seen as optimizations for speed, and that it wouldn't 
fit their model very well in the large, or wanting the ability to add 
serializability as a safety guarantee without tearing down your whole 
infrastructure.

Many DBs' default transaction mechanisms have semantics could lend 
themselves to higher availability, but their implementation just doesn't 
appear to support it.


From cmeiklejohn@REDACTED  Mon May  4 19:32:16 2015
From: cmeiklejohn@REDACTED (Christopher Meiklejohn)
Date: Mon, 4 May 2015 19:32:16 +0200
Subject: [erlang-questions]
 =?utf-8?q?=5Briak=5D_comile_riak_error_cannot_?=
 =?utf-8?b?c3RhdCDigJhzb2xyLTQuNy4wLXl6LTEudGd64oCZOiBObyBzdWNoIGZpbGUg?=
 =?utf-8?q?or_directory?=
In-Reply-To: 
References: 
Message-ID: 


> On Apr 24, 2015, at 5:58 AM, ?? <44290016@REDACTED> wrote:
> 
> Hi,
> 
> When I make all, got following error:
> Erlang R16B03, git clone https://github.com/basho/riak  git checkout riak-2.0.4
> 
> ==> yokozuna (compile)
> Create dir ../build
> Pulling Solr from S3
> cp: cannot stat ?solr-4.7.0-yz-1.tgz?: No such file or directory
> ERROR: Command [compile] failed!

Hi there,

Can you please provide the full output of ?make? when you attempt to compile Riak?

- Chris

Christopher Meiklejohn
Senior Software Engineer
Basho Technologies, Inc.
cmeiklejohn@REDACTED



From benhsu@REDACTED  Tue May  5 03:41:01 2015
From: benhsu@REDACTED (Ben Hsu)
Date: Mon, 4 May 2015 21:41:01 -0400
Subject: [erlang-questions] remote service discovery?
Message-ID: 

Hello Erlangers

I'm curious, is there a way to ask a remote erlang node what services it
provides? either in core Erlang or OTP?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From s.j.thompson@REDACTED  Tue May  5 10:43:40 2015
From: s.j.thompson@REDACTED (Simon Thompson)
Date: Tue, 5 May 2015 09:43:40 +0100
Subject: [erlang-questions] IFL 2015 -- call for papers
References: <201505041250.t44CoFq8031082@easychair.org>
Message-ID: <04C80D50-B2C9-4A68-8096-0E2E3DC538E1@kent.ac.uk>

IFL 2015 - Call for papers

27th SYMPOSIUM ON IMPLEMENTATION AND APPLICATION OF FUNCTIONAL LANGUAGES - IFL 2015

University of Koblenz-Landau, Koblenz, Germany

In cooperation with ACM SIGPLAN

September 14-16, 2015

http://ifl2015.wikidot.com/

Scope

The goal of the IFL symposia is to bring together researchers actively engaged in the implementation and application of functional and function-based programming languages. IFL 2015 will be a venue for researchers to present and discuss new ideas and concepts, work in progress, and publication-ripe results related to the implementation and application of functional languages and function-based programming.

Peer-review

Following the IFL tradition, IFL 2015 will use a post-symposium review process to produce the formal proceedings. All participants of IFL2015 are invited to submit either a draft paper or an extended abstract describing work to be presented at the symposium. At no time may work submitted to IFL be simultaneously submitted to other venues; submissions must adhere to ACM SIGPLAN's republication policy:

http://www.sigplan.org/Resources/Policies/Republication

The submissions will be screened by the program committee chair to make sure they are within the scope of IFL, and will appear in the draft proceedings distributed at the symposium. Submissions appearing in the draft proceedings are not peer-reviewed publications. Hence, publications that appear only in the draft proceedings do not count as publication for the ACM SIGPLAN republication policy. After the symposium, authors will be given the opportunity to incorporate the feedback from discussions at the symposium and will be invited to submit a revised full article for the formal review process. From the revised submissions, the program committee will select papers for the formal proceedings considering their correctness, novelty, originality, relevance, significance, and clarity.

Important dates

August 10: Submission deadline draft papers
August 12: Notification of acceptance for presentation
August 14: Early registration deadline
August 21: Late registration deadline
September 7: Submission deadline for pre-symposium proceedings
September 14-16: IFL Symposium
December 1: Submission deadline for post-symposium proceedings
January 15, 2016: Notification of acceptance for post-symposium proceedings
March 1, 2016: Camera-ready version for post-symposium proceedings

Submission details

Prospective authors are encouraged to submit papers or extended abstracts to be published in the draft proceedings and to present them at the symposium. All contributions must be written in English. Papers must adhere to the standard ACM two columns conference format. For the pre-symposium proceedings we adopt a 'weak' page limit of 12 pages. For the post-symposium proceedings the page limit of 12 pages is firm. A suitable document template for LaTeX can be found at:

http://www.acm.org/sigs/sigplan/authorInformation.htm

Authors submit through EasyChair:

https://easychair.org/conferences/?conf=ifl2015

Topics

IFL welcomes submissions describing practical and theoretical work as well as submissions describing applications and tools in the context of functional programming. If you are not sure whether your work is appropriate for IFL 2015, please contact the PC chair at rlaemmel@REDACTED Topics of interest include, but are not limited to:

- language concepts
- type systems, type checking, type inferencing
- compilation techniques
- staged compilation
- run-time function specialization
- run-time code generation
- partial evaluation
- (abstract) interpretation
- metaprogramming
- generic programming
- automatic program generation
- array processing
- concurrent/parallel programming
- concurrent/parallel program execution
- embedded systems
- web applications
- (embedded) domain specific languages
- security
- novel memory management techniques
- run-time profiling performance measurements
- debugging and tracing
- virtual/abstract machine architectures
- validation, verification of functional programs
- tools and programming techniques
- (industrial) applications

Peter Landin Prize

The Peter Landin Prize is awarded to the best paper presented at the symposium every year. The honored article is selected by the program committee based on the submissions received for the formal review process. The prize carries a cash award equivalent to 150 Euros.

Programme committee

Chair: Ralf L?mmel, University of Koblenz-Landau, Germany

- Malgorzata Biernacka, University of Wroclaw, Poland
- Laura M. Castro, University of A Coru?a, Spain
- Martin Erwig, Oregon State University, USA
- Dan Ghica, University of Birmingham, UK
- Andrew Gill, University of Kansas, USA
- Stephan Herhut, Google, USA
- Zhenjiang Hu, National Institute of Informatics (NII), Japan
- Mauro Jaskelioff, CIFASIS/Universidad Nacional de Rosario, Argentina
- Fr?d?ric Jouault, ESEO, France
- Oleg Kiselyov, Tohoku University, Japan
- Lindsey Kuper, Indiana University, USA
- Rita Loogen, Philipps-Universit?t Marburg, Germany
- Akimasa Morihata, University of Tokyo, Japan
- Atsushi Ohori, Tohoku University, Japan
- Bruno C. D. S. Oliveira, The University of Hong Kong, Hong Kong
- Frank Piessens, Katholieke Universiteit Leuven, Belgium
- Norman Ramsey, Tufts University, USA
- Matthew Roberts, Macquarie University, Australia
- Manfred Schmidt-Schauss, Goethe-University Frankfurt, Germany
- Simon Thompson, University of Kent, UK
- Stephanie Weirich, University of Pennsylvania, USA
- Steve Zdancewic, University of Pennsylvania , USA

Venue

The 27th IFL will be held in association with the Faculty of Computer Science, University of Koblenz-Landau, Campus Koblenz. Koblenz is well connected by train to several international airports. For instance, Koblenz can be reached from Frankfurt by high-speed train ICE within an hour. The modern Koblenz campus is close to the city center and can be reached by foot, bus, or cab. See the website for more information on the venue.

Simon Thompson | Professor of Logic and Computation 
School of Computing | University of Kent | Canterbury, CT2 7NF, UK
s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt




From Andras.Bekes@REDACTED  Tue May  5 09:44:47 2015
From: Andras.Bekes@REDACTED (Bekes, Andras G)
Date: Tue, 5 May 2015 07:44:47 +0000
Subject: [erlang-questions] {error,closed} vs. {error,econnreset}
In-Reply-To: <20150502103209.GA19880@nybek.com>
References: 
 <54E4A07E.5010307@erlang.org>
 
 <20150502103209.GA19880@nybek.com>
Message-ID: 

Thank you very much for your efforts Rory.

The ability "to set a socket option that shows all econnreset errors" sounds like the right solution. I am wondering why hiding this detail is the default, but I believe there were good enough reasons to design it that way.

I accept that your solution will not notice the connection reset event in some corner cases. I think this will not apply in my case: I am sending a small amount of data (<1KB) and wait for the reply.

I am looking forward to see your patch in the next release of Erlang/OTP!

Thanks,
   Andras

-----Original Message-----
From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Rory Byrne
Sent: Saturday, May 02, 2015 12:32 PM
To: erlang-questions@REDACTED
Subject: Re: [erlang-questions] {error,closed} vs. {error,econnreset}

Hi Andras,

On Fri, May 01, 2015 at 11:53:13AM +0000, Bekes, Andras G wrote:
> 
> Please see the trace in the attached file. Do you have suggestions on what else shall I trace?
> 
>>> On 2015-02-10 14:39, Bekes, Andras G wrote:
>>> 
>>> Looks like I have a problem with the erroneous result of gen_tcp:recv/2, as it returns {error,closed} instead of my expectation of {error,econnreset}.
>>> 
...
>>> 
>>> Unfortunately it looks like I really need to separate these two results.
>>> 
>>> I also tried gen_tcp in active mode, but no difference, the result is {error,closed} instead of {error,econnreset}.
>>> 
>>> Can someone explain why the econnreset error is masked? Is there any way I can separate the two kinds of events?
>>> 

It seems it was a design decision to mask both ECONNRESET recv errors and
all send errors. You can see where this happens for recv errors at around
line 9988 in erts/emulator/drivers/common/inet_drv.c. Open that file and
search for ECONNRESET and you will find the following section of code.

    n = sock_recv(desc->inet.s, desc->i_ptr, nread, 0);

    if (IS_SOCKET_ERROR(n)) {
        int err = sock_errno();
        if (err == ECONNRESET) {
            DEBUGF((" => detected close (connreset)\r\n"));
            return tcp_recv_closed(desc);
        }
        if (err == ERRNO_BLOCK) {
            DEBUGF((" => would block\r\n"));
            return 0;
        }
        else {
            DEBUGF((" => error: %d\r\n", err));
            return tcp_recv_error(desc, err);
        }
    }
    else if (n == 0) {
        DEBUGF(("  => detected close\r\n"));
        return tcp_recv_closed(desc);
    }

As you can see from this, ECONNRESET is being treated as though it's an EOF 
rather than an error: both end up calling tcp_recv_closed(). As a quick fix to
your problem, change the code to read:

        if (err == ECONNRESET) {
            DEBUGF((" => detected close (connreset)\r\n"));
            return tcp_recv_error(desc, err);
        }

However, this is an indiscriminate change which will effect all socket code,
including the networking code for distributed Erlang, and any third party
Erlang code you are using.

Also, be warned that this fix won't alert you to every incoming RST. For
example, if you have a large payload buffered in the socket driver in Erlang 
(i.e. a payload that is too large for the kernel socket send buffer) and you 
receive an RST, then your next call to gen_tcp:recv/2 will return 
{error, closed} rather than {error, econnreset}. See below for example code 
which shows this. The reason this happens is as follows:

 1. When there is outbound data buffered in the socket driver queue, 
    gen_tcp:send/2 becomes an asynchronous call. Which is to say, the new data
    is added to the driver queue and the call returns 'ok'. Then the actual 
    send syscall takes place at a later time, according to epoll, select or 
    whatever.

 2. Then, when you are in passive mode and an error is detected on the send 
    syscall there is no caller to return it to, so it is marked as a 
    "delayed close error". This has two consequences: 

      (a) it is masked to a generic {error, closed}; and
      (b) it is returned on the next call to gen_tcp:recv/2 or gen_tcp:send/2.

So, the send error is ultimately returned on a gen_tcp:recv/2 call, and all
send errors are masked as {error, closed}.

In active mode the problems are similar.

I've got a patch for the recv errors and I'm working on a patch for the send 
errors. Both patches will allow the user to set a socket option that shows all 
econnreset errors (changing some epipe errors to econnreset in the case of send
errors). With any luck, I'll release these patches over the next week or
two as part of a larger set of patches. No guarantee they'll be accepted 
though.

Rory

%%--- Server Code ---
-module(server).
-export([run/0]).

-define(PORT, 7777).
-define(RECBUF, (4 * 1024)).

run() ->
    SockOpts = [{recbuf, ?RECBUF}, {reuseaddr, true}, {active, false}],
    {ok, LSock} = gen_tcp:listen(?PORT, SockOpts),
    {ok, Sock} = gen_tcp:accept(LSock),
    ok = gen_tcp:close(LSock),
    ok = inet:setopts(Sock, [{linger, {true, 0}}]),
    ok = gen_tcp:send(Sock, "Any payload"),
    timer:sleep(1000),
    ok = gen_tcp:close(Sock).

%%--- Client Code ---
-module(client).
-export([run/0]).

-define(PORT, 7777).
-define(SNDBUF, (4 * 1024)).
-define(PAYLOAD_SIZE, (30 * 1024)).

run() ->
    SockOpts = [{sndbuf, ?SNDBUF}, {active, false}],
    {ok, Sock} = gen_tcp:connect("localhost", ?PORT, SockOpts),
    Payload = lists:duplicate(?PAYLOAD_SIZE, $.),
    ok = gen_tcp:send(Sock, Payload),
    ok = timer:sleep(2000),
    Res = gen_tcp:recv(Sock, 0),
    io:format("Result: ~p~n", [Res]),
    gen_tcp:close(Sock).
_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://erlang.org/mailman/listinfo/erlang-questions




From cmeiklejohn@REDACTED  Tue May  5 12:50:55 2015
From: cmeiklejohn@REDACTED (Christopher Meiklejohn)
Date: Tue, 5 May 2015 12:50:55 +0200
Subject: [erlang-questions] [ANN] Curry On Prague! 2015 Schedule Announced
Message-ID: <6042D16A-1C41-43A4-9666-774F49042EB7@basho.com>

Curry On is a new conference focused on the intersection of emerging languages and emerging challenges in industry (e.g. big data or security), as well as new ideas and paradigms in software development. 

Curry On also seeks to act as a conduit for ferrying understanding and ideas back and forth between industry and academic programming languages, software engineering, and systems research communities (amongst others).

Curry On is a rare event where academic minds responsible for concepts and tools now invaluable to everyday software development ? like functional programming, or generics in Java ? collide with the movers and shakers in industry that are building next-generation systems and developing software engineering practices central to our entire industry. 

Curry On will be held in a different European city year-to-year, and it will always be co-located with one of the top academic conferences in programming languages. This year, it's co-located with ECOOP.

Join us for two days of fruitful and mind-altering ideas and discussions!

http://curry-on.org

Christopher Meiklejohn
Senior Software Engineer
Basho Technologies, Inc.
cmeiklejohn@REDACTED



From Antonio.Musumeci@REDACTED  Tue May  5 13:06:38 2015
From: Antonio.Musumeci@REDACTED (Musumeci, Antonio S)
Date: Tue, 5 May 2015 11:06:38 +0000
Subject: [erlang-questions] Re:  {error,
 closed} vs. {error, econnreset}
In-Reply-To: 
References: 
 <54E4A07E.5010307@erlang.org>
 
 <20150502103209.GA19880@nybek.com>
 
Message-ID: <51C6F20DC46369418387C5250127649B122FE4B9@HZWEX2014N4.msad.ms.com>

?It feels like there really should be two code paths. One lower level which mimics traditional socket behavior and then something on top which may or may not hide some of the details on account of the abstraction. Masking the errors here can cause a lot of useful info to be lost.

-----Original Message-----
From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Bekes, Andras G (ICT)
Sent: Tuesday, May 05, 2015 3:45 AM
To: Rory Byrne; erlang-questions@REDACTED
Subject: Re: [erlang-questions] {error,closed} vs. {error,econnreset}

Note: This e-mail has been flagged because although the sender appears to be internal, the e-mail originated outside the Firm. The sender?s address may have been forged, which can be an indicator of malicious activity. If you have any concerns about the content or sender of this e-mail, please report it immediately at http://irespond.

Thank you very much for your efforts Rory.

The ability "to set a socket option that shows all econnreset errors" sounds like the right solution. I am wondering why hiding this detail is the default, but I believe there were good enough reasons to design it that way.

I accept that your solution will not notice the connection reset event in some corner cases. I think this will not apply in my case: I am sending a small amount of data (<1KB) and wait for the reply.

I am looking forward to see your patch in the next release of Erlang/OTP!

Thanks,
   Andras

-----Original Message-----
From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Rory Byrne
Sent: Saturday, May 02, 2015 12:32 PM
To: erlang-questions@REDACTED
Subject: Re: [erlang-questions] {error,closed} vs. {error,econnreset}

Hi Andras,

On Fri, May 01, 2015 at 11:53:13AM +0000, Bekes, Andras G wrote:
>
> Please see the trace in the attached file. Do you have suggestions on what else shall I trace?
>
>>> On 2015-02-10 14:39, Bekes, Andras G wrote:
>>>
>>> Looks like I have a problem with the erroneous result of gen_tcp:recv/2, as it returns {error,closed} instead of my expectation of {error,econnreset}.
>>>
...
>>>
>>> Unfortunately it looks like I really need to separate these two results.
>>>
>>> I also tried gen_tcp in active mode, but no difference, the result is {error,closed} instead of {error,econnreset}.
>>>
>>> Can someone explain why the econnreset error is masked? Is there any way I can separate the two kinds of events?
>>>

It seems it was a design decision to mask both ECONNRESET recv errors and
all send errors. You can see where this happens for recv errors at around
line 9988 in erts/emulator/drivers/common/inet_drv.c. Open that file and
search for ECONNRESET and you will find the following section of code.

    n = sock_recv(desc->inet.s, desc->i_ptr, nread, 0);

    if (IS_SOCKET_ERROR(n)) {
        int err = sock_errno();
        if (err == ECONNRESET) {
            DEBUGF((" => detected close (connreset)\r\n"));
            return tcp_recv_closed(desc);
        }
        if (err == ERRNO_BLOCK) {
            DEBUGF((" => would block\r\n"));
            return 0;
        }
        else {
            DEBUGF((" => error: %d\r\n", err));
            return tcp_recv_error(desc, err);
        }
    }
    else if (n == 0) {
        DEBUGF(("  => detected close\r\n"));
        return tcp_recv_closed(desc);
    }

As you can see from this, ECONNRESET is being treated as though it's an EOF
rather than an error: both end up calling tcp_recv_closed(). As a quick fix to
your problem, change the code to read:

        if (err == ECONNRESET) {
            DEBUGF((" => detected close (connreset)\r\n"));
            return tcp_recv_error(desc, err);
        }

However, this is an indiscriminate change which will effect all socket code,
including the networking code for distributed Erlang, and any third party
Erlang code you are using.

Also, be warned that this fix won't alert you to every incoming RST. For
example, if you have a large payload buffered in the socket driver in Erlang
(i.e. a payload that is too large for the kernel socket send buffer) and you
receive an RST, then your next call to gen_tcp:recv/2 will return
{error, closed} rather than {error, econnreset}. See below for example code
which shows this. The reason this happens is as follows:

 1. When there is outbound data buffered in the socket driver queue,
    gen_tcp:send/2 becomes an asynchronous call. Which is to say, the new data
    is added to the driver queue and the call returns 'ok'. Then the actual
    send syscall takes place at a later time, according to epoll, select or
    whatever.

 2. Then, when you are in passive mode and an error is detected on the send
    syscall there is no caller to return it to, so it is marked as a
    "delayed close error". This has two consequences:

      (a) it is masked to a generic {error, closed}; and
      (b) it is returned on the next call to gen_tcp:recv/2 or gen_tcp:send/2.

So, the send error is ultimately returned on a gen_tcp:recv/2 call, and all
send errors are masked as {error, closed}.

In active mode the problems are similar.

I've got a patch for the recv errors and I'm working on a patch for the send
errors. Both patches will allow the user to set a socket option that shows all
econnreset errors (changing some epipe errors to econnreset in the case of send
errors). With any luck, I'll release these patches over the next week or
two as part of a larger set of patches. No guarantee they'll be accepted
though.

Rory

%%--- Server Code ---
-module(server).
-export([run/0]).

-define(PORT, 7777).
-define(RECBUF, (4 * 1024)).

run() ->
    SockOpts = [{recbuf, ?RECBUF}, {reuseaddr, true}, {active, false}],
    {ok, LSock} = gen_tcp:listen(?PORT, SockOpts),
    {ok, Sock} = gen_tcp:accept(LSock),
    ok = gen_tcp:close(LSock),
    ok = inet:setopts(Sock, [{linger, {true, 0}}]),
    ok = gen_tcp:send(Sock, "Any payload"),
    timer:sleep(1000),
    ok = gen_tcp:close(Sock).

%%--- Client Code ---
-module(client).
-export([run/0]).

-define(PORT, 7777).
-define(SNDBUF, (4 * 1024)).
-define(PAYLOAD_SIZE, (30 * 1024)).

run() ->
    SockOpts = [{sndbuf, ?SNDBUF}, {active, false}],
    {ok, Sock} = gen_tcp:connect("localhost", ?PORT, SockOpts),
    Payload = lists:duplicate(?PAYLOAD_SIZE, $.),
    ok = gen_tcp:send(Sock, Payload),
    ok = timer:sleep(2000),
    Res = gen_tcp:recv(Sock, 0),
    io:format("Result: ~p~n", [Res]),
    gen_tcp:close(Sock).
_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://erlang.org/mailman/listinfo/erlang-questions



From tuncer.ayaz@REDACTED  Tue May  5 19:58:09 2015
From: tuncer.ayaz@REDACTED (Tuncer Ayaz)
Date: Tue, 5 May 2015 19:58:09 +0200
Subject: [erlang-questions] my first crash.dump
In-Reply-To: 
References: 
 
Message-ID: 

On Mon, May 4, 2015 at 12:34 PM, Tuncer Ayaz wrote:
> On Mon, May 4, 2015 at 11:04 AM, Peter J Etheridge wrote:
> > dear list,
> > in learning Erlang i am trialling Intellij IDEA.
> > as a test, i entered from p.165 of joe's book, his module fac1.
> > although no errors were evident in what i had transcribed,
> > when i ran debugger i received my first;
> > erl_crash.dump
> > 3,258 typos found.
> > from just 9 lines of joe's code?
> > any clues?
>
> For reference, can you post the code here?
>
> Also, what Erlang version did you use?

Here's what Peter sent me and forgot to CC the list:

-module(fac). %fac1 from p.165 of joe's book
-author("BB").
-include_lib("kernel/include/file.hrl"). %another experiment
%% API
-export([main/1]).

main([A])->
    I = list_to_integer(atom_to_list(A)),
    F = fac(I),
    io:format("factorial ~w = ~w~n", [I,F]),
    init:stop().

fac(0) -> 1;
fac(N) -> N*fac(N-1).

I don't know if this is exactly the code from the book, but it's easy
to see where some confusion might arise.

main/1 accepts a list of a single atom to later use as the argument
for fac/1. Maybe fac.erl is meant to be a valid escript as well.

$ erl
> c(fac).
{ok,fac}
> fac:main(['2']).
factorial 2 = 2

If you export fac/2, you can call it like this:

> fac:fac(2).
2

Also, I don't see where file.hrl would be required, so you can omit
that. You'd include it only for the record declarations contained
within (#file_info, #file_description).


From elbrujohalcon@REDACTED  Tue May  5 20:41:20 2015
From: elbrujohalcon@REDACTED (Fernando 'Brujo' Benavides)
Date: Tue, 5 May 2015 15:41:20 -0300
Subject: [erlang-questions] ErlangBA 2015 - CFP
Message-ID: <31A2DAFA-5AE2-44EA-8286-96628B8315F2@inaka.net>

Hi everybody, 

We just started organizing the ErlangBA 2015 . We're currently looking for speakers. Do you have something to share with the Erlang community around Buenos Aires? Have you developed something with Elixir worth showing? Your company realised that they needed Erlang or Elixir to grow and you want to tell us why? This is your place!
Send us your talk proposal through this form .
We'll be receiving proposals until May 15th. We welcome remote speakers that want to share their talks with us using google-hangout, skype or other similar technology.
Cheers!
 

Fernando "Brujo" Benavides
about.me/elbrujohalcon

  				
 



-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From petergi@REDACTED  Wed May  6 06:23:40 2015
From: petergi@REDACTED (Peter J Etheridge)
Date: Wed, 06 May 2015 14:23:40 +1000
Subject: [erlang-questions]
	
Message-ID: 

thank you for your insights and suggestions, tuncer.
best regards,
peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From lee.sylvester@REDACTED  Wed May  6 11:04:19 2015
From: lee.sylvester@REDACTED (Lee Sylvester)
Date: Wed, 06 May 2015 10:04:19 +0100
Subject: [erlang-questions] Erlang update on CentOS
Message-ID: <5549D913.8050701@gmail.com>

Hey guys,

So, I'm updating a server from R16.1B to 17.5.  I now have 17 existing 
in /usr/lib64/erlang and R16B in /usr/local/lib/erlang, but when I run 
erl, it runs the R16B version.

Does anyone know where the path is that I need to change to update the 
wanted erlang install location?

Thanks,
Lee


From bengt.kleberg@REDACTED  Wed May  6 11:06:16 2015
From: bengt.kleberg@REDACTED (Bengt Kleberg)
Date: Wed, 6 May 2015 11:06:16 +0200
Subject: [erlang-questions] Erlang update on CentOS
In-Reply-To: <5549D913.8050701@gmail.com>
References: <5549D913.8050701@gmail.com>
Message-ID: <5549D988.3070608@ericsson.com>

Greetings,

What do you get from "which erl"?


bengt

On 05/06/2015 11:04 AM, Lee Sylvester wrote:
> Hey guys,
>
> So, I'm updating a server from R16.1B to 17.5.  I now have 17 existing 
> in /usr/lib64/erlang and R16B in /usr/local/lib/erlang, but when I run 
> erl, it runs the R16B version.
>
> Does anyone know where the path is that I need to change to update the 
> wanted erlang install location?
>
> Thanks,
> Lee
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions



From zandra@REDACTED  Wed May  6 11:43:19 2015
From: zandra@REDACTED (Zandra Hird)
Date: Wed, 6 May 2015 11:43:19 +0200
Subject: [erlang-questions] Patch Package OTP 17.5.3 Released
Message-ID: <5549E237.9050503@erlang.org>

Patch Package:           OTP 17.5.3
Git Tag:                 OTP-17.5.3

  Check out the git tag OTP-17.5.3, and build a full OTP system
  including documentation. Apply one or more applications from this
  build as patches to your installation using the 'otp_patch_apply'
  tool. For information on install requirements, see descriptions for
  each application version below.

  ---------------------------------------------------------------------
  --- common_test-1.10.1 ----------------------------------------------
  ---------------------------------------------------------------------

  Note! The common_test-1.10.1 application can *not* be applied
        independently of other applications on an arbitrary OTP 17
        installation.

        On a full OTP 17 installation, also the following runtime
        dependency has to be satisfied:
        -- test_server-3.7.1 (first satisfied in OTP 17.1)


  --- Fixed Bugs and Malfunctions ---

   OTP-12643    Application(s): common_test

                A fault in the Common Test logger process, that caused
                the application to crash when running on a long name
                node, has been corrected.


   OTP-12688    Application(s): common_test
                Related Id(s): seq12818

                A 'wait_for_prompt' option in ct_telnet:expect/3 has
                been introduced which forces the function to not return
                until a prompt string has been received, even if other
                expect patterns have already been found.


   OTP-12697    Application(s): common_test, test_server
                Related Id(s): seq12848

                If the last expression in a test case causes a timetrap
                timeout, the stack trace is ignored and not printed to
                the test case log file. This happens because the
                {Suite,TestCase,Line} info is not available in the
                stack trace in this scenario, due to tail call
                elimination. Common Test has been modified to handle
                this situation by inserting a
                {Suite,TestCase,last_expr} tuple in the correct place
                and printing the stack trace as expected.


   OTP-12698    Application(s): common_test
                Related Id(s): seq12844

                Fixed a buffer problem in ct_netconfc which could cause
                that some messages where buffered forever.


   OTP-12704    Application(s): common_test, erts
                Related Id(s): OTP-10922

                The VTS mode in Common Test has been modified to use a
                private version of the Webtool application
                (ct_webtool).


   OTP-12707    Application(s): common_test
                Related Id(s): seq12846

                Add possibility to add user capabilities in
                ct_netconfc:hello/3.


  Full runtime dependencies of common_test-1.10.1: compiler-5.0,
  crypto-3.3, debugger-4.0, erts-6.0, inets-5.10, kernel-3.0,
  runtime_tools-1.8.14, sasl-2.4, snmp-4.25.1, ssh-3.0.1, stdlib-2.0,
  test_server-3.7.1, tools-2.6.14, webtool-0.8.10, xmerl-1.3.7


  ---------------------------------------------------------------------
  --- diameter-1.9.1 --------------------------------------------------
  ---------------------------------------------------------------------

  The diameter-1.9.1 application can be applied independently of other
  applications on a full OTP 17 installation.

  --- Known Bugs and Problems ---

   OTP-12642    Application(s): diameter

                Don't leave extra bit in decoded AVP data.

                OTP-12074 in OTP 17.3 missed one case: a length error
                on a trailing AVP unknown to the dictionary in
                question.


   OTP-12654    Application(s): diameter
                Related Id(s): seq12851

                Don't confuse Result-Code and Experimental-Result

                The errors field of a decoded diameter_packet record
                was populated with a Result-Code AVP when an
                Experimental-Result containing a 3xxx Result-Code was
                received in an answer not setting the E-bit. The
                correct AVP is now extracted from the incoming message.


   OTP-12701    Application(s): diameter

                Don't count on unknown Application Id.

                OTP-11721 in OTP 17.1 missed the case of an Application
                Id not agreeing with that of the dictionary in
                question, causing counters to be accumulated on keys
                containing the unknown id.


  Full runtime dependencies of diameter-1.9.1: erts-6.0, kernel-3.0,
  ssl-5.3.4, stdlib-2.0


  ---------------------------------------------------------------------
  --- erts-6.4.1 ------------------------------------------------------
  ---------------------------------------------------------------------

  The erts-6.4.1 application can be applied independently of other
  applications on a full OTP 17 installation.

  --- Fixed Bugs and Malfunctions ---

   OTP-12704    Application(s): common_test, erts
                Related Id(s): OTP-10922

                The VTS mode in Common Test has been modified to use a
                private version of the Webtool application
                (ct_webtool).


  Full runtime dependencies of erts-6.4.1: kernel-3.0, sasl-2.4,
  stdlib-2.0


  ---------------------------------------------------------------------
  --- snmp-5.1.2 ------------------------------------------------------
  ---------------------------------------------------------------------

  The snmp-5.1.2 application can be applied independently of other
  applications on a full OTP 17 installation.

  --- Fixed Bugs and Malfunctions ---

   OTP-12669    Application(s): snmp
                Related Id(s): seq12841

                A bug in the SNMP Agent has been corrected; when
                opening a port using the command line argument
                -snmpa_fd the Port should be 0 when calling
                gen_udp:open.

                A bug in the SNMP manager has been corrected; it should
                not look at the -snmp_fd command line argument, but
                instead at -snmpm_fd.


  --- Improvements and New Features ---

   OTP-12452    Application(s): snmp

                Improved cryptocraphic capability.


  Full runtime dependencies of snmp-5.1.2: crypto-3.3, erts-6.0,
  kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, stdlib-2.0


  ---------------------------------------------------------------------
  --- test_server-3.8.1 -----------------------------------------------
  ---------------------------------------------------------------------

  Note! The test_server-3.8.1 application can *not* be applied
        independently of other applications on an arbitrary OTP 17
        installation.

        On a full OTP 17 installation, also the following runtime
        dependency has to be satisfied:
        -- syntax_tools-1.6.16 (first satisfied in OTP 17.1.1)


  --- Fixed Bugs and Malfunctions ---

   OTP-12697    Application(s): common_test, test_server
                Related Id(s): seq12848

                If the last expression in a test case causes a timetrap
                timeout, the stack trace is ignored and not printed to
                the test case log file. This happens because the
                {Suite,TestCase,Line} info is not available in the
                stack trace in this scenario, due to tail call
                elimination. Common Test has been modified to handle
                this situation by inserting a
                {Suite,TestCase,last_expr} tuple in the correct place
                and printing the stack trace as expected.


  Full runtime dependencies of test_server-3.8.1: erts-6.0, inets-5.10,
  kernel-3.0, observer-2.0, runtime_tools-1.8.14, stdlib-2.0,
  syntax_tools-1.6.16, tools-2.6.14


  ---------------------------------------------------------------------
  ---------------------------------------------------------------------
  ---------------------------------------------------------------------


From zxq9@REDACTED  Wed May  6 12:11:35 2015
From: zxq9@REDACTED (zxq9)
Date: Wed, 06 May 2015 19:11:35 +0900
Subject: [erlang-questions] Erlang update on CentOS
In-Reply-To: <5549D913.8050701@gmail.com>
References: <5549D913.8050701@gmail.com>
Message-ID: <2887117.eREsdjHPy6@changa>

On 2015?5?6? ??? 10:04:19 Lee Sylvester wrote:
> Hey guys,
> 
> So, I'm updating a server from R16.1B to 17.5.  I now have 17 existing 
> in /usr/lib64/erlang and R16B in /usr/local/lib/erlang, but when I run 
> erl, it runs the R16B version.
> 
> Does anyone know where the path is that I need to change to update the 
> wanted erlang install location?

Since none of us know how your packages are installed (or if they are packages)...

To see what path is being found first, try:

  which erl

Using the "locate" command can be helpful, especially if you search for "bin/erlc" instead of just erl. On a Debian system, for example, I see:

  ceverett@REDACTED:~$ locate bin/erlc
  /usr/bin/erlc
  /usr/lib/erlang/bin/erlc
  /usr/lib/erlang/erts-6.4/bin/erlc

Looking a bit closer:

  ceverett@REDACTED:~$ which erlc
  /usr/bin/erlc

And looking even closer:

  ceverett@REDACTED:~$ ls -l $(which erlc)
  lrwxrwxrwx 1 root root 22  4?  2 19:23 /usr/bin/erlc -> ../lib/erlang/bin/erlc

Hopefully this gives you a place to start.

On a Fedora derivative You may want to use the alternatives command to simplify management of this sort of thing:
( http://zxq9.com/archives/934 )

-Craig


From lee.sylvester@REDACTED  Wed May  6 12:13:23 2015
From: lee.sylvester@REDACTED (Lee Sylvester)
Date: Wed, 06 May 2015 11:13:23 +0100
Subject: [erlang-questions] Erlang update on CentOS
In-Reply-To: <2887117.eREsdjHPy6@changa>
References: <5549D913.8050701@gmail.com> <2887117.eREsdjHPy6@changa>
Message-ID: <5549E943.80807@gmail.com>

Thanks zxq9,

I was able to solve it by editing the script in /usr/local/bin

Kind regards,
Lee


On 06/05/15 11:11, zxq9 wrote:
> On 2015?5?6? ??? 10:04:19 Lee Sylvester wrote:
>> Hey guys,
>>
>> So, I'm updating a server from R16.1B to 17.5.  I now have 17 existing
>> in /usr/lib64/erlang and R16B in /usr/local/lib/erlang, but when I run
>> erl, it runs the R16B version.
>>
>> Does anyone know where the path is that I need to change to update the
>> wanted erlang install location?
> Since none of us know how your packages are installed (or if they are packages)...
>
> To see what path is being found first, try:
>
>    which erl
>
> Using the "locate" command can be helpful, especially if you search for "bin/erlc" instead of just erl. On a Debian system, for example, I see:
>
>    ceverett@REDACTED:~$ locate bin/erlc
>    /usr/bin/erlc
>    /usr/lib/erlang/bin/erlc
>    /usr/lib/erlang/erts-6.4/bin/erlc
>
> Looking a bit closer:
>
>    ceverett@REDACTED:~$ which erlc
>    /usr/bin/erlc
>
> And looking even closer:
>
>    ceverett@REDACTED:~$ ls -l $(which erlc)
>    lrwxrwxrwx 1 root root 22  4?  2 19:23 /usr/bin/erlc -> ../lib/erlang/bin/erlc
>
> Hopefully this gives you a place to start.
>
> On a Fedora derivative You may want to use the alternatives command to simplify management of this sort of thing:
> ( http://zxq9.com/archives/934 )
>
> -Craig
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions



From david.koch@REDACTED  Wed May  6 13:01:03 2015
From: david.koch@REDACTED (david.koch@REDACTED)
Date: Wed, 6 May 2015 13:01:03 +0200 (CEST)
Subject: [erlang-questions] =?utf-8?q?Re=C2=A0=3A_Re=3A__Patch_Package_OTP?=
 =?utf-8?q?_17=2E5=2E3_Released?=
In-Reply-To: <1237988676.427650.1430909917904.JavaMail.root@zimbra30-e5.priv.proxad.net>
Message-ID: <750071878.431566.1430910063833.JavaMail.root@zimbra30-e5.priv.proxad.net>

This, I guess :

https://github.com/erlang/otp/tree/OTP-17.5.3

----- Mail d'origine -----
De: Zandra Hird 
Envoy?: Wed, 06 May 2015 11:43:19 +0000 (CEST)
Objet: Re: [erlang-questions] Patch Package OTP 17.5.3 Released

Patch Package:           OTP 17.5.3
Git Tag:                 OTP-17.5.3

  Check out the git tag OTP-17.5.3, and build a full OTP system
  including documentation. Apply one or more applications from this
  build as patches to your installation using the 'otp_patch_apply'
  tool. For information on install requirements, see descriptions for
  each application version below.

  ---------------------------------------------------------------------
  --- common_test-1.10.1 ----------------------------------------------
  ---------------------------------------------------------------------

  Note! The common_test-1.10.1 application can *not* be applied
        independently of other applications on an arbitrary OTP 17
        installation.

        On a full OTP 17 installation, also the following runtime
        dependency has to be satisfied:
        -- test_server-3.7.1 (first satisfied in OTP 17.1)


  --- Fixed Bugs and Malfunctions ---

   OTP-12643    Application(s): common_test

                A fault in the Common Test logger process, that caused
                the application to crash when running on a long name
                node, has been corrected.


   OTP-12688    Application(s): common_test
                Related Id(s): seq12818

                A 'wait_for_prompt' option in ct_telnet:expect/3 has
                been introduced which forces the function to not return
                until a prompt string has been received, even if other
                expect patterns have already been found.


   OTP-12697    Application(s): common_test, test_server
                Related Id(s): seq12848

                If the last expression in a test case causes a timetrap
                timeout, the stack trace is ignored and not printed to
                the test case log file. This happens because the
                {Suite,TestCase,Line} info is not available in the
                stack trace in this scenario, due to tail call
                elimination. Common Test has been modified to handle
                this situation by inserting a
                {Suite,TestCase,last_expr} tuple in the correct place
                and printing the stack trace as expected.


   OTP-12698    Application(s): common_test
                Related Id(s): seq12844

                Fixed a buffer problem in ct_netconfc which could cause
                that some messages where buffered forever.


   OTP-12704    Application(s): common_test, erts
                Related Id(s): OTP-10922

                The VTS mode in Common Test has been modified to use a
                private version of the Webtool application
                (ct_webtool).


   OTP-12707    Application(s): common_test
                Related Id(s): seq12846

                Add possibility to add user capabilities in
                ct_netconfc:hello/3.


  Full runtime dependencies of common_test-1.10.1: compiler-5.0,
  crypto-3.3, debugger-4.0, erts-6.0, inets-5.10, kernel-3.0,
  runtime_tools-1.8.14, sasl-2.4, snmp-4.25.1, ssh-3.0.1, stdlib-2.0,
  test_server-3.7.1, tools-2.6.14, webtool-0.8.10, xmerl-1.3.7


  ---------------------------------------------------------------------
  --- diameter-1.9.1 --------------------------------------------------
  ---------------------------------------------------------------------

  The diameter-1.9.1 application can be applied independently of other
  applications on a full OTP 17 installation.

  --- Known Bugs and Problems ---

   OTP-12642    Application(s): diameter

                Don't leave extra bit in decoded AVP data.

                OTP-12074 in OTP 17.3 missed one case: a length error
                on a trailing AVP unknown to the dictionary in
                question.


   OTP-12654    Application(s): diameter
                Related Id(s): seq12851

                Don't confuse Result-Code and Experimental-Result

                The errors field of a decoded diameter_packet record
                was populated with a Result-Code AVP when an
                Experimental-Result containing a 3xxx Result-Code was
                received in an answer not setting the E-bit. The
                correct AVP is now extracted from the incoming message.


   OTP-12701    Application(s): diameter

                Don't count on unknown Application Id.

                OTP-11721 in OTP 17.1 missed the case of an Application
                Id not agreeing with that of the dictionary in
                question, causing counters to be accumulated on keys
                containing the unknown id.


  Full runtime dependencies of diameter-1.9.1: erts-6.0, kernel-3.0,
  ssl-5.3.4, stdlib-2.0


  ---------------------------------------------------------------------
  --- erts-6.4.1 ------------------------------------------------------
  ---------------------------------------------------------------------

  The erts-6.4.1 application can be applied independently of other
  applications on a full OTP 17 installation.

  --- Fixed Bugs and Malfunctions ---

   OTP-12704    Application(s): common_test, erts
                Related Id(s): OTP-10922

                The VTS mode in Common Test has been modified to use a
                private version of the Webtool application
                (ct_webtool).


  Full runtime dependencies of erts-6.4.1: kernel-3.0, sasl-2.4,
  stdlib-2.0


  ---------------------------------------------------------------------
  --- snmp-5.1.2 ------------------------------------------------------
  ---------------------------------------------------------------------

  The snmp-5.1.2 application can be applied independently of other
  applications on a full OTP 17 installation.

  --- Fixed Bugs and Malfunctions ---

   OTP-12669    Application(s): snmp
                Related Id(s): seq12841

                A bug in the SNMP Agent has been corrected; when
                opening a port using the command line argument
                -snmpa_fd the Port should be 0 when calling
                gen_udp:open.

                A bug in the SNMP manager has been corrected; it should
                not look at the -snmp_fd command line argument, but
                instead at -snmpm_fd.


  --- Improvements and New Features ---

   OTP-12452    Application(s): snmp

                Improved cryptocraphic capability.


  Full runtime dependencies of snmp-5.1.2: crypto-3.3, erts-6.0,
  kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, stdlib-2.0


  ---------------------------------------------------------------------
  --- test_server-3.8.1 -----------------------------------------------
  ---------------------------------------------------------------------

  Note! The test_server-3.8.1 application can *not* be applied
        independently of other applications on an arbitrary OTP 17
        installation.

        On a full OTP 17 installation, also the following runtime
        dependency has to be satisfied:
        -- syntax_tools-1.6.16 (first satisfied in OTP 17.1.1)


  --- Fixed Bugs and Malfunctions ---

   OTP-12697    Application(s): common_test, test_server
                Related Id(s): seq12848

                If the last expression in a test case causes a timetrap
                timeout, the stack trace is ignored and not printed to
                the test case log file. This happens because the
                {Suite,TestCase,Line} info is not available in the
                stack trace in this scenario, due to tail call
                elimination. Common Test has been modified to handle
                this situation by inserting a
                {Suite,TestCase,last_expr} tuple in the correct place
                and printing the stack trace as expected.


  Full runtime dependencies of test_server-3.8.1: erts-6.0, inets-5.10,
  kernel-3.0, observer-2.0, runtime_tools-1.8.14, stdlib-2.0,
  syntax_tools-1.6.16, tools-2.6.14


  ---------------------------------------------------------------------
  ---------------------------------------------------------------------
  ---------------------------------------------------------------------


From bog495@REDACTED  Wed May  6 13:20:31 2015
From: bog495@REDACTED (Bogdan Andu)
Date: Wed, 6 May 2015 14:20:31 +0300
Subject: [erlang-questions] gen_event handler error_logger_file_h crashed
Message-ID: 

Hi,

The following line:
error_logger:info_msg("[~p]:~ts~n", [PPid, Doc])

prints (Doc is data from the wire - untouched):

[<0.99.0>]:
        
                
                        
                                AUTO-1232
                                
                                        Nume
Det
                                        
                                        
                                                Address
#1
                                                Address #2
\276 \320 \350 \352 \353 
                                                Address #3
- \403\356\342\x{219}\x{21B}\402\316\x{218}\x{21A}\302

City

Province

102030
                                                ro
                                        
                                
                                +40.233232313
                                +40.722565195
                                bogdan@REDACTED

                                
                                        
                                
                        
                

                5549ee7e3a208
        


it logs to the file log: /var/xpp/logs/error_log
and also to the stdout and then generates the error:
'gen_event handler error_logger_file_h crashed'

and logging is not performed anymore in file log,
only to stdout.

the app is running in shell mode

The full (lengthy) error follows (sorry for the length of error):

=ERROR REPORT==== 6-May-2015::13:37:39 ===
** gen_event handler error_logger_file_h crashed.
** Was installed in error_logger
** Last event was: {info_msg,<0.80.0>,
                             {<0.123.0>,"[~p]:~ts~n",
                              [<0.99.0>,
                               <<60,63,120,109,108,32,118,101,114,115,105,
                                 111,110,61,34,49,46,48,34,32,101,110,99,
                                 111,100,105,110,103,61,34,85,84,70,45,56,
                                 34,63,62,10,60,101,112,112,32,120,109,108,
                                 110,115,61,34,117,114,110,58,105,101,116,
                                 102,58,112,97,114,97,109,115,58,120,109,
                                 108,58,110,115,58,101,112,112,45,49,46,48,
                                 34,32,120,109,108,110,115,58,120,115,105,
                                 61,34,104,116,116,112,58,47,47,119,119,119,
                                 46,119,51,46,111,114,103,47,50,48,48,49,47,
                                 88,77,76,83,99,104,101,109,97,45,105,110,
                                 115,116,97,110,99,101,34,10,9,120,115,105,
                                 58,115,99,104,101,109,97,76,111,99,97,116,
                                 105,111,110,61,34,117,114,110,58,105,101,
                                 116,102,58,112,97,114,97,109,115,58,120,
                                 109,108,58,110,115,58,101,112,112,45,49,46,
                                 48,32,101,112,112,45,49,46,48,46,120,115,
                                 100,34,62,10,9,60,99,111,109,109,97,110,
                                 100,62,10,9,9,60,99,114,101,97,116,101,62,
                                 10,9,9,9,60,99,111,110,116,97,99,116,58,99,
                                 114,101,97,116,101,32,120,109,108,110,115,
                                 58,99,111,110,116,97,99,116,61,34,117,114,
                                 110,58,105,101,116,102,58,112,97,114,97,
                                 109,115,58,120,109,108,58,110,115,58,99,
                                 111,110,116,97,99,116,45,49,46,48,34,10,9,
                                 9,9,9,120,115,105,58,115,99,104,101,109,97,
                                 76,111,99,97,116,105,111,110,61,34,117,114,
                                 110,58,105,101,116,102,58,112,97,114,97,
                                 109,115,58,120,109,108,58,110,115,58,99,
                                 111,110,116,97,99,116,45,49,46,48,32,99,
                                 111,110,116,97,99,116,45,49,46,48,46,120,
                                 115,100,34,62,10,9,9,9,9,60,99,111,110,116,
                                 97,99,116,58,105,100,62,65,85,84,79,45,49,
                                 50,51,50,60,47,99,111,110,116,97,99,116,58,
                                 105,100,62,10,9,9,9,9,60,99,111,110,116,97,
                                 99,116,58,112,111,115,116,97,108,73,110,
                                 102,111,32,116,121,112,101,61,34,105,110,
                                 116,34,62,10,9,9,9,9,9,60,99,111,110,116,
                                 97,99,116,58,110,97,109,101,62,78,117,109,
                                 101,32,68,101,116,60,47,99,111,110,116,97,
                                 99,116,58,110,97,109,101,62,10,9,9,9,9,9,
                                 60,99,111,110,116,97,99,116,58,111,114,103,
                                 62,60,47,99,111,110,116,97,99,116,58,111,
                                 114,103,62,10,9,9,9,9,9,60,99,111,110,116,
                                 97,99,116,58,97,100,100,114,62,10,9,9,9,9,
                                 9,9,60,99,111,110,116,97,99,116,58,115,116,
                                 114,101,101,116,62,65,100,100,114,101,115,
                                 115,32,35,49,60,47,99,111,110,116,97,99,
                                 116,58,115,116,114,101,101,116,62,10,9,9,9,
                                 9,9,9,60,99,111,110,116,97,99,116,58,115,
                                 116,114,101,101,116,62,65,100,100,114,101,
                                 115,115,32,35,50,32,194,190,32,195,144,32,
                                 195,168,32,195,170,32,195,171,32,60,47,99,
                                 111,110,116,97,99,116,58,115,116,114,101,
                                 101,116,62,10,9,9,9,9,9,9,60,99,111,110,
                                 116,97,99,116,58,115,116,114,101,101,116,
                                 62,65,100,100,114,101,115,115,32,35,51,32,
                                 45,32,196,131,195,174,195,162,200,153,200,
                                 155,196,130,195,142,200,152,200,154,195,
                                 130,60,47,99,111,110,116,97,99,116,58,115,
                                 116,114,101,101,116,62,10,9,9,9,9,9,9,60,
                                 99,111,110,116,97,99,116,58,99,105,116,121,
                                 62,67,105,116,121,60,47,99,111,110,116,97,
                                 99,116,58,99,105,116,121,62,10,9,9,9,9,9,9,
                                 60,99,111,110,116,97,99,116,58,115,112,62,
                                 80,114,111,118,105,110,99,101,60,47,99,111,
                                 110,116,97,99,116,58,115,112,62,10,9,9,9,9,
                                 9,9,60,99,111,110,116,97,99,116,58,112,99,
                                 62,49,48,50,48,51,48,60,47,99,111,110,116,
                                 97,99,116,58,112,99,62,10,9,9,9,9,9,9,60,
                                 99,111,110,116,97,99,116,58,99,99,62,114,
                                 111,60,47,99,111,110,116,97,99,116,58,99,
                                 99,62,10,9,9,9,9,9,60,47,99,111,110,116,97,
                                 99,116,58,97,100,100,114,62,10,9,9,9,9,60,
                                 47,99,111,110,116,97,99,116,58,112,111,115,
                                 116,97,108,73,110,102,111,62,10,9,9,9,9,60,
                                 99,111,110,116,97,99,116,58,118,111,105,99,
                                 101,62,43,52,48,46,50,51,51,50,51,50,51,49,
                                 51,60,47,99,111,110,116,97,99,116,58,118,
                                 111,105,99,101,62,10,9,9,9,9,60,99,111,110,
                                 116,97,99,116,58,102,97,120,62,43,52,48,46,
                                 55,50,50,53,54,53,49,57,53,60,47,99,111,
                                 110,116,97,99,116,58,102,97,120,62,10,9,9,
                                 9,9,60,99,111,110,116,97,99,116,58,101,109,
                                 97,105,108,62,98,111,103,100,97,110,64,114,
                                 111,116,108,100,46,114,111,60,47,99,111,
                                 110,116,97,99,116,58,101,109,97,105,108,62,
                                 10,9,9,9,9,60,99,111,110,116,97,99,116,58,
                                 97,117,116,104,73,110,102,111,62,10,9,9,9,
                                 9,9,60,99,111,110,116,97,99,116,58,112,119,
                                 62,60,47,99,111,110,116,97,99,116,58,112,
                                 119,62,10,9,9,9,9,60,47,99,111,110,116,97,
                                 99,116,58,97,117,116,104,73,110,102,111,62,
                                 10,9,9,9,60,47,99,111,110,116,97,99,116,58,
                                 99,114,101,97,116,101,62,10,9,9,60,47,99,
                                 114,101,97,116,101,62,10,9,9,60,101,120,
                                 116,101,110,115,105,111,110,62,10,9,9,9,60,
                                 114,111,116,108,100,58,101,120,116,32,120,
                                 109,108,110,115,58,114,111,116,108,100,61,
                                 34,104,116,116,112,58,47,47,119,119,119,46,
                                 114,111,116,108,100,46,114,111,47,120,109,
                                 108,47,101,112,112,47,114,111,116,108,100,
                                 45,49,46,48,34,62,10,9,9,9,9,60,114,111,
                                 116,108,100,58,99,114,101,97,116,101,62,10,
                                 9,9,9,9,9,60,114,111,116,108,100,58,99,111,
                                 110,116,97,99,116,62,10,9,9,9,9,9,9,60,114,
                                 111,116,108,100,58,99,110,112,95,102,105,
                                 115,99,97,108,95,99,111,100,101,62,49,51,
                                 56,49,48,53,48,53,60,47,114,111,116,108,
                                 100,58,99,110,112,95,102,105,115,99,97,108,
                                 95,99,111,100,101,62,10,9,9,9,9,9,9,60,114,
                                 111,116,108,100,58,114,101,103,105,115,116,
                                 114,97,116,105,111,110,95,110,117,109,98,
                                 101,114,62,55,47,80,74,47,49,55,46,48,49,
                                 46,50,48,48,49,60,47,114,111,116,108,100,
                                 58,114,101,103,105,115,116,114,97,116,105,
                                 111,110,95,110,117,109,98,101,114,62,10,9,
                                 9,9,9,9,9,60,114,111,116,108,100,58,112,
                                 101,114,115,111,110,95,116,121,112,101,62,
                                 99,60,47,114,111,116,108,100,58,112,101,
                                 114,115,111,110,95,116,121,112,101,62,10,9,
                                 9,9,9,9,60,47,114,111,116,108,100,58,99,
                                 111,110,116,97,99,116,62,10,9,9,9,9,60,47,
                                 114,111,116,108,100,58,99,114,101,97,116,
                                 101,62,10,9,9,9,60,47,114,111,116,108,100,
                                 58,101,120,116,62,10,9,9,60,47,101,120,116,
                                 101,110,115,105,111,110,62,10,9,9,60,99,
                                 108,84,82,73,68,62,53,53,52,57,101,101,55,
                                 101,51,97,50,48,56,60,47,99,108,84,82,73,
                                 68,62,10,9,60,47,99,111,109,109,97,110,100,
                                 62,10,60,47,101,112,112,62>>]}}
** When handler state == {<0.46.0>,"/var/xpp/logs/error_log",[]}
** Reason == {no_translation,
                 [{io,format,
                      [<0.46.0>,
                       ["\n",61,"INFO
REPORT",61,61,61,61,32,"6",45,"May",45,

"2015",58,58,"13",58,"37",58,"39",32,61,61,61,"\n",91,
                        "<0.99.0>",93,58,

[60,63,120,109,108,32,118,101,114,115,105,111,110,61,

34,49,46,48,34,32,101,110,99,111,100,105,110,103,61,

34,85,84,70,45,56,34,63,62,10,60,101,112,112,32,120,

109,108,110,115,61,34,117,114,110,58,105,101,116,102,
                         58,112,97,114,97,109,115,58,120,109,108,58,110,115,

58,101,112,112,45,49,46,48,34,32,120,109,108,110,115,
                         58,120,115,105,61,34,104,116,116,112,58,47,47,119,
                         119,119,46,119,51,46,111,114,103,47,50,48,48,49,47,

88,77,76,83,99,104,101,109,97,45,105,110,115,116,97,

110,99,101,34,10,9,120,115,105,58,115,99,104,101,109,

97,76,111,99,97,116,105,111,110,61,34,117,114,110,58,

105,101,116,102,58,112,97,114,97,109,115,58,120,109,

108,58,110,115,58,101,112,112,45,49,46,48,32,101,112,

112,45,49,46,48,46,120,115,100,34,62,10,9,60,99,111,
                         109,109,97,110,100,62,10,9,9,60,99,114,101,97,116,
                         101,62,10,9,9,9,60,99,111,110,116,97,99,116,58,99,

114,101,97,116,101,32,120,109,108,110,115,58,99,111,
                         110,116,97,99,116,61,34,117,114,110,58,105,101,116,
                         102,58,112,97,114,97,109,115,58,120,109,108,58,110,

115,58,99,111,110,116,97,99,116,45,49,46,48,34,10,9,

9,9,9,120,115,105,58,115,99,104,101,109,97,76,111,99,

97,116,105,111,110,61,34,117,114,110,58,105,101,116,
                         102,58,112,97,114,97,109,115,58,120,109,108,58,110,
                         115,58,99,111,110,116,97,99,116,45,49,46,48,32,99,

111,110,116,97,99,116,45,49,46,48,46,120,115,100,34,

62,10,9,9,9,9,60,99,111,110,116,97,99,116,58,105,100,
                         62,65,85,84,79,45,49,50,51,50,60,47,99,111,110,116,

97,99,116,58,105,100,62,10,9,9,9,9,60,99,111,110,116,
                         97,99,116,58,112,111,115,116,97,108,73,110,102,111,

32,116,121,112,101,61,34,105,110,116,34,62,10,9,9,9,

9,9,60,99,111,110,116,97,99,116,58,110,97,109,101,62,

78,117,109,101,32,68,101,116,60,47,99,111,110,116,97,
                         99,116,58,110,97,109,101,62,10,9,9,9,9,9,60,99,111,

110,116,97,99,116,58,111,114,103,62,60,47,99,111,110,
                         116,97,99,116,58,111,114,103,62,10,9,9,9,9,9,60,99,

111,110,116,97,99,116,58,97,100,100,114,62,10,9,9,9,

9,9,9,60,99,111,110,116,97,99,116,58,115,116,114,101,

101,116,62,65,100,100,114,101,115,115,32,35,49,60,47,

99,111,110,116,97,99,116,58,115,116,114,101,101,116,

62,10,9,9,9,9,9,9,60,99,111,110,116,97,99,116,58,115,

116,114,101,101,116,62,65,100,100,114,101,115,115,32,

35,50,32,190,32,208,32,232,32,234,32,235,32,60,47,99,

111,110,116,97,99,116,58,115,116,114,101,101,116,62,
                         10,9,9,9,9,9,9,60,99,111,110,116,97,99,116,58,115,

116,114,101,101,116,62,65,100,100,114,101,115,115,32,
                         35,51,32,45,32,259,238,226,537,539,258,206,536,538,
                         194,60,47,99,111,110,116,97,99,116,58,115,116,114,
                         101,101,116,62,10,9,9,9,9,9,9,60,99,111,110,116,97,

99,116,58,99,105,116,121,62,67,105,116,121,60,47,99,

111,110,116,97,99,116,58,99,105,116,121,62,10,9,9,9,
                         9,9,9,60,99,111,110,116,97,99,116,58,115,112,62,80,
                         114,111,118,105,110,99,101,60,47,99,111,110,116,97,
                         99,116,58,115,112,62,10,9,9,9,9,9,9,60,99,111,110,
                         116,97,99,116,58,112,99,62,49,48,50,48,51,48,60,47,

99,111,110,116,97,99,116,58,112,99,62,10,9,9,9,9,9,9,
                         60,99,111,110,116,97,99,116,58,99,99,62,114,111,60,

47,99,111,110,116,97,99,116,58,99,99,62,10,9,9,9,9,9,

60,47,99,111,110,116,97,99,116,58,97,100,100,114,62,

10,9,9,9,9,60,47,99,111,110,116,97,99,116,58,112,111,
                         115,116,97,108,73,110,102,111,62,10,9,9,9,9,60,99,

111,110,116,97,99,116,58,118,111,105,99,101,62,43,52,
                         48,46,50,51,51,50,51,50,51,49,51,60,47,99,111,110,

116,97,99,116,58,118,111,105,99,101,62,10,9,9,9,9,60,
                         99,111,110,116,97,99,116,58,102,97,120,62,43,52,48,
                         46,55,50,50,53,54,53,49,57,53,60,47,99,111,110,116,

97,99,116,58,102,97,120,62,10,9,9,9,9,60,99,111,110,
                         116,97,99,116,58,101,109,97,105,108,62,98,111,103,
                         100,97,110,64,114,111,116,108,100,46,114,111,60,47,

99,111,110,116,97,99,116,58,101,109,97,105,108,62,10,
                         9,9,9,9,60,99,111,110,116,97,99,116,58,97,117,116,

104,73,110,102,111,62,10,9,9,9,9,9,60,99,111,110,116,
                         97,99,116,58,112,119,62,60,47,99,111,110,116,97,99,

116,58,112,119,62,10,9,9,9,9,60,47,99,111,110,116,97,

99,116,58,97,117,116,104,73,110,102,111,62,10,9,9,9,

60,47,99,111,110,116,97,99,116,58,99,114,101,97,116,

101,62,10,9,9,60,47,99,114,101,97,116,101,62,10,9,9,
                         60,101,120,116,101,110,115,105,111,110,62,10,9,9,9,

60,114,111,116,108,100,58,101,120,116,32,120,109,108,

110,115,58,114,111,116,108,100,61,34,104,116,116,112,
                         58,47,47,119,119,119,46,114,111,116,108,100,46,114,

111,47,120,109,108,47,101,112,112,47,114,111,116,108,

100,45,49,46,48,34,62,10,9,9,9,9,60,114,111,116,108,

100,58,99,114,101,97,116,101,62,10,9,9,9,9,9,60,114,

111,116,108,100,58,99,111,110,116,97,99,116,62,10,9,
                         9,9,9,9,9,60,114,111,116,108,100,58,99,110,112,95,

102,105,115,99,97,108,95,99,111,100,101,62,49,51,56,
                         49,48,53,48,53,60,47,114,111,116,108,100,58,99,110,

112,95,102,105,115,99,97,108,95,99,111,100,101,62,10,
                         9,9,9,9,9,9,60,114,111,116,108,100,58,114,101,103,

105,115,116,114,97,116,105,111,110,95,110,117,109,98,

101,114,62,55,47,80,74,47,49,55,46,48,49,46,50,48,48,

49,60,47,114,111,116,108,100,58,114,101,103,105,115,

116,114,97,116,105,111,110,95,110,117,109,98,101,114,

62,10,9,9,9,9,9,9,60,114,111,116,108,100,58,112,101,
                         114,115,111,110,95,116,121,112,101,62,99,60,47,114,
                         111,116,108,100,58,112,101,114,115,111,110,95,116,
                         121,112,101,62,10,9,9,9,9,9,60,47,114,111,116,108,

100,58,99,111,110,116,97,99,116,62,10,9,9,9,9,60,47,

114,111,116,108,100,58,99,114,101,97,116,101,62,10,9,

9,9,60,47,114,111,116,108,100,58,101,120,116,62,10,9,

9,60,47,101,120,116,101,110,115,105,111,110,62,10,9,
                         9,60,99,108,84,82,73,68,62,53,53,52,57,101,101,55,

101,51,97,50,48,56,60,47,99,108,84,82,73,68,62,10,9,

60,47,99,111,109,109,97,110,100,62,10,60,47,101,112,
                         112,62],
                        "\n"],
                       []],
                      []},
                  {error_logger_file_h,handle_event,2,
                      [{file,"error_logger_file_h.erl"},{line,63}]},
                  {gen_event,server_update,4,
                      [{file,"gen_event.erl"},{line,525}]},
                  {gen_event,server_notify,4,
                      [{file,"gen_event.erl"},{line,507}]},

{gen_event,handle_msg,5,[{file,"gen_event.erl"},{line,248}]},
                  {proc_lib,init_p_do_apply,3,
                      [{file,"proc_lib.erl"},{line,237}]}]}

------END-OF-ERROR--------

it seems error_logger is unable to
interpret romanian diacritics.

without t modifier from "[~p]:~ts~n"
the crash does not happens.

What can be done to at least make
error_logger log to the file log again
after this crash occurs?

Bogdan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roger@REDACTED  Wed May  6 13:32:51 2015
From: roger@REDACTED (Roger Lipscombe)
Date: Wed, 6 May 2015 12:32:51 +0100
Subject: [erlang-questions] remote service discovery?
In-Reply-To: 
References: 
Message-ID: 

Define "services"...?

On 5 May 2015 at 02:41, Ben Hsu  wrote:
> Hello Erlangers
>
> I'm curious, is there a way to ask a remote erlang node what services it
> provides? either in core Erlang or OTP?
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>


From vychodil.hynek@REDACTED  Wed May  6 15:01:41 2015
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Wed, 6 May 2015 15:01:41 +0200
Subject: [erlang-questions] remote service discovery?
In-Reply-To: 
References: 
Message-ID: 

Poor man's service discovery:

[{Node, rpc:call(Node, application, which_applications, [])} || Node <-
nodes() ]

It returns list of all application installed in all nodes in a form:
[{'foo@REDACTED',[{stdlib,"ERTS  CXC 138 10","2.2"},
                        {kernel,"ERTS  CXC 138 10","3.0.3"}]}]


On Tue, May 5, 2015 at 3:41 AM, Ben Hsu  wrote:

> Hello Erlangers
>
> I'm curious, is there a way to ask a remote erlang node what services it
> provides? either in core Erlang or OTP?
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From benhsu@REDACTED  Wed May  6 13:43:19 2015
From: benhsu@REDACTED (Ben Hsu)
Date: Wed, 6 May 2015 07:43:19 -0400
Subject: [erlang-questions] remote service discovery?
In-Reply-To: 
References: 
 
Message-ID: 

a list of all the registered processes on the remote node?

On Wed, May 6, 2015 at 7:32 AM, Roger Lipscombe 
wrote:

> Define "services"...?
>
> On 5 May 2015 at 02:41, Ben Hsu  wrote:
> > Hello Erlangers
> >
> > I'm curious, is there a way to ask a remote erlang node what services it
> > provides? either in core Erlang or OTP?
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From john.foldager@REDACTED  Wed May  6 14:42:29 2015
From: john.foldager@REDACTED (John Foldager)
Date: Wed, 6 May 2015 14:42:29 +0200
Subject: [erlang-questions] Possibly to change TLS record size?
In-Reply-To: 
References: 
 
Message-ID: 

Hi Roger, I just found out that I was replying only to you and not the
mailing list, so I will reply with our conversations below:

ME:
This is EXACTLY what is needed. I used the
SSL_CTRL_SET_MAX_SEND_FRAGMENT to search and found you comment about
the patch on StackOverflow:
http://stackoverflow.com/questions/19276598/erlang-ssl-set-max-send-fragment-size

Any idea who could possible approve this patch for Erlang?

ROGER:
Not really, no. If you're interested in picking it up, I can forward
you Ingela's review comments on my original patch. 

ME:
Thanks. I would like to see Ingela's review comments if possible.


ROGER:
Ingela wrote:

> RFC 6066 obsoletes the RFC 4366 that according to my understanding specifies
> the extension for TLS 1.1 and TLS 1.0 also.  I think that that the best
> approach  would  be to implement handling of the max_fragment_length
> extension, that defaults to the current max if no extension is sent.  And
> also have an application environment variable as a way to change the default.
> (You can look at the session_lifetime application environment variable as an
> example.) The reason I would like to have it as an application environment
> variable is that I do not want to encourage the us of it, if you do not know
> exactly what you are doing, if it is a listen option it is so easily
> accessible!

> When it comes to header files I think that max_plain_text_length numbers
> belongs in ssl_record.hrl.  I could not see that this should be a problem,
> note that tls_record.hrl (includes ssl_record.hrl)


ME (now):
So, Ingela (or anyone else), could we have this TLS record size configurable?

On Tue, Apr 21, 2015 at 4:33 PM, Roger Lipscombe  wrote:
> On 21 April 2015 at 14:03, John Foldager  wrote:
>> Using RabbitMQ I would like to know if it is possible somehow to
>> configure/set the running Erlang process to change the size of the TLS
>> records?
>
> Do you mean an equivalent to OpenSSL's SSL_CTRL_SET_MAX_SEND_FRAGMENT option?
>
> As far as I know, it's not possible to set it in Erlang. Way back in
> Jan 2014, I had a patch (see
> https://github.com/rlipscombe/otp/commit/71c53d20191d3ddf43fc0aa87fabf5ac84ef70f3),
> but it didn't make it into OTP -- I had some feedback from Ingela on
> the OTP team, but didn't get around to dealing with it -- and I've not
> updated it since.


From benhsu@REDACTED  Wed May  6 16:07:17 2015
From: benhsu@REDACTED (Ben Hsu)
Date: Wed, 6 May 2015 10:07:17 -0400
Subject: [erlang-questions] remote service discovery?
In-Reply-To: 
References: 
 
Message-ID: 

Thank you! This helped!

On Wed, May 6, 2015 at 9:01 AM, Hynek Vychodil 
wrote:

> Poor man's service discovery:
>
> [{Node, rpc:call(Node, application, which_applications, [])} || Node <-
> nodes() ]
>
> It returns list of all application installed in all nodes in a form:
> [{'foo@REDACTED',[{stdlib,"ERTS  CXC 138 10","2.2"},
>                         {kernel,"ERTS  CXC 138 10","3.0.3"}]}]
>
>
> On Tue, May 5, 2015 at 3:41 AM, Ben Hsu  wrote:
>
>> Hello Erlangers
>>
>> I'm curious, is there a way to ask a remote erlang node what services it
>> provides? either in core Erlang or OTP?
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From vychodil.hynek@REDACTED  Wed May  6 19:24:34 2015
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Wed, 6 May 2015 19:24:34 +0200
Subject: [erlang-questions] remote service discovery?
In-Reply-To: 
References: 
 
 
Message-ID: 

[ {Node, rpc:call(Node, erlang, registered, [])} || Node <- nodes() ]

On Wed, May 6, 2015 at 1:43 PM, Ben Hsu  wrote:

> a list of all the registered processes on the remote node?
>
> On Wed, May 6, 2015 at 7:32 AM, Roger Lipscombe 
> wrote:
>
>> Define "services"...?
>>
>> On 5 May 2015 at 02:41, Ben Hsu  wrote:
>> > Hello Erlangers
>> >
>> > I'm curious, is there a way to ask a remote erlang node what services it
>> > provides? either in core Erlang or OTP?
>> >
>> > _______________________________________________
>> > erlang-questions mailing list
>> > erlang-questions@REDACTED
>> > http://erlang.org/mailman/listinfo/erlang-questions
>> >
>>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From otpcoder@REDACTED  Thu May  7 12:32:47 2015
From: otpcoder@REDACTED (otpcoder)
Date: Thu, 7 May 2015 11:32:47 +0100
Subject: [erlang-questions] Reporting vulnerabilities in Erlang/OTP
Message-ID: 

Are there particular procedures in place for reporting vulnerabilities in
Erlang/OTP both with and without patches? I don't see anything specifically
related to security disclosures in the wiki [1].

Thanks

[1] https://github.com/erlang/otp/wiki
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From ngreco@REDACTED  Thu May  7 15:07:37 2015
From: ngreco@REDACTED (Nahuel Greco)
Date: Thu, 7 May 2015 10:07:37 -0300
Subject: [erlang-questions] Why intra-node messages are copied and not
	refcounted, rationale?
Message-ID: 

Given erlang data structures are immutable and acyclic, what's the
rationale for copying them when a process sends a message to another
process in the same node? What's the rationale of accepting a copying
cpu/memory overhead? Why a refcounting mechanism like the used for binaries
is not desirable for all the data structures?

Saludos,
Nahuel Greco.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From henrik@REDACTED  Thu May  7 16:01:45 2015
From: henrik@REDACTED (Henrik Nord)
Date: Thu, 7 May 2015 16:01:45 +0200
Subject: [erlang-questions] Reporting vulnerabilities in Erlang/OTP
In-Reply-To: 
References: 
Message-ID: <554B7049.1070402@erlang.org>

Hello

No there is no procedures other than that we would like you to keep it 
private.
If it is ssh/ssl related send a email to ingela@REDACTED
If it is something else or if you are having any other thoughts you can 
always send a email to kenneth@REDACTED or kenneth.lundin@REDACTED 
and he will make sure that it is handled by the correct person.


hope this helps.
/Henrik

On 2015-05-07 12:32, otpcoder wrote:
> Are there particular procedures in place for reporting vulnerabilities 
> in Erlang/OTP both with and without patches? I don't see anything 
> specifically related to security disclosures in the wiki [1].
>
> Thanks
>
> [1] https://github.com/erlang/otp/wiki
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-- 
/Henrik Nord Erlang/OTP

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From ingela.andin@REDACTED  Thu May  7 16:24:33 2015
From: ingela.andin@REDACTED (Ingela Andin)
Date: Thu, 7 May 2015 16:24:33 +0200
Subject: [erlang-questions] Fwd:  Possibly to change TLS record size?
In-Reply-To: 
References: 
 
 
 
Message-ID: 

Hi!

Let me put it this way.  If someone makes a PR for implementing the Maximum
Fragment Length extension from RFC 6066 we will accept the PR.
The implementation should be fairly straight forward. I could easily do it
myself, but when and if this will be prioritised by Ericsson so that I have
the time to do it, is an entirely different question.

Regards Ingela Erlang/OTP team - Ericsson AB

2015-05-06 14:42 GMT+02:00 John Foldager :

> Hi Roger, I just found out that I was replying only to you and not the
> mailing list, so I will reply with our conversations below:
>
> ME:
> This is EXACTLY what is needed. I used the
> SSL_CTRL_SET_MAX_SEND_FRAGMENT to search and found you comment about
> the patch on StackOverflow:
>
> http://stackoverflow.com/questions/19276598/erlang-ssl-set-max-send-fragment-size
>
> Any idea who could possible approve this patch for Erlang?
>
> ROGER:
> Not really, no. If you're interested in picking it up, I can forward
> you Ingela's review comments on my original patch.
> 
>
> ME:
> Thanks. I would like to see Ingela's review comments if possible.
> 
>
> ROGER:
> Ingela wrote:
>
> > RFC 6066 obsoletes the RFC 4366 that according to my understanding
> specifies
> > the extension for TLS 1.1 and TLS 1.0 also.  I think that that the best
> > approach  would  be to implement handling of the max_fragment_length
> > extension, that defaults to the current max if no extension is sent.  And
> > also have an application environment variable as a way to change the
> default.
> > (You can look at the session_lifetime application environment variable
> as an
> > example.) The reason I would like to have it as an application
> environment
> > variable is that I do not want to encourage the us of it, if you do not
> know
> > exactly what you are doing, if it is a listen option it is so easily
> > accessible!
>
> > When it comes to header files I think that max_plain_text_length numbers
> > belongs in ssl_record.hrl.  I could not see that this should be a
> problem,
> > note that tls_record.hrl (includes ssl_record.hrl)
>
>
> ME (now):
> So, Ingela (or anyone else), could we have this TLS record size
> configurable?
>
> On Tue, Apr 21, 2015 at 4:33 PM, Roger Lipscombe 
> wrote:
> > On 21 April 2015 at 14:03, John Foldager 
> wrote:
> >> Using RabbitMQ I would like to know if it is possible somehow to
> >> configure/set the running Erlang process to change the size of the TLS
> >> records?
> >
> > Do you mean an equivalent to OpenSSL's SSL_CTRL_SET_MAX_SEND_FRAGMENT
> option?
> >
> > As far as I know, it's not possible to set it in Erlang. Way back in
> > Jan 2014, I had a patch (see
> >
> https://github.com/rlipscombe/otp/commit/71c53d20191d3ddf43fc0aa87fabf5ac84ef70f3
> ),
> > but it didn't make it into OTP -- I had some feedback from Ingela on
> > the OTP team, but didn't get around to dealing with it -- and I've not
> > updated it since.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jesper.louis.andersen@REDACTED  Thu May  7 16:27:17 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Thu, 7 May 2015 16:27:17 +0200
Subject: [erlang-questions] Why intra-node messages are copied and not
 refcounted, rationale?
In-Reply-To: 
References: 
Message-ID: 

On Thu, May 7, 2015 at 3:07 PM, Nahuel Greco  wrote:

> What's the rationale of accepting a copying cpu/memory overhead? Why a
> refcounting mechanism like the used for binaries is not desirable for all
> the data structures?


Really good questions!

Why copy and not pass by reference? It is implementation specific, as the
Erlang language allows for both semantics, and there were a variant of the
VM which used varying reference-passing tricks in place of the current
copying solution. The biggest disadvantage is that if you send very large
messages, then the copying overhead is large and costs performance. But
there are also some important advantages to copying. Each process can have
its own heap arena for instance, so this breaks up the memory space into
many small chunks which can be individually garbage collected. Erlang is a
soft realtime system, so you can't afford long garbage collection pauses.
This is usually really good for GC pause times as they can be quite small.
The second big advantage is that the semantics are the same for local
processes as well as distributed processes. You *have* to copy if the
process lives on another node anyway. The third advantage is that data
becomes local to a process, and thus local to a processing core. There are
advantages to this model on a system where memory access is highly
non-uniform, because you can in theory pick memory close to the processing
core.

As an example, I have been brewing on some latency benchmarks for
webservers. A sneak peak is the median response time of this test:

out.go: 50.000%    1.72ms
out.undertow: 50.000%    1.53ms
out.cowboy: 50.000%    6.33ms

where clearly, Erlang is far slower than a Go or Undertow, a Java-based
webserver. But once we look at the 99.99th percentile, things are different:

out.go: 99.990%   58.75ms
out.undertow: 99.990%   47.90ms
out.cowboy: 99.990%   38.62ms

and at the 99.999th percentile it gets really funny:

out.go: 99.999%  216.45ms
out.undertow: 99.999%   64.61ms
out.cowboy: 99.999%   45.09ms

what you see here is garbage collection overhead because you have such
large heaps to traverse, or that there are some other factor imposing
additional latency for a few of the calls.

Why not use refcounting? One price of refcounting is unpredictable
performance. If you ref-count a large list and free it up, then the GC has
to get rid of this list, and it will take quite some time. Many refcounting
GC's runs this as a background job to handle this. In fact, Refcounting GCs
are dual to Mark&Sweep GCs[0]. In the binary heap, a binary can contain no
pointers, which means reclamation of memory is always constant.
Furthermore, without optimizations, refcounting GCs tend to perform badly.
With optimizations, they start to look like Mark&Sweep collectors, as
written in [0].

In other words, both decisions are design decisions which tend to yield
good soft realtime performance on current hardware, and predictable
performance, especially in the worst case. You may not want a system that
stalls for at least 216ms every 100.000th call. But there are no rules in
the Erlang semantics which constrains us from coming up with another
message passing scheme, should we invent one that is even better. It is
just that the current behavior is highly desirable in the systems where
Erlang tend to get used.

[0]
http://researcher.watson.ibm.com/researcher/files/us-bacon/Bacon04Unified.pdf


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From eric@REDACTED  Thu May  7 16:40:53 2015
From: eric@REDACTED (Eric Skoglund)
Date: Thu, 07 May 2015 16:40:53 +0200
Subject: [erlang-questions] Reporting vulnerabilities in Erlang/OTP
In-Reply-To: <554B7049.1070402@erlang.org>
References: 
 <554B7049.1070402@erlang.org>
Message-ID: <554B7975.1000201@pagefault.se>

I was at a meetup last night with some FOSS people and the question on
how to handle security bugs in open source projects came up. Why this
came up was due to a security bug that was found and there wasn't a
proper procedure set up, leading to the bug being made public before
everyone was properly notified.

I think it would be a good idea to have a discussion on how security
issues should be handled. So that something like the above can be prevented.

One thing that seems like it is popular for FOSS software is to have a
mail address specifically for security related bugs that a subset of
maintainers have access to (curl [0] or rails [1]). It might be a good
idea to set up security@REDACTED for something like this.

Just my 2 cents

// Eric Skoglund

[0] http://curl.haxx.se/docs/security.html
[1] http://rubyonrails.org/security/


From raimo+erlang-questions@REDACTED  Thu May  7 17:18:15 2015
From: raimo+erlang-questions@REDACTED (Raimo Niskanen)
Date: Thu, 7 May 2015 17:18:15 +0200
Subject: [erlang-questions] Reporting vulnerabilities in Erlang/OTP
In-Reply-To: <554B7975.1000201@pagefault.se>
References: 
 <554B7049.1070402@erlang.org> <554B7975.1000201@pagefault.se>
Message-ID: <20150507151815.GA7054@erix.ericsson.se>

On Thu, May 07, 2015 at 04:40:53PM +0200, Eric Skoglund wrote:
> I was at a meetup last night with some FOSS people and the question on
> how to handle security bugs in open source projects came up. Why this
> came up was due to a security bug that was found and there wasn't a
> proper procedure set up, leading to the bug being made public before
> everyone was properly notified.
> 
> I think it would be a good idea to have a discussion on how security
> issues should be handled. So that something like the above can be prevented.
> 
> One thing that seems like it is popular for FOSS software is to have a
> mail address specifically for security related bugs that a subset of
> maintainers have access to (curl [0] or rails [1]). It might be a good
> idea to set up security@REDACTED for something like this.

There is actually an erlang-security at erlang dot org that is intended for
this purpose.  security at erlang dot org goes to the website admin for
website security issues.

> 
> Just my 2 cents
> 
> // Eric Skoglund
> 
> [0] http://curl.haxx.se/docs/security.html
> [1] http://rubyonrails.org/security/
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB


From otpcoder@REDACTED  Thu May  7 17:43:57 2015
From: otpcoder@REDACTED (otpcoder)
Date: Thu, 7 May 2015 16:43:57 +0100
Subject: [erlang-questions] Reporting vulnerabilities in Erlang/OTP
In-Reply-To: <20150507151815.GA7054@erix.ericsson.se>
References: 
 <554B7049.1070402@erlang.org> <554B7975.1000201@pagefault.se>
 <20150507151815.GA7054@erix.ericsson.se>
Message-ID: 

On Thu, May 7, 2015 at 4:18 PM, Raimo Niskanen <
raimo+erlang-questions@REDACTED> wrote:

> On Thu, May 07, 2015 at 04:40:53PM +0200, Eric Skoglund wrote:
> > I was at a meetup last night with some FOSS people and the question on
> > how to handle security bugs in open source projects came up. Why this
> > came up was due to a security bug that was found and there wasn't a
> > proper procedure set up, leading to the bug being made public before
> > everyone was properly notified.
> >
> > I think it would be a good idea to have a discussion on how security
> > issues should be handled. So that something like the above can be
> prevented.
> >
> > One thing that seems like it is popular for FOSS software is to have a
> > mail address specifically for security related bugs that a subset of
> > maintainers have access to (curl [0] or rails [1]). It might be a good
> > idea to set up security@REDACTED for something like this.
>
> There is actually an erlang-security at erlang dot org that is intended for
> this purpose.  security at erlang dot org goes to the website admin for
> website security issues.
>

So is erlang-security at erlang dot org the right address to use, rather
than
Ingela or Kenneth's personal addresses?

I agree with Eric that having a formal procedure is a good idea. You should
decide on some procedure, no matter how minimal, and publish it on the
wiki. In the absence of such guidance it's very easy to assume that the
right
thing to do is to push a topic branch and issue a pull request.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From eric@REDACTED  Thu May  7 18:47:17 2015
From: eric@REDACTED (Eric Skoglund)
Date: Thu, 07 May 2015 18:47:17 +0200
Subject: [erlang-questions] Reporting vulnerabilities in Erlang/OTP
In-Reply-To: <20150507151815.GA7054@erix.ericsson.se>
References: 
 <554B7049.1070402@erlang.org> <554B7975.1000201@pagefault.se>
 <20150507151815.GA7054@erix.ericsson.se>
Message-ID: <554B9715.40607@pagefault.se>



On 05/07/2015 05:18 PM, Raimo Niskanen wrote:
> On Thu, May 07, 2015 at 04:40:53PM +0200, Eric Skoglund wrote:
>> I was at a meetup last night with some FOSS people and the question on
>> how to handle security bugs in open source projects came up. Why this
>> came up was due to a security bug that was found and there wasn't a
>> proper procedure set up, leading to the bug being made public before
>> everyone was properly notified.
>>
>> I think it would be a good idea to have a discussion on how security
>> issues should be handled. So that something like the above can be prevented.
>>
>> One thing that seems like it is popular for FOSS software is to have a
>> mail address specifically for security related bugs that a subset of
>> maintainers have access to (curl [0] or rails [1]). It might be a good
>> idea to set up security@REDACTED for something like this.
> 
> There is actually an erlang-security at erlang dot org that is intended for
> this purpose.  security at erlang dot org goes to the website admin for
> website security issues.
> 
>>

That's great :), although I can't seem to find that information
anywhere. It might be a good idea to publish this information on the
website and github.

// Eric



From community-manager@REDACTED  Fri May  8 11:56:56 2015
From: community-manager@REDACTED (Bruce Yinhe)
Date: Fri, 8 May 2015 11:56:56 +0200
Subject: [erlang-questions] New Erlang job openings at Erlang Central
Message-ID: 

Many exciting Erlang career opportunities are available at Erlang Central.
https://erlangcentral.org/jobs

To publish new Erlang jobs, free of charge:
https://erlangcentral.org/jobs/add

To subscribe for weekly Erlang job updates
https://erlangcentral.org/login/?action=register

*Americas*

Software Developer - Home Based at Basho Technologies - Home Based, USA
https://erlangcentral.org/software-developer-home-based-basho/

Software Developer Testing- Home Based at Basho Technologies - Home Based,
USA
http://erlangcentral.org/erlang-developer-competitive-salary-with-stock-options-expend-io/

Principal Software Engineer - Home Based at Basho Technologies - Home
Based, USA
https://erlangcentral.org/principal-software-engineer-home-based-basho/

Senior Software Developer - Home Based at Basho Technologies - Home Based,
USA
https://erlangcentral.org/senior-software-developer-home-based-basho/

Erlang Developer at Sqor Sports - San Francisco, CA, USA
https://erlangcentral.org/erlang-developer-sqor/

Erlang Developer at Inaka - Buenos Aires, Argentina
https://erlangcentral.org/erlang-developer-inaka/

Senior Staff & Lead Erlang Engineers at Raretec Consulting - California,
CA, USA
https://erlangcentral.org/senior-staff-lead-erlang-engineers-raretec-consulting/

Lead Functional Programmer at Visa Inc. - Foster City, CA, USA
https://erlangcentral.org/visa-lead-functional-programmer/

Sr. Staff SW Engineer at Visa Inc. - Foster City, CA, USA
https://erlangcentral.org/sr-staff-sw-engineer-visa/

Senior Software Engineer ? Distributed Systems at Couchbase - Mountain
View, CA, USA
https://erlangcentral.org/senior-software-engineer-distributed-systems-couchbase/

Software Engineer ? Tools and Packaging Development at Couchbase - Mountain
View, CA, USA
https://erlangcentral.org/software-engineer-tools-and-packaging-development-couchbase/

Erlang Developer at Tigertext - Mountain View, CA, USA
http://erlangcentral.org/erlang-developer-tigertext/

Senior Erlang Software Developer at IBM Cloudant - Remote (USA), USA
https://erlangcentral.org/senior-erlang-software-developer-ibm-cloudant/

Software Engineer at Riot Games - Saint Louis, MO. Santa Monica, CA, USA
http://erlangcentral.org/software-engineer-riot-games/

Sr. Engineer and Architect at Samvia - San Carlos, CA, USA
http://erlangcentral.org/sr-engineer-and-architect-samvia/

Senior Back End Engineer (Erlang) at Hyperledger - San Francisco, CA, USA
https://erlangcentral.org/senior-back-end-engineer-erlang-hyperledger/

Infrastructure Engineer at Getaround - San Francisco, CA, USA
http://erlangcentral.org/infrastructure-engineer-getaround/

Sr. Software Engineer, RTB at AdRoll - San Francisco, CA, USA
http://erlangcentral.org/sr-software-engineer-rtb-adroll/

*Europe*

Developer position in Stockholm at Erlang Solutions - Stockholm, Sweden
https://erlangcentral.org/developer-position-stockholm-erlang-solutions/

Erlang Developer at Klarna - Stockholm, Sweden
https://erlangcentral.org/erlang-developer-klarna/

RabbitMQ Implementation and Development Engineer at Erlang Solutions -
London, Stockholm, Krakow or Budapest,
https://erlangcentral.org/rabbitmq-implementation-and-development-engineer/

Erlang Developer (Junior/Senior) at Erlang Solutions - London, Budapest,
Krakow, Stockholm,
https://erlangcentral.org/erlang-developer-erlang-solutions/

Erlang Developer at EPSN Workforce Group - Frankfurt, Germany
http://erlangcentral.org/erlang-developer-epsn-workforce/

Back-end Developer at NGTI - Rotterdam, Netherlands
https://erlangcentral.org/back-end-developer-ngti-nl/

Senior Software Developer in Erlang at DEK Technologies Sweden AB -
Stockholm, Sweden
http://erlangcentral.org/senior-software-developer-in-erlang-dek-technologies/

Backend Engineer at Centralway - Zurich, Switzerland
https://erlangcentral.org/backend-engineer-centralway/

Senior Back-end Engineer at Spatch - London, UK
https://erlangcentral.org/senior-back-end-engineer-spatch/

Senior DevOps Engineer at Spatch - London, UK
https://erlangcentral.org/senior-devops-engineer-spatch/

Erlang Developer at Darwin Recruitment - London, UK
http://erlangcentral.org/erlang-developer-darwin-recruitment-bath/

Erlang Software Developers at AccionLabs - London, UK
https://erlangcentral.org/erlang-software-developers-accionlabs/

*Asia*

System Analyst (Erlang software engineer) at Chaatz Limited - Wan Chai,
Hong Kong, Hong Kong
https://erlangcentral.org/system-analyst-erlang-software-engineer-chaatz/


Bruce Yinhe


ErlangCentral.org | @ErlangCentral  | Your
Erlang community site
Sponsored by Industrial Erlang User Group

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From garry@REDACTED  Fri May  8 19:41:54 2015
From: garry@REDACTED (Garry Hodgson)
Date: Fri, 08 May 2015 13:41:54 -0400
Subject: [erlang-questions] can i tell what version or erlang a .beam was
	compiled with?
Message-ID: <554CF562.10307@research.att.com>

since we've upgraded from r15 to r17.4 we have been
having problems with existing code that either crashes,
or runs for hours then suddenly doubles memory consumption
until oom-killer kills it. in either case we see no crash dump
to mine for clues.

one possibility could be that we've overlooked some modules
in the system that may still be running r15 code. i would have
expected that erlang would balk at that, but i don't know that
for sure. is there any way i can determine from a .beam file
what version of erlang was used to compile it? or, for that
matter, any other way to rule out or confirm this possibility?

we're kind of grasping at straws here.

thanks

-- 
Garry Hodgson
Lead Member of Technical Staff
AT&T Chief Security Office (CSO)

"This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited."



From roger@REDACTED  Fri May  8 20:28:46 2015
From: roger@REDACTED (Roger Lipscombe)
Date: Fri, 8 May 2015 19:28:46 +0100
Subject: [erlang-questions] can i tell what version or erlang a .beam
 was compiled with?
In-Reply-To: <554CF562.10307@research.att.com>
References: <554CF562.10307@research.att.com>
Message-ID: 

On 8 May 2015 at 18:41, Garry Hodgson  wrote:
> is there any way i can determine from a .beam file
> what version of erlang was used to compile it? or, for that
> matter, any other way to rule out or confirm this possibility?

beam_lib has what you need:

<0.39.0> (myapp@REDACTED) 4> {_, Beam, _} = code:get_object_code(foo).
{foo,<<70,79,82,49,0,1,102,184,66,69,65,77,65,116,
               111,109,0,0,17,242,0,0,1,51,11,105,109,...>>,
             "/home/roger/Private/Source/myapp/apps/myapp/ebin/foo.beam"}


<0.39.0> (myapp@REDACTED) 5> beam_lib:chunks(Beam, [compile_info]).
{ok,{foo,[{compile_info,[{options,[{outdir,"ebin"},
                                           debug_info,
                                           {parse_transform,lager_transform},

warnings_as_errors,warn_unused_vars,warn_shadow_vars,

warn_unused_import,warn_unused_function,warn_bif_clash,

warn_unused_record,warn_deprecated_function,

warn_obsolete_guard,warn_export_vars,warn_exported_vars,
                                           debug_info,
                                           {i,"include"}]},
                                 {version,"4.9.4"},
                                 {time,{2015,5,5,14,3,14}},

{source,"/home/roger/Private/Source/myapp/apps/myapp/src/foo.erl"}]}]}}

What's that "4.9.4"? It's the compiler version...

<0.39.0> (myapp@REDACTED) 6> code:which(compile).
"/usr/local/erlang-R16B03-1/lib/compiler-4.9.4/ebin/compile.beam"


From garry@REDACTED  Fri May  8 22:00:03 2015
From: garry@REDACTED (Garry Hodgson)
Date: Fri, 08 May 2015 16:00:03 -0400
Subject: [erlang-questions] can i tell what version or erlang a .beam
 was compiled with?
In-Reply-To: 
References: <554CF562.10307@research.att.com>
 
Message-ID: <554D15C3.50409@research.att.com>

On 05/08/2015 02:28 PM, Roger Lipscombe wrote:
> On 8 May 2015 at 18:41, Garry Hodgson  wrote:
>> is there any way i can determine from a .beam file
>> what version of erlang was used to compile it? or, for that
>> matter, any other way to rule out or confirm this possibility?
> beam_lib has what you need:
> ...
> What's that "4.9.4"? It's the compiler version...
>
> <0.39.0> (myapp@REDACTED) 6> code:which(compile).
> "/usr/local/erlang-R16B03-1/lib/compiler-4.9.4/ebin/compile.beam"
ahh, perfect. i looked at beam_lib, but missed that.
thanks very much.


-- 
Garry Hodgson
Lead Member of Technical Staff
AT&T Chief Security Office (CSO)

"This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited."



From ngreco@REDACTED  Sat May  9 05:34:39 2015
From: ngreco@REDACTED (Nahuel Greco)
Date: Sat, 9 May 2015 00:34:39 -0300
Subject: [erlang-questions] Why intra-node messages are copied and not
 refcounted, rationale?
In-Reply-To: 
References: 
 
Message-ID: 

Thanks Jesper for your crystal clear explanation. I wrongly assumed when
you appended two Refc binaries (via process local ProcBin's) a new
null-data Refc binary were created in the shared heap pointing to the
previous two. Imagining BEAM having the binaries stored in a persistent
data structure schema on a shared refcounted heap, my next thought was "why
not use it for all data structures?". Need to dive more in the BEAM
internals.


Saludos,
Nahuel Greco.

On Thu, May 7, 2015 at 11:27 AM, Jesper Louis Andersen <
jesper.louis.andersen@REDACTED> wrote:

>
> On Thu, May 7, 2015 at 3:07 PM, Nahuel Greco  wrote:
>
>> What's the rationale of accepting a copying cpu/memory overhead? Why a
>> refcounting mechanism like the used for binaries is not desirable for all
>> the data structures?
>
>
> Really good questions!
>
> Why copy and not pass by reference? It is implementation specific, as the
> Erlang language allows for both semantics, and there were a variant of the
> VM which used varying reference-passing tricks in place of the current
> copying solution. The biggest disadvantage is that if you send very large
> messages, then the copying overhead is large and costs performance. But
> there are also some important advantages to copying. Each process can have
> its own heap arena for instance, so this breaks up the memory space into
> many small chunks which can be individually garbage collected. Erlang is a
> soft realtime system, so you can't afford long garbage collection pauses.
> This is usually really good for GC pause times as they can be quite small.
> The second big advantage is that the semantics are the same for local
> processes as well as distributed processes. You *have* to copy if the
> process lives on another node anyway. The third advantage is that data
> becomes local to a process, and thus local to a processing core. There are
> advantages to this model on a system where memory access is highly
> non-uniform, because you can in theory pick memory close to the processing
> core.
>
> As an example, I have been brewing on some latency benchmarks for
> webservers. A sneak peak is the median response time of this test:
>
> out.go: 50.000%    1.72ms
> out.undertow: 50.000%    1.53ms
> out.cowboy: 50.000%    6.33ms
>
> where clearly, Erlang is far slower than a Go or Undertow, a Java-based
> webserver. But once we look at the 99.99th percentile, things are different:
>
> out.go: 99.990%   58.75ms
> out.undertow: 99.990%   47.90ms
> out.cowboy: 99.990%   38.62ms
>
> and at the 99.999th percentile it gets really funny:
>
> out.go: 99.999%  216.45ms
> out.undertow: 99.999%   64.61ms
> out.cowboy: 99.999%   45.09ms
>
> what you see here is garbage collection overhead because you have such
> large heaps to traverse, or that there are some other factor imposing
> additional latency for a few of the calls.
>
> Why not use refcounting? One price of refcounting is unpredictable
> performance. If you ref-count a large list and free it up, then the GC has
> to get rid of this list, and it will take quite some time. Many refcounting
> GC's runs this as a background job to handle this. In fact, Refcounting GCs
> are dual to Mark&Sweep GCs[0]. In the binary heap, a binary can contain no
> pointers, which means reclamation of memory is always constant.
> Furthermore, without optimizations, refcounting GCs tend to perform badly.
> With optimizations, they start to look like Mark&Sweep collectors, as
> written in [0].
>
> In other words, both decisions are design decisions which tend to yield
> good soft realtime performance on current hardware, and predictable
> performance, especially in the worst case. You may not want a system that
> stalls for at least 216ms every 100.000th call. But there are no rules in
> the Erlang semantics which constrains us from coming up with another
> message passing scheme, should we invent one that is even better. It is
> just that the current behavior is highly desirable in the systems where
> Erlang tend to get used.
>
> [0]
> http://researcher.watson.ibm.com/researcher/files/us-bacon/Bacon04Unified.pdf
>
>
> --
> J.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From kronic.deth@REDACTED  Sat May  9 06:12:09 2015
From: kronic.deth@REDACTED (Luke Imhoff)
Date: Fri, 8 May 2015 23:12:09 -0500
Subject: [erlang-questions] Reporting vulnerabilities in Erlang/OTP
In-Reply-To: <554B9715.40607@pagefault.se>
References: 
 <554B7049.1070402@erlang.org> <554B7975.1000201@pagefault.se>
 <20150507151815.GA7054@erix.ericsson.se>
 <554B9715.40607@pagefault.se>
Message-ID: 

I would like to suggest that erlang-security@REDACTED setup a disclosure
page on erlang.org with a link to it from the home page in the footer.  At
Rapid7 we also post a PGP key so that researchers can encrypt their
finding.  You can see our disclosure page here:
https://www.rapid7.com/disclosure.jsp.  We handle zero-day vulnerability
disclosures often with outside vendor in our work on Metasploit
Framework and we've helped open source contriibutors disclose to vendors.
Tod Beardsley  could help explain how he setup
Rapid7's program.  By the way, after the vulnerability is public,
https://github.com/rapid7/metasploit-framework/pulls will accept pull
request with a Metasploit Module that users of the software can use to test
if they are vulnerable.

On Thu, May 7, 2015 at 11:47 AM, Eric Skoglund  wrote:

>
>
> On 05/07/2015 05:18 PM, Raimo Niskanen wrote:
> > On Thu, May 07, 2015 at 04:40:53PM +0200, Eric Skoglund wrote:
> >> I was at a meetup last night with some FOSS people and the question on
> >> how to handle security bugs in open source projects came up. Why this
> >> came up was due to a security bug that was found and there wasn't a
> >> proper procedure set up, leading to the bug being made public before
> >> everyone was properly notified.
> >>
> >> I think it would be a good idea to have a discussion on how security
> >> issues should be handled. So that something like the above can be
> prevented.
> >>
> >> One thing that seems like it is popular for FOSS software is to have a
> >> mail address specifically for security related bugs that a subset of
> >> maintainers have access to (curl [0] or rails [1]). It might be a good
> >> idea to set up security@REDACTED for something like this.
> >
> > There is actually an erlang-security at erlang dot org that is intended
> for
> > this purpose.  security at erlang dot org goes to the website admin for
> > website security issues.
> >
> >>
>
> That's great :), although I can't seem to find that information
> anywhere. It might be a good idea to publish this information on the
> website and github.
>
> // Eric
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From barco@REDACTED  Sat May  9 09:51:09 2015
From: barco@REDACTED (Barco You)
Date: Sat, 9 May 2015 15:51:09 +0800
Subject: [erlang-questions] [ANN] CTO Position is open for Erlang enthusiast
Message-ID: <000f01d08a2c$e9c8a7e0$bd59f7a0$@com>

Dear Erlangers,

 

Here we are a Chinese company located in the most beautiful city- Shenzhen
very nearby Hongkong!

 

Dasudian is a startup founded by Barco You (an ex- Ericsson Engineer) and
several key persons in Chinese manufactures, which has a vision of changing
people's life and making it better and beautiful with Big Data .[for more
information about the company, please write a mail in person to
barco@REDACTED].

 

[Job Descriptions]

1.Work as a C-level member of the management team of the company

2. Report to CEO and lead the technology teams

3. Steer the technical direction of the company and do the high-level
management work of technology

4. Design the architecture of the systems of the Cloud Platform

5. Manage the daily R&D activities.

 

[Desired Skills and Expertise]

1.     Master Degree or above in EE, Computer science or relevant majors

2.     More than 5 years working experience in Telecommunication Systems,
Internet Systems, Cloud Computing, Big data or relevant areas.

3.     Expert in C/C++, Erlang programming

4.     Strong knowledge base and skills in Machine-Leaning algorithm design,
particularly experience in data mining technologies.

5.     Expert in NoSQL/SQL databases and DB clusters development

6.     Expert in Network programming and system design

7.     Fluent English Speaking and writing are mandatory

8.     Honest, active, optimistic and confident

9.     Willing to live in China

 

[Salary and Treatment]

1.     100,000 - 150,000 USD for yearly base

2.     Big bonus at year end

3.     Options

4.     Subsidies for house rental

 

If you have a dream and if you want to change the world, let's do it
together here. China is the best place to make that dream come true!

 

Please just send your resume to barco@REDACTED, if you have interest in
this position.

 

Thank you!

Barco You

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From ahe.sanath@REDACTED  Sat May  9 13:03:18 2015
From: ahe.sanath@REDACTED (Sanath Prasanna)
Date: Sat, 9 May 2015 16:33:18 +0530
Subject: [erlang-questions] Emysql: Switch to Redundant MySQl DB connection
	When Primary Mysql DB down
Message-ID: 

Hi all,
Pls let me know how to implement logic using emysql to switch from Active
MySQL DB to Redundant MySQL DB instance.
I have 2 Mysql DB Instance - Master-Master in 2 servers.
Br,
Sana Jonson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From yashgt@REDACTED  Sat May  9 15:40:38 2015
From: yashgt@REDACTED (Yash Ganthe)
Date: Sat, 9 May 2015 19:10:38 +0530
Subject: [erlang-questions] What makes erlang scalable?
Message-ID: 

I am working on an article describing fundamentals of technologies used by
scalable systems. I have worked on Erlang before in a self-learning
excercise. I have gone through several articles but have not been able to
answer the following questions:
1. What is in the implementation of Erlang that makes it scalable? What
makes it able to run concurrent processes more efficiently than
technologies like Java?
2. What is the relation between functional programming and parallelization?
With the declarative syntax of Erlang, do we achieve run-time efficiency?
3. Does process state not make it heavy? If we have thousands of concurrent
users and spawn and equal number of processes as gen_server or any other
equivalent pattern, each process would maintain a state. With so many
processes, will it not be a drain on the RAM?
4. If a process has to make DB operations and we spawn multiple instances
of that process, eventually the DB will become a bottleneck. This happens
even if we use traditional models like Apache-PHP. Almost every business
application needs DB access. What then do we gain from using Erlang?
5. How does process restart help? A process crashes when something is wrong
in its logic or in the data. OTP allows you to restart a process. If the
logic or data does not change, why would the process not crash again and
keep crashing always?

Most articles sing praises about Erlang citing its use in Facebook and
Whatsapp. I salute Erlang for being scalable, but also want to technically
justify its scalability.

Even if I find answers to these queries on an existing link, that will help.

Regards,
Yash
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From boozelclark@REDACTED  Sat May  9 15:55:29 2015
From: boozelclark@REDACTED (Chris Clark)
Date: Sat, 09 May 2015 13:55:29 +0000
Subject: [erlang-questions] EXMPP and Rebar
Message-ID: 

Hi

I am trying to use this (https://github.com/Zert/exmpp) rebar-ized fork of
EXMPP with rebar3.

I can compile and generate a release but when I try to run the release
starting exmpp give the error:

{error,{exmpp,{{shutdown,{failed_to_start_child,stringprep,

{port_driver,load,{open_error,-10},exmpp_stringprep}}},
               {exmpp,start,[normal,[]]}}}}

=INFO REPORT==== 9-May-2015::15:54:25 ===
    application: exmpp
    exited: {{shutdown,
                 {failed_to_start_child,stringprep,
                     {port_driver,load,{open_error,-10},exmpp_stringprep}}},
             {exmpp,start,[normal,[]]}}
    type: temporary


Does any one know what I may be doing wrong?

Thanks in advance.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From eric.pailleau@REDACTED  Sat May  9 16:18:00 2015
From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=)
Date: Sat, 09 May 2015 16:18:00 +0200
Subject: [erlang-questions] What makes erlang scalable?
In-Reply-To: 
Message-ID: 

Hi,
The original paper of Joe is a start,  even if Erlang and Otp had big evolutions since this time...

http://www1.erlang.se/publications/inap96.pdf



Le?9 mai 2015 15:40, Yash Ganthe  a ?crit?:
>
> I am working on an article describing fundamentals of technologies used by scalable systems. I have worked on Erlang before in a self-learning excercise. I have gone through several articles but have not been able to answer the following questions:
> 1. What is in the implementation of Erlang that makes it scalable? What makes it able to run concurrent processes more efficiently than technologies like Java?
> 2. What is the relation between functional programming and parallelization? With the declarative syntax of Erlang, do we achieve run-time efficiency?
> 3. Does process state not make it heavy? If we have thousands of concurrent users and spawn and equal number of processes as gen_server or any other equivalent pattern, each process would maintain a state. With so many processes, will it not be a drain on the RAM?
> 4. If a process has to make DB operations and we spawn multiple instances of that process, eventually the DB will become a bottleneck. This happens even if we use traditional models like Apache-PHP. Almost every business application needs DB access. What then do we gain from using Erlang?
> 5. How does process restart help? A process crashes when something is wrong in its logic or in the data. OTP allows you to restart a process. If the logic or data does not change, why would the process not crash again and keep crashing always?
>
> Most articles sing praises about Erlang citing its use in Facebook and Whatsapp. I salute Erlang for being scalable, but also want to technically justify its scalability.
>
> Even if I find answers to these queries on an existing link, that will help.
>
> Regards,
> Yash

From lars@REDACTED  Sat May  9 16:47:26 2015
From: lars@REDACTED (Lars Hesel Christensen)
Date: Sat, 09 May 2015 16:47:26 +0200
Subject: [erlang-questions] EXMPP and Rebar
In-Reply-To: 
References: 
Message-ID: <554E1DFE.7030308@larshesel.dk>

Hi

Since the error mentions stringprep which is a part (on my system) of
libidn11, maybe you don't have that library properly installed or you
have a wrong version?

Cheers,
Lars

On 05/09/2015 03:55 PM, Chris Clark wrote:
> Hi
> 
> I am trying to use this (https://github.com/Zert/exmpp) rebar-ized fork
> of EXMPP with rebar3. 
> 
> I can compile and generate a release but when I try to run the release
> starting exmpp give the error:
> 
> {error,{exmpp,{{shutdown,{failed_to_start_child,stringprep,
>                                                
> {port_driver,load,{open_error,-10},exmpp_stringprep}}},
>                {exmpp,start,[normal,[]]}}}}
> 
> =INFO REPORT==== 9-May-2015::15:54:25 ===
>     application: exmpp
>     exited: {{shutdown,
>                  {failed_to_start_child,stringprep,
>                      {port_driver,load,{open_error,-10},exmpp_stringprep}}},
>              {exmpp,start,[normal,[]]}}
>     type: temporary
> 
> 
> Does any one know what I may be doing wrong?
> 
> Thanks in advance.
> 
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

From boozelclark@REDACTED  Sat May  9 16:52:50 2015
From: boozelclark@REDACTED (Chris Clark)
Date: Sat, 09 May 2015 14:52:50 +0000
Subject: [erlang-questions] EXMPP and Rebar
In-Reply-To: <554E1DFE.7030308@larshesel.dk>
References: 
 <554E1DFE.7030308@larshesel.dk>
Message-ID: 

Thanks Lars

apt-get install libidn11 says its already on the latest version.
I have installed the regular exmpp from process one as per their guide and
if I run erl including my app source it works but when I try generate a
release with rebar3 and run the erl it stops Woking

I assumed that the rebar-ized version of exmpp includes all the
dependencies but i'm not sure how to tell what is missing and include it.


On Sat, May 9, 2015 at 4:47 PM Lars Hesel Christensen 
wrote:

> Hi
>
> Since the error mentions stringprep which is a part (on my system) of
> libidn11, maybe you don't have that library properly installed or you
> have a wrong version?
>
> Cheers,
> Lars
>
> On 05/09/2015 03:55 PM, Chris Clark wrote:
> > Hi
> >
> > I am trying to use this (https://github.com/Zert/exmpp) rebar-ized fork
> > of EXMPP with rebar3.
> >
> > I can compile and generate a release but when I try to run the release
> > starting exmpp give the error:
> >
> > {error,{exmpp,{{shutdown,{failed_to_start_child,stringprep,
> >
> > {port_driver,load,{open_error,-10},exmpp_stringprep}}},
> >                {exmpp,start,[normal,[]]}}}}
> >
> > =INFO REPORT==== 9-May-2015::15:54:25 ===
> >     application: exmpp
> >     exited: {{shutdown,
> >                  {failed_to_start_child,stringprep,
> >
> {port_driver,load,{open_error,-10},exmpp_stringprep}}},
> >              {exmpp,start,[normal,[]]}}
> >     type: temporary
> >
> >
> > Does any one know what I may be doing wrong?
> >
> > Thanks in advance.
> >
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From flexchap@REDACTED  Sat May  9 14:35:38 2015
From: flexchap@REDACTED (Peter Johansson)
Date: Sat, 9 May 2015 14:35:38 +0200
Subject: [erlang-questions] Retrieving "semi-constant" data from a function
	versus Mnesia
Message-ID: 

Hi to all fellow Erlang-users out there !

I'm working with a web-application prototype, powered by Erlang, at the
current time .....
and is for the moment preoccupied with the choose of a suitable
implementation regarding in-memory storage of shared/"top-level"
configuration-data terms.

This configuration-data terms holds ejson-structures ( typically 5KB - 15KB
in size ) and will be consulted by the majority of the request-related
processes based on cookie-association & parameters.

Since this configuration-data relatively rarely will undergo updates but
still be read by almost every request-process I consider it's in-memory
storage
implementation as highly significant for the process-efficiency over time &
shifting payload-situations.

In the case of Mnesia the configuration-terms have to be retrieved by the
means of transactions of table-records into the different process-heaps,
that means in-memory copy-operations which obviously will cause some
overhead in the environment during the occurrence of peak-like situations.

The other case (the function case) is to template the updated
ejson-structures (as the sole "return"-structures) into dedicated
function-definitions,
each definition hold in it's own module, and then recompile & reload those
modules programmatically via special update-functions. The up to date
configuration-data can then be retrieved by common processes as simple
function-calls returning fixed data.

I assume/expect these sole ejson/"return"-structures to be stored into the
constant-pools of the modules when these becomes loaded in memory.
In such case the call to any such function into a variable-bound should
result in the creation of a memory-reference for that variable pointing to
the fixed structure in the module's constant-pool.

Retrieving the configuration-data in this later manner must be
significantly more efficient compare to the case of transactions from
Mnesia if considering
both the sizes of the data-structures & the frequency under which they will
be consulted/read.

Is this assumption of mine correct/true or have I missed/overlooked
something in my assessment of the situation ?


Sending my best regards to you erlangers reading this !  / Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From flexchap@REDACTED  Sat May  9 14:55:43 2015
From: flexchap@REDACTED (Peter Johansson)
Date: Sat, 9 May 2015 14:55:43 +0200
Subject: [erlang-questions] Retrieving "semi-constant" data from a function
 versus Mnesia ( corrected line-breaks ! )
Message-ID: 

2015-05-09 14:35 GMT+02:00 Peter Johansson :

> Hi to all fellow Erlang-users out there !
>
> I'm working with a web-application prototype, powered by Erlang, at the
> current time ..... and is for the moment preoccupied with the choose of a
> suitable implementation regarding in-memory storage of shared/"top-level"
> configuration-data terms.
>
> This configuration-data terms holds ejson-structures ( typically 5KB -
> 15KB in size ) and will be consulted by the majority of the request-related
> processes based on cookie-association & parameters.
>
> Since this configuration-data relatively rarely will undergo updates but
> still be read by almost every request-process I consider it's in-memory
> storage implementation as highly significant for the process-efficiency
> over time & shifting payload-situations.
>
> In the case of Mnesia the configuration-terms have to be retrieved by the
> means of transactions of table-records into the different process-heaps,
> that means in-memory copy-operations which obviously will cause some
> overhead in the environment during the occurrence of peak-like situations.
>
> The other case (the function case) is to template the updated
> ejson-structures (as the sole "return"-structures) into dedicated
> function-definitions, each definition hold in it's own module, and then
> recompile & reload those modules programmatically via special
> update-functions. The up to date configuration-data can then be retrieved
> by common processes as simple function-calls returning fixed data.
>
> I assume/expect these sole ejson/"return"-structures to be stored into the
> constant-pools of the modules when these becomes loaded in memory. In such
> case the call to any such function into a variable-bound should result in
> the creation of a memory-reference for that variable pointing to the fixed
> structure in the module's constant-pool.
>
> Retrieving the configuration-data in this later manner must be
> significantly more efficient compare to the case of transactions from
> Mnesia if considering both the sizes of the data-structures & the frequency
> under which they will be consulted/read.
>
> Is this assumption of mine correct/true or have I missed/overlooked
> something in my assessment of the situation ?
>
>
> Sending my best regards to you erlangers reading this !  / Peter
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From mononcqc@REDACTED  Sat May  9 20:19:13 2015
From: mononcqc@REDACTED (Fred Hebert)
Date: Sat, 9 May 2015 14:19:13 -0400
Subject: [erlang-questions] EXMPP and Rebar
In-Reply-To: 
References: 
 <554E1DFE.7030308@larshesel.dk>
 
Message-ID: <20150509181911.GA39305@ferdmbp.local>

Rebar3 doesn't come with a port driver out of the box anymore. We didn't 
feel safe enough providing that environment and maintaining it moving 
forward, and instead decided to put that responsibility back to the 
people shipping C code.

The two options supported are to use makefiles in hooks, as described in 
http://www.rebar3.org/v3.0/docs/building-cc, or alternatively add the 
plugin that ports the rebar 2.x port compiler to rebar3: 
https://github.com/blt/port_compiler

Follow the instructions, and any of these would bring back that 
compatibility.

Regards,
Fred.


From rdm@REDACTED  Sat May  9 20:07:07 2015
From: rdm@REDACTED (Rich Morin)
Date: Sat, 9 May 2015 11:07:07 -0700
Subject: [erlang-questions] information on actor-based algorithms?
Message-ID: <4DC5D9AD-A03E-4FAD-884B-6983294B629A@cfcl.com>

Googling around, I've found very little information on actor-based
algorithms (ie, algorithms which rely on the actor model).  If you
can suggest any relevant resources or keywords to try, please do!

I'm particularly interested in using actors to play with property
graphs (ala Neo4j and TinkerPop), doing spreading activation, force-
directed layout calculations, etc.

-r

 -- 
http://www.cfcl.com/rdm           Rich Morin           rdm@REDACTED
http://www.cfcl.com/rdm/resume    San Bruno, CA, USA   +1 650-873-7841

Software system design, development, and documentation




From desired.mta@REDACTED  Sun May 10 01:37:07 2015
From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=)
Date: Sun, 10 May 2015 01:37:07 +0200
Subject: [erlang-questions] Retrieving "semi-constant" data from a
 function versus Mnesia
In-Reply-To: 
References: 
Message-ID: 

On Sat, May 9, 2015 at 2:35 PM, Peter Johansson  wrote:
> Hi to all fellow Erlang-users out there !
>
> I'm working with a web-application prototype, powered by Erlang, at the
> current time .....
> and is for the moment preoccupied with the choose of a suitable
> implementation regarding in-memory storage of shared/"top-level"
> configuration-data terms.

This is a great question! However, the answer might be not what you
really expect.

The usual approach suggested by Erlang community is:
1. Build the simplest thing possible. It be sufficient for you in 95%
of the cases. Most likely, your bottle-neck will be somewhere else,
...
2. ... unless you figure out it's not by measuring it. Measure.
3. After (1) and (2) you know your bottle-neck, and have a much better
idea how performance can be improved. Also, when you have the
measurements and the infrastructure, it's much easier to just try
multiple approaches and pick the fastest one.

For me the simplest solution for your problem is to keep the
configuration where it belongs: the application environment (retrieve
with application:get_env/3).

-- 
Motiejus Jak?tys


From ahe.sanath@REDACTED  Sun May 10 09:38:16 2015
From: ahe.sanath@REDACTED (Sanath Prasanna)
Date: Sun, 10 May 2015 13:08:16 +0530
Subject: [erlang-questions] Emysql: Switch to Redundant MySQl DB
 connection When Primary Mysql DB down
In-Reply-To: 
References: 
Message-ID: 

Any comment on below requirement?

On Sat, May 9, 2015 at 4:33 PM, Sanath Prasanna 
wrote:

> Hi all,
> Pls let me know how to implement logic using emysql to switch from Active
> MySQL DB to Redundant MySQL DB instance.
> I have 2 Mysql DB Instance - Master-Master in 2 servers.
> Br,
> Sana Jonson
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From petergi@REDACTED  Sun May 10 08:24:33 2015
From: petergi@REDACTED (Peter J Etheridge)
Date: Sun, 10 May 2015 16:24:33 +1000
Subject: [erlang-questions] and the argument is...?
Message-ID: <3f13067c82b13e494226b8e5d7ebba994dfdda20@webmail.iinet.net.au>

dear list,i am learning Erlang and trialling IntelliJ IDEA.when i try
to 'edit configurations' before 'debug',i get crash dumps with
messages like;N is an unbound variable,init terminating in do_boot().
when entering the details into 'edit configurations'IJ does not
specifically ask for arity,rather, 'argument'.
-module(fac/1). % from p.165 of joe's book
-export([main/1]).
?main([A])->
?I = list_to_integer(atom_to_list(A)),
?F = fac(I),
?io:format("factorial ~w = ~w~n", [I,F]),
?init:stop().

?fac(0) -> 1;
?fac(N) -> N*fac(N-1).

I am using the current version of Erlang and the latest upgrade for IJ
on Win 7, 64..
if fac is the module,and main is the function,then, what would you say
is the argument?


-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From kostis@REDACTED  Sun May 10 10:02:20 2015
From: kostis@REDACTED (Kostis Sagonas)
Date: Sun, 10 May 2015 10:02:20 +0200
Subject: [erlang-questions] and the argument is...?
In-Reply-To: <3f13067c82b13e494226b8e5d7ebba994dfdda20@webmail.iinet.net.au>
References: <3f13067c82b13e494226b8e5d7ebba994dfdda20@webmail.iinet.net.au>
Message-ID: <554F108C.3000303@cs.ntua.gr>

On 05/10/2015 08:24 AM, Peter J Etheridge wrote:
> when entering the details into 'edit configurations'
> IJ does not specifically ask for arity,
> rather, 'argument'.
>
> -module(fac/1). % from p.165 of joe's book

I do not have Joe's book handy, but I doubt it contains the line above. 
Change it to

-module(fac).

and save this in a fac.erl file and most likely you will be able to 
continue and improve your experience with Erlang.

Kostis


From eric.pailleau@REDACTED  Sun May 10 10:09:06 2015
From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=)
Date: Sun, 10 May 2015 10:09:06 +0200
Subject: [erlang-questions] Emysql: Switch to Redundant MySQl DB
 connection When Primary Mysql DB down
In-Reply-To: 
Message-ID: <172c3acf-d62b-4af5-be6a-c8a14dcfdca2@email.android.com>

Hi,
I suppose you can use the pool functionality :

https://github.com/Eonblast/Emysql/blob/master/README.md#Samples

Regards

Le?10 mai 2015 09:38, Sanath Prasanna  a ?crit?:
>
> Any comment on below requirement?
>
> On Sat, May 9, 2015 at 4:33 PM, Sanath Prasanna  wrote:
>>
>> Hi all,
>> Pls let me know how to implement logic using emysql to switch from Active MySQL DB to Redundant MySQL DB instance.
>> I have 2 Mysql DB Instance - Master-Master in 2 servers.
>> Br,
>> Sana Jonson
>
>

From hokan.stenholm@REDACTED  Sun May 10 10:21:20 2015
From: hokan.stenholm@REDACTED (=?windows-1252?Q?H=E5kan_Stenholm?=)
Date: Sun, 10 May 2015 10:21:20 +0200
Subject: [erlang-questions] and the argument is...?
In-Reply-To: <3f13067c82b13e494226b8e5d7ebba994dfdda20@webmail.iinet.net.au>
References: <3f13067c82b13e494226b8e5d7ebba994dfdda20@webmail.iinet.net.au>
Message-ID: <554F1500.3050707@bredband.net>

On 2015-05-10 08:24, Peter J Etheridge wrote:
> -module(fac/1). % from p.165 of joe's book
> -export([main/1]).
>  main([A])->
>  I = list_to_integer(atom_to_list(A)),
>  F = fac(I),
>  io:format("factorial ~w = ~w~n", [I,F]),
>  init:stop().
>
>  fac(0) -> 1;
>  fac(N) -> N*fac(N-1).
Replace
-module(fac/1).
with
-module(fac).

I ran the code via:

erl -s fac main ...

on the command line and it appears to work fine. But I have to say that 
it is probably simpler to compile using erlc on the command line and run 
the resulting .beam file interactively in the erlang shell (see the erl 
command) as you can then simplify the code into:

-module(fac).
-export([fac/1]).

  fac(0) -> 1;
  fac(N) -> N*fac(N-1).

This allow you to call fac:fac(...) directly, multiple times, with 
different input as well as many other things ...

The erlang shell is also much more convenient if you need to pass more 
complex arguments or if you want to pass the result of previously 
executed commands as input.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From boozelclark@REDACTED  Sun May 10 10:39:46 2015
From: boozelclark@REDACTED (Chris Clark)
Date: Sun, 10 May 2015 08:39:46 +0000
Subject: [erlang-questions] EXMPP and Rebar
In-Reply-To: <20150509181911.GA39305@ferdmbp.local>
References: 
 <554E1DFE.7030308@larshesel.dk>
 
 <20150509181911.GA39305@ferdmbp.local>
Message-ID: 

Thanks very much. I managed to get some help on the #rebar irc channel. I
started using the port compiler https://github.com/blt/port_compiler but I
still had a few issues with getting EXMPP compiled. Turns out that there
was something wrong with rebar3 but I believe that its being patched.

Thanks for your help.

On Sat, May 9, 2015 at 8:19 PM Fred Hebert  wrote:

> Rebar3 doesn't come with a port driver out of the box anymore. We didn't
> feel safe enough providing that environment and maintaining it moving
> forward, and instead decided to put that responsibility back to the
> people shipping C code.
>
> The two options supported are to use makefiles in hooks, as described in
> http://www.rebar3.org/v3.0/docs/building-cc, or alternatively add the
> plugin that ports the rebar 2.x port compiler to rebar3:
> https://github.com/blt/port_compiler
>
> Follow the instructions, and any of these would bring back that
> compatibility.
>
> Regards,
> Fred.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From ahe.sanath@REDACTED  Sun May 10 12:00:17 2015
From: ahe.sanath@REDACTED (Sanath Prasanna)
Date: Sun, 10 May 2015 15:30:17 +0530
Subject: [erlang-questions] Emysql: Switch to Redundant MySQl DB
 connection When Primary Mysql DB down
In-Reply-To: <172c3acf-d62b-4af5-be6a-c8a14dcfdca2@email.android.com>
References: 
 <172c3acf-d62b-4af5-be6a-c8a14dcfdca2@email.android.com>
Message-ID: 

Hi Eric,
Tx for info. However my program run in server A. My DB run on server B &
Server C. (master-master) I need to add 2 database to connection pool.
if one db down (Say Server B) , then prohram need to refer  Server C
connection. automatically through emysql.That is my requirement.
Br,

Sana Jonson

On Sun, May 10, 2015 at 1:39 PM, ?ric Pailleau 
wrote:

> Hi,
> I suppose you can use the pool functionality :
>
> https://github.com/Eonblast/Emysql/blob/master/README.md#Samples
>
> Regards
>
> Le 10 mai 2015 09:38, Sanath Prasanna  a ?crit :
> >
> > Any comment on below requirement?
> >
> > On Sat, May 9, 2015 at 4:33 PM, Sanath Prasanna 
> wrote:
> >>
> >> Hi all,
> >> Pls let me know how to implement logic using emysql to switch from
> Active MySQL DB to Redundant MySQL DB instance.
> >> I have 2 Mysql DB Instance - Master-Master in 2 servers.
> >> Br,
> >> Sana Jonson
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From tuncer.ayaz@REDACTED  Sun May 10 12:00:19 2015
From: tuncer.ayaz@REDACTED (Tuncer Ayaz)
Date: Sun, 10 May 2015 12:00:19 +0200
Subject: [erlang-questions] Observer Crashdump Viewer bug?
In-Reply-To: 
References: 
Message-ID: 

On Fri, Apr 24, 2015 at 10:35 PM, Tuncer Ayaz wrote:
> Can anyone else reproduce a responsive but blocked Observer
> gui following these steps?
>
> # install otp 18.0 from master
>
> # clone and build erlguten
> $ git clone https://github.com/richcarl/erlguten
> $ cd erlguten
> $ make
> $ ./erlguten test/test1.xml
>
> # start Observer use File->Examine Crash Dump to load the
> # erlguten crash dump
> $ erl
> 1> observer:start().
> ok
> WARNING: Found unexpected tag:scheduler
> WARNING: Found unexpected tag:scheduler
> WARNING: Found unexpected line in general information:
> Calling Thread
> WARNING: Found unexpected line in ETS info:
> Chain Length Avg
>
> # now it stalls in the GUI at "Processing ets"
>
> This does not happen if you instead start crashdump_viewer:start/0.

So, can nobody else reproduce this?


From eric.pailleau@REDACTED  Sun May 10 12:37:34 2015
From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=)
Date: Sun, 10 May 2015 12:37:34 +0200
Subject: [erlang-questions] Emysql: Switch to Redundant MySQl DB
 connection When Primary Mysql DB down
In-Reply-To: 
Message-ID: 

An HTML attachment was scrubbed...
URL: 

From ahe.sanath@REDACTED  Sun May 10 16:07:41 2015
From: ahe.sanath@REDACTED (Sanath Prasanna)
Date: Sun, 10 May 2015 19:37:41 +0530
Subject: [erlang-questions] Emysql: Switch to Redundant MySQl DB
 connection When Primary Mysql DB down
In-Reply-To: 
References: 
 
Message-ID: 

Hi Eric,
it means I can't do above using emysql?
Br,
Sana Johnson

On Sun, May 10, 2015 at 4:07 PM, ?ric Pailleau 
wrote:

> Hi,
>
> You can set different datasources to connection. See erlmysql...
> Regards
> Le 10 mai 2015 12:00, Sanath Prasanna  a ?crit :
>
> Hi Eric,
> Tx for info. However my program run in server A. My DB run on server B &
> Server C. (master-master) I need to add 2 database to connection pool.
> if one db down (Say Server B) , then prohram need to refer  Server C
> connection. automatically through emysql.That is my requirement.
> Br,
>
> Sana Jonson
>
> On Sun, May 10, 2015 at 1:39 PM, ?ric Pailleau 
> wrote:
>
> Hi,
> I suppose you can use the pool functionality :
>
> https://github.com/Eonblast/Emysql/blob/master/README.md#Samples
>
> Regards
>
> Le 10 mai 2015 09:38, Sanath Prasanna  a ?crit :
> >
> > Any comment on below requirement?
> >
> > On Sat, May 9, 2015 at 4:33 PM, Sanath Prasanna 
> wrote:
> >>
> >> Hi all,
> >> Pls let me know how to implement logic using emysql to switch from
> Active MySQL DB to Redundant MySQL DB instance.
> >> I have 2 Mysql DB Instance - Master-Master in 2 servers.
> >> Br,
> >> Sana Jonson
> >
> >
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From eric.pailleau@REDACTED  Sun May 10 16:45:20 2015
From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=)
Date: Sun, 10 May 2015 16:45:20 +0200
Subject: [erlang-questions] Emysql: Switch to Redundant MySQl DB
 connection When Primary Mysql DB down
In-Reply-To: 
Message-ID: 

An HTML attachment was scrubbed...
URL: 

From jesper.louis.andersen@REDACTED  Sun May 10 20:03:35 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Sun, 10 May 2015 20:03:35 +0200
Subject: [erlang-questions] What makes erlang scalable?
In-Reply-To: 
References: 
Message-ID: 

On Sat, May 9, 2015 at 3:40 PM, Yash Ganthe  wrote:

I think all of these are good questions. Let me try to answer them one by
one.

am working on an article describing fundamentals of technologies used by
> scalable systems.
>

First, you have to come up with a definition of what "scalable" means in
this setting. Usually when you add load to a system, its system behavior
changes. Usually "scalable" is meant to be understood such that there is no
noticeable change as we add load, or that the change in behavior is
graceful degradation. That is, if the system has a normal operating
capacity of 20 simultaneous users, and we add load to 100 simultaneous
users, we expect processing to either stay the same, or become roughly 5
times slower for each user. But we don't expect the system to fail for the
users, or take 1000 times as much processing time all of a sudden. Making
sure you know exactly what measurable quantity you are concerned about when
scaling the system up is important.


> 1. What is in the implementation of Erlang that makes it scalable? What
> makes it able to run concurrent processes more efficiently than
> technologies like Java?
>

Each concurrent process will take up resources. The more resources a
process takes, the less of them can be had. The Erlang BEAM VM is pretty
efficient and can have millions of processes. A naive Java-solution based
on threads will quickly come into trouble, running out of resources.
Furthermore, thread overhead is typically larger than process overhead in
the BEAM VM.

You can, however, implement the central aspects of the Erlang VM on top of
Java's JVM which gives you some of the same capabilities. See for instance,
the Erjang project. This yields many of the same advantages. The idea is
roughly the same: keep the processes in user-space and do not involve the
kernel at all. Then map a number of process schedulers onto threads toward
the kernel in an N:M threading architecture.


> 2. What is the relation between functional programming and
> parallelization? With the declarative syntax of Erlang, do we achieve
> run-time efficiency?
>

To execute part of a program in parallel, you need to have at least two
independent tasks that can be computed without depending directly on each
other. This would allow us to spread the work over two cores. If we have
8-way parallel sections, we can utilize 8 cores in principle and so on.

If you have a functional program without any side-effects, it tends to
expose many places where such independent tasks can be identified. In
principle, this means you have an easy way to turn a sequential program
into a parallel one. And this is the lure of the functional programming
paradigm w.r.t. parallel execution. It isn't limited to functional
programming only however. Logic programming (Prolog, Mercury) is also
highly amenable to parallel execution. Any model which looks like the Actor
model is also amenable because each actor can execute independently of each
other.

Interestingly, Erlang doesn't employ the "functional programming" parallel
methodology, but rather the one based on an actor-like model. There is no
part of the Erlang runtime that will try to run a list comprehension such
as [F(X) || X <- List] in parallel currently, though in many cases this
could be done.


> 3. Does process state not make it heavy? If we have thousands of
> concurrent users and spawn and equal number of processes as gen_server or
> any other equivalent pattern, each process would maintain a state. With so
> many processes, will it not be a drain on the RAM?
>

Erlang processes grows dynamically from around 1-1.5 Kilobytes of memory.
In practice this has proven to be quite efficient, even for a large number
of processes. it corresponds to around 1.5 Gigabyte of memory at one
million processes currently. For a server environment, this is hardly a
problematic drain. For a small embedded device, this can be a problem, if
you want to run millions of processes on it. But even in that space, RAM is
getting cheaper every day.


> 4. If a process has to make DB operations and we spawn multiple instances
> of that process, eventually the DB will become a bottleneck. This happens
> even if we use traditional models like Apache-PHP. Almost every business
> application needs DB access. What then do we gain from using Erlang?
>

The key difference is that we can keep part of the computation in memory
and only push it back to the DB whenever we want. In a classical PHP
application, state is lost between each invocation. Suppose we are
implementing a server keeping track of Chess games. In PHP we would have to
store the chess game state in the DB for each move made. In Erlang, we can
keep that state in-memory in a process. Every time 5 minutes pass, we send
a message to the DB asynchronously telling it what moves were made in the
meantime. This lowers the load on the DB considerably. In fact, we can
control the load on the DB by dynamically optimizing the interval at which
we store data back to the database.

The PHP solution would be able to track the in-application state in Redis
or Memcached. But this means we would still have to retrieve the state from
another unix-process whenever we wanted to do a move. This is usually a
waste of resources.

5. How does process restart help? A process crashes when something is wrong
> in its logic or in the data. OTP allows you to restart a process. If the
> logic or data does not change, why would the process not crash again and
> keep crashing always?
>

The gist of this has to do with non-deterministic errors. Many errors in
distributed/asynchronous systems are "spurious" in that they only occur if
certain events happens in certain orders. If the bad event-order is highly
unlikely, retrying the operation often suceeds and lets the system continue
on. Other times, a user will do things in a different order the next time,
and this order avoids the problem. In the network, TCP window stalls might
make data arrive in an opposite order than what was normally intended.

On the other hand, as you say, if there is a genuine programmer error and
we always hit the same code-path, then there is nothing fault tolerance can
do. Restarting the system has no effect and won't correct the error.


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jay@REDACTED  Sun May 10 22:04:57 2015
From: jay@REDACTED (Jay Nelson)
Date: Sun, 10 May 2015 13:04:57 -0700
Subject: [erlang-questions] Retrieving "semi-constant" data from a
	function versus Mnesia
Message-ID: 

I did an experiment recently with app.config vs https://github.com/jaynel/vbisect
vs compiled behaviour implementation of configuration. I was worried that
ets locks and row contention would penalize app.config as the number of
workers accessing the same config scaled up. I only tried it on my small
4-core laptop so far, I expect different results on a 16-core or more server.
(And all my apprehension may be misplaced as the app.config ets table
is almost always exclusively read-only.)

In theory the code-compiled constants would have the most distributed
access, followed by the binary encoded dictionary (vbisect) because a
single binary lives on the heap and is accessed from multiple places with
only a pointer pass, and lastly app.config. Turns out that the compiled
version was the slowest for one reason I completely overlooked and was
quite surprised by: Module:config_fun/N is 6x slower than a compiled
module:config_fun/N call according to the documentation.

This was enough of an impact to reduce the number of transactions to
Cassandra in half! (I was testing with https://github.com/duomark/elysium
which dynamically manages distributed sockets with stochastic decay
and 10x or more config accesses per query.)  When testing with app.config
vs binary there is a slight trend to ets slowdown with worker scale up, but
not very significant and I would want to see on a bigger machine with more
cores what the impact is.

My guess is that mochiglobal would outperform compiled code, IF you
have a constant module name. I was using a behaviour so that I could
swap configs and therefore had to use Module:fn(?) calls everywhere.
This slowdown would impact any behaviour (e.g., gen_server, gen_fsm)
you implement if it is a hotspot, as the slow function call mechanism is
fundamental to behaviour implementation.

jay



From erlang@REDACTED  Sun May 10 22:23:11 2015
From: erlang@REDACTED (Joe Armstrong)
Date: Sun, 10 May 2015 22:23:11 +0200
Subject: [erlang-questions] Retrieving "semi-constant" data from a
 function versus Mnesia
In-Reply-To: 
References: 
Message-ID: 

How large is the total data?

If it's small you could define this in a module and not use a database
or process at all.

-module(global_data).
-export(...)

data_one() ->
    ....

Then dynamically change and recompile the module. Any call
global_data:data_one() will pick up the value from the "latest version"

/Joe

On Sat, May 9, 2015 at 2:35 PM, Peter Johansson  wrote:
> Hi to all fellow Erlang-users out there !
>
> I'm working with a web-application prototype, powered by Erlang, at the
> current time .....
> and is for the moment preoccupied with the choose of a suitable
> implementation regarding in-memory storage of shared/"top-level"
> configuration-data terms.
>
> This configuration-data terms holds ejson-structures ( typically 5KB - 15KB
> in size ) and will be consulted by the majority of the request-related
> processes based on cookie-association & parameters.
>
> Since this configuration-data relatively rarely will undergo updates but
> still be read by almost every request-process I consider it's in-memory
> storage
> implementation as highly significant for the process-efficiency over time &
> shifting payload-situations.
>
> In the case of Mnesia the configuration-terms have to be retrieved by the
> means of transactions of table-records into the different process-heaps,
> that means in-memory copy-operations which obviously will cause some
> overhead in the environment during the occurrence of peak-like situations.
>
> The other case (the function case) is to template the updated
> ejson-structures (as the sole "return"-structures) into dedicated
> function-definitions,
> each definition hold in it's own module, and then recompile & reload those
> modules programmatically via special update-functions. The up to date
> configuration-data can then be retrieved by common processes as simple
> function-calls returning fixed data.
>
> I assume/expect these sole ejson/"return"-structures to be stored into the
> constant-pools of the modules when these becomes loaded in memory.
> In such case the call to any such function into a variable-bound should
> result in the creation of a memory-reference for that variable pointing to
> the fixed structure in the module's constant-pool.
>
> Retrieving the configuration-data in this later manner must be significantly
> more efficient compare to the case of transactions from Mnesia if
> considering
> both the sizes of the data-structures & the frequency under which they will
> be consulted/read.
>
> Is this assumption of mine correct/true or have I missed/overlooked
> something in my assessment of the situation ?
>
>
> Sending my best regards to you erlangers reading this !  / Peter
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>


From mjtruog@REDACTED  Sun May 10 22:28:43 2015
From: mjtruog@REDACTED (Michael Truog)
Date: Sun, 10 May 2015 13:28:43 -0700
Subject: [erlang-questions] Retrieving "semi-constant" data from a
 function versus Mnesia
In-Reply-To: 
References: 
Message-ID: <554FBF7B.3010208@gmail.com>

On 05/10/2015 01:04 PM, Jay Nelson wrote:
> I did an experiment recently with app.config vs https://github.com/jaynel/vbisect
> vs compiled behaviour implementation of configuration. I was worried that
You might want to try https://github.com/knutin/bisect too.


> ets locks and row contention would penalize app.config as the number of
> workers accessing the same config scaled up. I only tried it on my small
> 4-core laptop so far, I expect different results on a 16-core or more server.
> (And all my apprehension may be misplaced as the app.config ets table
> is almost always exclusively read-only.)
>
> In theory the code-compiled constants would have the most distributed
> access, followed by the binary encoded dictionary (vbisect) because a
> single binary lives on the heap and is accessed from multiple places with
> only a pointer pass, and lastly app.config. Turns out that the compiled
> version was the slowest for one reason I completely overlooked and was
> quite surprised by: Module:config_fun/N is 6x slower than a compiled
> module:config_fun/N call according to the documentation.
>
> This was enough of an impact to reduce the number of transactions to
> Cassandra in half! (I was testing with https://github.com/duomark/elysium
> which dynamically manages distributed sockets with stochastic decay
> and 10x or more config accesses per query.)  When testing with app.config
> vs binary there is a slight trend to ets slowdown with worker scale up, but
> not very significant and I would want to see on a bigger machine with more
> cores what the impact is.
>
> My guess is that mochiglobal would outperform compiled code, IF you
> have a constant module name. I was using a behaviour so that I could
> swap configs and therefore had to use Module:fn(?) calls everywhere.
> This slowdown would impact any behaviour (e.g., gen_server, gen_fsm)
> you implement if it is a hotspot, as the slow function call mechanism is
> fundamental to behaviour implementation.
>
> jay
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions



From bchesneau@REDACTED  Sun May 10 22:32:44 2015
From: bchesneau@REDACTED (Benoit Chesneau)
Date: Sun, 10 May 2015 20:32:44 +0000
Subject: [erlang-questions] Retrieving "semi-constant" data from a
 function versus Mnesia
In-Reply-To: 
References: 
 
Message-ID: 

On Sun, May 10, 2015 at 10:23 PM Joe Armstrong  wrote:

> How large is the total data?
>
> If it's small you could define this in a module and not use a database
> or process at all.
>

How small should it be? What if the compiled bean is about 888 kilobytes?

- benoit
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From publicityifl@REDACTED  Sun May 10 22:02:37 2015
From: publicityifl@REDACTED (publicityifl@REDACTED)
Date: Sun, 10 May 2015 13:02:37 -0700 (PDT)
Subject: [erlang-questions] First Call for Papers for IFL 2015
Message-ID: <554fb95d.2853b40a.5b6d.113b@mx.google.com>

Hello,

Please, find below the first call for papers for IFL 2015.
Please forward these to anyone you think may be interested.
Apologies for any duplicates you may receive.

best regards,
Jurriaan Hage
Publicity Chair of IFL

---

IFL 2015 - Call for papers

27th SYMPOSIUM ON IMPLEMENTATION AND APPLICATION OF FUNCTIONAL LANGUAGES - IFL 2015

University of Koblenz-Landau, Koblenz, Germany

In cooperation with ACM SIGPLAN

September 14-16, 2015

http://ifl2015.wikidot.com/

Scope

The goal of the IFL symposia is to bring together researchers actively engaged in the implementation and application of functional and function-based programming languages. IFL 2015 will be a venue for researchers to present and discuss new ideas and concepts, work in progress, and publication-ripe results related to the implementation and application of functional languages and function-based programming.

Peer-review

Following the IFL tradition, IFL 2015 will use a post-symposium review process to produce the formal proceedings. All participants of IFL2015 are invited to submit either a draft paper or an extended abstract describing work to be presented at the symposium. At no time may work submitted to IFL be simultaneously submitted to other venues; submissions must adhere to ACM SIGPLAN's republication policy:

http://www.sigplan.org/Resources/Policies/Republication

The submissions will be screened by the program committee chair to make sure they are within the scope of IFL, and will appear in the draft proceedings distributed at the symposium. Submissions appearing in the draft proceedings are not peer-reviewed publications. Hence, publications that appear only in the draft proceedings do not count as publication for the ACM SIGPLAN republication policy. After the symposium, authors will be given the opportunity to incorporate the feedback from discussions at the symposium and will be invited to submit a revised full article for the formal review process. From the revised submissions, the program committee will select papers for the formal proceedings considering their correctness, novelty, originality, relevance, significance, and clarity.

Important dates

August 10: Submission deadline draft papers
August 12: Notification of acceptance for presentation
August 14: Early registration deadline
August 21: Late registration deadline
September 7: Submission deadline for pre-symposium proceedings
September 14-16: IFL Symposium
December 1: Submission deadline for post-symposium proceedings
January 15, 2016: Notification of acceptance for post-symposium proceedings
March 1, 2016: Camera-ready version for post-symposium proceedings

Submission details

Prospective authors are encouraged to submit papers or extended abstracts to be published in the draft proceedings and to present them at the symposium. All contributions must be written in English. Papers must adhere to the standard ACM two columns conference format. For the pre-symposium proceedings we adopt a 'weak' page limit of 12 pages. For the post-symposium proceedings the page limit of 12 pages is firm. A suitable document template for LaTeX can be found at:

http://www.acm.org/sigs/sigplan/authorInformation.htm

Authors submit through EasyChair:

https://easychair.org/conferences/?conf=ifl2015

Topics

IFL welcomes submissions describing practical and theoretical work as well as submissions describing applications and tools in the context of functional programming. If you are not sure whether your work is appropriate for IFL 2015, please contact the PC chair at rlaemmel@REDACTED Topics of interest include, but are not limited to:

- language concepts
- type systems, type checking, type inferencing
- compilation techniques
- staged compilation
- run-time function specialization
- run-time code generation
- partial evaluation
- (abstract) interpretation
- metaprogramming
- generic programming
- automatic program generation
- array processing
- concurrent/parallel programming
- concurrent/parallel program execution
- embedded systems
- web applications
- (embedded) domain specific languages
- security
- novel memory management techniques
- run-time profiling performance measurements
- debugging and tracing
- virtual/abstract machine architectures
- validation, verification of functional programs
- tools and programming techniques
- (industrial) applications

Peter Landin Prize

The Peter Landin Prize is awarded to the best paper presented at the symposium every year. The honored article is selected by the program committee based on the submissions received for the formal review process. The prize carries a cash award equivalent to 150 Euros.

Programme committee

Chair: Ralf L??mmel, University of Koblenz-Landau, Germany

- Malgorzata Biernacka, University of Wroclaw, Poland
- Laura M. Castro, University of A Coru??a, Spain
- Martin Erwig, Oregon State University, USA
- Dan Ghica, University of Birmingham, UK
- Andrew Gill, University of Kansas, USA
- Stephan Herhut, Google, USA
- Zhenjiang Hu, National Institute of Informatics (NII), Japan
- Mauro Jaskelioff, CIFASIS/Universidad Nacional de Rosario, Argentina
- Fr??d??ric Jouault, ESEO, France
- Oleg Kiselyov, Tohoku University, Japan
- Lindsey Kuper, Indiana University, USA
- Rita Loogen, Philipps-Universit??t Marburg, Germany
- Akimasa Morihata, University of Tokyo, Japan
- Atsushi Ohori, Tohoku University, Japan
- Bruno C. D. S. Oliveira, The University of Hong Kong, Hong Kong
- Frank Piessens, Katholieke Universiteit Leuven, Belgium
- Norman Ramsey, Tufts University, USA
- Matthew Roberts, Macquarie University, Australia
- Manfred Schmidt-Schauss, Goethe-University Frankfurt, Germany
- Simon Thompson, University of Kent, UK
- Stephanie Weirich, University of Pennsylvania, USA
- Steve Zdancewic, University of Pennsylvania , USA

Venue

The 27th IFL will be held in association with the Faculty of Computer Science, University of Koblenz-Landau, Campus Koblenz. Koblenz is well connected by train to several international airports. For instance, Koblenz can be reached from Frankfurt by high-speed train ICE within an hour. The modern Koblenz campus is close to the city center and can be reached by foot, bus, or cab. See the website for more information on the venue.


From petergi@REDACTED  Mon May 11 09:08:25 2015
From: petergi@REDACTED (Peter J Etheridge)
Date: Mon, 11 May 2015 17:08:25 +1000
Subject: [erlang-questions]
	<3f13067c82b13e494226b8e5d7ebba994dfdda20@webmail.iinet.net.au>
Message-ID: <73b7b8234619a7ba073e9cb7ec900b0a5638d241@webmail.iinet.net.au>

dear hokan & kostis,thank you for sharing your suggestions and
guidance in your responses.
best regards,peter


-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From erlang@REDACTED  Mon May 11 14:32:11 2015
From: erlang@REDACTED (Joe Armstrong)
Date: Mon, 11 May 2015 14:32:11 +0200
Subject: [erlang-questions] Retrieving "semi-constant" data from a
 function versus Mnesia
In-Reply-To: 
References: 
 
 
Message-ID: 

On Sun, May 10, 2015 at 10:32 PM, Benoit Chesneau  wrote:
>
>
> On Sun, May 10, 2015 at 10:23 PM Joe Armstrong  wrote:
>>
>> How large is the total data?
>>
>> If it's small you could define this in a module and not use a database
>> or process at all.
>
>
> How small should it be? What if the compiled bean is about 888 kilobytes?

It depends :-)

There are many factors involved:

   a) How much memory do you have
   b) How often do you want to change the configuration data
   c) How quick do you the change to be

Assume

  a) is a smallish number of GBytes (normal these days)
  b) is "relatively rarely" (I don't know what this means - (aside - this is why
      I always ask people to provide numbers in their questions))
     Say once a day
  c) is a few seconds

Then it should be perfectly doable. Making a module containing all the
config data
and recompiling when necessary is certainly the fastest way to access the data -
this has very fast reads but very slow writes - you could think of a
module as a database
optimized for reads but not writes :-)

/Joe



>
> - benoit


From rdm@REDACTED  Mon May 11 15:16:52 2015
From: rdm@REDACTED (Rich Morin)
Date: Mon, 11 May 2015 06:16:52 -0700
Subject: [erlang-questions] information on actor-based algorithms?
In-Reply-To: <4DC5D9AD-A03E-4FAD-884B-6983294B629A@cfcl.com>
References: <4DC5D9AD-A03E-4FAD-884B-6983294B629A@cfcl.com>
Message-ID: <6525E7F1-F44E-47CA-9386-1CEACE5E409A@cfcl.com>

I've done a bit of browsing through Wikipedia, speculation, etc.
Here is my current set of notes:

  http://wiki.cfcl.com/Projects/Elixir/PG/Algorithms

Comments and suggestions welcome!

-r

 -- 
http://www.cfcl.com/rdm           Rich Morin           rdm@REDACTED
http://www.cfcl.com/rdm/resume    San Bruno, CA, USA   +1 650-873-7841

Software system design, development, and documentation




From sean@REDACTED  Mon May 11 18:00:02 2015
From: sean@REDACTED (Functional Jobs)
Date: Mon, 11 May 2015 12:00:02 -0400
Subject: [erlang-questions] New Functional Programming Job Opportunities
Message-ID: <5550d203c6d7b@functionaljobs.com>

Here are some functional programming job opportunities that were posted
recently:

Sr. Software Engineer at McGraw-Hill Education
http://functionaljobs.com/jobs/8822-sr-software-engineer-at-mcgraw-hill-education

Cheers,
Sean Murphy
FunctionalJobs.com



From jay@REDACTED  Mon May 11 20:02:11 2015
From: jay@REDACTED (Jay Nelson)
Date: Mon, 11 May 2015 11:02:11 -0700
Subject: [erlang-questions] Retrieving "semi-constant" data from a
	function versus Mnesia
Message-ID: 

(Edited previous posts for relevance, including only Joe?s comments
interspersed with my responses.)
> If it's small you could define this in a module and not use a database
> or process at all.

> -module(global_data).
> -export(...)

> data_one() ->
>    ....

> Then dynamically change and recompile the module. Any call
> global_data:data_one() will pick up the value from the "latest version"

> /Joe
This is exactly the approach I was taking, but I assumed that there would be
various implementations so I used a behaviour to identify the specific config:

https://github.com/duomark/elysium/blob/master/src/elysium_config.erl#L95-L108 
https://github.com/duomark/elysium/blob/master/src/elysium_default_config.erl#L19-L39 

The data set is quite small as it is normal app configuration data.
I implemented in the most straightforward obvious way:

https://github.com/duomark/elysium/blob/master/src/elysium_config.erl#L268-L273 

It turns out the Config_Module:fn(?) is the killer of performance. This alternative
approach is much more performant if you are ever faced with a similar situation
(example from some code I was browsing recently):

https://github.com/knutin/elli/blob/master/src/elli_tcp.erl#L51-L54 

> Making a module containing all the config data
> and recompiling when necessary is certainly the fastest way
> to access the data -this has very fast reads but very slow writes - you could think of a
> module as a database optimized for reads but not writes :-)
So I thought as well, but you *must* call it with a compiled module name or
eliminate the performance benefit. I mostly wanted to guarantee distributed
lock-free code access for many concurrent readers to avoid contention, but
it proved the slowest of the 3 approaches when using a behaviour.

jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roberto.ostinelli@REDACTED  Mon May 11 21:13:52 2015
From: roberto.ostinelli@REDACTED (Roberto Ostinelli)
Date: Mon, 11 May 2015 21:13:52 +0200
Subject: [erlang-questions] HEART
Message-ID: 

Dear list,
I'm a little confused and would like to understand whether the HEART command should be used or not. 

In non-erlang systems, I would have standard watchdogs that launch an application on OS boot, and then monitor it and relaunch it if necessary. 

What is the best practice in the erlang world? A script to start an erlang release at boot time, and use the HEART command? Others?

The warning in the docs makes me hesitate.

Any input welcome. 

Thanks!
r.

From erlang-questions-mailing-list@REDACTED  Mon May 11 19:37:15 2015
From: erlang-questions-mailing-list@REDACTED (Luis Gerhorst)
Date: Mon, 11 May 2015 19:37:15 +0200
Subject: [erlang-questions] Apply Erlang configuration file (for multiple
	applications) to Common Test with rebar
Message-ID: 

Hi,
I've already posted this question on Stack Overflow but nobody has answered it. Maybe anyone here know's how I can do that:

http://stackoverflow.com/questions/30161772/apply-erlang-configuration-file-for-multiple-applications-to-common-test-with 

If you know an answer / need more information but don't have a Stack Exchange account you can of course also answer on the mailing list.

Thank you very much and best regards,
Luis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From eric.pailleau@REDACTED  Mon May 11 22:11:38 2015
From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=)
Date: Mon, 11 May 2015 22:11:38 +0200
Subject: [erlang-questions] HEART
In-Reply-To: 
Message-ID: <327d3157-4eda-4111-a9d4-5df686606ebe@email.android.com>

Hi Roberto,

Good question. Heart was probably initially used for embedded systems. 
Monitoring your node is not that much easy,
Because no pid file is created out of the box. It ends by using a wrapper that create a pidfile.
Heart is also needful for hot upgrades when reboot of emulator is necessary.
Mho,  you should set the HEART_command to your starter script in order to create another pidfile on reboot. See heart module documentation. 
Regards

Le?11 mai 2015 21:13, Roberto Ostinelli  a ?crit?:
>
> Dear list, 
> I'm a little confused and would like to understand whether the HEART command should be used or not. 
>
> In non-erlang systems, I would have standard watchdogs that launch an application on OS boot, and then monitor it and relaunch it if necessary. 
>
> What is the best practice in the erlang world? A script to start an erlang release at boot time, and use the HEART command? Others? 
>
> The warning in the docs makes me hesitate. 
>
> Any input welcome. 
>
> Thanks! 
> r. 
> _______________________________________________ 
> erlang-questions mailing list 
> erlang-questions@REDACTED 
> http://erlang.org/mailman/listinfo/erlang-questions 

From roberto.ostinelli@REDACTED  Mon May 11 22:23:04 2015
From: roberto.ostinelli@REDACTED (Roberto Ostinelli)
Date: Mon, 11 May 2015 22:23:04 +0200
Subject: [erlang-questions] HEART
In-Reply-To: <327d3157-4eda-4111-a9d4-5df686606ebe@email.android.com>
References: <327d3157-4eda-4111-a9d4-5df686606ebe@email.android.com>
Message-ID: <581CD4C7-6353-4449-86E9-5EEE522A233E@widetag.com>

Hi Eric,
Thank you. Do you have a recommended starter script?



> On 11/mag/2015, at 22:11, ?ric Pailleau  wrote:
> 
> Hi Roberto,
> 
> Good question. Heart was probably initially used for embedded systems. 
> Monitoring your node is not that much easy,
> Because no pid file is created out of the box. It ends by using a wrapper that create a pidfile.
> Heart is also needful for hot upgrades when reboot of emulator is necessary.
> Mho,  you should set the HEART_command to your starter script in order to create another pidfile on reboot. See heart module documentation. 
> Regards
> 
> Le 11 mai 2015 21:13, Roberto Ostinelli  a ?crit :
>> 
>> Dear list, 
>> I'm a little confused and would like to understand whether the HEART command should be used or not. 
>> 
>> In non-erlang systems, I would have standard watchdogs that launch an application on OS boot, and then monitor it and relaunch it if necessary. 
>> 
>> What is the best practice in the erlang world? A script to start an erlang release at boot time, and use the HEART command? Others? 
>> 
>> The warning in the docs makes me hesitate. 
>> 
>> Any input welcome. 
>> 
>> Thanks! 
>> r. 
>> _______________________________________________ 
>> erlang-questions mailing list 
>> erlang-questions@REDACTED 
>> http://erlang.org/mailman/listinfo/erlang-questions 


From eric.pailleau@REDACTED  Mon May 11 22:32:48 2015
From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=)
Date: Mon, 11 May 2015 22:32:48 +0200
Subject: [erlang-questions] HEART
In-Reply-To: <581CD4C7-6353-4449-86E9-5EEE522A233E@widetag.com>
Message-ID: <4c245964-edc8-4163-b19f-ba05698f9ee3@email.android.com>

Hi, 
No, in fact, I m working on this.
To be short, the wrapper might be posix compliant and trap signals, create the pidfile set the heart command and launch the release.
I will maybe share my work once ready, but so many thing to do and do little time... ;-) 

Le?11 mai 2015 22:23, Roberto Ostinelli  a ?crit?:
>
> Hi Eric, 
> Thank you. Do you have a recommended starter script? 
>
>
>
> > On 11/mag/2015, at 22:11, ?ric Pailleau  wrote: 
> > 
> > Hi Roberto, 
> > 
> > Good question. Heart was probably initially used for embedded systems. 
> > Monitoring your node is not that much easy, 
> > Because no pid file is created out of the box. It ends by using a wrapper that create a pidfile. 
> > Heart is also needful for hot upgrades when reboot of emulator is necessary. 
> > Mho,? you should set the HEART_command to your starter script in order to create another pidfile on reboot. See heart module documentation. 
> > Regards 
> > 
> > Le 11 mai 2015 21:13, Roberto Ostinelli  a ?crit : 
> >> 
> >> Dear list, 
> >> I'm a little confused and would like to understand whether the HEART command should be used or not. 
> >> 
> >> In non-erlang systems, I would have standard watchdogs that launch an application on OS boot, and then monitor it and relaunch it if necessary. 
> >> 
> >> What is the best practice in the erlang world? A script to start an erlang release at boot time, and use the HEART command? Others? 
> >> 
> >> The warning in the docs makes me hesitate. 
> >> 
> >> Any input welcome. 
> >> 
> >> Thanks! 
> >> r. 
> >> _______________________________________________ 
> >> erlang-questions mailing list 
> >> erlang-questions@REDACTED 
> >> http://erlang.org/mailman/listinfo/erlang-questions 

From lucafinzicontini@REDACTED  Tue May 12 10:02:47 2015
From: lucafinzicontini@REDACTED (Luca Finzi Contini)
Date: Tue, 12 May 2015 10:02:47 +0200
Subject: [erlang-questions] Which applications does Observer show ?
Message-ID: 

Hi all,
in my first experiments with Observer tool, I noticed that it shows lots of
very useful information, but apparently not all applications are shown
explicitly.
For example, in our Erlang-based software there's a couple of applications
that are related to each other:
- erlpass
- bcrypt, used by erlpass

Apparently though, only bcrypt is shown in Observer
My understanding is that perhaps erlpass is started in a different way than
bcrypt, so that it is maybe 'registered' in a different way.
Any pointers?
Thank in advance
Luca
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roger@REDACTED  Tue May 12 11:22:08 2015
From: roger@REDACTED (Roger Lipscombe)
Date: Tue, 12 May 2015 10:22:08 +0100
Subject: [erlang-questions] Which applications does Observer show ?
In-Reply-To: 
References: 
Message-ID: 

On 12 May 2015 at 09:02, Luca Finzi Contini  wrote:
> Apparently though, only bcrypt is shown in Observer
> My understanding is that perhaps erlpass is started in a different way than
> bcrypt, so that it is maybe 'registered' in a different way.
> Any pointers?

I suspect that it's down to the difference between "application" and
"library application"; see
http://www.erlang.org/doc/design_principles/des_princ.html#id56438 and
http://www.erlang.org/doc/design_principles/applications.html#id73722.


From sdl.web@REDACTED  Tue May 12 11:29:43 2015
From: sdl.web@REDACTED (Leo Liu)
Date: Tue, 12 May 2015 17:29:43 +0800
Subject: [erlang-questions] Which applications does Observer show ?
References: 
 
Message-ID: 

On 2015-05-12 17:22 +0800, Roger Lipscombe wrote:
> I suspect that it's down to the difference between "application" and
> "library application"; see
> http://www.erlang.org/doc/design_principles/des_princ.html#id56438 and
> http://www.erlang.org/doc/design_principles/applications.html#id73722.

Yes, observer shows only applications with supervision tree.

Leo



From jesper.louis.andersen@REDACTED  Tue May 12 17:21:08 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Tue, 12 May 2015 17:21:08 +0200
Subject: [erlang-questions] Apply Erlang configuration file (for
 multiple applications) to Common Test with rebar
In-Reply-To: 
References: 
Message-ID: 

I think my solutions would be something along the lines of:

setup_app(App, Config) ->
  application:load(App),
  [application:set_env(App, Par, Val) || C <- ?config(App, Config), {Par,
Val} <- C],
  application:start(App).

This makes the configuration of the application part of the code, which
means it is much easier to parameterize the configuration later on when
needed. A static configuration file means you have to re-run CT for each
test you want to make with a new configuration, but this allows you to
change the configuration as you go along. It also cuts down on the command
line configuration considerable, since you only need a single configuration
now.

On Mon, May 11, 2015 at 7:37 PM, Luis Gerhorst <
erlang-questions-mailing-list@REDACTED> wrote:

> Hi,
> I've already posted this question on Stack Overflow but nobody has
> answered it. Maybe anyone here know's how I can do that:
>
>
> http://stackoverflow.com/questions/30161772/apply-erlang-configuration-file-for-multiple-applications-to-common-test-with
>
> If you know an answer / need more information but don't have a Stack
> Exchange account you can of course also answer on the mailing list.
>
> Thank you very much and best regards,
> Luis
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From t@REDACTED  Tue May 12 17:24:41 2015
From: t@REDACTED (Tristan Sloughter)
Date: Tue, 12 May 2015 10:24:41 -0500
Subject: [erlang-questions] Apply Erlang configuration file (for
 multiple applications) to Common Test with rebar
In-Reply-To: 
References: 
Message-ID: <1431444281.2691880.266808497.0EC9001D@webmail.messagingengine.com>

> *Update:* ct_run -dir apps/app1/test -pa deps/*/ebin -pa apps/*/ebin
> -erl_args -config rpm
 also works as expected. I guess the problem is that rebar changes the
 cwd when running the tests for each application and therefore the
 -config rpm option does not point to a existing file. I was anyway not
 able to find a workaround.

I thought I'd point out that rebar3 (http://rebar3.org) does not change
the cwd, so you may have better luck trying rebar3 ct.

--
Tristan Sloughter t@REDACTED



On Mon, May 11, 2015, at 12:37 PM, Luis Gerhorst wrote:
> Hi, I've already posted this question on Stack Overflow but nobody has
> answered it. Maybe anyone here know's how I can do that:
>
> http://stackoverflow.com/questions/30161772/apply-erlang-configuration-file-for-multiple-applications-to-common-test-with
>
> If you know an answer / need more information but don't have a Stack
> Exchange account you can of course also answer on the mailing list.
>
> Thank you very much and best regards, Luis
> _________________________________________________
> erlang-questions mailing list erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roberto@REDACTED  Tue May 12 17:31:05 2015
From: roberto@REDACTED (Roberto Ostinelli)
Date: Tue, 12 May 2015 17:31:05 +0200
Subject: [erlang-questions] HEART
In-Reply-To: <4c245964-edc8-4163-b19f-ba05698f9ee3@email.android.com>
References: <581CD4C7-6353-4449-86E9-5EEE522A233E@widetag.com>
 <4c245964-edc8-4163-b19f-ba05698f9ee3@email.android.com>
Message-ID: 

I still can't seem to find anything decent on a proper way to start an
Erlang Release on system boot, and a watchdog to restart it if the VM
crashes.
I only find a variety of hacks, which include running releases *not* in
detached mode.

Any other recommendations?

On Mon, May 11, 2015 at 10:32 PM, ?ric Pailleau 
wrote:

> Hi,
> No, in fact, I m working on this.
> To be short, the wrapper might be posix compliant and trap signals, create
> the pidfile set the heart command and launch the release.
> I will maybe share my work once ready, but so many thing to do and do
> little time... ;-)
>
> Le 11 mai 2015 22:23, Roberto Ostinelli  a
> ?crit :
> >
> > Hi Eric,
> > Thank you. Do you have a recommended starter script?
> >
> >
> >
> > > On 11/mag/2015, at 22:11, ?ric Pailleau 
> wrote:
> > >
> > > Hi Roberto,
> > >
> > > Good question. Heart was probably initially used for embedded systems.
> > > Monitoring your node is not that much easy,
> > > Because no pid file is created out of the box. It ends by using a
> wrapper that create a pidfile.
> > > Heart is also needful for hot upgrades when reboot of emulator is
> necessary.
> > > Mho,  you should set the HEART_command to your starter script in order
> to create another pidfile on reboot. See heart module documentation.
> > > Regards
> > >
> > > Le 11 mai 2015 21:13, Roberto Ostinelli 
> a ?crit :
> > >>
> > >> Dear list,
> > >> I'm a little confused and would like to understand whether the HEART
> command should be used or not.
> > >>
> > >> In non-erlang systems, I would have standard watchdogs that launch an
> application on OS boot, and then monitor it and relaunch it if necessary.
> > >>
> > >> What is the best practice in the erlang world? A script to start an
> erlang release at boot time, and use the HEART command? Others?
> > >>
> > >> The warning in the docs makes me hesitate.
> > >>
> > >> Any input welcome.
> > >>
> > >> Thanks!
> > >> r.
> > >> _______________________________________________
> > >> erlang-questions mailing list
> > >> erlang-questions@REDACTED
> > >> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jesper.louis.andersen@REDACTED  Tue May 12 17:52:08 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Tue, 12 May 2015 17:52:08 +0200
Subject: [erlang-questions] HEART
In-Reply-To: 
References: 
Message-ID: 

On Mon, May 11, 2015 at 9:13 PM, Roberto Ostinelli <
roberto.ostinelli@REDACTED> wrote:

> In non-erlang systems, I would have standard watchdogs that launch an
> application on OS boot, and then monitor it and relaunch it if necessary.


The heart system in Erlang is a simple watchdog, mostly used if you nothing
else that will restart your application. In an SysV init system, there is
no automatic watching and restart. In RcNG in FreeBSD, there is no restart.
In OpenBSDs rc, there is no automatic restart.

But as you say, many modern init systems also provides watching of the
applications it starts, and in that case I wouldn't run with heart enabled.
Better to let the system above handle it, and plug into its
monitoring/reporting and so on.

Heart is very useful if you start nodes outside the control of the system
watchdog though. In that case, they won't be restarted, and you can run
heart on those.


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roberto@REDACTED  Tue May 12 18:07:11 2015
From: roberto@REDACTED (Roberto Ostinelli)
Date: Tue, 12 May 2015 18:07:11 +0200
Subject: [erlang-questions] HEART
In-Reply-To: 
References: 
 
Message-ID: 

Fair enough, however at this point I cannot even a single one of these
systems to simply START *and* STOP an Erlang release.
I can easily start it (on ubuntu, `sudo service myapp start`) but the STOP
command will always fail. That's because the init script cannot get a grasp
of the BEAM process' pid (for some reason), hence it cannot stop it.

Any ideas on how to do that?


On Tue, May 12, 2015 at 5:52 PM, Jesper Louis Andersen <
jesper.louis.andersen@REDACTED> wrote:

>
> On Mon, May 11, 2015 at 9:13 PM, Roberto Ostinelli <
> roberto.ostinelli@REDACTED> wrote:
>
>> In non-erlang systems, I would have standard watchdogs that launch an
>> application on OS boot, and then monitor it and relaunch it if necessary.
>
>
> The heart system in Erlang is a simple watchdog, mostly used if you
> nothing else that will restart your application. In an SysV init system,
> there is no automatic watching and restart. In RcNG in FreeBSD, there is no
> restart. In OpenBSDs rc, there is no automatic restart.
>
> But as you say, many modern init systems also provides watching of the
> applications it starts, and in that case I wouldn't run with heart enabled.
> Better to let the system above handle it, and plug into its
> monitoring/reporting and so on.
>
> Heart is very useful if you start nodes outside the control of the system
> watchdog though. In that case, they won't be restarted, and you can run
> heart on those.
>
>
> --
> J.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From dmkolesnikov@REDACTED  Tue May 12 18:48:31 2015
From: dmkolesnikov@REDACTED (Dmitry Kolesnikov)
Date: Tue, 12 May 2015 19:48:31 +0300
Subject: [erlang-questions] HEART
In-Reply-To: 
References: 
 
 
Message-ID: 

Hello Roberto,

The rebar makes a dirty work to generate "node release? and necessary bootstrap scripts.  Please check create-node command and concept of erlang releases: 
https://github.com/rebar/rebar/wiki/Rebar-commands
http://www.erlang.org/doc/man/reltool.html

It generates script looks something similar to this one. It also hooks the heart to watchdog the node.  
https://github.com/fogfish/hyperion/blob/master/rel/files/hyperion

Personally, I am following an idea to package the application to release and ship it to destination hosts.
Long time ago, I?ve made an empty ?erlang release? project as show-case to study and debug various scenarios. You might look on it here:

https://github.com/fogfish/hyperion  

Best Regards, 
Dmitry


> On 12 May 2015, at 19:07, Roberto Ostinelli  wrote:
> 
> Fair enough, however at this point I cannot even a single one of these systems to simply START *and* STOP an Erlang release.
> I can easily start it (on ubuntu, `sudo service myapp start`) but the STOP command will always fail. That's because the init script cannot get a grasp of the BEAM process' pid (for some reason), hence it cannot stop it.
> 
> Any ideas on how to do that?
> 
> 
> On Tue, May 12, 2015 at 5:52 PM, Jesper Louis Andersen > wrote:
> 
> On Mon, May 11, 2015 at 9:13 PM, Roberto Ostinelli > wrote:
> In non-erlang systems, I would have standard watchdogs that launch an application on OS boot, and then monitor it and relaunch it if necessary.
> 
> The heart system in Erlang is a simple watchdog, mostly used if you nothing else that will restart your application. In an SysV init system, there is no automatic watching and restart. In RcNG in FreeBSD, there is no restart. In OpenBSDs rc, there is no automatic restart.
> 
> But as you say, many modern init systems also provides watching of the applications it starts, and in that case I wouldn't run with heart enabled. Better to let the system above handle it, and plug into its monitoring/reporting and so on.
> 
> Heart is very useful if you start nodes outside the control of the system watchdog though. In that case, they won't be restarted, and you can run heart on those.
> 
> 
> -- 
> J.
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From t@REDACTED  Tue May 12 18:58:20 2015
From: t@REDACTED (Tristan Sloughter)
Date: Tue, 12 May 2015 11:58:20 -0500
Subject: [erlang-questions] HEART
In-Reply-To: 
References: 
 
 
 
Message-ID: <1431449900.2713951.266897849.0D1D4324@webmail.messagingengine.com>

I'd suggest relx (github.com/erlware/relx) or rebar3 which use relx
http://www.rebar3.org/v3.0/docs/releases

I've never had to do what you are trying to do but I know the
extended_start_script has the ability to return the pid of the running
node and `start` uses `run_erl` for running as a daemon and allows
attaching through a fifo pipe.

It would be great to improve the generated release scripts and whatever
else we can to make this easier.

--
Tristan Sloughter t@REDACTED



On Tue, May 12, 2015, at 11:48 AM, Dmitry Kolesnikov wrote:
> Hello Roberto,
>
> The rebar makes a dirty work to generate "node release? and necessary
> bootstrap scripts. Please check create-node command and concept of
> erlang releases: https://github.com/rebar/rebar/wiki/Rebar-commands
> http://www.erlang.org/doc/man/reltool.html
>
> It generates script looks something similar to this one. It also hooks
> the heart to watchdog the node.
> https://github.com/fogfish/hyperion/blob/master/rel/files/hyperion
>
> Personally, I am following an idea to package the application to
> release and ship it to destination hosts. Long time ago, I?ve made an
> empty ?erlang release? project as show-case to study and debug various
> scenarios. You might look on it here:
>
> https://github.com/fogfish/hyperion
>
> Best Regards, Dmitry
>
>
>> On 12 May 2015, at 19:07, Roberto Ostinelli
>>  wrote:
>>
>> Fair enough, however at this point I cannot even a single one of
>> these systems to simply START *and* STOP an Erlang release. I can
>> easily start it (on ubuntu, `sudo service myapp start`) but the STOP
>> command will always fail. That's because the init script cannot get a
>> grasp of the BEAM process' pid (for some reason), hence it cannot
>> stop it.
>>
>> Any ideas on how to do that?
>>
>>
>> On Tue, May 12, 2015 at 5:52 PM, Jesper Louis Andersen
>>  wrote:
>>>
>>>
>>> On Mon, May 11, 2015 at 9:13 PM, Roberto Ostinelli
>>>  wrote:
>>>
>>>> In non-erlang systems, I would have standard watchdogs that launch
>>>> an application on OS boot, and then monitor it and relaunch it if
>>>> necessary.
>>>
>>> The heart system in Erlang is a simple watchdog, mostly used if you
>>> nothing else that will restart your application. In an SysV init
>>> system, there is no automatic watching and restart. In RcNG in
>>> FreeBSD, there is no restart. In OpenBSDs rc, there is no automatic
>>> restart.
>>>
>>> But as you say, many modern init systems also provides watching of
>>> the applications it starts, and in that case I wouldn't run with
>>> heart enabled. Better to let the system above handle it, and plug
>>> into its monitoring/reporting and so on.
>>>
>>> Heart is very useful if you start nodes outside the control of the
>>> system watchdog though. In that case, they won't be restarted, and
>>> you can run heart on those.
>>>
>>>
>>>
>>> --
>>>
>>> J.
>>>
>>
>> _______________________________________________
>> erlang-questions mailing list erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>
> _________________________________________________
> erlang-questions mailing list erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roberto.ostinelli@REDACTED  Tue May 12 19:28:13 2015
From: roberto.ostinelli@REDACTED (Roberto Ostinelli)
Date: Tue, 12 May 2015 19:28:13 +0200
Subject: [erlang-questions] HEART
In-Reply-To: 
References: 
 
 
 
Message-ID: <08E64BBA-163C-4CFD-885C-0F2FC9BAD36B@widetag.com>

Hi Dmitry,
I am using rebar to generate the release. Thing is: how do you make the generated script start at boot time of, say, ubuntu? And more importantly, how do you stop it since the pid is unknown to the launching script?



> On 12/mag/2015, at 18:48, Dmitry Kolesnikov  wrote:
> 
> Hello Roberto,
> 
> The rebar makes a dirty work to generate "node release? and necessary bootstrap scripts.  Please check create-node command and concept of erlang releases: 
> https://github.com/rebar/rebar/wiki/Rebar-commands
> http://www.erlang.org/doc/man/reltool.html
> 
> It generates script looks something similar to this one. It also hooks the heart to watchdog the node.  
> https://github.com/fogfish/hyperion/blob/master/rel/files/hyperion
> 
> Personally, I am following an idea to package the application to release and ship it to destination hosts.
> Long time ago, I?ve made an empty ?erlang release? project as show-case to study and debug various scenarios. You might look on it here:
> 
> https://github.com/fogfish/hyperion  
> 
> Best Regards, 
> Dmitry
> 
> 
>> On 12 May 2015, at 19:07, Roberto Ostinelli  wrote:
>> 
>> Fair enough, however at this point I cannot even a single one of these systems to simply START *and* STOP an Erlang release.
>> I can easily start it (on ubuntu, `sudo service myapp start`) but the STOP command will always fail. That's because the init script cannot get a grasp of the BEAM process' pid (for some reason), hence it cannot stop it.
>> 
>> Any ideas on how to do that?
>> 
>> 
>>> On Tue, May 12, 2015 at 5:52 PM, Jesper Louis Andersen  wrote:
>>> 
>>>> On Mon, May 11, 2015 at 9:13 PM, Roberto Ostinelli  wrote:
>>>> In non-erlang systems, I would have standard watchdogs that launch an application on OS boot, and then monitor it and relaunch it if necessary.
>>> 
>>> The heart system in Erlang is a simple watchdog, mostly used if you nothing else that will restart your application. In an SysV init system, there is no automatic watching and restart. In RcNG in FreeBSD, there is no restart. In OpenBSDs rc, there is no automatic restart.
>>> 
>>> But as you say, many modern init systems also provides watching of the applications it starts, and in that case I wouldn't run with heart enabled. Better to let the system above handle it, and plug into its monitoring/reporting and so on.
>>> 
>>> Heart is very useful if you start nodes outside the control of the system watchdog though. In that case, they won't be restarted, and you can run heart on those.
>>> 
>>> 
>>> -- 
>>> J.
>> 
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roberto.ostinelli@REDACTED  Tue May 12 19:29:19 2015
From: roberto.ostinelli@REDACTED (Roberto Ostinelli)
Date: Tue, 12 May 2015 19:29:19 +0200
Subject: [erlang-questions] HEART
In-Reply-To: <1431449900.2713951.266897849.0D1D4324@webmail.messagingengine.com>
References: 
 
 
 
 <1431449900.2713951.266897849.0D1D4324@webmail.messagingengine.com>
Message-ID: <9B082D3C-0E45-4D21-B914-4AD4CECE03C3@widetag.com>

Indeed, I'm usong rebar (not relx, not rebar3). AFAIK this option is a relx option unfortunately.

Is is worthy to move to relx for this?



> On 12/mag/2015, at 18:58, Tristan Sloughter  wrote:
> 
> I'd suggest relx (github.com/erlware/relx) or rebar3 which use relx http://www.rebar3.org/v3.0/docs/releases
>  
> I've never had to do what you are trying to do but I know the extended_start_script has the ability to return the pid of the running node and `start` uses `run_erl` for running as a daemon and allows attaching through a fifo pipe.
>  
> It would be great to improve the generated release scripts and whatever else we can to make this easier.
>  
> --
> Tristan Sloughter
> t@REDACTED
>  
>  
>  
>> On Tue, May 12, 2015, at 11:48 AM, Dmitry Kolesnikov wrote:
>> Hello Roberto,
>>  
>> The rebar makes a dirty work to generate "node release? and necessary bootstrap scripts.  Please check create-node command and concept of erlang releases: 
>> https://github.com/rebar/rebar/wiki/Rebar-commands
>> http://www.erlang.org/doc/man/reltool.html
>>  
>> It generates script looks something similar to this one. It also hooks the heart to watchdog the node.  
>> https://github.com/fogfish/hyperion/blob/master/rel/files/hyperion
>>  
>> Personally, I am following an idea to package the application to release and ship it to destination hosts.
>> Long time ago, I?ve made an empty ?erlang release? project as show-case to study and debug various scenarios. You might look on it here:
>>  
>> https://github.com/fogfish/hyperion
>>  
>> Best Regards, 
>> Dmitry
>>  
>>  
>>> On 12 May 2015, at 19:07, Roberto Ostinelli  wrote:
>>>  
>>> Fair enough, however at this point I cannot even a single one of these systems to simply START *and* STOP an Erlang release.
>>> I can easily start it (on ubuntu, `sudo service myapp start`) but the STOP command will always fail. That's because the init script cannot get a grasp of the BEAM process' pid (for some reason), hence it cannot stop it.
>>>  
>>> Any ideas on how to do that?
>>>  
>>>  
>>> On Tue, May 12, 2015 at 5:52 PM, Jesper Louis Andersen  wrote:
>>>  
>>>  
>>> On Mon, May 11, 2015 at 9:13 PM, Roberto Ostinelli  wrote:
>>>  
>>> In non-erlang systems, I would have standard watchdogs that launch an application on OS boot, and then monitor it and relaunch it if necessary.
>>>  
>>> The heart system in Erlang is a simple watchdog, mostly used if you nothing else that will restart your application. In an SysV init system, there is no automatic watching and restart. In RcNG in FreeBSD, there is no restart. In OpenBSDs rc, there is no automatic restart.
>>>  
>>> But as you say, many modern init systems also provides watching of the applications it starts, and in that case I wouldn't run with heart enabled. Better to let the system above handle it, and plug into its monitoring/reporting and so on.
>>>  
>>> Heart is very useful if you start nodes outside the control of the system watchdog though. In that case, they won't be restarted, and you can run heart on those.
>>>  
>>>  
>>>  
>>> -- 
>>>  
>>> J.
>>>  
>>>  
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>> 
>>  
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jesper.louis.andersen@REDACTED  Tue May 12 19:31:18 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Tue, 12 May 2015 17:31:18 +0000
Subject: [erlang-questions] HEART
In-Reply-To: <9B082D3C-0E45-4D21-B914-4AD4CECE03C3@widetag.com>
References: 
 
 
 
 <1431449900.2713951.266897849.0D1D4324@webmail.messagingengine.com>
 <9B082D3C-0E45-4D21-B914-4AD4CECE03C3@widetag.com>
Message-ID: 

It is known through epmd if I remember correctly. So that way you should be
able to stop it.

On Tue, May 12, 2015, 19:29 Roberto Ostinelli 
wrote:

> Indeed, I'm usong rebar (not relx, not rebar3). AFAIK this option is a
> relx option unfortunately.
>
> Is is worthy to move to relx for this?
>
>
>
> On 12/mag/2015, at 18:58, Tristan Sloughter  wrote:
>
> I'd suggest relx (github.com/erlware/relx) or rebar3 which use relx
> http://www.rebar3.org/v3.0/docs/releases
>
> I've never had to do what you are trying to do but I know the
> extended_start_script has the ability to return the pid of the running node
> and `start` uses `run_erl` for running as a daemon and allows attaching
> through a fifo pipe.
>
> It would be great to improve the generated release scripts and whatever
> else we can to make this easier.
>
> --
> Tristan Sloughter
> t@REDACTED
>
>
>
> On Tue, May 12, 2015, at 11:48 AM, Dmitry Kolesnikov wrote:
>
> Hello Roberto,
>
> The rebar makes a dirty work to generate "node release? and necessary
> bootstrap scripts.  Please check create-node command and concept of erlang
> releases:
> https://github.com/rebar/rebar/wiki/Rebar-commands
> http://www.erlang.org/doc/man/reltool.html
>
> It generates script looks something similar to this one. It also hooks the
> heart to watchdog the node.
> https://github.com/fogfish/hyperion/blob/master/rel/files/hyperion
>
> Personally, I am following an idea to package the application to release
> and ship it to destination hosts.
> Long time ago, I?ve made an empty ?erlang release? project as show-case to
> study and debug various scenarios. You might look on it here:
>
> https://github.com/fogfish/hyperion
>
> Best Regards,
> Dmitry
>
>
>
> On 12 May 2015, at 19:07, Roberto Ostinelli  wrote:
>
> Fair enough, however at this point I cannot even a single one of these
> systems to simply START *and* STOP an Erlang release.
> I can easily start it (on ubuntu, `sudo service myapp start`) but the STOP
> command will always fail. That's because the init script cannot get a grasp
> of the BEAM process' pid (for some reason), hence it cannot stop it.
>
> Any ideas on how to do that?
>
>
> On Tue, May 12, 2015 at 5:52 PM, Jesper Louis Andersen <
> jesper.louis.andersen@REDACTED> wrote:
>
>
>
> On Mon, May 11, 2015 at 9:13 PM, Roberto Ostinelli <
> roberto.ostinelli@REDACTED> wrote:
>
>
> In non-erlang systems, I would have standard watchdogs that launch an
> application on OS boot, and then monitor it and relaunch it if necessary.
>
>
> The heart system in Erlang is a simple watchdog, mostly used if you
> nothing else that will restart your application. In an SysV init system,
> there is no automatic watching and restart. In RcNG in FreeBSD, there is no
> restart. In OpenBSDs rc, there is no automatic restart.
>
> But as you say, many modern init systems also provides watching of the
> applications it starts, and in that case I wouldn't run with heart enabled.
> Better to let the system above handle it, and plug into its
> monitoring/reporting and so on.
>
> Heart is very useful if you start nodes outside the control of the system
> watchdog though. In that case, they won't be restarted, and you can run
> heart on those.
>
>
>
> --
>
> J.
>
>
>
>  _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
> *_______________________________________________*
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From t@REDACTED  Tue May 12 19:34:33 2015
From: t@REDACTED (Tristan Sloughter)
Date: Tue, 12 May 2015 12:34:33 -0500
Subject: [erlang-questions] HEART
In-Reply-To: <9B082D3C-0E45-4D21-B914-4AD4CECE03C3@widetag.com>
References: 
 
 
 
 <1431449900.2713951.266897849.0D1D4324@webmail.messagingengine.com>
 <9B082D3C-0E45-4D21-B914-4AD4CECE03C3@widetag.com>
Message-ID: <1431452073.2721867.266937257.35C09C47@webmail.messagingengine.com>

The start script generated using rebar should also have a getpid
supported argument and a start that works the same. The scripts between
relx and rebar are nearly identical.

But I'd still suggest moving to relx over reltool :)

--
Tristan Sloughter t@REDACTED



On Tue, May 12, 2015, at 12:29 PM, Roberto Ostinelli wrote:
> Indeed, I'm usong rebar (not relx, not rebar3). AFAIK this option is a
> relx option unfortunately.
>
> Is is worthy to move to relx for this?
>
>
>
> On 12/mag/2015, at 18:58, Tristan Sloughter  wrote:
>
>> I'd suggest relx (github.com/erlware/relx) or rebar3 which use relx
>> http://www.rebar3.org/v3.0/docs/releases
>>
>> I've never had to do what you are trying to do but I know the
>> extended_start_script has the ability to return the pid of the
>> running node and `start` uses `run_erl` for running as a daemon and
>> allows attaching through a fifo pipe.
>>
>> It would be great to improve the generated release scripts and
>> whatever else we can to make this easier.
>>
>> --
>> Tristan Sloughter t@REDACTED
>>
>>
>>
>> On Tue, May 12, 2015, at 11:48 AM, Dmitry Kolesnikov wrote:
>>> Hello Roberto,
>>>
>>> The rebar makes a dirty work to generate "node release? and
>>> necessary bootstrap scripts. Please check create-node command and
>>> concept of erlang releases:
>>> https://github.com/rebar/rebar/wiki/Rebar-commands
>>> http://www.erlang.org/doc/man/reltool.html
>>>
>>> It generates script looks something similar to this one. It also
>>> hooks the heart to watchdog the node.
>>> https://github.com/fogfish/hyperion/blob/master/rel/files/hyperion
>>>
>>> Personally, I am following an idea to package the application to
>>> release and ship it to destination hosts. Long time ago, I?ve made
>>> an empty ?erlang release? project as show-case to study and debug
>>> various scenarios. You might look on it here:
>>>
>>> https://github.com/fogfish/hyperion
>>>
>>> Best Regards, Dmitry
>>>
>>>
>>>> On 12 May 2015, at 19:07, Roberto Ostinelli 
>>>> wrote:
>>>>
>>>> Fair enough, however at this point I cannot even a single one of
>>>> these systems to simply START *and* STOP an Erlang release. I can
>>>> easily start it (on ubuntu, `sudo service myapp start`) but the
>>>> STOP command will always fail. That's because the init script
>>>> cannot get a grasp of the BEAM process' pid (for some reason),
>>>> hence it cannot stop it.
>>>>
>>>> Any ideas on how to do that?
>>>>
>>>>
>>>> On Tue, May 12, 2015 at 5:52 PM, Jesper Louis Andersen
>>>>  wrote:
>>>>>
>>>>>
>>>>> On Mon, May 11, 2015 at 9:13 PM, Roberto Ostinelli
>>>>>  wrote:
>>>>>
>>>>>> In non-erlang systems, I would have standard watchdogs that
>>>>>> launch an application on OS boot, and then monitor it and
>>>>>> relaunch it if necessary.
>>>>>
>>>>> The heart system in Erlang is a simple watchdog, mostly used if
>>>>> you nothing else that will restart your application. In an SysV
>>>>> init system, there is no automatic watching and restart. In RcNG
>>>>> in FreeBSD, there is no restart. In OpenBSDs rc, there is no
>>>>> automatic restart.
>>>>>
>>>>> But as you say, many modern init systems also provides watching of
>>>>> the applications it starts, and in that case I wouldn't run with
>>>>> heart enabled. Better to let the system above handle it, and plug
>>>>> into its monitoring/reporting and so on.
>>>>>
>>>>> Heart is very useful if you start nodes outside the control of the
>>>>> system watchdog though. In that case, they won't be restarted, and
>>>>> you can run heart on those.
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> J.
>>>>>
>>>>
>>>> _______________________________________________
>>>> erlang-questions mailing list erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>> _________________________________________________
>>> erlang-questions mailing list erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From dmkolesnikov@REDACTED  Tue May 12 19:35:21 2015
From: dmkolesnikov@REDACTED (Dmitry Kolesnikov)
Date: Tue, 12 May 2015 20:35:21 +0300
Subject: [erlang-questions] HEART
In-Reply-To: <08E64BBA-163C-4CFD-885C-0F2FC9BAD36B@widetag.com>
References: 
 
 
 
 <08E64BBA-163C-4CFD-885C-0F2FC9BAD36B@widetag.com>
Message-ID: 

Hello,

This is business as usual. If you generate myapp release and deploy it to /usr/local then you can create very simple script to init.d

See it here
https://github.com/fogfish/hyperion/blob/master/rel/deploy.sh#L19

e.g.

#!/bin/bash
export HOME=/root
/usr/local/myapp/bin/myapp $1

- Dmitry


> On 12 May 2015, at 20:28, Roberto Ostinelli  wrote:
> 
> Hi Dmitry,
> I am using rebar to generate the release. Thing is: how do you make the generated script start at boot time of, say, ubuntu? And more importantly, how do you stop it since the pid is unknown to the launching script?
> 
> 
> 
> On 12/mag/2015, at 18:48, Dmitry Kolesnikov > wrote:
> 
>> Hello Roberto,
>> 
>> The rebar makes a dirty work to generate "node release? and necessary bootstrap scripts.  Please check create-node command and concept of erlang releases: 
>> https://github.com/rebar/rebar/wiki/Rebar-commands 
>> http://www.erlang.org/doc/man/reltool.html 
>> 
>> It generates script looks something similar to this one. It also hooks the heart to watchdog the node.  
>> https://github.com/fogfish/hyperion/blob/master/rel/files/hyperion 
>> 
>> Personally, I am following an idea to package the application to release and ship it to destination hosts.
>> Long time ago, I?ve made an empty ?erlang release? project as show-case to study and debug various scenarios. You might look on it here:
>> 
>> https://github.com/fogfish/hyperion   
>> 
>> Best Regards, 
>> Dmitry
>> 
>> 
>>> On 12 May 2015, at 19:07, Roberto Ostinelli > wrote:
>>> 
>>> Fair enough, however at this point I cannot even a single one of these systems to simply START *and* STOP an Erlang release.
>>> I can easily start it (on ubuntu, `sudo service myapp start`) but the STOP command will always fail. That's because the init script cannot get a grasp of the BEAM process' pid (for some reason), hence it cannot stop it.
>>> 
>>> Any ideas on how to do that?
>>> 
>>> 
>>> On Tue, May 12, 2015 at 5:52 PM, Jesper Louis Andersen > wrote:
>>> 
>>> On Mon, May 11, 2015 at 9:13 PM, Roberto Ostinelli > wrote:
>>> In non-erlang systems, I would have standard watchdogs that launch an application on OS boot, and then monitor it and relaunch it if necessary.
>>> 
>>> The heart system in Erlang is a simple watchdog, mostly used if you nothing else that will restart your application. In an SysV init system, there is no automatic watching and restart. In RcNG in FreeBSD, there is no restart. In OpenBSDs rc, there is no automatic restart.
>>> 
>>> But as you say, many modern init systems also provides watching of the applications it starts, and in that case I wouldn't run with heart enabled. Better to let the system above handle it, and plug into its monitoring/reporting and so on.
>>> 
>>> Heart is very useful if you start nodes outside the control of the system watchdog though. In that case, they won't be restarted, and you can run heart on those.
>>> 
>>> 
>>> -- 
>>> J.
>>> 
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED 
>>> http://erlang.org/mailman/listinfo/erlang-questions 
>> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roberto.ostinelli@REDACTED  Tue May 12 19:36:29 2015
From: roberto.ostinelli@REDACTED (Roberto Ostinelli)
Date: Tue, 12 May 2015 19:36:29 +0200
Subject: [erlang-questions] HEART
In-Reply-To: 
References: 
 
 
 
 <1431449900.2713951.266897849.0D1D4324@webmail.messagingengine.com>
 <9B082D3C-0E45-4D21-B914-4AD4CECE03C3@widetag.com>
 
Message-ID: <210B32A7-4EC4-4BE6-B935-43D48D78AC77@widetag.com>



> On 12/mag/2015, at 19:31, Jesper Louis Andersen  wrote:
> 
> It is known through epmd if I remember correctly. So that way you should be able to stop it.

Thank you Jesper,
Do you hve an example on how to do so?

From roberto.ostinelli@REDACTED  Tue May 12 19:45:03 2015
From: roberto.ostinelli@REDACTED (Roberto Ostinelli)
Date: Tue, 12 May 2015 19:45:03 +0200
Subject: [erlang-questions] HEART
In-Reply-To: <1431452073.2721867.266937257.35C09C47@webmail.messagingengine.com>
References: 
 
 
 
 <1431449900.2713951.266897849.0D1D4324@webmail.messagingengine.com>
 <9B082D3C-0E45-4D21-B914-4AD4CECE03C3@widetag.com>
 <1431452073.2721867.266937257.35C09C47@webmail.messagingengine.com>
Message-ID: 


> On 12/mag/2015, at 19:34, Tristan Sloughter  wrote:
> 
> The start script generated using rebar should also have a getpid supported argument and a start that works the same. The scripts between relx and rebar are nearly identical.

Right. Unfortunately I can't find a way to oass this pid to the original script that starts it (using upstart).

Oh well. Will dig further. :)

thanks!


From roger@REDACTED  Tue May 12 20:44:22 2015
From: roger@REDACTED (Roger Lipscombe)
Date: Tue, 12 May 2015 19:44:22 +0100
Subject: [erlang-questions] HEART
In-Reply-To: 
References: 
 
 
 
 <1431449900.2713951.266897849.0D1D4324@webmail.messagingengine.com>
 <9B082D3C-0E45-4D21-B914-4AD4CECE03C3@widetag.com>
 <1431452073.2721867.266937257.35C09C47@webmail.messagingengine.com>
 
Message-ID: 

On 12 May 2015 at 18:45, Roberto Ostinelli
 wrote:
> Right. Unfortunately I can't find a way to oass this pid to the original script that starts it (using upstart).

We use relx-generated releases with upstart. Simply run "bin/myapp
foreground" from the upstart script.


From roberto.ostinelli@REDACTED  Tue May 12 20:53:49 2015
From: roberto.ostinelli@REDACTED (Roberto Ostinelli)
Date: Tue, 12 May 2015 20:53:49 +0200
Subject: [erlang-questions] HEART
In-Reply-To: 
References: 
 
 
 
 <1431449900.2713951.266897849.0D1D4324@webmail.messagingengine.com>
 <9B082D3C-0E45-4D21-B914-4AD4CECE03C3@widetag.com>
 <1431452073.2721867.266937257.35C09C47@webmail.messagingengine.com>
 
 
Message-ID: <7978EEA2-A2F1-4A7E-A07F-D381E2F99A4C@widetag.com>



> On 12/mag/2015, at 20:44, Roger Lipscombe  wrote:
> 
> We use relx-generated releases with upstart. Simply run "bin/myapp
> foreground" from the upstart script.

Yes this seems to be a good idea. Thank you. 


From eric.pailleau@REDACTED  Tue May 12 21:07:07 2015
From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=)
Date: Tue, 12 May 2015 21:07:07 +0200
Subject: [erlang-questions] HEART
In-Reply-To: <7978EEA2-A2F1-4A7E-A07F-D381E2F99A4C@widetag.com>
Message-ID: <6783310e-a302-4fae-b7ae-51ba24ceafa4@email.android.com>

Hi,

Or let your program inherit of your current pid, something like :

echo $$ > /var/run/myapp.pid
exec bin/myapp start

Regards

Le?12 mai 2015 20:53, Roberto Ostinelli  a ?crit?:
>
>
>
> > On 12/mag/2015, at 20:44, Roger Lipscombe  wrote: 
> > 
> > We use relx-generated releases with upstart. Simply run "bin/myapp 
> > foreground" from the upstart script. 
>
> Yes this seems to be a good idea. Thank you. 
> _______________________________________________ 
> erlang-questions mailing list 
> erlang-questions@REDACTED 
> http://erlang.org/mailman/listinfo/erlang-questions 

From hanssv@REDACTED  Tue May 12 21:44:53 2015
From: hanssv@REDACTED (Hans Svensson)
Date: Tue, 12 May 2015 21:44:53 +0200
Subject: [erlang-questions] Final Call for Papers: Erlang Workshop 2015
Message-ID: <55525835.4060205@gmail.com>

Hello all,

Please find below the Final Call for Papers for the Fourteenth ACM
SIGPLAN Erlang Workshop. Submission deadline is 10 days from now 
(Friday, May 22).

Apologies for any duplicates you may receive.

CALL FOR PAPERS
===============

Fourteenth ACM SIGPLAN Erlang Workshop
-----------------------------------------------------------

Vancouver, Canada, September 4, 2015
Satellite event of the 20th ACM SIGPLAN International Conference on
Functional Programming (ICFP 2015)
August 30 - September 5, 2015

http://www.erlang.org/workshop/2015/ErlangWorkshop2015.html

Erlang is a concurrent, distributed functional programming language
aimed at systems with requirements of massive concurrency, soft real
time response, fault tolerance, and high availability. It has been
available as open source for 16 years, creating a community that
actively contributes to its already existing rich set of libraries and
applications. Originally created for telecom applications, its usage
has spread to other domains including e-commerce, banking, databases,
and computer telephony and messaging.

Erlang programs are today among the largest applications written in
any functional programming language. These applications offer new
opportunities to evaluate functional programming and functional
programming methods on a very large scale and suggest new problems for
the research community to solve.

This workshop will bring together the open source, academic, and
industrial programming communities of Erlang. It will enable
participants to familiarize themselves with recent developments on new
techniques and tools tailored to Erlang, novel applications, draw
lessons from users' experiences and identify research problems and
common areas relevant to the practice of Erlang and functional
programming.

We invite three types of submissions.

1. Technical papers describing language extensions, critical
discussions of the status quo, formal semantics of language
constructs, program analysis and transformation, virtual machine
extensions and compilation techniques, implementations and interfaces
of Erlang in/with other languages, and new tools (profilers, tracers,
debuggers, testing frameworks, etc.). The maximum length for technical
papers is restricted to 12 pages.

2. Practice and application papers describing uses of Erlang in the
"real-world", Erlang libraries for specific tasks, experiences from
using Erlang in specific application domains, reusable programming
idioms and elegant new ways of using Erlang to approach or solve a
particular problem. The maximum length for the practice and
application papers is restricted to 12 pages. Note that this is a
maximum length; we welcome shorter papers also, and the program
committee will evaluate all papers on an equal basis independent of
their lengths.

3. Poster presentations describing topics related to the workshop
goals. Each includes a maximum of 2 pages of the abstract and summary.
Presentations in this category will be given an hour of shared
simultaneous demonstration time.

Workshop Co-Chairs
------------------
Hans Svensson, QuviQ AB, Sweden
Melinda T?th, E?tv?s Lor?nd University, Hungary

Program Committee
-----------------------------
(Note: the Workshop Co-Chairs are also committee members)

Jesper L. Andersen, Independent
Clara Benac Earle, Technical University of Madrid, Spain
Laura M. Castro, University of  A Coru?a, Spain
Christopher Meiklejohn, Basho Technologies, Inc., US
Samuel Rivas, Klarna AB, Sweden
Tee Teoh, Erlang Solutions Ltd, UK
Simon Thompson, University of Kent, UK


Important Dates
-----------------------
Submissions due: Friday, 22 May, 2015
Author notification: Friday, 26 June, 2015
Final copy due: Sunday, 19 July, 2015
Workshop date: September 4, 2015

Instructions to authors
--------------------------------
Papers must be submitted online via EasyChair (via the "Erlang2015"
event). The submission page is
https://www.easychair.org/conferences/?conf=erlang2015

Submitted papers should be in portable document format (PDF),
formatted using the ACM SIGPLAN style guidelines.

Each submission must adhere to SIGPLAN's republication policy.
Violation risks summary rejection of the offending submission.
Accepted papers will be published by the ACM and will appear in the
ACM Digital Library.

The proceedings will be freely available for download from the ACM Digital
Library from one week before the start of the conference until two weeks
after the conference.

Paper submissions will be considered for poster submission in the case
they are not accepted as full papers.

Venue & Registration Details
------------------------------------------
For registration, please see the ICFP 2015 web site at:
http://icfpconference.org/icfp2015/

Related Links
--------------------
ICFP 2015 web site: http://www.icfpconference.org/icfp2015/
Past ACM SIGPLAN Erlang workshops: http://www.erlang.org/workshop/
Open Source Erlang: http://www.erlang.org/
EasyChair submission site:
https://www.easychair.org/conferences/?conf=erlang2015
Author Information for SIGPLAN Conferences:
http://www.sigplan.org/authorInformation.htm
Atendee Information for SIGPLAN Events:
http://www.sigplan.org/Resources/Policies/Anti-harassment


From mfidelman@REDACTED  Wed May 13 00:37:50 2015
From: mfidelman@REDACTED (Miles Fidelman)
Date: Tue, 12 May 2015 18:37:50 -0400
Subject: [erlang-questions] HEART
In-Reply-To: 
References: 
 
 
 
 <08E64BBA-163C-4CFD-885C-0F2FC9BAD36B@widetag.com>
 
Message-ID: <555280BE.7030605@meetinghouse.net>

> On Mon, May 11, 2015 at 9:13 PM, Roberto Ostinelli <
> > wrote:
>
> >/  In non-erlang systems, I would have standard watchdogs that launch an
> />/  application on OS boot, and then monitor it and relaunch it if necessary.
> /
>
> The heart system in Erlang is a simple watchdog, mostly used if you nothing
> else that will restart your application. In an SysV init system, there is
> no automatic watching and restart. In RcNG in FreeBSD, there is no restart.
> In OpenBSDs rc, there is no automatic restart.
>

Wait a minute - isn't that what respawn does in a SysV init environment?

Of course that only works if the VM well and truly dies.  If it locks up, you still have a problem.

Anybody see a reason you couldn't:
1. start BEAM with respawn
2. start a separate watchdog process that periodically runs a few tests to see if BEAM is running properly,
and if not, KILLs the process - at which point respawn would take care of a restart.

Miles Fidelman


-- 
In theory, there is no difference between theory and practice.
In practice, there is.   .... Yogi Berra



From mmartin4242@REDACTED  Wed May 13 02:59:55 2015
From: mmartin4242@REDACTED (Michael L Martin)
Date: Tue, 12 May 2015 19:59:55 -0500
Subject: [erlang-questions] HEART
In-Reply-To: <555280BE.7030605@meetinghouse.net>
References: 
 
 
 
 <08E64BBA-163C-4CFD-885C-0F2FC9BAD36B@widetag.com>
 
 <555280BE.7030605@meetinghouse.net>
Message-ID: <5552A20B.7090104@gmail.com>

But who watches the watchdog?


On 05/12/2015 05:37 PM, Miles Fidelman wrote:
>> On Mon, May 11, 2015 at 9:13 PM, Roberto Ostinelli <
>> > wrote:
>>
>> >/  In non-erlang systems, I would have standard watchdogs that 
>> launch an
>> />/  application on OS boot, and then monitor it and relaunch it if 
>> necessary.
>> /
>>
>> The heart system in Erlang is a simple watchdog, mostly used if you 
>> nothing
>> else that will restart your application. In an SysV init system, 
>> there is
>> no automatic watching and restart. In RcNG in FreeBSD, there is no 
>> restart.
>> In OpenBSDs rc, there is no automatic restart.
>>
>
> Wait a minute - isn't that what respawn does in a SysV init environment?
>
> Of course that only works if the VM well and truly dies.  If it locks 
> up, you still have a problem.
>
> Anybody see a reason you couldn't:
> 1. start BEAM with respawn
> 2. start a separate watchdog process that periodically runs a few 
> tests to see if BEAM is running properly,
> and if not, KILLs the process - at which point respawn would take care 
> of a restart.
>
> Miles Fidelman
>
>



From zxq9@REDACTED  Wed May 13 03:31:58 2015
From: zxq9@REDACTED (zxq9)
Date: Wed, 13 May 2015 10:31:58 +0900
Subject: [erlang-questions] HEART
In-Reply-To: <5552A20B.7090104@gmail.com>
References: 
 <555280BE.7030605@meetinghouse.net> <5552A20B.7090104@gmail.com>
Message-ID: <8643859.87U046Nyue@changa>

On 2015?5?12? ??? 19:59:55 Michael L Martin wrote:
> But who watches the watchdog?

Depends on which type you're talking about.
Some (most?) watchdog/procdoc type systems start two processes that watch each other in addition to the target process(es).

When writing these for my own daemons I usually make an "undead mode" where a monitor daemon is created to watch the service daemon, and the service daemon itself acts as the monitor for the monitor daemon.

-Craig


From flexchap@REDACTED  Wed May 13 01:00:02 2015
From: flexchap@REDACTED (Peter Johansson)
Date: Wed, 13 May 2015 01:00:02 +0200
Subject: [erlang-questions] Retrieving "semi-constant" data from a
 function versus Mnesia
In-Reply-To: 
References: 
 
 
 
Message-ID: 

Hi Joe !

Thank you very much for a resolving answer/response indeed !

This is my debut / first ever communication/message-interchange with you by
the way   :-)
( but I have been aware of Erlang  ....you ..and some of the other
community-active users for a few years now ).

And also ... Thank you way ...way more for your contribution of/to a
language being absolutely marvelously flexible & simplistic to use in real
life prototyping/"testing out"-situations ( it even beats Python in this
regard if you ask me ).


It struck me yesterday that I actually have two other more "generic"
Erlang-question
as well .....but which happen to fit in completely naturally & closely
under this current question-thread by their nature .....so I put them here
too.

1:
Are fixed data, defined & compiled inside functions, always stored inside &
referenced from the constant-pool of the particular modules regardless of
what Erlang term-types/structures they holds ....or does it exist special
situations when such fixed data becomes partly or fully copied into the
heap/stack of the calling process ?

2:
In the current web-application project I work with (implemented on top of
Yaws) I have the following type of function-call construct ( to achieve
server-side method-dispatching )

Variable_module_name:fixed_function_name(),

According to the efficiency-guide of the Erlang-documentation this type of
call-construct is more "expensive" (about six times) compare to a fully
fixed name function-call.

In what sense is it more expensive ?  ....is it about the time-lag between
the point when the VM catch/discovers this call-construct and the point
when the functional content (the prepared sub-routines) actually can be
executed ?


Once again ....thank you very much for contributing this language to the
programmer-community.
Sending my best regards !

Peter , Lund Sverige

2015-05-11 14:32 GMT+02:00 Joe Armstrong :

> On Sun, May 10, 2015 at 10:32 PM, Benoit Chesneau 
> wrote:
> >
> >
> > On Sun, May 10, 2015 at 10:23 PM Joe Armstrong  wrote:
> >>
> >> How large is the total data?
> >>
> >> If it's small you could define this in a module and not use a database
> >> or process at all.
> >
> >
> > How small should it be? What if the compiled bean is about 888 kilobytes?
>
> It depends :-)
>
> There are many factors involved:
>
>    a) How much memory do you have
>    b) How often do you want to change the configuration data
>    c) How quick do you the change to be
>
> Assume
>
>   a) is a smallish number of GBytes (normal these days)
>   b) is "relatively rarely" (I don't know what this means - (aside - this
> is why
>       I always ask people to provide numbers in their questions))
>      Say once a day
>   c) is a few seconds
>
> Then it should be perfectly doable. Making a module containing all the
> config data
> and recompiling when necessary is certainly the fastest way to access the
> data -
> this has very fast reads but very slow writes - you could think of a
> module as a database
> optimized for reads but not writes :-)
>
> /Joe
>
>
>
> >
> > - benoit
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From vychodil.hynek@REDACTED  Wed May 13 09:07:55 2015
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Wed, 13 May 2015 09:07:55 +0200
Subject: [erlang-questions] Retrieving "semi-constant" data from a
 function versus Mnesia
In-Reply-To: 
References: 
Message-ID: 

What prevents you using

generic_config:data_one(Config, ...)

instead of

Config_Module:data_one(...)

It is just one more step of indirection. It can be a viable way in the case
when write operations are very rare.

Hynek

On Mon, May 11, 2015 at 8:02 PM, Jay Nelson  wrote:

> (Edited previous posts for relevance, including only Joe?s comments
>
> interspersed with my responses.)
>
> > If it's small you could define this in a module and not use a database
> > or process at all.
>
> > -module(global_data).
> > -export(...)
>
> > data_one() ->
> >    ....
>
> > Then dynamically change and recompile the module. Any call
> > global_data:data_one() will pick up the value from the "latest version"
>
> > /Joe
>
> This is exactly the approach I was taking, but I assumed that there would
> be
> various implementations so I used a behaviour to identify the specific
> config:
>
>
> https://github.com/duomark/elysium/blob/master/src/elysium_config.erl#L95-L108
>
> https://github.com/duomark/elysium/blob/master/src/elysium_default_config.erl#L19-L39
>
> The data set is quite small as it is normal app configuration data.
> I implemented in the most straightforward obvious way:
>
>
> https://github.com/duomark/elysium/blob/master/src/elysium_config.erl#L268-L273
>
> It turns out the Config_Module:fn(?) is the killer of performance. This
> alternative
> approach is much more performant if you are ever faced with a similar
> situation
> (example from some code I was browsing recently):
>
> https://github.com/knutin/elli/blob/master/src/elli_tcp.erl#L51-L54
>
> > Making a module containing all the config data
> > and recompiling when necessary is certainly the fastest way
>
> > to access the data -this has very fast reads but very slow writes - you could think of a
>
> > module as a database optimized for reads but not writes :-)
>
> So I thought as well, but you *must* call it with a compiled module name or
> eliminate the performance benefit. I mostly wanted to guarantee distributed
> lock-free code access for many concurrent readers to avoid contention, but
> it proved the slowest of the 3 approaches when using a behaviour.
>
> jay
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From mailparmalat@REDACTED  Wed May 13 09:21:05 2015
From: mailparmalat@REDACTED (Steven)
Date: Wed, 13 May 2015 09:21:05 +0200
Subject: [erlang-questions] beam.smp[20162]: segfault at 00002aaad3e84bc8
 rip 00000000004ee448 rsp 0000000042877c60 error 4
Message-ID: 

Good day everyone

We recently experienced a segmentation fault on one of our runtimes. The
runtime was carrying traffic and been running for about a year. Running
R13B4 64bit with smp enabled and OS is redhat 5.2 64bit. The runtime is
started with +S 8 +A 32 and not using any nifs. Unfortunately, no core dump
and this is the first time we seeing it. There's another 5 nodes with the
same deployment and hasn't happened on those runtimes (touch wood). Below
is the information I could extract. I can enable core dump but won't know
when this will happen again.

*Erlang
/usr/local/erlang/R13B-4/lib/erlang/bin/erl
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:16:16] [rq:16]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)

*beam.smp
strings -a /usr/local/erlang/R13B-4/lib/erlang/erts-5.7.5/bin/beam.smp |
fgrep GCC | more
GCC: (GNU) 4.1.2 20071124 (Red Hat 4.1.2-41)

*Disassembled the beam.smp, instruction pointer where the fault occurred
shown below.

 cat beam.smp.objdump | grep -C 10 4ee448
  4ee3fe:       48 3b 54 24 08          cmp    0x8(%rsp),%rdx
  4ee403:       48 89 74 24 68          mov    %rsi,0x68(%rsp)
  4ee408:       74 12                   je     4ee41c

  4ee40a:       48 8b 35 87 21 37 00    mov    3613063(%rip),%rsi        #
860598 
  4ee411:       bf 09 00 00 00          mov    $0x9,%edi
  4ee416:       ff 15 74 21 37 00       callq  *3613044(%rip)        #
860590 
  4ee41c:       4d 85 ff                test   %r15,%r15
  4ee41f:       0f 84 b1 08 00 00       je     4eecd6

  4ee425:       48 8b 54 24 58          mov    0x58(%rsp),%rdx
  4ee42a:       48 39 54 24 68          cmp    %rdx,0x68(%rsp)
  4ee42f:       75 17                   jne    4ee448

  4ee431:       eb 52                   jmp    4ee485

  4ee433:       48 83 f8 02             cmp    $0x2,%rax
  4ee437:       0f 84 dc 05 00 00       je     4eea19

  4ee43d:       48 83 c2 08             add    $0x8,%rdx
  4ee441:       48 39 54 24 68          cmp    %rdx,0x68(%rsp)
  4ee446:       74 3d                   je     4ee485

  4ee448:       48 8b 32                mov    (%rdx),%rsi
  4ee44b:       48 89 f0                mov    %rsi,%rax
  4ee44e:       83 e0 03                and    $0x3,%eax
  4ee451:       48 83 f8 01             cmp    $0x1,%rax
  4ee455:       0f 84 eb 04 00 00       je     4ee946

  4ee45b:       73 d6                   jae    4ee433

  4ee45d:       48 89 f0                mov    %rsi,%rax
  4ee460:       83 e0 3c                and    $0x3c,%eax
  4ee463:       74 d8                   je     4ee43d

  4ee465:       48 83 f8 04             cmp    $0x4,%rax
  4ee469:       4c 8d 4a 08             lea    0x8(%rdx),%r9
  4ee46d:       0f 1f 00                nopl   (%rax)
  4ee470:       0f 84 5f 07 00 00       je     4eebd5

  4ee476:       48 c1 ee 06             shr    $0x6,%rsi
  4ee47a:       49 8d 14 f1             lea    (%r9,%rsi,8),%rdx
  4ee47e:       48 39 54 24 68          cmp    %rdx,0x68(%rsp)
  4ee483:       75 c3                   jne    4ee448

  4ee485:       48 8b bd 20 02 00 00    mov    0x220(%rbp),%rdi
  4ee48c:       48 39 fb                cmp    %rdi,%rbx
  4ee48f:       0f 87 27 08 00 00       ja     4eecbc

  4ee495:       48 8b b5 10 02 00 00    mov    0x210(%rbp),%rsi
  4ee49c:       48 39 75 10             cmp    %rsi,0x10(%rbp)
  4ee4a0:       48 8b 44 24 58          mov    0x58(%rsp),%rax
  4ee4a5:       48 0f 44 44 24 68       cmove  0x68(%rsp),%rax
  4ee4ab:       48 83 bd 38 02 00 00    cmpq   $0x0,0x238(%rbp)
  4ee4b2:       00
  4ee4b3:       48 89 9d 20 02 00 00    mov    %rbx,0x220(%rbp)

Any help or knowledge would be appreciated. Thanks all

Regards,
Steven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From matwey.kornilov@REDACTED  Wed May 13 09:30:41 2015
From: matwey.kornilov@REDACTED (Matwey V. Kornilov)
Date: Wed, 13 May 2015 10:30:41 +0300
Subject: [erlang-questions] Patch Package OTP 17.5.3 Released
In-Reply-To: <5549E237.9050503@erlang.org>
References: <5549E237.9050503@erlang.org>
Message-ID: 


Hi,

Why is 17.5.3 not available as prebuild package at 
http://www.erlang.org/download.html ?

06.05.2015 12:43, Zandra Hird ?????:
> Patch Package:           OTP 17.5.3
> Git Tag:                 OTP-17.5.3
>
>   Check out the git tag OTP-17.5.3, and build a full OTP system
>   including documentation. Apply one or more applications from this
>   build as patches to your installation using the 'otp_patch_apply'
>   tool. For information on install requirements, see descriptions for
>   each application version below.
>
>   ---------------------------------------------------------------------
>   --- common_test-1.10.1 ----------------------------------------------
>   ---------------------------------------------------------------------
>
>   Note! The common_test-1.10.1 application can *not* be applied
>         independently of other applications on an arbitrary OTP 17
>         installation.
>
>         On a full OTP 17 installation, also the following runtime
>         dependency has to be satisfied:
>         -- test_server-3.7.1 (first satisfied in OTP 17.1)
>
>
>   --- Fixed Bugs and Malfunctions ---
>
>    OTP-12643    Application(s): common_test
>
>                 A fault in the Common Test logger process, that caused
>                 the application to crash when running on a long name
>                 node, has been corrected.
>
>
>    OTP-12688    Application(s): common_test
>                 Related Id(s): seq12818
>
>                 A 'wait_for_prompt' option in ct_telnet:expect/3 has
>                 been introduced which forces the function to not return
>                 until a prompt string has been received, even if other
>                 expect patterns have already been found.
>
>
>    OTP-12697    Application(s): common_test, test_server
>                 Related Id(s): seq12848
>
>                 If the last expression in a test case causes a timetrap
>                 timeout, the stack trace is ignored and not printed to
>                 the test case log file. This happens because the
>                 {Suite,TestCase,Line} info is not available in the
>                 stack trace in this scenario, due to tail call
>                 elimination. Common Test has been modified to handle
>                 this situation by inserting a
>                 {Suite,TestCase,last_expr} tuple in the correct place
>                 and printing the stack trace as expected.
>
>
>    OTP-12698    Application(s): common_test
>                 Related Id(s): seq12844
>
>                 Fixed a buffer problem in ct_netconfc which could cause
>                 that some messages where buffered forever.
>
>
>    OTP-12704    Application(s): common_test, erts
>                 Related Id(s): OTP-10922
>
>                 The VTS mode in Common Test has been modified to use a
>                 private version of the Webtool application
>                 (ct_webtool).
>
>
>    OTP-12707    Application(s): common_test
>                 Related Id(s): seq12846
>
>                 Add possibility to add user capabilities in
>                 ct_netconfc:hello/3.
>
>
>   Full runtime dependencies of common_test-1.10.1: compiler-5.0,
>   crypto-3.3, debugger-4.0, erts-6.0, inets-5.10, kernel-3.0,
>   runtime_tools-1.8.14, sasl-2.4, snmp-4.25.1, ssh-3.0.1, stdlib-2.0,
>   test_server-3.7.1, tools-2.6.14, webtool-0.8.10, xmerl-1.3.7
>
>
>   ---------------------------------------------------------------------
>   --- diameter-1.9.1 --------------------------------------------------
>   ---------------------------------------------------------------------
>
>   The diameter-1.9.1 application can be applied independently of other
>   applications on a full OTP 17 installation.
>
>   --- Known Bugs and Problems ---
>
>    OTP-12642    Application(s): diameter
>
>                 Don't leave extra bit in decoded AVP data.
>
>                 OTP-12074 in OTP 17.3 missed one case: a length error
>                 on a trailing AVP unknown to the dictionary in
>                 question.
>
>
>    OTP-12654    Application(s): diameter
>                 Related Id(s): seq12851
>
>                 Don't confuse Result-Code and Experimental-Result
>
>                 The errors field of a decoded diameter_packet record
>                 was populated with a Result-Code AVP when an
>                 Experimental-Result containing a 3xxx Result-Code was
>                 received in an answer not setting the E-bit. The
>                 correct AVP is now extracted from the incoming message.
>
>
>    OTP-12701    Application(s): diameter
>
>                 Don't count on unknown Application Id.
>
>                 OTP-11721 in OTP 17.1 missed the case of an Application
>                 Id not agreeing with that of the dictionary in
>                 question, causing counters to be accumulated on keys
>                 containing the unknown id.
>
>
>   Full runtime dependencies of diameter-1.9.1: erts-6.0, kernel-3.0,
>   ssl-5.3.4, stdlib-2.0
>
>
>   ---------------------------------------------------------------------
>   --- erts-6.4.1 ------------------------------------------------------
>   ---------------------------------------------------------------------
>
>   The erts-6.4.1 application can be applied independently of other
>   applications on a full OTP 17 installation.
>
>   --- Fixed Bugs and Malfunctions ---
>
>    OTP-12704    Application(s): common_test, erts
>                 Related Id(s): OTP-10922
>
>                 The VTS mode in Common Test has been modified to use a
>                 private version of the Webtool application
>                 (ct_webtool).
>
>
>   Full runtime dependencies of erts-6.4.1: kernel-3.0, sasl-2.4,
>   stdlib-2.0
>
>
>   ---------------------------------------------------------------------
>   --- snmp-5.1.2 ------------------------------------------------------
>   ---------------------------------------------------------------------
>
>   The snmp-5.1.2 application can be applied independently of other
>   applications on a full OTP 17 installation.
>
>   --- Fixed Bugs and Malfunctions ---
>
>    OTP-12669    Application(s): snmp
>                 Related Id(s): seq12841
>
>                 A bug in the SNMP Agent has been corrected; when
>                 opening a port using the command line argument
>                 -snmpa_fd the Port should be 0 when calling
>                 gen_udp:open.
>
>                 A bug in the SNMP manager has been corrected; it should
>                 not look at the -snmp_fd command line argument, but
>                 instead at -snmpm_fd.
>
>
>   --- Improvements and New Features ---
>
>    OTP-12452    Application(s): snmp
>
>                 Improved cryptocraphic capability.
>
>
>   Full runtime dependencies of snmp-5.1.2: crypto-3.3, erts-6.0,
>   kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, stdlib-2.0
>
>
>   ---------------------------------------------------------------------
>   --- test_server-3.8.1 -----------------------------------------------
>   ---------------------------------------------------------------------
>
>   Note! The test_server-3.8.1 application can *not* be applied
>         independently of other applications on an arbitrary OTP 17
>         installation.
>
>         On a full OTP 17 installation, also the following runtime
>         dependency has to be satisfied:
>         -- syntax_tools-1.6.16 (first satisfied in OTP 17.1.1)
>
>
>   --- Fixed Bugs and Malfunctions ---
>
>    OTP-12697    Application(s): common_test, test_server
>                 Related Id(s): seq12848
>
>                 If the last expression in a test case causes a timetrap
>                 timeout, the stack trace is ignored and not printed to
>                 the test case log file. This happens because the
>                 {Suite,TestCase,Line} info is not available in the
>                 stack trace in this scenario, due to tail call
>                 elimination. Common Test has been modified to handle
>                 this situation by inserting a
>                 {Suite,TestCase,last_expr} tuple in the correct place
>                 and printing the stack trace as expected.
>
>
>   Full runtime dependencies of test_server-3.8.1: erts-6.0, inets-5.10,
>   kernel-3.0, observer-2.0, runtime_tools-1.8.14, stdlib-2.0,
>   syntax_tools-1.6.16, tools-2.6.14
>
>
>   ---------------------------------------------------------------------
>   ---------------------------------------------------------------------
>   ---------------------------------------------------------------------




From ok@REDACTED  Wed May 13 11:11:51 2015
From: ok@REDACTED (ok@REDACTED)
Date: Wed, 13 May 2015 21:11:51 +1200
Subject: [erlang-questions] Retrieving "semi-constant" data from a
 function versus Mnesia
In-Reply-To: 
References: 
 
 
 
 
Message-ID: <2fef38bc3a8e928cfefb6714c6bd546a.squirrel@chasm.otago.ac.nz>

> 2:
> In the current web-application project I work with (implemented on top of
> Yaws) I have the following type of function-call construct ( to achieve
> server-side method-dispatching )
>
> Variable_module_name:fixed_function_name(),
>
> According to the efficiency-guide of the Erlang-documentation this type of
> call-construct is more "expensive" (about six times) compare to a fully
> fixed name function-call.
>
> In what sense is it more expensive ?

Time.

> ....is it about the time-lag between
> the point when the VM catch/discovers this call-construct and the point
> when the functional content (the prepared sub-routines) actually can be
> executed ?

If you compile a little example using erlc +"'S'" and then poke
around in beam_emu.c, you'll see that the dynamic function call
takes a couple of C function calls to find the place to go.
One of them is or was erts_find_export_entry(module, function, arity),
surprise surprise, which looks the triple up in a hash table, so it's
fairly clear where the time is going.

Another approach would be to do something like this:
if module m exports fn, generate
    ' apply f'(X1, ..., Xn, m) -> m:f(X1, ..., Xn);
and translate a csll M:f(E1, ..., En) as
    ' apply f'(E1, ..., En, M).

Looking at ...:...(...) calls in an Erlang/OTP release,
I found
    17.5% ?MODULE:function  -- ?MODULE &c
    81.6%  module:function
     0.0%  module:Function  -- some but very few
     0.8%  Module:function
     0.0%  Module:Function  -- some but very few
These figures are a bit dodgy but wouldn't be too far wrong.
In any case they're static figures, not dynamic.

While I think this kind of call _could_ be faster, I suspect
that they are fast _enough_ given their rarity and the other
things going on in a program.




From jesper.louis.andersen@REDACTED  Wed May 13 11:28:15 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Wed, 13 May 2015 11:28:15 +0200
Subject: [erlang-questions] Patch Package OTP 17.5.3 Released
In-Reply-To: 
References: <5549E237.9050503@erlang.org> 
Message-ID: 

On Wed, May 13, 2015 at 9:30 AM, Matwey V. Kornilov <
matwey.kornilov@REDACTED> wrote:

> Why is 17.5.3 not available as prebuild package at
> http://www.erlang.org/download.html ?
>

The smaller point-releases are normally meant for people/companies who had
a specific problem that got solved with the point release, and there is
overhead in producing official builds of quality.

This is more of an informational service about the changes, so you can
decide if it is worth picking up, or you better wait until the next minor
release.

-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From nico.kruber@REDACTED  Wed May 13 11:51:23 2015
From: nico.kruber@REDACTED (Nico Kruber)
Date: Wed, 13 May 2015 11:51:23 +0200
Subject: [erlang-questions] FYI: current erlang master (soon to be 18.0-rc2)
	fails using common test
Message-ID: <2148588.3UGZXaZKbP@csr-pc40.zib.de>

I tried "make test" in Scalaris [1]

> git clone https://github.com/scalaris-team/scalaris.git scalaris
> cd scalaris
> ./configure && make
> make test
ct_run  -name ct@`hostname -s`  -pa `pwd`/ebin `pwd`/test 
`pwd`/contrib/yaws/ebin `pwd`/contrib/log4erl/ebin -spec 
test/scalaris.runtests-default.cfg -ct_hooks scalaris_cth -noshell | tee 
ct.log ; \
        grep " 0 failed" ct.log


Common Test v1.10.1 starting (cwd is /)


Common Test: Running make in test directories...

CWD set to: "//ct_run.ct@REDACTED"

TEST INFO: 1 test(s), 62 suite(s)

Testing nico.scalaris: Starting test (with repeated test cases)
EXIT, reason {badarg,
                 [{erl_anno,anno_info,
                      [macro],
                      [{file,"erl_anno.erl"},{line,426}]},
                  {erl_anno,location,1,[{file,"erl_anno.erl"},{line,234}]},
                  {erl_anno,line,1,[{file,"erl_anno.erl"},{line,219}]},
                  {erl2html2,parse_non_preprocessed_file,3,
                      [{file,"erl2html2.erl"},{line,155}]},
                  {erl2html2,parse_non_preprocessed_file,3,
                      [{file,"erl2html2.erl"},{line,156}]},
                  {erl2html2,parse_non_preprocessed_file,1,
                      [{file,"erl2html2.erl"},{line,140}]},
                  {erl2html2,convert,4,[{file,"erl2html2.erl"},{line,56}]},
                  {test_server_ctrl,html_convert_modules,1,
                      [{file,"test_server_ctrl.erl"},{line,1928}]}]}Updating 
//index.html... done
Updating //all_runs.html... done
Test run crashed! This could be an internal error - please report!

{{case_clause,
     {value,
         {'EXIT',
             {badarg,
                 [{erl_anno,anno_info,
                      [macro],
                      [{file,"erl_anno.erl"},{line,426}]},
                  {erl_anno,location,1,[{file,"erl_anno.erl"},{line,234}]},
                  {erl_anno,line,1,[{file,"erl_anno.erl"},{line,219}]},
                  {erl2html2,parse_non_preprocessed_file,3,
                      [{file,"erl2html2.erl"},{line,155}]},
                  {erl2html2,parse_non_preprocessed_file,3,
                      [{file,"erl2html2.erl"},{line,156}]},
                  {erl2html2,parse_non_preprocessed_file,1,
                      [{file,"erl2html2.erl"},{line,140}]},
                  {erl2html2,convert,4,[{file,"erl2html2.erl"},{line,56}]},
                  {test_server_ctrl,html_convert_modules,1,
                      [{file,"test_server_ctrl.erl"},{line,1928}]}]}}}},
 [{ct_run,execute_all_specs,4,[{file,"ct_run.erl"},{line,459}]},
  {ct_run,script_start1,2,[{file,"ct_run.erl"},{line,376}]}]}



=ERROR REPORT==== 13-May-2015::11:48:24 ===
Error in process <0.52.0> on node 'ct@REDACTED' with exit value: 
{{case_clause,{value,{'EXIT',{badarg,[{erl_anno,anno_info,[macro],
[{file,"erl_anno.erl"},{line,426}]},{erl_anno,location,1,
[{file,"erl_anno.erl"},{line,234}]},{erl_anno,line,1,[{file,"erl_anno.erl"},
{line,219}]},{erl2html2,parse_non_preprocessed_file... 

make: *** [test] Error 1


[1] https://github.com/scalaris-team/scalaris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: This is a digitally signed message part.
URL: 

From erlang@REDACTED  Wed May 13 12:26:54 2015
From: erlang@REDACTED (Joe Armstrong)
Date: Wed, 13 May 2015 12:26:54 +0200
Subject: [erlang-questions] Retrieving "semi-constant" data from a
 function versus Mnesia
In-Reply-To: 
References: 
 
 
 
 
Message-ID: 

On Wed, May 13, 2015 at 1:00 AM, Peter Johansson  wrote:
> Hi Joe !
>
> Thank you very much for a resolving answer/response indeed !
>
> This is my debut / first ever communication/message-interchange with you by
> the way   :-)
> ( but I have been aware of Erlang  ....you ..and some of the other
> community-active users for a few years now ).

Welcome to the Erlang mailing list, the place were we try to solve all the
world's programming problems :-)

>
> And also ... Thank you way ...way more for your contribution of/to a
> language being absolutely marvelously flexible & simplistic to use in real
> life prototyping/"testing out"-situations ( it even beats Python in this
> regard if you ask me ).
>
>
> It struck me yesterday that I actually have two other more "generic"
> Erlang-question as well .....but which happen to fit in completely naturally
> & closely under this current question-thread by their nature .....so I put
> them here too.
>
> 1:
> Are fixed data, defined & compiled inside functions, always stored inside &
> referenced from the constant-pool of the particular modules regardless of
> what Erlang term-types/structures they holds ....or does it exist special
> situations when such fixed data becomes partly or fully copied into the
> heap/stack of the calling process ?

Answer 1)

   You're not supposed to know. Your expectation should be that
fixed data will be compiled and implemented as efficiently as possible.
If this is not the case and your expectation is not met you should
complain loudly.

aside: this has happened on several occasions. If your expectations are
not met then tell us. Inefficient handling of fixed data is a bug.

Even if I told you the answer - then whatever you observe today might
not be true in a years time.

It seems to me to be crazy to optimise code on the basis of how it
performs *today* and expect these optimisation to be true in a few
years time. The hardware will change.

Optimisation should be the very last thing you do.

You should

    - write clean short easy to understand code

      (the goal is that *you* can understand your own code in a years time)

     


    - measure, measure, measure
    - optimize if *absolutely* necessary

If you want your program to go a thousand times faster wait 10 years. Do
nothing and wait. This is by far the easiest way to optimize a program.




This is why having  small clean code base wins in the long run. Erlang
(today) is million of times faster than the mid 1980s version -
this speed up has *not* come from software optimisations but from hardware
changes.

The desire to resist optimize requires saintly dedication - you can
optimize if and only if your program becomes clearer shorter and more
beautiful.

Answer 2)

Measure measure measure

Answer 3)

Wait ten years

Answer 4)

Buy/Borrow a faster machine

Answer 5)

Yes (ish) it is my understanding that fixed data is special cased to keep
it off-stack and heap (or at least it should be)


>
> 2:
> In the current web-application project I work with (implemented on top of
> Yaws) I have the following type of function-call construct ( to achieve
> server-side method-dispatching )
>
> Variable_module_name:fixed_function_name(),
>
> According to the efficiency-guide of the Erlang-documentation this type of
> call-construct is more "expensive" (about six times) compare to a fully
> fixed name function-call.
>
> In what sense is it more expensive ?  ....is it about the time-lag between
> the point when the VM catch/discovers this call-construct and the point when
> the functional content (the prepared sub-routines) actually can be executed
> ?

measure ^ 3 (again)

Cheers

/Joe

>
>
> Once again ....thank you very much for contributing this language to the
> programmer-community.
> Sending my best regards !
>
> Peter , Lund Sverige
>
> 2015-05-11 14:32 GMT+02:00 Joe Armstrong :
>>
>> On Sun, May 10, 2015 at 10:32 PM, Benoit Chesneau 
>> wrote:
>> >
>> >
>> > On Sun, May 10, 2015 at 10:23 PM Joe Armstrong  wrote:
>> >>
>> >> How large is the total data?
>> >>
>> >> If it's small you could define this in a module and not use a database
>> >> or process at all.
>> >
>> >
>> > How small should it be? What if the compiled bean is about 888
>> > kilobytes?
>>
>> It depends :-)
>>
>> There are many factors involved:
>>
>>    a) How much memory do you have
>>    b) How often do you want to change the configuration data
>>    c) How quick do you the change to be
>>
>> Assume
>>
>>   a) is a smallish number of GBytes (normal these days)
>>   b) is "relatively rarely" (I don't know what this means - (aside - this
>> is why
>>       I always ask people to provide numbers in their questions))
>>      Say once a day
>>   c) is a few seconds
>>
>> Then it should be perfectly doable. Making a module containing all the
>> config data
>> and recompiling when necessary is certainly the fastest way to access the
>> data -
>> this has very fast reads but very slow writes - you could think of a
>> module as a database
>> optimized for reads but not writes :-)
>>
>> /Joe
>>
>>
>>
>> >
>> > - benoit
>
>


From roberto@REDACTED  Wed May 13 13:41:53 2015
From: roberto@REDACTED (Roberto Ostinelli)
Date: Wed, 13 May 2015 13:41:53 +0200
Subject: [erlang-questions] HEART
In-Reply-To: 
References: 
 
 
 
 <1431449900.2713951.266897849.0D1D4324@webmail.messagingengine.com>
 <9B082D3C-0E45-4D21-B914-4AD4CECE03C3@widetag.com>
 <1431452073.2721867.266937257.35C09C47@webmail.messagingengine.com>
 
 
Message-ID: 

Hi,
Still experiencing weirdnesses though.

My upstart script is:

script
  export HOME=/root
  cd /usr/local/myapp
    exec bin/myapp foreground > /dev/null 2>&1
end script


When I start to attach to the app's node, I get:

$ /usr/local/myapp/bin/myapp attach
pong
Can't access pipe directory /tmp//usr/local/myapp/: No such file or
directory


However, if I start my app manually:

$ /usr/local/myapp/bin/myapp start

Then everything works fine:

$ /usr/local/cometa/bin/cometa attach
pong
Attaching to /tmp//usr/local/myapp/erlang.pipe.1 (^D to exit)

(myapp@REDACTED)1>


Can some kind soul explain to me what is going on?

Thank you,
r.







On Tue, May 12, 2015 at 8:44 PM, Roger Lipscombe 
wrote:

> On 12 May 2015 at 18:45, Roberto Ostinelli
>  wrote:
> > Right. Unfortunately I can't find a way to oass this pid to the original
> script that starts it (using upstart).
>
> We use relx-generated releases with upstart. Simply run "bin/myapp
> foreground" from the upstart script.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From kenneth@REDACTED  Wed May 13 13:48:45 2015
From: kenneth@REDACTED (Kenneth Lundin)
Date: Wed, 13 May 2015 13:48:45 +0200
Subject: [erlang-questions] Erlang/OTP 18.0-rc2 is available for testing
Message-ID: 

Erlang/OTP 18.0-rc2 is available for testing.

This is the second and last release candidate before the final OTP 18.0
product release in June 2015.

Between the 18.0 rc1 and 18.0 rc 2 and the final release there will be new
updates of
the master branch with corrections and minor new features.
Occasionally there might be new tags which we in that
case will communicate and ask you to test.

Erlang/OTP 18.0 is a new major release with new features, quite a few
(characteristics) improvements, as well as a few incompatibilities.

See the Release Notes and the documentation for more details.

We would like to ask you to build and test this release candidate and send
us
your feedback as soon as possible, so that we can make the necessary
corrections before OTP 18.0.

The release contains many changes; thus, some unexpected incompatibilities
or issues may have slipped through our tests.
Please try to build and run your current products/applications and let us
know about any problems.


*IMPORTANT INFO when building your own code with this OTP release*
Since erlang:now is deprecated your build might stop if you are using
"warnings as errors".
To let the build through you can turn of warnings for deprecated functions
by setting an environment variable like this:
  export ERL_COMPILER_OPTIONS=nowarn_deprecated_function

Some highlights of the release are:

   - dialyzer: The -dialyzer() attribute can be used for suppressing
   warnings
   in a module by specifying functions or warning options.
   It can also be used for requesting warnings in a module.
   - erts: The time functionality has been extended. This includes a new
   API for
   time, as well as "time warp" modes which alters the behavior when system
   time changes. You are strongly encouraged to use the new API instead of the
   old API based on erlang:now/0. erlang:now/0 has been deprecated since it
   will always be a scalability bottleneck.
   For more information see the Time and Time Correction chapter of the
   ERTS User's Guide. Here is a link
   http://www.erlang.org/documentation/doc-7.0-rc2/erts-7.0/doc/html/time_correction.html
   - erts: Beside the API changes and time warp modes a lot of scalability
   and performance improvements regarding time management has been made.
   Examples are:
      - scheduler specific timer wheels,
      - scheduler specific BIF timer management,
      - parallel retrieval of monotonic time and system time on OS:es that
      support it.
   - erts: The previously introduced "eager check I/O" feature is now
   enabled by default.
   - erts/compiler: enhanced support for maps. Big maps new uses a HAMT
   (Hash Array Mapped Trie) representation internally which makes them more
   efficient. There is now also support for variables as map keys.
   - ssl: Remove default support for SSL-3.0 and added padding check for
   TLS-1.0 due to the Poodle vulnerability.
   - ssl: Remove default support for RC4 cipher suites, as they are
   consider too weak.
   - stdlib: Allow maps for supervisor flags and child specs


You can find the Release Notes with more detailed info at

  http://www.erlang.org/download/OTP-18.0-rc2.README

You find the source code at github.com in the official Erlang repository.

Git tag OTP-18.0-rc2

https://github.com/erlang/otp/tree/OTP-18.0-rc2

You can also read the documentation on-line here:
(see the Release Notes mentioned above for release notes which
are not updated in the doc, but the new functionality is)

http://www.erlang.org/documentation/doc-7.0-rc2/doc/

We also want to thank those that sent us patches, suggestions and bug
reports.

The Erlang/OTP Team at Ericsson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From dmkolesnikov@REDACTED  Wed May 13 14:02:43 2015
From: dmkolesnikov@REDACTED (Dmitry Kolesnikov)
Date: Wed, 13 May 2015 15:02:43 +0300
Subject: [erlang-questions] HEART
In-Reply-To: 
References: 
 
 
 
 <1431449900.2713951.266897849.0D1D4324@webmail.messagingengine.com>
 <9B082D3C-0E45-4D21-B914-4AD4CECE03C3@widetag.com>
 <1431452073.2721867.266937257.35C09C47@webmail.messagingengine.com>
 
 
 
Message-ID: <89770C31-F1B8-4894-907C-9C6611172DC2@gmail.com>

Hi,

You are using ?start? command when you start node manually but upstart script uses ?foreground?. 
Try to use start in both places. I think foreground bypassed pipe creation.    

- Dmitry

> On 13 May 2015, at 14:41, Roberto Ostinelli  wrote:
> 
> Hi,
> Still experiencing weirdnesses though.
> 
> My upstart script is:
> 
> script
>     export HOME=/root
>     cd /usr/local/myapp
>     exec bin/myapp foreground > /dev/null 2>&1
> end script
> 
> 
> When I start to attach to the app's node, I get:
> 
> $ /usr/local/myapp/bin/myapp attach
> pong
> Can't access pipe directory /tmp//usr/local/myapp/: No such file or directory
> 
> 
> However, if I start my app manually:
> 
> $ /usr/local/myapp/bin/myapp start
> 
> Then everything works fine:
> 
> $ /usr/local/cometa/bin/cometa attach
> pong
> Attaching to /tmp//usr/local/myapp/erlang.pipe.1 (^D to exit)
> 
> (myapp@REDACTED)1>
> 
> 
> Can some kind soul explain to me what is going on?
> 
> Thank you,
> r.
> 
> 
> 
> 
> 
> 
> 
> On Tue, May 12, 2015 at 8:44 PM, Roger Lipscombe > wrote:
> On 12 May 2015 at 18:45, Roberto Ostinelli
> > wrote:
> > Right. Unfortunately I can't find a way to oass this pid to the original script that starts it (using upstart).
> 
> We use relx-generated releases with upstart. Simply run "bin/myapp
> foreground" from the upstart script.
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From essen@REDACTED  Wed May 13 14:51:07 2015
From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=)
Date: Wed, 13 May 2015 15:51:07 +0300
Subject: [erlang-questions] Erlang/OTP 18.0-rc2 is available for testing
In-Reply-To: 
References: 
Message-ID: <555348BB.8090303@ninenines.eu>

Everything looks fine after running the test suites.

Good job everyone involved!

On 05/13/2015 02:48 PM, Kenneth Lundin wrote:
> Erlang/OTP 18.0-rc2 is available for testing.
>
> This is the second and last release candidate before the final OTP 18.0
> product release in June 2015.
>
> Between the 18.0 rc1 and 18.0 rc 2 and the final release there will be
> new updates of
> the master branch with corrections and minor new features.
> Occasionally there might be new tags which we in that
> case will communicate and ask you to test.
>
> Erlang/OTP 18.0 is a new major release with new features, quite a few
> (characteristics) improvements, as well as a few incompatibilities.
>
> See the Release Notes and the documentation for more details.
>
> We would like to ask you to build and test this release candidate and
> send us
> your feedback as soon as possible, so that we can make the necessary
> corrections before OTP 18.0.
>
> The release contains many changes; thus, some unexpected incompatibilities
> or issues may have slipped through our tests.
> Please try to build and run your current products/applications and let us
> know about any problems.
>
> *IMPORTANT INFO when building your own code with this OTP release
> *
> Since erlang:now is deprecated your build might stop if you are using
> "warnings as errors".
> To let the build through you can turn of warnings for deprecated functions
> by setting an environment variable like this:
>    export ERL_COMPILER_OPTIONS=nowarn_deprecated_function
>
> Some highlights of the release are:
>
>   * dialyzer: The -dialyzer() attribute can be used for suppressing
>     warnings
>     in a module by specifying functions or warning options.
>     It can also be used for requesting warnings in a module.
>   * erts: The time functionality has been extended. This includes a new
>     API for
>     time, as well as "time warp" modes which alters the behavior when
>     system time changes. You are strongly encouraged to use the new API
>     instead of the old API based on erlang:now/0. erlang:now/0 has been
>     deprecated since it will always be a scalability bottleneck.
>     For more information see the Time and Time Correction chapter of the
>     ERTS User's Guide. Here is a link
>     http://www.erlang.org/documentation/doc-7.0-rc2/erts-7.0/doc/html/time_correction.html
>
>   * erts: Beside the API changes and time warp modes a lot of
>     scalability and performance improvements regarding time management
>     has been made. Examples are:
>       o scheduler specific timer wheels,
>       o scheduler specific BIF timer management,
>       o parallel retrieval of monotonic time and system time on OS:es
>         that support it.
>   * erts: The previously introduced "eager check I/O" feature is now
>     enabled by default.
>   * erts/compiler: enhanced support for maps. Big maps new uses a HAMT
>     (Hash Array Mapped Trie) representation internally which makes them
>     more efficient. There is now also support for variables as map keys.
>   * ssl: Remove default support for SSL-3.0 and added padding check for
>     TLS-1.0 due to the Poodle vulnerability.
>   * ssl: Remove default support for RC4 cipher suites, as they are
>     consider too weak.
>   * stdlib: Allow maps for supervisor flags and child specs
>
>
> You can find the Release Notes with more detailed info at
>
> http://www.erlang.org/download/OTP-18.0-rc2.README
>
> You find the source code at github.com  in the
> official Erlang repository.
>
> Git tag OTP-18.0-rc2
>
> https://github.com/erlang/otp/tree/OTP-18.0-rc2
>
> You can also read the documentation on-line here:
> (see the Release Notes mentioned above for release notes which
> are not updated in the doc, but the new functionality is)
>
> http://www.erlang.org/documentation/doc-7.0-rc2/doc/
>
> We also want to thank those that sent us patches, suggestions and bug
> reports.
>
> The Erlang/OTP Team at Ericsson
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>

-- 
Lo?c Hoguin
http://ninenines.eu


From jesper.louis.andersen@REDACTED  Wed May 13 15:04:34 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Wed, 13 May 2015 15:04:34 +0200
Subject: [erlang-questions] Erlang/OTP 18.0-rc2 is available for testing
In-Reply-To: 
References: 
Message-ID: 

I'll just attach this here, but on my machine the BEAM emulator segfaults
under the build if passed the --enable-dirty-schedulers option:

git checkout -b t OTP-18.0-rc2
git clean -dfxq
./otp_build autoconf
./configure --enable-dirty-schedulers --prefix=/usr/local/stow/$(git
describe)
make -j 10

I had expected this to complete and give my an 'erl' with dirty schedulers
enabled, but once it gets into 'erlc' compiling it's first binary, the
command segfaults. Not passing --enable-dirty-schedulers works.

Is this confirmable by anyone else, just to rule out my machine from the
loop?


On Wed, May 13, 2015 at 1:48 PM, Kenneth Lundin  wrote:

> Erlang/OTP 18.0-rc2 is available for testing.
>
> This is the second and last release candidate before the final OTP 18.0
> product release in June 2015.
>
> Between the 18.0 rc1 and 18.0 rc 2 and the final release there will be new
> updates of
> the master branch with corrections and minor new features.
> Occasionally there might be new tags which we in that
> case will communicate and ask you to test.
>
> Erlang/OTP 18.0 is a new major release with new features, quite a few
> (characteristics) improvements, as well as a few incompatibilities.
>
> See the Release Notes and the documentation for more details.
>
> We would like to ask you to build and test this release candidate and send
> us
> your feedback as soon as possible, so that we can make the necessary
> corrections before OTP 18.0.
>
> The release contains many changes; thus, some unexpected incompatibilities
> or issues may have slipped through our tests.
> Please try to build and run your current products/applications and let us
> know about any problems.
>
>
> *IMPORTANT INFO when building your own code with this OTP release*
> Since erlang:now is deprecated your build might stop if you are using
> "warnings as errors".
> To let the build through you can turn of warnings for deprecated functions
> by setting an environment variable like this:
>   export ERL_COMPILER_OPTIONS=nowarn_deprecated_function
>
> Some highlights of the release are:
>
>    - dialyzer: The -dialyzer() attribute can be used for suppressing
>    warnings
>    in a module by specifying functions or warning options.
>    It can also be used for requesting warnings in a module.
>    - erts: The time functionality has been extended. This includes a new
>    API for
>    time, as well as "time warp" modes which alters the behavior when
>    system time changes. You are strongly encouraged to use the new API instead
>    of the old API based on erlang:now/0. erlang:now/0 has been deprecated
>    since it will always be a scalability bottleneck.
>    For more information see the Time and Time Correction chapter of the
>    ERTS User's Guide. Here is a link
>    http://www.erlang.org/documentation/doc-7.0-rc2/erts-7.0/doc/html/time_correction.html
>    - erts: Beside the API changes and time warp modes a lot of
>    scalability and performance improvements regarding time management has been
>    made. Examples are:
>       - scheduler specific timer wheels,
>       - scheduler specific BIF timer management,
>       - parallel retrieval of monotonic time and system time on OS:es
>       that support it.
>    - erts: The previously introduced "eager check I/O" feature is now
>    enabled by default.
>    - erts/compiler: enhanced support for maps. Big maps new uses a HAMT
>    (Hash Array Mapped Trie) representation internally which makes them more
>    efficient. There is now also support for variables as map keys.
>    - ssl: Remove default support for SSL-3.0 and added padding check for
>    TLS-1.0 due to the Poodle vulnerability.
>    - ssl: Remove default support for RC4 cipher suites, as they are
>    consider too weak.
>    - stdlib: Allow maps for supervisor flags and child specs
>
>
> You can find the Release Notes with more detailed info at
>
>   http://www.erlang.org/download/OTP-18.0-rc2.README
>
> You find the source code at github.com in the official Erlang repository.
>
> Git tag OTP-18.0-rc2
>
> https://github.com/erlang/otp/tree/OTP-18.0-rc2
>
> You can also read the documentation on-line here:
> (see the Release Notes mentioned above for release notes which
> are not updated in the doc, but the new functionality is)
>
> http://www.erlang.org/documentation/doc-7.0-rc2/doc/
>
> We also want to thank those that sent us patches, suggestions and bug
> reports.
>
> The Erlang/OTP Team at Ericsson
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jesper.louis.andersen@REDACTED  Wed May 13 15:06:13 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Wed, 13 May 2015 15:06:13 +0200
Subject: [erlang-questions] Erlang/OTP 18.0-rc2 is available for testing
In-Reply-To: 
References: 
 
Message-ID: 

And edited to add, I know the version given by sha ce96ab6 compiled
(OTP-18.0-rc1-503-gce96ab6).


On Wed, May 13, 2015 at 3:04 PM, Jesper Louis Andersen <
jesper.louis.andersen@REDACTED> wrote:

> I'll just attach this here, but on my machine the BEAM emulator segfaults
> under the build if passed the --enable-dirty-schedulers option:
>
> git checkout -b t OTP-18.0-rc2
> git clean -dfxq
> ./otp_build autoconf
> ./configure --enable-dirty-schedulers --prefix=/usr/local/stow/$(git
> describe)
> make -j 10
>
> I had expected this to complete and give my an 'erl' with dirty schedulers
> enabled, but once it gets into 'erlc' compiling it's first binary, the
> command segfaults. Not passing --enable-dirty-schedulers works.
>
> Is this confirmable by anyone else, just to rule out my machine from the
> loop?
>
>
> On Wed, May 13, 2015 at 1:48 PM, Kenneth Lundin 
> wrote:
>
>> Erlang/OTP 18.0-rc2 is available for testing.
>>
>> This is the second and last release candidate before the final OTP 18.0
>> product release in June 2015.
>>
>> Between the 18.0 rc1 and 18.0 rc 2 and the final release there will be
>> new updates of
>> the master branch with corrections and minor new features.
>> Occasionally there might be new tags which we in that
>> case will communicate and ask you to test.
>>
>> Erlang/OTP 18.0 is a new major release with new features, quite a few
>> (characteristics) improvements, as well as a few incompatibilities.
>>
>> See the Release Notes and the documentation for more details.
>>
>> We would like to ask you to build and test this release candidate and
>> send us
>> your feedback as soon as possible, so that we can make the necessary
>> corrections before OTP 18.0.
>>
>> The release contains many changes; thus, some unexpected
>> incompatibilities
>> or issues may have slipped through our tests.
>> Please try to build and run your current products/applications and let us
>> know about any problems.
>>
>>
>> *IMPORTANT INFO when building your own code with this OTP release*
>> Since erlang:now is deprecated your build might stop if you are using
>> "warnings as errors".
>> To let the build through you can turn of warnings for deprecated functions
>> by setting an environment variable like this:
>>   export ERL_COMPILER_OPTIONS=nowarn_deprecated_function
>>
>> Some highlights of the release are:
>>
>>    - dialyzer: The -dialyzer() attribute can be used for suppressing
>>    warnings
>>    in a module by specifying functions or warning options.
>>    It can also be used for requesting warnings in a module.
>>    - erts: The time functionality has been extended. This includes a new
>>    API for
>>    time, as well as "time warp" modes which alters the behavior when
>>    system time changes. You are strongly encouraged to use the new API instead
>>    of the old API based on erlang:now/0. erlang:now/0 has been deprecated
>>    since it will always be a scalability bottleneck.
>>    For more information see the Time and Time Correction chapter of the
>>    ERTS User's Guide. Here is a link
>>    http://www.erlang.org/documentation/doc-7.0-rc2/erts-7.0/doc/html/time_correction.html
>>    - erts: Beside the API changes and time warp modes a lot of
>>    scalability and performance improvements regarding time management has been
>>    made. Examples are:
>>       - scheduler specific timer wheels,
>>       - scheduler specific BIF timer management,
>>       - parallel retrieval of monotonic time and system time on OS:es
>>       that support it.
>>    - erts: The previously introduced "eager check I/O" feature is now
>>    enabled by default.
>>    - erts/compiler: enhanced support for maps. Big maps new uses a HAMT
>>    (Hash Array Mapped Trie) representation internally which makes them more
>>    efficient. There is now also support for variables as map keys.
>>    - ssl: Remove default support for SSL-3.0 and added padding check for
>>    TLS-1.0 due to the Poodle vulnerability.
>>    - ssl: Remove default support for RC4 cipher suites, as they are
>>    consider too weak.
>>    - stdlib: Allow maps for supervisor flags and child specs
>>
>>
>> You can find the Release Notes with more detailed info at
>>
>>   http://www.erlang.org/download/OTP-18.0-rc2.README
>>
>> You find the source code at github.com in the official Erlang repository.
>>
>> Git tag OTP-18.0-rc2
>>
>> https://github.com/erlang/otp/tree/OTP-18.0-rc2
>>
>> You can also read the documentation on-line here:
>> (see the Release Notes mentioned above for release notes which
>> are not updated in the doc, but the new functionality is)
>>
>> http://www.erlang.org/documentation/doc-7.0-rc2/doc/
>>
>> We also want to thank those that sent us patches, suggestions and bug
>> reports.
>>
>> The Erlang/OTP Team at Ericsson
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
>
> --
> J.
>



-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From vinoski@REDACTED  Wed May 13 15:24:22 2015
From: vinoski@REDACTED (Steve Vinoski)
Date: Wed, 13 May 2015 09:24:22 -0400
Subject: [erlang-questions] Erlang/OTP 18.0-rc2 is available for testing
In-Reply-To: 
References: 
 
 
Message-ID: 

Hi Jesper, I see this too. It must be due to a very recent change, as I've
been successfully building and running quite a bit recently from master
with dirty schedulers enabled.

It's got something to do with timer wheel:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x1113 of process 16360]
0x000000010017480c in erts_check_next_timeout_time (esdp=0x10370a440) at
beam/time.c:280
280    if (tiw->true_next_timeout_time)
(gdb) bt
#0  0x000000010017480c in erts_check_next_timeout_time (esdp=0x10370a440)
at beam/time.c:280
#1  0x000000010014c5bb in scheduler_wait (fcalls=0x105b47b20,
esdp=0x10370a440, rq=0x101e47140) at beam/erl_process.c:2902
#2  0x0000000100145e1a in schedule (p=0x0, calls=0) at
beam/erl_process.c:9340
#3  0x0000000100000e6e in process_main () at beam/beam_emu.c:1253
#4  0x0000000100142428 in sched_dirty_cpu_thread_func (vesdp=0x10370a440)
at beam/erl_process.c:7946
#5  0x000000010033e7a1 in thr_wrapper (vtwd=0x7fff5fbfee88) at
pthread/ethread.c:113
#6  0x00007fff8eae3268 in _pthread_body () from
/usr/lib/system/libsystem_pthread.dylib
#7  0x00007fff8eae31e5 in _pthread_start () from
/usr/lib/system/libsystem_pthread.dylib
#8  0x00007fff8eae141d in thread_start () from
/usr/lib/system/libsystem_pthread.dylib
#9  0x0000000000000000 in ?? ()
(gdb) p tiw
$1 = (ErtsTimerWheel *) 0x0

The dirty scheduler threads are restricted in what they can do as compared
to normal schedulers, so my guess is that the dirty scheduler threads
should not be entering this code.

--steve


On Wed, May 13, 2015 at 9:06 AM, Jesper Louis Andersen <
jesper.louis.andersen@REDACTED> wrote:

> And edited to add, I know the version given by sha ce96ab6 compiled
> (OTP-18.0-rc1-503-gce96ab6).
>
>
> On Wed, May 13, 2015 at 3:04 PM, Jesper Louis Andersen <
> jesper.louis.andersen@REDACTED> wrote:
>
>> I'll just attach this here, but on my machine the BEAM emulator segfaults
>> under the build if passed the --enable-dirty-schedulers option:
>>
>> git checkout -b t OTP-18.0-rc2
>> git clean -dfxq
>> ./otp_build autoconf
>> ./configure --enable-dirty-schedulers --prefix=/usr/local/stow/$(git
>> describe)
>> make -j 10
>>
>> I had expected this to complete and give my an 'erl' with dirty
>> schedulers enabled, but once it gets into 'erlc' compiling it's first
>> binary, the command segfaults. Not passing --enable-dirty-schedulers works.
>>
>> Is this confirmable by anyone else, just to rule out my machine from the
>> loop?
>>
>>
>> On Wed, May 13, 2015 at 1:48 PM, Kenneth Lundin 
>> wrote:
>>
>>> Erlang/OTP 18.0-rc2 is available for testing.
>>>
>>> This is the second and last release candidate before the final OTP 18.0
>>> product release in June 2015.
>>>
>>> Between the 18.0 rc1 and 18.0 rc 2 and the final release there will be
>>> new updates of
>>> the master branch with corrections and minor new features.
>>> Occasionally there might be new tags which we in that
>>> case will communicate and ask you to test.
>>>
>>> Erlang/OTP 18.0 is a new major release with new features, quite a few
>>> (characteristics) improvements, as well as a few incompatibilities.
>>>
>>> See the Release Notes and the documentation for more details.
>>>
>>> We would like to ask you to build and test this release candidate and
>>> send us
>>> your feedback as soon as possible, so that we can make the necessary
>>> corrections before OTP 18.0.
>>>
>>> The release contains many changes; thus, some unexpected
>>> incompatibilities
>>> or issues may have slipped through our tests.
>>> Please try to build and run your current products/applications and let
>>> us
>>> know about any problems.
>>>
>>>
>>> *IMPORTANT INFO when building your own code with this OTP release*
>>> Since erlang:now is deprecated your build might stop if you are using
>>> "warnings as errors".
>>> To let the build through you can turn of warnings for deprecated
>>> functions
>>> by setting an environment variable like this:
>>>   export ERL_COMPILER_OPTIONS=nowarn_deprecated_function
>>>
>>> Some highlights of the release are:
>>>
>>>    - dialyzer: The -dialyzer() attribute can be used for suppressing
>>>    warnings
>>>    in a module by specifying functions or warning options.
>>>    It can also be used for requesting warnings in a module.
>>>    - erts: The time functionality has been extended. This includes a
>>>    new API for
>>>    time, as well as "time warp" modes which alters the behavior when
>>>    system time changes. You are strongly encouraged to use the new API instead
>>>    of the old API based on erlang:now/0. erlang:now/0 has been deprecated
>>>    since it will always be a scalability bottleneck.
>>>    For more information see the Time and Time Correction chapter of the
>>>    ERTS User's Guide. Here is a link
>>>    http://www.erlang.org/documentation/doc-7.0-rc2/erts-7.0/doc/html/time_correction.html
>>>    - erts: Beside the API changes and time warp modes a lot of
>>>    scalability and performance improvements regarding time management has been
>>>    made. Examples are:
>>>       - scheduler specific timer wheels,
>>>       - scheduler specific BIF timer management,
>>>       - parallel retrieval of monotonic time and system time on OS:es
>>>       that support it.
>>>    - erts: The previously introduced "eager check I/O" feature is now
>>>    enabled by default.
>>>    - erts/compiler: enhanced support for maps. Big maps new uses a HAMT
>>>    (Hash Array Mapped Trie) representation internally which makes them more
>>>    efficient. There is now also support for variables as map keys.
>>>    - ssl: Remove default support for SSL-3.0 and added padding check
>>>    for TLS-1.0 due to the Poodle vulnerability.
>>>    - ssl: Remove default support for RC4 cipher suites, as they are
>>>    consider too weak.
>>>    - stdlib: Allow maps for supervisor flags and child specs
>>>
>>>
>>> You can find the Release Notes with more detailed info at
>>>
>>>   http://www.erlang.org/download/OTP-18.0-rc2.README
>>>
>>> You find the source code at github.com in the official Erlang
>>> repository.
>>>
>>> Git tag OTP-18.0-rc2
>>>
>>> https://github.com/erlang/otp/tree/OTP-18.0-rc2
>>>
>>> You can also read the documentation on-line here:
>>> (see the Release Notes mentioned above for release notes which
>>> are not updated in the doc, but the new functionality is)
>>>
>>> http://www.erlang.org/documentation/doc-7.0-rc2/doc/
>>>
>>> We also want to thank those that sent us patches, suggestions and bug
>>> reports.
>>>
>>> The Erlang/OTP Team at Ericsson
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>
>>
>> --
>> J.
>>
>
>
>
> --
> J.
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From rickard@REDACTED  Wed May 13 15:32:38 2015
From: rickard@REDACTED (Rickard Green)
Date: Wed, 13 May 2015 15:32:38 +0200
Subject: [erlang-questions] Erlang/OTP 18.0-rc2 is available for testing
In-Reply-To: 
References: 
 
 
 
Message-ID: 

I broke it :-/ I'll unbreak it and post a notification here when it
has been fixed in master. I can not promise exactly when that will
happen, but at least before the actual release.

Regards,
Rickard

On Wed, May 13, 2015 at 3:24 PM, Steve Vinoski  wrote:
> Hi Jesper, I see this too. It must be due to a very recent change, as I've
> been successfully building and running quite a bit recently from master with
> dirty schedulers enabled.
>
> It's got something to do with timer wheel:
>
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x1113 of process 16360]
> 0x000000010017480c in erts_check_next_timeout_time (esdp=0x10370a440) at
> beam/time.c:280
> 280    if (tiw->true_next_timeout_time)
> (gdb) bt
> #0  0x000000010017480c in erts_check_next_timeout_time (esdp=0x10370a440) at
> beam/time.c:280
> #1  0x000000010014c5bb in scheduler_wait (fcalls=0x105b47b20,
> esdp=0x10370a440, rq=0x101e47140) at beam/erl_process.c:2902
> #2  0x0000000100145e1a in schedule (p=0x0, calls=0) at
> beam/erl_process.c:9340
> #3  0x0000000100000e6e in process_main () at beam/beam_emu.c:1253
> #4  0x0000000100142428 in sched_dirty_cpu_thread_func (vesdp=0x10370a440) at
> beam/erl_process.c:7946
> #5  0x000000010033e7a1 in thr_wrapper (vtwd=0x7fff5fbfee88) at
> pthread/ethread.c:113
> #6  0x00007fff8eae3268 in _pthread_body () from
> /usr/lib/system/libsystem_pthread.dylib
> #7  0x00007fff8eae31e5 in _pthread_start () from
> /usr/lib/system/libsystem_pthread.dylib
> #8  0x00007fff8eae141d in thread_start () from
> /usr/lib/system/libsystem_pthread.dylib
> #9  0x0000000000000000 in ?? ()
> (gdb) p tiw
> $1 = (ErtsTimerWheel *) 0x0
>
> The dirty scheduler threads are restricted in what they can do as compared
> to normal schedulers, so my guess is that the dirty scheduler threads should
> not be entering this code.
>
> --steve
>
>
> On Wed, May 13, 2015 at 9:06 AM, Jesper Louis Andersen
>  wrote:
>>
>> And edited to add, I know the version given by sha ce96ab6 compiled
>> (OTP-18.0-rc1-503-gce96ab6).
>>
>>
>> On Wed, May 13, 2015 at 3:04 PM, Jesper Louis Andersen
>>  wrote:
>>>
>>> I'll just attach this here, but on my machine the BEAM emulator segfaults
>>> under the build if passed the --enable-dirty-schedulers option:
>>>
>>> git checkout -b t OTP-18.0-rc2
>>> git clean -dfxq
>>> ./otp_build autoconf
>>> ./configure --enable-dirty-schedulers --prefix=/usr/local/stow/$(git
>>> describe)
>>> make -j 10
>>>
>>> I had expected this to complete and give my an 'erl' with dirty
>>> schedulers enabled, but once it gets into 'erlc' compiling it's first
>>> binary, the command segfaults. Not passing --enable-dirty-schedulers works.
>>>
>>> Is this confirmable by anyone else, just to rule out my machine from the
>>> loop?
>>>
>>>
>>> On Wed, May 13, 2015 at 1:48 PM, Kenneth Lundin 
>>> wrote:
>>>>
>>>> Erlang/OTP 18.0-rc2 is available for testing.
>>>>
>>>> This is the second and last release candidate before the final OTP 18.0
>>>> product release in June 2015.
>>>>
>>>> Between the 18.0 rc1 and 18.0 rc 2 and the final release there will be
>>>> new updates of
>>>> the master branch with corrections and minor new features.
>>>> Occasionally there might be new tags which we in that
>>>> case will communicate and ask you to test.
>>>>
>>>> Erlang/OTP 18.0 is a new major release with new features, quite a few
>>>> (characteristics) improvements, as well as a few incompatibilities.
>>>>
>>>> See the Release Notes and the documentation for more details.
>>>>
>>>> We would like to ask you to build and test this release candidate and
>>>> send us
>>>> your feedback as soon as possible, so that we can make the necessary
>>>> corrections before OTP 18.0.
>>>>
>>>> The release contains many changes; thus, some unexpected
>>>> incompatibilities
>>>> or issues may have slipped through our tests.
>>>> Please try to build and run your current products/applications and let
>>>> us
>>>> know about any problems.
>>>>
>>>> IMPORTANT INFO when building your own code with this OTP release
>>>>
>>>> Since erlang:now is deprecated your build might stop if you are using
>>>> "warnings as errors".
>>>> To let the build through you can turn of warnings for deprecated
>>>> functions
>>>> by setting an environment variable like this:
>>>>   export ERL_COMPILER_OPTIONS=nowarn_deprecated_function
>>>>
>>>> Some highlights of the release are:
>>>>
>>>> dialyzer: The -dialyzer() attribute can be used for suppressing warnings
>>>> in a module by specifying functions or warning options.
>>>> It can also be used for requesting warnings in a module.
>>>> erts: The time functionality has been extended. This includes a new API
>>>> for
>>>> time, as well as "time warp" modes which alters the behavior when system
>>>> time changes. You are strongly encouraged to use the new API instead of the
>>>> old API based on erlang:now/0. erlang:now/0 has been deprecated since it
>>>> will always be a scalability bottleneck.
>>>> For more information see the Time and Time Correction chapter of the
>>>> ERTS User's Guide. Here is a link
>>>> http://www.erlang.org/documentation/doc-7.0-rc2/erts-7.0/doc/html/time_correction.html
>>>> erts: Beside the API changes and time warp modes a lot of scalability
>>>> and performance improvements regarding time management has been made.
>>>> Examples are:
>>>>
>>>> scheduler specific timer wheels,
>>>> scheduler specific BIF timer management,
>>>> parallel retrieval of monotonic time and system time on OS:es that
>>>> support it.
>>>>
>>>> erts: The previously introduced "eager check I/O" feature is now enabled
>>>> by default.
>>>> erts/compiler: enhanced support for maps. Big maps new uses a HAMT (Hash
>>>> Array Mapped Trie) representation internally which makes them more
>>>> efficient. There is now also support for variables as map keys.
>>>> ssl: Remove default support for SSL-3.0 and added padding check for
>>>> TLS-1.0 due to the Poodle vulnerability.
>>>> ssl: Remove default support for RC4 cipher suites, as they are consider
>>>> too weak.
>>>> stdlib: Allow maps for supervisor flags and child specs
>>>>
>>>>
>>>> You can find the Release Notes with more detailed info at
>>>>
>>>>   http://www.erlang.org/download/OTP-18.0-rc2.README
>>>>
>>>> You find the source code at github.com in the official Erlang
>>>> repository.
>>>>
>>>> Git tag OTP-18.0-rc2
>>>>
>>>> https://github.com/erlang/otp/tree/OTP-18.0-rc2
>>>>
>>>> You can also read the documentation on-line here:
>>>> (see the Release Notes mentioned above for release notes which
>>>> are not updated in the doc, but the new functionality is)
>>>>
>>>> http://www.erlang.org/documentation/doc-7.0-rc2/doc/
>>>>
>>>> We also want to thank those that sent us patches, suggestions and bug
>>>> reports.
>>>>
>>>> The Erlang/OTP Team at Ericsson
>>>>
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>
>>>
>>>
>>>
>>> --
>>> J.
>>
>>
>>
>>
>> --
>> J.
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>



-- 
Rickard Green, Erlang/OTP, Ericsson AB


From richard.youngkin@REDACTED  Wed May 13 15:39:26 2015
From: richard.youngkin@REDACTED (Youngkin, Rich)
Date: Wed, 13 May 2015 07:39:26 -0600
Subject: [erlang-questions] The upcoming leap second
Message-ID: 

Hi all,

There's an upcoming leap second on June 30th.  There's a bit of buzz about
how it affects Linux and Java, as well as problems encountered in 2012 [1].
I was wondering if there are any known issues with Erlang. I searched for
"erlang leap second" and a few things popped up [2][3]. The link for [3] is
stale.

Thanks,
Rich

[1] -
http://www.networkworld.com/article/2872578/business-continuity/watch-out-for-an-upcoming-leap-second.html
[2] - http://erlang.org/pipermail/erlang-bugs/2009-April/001254.html
[3] -
http://www.erlang.org/documentation/doc-7.0-rc1/erts-7.0/doc/html/time_correction.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From t@REDACTED  Wed May 13 15:54:53 2015
From: t@REDACTED (Tristan Sloughter)
Date: Wed, 13 May 2015 08:54:53 -0500
Subject: [erlang-questions] HEART
In-Reply-To: 
References: 
 
 
 
 <1431449900.2713951.266897849.0D1D4324@webmail.messagingengine.com>
 <9B082D3C-0E45-4D21-B914-4AD4CECE03C3@widetag.com>
 <1431452073.2721867.266937257.35C09C47@webmail.messagingengine.com>
 
 
 
Message-ID: <1431525293.3002069.267735585.0D9F5F7F@webmail.messagingengine.com>

Use remote_console instead of attach.

--
Tristan Sloughter t@REDACTED



On Wed, May 13, 2015, at 06:41 AM, Roberto Ostinelli wrote:
> Hi, Still experiencing weirdnesses though.
>
> My upstart script is:
>
> script export HOME=/root
> cd /usr/local/myapp exec bin/myapp foreground > /dev/null 2>&1 end
>     script
>
>
> When I start to attach to the app's node, I get:
>
> $ /usr/local/myapp/bin/myapp attach pong Can't access pipe directory
> /tmp//usr/local/myapp/: No such file or directory
>
>
> However, if I start my app manually:
>
> $ /usr/local/myapp/bin/myapp start
>
> Then everything works fine:
>
> $ /usr/local/cometa/bin/cometa attach pong Attaching to
> /tmp//usr/local/myapp/erlang.pipe.1 (^D to exit)
>
> (myapp@REDACTED)1>
>
>
> Can some kind soul explain to me what is going on?
>
> Thank you,
> r.
>
>
>
>
>
>
>
> On Tue, May 12, 2015 at 8:44 PM, Roger Lipscombe
>  wrote:
>> On 12 May 2015 at 18:45, Roberto Ostinelli
>>  wrote:
>>
> Right. Unfortunately I can't find a way to oass this pid to the
> original script that starts it (using upstart).
>>
>> We use relx-generated releases with upstart. Simply run "bin/myapp
>>
foreground" from the upstart script.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roberto@REDACTED  Wed May 13 16:08:32 2015
From: roberto@REDACTED (Roberto Ostinelli)
Date: Wed, 13 May 2015 16:08:32 +0200
Subject: [erlang-questions] HEART
In-Reply-To: <1431525293.3002069.267735585.0D9F5F7F@webmail.messagingengine.com>
References: 
 
 
 
 <1431449900.2713951.266897849.0D1D4324@webmail.messagingengine.com>
 <9B082D3C-0E45-4D21-B914-4AD4CECE03C3@widetag.com>
 <1431452073.2721867.266937257.35C09C47@webmail.messagingengine.com>
 
 
 
 <1431525293.3002069.267735585.0D9F5F7F@webmail.messagingengine.com>
Message-ID: 

Ok,
After a while of experimenting I want to provide some feedback to the great
help that was provided here.
I've come up with a init.d script (currently running on Ubuntu) that
provides me with what I needed.

As a reminder:

   - I have an Erlang 17.4 release generated with rebar.
   - I want to have the release started on system boot.
   - I want to the VM monitored and restarted it if it crashes.


First, ensure that the `-heart` option is used in your `vm.args` file.
Heart will monitor the VM and restart it if needed.

Second, create the file `/etc/init.d/myapp`:


########################################################################

#!/usr/bin/env bash
# myapp daemon
# chkconfig: 345 20 80
# description: myapp daemon
# processname: myapp

NAME=myapp
PROJECT_ROOT_PATH=/usr/local/$NAME
APP_SCRIPT="bin/$NAME"

# export
export HOME=/root

case "$1" in
start)
    printf "%-50s" "Starting $NAME..."

    # start
    cd $PROJECT_ROOT_PATH
    $APP_SCRIPT start > /dev/null 2>&1;

    # wait for pid
    for (( i=0; i<10; ++i )); do
        OUT=`$APP_SCRIPT getpid`;
        if [ $? == 0 ]; then PID=$OUT; break; fi
        sleep 1;
    done

    if [ -z "$PID" ]; then
        printf "%s\n" "Failsd"
    else
        printf "%s\n" "Ok"
    fi
;;
status)
    printf "%-50s" "Checking $NAME..."

    # wait for pid
    cd $PROJECT_ROOT_PATH
    $APP_SCRIPT getpid > /dev/null 2>&1;

    if [ $? != 0 ]; then
        printf "%s\n" "Node is not running!"
    else
        printf "%s\n" "Ok"
    fi
;;
stop)
    printf "%-50s" "Stopping $NAME..."

    # cd and stop
    cd $PROJECT_ROOT_PATH
    $APP_SCRIPT stop > /dev/null 2>&1;

    if [ $? != 0 ]; then
        printf "%s\n" "Node is not running!"
    else
        printf "%s\n" "Ok"
    fi
;;

restart)
    $0 stop
    $0 start
;;

*)
    echo "Usage: $0 {status|start|stop|restart}"
    exit 1
esac


########################################################################

You can use this file as normal services:

```
$ sudo service cometa start
Starting myapp...                                Ok
```


Third, ensure this script is used at boot time:

`sudo update-rc.d myapp defaults`



Side note: you can see that the script waits to exit the start function
until the PID is retrieved from the VM.
This is not strictly necessary, although in this way you can even consider
dumping it into PID files or perform other types of monitoring actions
instead of using HEART.


Hope this helps someone in my same spot.

Best,
r.




On Wed, May 13, 2015 at 3:54 PM, Tristan Sloughter  wrote:

>  Use remote_console instead of attach.
>
> --
> Tristan Sloughter
> t@REDACTED
>
>
>
> On Wed, May 13, 2015, at 06:41 AM, Roberto Ostinelli wrote:
>
> Hi,
> Still experiencing weirdnesses though.
>
> My upstart script is:
>
> script
> export HOME=/root
> cd /usr/local/myapp
>     exec bin/myapp foreground > /dev/null 2>&1
> end script
>
>
> When I start to attach to the app's node, I get:
>
> $ /usr/local/myapp/bin/myapp attach
> pong
> Can't access pipe directory /tmp//usr/local/myapp/: No such file or
> directory
>
>
> However, if I start my app manually:
>
> $ /usr/local/myapp/bin/myapp start
>
> Then everything works fine:
>
> $ /usr/local/cometa/bin/cometa attach
> pong
> Attaching to /tmp//usr/local/myapp/erlang.pipe.1 (^D to exit)
>
> (myapp@REDACTED)1>
>
>
> Can some kind soul explain to me what is going on?
>
> Thank you,
> r.
>
>
>
>
>
>
>
> On Tue, May 12, 2015 at 8:44 PM, Roger Lipscombe 
> wrote:
>
> On 12 May 2015 at 18:45, Roberto Ostinelli
>  wrote:
> > Right. Unfortunately I can't find a way to oass this pid to the original
> script that starts it (using upstart).
>
> We use relx-generated releases with upstart. Simply run "bin/myapp
> foreground" from the upstart script.
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From peppe@REDACTED  Wed May 13 16:09:15 2015
From: peppe@REDACTED (Peter Andersson)
Date: Wed, 13 May 2015 16:09:15 +0200
Subject: [erlang-questions] FYI: current erlang master (soon to be
 18.0-rc2) fails using common test
In-Reply-To: <2148588.3UGZXaZKbP@csr-pc40.zib.de>
References: <2148588.3UGZXaZKbP@csr-pc40.zib.de>
Message-ID: <55535B0B.4040705@erlang.org>

Thanks for the error report!

The bug has been located and fixed (and new tests have been added). I
will push the update to master within the next few days.

Best,
Peter

Ericsson AB, Erlang/OTP

On 2015-05-13 11:51, Nico Kruber wrote:
> I tried "make test" in Scalaris [1]
>
> > git clone https://github.com/scalaris-team/scalaris.git scalaris
> > cd scalaris
> > ./configure && make
> > make test
> ct_run  -name ct@`hostname -s`  -pa `pwd`/ebin `pwd`/test 
> `pwd`/contrib/yaws/ebin `pwd`/contrib/log4erl/ebin -spec 
> test/scalaris.runtests-default.cfg -ct_hooks scalaris_cth -noshell | tee 
> ct.log ; \
>         grep " 0 failed" ct.log
>
>
> Common Test v1.10.1 starting (cwd is /)
>
>
> Common Test: Running make in test directories...
>
> CWD set to: "//ct_run.ct@REDACTED"
>
> TEST INFO: 1 test(s), 62 suite(s)
>
> Testing nico.scalaris: Starting test (with repeated test cases)
> EXIT, reason {badarg,
>                  [{erl_anno,anno_info,
>                       [macro],
>                       [{file,"erl_anno.erl"},{line,426}]},
>                   {erl_anno,location,1,[{file,"erl_anno.erl"},{line,234}]},
>                   {erl_anno,line,1,[{file,"erl_anno.erl"},{line,219}]},
>                   {erl2html2,parse_non_preprocessed_file,3,
>                       [{file,"erl2html2.erl"},{line,155}]},
>                   {erl2html2,parse_non_preprocessed_file,3,
>                       [{file,"erl2html2.erl"},{line,156}]},
>                   {erl2html2,parse_non_preprocessed_file,1,
>                       [{file,"erl2html2.erl"},{line,140}]},
>                   {erl2html2,convert,4,[{file,"erl2html2.erl"},{line,56}]},
>                   {test_server_ctrl,html_convert_modules,1,
>                       [{file,"test_server_ctrl.erl"},{line,1928}]}]}Updating 
> //index.html... done
> Updating //all_runs.html... done
> Test run crashed! This could be an internal error - please report!
>
> {{case_clause,
>      {value,
>          {'EXIT',
>              {badarg,
>                  [{erl_anno,anno_info,
>                       [macro],
>                       [{file,"erl_anno.erl"},{line,426}]},
>                   {erl_anno,location,1,[{file,"erl_anno.erl"},{line,234}]},
>                   {erl_anno,line,1,[{file,"erl_anno.erl"},{line,219}]},
>                   {erl2html2,parse_non_preprocessed_file,3,
>                       [{file,"erl2html2.erl"},{line,155}]},
>                   {erl2html2,parse_non_preprocessed_file,3,
>                       [{file,"erl2html2.erl"},{line,156}]},
>                   {erl2html2,parse_non_preprocessed_file,1,
>                       [{file,"erl2html2.erl"},{line,140}]},
>                   {erl2html2,convert,4,[{file,"erl2html2.erl"},{line,56}]},
>                   {test_server_ctrl,html_convert_modules,1,
>                       [{file,"test_server_ctrl.erl"},{line,1928}]}]}}}},
>  [{ct_run,execute_all_specs,4,[{file,"ct_run.erl"},{line,459}]},
>   {ct_run,script_start1,2,[{file,"ct_run.erl"},{line,376}]}]}
>
>
>
> =ERROR REPORT==== 13-May-2015::11:48:24 ===
> Error in process <0.52.0> on node 'ct@REDACTED' with exit value: 
> {{case_clause,{value,{'EXIT',{badarg,[{erl_anno,anno_info,[macro],
> [{file,"erl_anno.erl"},{line,426}]},{erl_anno,location,1,
> [{file,"erl_anno.erl"},{line,234}]},{erl_anno,line,1,[{file,"erl_anno.erl"},
> {line,219}]},{erl2html2,parse_non_preprocessed_file... 
>
> make: *** [test] Error 1
>
>
> [1] https://github.com/scalaris-team/scalaris
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From daniel.goertzen@REDACTED  Wed May 13 16:49:56 2015
From: daniel.goertzen@REDACTED (Daniel Goertzen)
Date: Wed, 13 May 2015 09:49:56 -0500
Subject: [erlang-questions] [ANN] Ruster-Unsafe and Ruster
Message-ID: 

Since Rust is hitting 1.0.0-release shortly, now is a good time to point
out some Rust NIF libraries I?ve been working on.  ?Rust? is a modern
systems language with a focus on speed, safety and concurrency.
 ?Ruster-Unsafe? is a bare binding of the C NIF API.  ?Ruster? is a higher
level binding that sits atop Ruster-Unsafe.

Ruster-Unsafe is usable today, but Ruster is still under heavy
construction.  It is premature for me to be announcing Ruster, but I
thought I would mention it since I'm already talking about Rust and
Ruster-Unsafe.

Ruster-Unsafe
demo: https://github.com/goertzenator/ruster_unsafe_demo
docs: http://goertzenator.github.io/ruster_unsafe/ruster_unsafe/index.html
source: https://github.com/goertzenator/ruster_unsafe

Ruster: https://github.com/goertzenator/ruster
Rust: http://www.rust-lang.org/

A minimal Ruster-Unsafe NIF module looks like ?


#[macro_use]
extern crate ruster_unsafe;
use ruster_unsafe::*;

nif_init!(b"ruster_unsafe_minimal\0", None, None, None, None,
    nif!(b"just123\0", 0, just123)
);

extern "C" fn just123(_env: *mut ErlNifEnv, _argc: c_int,
    _args: *const ERL_NIF_TERM) -> ERL_NIF_TERM {
unsafe { enif_make_int(env, 123) }
}



Example of the sort of thing I?m aiming for with Ruster ...

// crack the tuple {1, 2, {3, 4}}
let (a,b,(c,d)) = try!(from_term::<(u32,u32,(u32, u32))>(env, term));
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From rickard@REDACTED  Wed May 13 17:17:19 2015
From: rickard@REDACTED (Rickard Green)
Date: Wed, 13 May 2015 17:17:19 +0200
Subject: [erlang-questions] The upcoming leap second
In-Reply-To: 
References: 
Message-ID: 

On Wed, May 13, 2015 at 3:39 PM, Youngkin, Rich
 wrote:
> Hi all,
>
> There's an upcoming leap second on June 30th.  There's a bit of buzz about
> how it affects Linux and Java, as well as problems encountered in 2012 [1].
> I was wondering if there are any known issues with Erlang. I searched for
> "erlang leap second" and a few things popped up [2][3]. The link for [3] is
> stale.
>
> Thanks,
> Rich
>
> [1] -
> http://www.networkworld.com/article/2872578/business-continuity/watch-out-for-an-upcoming-leap-second.html
> [2] - http://erlang.org/pipermail/erlang-bugs/2009-April/001254.html
> [3] -
> http://www.erlang.org/documentation/doc-7.0-rc1/erts-7.0/doc/html/time_correction.html
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>

The Erlang runtime system as such will be able to handle it. This
assuming that the only effect the leap second have on the OS is a
change in system time (such as for example a freeze of time for a
second).

On pre OTP 18 systems there are two scenarios:

- Time correction is enabled: The frequency of the Erlang system time
will temporarily change (with 1%) in order to slowly align Erlang
system time with OS system time.
- Time correction is disabled: Erlang system time immediately follows
OS system time as long as OS system time does not go backwards. If OS
system time goes backwards, Erlang system time will freeze until OS
system time has caught up.

What the total effect will be on your system depends on how your
Erlang code reacts to the above.

On OTP 18 systems there are a few more scenarios depending on
configuration. See
http://www.erlang.org/documentation/doc-7.0-rc2/erts-7.0/doc/html/time_correction.html
(the rc1 version was removed when we published rc2).

Regards,
Rickard

-- 
Rickard Green, Erlang/OTP, Ericsson AB


From jay@REDACTED  Wed May 13 17:27:45 2015
From: jay@REDACTED (Jay Nelson)
Date: Wed, 13 May 2015 08:27:45 -0700
Subject: [erlang-questions] Retrieving "semi-constant" data from a
	function versus Mnesia
In-Reply-To: 
References: 
 
Message-ID: <1EF06091-2413-4A97-B1A6-FCA099A655F7@duomark.com>


> On May 13, 2015, at 12:07 AM, Hynek Vychodil  wrote:
> 
> What prevents you using 
> 
> generic_config:data_one(Config, ...)
> 
> instead of 
> 
> Config_Module:data_one(...)
> 
> It is just one more step of indirection. It can be a viable way in the case when write operations are very rare.
> 
> Hynek
> 

I was writing an OSS library to be reused by others. I employed
erlang?s behaviour mechanism as the ?most principled? approach
to genericity. Using erlang.mk or rebar expects the library to be
fetched and used without modification during the build process.
To make it performant with this approach, I have to have documentation
that instructs the user to mirror the example default, and use a
reserved constant pre-defined module name (hoping there is no conflict
with another module already defined in their application).

I will probably benchmark this one day, but using vbisect is a more
reasonable approach and so far the traditional app.config has not
proven to be a big penalty (maybe 10% slowdown, maybe nothing
significant, I don?t have definitive production numbers).

/aside

Another reply had mentioned knutin/bisect? Bisect requires
fixed length key/value, so while it may be faster (I would expect
it?s possible but I don?t know) it has a larger memory usage and
when including things like URLs as values, that memory waste
could be vast for other values like integers. Vbisect is Kresten
Krab?s inspiration from bisect re-implemented with variable-sized
keys and values. He wanted a generic technique to store data in
Hanoi DB which is his erlang implementation of Level DB, a log
merge KV store. Performance testing and PropEr tests give results
that are similar to ets with 10K or so keys in a single dictionary,
although it will slow down with more data as it is a sorted tree of KV.
My fork adds more structure to the code, PropEr testing and some
additional API calls to mirror dict and friends of erlang, so it can be
used as a swap out alternative.

jay



From jay@REDACTED  Wed May 13 18:59:57 2015
From: jay@REDACTED (Jay Nelson)
Date: Wed, 13 May 2015 09:59:57 -0700
Subject: [erlang-questions] Retrieving "semi-constant" data from a
	function versus Mnesia
In-Reply-To: 
References: 
 
Message-ID: <003AEB56-9546-4861-9FA9-84E2BC64402B@duomark.com>


> On May 13, 2015, at 12:07 AM, Hynek Vychodil  wrote:
> 
> What prevents you using 
> 
> generic_config:data_one(Config, ...)
> 
> instead of 
> 
> Config_Module:data_one(...)
> 
> It is just one more step of indirection. It can be a viable way in the case when write operations are very rare.
> 
> Hynek

You?ve inspired me to use a Macro and a build hook to generate
a single line include file with the user-provided module name.
Hopefully I can make it simple enough for 3rd party users to
customize.

Thanks!
jay



From mjtruog@REDACTED  Wed May 13 19:29:15 2015
From: mjtruog@REDACTED (Michael Truog)
Date: Wed, 13 May 2015 10:29:15 -0700
Subject: [erlang-questions] Erlang/OTP 18.0-rc2 is available for testing
In-Reply-To: 
References: 
 
 
 
 
Message-ID: <555389EB.5070102@gmail.com>

On 05/13/2015 06:32 AM, Rickard Green wrote:
> I broke it :-/ I'll unbreak it and post a notification here when it
> has been fixed in master. I can not promise exactly when that will
> happen, but at least before the actual release.
When 18.0-rc1 was released the timer wheel size per scheduler was larger (http://erlang.org/pipermail/erlang-bugs/2015-April/004907.html) and the size was decreased due to problems related to the larger size. I was wondering if there are any plans to add an erts option that can be adjusted from the erl command line, so that testing can determine if a larger timer wheel size per scheduler is beneficial.

Thanks,
Michael
>
> Regards,
> Rickard
>
> On Wed, May 13, 2015 at 3:24 PM, Steve Vinoski  wrote:
>> Hi Jesper, I see this too. It must be due to a very recent change, as I've
>> been successfully building and running quite a bit recently from master with
>> dirty schedulers enabled.
>>
>> It's got something to do with timer wheel:
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> [Switching to Thread 0x1113 of process 16360]
>> 0x000000010017480c in erts_check_next_timeout_time (esdp=0x10370a440) at
>> beam/time.c:280
>> 280    if (tiw->true_next_timeout_time)
>> (gdb) bt
>> #0  0x000000010017480c in erts_check_next_timeout_time (esdp=0x10370a440) at
>> beam/time.c:280
>> #1  0x000000010014c5bb in scheduler_wait (fcalls=0x105b47b20,
>> esdp=0x10370a440, rq=0x101e47140) at beam/erl_process.c:2902
>> #2  0x0000000100145e1a in schedule (p=0x0, calls=0) at
>> beam/erl_process.c:9340
>> #3  0x0000000100000e6e in process_main () at beam/beam_emu.c:1253
>> #4  0x0000000100142428 in sched_dirty_cpu_thread_func (vesdp=0x10370a440) at
>> beam/erl_process.c:7946
>> #5  0x000000010033e7a1 in thr_wrapper (vtwd=0x7fff5fbfee88) at
>> pthread/ethread.c:113
>> #6  0x00007fff8eae3268 in _pthread_body () from
>> /usr/lib/system/libsystem_pthread.dylib
>> #7  0x00007fff8eae31e5 in _pthread_start () from
>> /usr/lib/system/libsystem_pthread.dylib
>> #8  0x00007fff8eae141d in thread_start () from
>> /usr/lib/system/libsystem_pthread.dylib
>> #9  0x0000000000000000 in ?? ()
>> (gdb) p tiw
>> $1 = (ErtsTimerWheel *) 0x0
>>
>> The dirty scheduler threads are restricted in what they can do as compared
>> to normal schedulers, so my guess is that the dirty scheduler threads should
>> not be entering this code.
>>
>> --steve
>>
>>
>> On Wed, May 13, 2015 at 9:06 AM, Jesper Louis Andersen
>>  wrote:
>>> And edited to add, I know the version given by sha ce96ab6 compiled
>>> (OTP-18.0-rc1-503-gce96ab6).
>>>
>>>
>>> On Wed, May 13, 2015 at 3:04 PM, Jesper Louis Andersen
>>>  wrote:
>>>> I'll just attach this here, but on my machine the BEAM emulator segfaults
>>>> under the build if passed the --enable-dirty-schedulers option:
>>>>
>>>> git checkout -b t OTP-18.0-rc2
>>>> git clean -dfxq
>>>> ./otp_build autoconf
>>>> ./configure --enable-dirty-schedulers --prefix=/usr/local/stow/$(git
>>>> describe)
>>>> make -j 10
>>>>
>>>> I had expected this to complete and give my an 'erl' with dirty
>>>> schedulers enabled, but once it gets into 'erlc' compiling it's first
>>>> binary, the command segfaults. Not passing --enable-dirty-schedulers works.
>>>>
>>>> Is this confirmable by anyone else, just to rule out my machine from the
>>>> loop?
>>>>
>>>>
>>>> On Wed, May 13, 2015 at 1:48 PM, Kenneth Lundin 
>>>> wrote:
>>>>> Erlang/OTP 18.0-rc2 is available for testing.
>>>>>
>>>>> This is the second and last release candidate before the final OTP 18.0
>>>>> product release in June 2015.
>>>>>
>>>>> Between the 18.0 rc1 and 18.0 rc 2 and the final release there will be
>>>>> new updates of
>>>>> the master branch with corrections and minor new features.
>>>>> Occasionally there might be new tags which we in that
>>>>> case will communicate and ask you to test.
>>>>>
>>>>> Erlang/OTP 18.0 is a new major release with new features, quite a few
>>>>> (characteristics) improvements, as well as a few incompatibilities.
>>>>>
>>>>> See the Release Notes and the documentation for more details.
>>>>>
>>>>> We would like to ask you to build and test this release candidate and
>>>>> send us
>>>>> your feedback as soon as possible, so that we can make the necessary
>>>>> corrections before OTP 18.0.
>>>>>
>>>>> The release contains many changes; thus, some unexpected
>>>>> incompatibilities
>>>>> or issues may have slipped through our tests.
>>>>> Please try to build and run your current products/applications and let
>>>>> us
>>>>> know about any problems.
>>>>>
>>>>> IMPORTANT INFO when building your own code with this OTP release
>>>>>
>>>>> Since erlang:now is deprecated your build might stop if you are using
>>>>> "warnings as errors".
>>>>> To let the build through you can turn of warnings for deprecated
>>>>> functions
>>>>> by setting an environment variable like this:
>>>>>    export ERL_COMPILER_OPTIONS=nowarn_deprecated_function
>>>>>
>>>>> Some highlights of the release are:
>>>>>
>>>>> dialyzer: The -dialyzer() attribute can be used for suppressing warnings
>>>>> in a module by specifying functions or warning options.
>>>>> It can also be used for requesting warnings in a module.
>>>>> erts: The time functionality has been extended. This includes a new API
>>>>> for
>>>>> time, as well as "time warp" modes which alters the behavior when system
>>>>> time changes. You are strongly encouraged to use the new API instead of the
>>>>> old API based on erlang:now/0. erlang:now/0 has been deprecated since it
>>>>> will always be a scalability bottleneck.
>>>>> For more information see the Time and Time Correction chapter of the
>>>>> ERTS User's Guide. Here is a link
>>>>> http://www.erlang.org/documentation/doc-7.0-rc2/erts-7.0/doc/html/time_correction.html
>>>>> erts: Beside the API changes and time warp modes a lot of scalability
>>>>> and performance improvements regarding time management has been made.
>>>>> Examples are:
>>>>>
>>>>> scheduler specific timer wheels,
>>>>> scheduler specific BIF timer management,
>>>>> parallel retrieval of monotonic time and system time on OS:es that
>>>>> support it.
>>>>>
>>>>> erts: The previously introduced "eager check I/O" feature is now enabled
>>>>> by default.
>>>>> erts/compiler: enhanced support for maps. Big maps new uses a HAMT (Hash
>>>>> Array Mapped Trie) representation internally which makes them more
>>>>> efficient. There is now also support for variables as map keys.
>>>>> ssl: Remove default support for SSL-3.0 and added padding check for
>>>>> TLS-1.0 due to the Poodle vulnerability.
>>>>> ssl: Remove default support for RC4 cipher suites, as they are consider
>>>>> too weak.
>>>>> stdlib: Allow maps for supervisor flags and child specs
>>>>>
>>>>>
>>>>> You can find the Release Notes with more detailed info at
>>>>>
>>>>>    http://www.erlang.org/download/OTP-18.0-rc2.README
>>>>>
>>>>> You find the source code at github.com in the official Erlang
>>>>> repository.
>>>>>
>>>>> Git tag OTP-18.0-rc2
>>>>>
>>>>> https://github.com/erlang/otp/tree/OTP-18.0-rc2
>>>>>
>>>>> You can also read the documentation on-line here:
>>>>> (see the Release Notes mentioned above for release notes which
>>>>> are not updated in the doc, but the new functionality is)
>>>>>
>>>>> http://www.erlang.org/documentation/doc-7.0-rc2/doc/
>>>>>
>>>>> We also want to thank those that sent us patches, suggestions and bug
>>>>> reports.
>>>>>
>>>>> The Erlang/OTP Team at Ericsson
>>>>>
>>>>> _______________________________________________
>>>>> erlang-questions mailing list
>>>>> erlang-questions@REDACTED
>>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>>
>>>>
>>>>
>>>> --
>>>> J.
>>>
>>>
>>>
>>> --
>>> J.
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
>



From elbrujohalcon@REDACTED  Wed May 13 20:52:35 2015
From: elbrujohalcon@REDACTED (Fernando 'Brujo' Benavides)
Date: Wed, 13 May 2015 15:52:35 -0300
Subject: [erlang-questions] [ANN] ErlangBA CFP
Message-ID: <11BD9639-5B3B-4B2C-8F23-B03FD1D870E5@inaka.net>

Hi everybody,
Just a reminder that the Call for Presentations for ErlangBA 2015  closes in 2 days. Send us your talk proposal through this form .
We'll be receiving proposals until May 15th. We welcome remote speakers that want to share their talks with us using google-hangout, skype or other similar technology.
Cheers!
 

Fernando "Brujo" Benavides
about.me/elbrujohalcon

  				
 



-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jay@REDACTED  Thu May 14 00:58:29 2015
From: jay@REDACTED (Jay Nelson)
Date: Wed, 13 May 2015 15:58:29 -0700
Subject: [erlang-questions] Retrieving "semi-constant" data from a
	function versus Mnesia
Message-ID: <8FF4E9C9-2FE6-4788-B83B-49E2D9CD2DA4@duomark.com>

ROK wrote:

> One of them is or was erts_find_export_entry(module, function, arity),
> surprise surprise, which looks the triple up in a hash table, so it?s
> fairly clear where the time is going.

It?s funny because I recall from the CLOS "Art of the Metaobject Protocol?
that this was a high-speed solution to the method dispatch problem. The
memoization was bragging rights and what made it feasible.

> While I think this kind of call _could_ be faster, I suspect
> that they are fast _enough_ given their rarity and the other
> things going on in a program.

The OTP baseline is not reflective of application developers and their
expected frequent use of behaviours. These always use some form
of Module:function(?).  Or maybe the few places in the OTP baseline
are invoked frequently from a gen_* during normal operation (as you
said, there is a difference between static and dynamic analysis). But as
you also say, if it were really a _practical_ problem the complaints would
cause optimization patches.

jay



From ok@REDACTED  Thu May 14 04:24:47 2015
From: ok@REDACTED (Richard A. O'Keefe)
Date: Thu, 14 May 2015 14:24:47 +1200
Subject: [erlang-questions] Retrieving "semi-constant" data from a
	function versus Mnesia
In-Reply-To: <8FF4E9C9-2FE6-4788-B83B-49E2D9CD2DA4@duomark.com>
References: <8FF4E9C9-2FE6-4788-B83B-49E2D9CD2DA4@duomark.com>
Message-ID: 


On 14/05/2015, at 10:58 am, Jay Nelson  wrote:

> ROK wrote:
> 
>> One of them is or was erts_find_export_entry(module, function, arity),
>> surprise surprise, which looks the triple up in a hash table, so it?s
>> fairly clear where the time is going.
> 
> It?s funny because I recall from the CLOS "Art of the Metaobject Protocol?
> that this was a high-speed solution to the method dispatch problem. The
> memoization was bragging rights and what made it feasible.

Memoization in dynamic function calling is old technology.
As I recall, Smalltalk-80 used a cache for dynamic dispatch,
and later implementations used distributed "polymorphic inline
caches".

Simple inline caching would turn

    Module:func(E1, ..., En)
into
    static atomic { mod, ptr } cache;
    atomic {
        if (Module == cache.mod) {
            p = cache.ptr;
        } else {
            p = lookup(Module, func, n);
            cache.mod = Module;
            cache.ptr = p;
        }
    }
    (*p)(E1, ..., En);

One form of polymorphic inline caching would yield

    static atomic { mod, ptr } cache[N];
    q = &cache[hash(Module)%N];
    atomic {
        if (Module = q->mod) {
            p = q->ptr;
        } else {
            p = lookup(Module, func, n);
            q->mod = Module;
            q->ptr = p;
        }
    }
    (*p)(E1, ..., En);

This is typically implemented so that the cache can grow.

Doing this in a lock-free way is a bit tricky, but possible.

My actual point was that since a dynamic call in Erlang
involves at least two C function calls in addition to the
intended Erlang call, 6 times slower than a direct call
is not unreasonable.

Oh, that's emulated.  I don't know what HiPE does with
dynamic function calls, but it *probably* makes normal
calls faster without touching dynamic ones much.



From jesper.louis.andersen@REDACTED  Thu May 14 10:10:32 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Thu, 14 May 2015 10:10:32 +0200
Subject: [erlang-questions] The upcoming leap second
In-Reply-To: 
References: 
Message-ID: 

On Wed, May 13, 2015 at 3:39 PM, Youngkin, Rich <
richard.youngkin@REDACTED> wrote:

> here's an upcoming leap second on June 30th.  There's a bit of buzz about
> how it affects Linux and Java, as well as problems encountered in 2012 [1].
>

In addition to what Rickard wrote:

The two major problems to look out for is repetition and precision.
Repetition happens because the POSIX clock doesn't understand leap seconds,
so it repeats a second. If UTC is 58, 59, 60, 00, ... then POSIX will be N,
N+1, N+2, N+2, ... Now, if you use the equivalent of `os:timestamp()` (Pre
18) in the 60 and 00 seconds, then you may get the wrong order. Say you
call os timestamp twice:

TS1 = os:timestamp(),
...
TS2 = os:timestamp(),

You expect TS1 < TS2, but when time repeats, you may get fractions of a
second and suddenly this invariant breaks. For example if TS1 = {X, Y,
700000} and TS2 = {X, Y, 200000}. It leads to all kinds of trouble if you
rely on the time ordering in your system, and such errors can sometimes
cascade through subsystems creating an avalanche of failure ultimately
bringing the system down.

In Pre 18 systems, erlang:now() performs what is called "smearing"
nowadays, so it is guaranteed to be monotonic and unique. This means the
above repetition problem doesn't happen. From 18.x and onwards, the new
Time API gives you far more insight in what happens to the time in the
system, so you are able to program your own solution, which is correct for
your problem. Also note that Google altered their NTPd's to perform
smearing systemwide for clusters where they knew it was not a problem.

The other problem is precision. Some NTP daemons can't cope with leap
seconds, so when one happens, they are "kicked" and loses time precision.
Smearing also alters the clock speed, so 1000ms could suddenly be 1010ms or
1001ms in your system. For some systems, where high-resolution timing is a
necessity, this is trouble. Air Traffic control needs high precision
because planes move 300m in 1 second. The same is true for high speed
trains. Manufacturing plants some times needs high precision time keeping
because of the work they do. Systems can suddenly be off from each other by
up to a second, and this can end up in disaster.

Erlang/OTP 18.x decouples monotonic time from the system time. This means
you can use monotonic time when you need the high resolution/precision
timing. Using time for event ordering is usually a programming mistake
because leap seconds violate the invariant that time is always moving
forwards. Also, there are subtle bugs to look out for: one, distributed
systems will never be able to use time as a resolver for ordering, unless
it is known what drift there are on the clocks. Google's Spanner system
employs GPS clocks in the data centers to make sure time is accurate. And
then they can make guarantees about a time window across data centers. The
other bug is related to what Justin Sheehy so succinctly wrote in the
sentence, "There is no now". Imagine the following code:

TS = os:timestamp(),

f(..., TS, ...)

If Erlang preempts you in , or the kernel does the same, then in
principle, any amount of time might happen between the def of TS and it's
use inside 'f'. That is, any timestamp you draw always "lags behind"
reality by some value ?, and in some cases this ? varies by quite a lot in
a non-deterministic way. If two Erlang processes both draw a timestamp in
this fashion and only one of them gets blocked, then the event ordering
might be inverted from what you expect. Coping with this is a real problem.
In cooperatively scheduled systems, Node.js for instance, this is less of a
problem because a task always runs to completion, even after a kernel
preemption. But in a fully preemptive system, like Erlang, Go or
Haskell[0], this is something to look out for.

[0] Go preempts on a function call boundary nowadays. So it won't be
preempted in a for (;;) { ... } loop. Haskell preempts on memory
allocation. In practice, both models are fully preemptive. Erlang also
preempts on a funcall boundary, but its functional nature means that the
end of any basic block of execution has a function call.


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From kovyl2404@REDACTED  Thu May 14 12:29:00 2015
From: kovyl2404@REDACTED (Viacheslav V. Kovalev)
Date: Thu, 14 May 2015 13:29:00 +0300
Subject: [erlang-questions] Ets read concurrency tweak
Message-ID: 

Hi, List!

I'm playing with ets tweaks and specifically with read_concurrency.
I've written simple test to measure how this tweak impacts on read
performance. Test implementations is here
https://gist.github.com/kovyl2404/826a51b27ba869527910

Briefly, this test sequentially creates three [public, set] ets tables
with different read_concurrency options (without any tweaks, with
{read_concurrency, true} and with {read_concurrency, false}). After
one table created, test pupulates table with some random data and runs
N readers (N is power of 2 from 4 to 1024). Readers performs K random
reads and exists.

Result is quite surprising for me. There absolutely no difference
between 3 these tests. I have run this test on Erlang 17 on 2-, 8-,
and 64-core machines and could not find any significant performance
impact from this tweak.

Could anybody explain usecase of this tweak? What should I do to see
any difference and understand when to use this option?


From richard.youngkin@REDACTED  Thu May 14 16:40:50 2015
From: richard.youngkin@REDACTED (Youngkin, Rich)
Date: Thu, 14 May 2015 08:40:50 -0600
Subject: [erlang-questions] The upcoming leap second
In-Reply-To: 
References: 
 
Message-ID: 

Rickard & Jesper,

Thanks for the additional info, very useful. One item I'm concerned about
is uuid generation. I'll have to look into this a bit more. Thanks again
for your help!

Cheers,
Rich
On May 14, 2015 2:10 AM, "Jesper Louis Andersen" <
jesper.louis.andersen@REDACTED> wrote:

>
> On Wed, May 13, 2015 at 3:39 PM, Youngkin, Rich <
> richard.youngkin@REDACTED> wrote:
>
>> here's an upcoming leap second on June 30th.  There's a bit of buzz
>> about how it affects Linux and Java, as well as problems encountered in
>> 2012 [1].
>>
>
> In addition to what Rickard wrote:
>
> The two major problems to look out for is repetition and precision.
> Repetition happens because the POSIX clock doesn't understand leap seconds,
> so it repeats a second. If UTC is 58, 59, 60, 00, ... then POSIX will be N,
> N+1, N+2, N+2, ... Now, if you use the equivalent of `os:timestamp()` (Pre
> 18) in the 60 and 00 seconds, then you may get the wrong order. Say you
> call os timestamp twice:
>
> TS1 = os:timestamp(),
> ...
> TS2 = os:timestamp(),
>
> You expect TS1 < TS2, but when time repeats, you may get fractions of a
> second and suddenly this invariant breaks. For example if TS1 = {X, Y,
> 700000} and TS2 = {X, Y, 200000}. It leads to all kinds of trouble if you
> rely on the time ordering in your system, and such errors can sometimes
> cascade through subsystems creating an avalanche of failure ultimately
> bringing the system down.
>
> In Pre 18 systems, erlang:now() performs what is called "smearing"
> nowadays, so it is guaranteed to be monotonic and unique. This means the
> above repetition problem doesn't happen. From 18.x and onwards, the new
> Time API gives you far more insight in what happens to the time in the
> system, so you are able to program your own solution, which is correct for
> your problem. Also note that Google altered their NTPd's to perform
> smearing systemwide for clusters where they knew it was not a problem.
>
> The other problem is precision. Some NTP daemons can't cope with leap
> seconds, so when one happens, they are "kicked" and loses time precision.
> Smearing also alters the clock speed, so 1000ms could suddenly be 1010ms or
> 1001ms in your system. For some systems, where high-resolution timing is a
> necessity, this is trouble. Air Traffic control needs high precision
> because planes move 300m in 1 second. The same is true for high speed
> trains. Manufacturing plants some times needs high precision time keeping
> because of the work they do. Systems can suddenly be off from each other by
> up to a second, and this can end up in disaster.
>
> Erlang/OTP 18.x decouples monotonic time from the system time. This means
> you can use monotonic time when you need the high resolution/precision
> timing. Using time for event ordering is usually a programming mistake
> because leap seconds violate the invariant that time is always moving
> forwards. Also, there are subtle bugs to look out for: one, distributed
> systems will never be able to use time as a resolver for ordering, unless
> it is known what drift there are on the clocks. Google's Spanner system
> employs GPS clocks in the data centers to make sure time is accurate. And
> then they can make guarantees about a time window across data centers. The
> other bug is related to what Justin Sheehy so succinctly wrote in the
> sentence, "There is no now". Imagine the following code:
>
> TS = os:timestamp(),
> 
> f(..., TS, ...)
>
> If Erlang preempts you in , or the kernel does the same, then in
> principle, any amount of time might happen between the def of TS and it's
> use inside 'f'. That is, any timestamp you draw always "lags behind"
> reality by some value ?, and in some cases this ? varies by quite a lot in
> a non-deterministic way. If two Erlang processes both draw a timestamp in
> this fashion and only one of them gets blocked, then the event ordering
> might be inverted from what you expect. Coping with this is a real problem.
> In cooperatively scheduled systems, Node.js for instance, this is less of a
> problem because a task always runs to completion, even after a kernel
> preemption. But in a fully preemptive system, like Erlang, Go or
> Haskell[0], this is something to look out for.
>
> [0] Go preempts on a function call boundary nowadays. So it won't be
> preempted in a for (;;) { ... } loop. Haskell preempts on memory
> allocation. In practice, both models are fully preemptive. Erlang also
> preempts on a funcall boundary, but its functional nature means that the
> end of any basic block of execution has a function call.
>
>
> --
> J.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From chandrashekhar.mullaparthi@REDACTED  Thu May 14 16:52:01 2015
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Thu, 14 May 2015 15:52:01 +0100
Subject: [erlang-questions] Ets read concurrency tweak
In-Reply-To: 
References: 
Message-ID: 

Hi,

A few comments.

- Your two tests which use the read_concurrency option are both setting it
to 'true' (lines 31 and 50 in the gist)

- I wouldn't run profiling when trying to compare two different options. It
could be that the overhead of profiling is masking any differences between
the two use cases.

- Setting {read_concurrency, false} is the same as not specifying it (so in
effect 2 of the tests are the same)

cheers
Chandru


On 14 May 2015 at 11:29, Viacheslav V. Kovalev  wrote:

> Hi, List!
>
> I'm playing with ets tweaks and specifically with read_concurrency.
> I've written simple test to measure how this tweak impacts on read
> performance. Test implementations is here
> https://gist.github.com/kovyl2404/826a51b27ba869527910
>
> Briefly, this test sequentially creates three [public, set] ets tables
> with different read_concurrency options (without any tweaks, with
> {read_concurrency, true} and with {read_concurrency, false}). After
> one table created, test pupulates table with some random data and runs
> N readers (N is power of 2 from 4 to 1024). Readers performs K random
> reads and exists.
>
> Result is quite surprising for me. There absolutely no difference
> between 3 these tests. I have run this test on Erlang 17 on 2-, 8-,
> and 64-core machines and could not find any significant performance
> impact from this tweak.
>
> Could anybody explain usecase of this tweak? What should I do to see
> any difference and understand when to use this option?
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From kovyl2404@REDACTED  Thu May 14 17:50:09 2015
From: kovyl2404@REDACTED (Viacheslav V. Kovalev)
Date: Thu, 14 May 2015 18:50:09 +0300
Subject: [erlang-questions] Ets read concurrency tweak
In-Reply-To: 
References: 
 
Message-ID: 

>> - Your two tests which use the read_concurrency option are both setting it to 'true' (lines 31 and 50 in the gist)
Ah yep, my fault. Shameful copypaste. However, as you noticed,
`{read_concurrency, false}` should be the same as first test with
default parameters. When I've fixed this, nothing was changed.

What about eprof - it is the last thing I've added to this test. With
or without profiler - same results (except of absolute values). Main
purpose of profiling was to see if `ets:lookup` is really most
expensive operation in this test (and this is true).

Also I've experimented with different sizes of table and different
sizes of keys in table. Seems all this things doesn't matter, so I am
really confused with this option and have no ideas.

2015-05-14 17:52 GMT+03:00 Chandru :
> Hi,
>
> A few comments.
>
> - Your two tests which use the read_concurrency option are both setting it
> to 'true' (lines 31 and 50 in the gist)
>
> - I wouldn't run profiling when trying to compare two different options. It
> could be that the overhead of profiling is masking any differences between
> the two use cases.
>
> - Setting {read_concurrency, false} is the same as not specifying it (so in
> effect 2 of the tests are the same)
>
> cheers
> Chandru
>
>
> On 14 May 2015 at 11:29, Viacheslav V. Kovalev  wrote:
>>
>> Hi, List!
>>
>> I'm playing with ets tweaks and specifically with read_concurrency.
>> I've written simple test to measure how this tweak impacts on read
>> performance. Test implementations is here
>> https://gist.github.com/kovyl2404/826a51b27ba869527910
>>
>> Briefly, this test sequentially creates three [public, set] ets tables
>> with different read_concurrency options (without any tweaks, with
>> {read_concurrency, true} and with {read_concurrency, false}). After
>> one table created, test pupulates table with some random data and runs
>> N readers (N is power of 2 from 4 to 1024). Readers performs K random
>> reads and exists.
>>
>> Result is quite surprising for me. There absolutely no difference
>> between 3 these tests. I have run this test on Erlang 17 on 2-, 8-,
>> and 64-core machines and could not find any significant performance
>> impact from this tweak.
>>
>> Could anybody explain usecase of this tweak? What should I do to see
>> any difference and understand when to use this option?
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>
>


From sergej.jurecko@REDACTED  Thu May 14 18:58:17 2015
From: sergej.jurecko@REDACTED (Sergej Jurecko)
Date: Thu, 14 May 2015 18:58:17 +0200
Subject: [erlang-questions] Ets read concurrency tweak
In-Reply-To: 
References: 
 
 
Message-ID: 

I think read_concurrency/write_concurrency are about mutexes between schedulers. So if you have 1000 processes on a single scheduler it makes no difference. How many logical cpus did the machine you were executing tests have?


Sergej

On 14 May 2015, at 17:50, Viacheslav V. Kovalev  wrote:

>>> - Your two tests which use the read_concurrency option are both setting it to 'true' (lines 31 and 50 in the gist)
> Ah yep, my fault. Shameful copypaste. However, as you noticed,
> `{read_concurrency, false}` should be the same as first test with
> default parameters. When I've fixed this, nothing was changed.
> 
> What about eprof - it is the last thing I've added to this test. With
> or without profiler - same results (except of absolute values). Main
> purpose of profiling was to see if `ets:lookup` is really most
> expensive operation in this test (and this is true).
> 
> Also I've experimented with different sizes of table and different
> sizes of keys in table. Seems all this things doesn't matter, so I am
> really confused with this option and have no ideas.
> 
> 2015-05-14 17:52 GMT+03:00 Chandru :
>> Hi,
>> 
>> A few comments.
>> 
>> - Your two tests which use the read_concurrency option are both setting it
>> to 'true' (lines 31 and 50 in the gist)
>> 
>> - I wouldn't run profiling when trying to compare two different options. It
>> could be that the overhead of profiling is masking any differences between
>> the two use cases.
>> 
>> - Setting {read_concurrency, false} is the same as not specifying it (so in
>> effect 2 of the tests are the same)
>> 
>> cheers
>> Chandru
>> 
>> 
>> On 14 May 2015 at 11:29, Viacheslav V. Kovalev  wrote:
>>> 
>>> Hi, List!
>>> 
>>> I'm playing with ets tweaks and specifically with read_concurrency.
>>> I've written simple test to measure how this tweak impacts on read
>>> performance. Test implementations is here
>>> https://gist.github.com/kovyl2404/826a51b27ba869527910
>>> 
>>> Briefly, this test sequentially creates three [public, set] ets tables
>>> with different read_concurrency options (without any tweaks, with
>>> {read_concurrency, true} and with {read_concurrency, false}). After
>>> one table created, test pupulates table with some random data and runs
>>> N readers (N is power of 2 from 4 to 1024). Readers performs K random
>>> reads and exists.
>>> 
>>> Result is quite surprising for me. There absolutely no difference
>>> between 3 these tests. I have run this test on Erlang 17 on 2-, 8-,
>>> and 64-core machines and could not find any significant performance
>>> impact from this tweak.
>>> 
>>> Could anybody explain usecase of this tweak? What should I do to see
>>> any difference and understand when to use this option?
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>> 
>> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions



From bchesneau@REDACTED  Thu May 14 19:10:46 2015
From: bchesneau@REDACTED (Benoit Chesneau)
Date: Thu, 14 May 2015 17:10:46 +0000
Subject: [erlang-questions] [ANN] Ruster-Unsafe and Ruster
In-Reply-To: 
References: 
Message-ID: 

thanks!

On Wed, May 13, 2015 at 4:50 PM Daniel Goertzen 
wrote:

> Since Rust is hitting 1.0.0-release shortly, now is a good time to point
> out some Rust NIF libraries I?ve been working on.  ?Rust? is a modern
> systems language with a focus on speed, safety and concurrency.
>  ?Ruster-Unsafe? is a bare binding of the C NIF API.  ?Ruster? is a higher
> level binding that sits atop Ruster-Unsafe.
>
> Ruster-Unsafe is usable today, but Ruster is still under heavy
> construction.  It is premature for me to be announcing Ruster, but I
> thought I would mention it since I'm already talking about Rust and
> Ruster-Unsafe.
>
> Ruster-Unsafe
> demo: https://github.com/goertzenator/ruster_unsafe_demo
> docs: http://goertzenator.github.io/ruster_unsafe/ruster_unsafe/index.html
> source: https://github.com/goertzenator/ruster_unsafe
>
> Ruster: https://github.com/goertzenator/ruster
> Rust: http://www.rust-lang.org/
>
> A minimal Ruster-Unsafe NIF module looks like ?
>
>
> #[macro_use]
> extern crate ruster_unsafe;
> use ruster_unsafe::*;
>
> nif_init!(b"ruster_unsafe_minimal\0", None, None, None, None,
>     nif!(b"just123\0", 0, just123)
> );
>
> extern "C" fn just123(_env: *mut ErlNifEnv, _argc: c_int,
>     _args: *const ERL_NIF_TERM) -> ERL_NIF_TERM {
> unsafe { enif_make_int(env, 123) }
> }
>
>
>
> Example of the sort of thing I?m aiming for with Ruster ...
>
> // crack the tuple {1, 2, {3, 4}}
> let (a,b,(c,d)) = try!(from_term::<(u32,u32,(u32, u32))>(env, term));
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From community-manager@REDACTED  Thu May 14 19:38:36 2015
From: community-manager@REDACTED (Bruce Yinhe)
Date: Thu, 14 May 2015 19:38:36 +0200
Subject: [erlang-questions] OSCON Europe, Amsterdam
Message-ID: 

Hello everybody,

OSCON, O'Reilly Open Source Convention is coming back to Europe for the
first time since 2006. On 26-28 October 2015, you will meet developers,
innovators, business people, and investors and discuss groundbreaking open
source projects and products during three exciting days in Amsterdam,
Netherlands.

If you are interested in presenting Erlang and would like to submit a talk
and/or tutorials, please get in touch. It is a great place for Erlang talks.

Cheers
Bruce Yinhe

community-manager@REDACTED
+46 72 311 43 89
Community Manager
Industrial Erlang User Group



ErlangCentral.org | @ErlangCentral  | Your
Erlang community site
Sponsored by Industrial Erlang User Group

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From kovyl2404@REDACTED  Thu May 14 20:40:46 2015
From: kovyl2404@REDACTED (Viacheslav V. Kovalev)
Date: Thu, 14 May 2015 21:40:46 +0300
Subject: [erlang-questions] Ets read concurrency tweak
In-Reply-To: 
References: 
 
 
 
Message-ID: 

I have run this test on following installations (without any
dramatical changes, except absolute values).

2 logical, 1 physical CPU, Erlang/OTP 17 [erts-6.1] [64-bit] [smp:2:2]
[async-threads:10] [hipe]
8 logical, 1 physical CPU, Erlang/OTP 17 [erts-6.4] [64-bit] [smp:8:8]
[async-threads:10] [hipe]
64 logical, 4 physical CPU, Erlang/OTP 17 [erts-6.3] [64-bit]
[smp:64:64] [async-threads:10] [hipe]

Also I've experimented with kernel-poll options. I believe it is about
another things, but anyway was interested.

2015-05-14 19:58 GMT+03:00 Sergej Jurecko :
> I think read_concurrency/write_concurrency are about mutexes between schedulers. So if you have 1000 processes on a single scheduler it makes no difference. How many logical cpus did the machine you were executing tests have?
>
>
> Sergej
>
> On 14 May 2015, at 17:50, Viacheslav V. Kovalev  wrote:
>
>>>> - Your two tests which use the read_concurrency option are both setting it to 'true' (lines 31 and 50 in the gist)
>> Ah yep, my fault. Shameful copypaste. However, as you noticed,
>> `{read_concurrency, false}` should be the same as first test with
>> default parameters. When I've fixed this, nothing was changed.
>>
>> What about eprof - it is the last thing I've added to this test. With
>> or without profiler - same results (except of absolute values). Main
>> purpose of profiling was to see if `ets:lookup` is really most
>> expensive operation in this test (and this is true).
>>
>> Also I've experimented with different sizes of table and different
>> sizes of keys in table. Seems all this things doesn't matter, so I am
>> really confused with this option and have no ideas.
>>
>> 2015-05-14 17:52 GMT+03:00 Chandru :
>>> Hi,
>>>
>>> A few comments.
>>>
>>> - Your two tests which use the read_concurrency option are both setting it
>>> to 'true' (lines 31 and 50 in the gist)
>>>
>>> - I wouldn't run profiling when trying to compare two different options. It
>>> could be that the overhead of profiling is masking any differences between
>>> the two use cases.
>>>
>>> - Setting {read_concurrency, false} is the same as not specifying it (so in
>>> effect 2 of the tests are the same)
>>>
>>> cheers
>>> Chandru
>>>
>>>
>>> On 14 May 2015 at 11:29, Viacheslav V. Kovalev  wrote:
>>>>
>>>> Hi, List!
>>>>
>>>> I'm playing with ets tweaks and specifically with read_concurrency.
>>>> I've written simple test to measure how this tweak impacts on read
>>>> performance. Test implementations is here
>>>> https://gist.github.com/kovyl2404/826a51b27ba869527910
>>>>
>>>> Briefly, this test sequentially creates three [public, set] ets tables
>>>> with different read_concurrency options (without any tweaks, with
>>>> {read_concurrency, true} and with {read_concurrency, false}). After
>>>> one table created, test pupulates table with some random data and runs
>>>> N readers (N is power of 2 from 4 to 1024). Readers performs K random
>>>> reads and exists.
>>>>
>>>> Result is quite surprising for me. There absolutely no difference
>>>> between 3 these tests. I have run this test on Erlang 17 on 2-, 8-,
>>>> and 64-core machines and could not find any significant performance
>>>> impact from this tweak.
>>>>
>>>> Could anybody explain usecase of this tweak? What should I do to see
>>>> any difference and understand when to use this option?
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>


From vychodil.hynek@REDACTED  Thu May 14 20:59:20 2015
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Thu, 14 May 2015 20:59:20 +0200
Subject: [erlang-questions] Ets read concurrency tweak
In-Reply-To: 
References: 
 
 
 
 
Message-ID: 

I think you have to add at least few writes to see the difference. In your
case, all your scheduler threads have soon read lock (RO copy in their CPU
cache), so there is no effect of more granular locks.

On Thu, May 14, 2015 at 8:40 PM, Viacheslav V. Kovalev 
wrote:

> I have run this test on following installations (without any
> dramatical changes, except absolute values).
>
> 2 logical, 1 physical CPU, Erlang/OTP 17 [erts-6.1] [64-bit] [smp:2:2]
> [async-threads:10] [hipe]
> 8 logical, 1 physical CPU, Erlang/OTP 17 [erts-6.4] [64-bit] [smp:8:8]
> [async-threads:10] [hipe]
> 64 logical, 4 physical CPU, Erlang/OTP 17 [erts-6.3] [64-bit]
> [smp:64:64] [async-threads:10] [hipe]
>
> Also I've experimented with kernel-poll options. I believe it is about
> another things, but anyway was interested.
>
> 2015-05-14 19:58 GMT+03:00 Sergej Jurecko :
> > I think read_concurrency/write_concurrency are about mutexes between
> schedulers. So if you have 1000 processes on a single scheduler it makes no
> difference. How many logical cpus did the machine you were executing tests
> have?
> >
> >
> > Sergej
> >
> > On 14 May 2015, at 17:50, Viacheslav V. Kovalev 
> wrote:
> >
> >>>> - Your two tests which use the read_concurrency option are both
> setting it to 'true' (lines 31 and 50 in the gist)
> >> Ah yep, my fault. Shameful copypaste. However, as you noticed,
> >> `{read_concurrency, false}` should be the same as first test with
> >> default parameters. When I've fixed this, nothing was changed.
> >>
> >> What about eprof - it is the last thing I've added to this test. With
> >> or without profiler - same results (except of absolute values). Main
> >> purpose of profiling was to see if `ets:lookup` is really most
> >> expensive operation in this test (and this is true).
> >>
> >> Also I've experimented with different sizes of table and different
> >> sizes of keys in table. Seems all this things doesn't matter, so I am
> >> really confused with this option and have no ideas.
> >>
> >> 2015-05-14 17:52 GMT+03:00 Chandru <
> chandrashekhar.mullaparthi@REDACTED>:
> >>> Hi,
> >>>
> >>> A few comments.
> >>>
> >>> - Your two tests which use the read_concurrency option are both
> setting it
> >>> to 'true' (lines 31 and 50 in the gist)
> >>>
> >>> - I wouldn't run profiling when trying to compare two different
> options. It
> >>> could be that the overhead of profiling is masking any differences
> between
> >>> the two use cases.
> >>>
> >>> - Setting {read_concurrency, false} is the same as not specifying it
> (so in
> >>> effect 2 of the tests are the same)
> >>>
> >>> cheers
> >>> Chandru
> >>>
> >>>
> >>> On 14 May 2015 at 11:29, Viacheslav V. Kovalev 
> wrote:
> >>>>
> >>>> Hi, List!
> >>>>
> >>>> I'm playing with ets tweaks and specifically with read_concurrency.
> >>>> I've written simple test to measure how this tweak impacts on read
> >>>> performance. Test implementations is here
> >>>> https://gist.github.com/kovyl2404/826a51b27ba869527910
> >>>>
> >>>> Briefly, this test sequentially creates three [public, set] ets tables
> >>>> with different read_concurrency options (without any tweaks, with
> >>>> {read_concurrency, true} and with {read_concurrency, false}). After
> >>>> one table created, test pupulates table with some random data and runs
> >>>> N readers (N is power of 2 from 4 to 1024). Readers performs K random
> >>>> reads and exists.
> >>>>
> >>>> Result is quite surprising for me. There absolutely no difference
> >>>> between 3 these tests. I have run this test on Erlang 17 on 2-, 8-,
> >>>> and 64-core machines and could not find any significant performance
> >>>> impact from this tweak.
> >>>>
> >>>> Could anybody explain usecase of this tweak? What should I do to see
> >>>> any difference and understand when to use this option?
> >>>> _______________________________________________
> >>>> erlang-questions mailing list
> >>>> erlang-questions@REDACTED
> >>>> http://erlang.org/mailman/listinfo/erlang-questions
> >>>
> >>>
> >> _______________________________________________
> >> erlang-questions mailing list
> >> erlang-questions@REDACTED
> >> http://erlang.org/mailman/listinfo/erlang-questions
> >
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roberto@REDACTED  Thu May 14 22:33:41 2015
From: roberto@REDACTED (Roberto Ostinelli)
Date: Thu, 14 May 2015 22:33:41 +0200
Subject: [erlang-questions] {tls_alert,"certificate unknown"}
Message-ID: 

Dear list,
I'm using hackney to perform http requests from an Erlang release.

When I run the release manually, i.e. myapp/bin/myapp start, hackney works
as expected. When the release is started from an init.d script, I get these
errors:

ssl_handshake.erl:438:Fatal error: certificate unknown
{tls_alert,"certificate unknown"}

Any ideas of why this might be happening?

Thank you,
r.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From diptansu@REDACTED  Fri May 15 04:57:30 2015
From: diptansu@REDACTED (Diptansu Das)
Date: Thu, 14 May 2015 22:57:30 -0400
Subject: [erlang-questions] How to load a port driver which itself uses a
	shared library?
Message-ID: 

I need to create a port driver which will need to link against some other
shared library. When I try to load the driver I get an {error, {open_error,
-10}}. I tracked this down to the call to the shared library functions. If
I comment out the calls to the shared library it loads fine. It seems the
erl_ddll loader cannot load a driver if it links to a shared library... Or
maybe there is some other way to accomplish this?
In case I cannot create a normal dynamically loaded port driver, how can I
statically link it to the erlang build?
Thanks,
D
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From boris.muehmer@REDACTED  Fri May 15 09:36:45 2015
From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=)
Date: Fri, 15 May 2015 09:36:45 +0200
Subject: [erlang-questions] How to load a port driver which itself uses
 a shared library?
In-Reply-To: 
References: 
Message-ID: 

Is Your work Linux based? If so, did You try to set LD_LIBRARY_PATH? But
this is only needed, if Your shared library dependency is in an unusual
location. There are other ways, to specify non-default shared library
pathes as well, but LD_LIBRARY_PATH may be the fastest to test for
non-default pathes.

For gcc you can specify -static when linking to force to use no dynamic
libraries. Another approach would be to use "-Wl,-rpath=...".
Maybe You should have a look at
http://stackoverflow.com/questions/8835108/specify-non-default-shared-library-path-in-gcc-linux

Regards,
Boris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From sergej.jurecko@REDACTED  Fri May 15 09:41:38 2015
From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=)
Date: Fri, 15 May 2015 09:41:38 +0200
Subject: [erlang-questions] How to load a port driver which itself uses
 a shared library?
In-Reply-To: 
References: 
Message-ID: 

To check shared library paths that your driver is looking for:

Linux: ldd your_driver.so
Osx: otool -L your_driver.so


Sergej

On Fri, May 15, 2015 at 4:57 AM, Diptansu Das  wrote:

> I need to create a port driver which will need to link against some other
> shared library. When I try to load the driver I get an {error, {open_error,
> -10}}. I tracked this down to the call to the shared library functions. If
> I comment out the calls to the shared library it loads fine. It seems the
> erl_ddll loader cannot load a driver if it links to a shared library... Or
> maybe there is some other way to accomplish this?
> In case I cannot create a normal dynamically loaded port driver, how can I
> statically link it to the erlang build?
> Thanks,
> D
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roberto@REDACTED  Fri May 15 11:11:32 2015
From: roberto@REDACTED (Roberto Ostinelli)
Date: Fri, 15 May 2015 11:11:32 +0200
Subject: [erlang-questions] Cowboy and links: weird delayed times
Message-ID: 

Dear list,
I am using cowboy to handle long lived http connections.

When a new connection is successful, I link a router process to it with
erlang:link/1, which traps exits, so that when I receive the
message {'EXIT', Pid, Reason} I can remove the connection reference from
the router process. Pretty standard stuff.

After Cowboy's terminate/3 callback is called, I immediately see the log
that the connection has been removed from the router process too.

However, in production I see that it can take almost 2 minutes from the
moment the terminate/3 function was called, and the router receiving the
{'EXIT', Pid, Reason}, which means that after exiting from the terminate/3
function the Cowboy process does not exit for almost 2 minutes:

2015-05-14 21:16:25.302 [debug] <0.1714.0>@myapp_device:terminate:180
Terminated robot request
[...]
2015-05-14 21:18:15.304 [info] <0.1717.0>@myapp_router:handle_info:157 Now
offline

My question is: have you seen this before? Does this mean that the client
is badly formed and is refusing to shutdown gracefully? If so, is there a
way to close the socket abruptly instead?

Thank you,
r.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roberto@REDACTED  Fri May 15 11:33:28 2015
From: roberto@REDACTED (Roberto Ostinelli)
Date: Fri, 15 May 2015 11:33:28 +0200
Subject: [erlang-questions] Cowboy and links: weird delayed times
In-Reply-To: 
References: 
Message-ID: 

Answering my own here.
This is pretty simple actually: if the connection header of the clients is
keep-alive, Cowboy will wait the timeout before killing the process, so
obviously it isn't removed from the router.

Duh.

Best,
r.


On Fri, May 15, 2015 at 11:11 AM, Roberto Ostinelli 
wrote:

> Dear list,
> I am using cowboy to handle long lived http connections.
>
> When a new connection is successful, I link a router process to it with
> erlang:link/1, which traps exits, so that when I receive the
> message {'EXIT', Pid, Reason} I can remove the connection reference from
> the router process. Pretty standard stuff.
>
> After Cowboy's terminate/3 callback is called, I immediately see the log
> that the connection has been removed from the router process too.
>
> However, in production I see that it can take almost 2 minutes from the
> moment the terminate/3 function was called, and the router receiving the
> {'EXIT', Pid, Reason}, which means that after exiting from the terminate/3
> function the Cowboy process does not exit for almost 2 minutes:
>
> 2015-05-14 21:16:25.302 [debug] <0.1714.0>@myapp_device:terminate:180
> Terminated robot request
> [...]
> 2015-05-14 21:18:15.304 [info] <0.1717.0>@myapp_router:handle_info:157 Now
> offline
>
> My question is: have you seen this before? Does this mean that the client
> is badly formed and is refusing to shutdown gracefully? If so, is there a
> way to close the socket abruptly instead?
>
> Thank you,
> r.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roger@REDACTED  Fri May 15 12:14:35 2015
From: roger@REDACTED (Roger Lipscombe)
Date: Fri, 15 May 2015 11:14:35 +0100
Subject: [erlang-questions] Cowboy and links: weird delayed times
In-Reply-To: 
References: 
 
Message-ID: 

On 15 May 2015 at 10:33, Roberto Ostinelli  wrote:
> Answering my own here.
> This is pretty simple actually: if the connection header of the clients is
> keep-alive, Cowboy will wait the timeout before killing the process, so
> obviously it isn't removed from the router.

Yes, per http://ninenines.eu/docs/en/cowboy/1.0/guide/http_handlers/#cleaning_up:

"If you used the process dictionary, timers, monitors or may be
receiving messages, then you can use this function to clean them up,
as Cowboy might reuse the process for the next keep-alive request."


From rickard@REDACTED  Fri May 15 12:50:40 2015
From: rickard@REDACTED (Rickard Green)
Date: Fri, 15 May 2015 12:50:40 +0200
Subject: [erlang-questions] Erlang/OTP 18.0-rc2 is available for testing
In-Reply-To: <555389EB.5070102@gmail.com>
References: 
 
 
 
 
 <555389EB.5070102@gmail.com>
Message-ID: 

On Wed, May 13, 2015 at 7:29 PM, Michael Truog  wrote:
> On 05/13/2015 06:32 AM, Rickard Green wrote:
>>
>> I broke it :-/ I'll unbreak it and post a notification here when it
>> has been fixed in master. I can not promise exactly when that will
>> happen, but at least before the actual release.
>
> When 18.0-rc1 was released the timer wheel size per scheduler was larger
> (http://erlang.org/pipermail/erlang-bugs/2015-April/004907.html) and the
> size was decreased due to problems related to the larger size. I was
> wondering if there are any plans to add an erts option that can be adjusted
> from the erl command line, so that testing can determine if a larger timer
> wheel size per scheduler is beneficial.
>
> Thanks,
> Michael
>

No plans for that. This since the usage of the wheel has been changed,
and by this the main reason for increasing the size is gone.

Regards,
Rickard
-- 
Rickard Green, Erlang/OTP, Ericsson AB


From kovyl2404@REDACTED  Fri May 15 13:00:26 2015
From: kovyl2404@REDACTED (Viacheslav V. Kovalev)
Date: Fri, 15 May 2015 14:00:26 +0300
Subject: [erlang-questions] Ets read concurrency tweak
In-Reply-To: 
References: 
 
 
 
 
 
Message-ID: 

I've added writer process to my test. It perfoms random write per each
100 ms. Implementation is here:
https://gist.github.com/kovyl2404/622a9908c4e8a2abc214

There is example of run on 8-core machine.

29> ets_read_concurrency:analyze(WithoutWriter).
[{{procs,1},{percent,-8.998057178933019}},
 {{procs,2},{percent,1.474232611754453}},
 {{procs,4},{percent,-0.4318193657099243}},
 {{procs,8},{percent,4.796912026714365}},
 {{procs,16},{percent,4.926194326111598}},
 {{procs,32},{percent,7.4091491628647805}},
 {{procs,64},{percent,6.7226404897426315}},
 {{procs,128},{percent,7.129140726319386}},
 {{procs,256},{percent,-28.200373148451757}},
 {{procs,512},{percent,10.229583687247757}},
 {{procs,1024},{percent,25.824989572270635}}]
30> ets_read_concurrency:analyze(WithWriter).
[{{procs,1},{percent,-9.233383915316411}},
 {{procs,2},{percent,1.3554058972355476}},
 {{procs,4},{percent,-1.3437122387165232}},
 {{procs,8},{percent,0.3944371727018411}},
 {{procs,16},{percent,21.719493229803017}},
 {{procs,32},{percent,-26.32711009412866}},
 {{procs,64},{percent,11.577461884371825}},
 {{procs,128},{percent,19.608517106505893}},
 {{procs,256},{percent,11.362311552960543}},
 {{procs,512},{percent,20.935963863004808}},
 {{procs,1024},{percent,22.512472513575506}}]

Where percent calculated as `(NonTweakedTime - TweakedTime)/NonTweaked*100`

Tweaked table shows stable performance advantage either with or
without writer process. I think I was wrong when interpreted former
results, but frankly I expected to see more drastic changes. Again, I
don't know how to interpret this artifacts:

{{procs,256},{percent,-28.200373148451757}}. %% WithoutWriter
{{procs,32},{percent,-26.32711009412866}}. %% WithWriter

Is this just statistical error or something else?


From roberto@REDACTED  Fri May 15 13:05:27 2015
From: roberto@REDACTED (Roberto Ostinelli)
Date: Fri, 15 May 2015 13:05:27 +0200
Subject: [erlang-questions] Cowboy and links: weird delayed times
In-Reply-To: 
References: 
 
 
Message-ID: 

Well no, since there's no guarantee that terminate/3 gets called, if you
want to use monitors you shouldn't rely on that function to cleaning them
out.

Anyway, my problem is solved by forcing the shutdown of the connection.

Best,
r.


On Fri, May 15, 2015 at 12:14 PM, Roger Lipscombe 
wrote:

> On 15 May 2015 at 10:33, Roberto Ostinelli  wrote:
> > Answering my own here.
> > This is pretty simple actually: if the connection header of the clients
> is
> > keep-alive, Cowboy will wait the timeout before killing the process, so
> > obviously it isn't removed from the router.
>
> Yes, per
> http://ninenines.eu/docs/en/cowboy/1.0/guide/http_handlers/#cleaning_up:
>
> "If you used the process dictionary, timers, monitors or may be
> receiving messages, then you can use this function to clean them up,
> as Cowboy might reuse the process for the next keep-alive request."
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From steven.charles.davis@REDACTED  Fri May 15 13:06:27 2015
From: steven.charles.davis@REDACTED (Steve Davis)
Date: Fri, 15 May 2015 06:06:27 -0500
Subject: [erlang-questions] [ANN] Ruster-Unsafe and Ruster
Message-ID: <7ED68E23-39BE-4319-8988-16B340AC6FCF@gmail.com>

+ 1 on interest level 

I had been wondering myself if Rust would be a good fit for NIFs


From chandrashekhar.mullaparthi@REDACTED  Fri May 15 13:11:24 2015
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Fri, 15 May 2015 12:11:24 +0100
Subject: [erlang-questions] Ets read concurrency tweak
In-Reply-To: 
References: 
 
 
 
 
 
 
Message-ID: 

Hi,

You might want to read this to get some insights.



http://www.researchgate.net/profile/Konstantinos_Sagonas/publication/262172496_On_the_scalability_of_the_Erlang_term_storage/links/53f3a22f0cf2155be351bc3b.pdf?origin=publication_detail

cheers,
Chandru


On 15 May 2015 at 12:00, Viacheslav V. Kovalev  wrote:

> I've added writer process to my test. It perfoms random write per each
> 100 ms. Implementation is here:
> https://gist.github.com/kovyl2404/622a9908c4e8a2abc214
>
> There is example of run on 8-core machine.
>
> 29> ets_read_concurrency:analyze(WithoutWriter).
> [{{procs,1},{percent,-8.998057178933019}},
>  {{procs,2},{percent,1.474232611754453}},
>  {{procs,4},{percent,-0.4318193657099243}},
>  {{procs,8},{percent,4.796912026714365}},
>  {{procs,16},{percent,4.926194326111598}},
>  {{procs,32},{percent,7.4091491628647805}},
>  {{procs,64},{percent,6.7226404897426315}},
>  {{procs,128},{percent,7.129140726319386}},
>  {{procs,256},{percent,-28.200373148451757}},
>  {{procs,512},{percent,10.229583687247757}},
>  {{procs,1024},{percent,25.824989572270635}}]
> 30> ets_read_concurrency:analyze(WithWriter).
> [{{procs,1},{percent,-9.233383915316411}},
>  {{procs,2},{percent,1.3554058972355476}},
>  {{procs,4},{percent,-1.3437122387165232}},
>  {{procs,8},{percent,0.3944371727018411}},
>  {{procs,16},{percent,21.719493229803017}},
>  {{procs,32},{percent,-26.32711009412866}},
>  {{procs,64},{percent,11.577461884371825}},
>  {{procs,128},{percent,19.608517106505893}},
>  {{procs,256},{percent,11.362311552960543}},
>  {{procs,512},{percent,20.935963863004808}},
>  {{procs,1024},{percent,22.512472513575506}}]
>
> Where percent calculated as `(NonTweakedTime - TweakedTime)/NonTweaked*100`
>
> Tweaked table shows stable performance advantage either with or
> without writer process. I think I was wrong when interpreted former
> results, but frankly I expected to see more drastic changes. Again, I
> don't know how to interpret this artifacts:
>
> {{procs,256},{percent,-28.200373148451757}}. %% WithoutWriter
> {{procs,32},{percent,-26.32711009412866}}. %% WithWriter
>
> Is this just statistical error or something else?
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From essen@REDACTED  Fri May 15 13:57:31 2015
From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=)
Date: Fri, 15 May 2015 14:57:31 +0300
Subject: [erlang-questions] Cowboy and links: weird delayed times
In-Reply-To: 
References: 
 
 
 
Message-ID: <5555DF2B.9080105@ninenines.eu>

It's there to handle the case where you have keep-alive and have two 
unrelated requests one after another, not the case where the process ends.

Force closing in your case is the right solution.

On 05/15/2015 02:05 PM, Roberto Ostinelli wrote:
> Well no, since there's no guarantee that terminate/3 gets called, if you
> want to use monitors you shouldn't rely on that function to cleaning
> them out.
>
> Anyway, my problem is solved by forcing the shutdown of the connection.
>
> Best,
> r.
>
>
> On Fri, May 15, 2015 at 12:14 PM, Roger Lipscombe
> > wrote:
>
>     On 15 May 2015 at 10:33, Roberto Ostinelli      > wrote:
>     > Answering my own here.
>     > This is pretty simple actually: if the connection header of the clients is
>     > keep-alive, Cowboy will wait the timeout before killing the process, so
>     > obviously it isn't removed from the router.
>
>     Yes, per
>     http://ninenines.eu/docs/en/cowboy/1.0/guide/http_handlers/#cleaning_up:
>
>     "If you used the process dictionary, timers, monitors or may be
>     receiving messages, then you can use this function to clean them up,
>     as Cowboy might reuse the process for the next keep-alive request."
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>

-- 
Lo?c Hoguin
http://ninenines.eu


From diptansu@REDACTED  Fri May 15 15:25:41 2015
From: diptansu@REDACTED (Diptansu Das)
Date: Fri, 15 May 2015 09:25:41 -0400
Subject: [erlang-questions] How to load a port driver which itself uses
 a shared library?
In-Reply-To: 
References: 
 
Message-ID: 

Yes, I tried with LD_LIBRARY_PATH and -rpath. Neither worked. This is on
Linux. I also tried using the erl_ddll:format_error/1 and that said that
one of the functions was undefined. The function is actually in a .so file
in the same directory.

On Fri, May 15, 2015 at 3:36 AM, Boris M?hmer 
wrote:

> Is Your work Linux based? If so, did You try to set LD_LIBRARY_PATH? But
> this is only needed, if Your shared library dependency is in an unusual
> location. There are other ways, to specify non-default shared library
> pathes as well, but LD_LIBRARY_PATH may be the fastest to test for
> non-default pathes.
>
> For gcc you can specify -static when linking to force to use no dynamic
> libraries. Another approach would be to use "-Wl,-rpath=...".
> Maybe You should have a look at
>
> http://stackoverflow.com/questions/8835108/specify-non-default-shared-library-path-in-gcc-linux
>
> Regards,
> Boris
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From diptansu@REDACTED  Fri May 15 15:30:40 2015
From: diptansu@REDACTED (Diptansu Das)
Date: Fri, 15 May 2015 09:30:40 -0400
Subject: [erlang-questions] How to load a port driver which itself uses
 a shared library?
In-Reply-To: 
References: 
 
Message-ID: 

 I used the erl_ddll:format_error/1 and that said that one of the functions
(foo) was undefined. The function doo is actually in another .so file in
the same directory (lets call that foo.so). Weirdly, when I run ldd on my
driver.so it does not show any dependency on this foo.so . I am compiling
the driver by using:
 g++ -o simple_drv.so -i$ERL_TOP/erts/emulator/beam -fpic -shared
simple_drv.cpp

On Fri, May 15, 2015 at 3:41 AM, Sergej Jure?ko 
wrote:

> To check shared library paths that your driver is looking for:
>
> Linux: ldd your_driver.so
> Osx: otool -L your_driver.so
>
>
> Sergej
>
> On Fri, May 15, 2015 at 4:57 AM, Diptansu Das  wrote:
>
>> I need to create a port driver which will need to link against some other
>> shared library. When I try to load the driver I get an {error, {open_error,
>> -10}}. I tracked this down to the call to the shared library functions. If
>> I comment out the calls to the shared library it loads fine. It seems the
>> erl_ddll loader cannot load a driver if it links to a shared library... Or
>> maybe there is some other way to accomplish this?
>> In case I cannot create a normal dynamically loaded port driver, how can
>> I statically link it to the erlang build?
>> Thanks,
>> D
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From diptansu@REDACTED  Fri May 15 16:20:20 2015
From: diptansu@REDACTED (Diptansu Das)
Date: Fri, 15 May 2015 10:20:20 -0400
Subject: [erlang-questions] How to load a port driver which itself uses
 a shared library?
In-Reply-To: 
References: 
 
 
Message-ID: 

I figure out my problem. I wasn't putting a -l in the link line,
so of course it didn't know which library to lookup. Idiotic.
Thanks for the help everyone. The tip about running ldd got me thinking.
Thanks.

On Fri, May 15, 2015 at 9:30 AM, Diptansu Das  wrote:

>  I used the erl_ddll:format_error/1 and that said that one of the
> functions (foo) was undefined. The function doo is actually in another .so
> file in the same directory (lets call that foo.so). Weirdly, when I run ldd
> on my driver.so it does not show any dependency on this foo.so . I am
> compiling the driver by using:
>  g++ -o simple_drv.so -i$ERL_TOP/erts/emulator/beam -fpic -shared
> simple_drv.cpp
>
> On Fri, May 15, 2015 at 3:41 AM, Sergej Jure?ko 
> wrote:
>
>> To check shared library paths that your driver is looking for:
>>
>> Linux: ldd your_driver.so
>> Osx: otool -L your_driver.so
>>
>>
>> Sergej
>>
>> On Fri, May 15, 2015 at 4:57 AM, Diptansu Das  wrote:
>>
>>> I need to create a port driver which will need to link against some
>>> other shared library. When I try to load the driver I get an {error,
>>> {open_error, -10}}. I tracked this down to the call to the shared library
>>> functions. If I comment out the calls to the shared library it loads fine.
>>> It seems the erl_ddll loader cannot load a driver if it links to a shared
>>> library... Or maybe there is some other way to accomplish this?
>>> In case I cannot create a normal dynamically loaded port driver, how can
>>> I statically link it to the erlang build?
>>> Thanks,
>>> D
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From ingela.andin@REDACTED  Fri May 15 17:05:13 2015
From: ingela.andin@REDACTED (Ingela Andin)
Date: Fri, 15 May 2015 17:05:13 +0200
Subject: [erlang-questions] {tls_alert,"certificate unknown"}
In-Reply-To: 
References: 
Message-ID: 

Hi!

Do you have the same version of the asn1 and public_key applications in
both cases?  This error typically means that something went wrong when
trying to decode the ASN1 decode certificate.

Regards Ingela Erlang/OTP team - Ericsson AB

2015-05-14 22:33 GMT+02:00 Roberto Ostinelli :

> Dear list,
> I'm using hackney to perform http requests from an Erlang release.
>
> When I run the release manually, i.e. myapp/bin/myapp start, hackney works
> as expected. When the release is started from an init.d script, I get these
> errors:
>
> ssl_handshake.erl:438:Fatal error: certificate unknown
> {tls_alert,"certificate unknown"}
>
> Any ideas of why this might be happening?
>
> Thank you,
> r.
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From max.lapshin@REDACTED  Fri May 15 19:03:50 2015
From: max.lapshin@REDACTED (Max Lapshin)
Date: Fri, 15 May 2015 20:03:50 +0300
Subject: [erlang-questions] Erlang/OTP 18.0-rc2 is available for testing
In-Reply-To: 
References: 
 
 
 
 
 <555389EB.5070102@gmail.com>
 
Message-ID: 

Steve, are your dirty nif patches included?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From vinoski@REDACTED  Fri May 15 20:18:32 2015
From: vinoski@REDACTED (Steve Vinoski)
Date: Fri, 15 May 2015 14:18:32 -0400
Subject: [erlang-questions] Erlang/OTP 18.0-rc2 is available for testing
In-Reply-To: 
References: 
 
 
 
 
 <555389EB.5070102@gmail.com>
 
 
Message-ID: 

Yes, the patch you're referring to merged a few weeks ago, but note that
18.0-rc2 doesn't work for dirty schedulers as discussed earlier in the
thread due to an issue related to timer wheel changes. Rickard and I will
have a fix soon.

--steve

On Fri, May 15, 2015 at 1:03 PM, Max Lapshin  wrote:

> Steve, are your dirty nif patches included?
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From radek@REDACTED  Fri May 15 22:21:09 2015
From: radek@REDACTED (Rad Gruchalski)
Date: Fri, 15 May 2015 22:21:09 +0200
Subject: [erlang-questions] UDP multicast on Raspberry Pi
Message-ID: 

Hi everyone,  

I?m seem to be having issues setting up multicast on Raspberry Pi running latest available Raspbian (2015-05-05 Wheezy).
I have created a minimal viable example to make sure that all the complexity of my software is out of the way.

First, verifying that multicast is working - on RPi:

iperf -s -u -B 239.0.0.250 -i 1

and on my machine:

iperf -c 239.0.0.250 -u -T 32 -t 3 -i 1

My RPi displays:

Server listening on UDP port 5001
Binding to local address 239.0.0.250
Joining multicast group  239.0.0.250
Receiving 1470 byte datagrams
UDP buffer size:  160 KByte (default)
------------------------------------------------------------
[  3] local 239.0.0.250 port 5001 connected with 10.128.30.104 port 55840
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3]  0.0- 1.0 sec  79.0 KBytes   647 Kbits/sec  12.543 ms    0/   55 (0%)
[  3]  1.0- 2.0 sec   169 KBytes  1.39 Mbits/sec  11.075 ms    0/  118 (0%)
[  3]  2.0- 3.0 sec  51.7 KBytes   423 Kbits/sec  15.568 ms    0/   36 (0%)
[  3]  3.0- 4.0 sec  83.3 KBytes   682 Kbits/sec  11.260 ms    0/   58 (0%)
[  3]  0.0- 4.3 sec   386 KBytes   728 Kbits/sec  54.013 ms    0/  268 (0%)
[  3]  0.0- 4.3 sec  1 datagrams received out-of-order


Suggesting that I can receive multicast. This works both ways.

Here is my Erlang UDP multicast server:

-module(udp_multicast).

-behaviour(gen_server).

-export([start_link/0, stop/0]).
-export([init/1, handle_call/3, handle_cast/2, handle_info/2, code_change/3, terminate/2]).

start_link() ->
  gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).

stop() -> gen_server:cast(?MODULE, stop).

init([]) ->
  Port = 6666,
  MulticastIp = {239,0,0,251},
  IfaceIp = {0,0,0,0},
  {ok, OverlaySocket} = gen_udp:open(Port, [ binary,
                                             {ip, MulticastIp},
                                             {multicast_ttl, 4},
                                             {multicast_loop, true},
                                             {broadcast, true},
                                             {add_membership, {MulticastIp, IfaceIp}} ] ),
  { ok, OverlaySocket }.

handle_info({udp, _ClientSocket, _ClientIp, _ClientPort, Msg}, State) ->
  error_logger:info_msg("Received multicast data: ~p", [ Msg ]),
  {noreply, State}.

handle_call( _, _From, State) ->
  { reply, ok, State }.

code_change(_OldVsn, State, _Extra) ->
  {ok, State}.

terminate(_, _) ->
  ok.

handle_cast(stop, LoopData) ->
  {noreply, LoopData}.


And this is the client I use to push multicast data:

-module(mcc).

-export([run/0]).

run() ->
  Port = 6666,
  MulticastIp = {239,0,0,251},
  IfaceIp = {0,0,0,0},
  {ok, OverlaySocket} = gen_udp:open(Port, [ binary,
                                             {ip, MulticastIp},
                                             {multicast_ttl, 4},
                                             {multicast_loop, true},
                                             {broadcast, true},
                                             {add_membership, {MulticastIp, IfaceIp}} ] ),
  gen_udp:send( OverlaySocket, MulticastIp, Port, <<"some random datas">> ),
  gen_udp:close( OverlaySocket ).


If I start the udp_multicast program on my development machine and run mcc from RPi, I receive the data. If I do it the other way around, udp_multicast listening on RPi and mcc run from development box, no data is received on RPi. There?s no firewall in between and the router puts no restriction on multicast.  
I?m baffled and looking for some ideas where could I start looking for possible cause. These tests have been done with OTP 17.4 and 17.5.

RPi system details:

pi@REDACTED ~ $ erl
Erlang/OTP 17 [erts-6.4] [source] [smp:4:4] [async-threads:10] [kernel-poll:false]

Eshell V6.4  (abort with ^G)
1> erlang:system_info(system_architecture).
"armv7l-unknown-linux-gnueabihf?


> uname -a
Linux raspberrypi 3.18.11-v7+ #781 SMP PREEMPT Tue Apr 21 18:07:59 BST 2015 armv7l GNU/Linux

Erlang built from sources using following instructions:
https://gist.github.com/radekg/93c1ca2c18c34830ac05










Kind regards,

Radek Gruchalski

radek@REDACTED (mailto:radek@REDACTED)
 (mailto:radek@REDACTED)
de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)

Confidentiality:
This communication is intended for the above-named person and may be confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From thomas.elsgaard@REDACTED  Sat May 16 03:59:14 2015
From: thomas.elsgaard@REDACTED (Thomas Elsgaard)
Date: Fri, 15 May 2015 23:59:14 -0200
Subject: [erlang-questions] Erlang microservice architecture ?
Message-ID: 

Hi list

If you should recommend the architecture for an microservice implementation
with around 20 Erlang microservices implemented on separate servers, how
would you implement it ?

1) using gen_servers on each server and use rpc calls between services ?
2) using web servers like yaws or webmachine on each server, and use http +
json/xml between services ?
3) roll my own erlang applications + message parsing on each server ?

Or any better ideas ?

At some point, thing like service discovery will also be interresting

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From max.lapshin@REDACTED  Sat May 16 08:25:06 2015
From: max.lapshin@REDACTED (Max Lapshin)
Date: Sat, 16 May 2015 09:25:06 +0300
Subject: [erlang-questions] UDP multicast on Raspberry Pi
In-Reply-To: 
References: 
Message-ID: 

Replace  {ip, MulticastIp},  with {ip, IfaceIp},
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From chandrashekhar.mullaparthi@REDACTED  Sat May 16 11:11:22 2015
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Sat, 16 May 2015 10:11:22 +0100
Subject: [erlang-questions] Erlang microservice architecture ?
In-Reply-To: 
References: 
Message-ID: 

Hi Thomas,

All the approaches you outlined below are valid and workable. The exact
answer depends on your particular set of requirements around system
integration, failure handling, load balancing, scalability, performance
required etc. Further comments inline below.


On 16 May 2015 at 02:59, Thomas Elsgaard  wrote:

> Hi list
>
> If you should recommend the architecture for an microservice
> implementation with around 20 Erlang microservices implemented on separate
> servers, how would you implement it ?
>
> 1) using gen_servers on each server and use rpc calls between services ?
>

The downside to using RPC is lack of flow control. When using RPC there is
no (easy) way for the server end to put back pressure on the clients. The
upside is obviously convenience. This approach probably has the lowest
barrier to entry so ideal for prototyping.



> 2) using web servers like yaws or webmachine on each server, and use http
> + json/xml between services ?
>

This approach is fine up to a certain point. Things to consider here are:
- Is strict ordering required between messages?
- What is the transaction volume? For very high volumes, the overhead of
HTTP becomes significant. HTTP isn't the best designed protocol either.
Sure, client libraries exist (I myself maintain one) which can hide this
for you, but the ugliness is still there.
- The upside is that integration between components (written in different
languages) is easy


> 3) roll my own erlang applications + message parsing on each server ?
>

This takes the most effort (compared to the rest, but still quite easy to
do in Erlang), but will also deliver maximum control and performance for
you. You can either roll your own protocol, or use things such as Thrift or
Protobuffs if integration between systems written in different languages is
important.


>
> Or any better ideas ?
>
> At some point, thing like service discovery will also be interresting
>

If the topology of your network of erlang nodes can be expressed in a few
hundred lines of config, I would say things like service discovery are less
useful I wouldn't spend time up front providing support for service
discovery.

cheers,
Chandru



>
> Thomas
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From eric.pailleau@REDACTED  Sat May 16 15:01:19 2015
From: eric.pailleau@REDACTED (PAILLEAU Eric)
Date: Sat, 16 May 2015 15:01:19 +0200
Subject: [erlang-questions] [ANN] geas 1.0.0 released
Message-ID: <55573F9F.5060600@wanadoo.fr>

Dear Erlangers,

This message announces the release of the unpretentious module :

geas 1.0.0  "Guess Erlang Application Scattering"

https://github.com/crownedgrouse/geas

Available in erlang.mk index too.

This module can give you useful informations, quickly, from beam files, 
even without source code. Mainly :

- if at least one module in project is compiled native and for which 
architecture
- What was the compiler version
- What are the Erlang versions compatible with this erts version

Example : http://erlang.org/pipermail/erlang-questions/2015-May/084481.html

Enjoy

Eric




From radek@REDACTED  Sat May 16 15:30:13 2015
From: radek@REDACTED (Rad Gruchalski)
Date: Sat, 16 May 2015 15:30:13 +0200
Subject: [erlang-questions] UDP multicast on Raspberry Pi
In-Reply-To: 
References: 
 
Message-ID: <699999C48B374C0FA700EE718B50197B@gruchalski.com>

Unfortunately, I get exactly the same result.  

No data received on RPi when using multicast group.
It?s very strange as data published over multicast from RPi arrives at the destination. Only RPi isn?t getting any data.
Just to make sure I was not having any general UDP issues - this is is the no multicast server running on RPi:

-module(udp_nomulticast).

-behaviour(gen_server).

-export([start_link/0, stop/0]).
-export([init/1, handle_call/3, handle_cast/2, handle_info/2, code_change/3, terminate/2]).

start_link() ->
  gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).

stop() -> gen_server:cast(?MODULE, stop).

init([]) ->
  Port = 6666,
  IfaceIp = {0,0,0,0},
  {ok, OverlaySocket} = gen_udp:open(Port, [ binary,
                                             {ip, IfaceIp} ] ),
  { ok, OverlaySocket }.

handle_info({udp, _ClientSocket, _ClientIp, _ClientPort, Msg}, State) ->
  error_logger:info_msg("Received multicast data: ~p", [ Msg ]),
  {noreply, State}.

handle_call( _, _From, State) ->
  { reply, ok, State }.

code_change(_OldVsn, State, _Extra) ->
  {ok, State}.

terminate(_, _) ->
  ok.

handle_cast(stop, LoopData) ->
  {noreply, LoopData}.


And the client sending to it from the dev box (RPi IP on the local network is 10.128.30.23):

-module(nmcc).

-export([run/0]).

run() ->
  Port = 6666,
  IfaceIp = {0,0,0,0},
  RPiIp = {10,128,30,23},
  {ok, OverlaySocket} = gen_udp:open(Port, [ binary,
                                             {ip, IfaceIp} ] ),
  gen_udp:send( OverlaySocket, RPiIp, Port, <<"some random datas">> ),
  gen_udp:close( OverlaySocket ).


This is arriving just fine:

pi@REDACTED ~ $ erl
Erlang/OTP 17 [erts-6.4] [source] [smp:4:4] [async-threads:10] [kernel-poll:false]

Eshell V6.4  (abort with ^G)
1> c("udp_nomulticast").
{ok,udp_nomulticast}
2> udp_nomulticast:start_link().
{ok,<0.40.0>}
3>
=INFO REPORT==== 16-May-2015::15:05:35 ===
Received multicast data: <<"some random datas?>>


I?m often jumping too fast to conclusions but I genuinely believe this could be some Erlang related problem but I am not sure how to diagnose it.

I tried multiple combinations, binding the server to {0,0,0,0} or multicast IP, different multicast groups. Result is always the same, traffic not arriving at RPi when using multicast UDP. What?s the best way to elevate? File a bug?










Kind regards,

Radek Gruchalski

radek@REDACTED (mailto:radek@REDACTED)
 (mailto:radek@REDACTED)
de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)

Confidentiality:
This communication is intended for the above-named person and may be confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.



On Saturday, 16 May 2015 at 08:25, Max Lapshin wrote:

> Replace  {ip, MulticastIp},  with {ip, IfaceIp},
>  
>  
>  


-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From peter@REDACTED  Sat May 16 16:05:16 2015
From: peter@REDACTED (Peter Membrey)
Date: Sat, 16 May 2015 22:05:16 +0800 (HKT)
Subject: [erlang-questions] UDP multicast on Raspberry Pi
In-Reply-To: <699999C48B374C0FA700EE718B50197B@gruchalski.com>
References: 
 
 <699999C48B374C0FA700EE718B50197B@gruchalski.com>
Message-ID: <286369735.1072.1431785116362.JavaMail.zimbra@membrey.hk>

Hi Max, 

I'm afraid I don't have a direct answer for your problem, but here's some things that might help you track it down. 

First, after running your Erlang listener run the following command: 

$ netstat -g 

This will show you all of the multicast groups that you are subscribing to and on which interface. Make sure that you see the group you care about in that list. 

If it is in that list, then start a tcpdump session on that interface as root and take a look at what goes on the wire i.e: 

# tcpdump -i  port  

# tcpdump -i eth0 port 10000 

This lot will tell you if your Pi is looking for multicast and whether or not they're hitting the Pi itself. In other words this should help you isolate the problem. 

In addition you can try: 

# tcpdump -i eth0 igmp 

Which should (I'm going from memory) isolate all igmp traffic. You should see packets sent when you join and leave a group. So if you start this command first, then start your Erlang app, you should see an IGMP join going out. 

Hope some of this helps! 

Kind Regards, 

Peter Membrey 


From: "Rad Gruchalski"  
To: "Max Lapshin"  
Cc: "erlang questions"  
Sent: Saturday, 16 May, 2015 21:30:13 
Subject: Re: [erlang-questions] UDP multicast on Raspberry Pi 

Unfortunately, I get exactly the same result. 

No data received on RPi when using multicast group. 
It?s very strange as data published over multicast from RPi arrives at the destination. Only RPi isn?t getting any data. 
Just to make sure I was not having any general UDP issues - this is is the no multicast server running on RPi: 

-module(udp_nomulticast). 

-behaviour(gen_server). 

-export([start_link/0, stop/0]). 
-export([init/1, handle_call/3, handle_cast/2, handle_info/2, code_change/3, terminate/2]). 

start_link() -> 
gen_server:start_link({local, ?MODULE}, ?MODULE, [], []). 

stop() -> gen_server:cast(?MODULE, stop). 

init([]) -> 
Port = 6666, 
IfaceIp = {0,0,0,0}, 
{ok, OverlaySocket} = gen_udp:open(Port, [ binary, 
{ip, IfaceIp} ] ), 
{ ok, OverlaySocket }. 

handle_info({udp, _ClientSocket, _ClientIp, _ClientPort, Msg}, State) -> 
error_logger:info_msg("Received multicast data: ~p", [ Msg ]), 
{noreply, State}. 

handle_call( _, _From, State) -> 
{ reply, ok, State }. 

code_change(_OldVsn, State, _Extra) -> 
{ok, State}. 

terminate(_, _) -> 
ok. 

handle_cast(stop, LoopData) -> 
{noreply, LoopData}. 

And the client sending to it from the dev box (RPi IP on the local network is 10.128.30.23): 

-module(nmcc). 

-export([run/0]). 

run() -> 
Port = 6666, 
IfaceIp = {0,0,0,0}, 
RPiIp = {10,128,30,23}, 
{ok, OverlaySocket} = gen_udp:open(Port, [ binary, 
{ip, IfaceIp} ] ), 
gen_udp:send( OverlaySocket, RPiIp, Port, <<"some random datas">> ), 
gen_udp:close( OverlaySocket ). 

This is arriving just fine: 

pi@REDACTED ~ $ erl 
Erlang/OTP 17 [erts-6.4] [source] [smp:4:4] [async-threads:10] [kernel-poll:false] 

Eshell V6.4 (abort with ^G) 
1> c("udp_nomulticast"). 
{ok,udp_nomulticast} 
2> udp_nomulticast:start_link(). 
{ok,<0.40.0>} 
3> 
=INFO REPORT==== 16-May-2015::15:05:35 === 
Received multicast data: <<"some random datas?>> 

I?m often jumping too fast to conclusions but I genuinely believe this could be some Erlang related problem but I am not sure how to diagnose it. 

I tried multiple combinations, binding the server to {0,0,0,0} or multicast IP, different multicast groups. Result is always the same, traffic not arriving at RPi when using multicast UDP. What?s the best way to elevate? File a bug? 








Kind regards, 
Radek Gruchalski 
radek@REDACTED 
de.linkedin.com/in/radgruchalski/ 

Confidentiality: 
This communication is intended for the above-named person and may be confidential and/or legally privileged. 
If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. 


On Saturday, 16 May 2015 at 08:25, Max Lapshin wrote: 


Replace {ip, MulticastIp}, with {ip, IfaceIp}, 





_______________________________________________ 
erlang-questions mailing list 
erlang-questions@REDACTED 
http://erlang.org/mailman/listinfo/erlang-questions 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From radek@REDACTED  Sat May 16 19:31:46 2015
From: radek@REDACTED (Rad Gruchalski)
Date: Sat, 16 May 2015 19:31:46 +0200
Subject: [erlang-questions] UDP multicast on Raspberry Pi
In-Reply-To: <286369735.1072.1431785116362.JavaMail.zimbra@membrey.hk>
References: 
 
 <699999C48B374C0FA700EE718B50197B@gruchalski.com>
 <286369735.1072.1431785116362.JavaMail.zimbra@membrey.hk>
Message-ID: <061B13BCD4054C90B6B49F9ED7256E07@gruchalski.com>

Hi Peter,  

Thank you for your suggestions. I?m getting somewhere but not exactly sure where to, yet.

I?m running this on RPi:

tcpdump -i eth0 igmp

When I join multicast group from my dev box, I get the following in the tcpdump output:

19:16:48.370585 IP drone?..german-fiber.net > 239.0.0.251: igmp v2 report 239.0.0.251

When I join with the RPi itself, I see:

19:18:15.245935 IP raspberrypi?..german-fiber.net > igmp.mcast.net: igmp v3 report, 1 group record(s)
19:18:16.035929 IP raspberrypi?..german-fiber.net > igmp.mcast.net: igmp v3 report, 1 group record(s)


I read is that Ubuntu/Debian IGMP default version is 3. OS X (which the host is), defaults to 3 as well:

$ sysctl net.inet.igmp.default_version
net.inet.igmp.default_version: 3


Erlang (?) for whatever reason announces multicast with version 2 from OS X but version 3 on Raspbian. The weird thing is that I also get double join.

I tried changing the IGMP version on Raspbian to 2 by using:

echo ?2? > /proc/sys/net/ipv4/conf/eth0/force_igmp_version

After the change, RPi join looks like this:

19:24:14.115945 IP raspberrypi?..german-fiber.net > 239.0.0.251: igmp v2 report 239.0.0.251
19:24:14.395920 IP raspberrypi?..german-fiber.net > 239.0.0.251: igmp v2 report 239.0.0.251
19:24:19.185936 IP raspberrypi?..german-fiber.net > 239.0.0.251: igmp v2 report 239.0.0.251


For whatever reason there are 3 joins. I?m not quite sure where the problem is. It would appear that Erlang add_membership is IGMPv2? Or would that be handled by the kernel? If it is handled by the kernel, why is it not v3 as suggested by OS X sysctl default? Following on that one, why would this behaviour be different on Raspbian?










Kind regards,

Radek Gruchalski

radek@REDACTED (mailto:radek@REDACTED)
 (mailto:radek@REDACTED)
de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)

Confidentiality:
This communication is intended for the above-named person and may be confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.



On Saturday, 16 May 2015 at 16:05, Peter Membrey wrote:

> Hi Max,
>  
> I'm afraid I don't have a direct answer for your problem, but here's some things that might help you track it down.
>  
> First, after running your Erlang listener run the following command:
>  
> $ netstat -g
>  
> This will show you all of the multicast groups that you are subscribing to and on which interface. Make sure that you see the group you care about in that list.
>  
> If it is in that list, then start a tcpdump session on that interface as root and take a look at what goes on the wire i.e:
>  
> # tcpdump -i  port 
>  
> # tcpdump -i eth0 port 10000
>  
> This lot will tell you if your Pi is looking for multicast and whether or not they're hitting the Pi itself. In other words this should help you isolate the problem.
>  
> In addition you can try:
>  
> # tcpdump -i eth0 igmp
>  
> Which should (I'm going from memory) isolate all igmp traffic. You should see packets sent when you join and leave a group. So if you start this command first, then start your Erlang app, you should see an IGMP join going out.
>  
> Hope some of this helps!
>  
> Kind Regards,
>  
> Peter Membrey
>  
> From: "Rad Gruchalski" 
> To: "Max Lapshin" 
> Cc: "erlang questions" 
> Sent: Saturday, 16 May, 2015 21:30:13
> Subject: Re: [erlang-questions] UDP multicast on Raspberry Pi
>  
> Unfortunately, I get exactly the same result.  
> No data received on RPi when using multicast group.
> It?s very strange as data published over multicast from RPi arrives at the destination. Only RPi isn?t getting any data.
> Just to make sure I was not having any general UDP issues - this is is the no multicast server running on RPi:
>  
> -module(udp_nomulticast).
>  
> -behaviour(gen_server).
>  
> -export([start_link/0, stop/0]).
> -export([init/1, handle_call/3, handle_cast/2, handle_info/2, code_change/3, terminate/2]).
>  
> start_link() ->
>   gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
>  
> stop() -> gen_server:cast(?MODULE, stop).
>  
> init([]) ->
>   Port = 6666,
>   IfaceIp = {0,0,0,0},
>   {ok, OverlaySocket} = gen_udp:open(Port, [ binary,
>                                              {ip, IfaceIp} ] ),
>   { ok, OverlaySocket }.
>  
> handle_info({udp, _ClientSocket, _ClientIp, _ClientPort, Msg}, State) ->
>   error_logger:info_msg("Received multicast data: ~p", [ Msg ]),
>   {noreply, State}.
>  
> handle_call( _, _From, State) ->
>   { reply, ok, State }.
>  
> code_change(_OldVsn, State, _Extra) ->
>   {ok, State}.
>  
> terminate(_, _) ->
>   ok.
>  
> handle_cast(stop, LoopData) ->
>   {noreply, LoopData}.
>  
>  
> And the client sending to it from the dev box (RPi IP on the local network is 10.128.30.23):
> -module(nmcc).
>  
> -export([run/0]).
>  
> run() ->
>   Port = 6666,
>   IfaceIp = {0,0,0,0},
>   RPiIp = {10,128,30,23},
>   {ok, OverlaySocket} = gen_udp:open(Port, [ binary,
>                                              {ip, IfaceIp} ] ),
>   gen_udp:send( OverlaySocket, RPiIp, Port, <<"some random datas">> ),
>   gen_udp:close( OverlaySocket ).
>  
>  
> This is arriving just fine:
> pi@REDACTED ~ $ erl
> Erlang/OTP 17 [erts-6.4] [source] [smp:4:4] [async-threads:10] [kernel-poll:false]
>  
> Eshell V6.4  (abort with ^G)
> 1> c("udp_nomulticast").
> {ok,udp_nomulticast}
> 2> udp_nomulticast:start_link().
> {ok,<0.40.0>}
> 3>
> =INFO REPORT==== 16-May-2015::15:05:35 ===
> Received multicast data: <<"some random datas?>>
>  
>  
> I?m often jumping too fast to conclusions but I genuinely believe this could be some Erlang related problem but I am not sure how to diagnose it.
> I tried multiple combinations, binding the server to {0,0,0,0} or multicast IP, different multicast groups. Result is always the same, traffic not arriving at RPi when using multicast UDP. What?s the best way to elevate? File a bug?
>  
>  
>  
>  
>  
>  
>  
>  
>  
>  
> Kind regards,

> Radek Gruchalski
> 
radek@REDACTED (mailto:radek@REDACTED)
 (mailto:radek@REDACTED)
> de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)
>  
> Confidentiality:
> This communication is intended for the above-named person and may be confidential and/or legally privileged.
> If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.
>  
>  
>  
> On Saturday, 16 May 2015 at 08:25, Max Lapshin wrote:
>  
> > Replace  {ip, MulticastIp},  with {ip, IfaceIp},
> >  
> >  
> >  
>  
>  
>  
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED (mailto:erlang-questions@REDACTED)
> http://erlang.org/mailman/listinfo/erlang-questions
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED (mailto:erlang-questions@REDACTED)
> http://erlang.org/mailman/listinfo/erlang-questions
>  
>  


-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From radek@REDACTED  Sat May 16 19:57:40 2015
From: radek@REDACTED (Rad Gruchalski)
Date: Sat, 16 May 2015 19:57:40 +0200
Subject: [erlang-questions] UDP multicast on Raspberry Pi
In-Reply-To: <061B13BCD4054C90B6B49F9ED7256E07@gruchalski.com>
References: 
 
 <699999C48B374C0FA700EE718B50197B@gruchalski.com>
 <286369735.1072.1431785116362.JavaMail.zimbra@membrey.hk>
 <061B13BCD4054C90B6B49F9ED7256E07@gruchalski.com>
Message-ID: <174797C397A2405AB67088C4468977C2@gruchalski.com>

Additionally, even when both machines have the udp_multicast program running and both were recorded by tcpdump (RPi 3 times), doing:  

$ ping 239.0.0.251

Results in:

PING 239.0.0.251 (239.0.0.251): 56 data bytes
64 bytes from 10.128.30.24: icmp_seq=0 ttl=64 time=0.101 ms
64 bytes from 10.128.30.24: icmp_seq=1 ttl=64 time=0.079 ms
64 bytes from 10.128.30.24: icmp_seq=2 ttl=64 time=0.125 ms
64 bytes from 10.128.30.24: icmp_seq=3 ttl=64 time=0.200 ms
64 bytes from 10.128.30.24: icmp_seq=4 ttl=64 time=0.047 ms
64 bytes from 10.128.30.24: icmp_seq=5 ttl=64 time=0.099 ms
64 bytes from 10.128.30.24: icmp_seq=6 ttl=64 time=0.087 ms


Only one host is replying. RPi isn?t in that group.  










Kind regards,

Radek Gruchalski

radek@REDACTED (mailto:radek@REDACTED)
 (mailto:radek@REDACTED)
de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)

Confidentiality:
This communication is intended for the above-named person and may be confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.



On Saturday, 16 May 2015 at 19:31, Rad Gruchalski wrote:

> Hi Peter,  
>  
> Thank you for your suggestions. I?m getting somewhere but not exactly sure where to, yet.
>  
> I?m running this on RPi:
>  
> tcpdump -i eth0 igmp
>  
> When I join multicast group from my dev box, I get the following in the tcpdump output:
>  
> 19:16:48.370585 IP drone?..german-fiber.net (http://german-fiber.net) > 239.0.0.251: igmp v2 report 239.0.0.251
>  
> When I join with the RPi itself, I see:
>  
> 19:18:15.245935 IP raspberrypi?..german-fiber.net (http://german-fiber.net) > igmp.mcast.net (http://igmp.mcast.net): igmp v3 report, 1 group record(s)
> 19:18:16.035929 IP raspberrypi?..german-fiber.net (http://german-fiber.net) > igmp.mcast.net (http://igmp.mcast.net): igmp v3 report, 1 group record(s)
>  
>  
> I read is that Ubuntu/Debian IGMP default version is 3. OS X (which the host is), defaults to 3 as well:
>  
> $ sysctl net.inet.igmp.default_version
> net.inet.igmp.default_version: 3
>  
>  
> Erlang (?) for whatever reason announces multicast with version 2 from OS X but version 3 on Raspbian. The weird thing is that I also get double join.
>  
> I tried changing the IGMP version on Raspbian to 2 by using:
>  
> echo ?2? > /proc/sys/net/ipv4/conf/eth0/force_igmp_version
>  
> After the change, RPi join looks like this:
>  
> 19:24:14.115945 IP raspberrypi?..german-fiber.net (http://german-fiber.net) > 239.0.0.251: igmp v2 report 239.0.0.251
> 19:24:14.395920 IP raspberrypi?..german-fiber.net (http://german-fiber.net) > 239.0.0.251: igmp v2 report 239.0.0.251
> 19:24:19.185936 IP raspberrypi?..german-fiber.net (http://german-fiber.net) > 239.0.0.251: igmp v2 report 239.0.0.251
>  
>  
> For whatever reason there are 3 joins. I?m not quite sure where the problem is. It would appear that Erlang add_membership is IGMPv2? Or would that be handled by the kernel? If it is handled by the kernel, why is it not v3 as suggested by OS X sysctl default? Following on that one, why would this behaviour be different on Raspbian?
>  
>  
>  
>  
>  
>  
>  
>  
>  
>  
> Kind regards,

> Radek Gruchalski
> 
radek@REDACTED (mailto:radek@REDACTED)
 (mailto:radek@REDACTED)
> de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)
>  
> Confidentiality:
> This communication is intended for the above-named person and may be confidential and/or legally privileged.
> If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.
>  
>  
>  
> On Saturday, 16 May 2015 at 16:05, Peter Membrey wrote:
>  
> > Hi Max,
> >  
> > I'm afraid I don't have a direct answer for your problem, but here's some things that might help you track it down.
> >  
> > First, after running your Erlang listener run the following command:
> >  
> > $ netstat -g
> >  
> > This will show you all of the multicast groups that you are subscribing to and on which interface. Make sure that you see the group you care about in that list.
> >  
> > If it is in that list, then start a tcpdump session on that interface as root and take a look at what goes on the wire i.e:
> >  
> > # tcpdump -i  port 
> >  
> > # tcpdump -i eth0 port 10000
> >  
> > This lot will tell you if your Pi is looking for multicast and whether or not they're hitting the Pi itself. In other words this should help you isolate the problem.
> >  
> > In addition you can try:
> >  
> > # tcpdump -i eth0 igmp
> >  
> > Which should (I'm going from memory) isolate all igmp traffic. You should see packets sent when you join and leave a group. So if you start this command first, then start your Erlang app, you should see an IGMP join going out.
> >  
> > Hope some of this helps!
> >  
> > Kind Regards,
> >  
> > Peter Membrey
> >  
> > From: "Rad Gruchalski" 
> > To: "Max Lapshin" 
> > Cc: "erlang questions" 
> > Sent: Saturday, 16 May, 2015 21:30:13
> > Subject: Re: [erlang-questions] UDP multicast on Raspberry Pi
> >  
> > Unfortunately, I get exactly the same result.  
> > No data received on RPi when using multicast group.
> > It?s very strange as data published over multicast from RPi arrives at the destination. Only RPi isn?t getting any data.
> > Just to make sure I was not having any general UDP issues - this is is the no multicast server running on RPi:
> >  
> > -module(udp_nomulticast).
> >  
> > -behaviour(gen_server).
> >  
> > -export([start_link/0, stop/0]).
> > -export([init/1, handle_call/3, handle_cast/2, handle_info/2, code_change/3, terminate/2]).
> >  
> > start_link() ->
> >   gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
> >  
> > stop() -> gen_server:cast(?MODULE, stop).
> >  
> > init([]) ->
> >   Port = 6666,
> >   IfaceIp = {0,0,0,0},
> >   {ok, OverlaySocket} = gen_udp:open(Port, [ binary,
> >                                              {ip, IfaceIp} ] ),
> >   { ok, OverlaySocket }.
> >  
> > handle_info({udp, _ClientSocket, _ClientIp, _ClientPort, Msg}, State) ->
> >   error_logger:info_msg("Received multicast data: ~p", [ Msg ]),
> >   {noreply, State}.
> >  
> > handle_call( _, _From, State) ->
> >   { reply, ok, State }.
> >  
> > code_change(_OldVsn, State, _Extra) ->
> >   {ok, State}.
> >  
> > terminate(_, _) ->
> >   ok.
> >  
> > handle_cast(stop, LoopData) ->
> >   {noreply, LoopData}.
> >  
> >  
> > And the client sending to it from the dev box (RPi IP on the local network is 10.128.30.23):
> > -module(nmcc).
> >  
> > -export([run/0]).
> >  
> > run() ->
> >   Port = 6666,
> >   IfaceIp = {0,0,0,0},
> >   RPiIp = {10,128,30,23},
> >   {ok, OverlaySocket} = gen_udp:open(Port, [ binary,
> >                                              {ip, IfaceIp} ] ),
> >   gen_udp:send( OverlaySocket, RPiIp, Port, <<"some random datas">> ),
> >   gen_udp:close( OverlaySocket ).
> >  
> >  
> > This is arriving just fine:
> > pi@REDACTED ~ $ erl
> > Erlang/OTP 17 [erts-6.4] [source] [smp:4:4] [async-threads:10] [kernel-poll:false]
> >  
> > Eshell V6.4  (abort with ^G)
> > 1> c("udp_nomulticast").
> > {ok,udp_nomulticast}
> > 2> udp_nomulticast:start_link().
> > {ok,<0.40.0>}
> > 3>
> > =INFO REPORT==== 16-May-2015::15:05:35 ===
> > Received multicast data: <<"some random datas?>>
> >  
> >  
> > I?m often jumping too fast to conclusions but I genuinely believe this could be some Erlang related problem but I am not sure how to diagnose it.
> > I tried multiple combinations, binding the server to {0,0,0,0} or multicast IP, different multicast groups. Result is always the same, traffic not arriving at RPi when using multicast UDP. What?s the best way to elevate? File a bug?
> >  
> >  
> >  
> >  
> >  
> >  
> >  
> >  
> >  
> >  
> > Kind regards,

> > Radek Gruchalski
> > 
radek@REDACTED (mailto:radek@REDACTED)
 (mailto:radek@REDACTED)
> > de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)
> >  
> > Confidentiality:
> > This communication is intended for the above-named person and may be confidential and/or legally privileged.
> > If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.
> >  
> >  
> >  
> > On Saturday, 16 May 2015 at 08:25, Max Lapshin wrote:
> >  
> > > Replace  {ip, MulticastIp},  with {ip, IfaceIp},
> > >  
> > >  
> > >  
> >  
> >  
> >  
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED)
> > http://erlang.org/mailman/listinfo/erlang-questions
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED)
> > http://erlang.org/mailman/listinfo/erlang-questions
> >  
> >  
> >  
>  
>  

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From mononcqc@REDACTED  Sat May 16 20:12:43 2015
From: mononcqc@REDACTED (Fred Hebert)
Date: Sat, 16 May 2015 14:12:43 -0400
Subject: [erlang-questions] Erlang microservice architecture ?
In-Reply-To: 
References: 
Message-ID: <20150516181242.GB1700@ferdmbp.local>

On 05/15, Thomas Elsgaard wrote:
>If you should recommend the architecture for an microservice 
>implementatio with around 20 Erlang microservices implemented on 
>separate servers, how would you implement it ?
>

I would implement 20 standalone OTP applications and put them onto the 
same server, same VM. Move them out of the VM and to a different server 
if/when the need arises.

Regards,
Fred.


From g@REDACTED  Sat May 16 20:35:42 2015
From: g@REDACTED (Garrett Smith)
Date: Sat, 16 May 2015 13:35:42 -0500
Subject: [erlang-questions] Erlang microservice architecture ?
In-Reply-To: <20150516181242.GB1700@ferdmbp.local>
References: 
 <20150516181242.GB1700@ferdmbp.local>
Message-ID: 

On Sat, May 16, 2015 at 1:12 PM, Fred Hebert  wrote:
> On 05/15, Thomas Elsgaard wrote:
>>
>> If you should recommend the architecture for an microservice implementatio
>> with around 20 Erlang microservices implemented on separate servers, how
>> would you implement it ?
>>
>
> I would implement 20 standalone OTP applications and put them onto the same
> server, same VM. Move them out of the VM and to a different server if/when
> the need arises.

Except Thomas asked about micro services. I think you're talking about
nano services. That hasn't been discovered yet.

Garrett


From thomas.elsgaard@REDACTED  Sat May 16 21:17:39 2015
From: thomas.elsgaard@REDACTED (Thomas Elsgaard)
Date: Sat, 16 May 2015 17:17:39 -0200
Subject: [erlang-questions] Erlang microservice architecture ?
In-Reply-To: 
References: 
 <20150516181242.GB1700@ferdmbp.local>
 
Message-ID: 

>
> Except Thomas asked about micro services. I think you're talking about
> nano services. That hasn't been discovered yet.
>
> Garrett
>

I should maybe add that this is for low volume but complex service
activation in the telco domain, the current approach has been the
traditional SOA solution with an ESB and process orchestration (all Java
based), but the number of services has increased, and the ESB (service bus
+ message queue) is suffering from memory leaks and other stability
issues which is hard to troubleshoot, and in general, the solution is just
complex.

Now the buzz is microservices (someone might call it SOA 2.0) but i would
like to split the architecture up in smaller isolated services, and of
course Erlang based. My original idea was to split the services up on
separate servers, install yaws on each server, and then install Erlang
applications as yapps on each server. This means that i can autoinstall the
servers just with yaws, build Erlang apps and deployed them on the servers
as needed. I could host several yapps on the same server, and if needed,
move individual yapps to another server.

The communication between services would then be http based.

But is there an smarter solution ? Chandru had some good input for why not
using RPC (back pressure issue). Protobuffs via sockets could also work,
but requires some more work.

Fred is defiantly having some good input about just having all the
applications on the same VM, I could do that, it's better than the current
 Java ESB monster, but if I then decide to split the services to different
servers, it will require some re-write.

Anyone who has done anything similar ? I do not need high performance, but
i need to handle complicated business logic between many services, and of
course keeping it fault tolerant ;-)

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From sergej.jurecko@REDACTED  Sat May 16 21:27:49 2015
From: sergej.jurecko@REDACTED (Sergej Jurecko)
Date: Sat, 16 May 2015 21:27:49 +0200
Subject: [erlang-questions] Erlang microservice architecture ?
In-Reply-To: 
References: 
 <20150516181242.GB1700@ferdmbp.local>
 
 
Message-ID: <26322861-91CD-4494-BF14-87246C78FEB2@gmail.com>

What about thrift? My experience with it has been pretty positive. 


Sergej

On 16 May 2015, at 21:17, Thomas Elsgaard  wrote:

> 
> 
> Except Thomas asked about micro services. I think you're talking about
> nano services. That hasn't been discovered yet.
> 
> Garrett
> 
> I should maybe add that this is for low volume but complex service activation in the telco domain, the current approach has been the traditional SOA solution with an ESB and process orchestration (all Java based), but the number of services has increased, and the ESB (service bus + message queue) is suffering from memory leaks and other stability issues which is hard to troubleshoot, and in general, the solution is just complex.
> 
> Now the buzz is microservices (someone might call it SOA 2.0) but i would like to split the architecture up in smaller isolated services, and of course Erlang based. My original idea was to split the services up on separate servers, install yaws on each server, and then install Erlang applications as yapps on each server. This means that i can autoinstall the servers just with yaws, build Erlang apps and deployed them on the servers as needed. I could host several yapps on the same server, and if needed, move individual yapps to another server.
> 
> The communication between services would then be http based. 
> 
> But is there an smarter solution ? Chandru had some good input for why not using RPC (back pressure issue). Protobuffs via sockets could also work, but requires some more work.
> 
> Fred is defiantly having some good input about just having all the applications on the same VM, I could do that, it's better than the current  Java ESB monster, but if I then decide to split the services to different servers, it will require some re-write.
> 
> Anyone who has done anything similar ? I do not need high performance, but i need to handle complicated business logic between many services, and of course keeping it fault tolerant ;-)
> 
> Thomas
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From mjtruog@REDACTED  Sat May 16 21:56:31 2015
From: mjtruog@REDACTED (Michael Truog)
Date: Sat, 16 May 2015 12:56:31 -0700
Subject: [erlang-questions] Erlang microservice architecture ?
In-Reply-To: 
References: 
 <20150516181242.GB1700@ferdmbp.local>
 
 
Message-ID: <5557A0EF.1070503@gmail.com>

On 05/16/2015 12:17 PM, Thomas Elsgaard wrote:
>
>
>     Except Thomas asked about micro services. I think you're talking about
>     nano services. That hasn't been discovered yet.
>
>     Garrett
>
>
> I should maybe add that this is for low volume but complex service activation in the telco domain, the current approach has been the traditional SOA solution with an ESB and process orchestration (all Java based), but the number of services has increased, and the ESB (service bus + message queue) is suffering from memory leaks and other stability issues which is hard to troubleshoot, and in general, the solution is just complex.
>
> Now the buzz is microservices (someone might call it SOA 2.0) but i would like to split the architecture up in smaller isolated services, and of course Erlang based. My original idea was to split the services up on separate servers, install yaws on each server, and then install Erlang applications as yapps on each server. This means that i can autoinstall the servers just with yaws, build Erlang apps and deployed them on the servers as needed. I could host several yapps on the same server, and if needed, move individual yapps to another server.
CloudI (http://cloudi.org) is a good fit to this situation if you want all the microservices to be fault-tolerant with service discovery in an AP-type distributed system (that is master-less). Each service only contains transient/temporary state that can be used for caching or related use.  Erlang services would use the cloudi_service behaviour which is similar to the gen_server OTP behaviour, but with many other features for its configuration and messaging.  If you only will be doing Erlang services, you could use the cloudi_core repository (https://github.com/CloudI/cloudi_core), otherwise if you need to create non-Erlang services also, use the main repository (https://github.com/CloudI/CloudI).

CloudI is using distributed Erlang to share service discovery information with other CloudI nodes and to exchange service requests with remote node services.  Each of the 20 services would want to have multiple instances created to be redundant (so by having the count_process set to > 1 for each service configuration entry of the 20 services) and would exist on a remote CloudI node for redundancy.
>
> The communication between services would then be http based.
If you used CloudI, you would be using service requests which contain a Request (body) and RequestInfo (request meta-data, e.g., HTTP headers) with additional parameters for various features (a unique transaction ID as a v1 UUID, a timeout that decrements based on any delays the service request encounters, a priority with 255 possible values).  Due to fault-tolerance concerns, service requests are randomly load balanced automatically among any services that subscribe to a single service name pattern that matches the service request's service name (which is why you want multiple processes for each of the 20 services, and CloudI handles that as a service configuration parameter).  If you want to exchange HTTP traffic, you can, the service request data is whatever you want it to be (in the RequestInfo/Request pair and the ResponseInfo/Response pair).  It is only if you are sending to an external service that you need to use binary data, to make sure the external service can 
utilize the data.
>
> But is there an smarter solution ? Chandru had some good input for why not using RPC (back pressure issue). Protobuffs via sockets could also work, but requires some more work.
>
> Fred is defiantly having some good input about just having all the applications on the same VM, I could do that, it's better than the current  Java ESB monster, but if I then decide to split the services to different servers, it will require some re-write.
With CloudI it would be simpler to reuse the Java source code by using the Java CloudI API and having CloudI execute the Java external services as an application server.  There is more information about why you would want this path at http://cloudi.org/faq.html#4_Erlang .  The CloudI source code is small when you consider everything is handled by the cloudi_core source code, the extra integration tests and example services help you get started and facilitate other usage and testing.
>
> Anyone who has done anything similar ? I do not need high performance, but i need to handle complicated business logic between many services, and of course keeping it fault tolerant ;-)
>
> Thomas
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From lostcolony@REDACTED  Sat May 16 21:57:56 2015
From: lostcolony@REDACTED (Christopher Phillips)
Date: Sat, 16 May 2015 15:57:56 -0400
Subject: [erlang-questions] erlang-questions Digest, Vol 217, Issue 9
In-Reply-To: 
References: 
Message-ID: 

>
>
> Date: Sat, 16 May 2015 17:17:39 -0200
> From: Thomas Elsgaard 
> To: Garrett Smith 
> Cc: "erlang-questions@REDACTED" 
> Subject: Re: [erlang-questions] Erlang microservice architecture ?
> Message-ID:
>         <
> CAKSYKuJLNc4NBrqCv0f_Kks52bDWKw-6m_7YVWB2zwo_DznX+g@REDACTED>
> Content-Type: text/plain; charset="iso-8859-1"
>
> >
> > Except Thomas asked about micro services. I think you're talking about
> > nano services. That hasn't been discovered yet.
> >
> > Garrett
> >
>
> I should maybe add that this is for low volume but complex service
> activation in the telco domain, the current approach has been the
> traditional SOA solution with an ESB and process orchestration (all Java
> based), but the number of services has increased, and the ESB (service bus
> + message queue) is suffering from memory leaks and other stability
> issues which is hard to troubleshoot, and in general, the solution is just
> complex.
>
> Now the buzz is microservices (someone might call it SOA 2.0) but i would
> like to split the architecture up in smaller isolated services, and of
> course Erlang based. My original idea was to split the services up on
> separate servers, install yaws on each server, and then install Erlang
> applications as yapps on each server. This means that i can autoinstall the
> servers just with yaws, build Erlang apps and deployed them on the servers
> as needed. I could host several yapps on the same server, and if needed,
> move individual yapps to another server.
>
> The communication between services would then be http based.
>
> But is there an smarter solution ? Chandru had some good input for why not
> using RPC (back pressure issue). Protobuffs via sockets could also work,
> but requires some more work.
>
> Fred is defiantly having some good input about just having all the
> applications on the same VM, I could do that, it's better than the current
>  Java ESB monster, but if I then decide to split the services to different
> servers, it will require some re-write.
>
> Anyone who has done anything similar ? I do not need high performance, but
> i need to handle complicated business logic between many services, and of
> course keeping it fault tolerant ;-)
>
> Thomas
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://erlang.org/pipermail/erlang-questions/attachments/20150516/cc99b101/attachment-0001.html
> >
>

Per Fred's suggestion, it's not entirely true that it would require a
rewrite. If it's sufficiently low volume, it could just be some
configuration changes.

That would require all inter-app communication passing through a well
defined interface, such as a registered gen_server. I.e., you have app_a,
app_b, and app_c, and each registers itself globally. All communication
from app_a to app_b goes through the registered app_b process, etc.

On a single node, where app_a, app_b, and app_c are all running on the same
VM, messages pass between them using the registered name.

If you later split them out across multiple nodes, spanning multiple
machines (such that app_a gets its own box, app_b gets its own box, and
app_c gets its own box, or any combination of sharing between them), you
can just ping the different nodes together, still globally register them,
and messages still pass between them using the registered name; nothing has
to change in the code. That's one of the nice things about Erlang's
distribution mechanism.

Now, if those servers need to be geographically distant, and/or netsplits
are going to be a common occurrence (and you need things to be partition
tolerant), etc, it gets tricky, and will require domain specific
considerations and implementations. But that's true regardless of what
technology or architecture you pick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From lostcolony@REDACTED  Sat May 16 22:00:00 2015
From: lostcolony@REDACTED (Christopher Phillips)
Date: Sat, 16 May 2015 16:00:00 -0400
Subject: [erlang-questions] Erlang microservice architecture ?
Message-ID: 

> Date: Sat, 16 May 2015 17:17:39 -0200
> From: Thomas Elsgaard 
> To: Garrett Smith 
> Cc: "erlang-questions@REDACTED" 
> Subject: Re: [erlang-questions] Erlang microservice architecture ?
> Message-ID:
>         <
> CAKSYKuJLNc4NBrqCv0f_Kks52bDWKw-6m_7YVWB2zwo_DznX+g@REDACTED>
> Content-Type: text/plain; charset="iso-8859-1"
>
> >
> > Except Thomas asked about micro services. I think you're talking about
> > nano services. That hasn't been discovered yet.
> >
> > Garrett
> >
>
> I should maybe add that this is for low volume but complex service
> activation in the telco domain, the current approach has been the
> traditional SOA solution with an ESB and process orchestration (all Java
> based), but the number of services has increased, and the ESB (service bus
> + message queue) is suffering from memory leaks and other stability
> issues which is hard to troubleshoot, and in general, the solution is just
> complex.
>
> Now the buzz is microservices (someone might call it SOA 2.0) but i would
> like to split the architecture up in smaller isolated services, and of
> course Erlang based. My original idea was to split the services up on
> separate servers, install yaws on each server, and then install Erlang
> applications as yapps on each server. This means that i can autoinstall the
> servers just with yaws, build Erlang apps and deployed them on the servers
> as needed. I could host several yapps on the same server, and if needed,
> move individual yapps to another server.
>
> The communication between services would then be http based.
>
> But is there an smarter solution ? Chandru had some good input for why not
> using RPC (back pressure issue). Protobuffs via sockets could also work,
> but requires some more work.
>
> Fred is defiantly having some good input about just having all the
> applications on the same VM, I could do that, it's better than the current
>  Java ESB monster, but if I then decide to split the services to different
> servers, it will require some re-write.
>
> Anyone who has done anything similar ? I do not need high performance, but
> i need to handle complicated business logic between many services, and of
> course keeping it fault tolerant ;-)
>
> Thomas
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://erlang.org/pipermail/erlang-questions/attachments/20150516/cc99b101/attachment-0001.html
> >
>

(Bah, broke the thread because forgot to replace the subject line. Kindly
disregard the other post)

Per Fred's suggestion, it's not entirely true that it would require a
rewrite. If it's sufficiently low volume, it could just be some
configuration changes.

That would require all inter-app communication passing through a well
defined interface, such as a registered gen_server. I.e., you have app_a,
app_b, and app_c, and each registers itself globally. All communication
from app_a to app_b goes through the registered app_b process, etc.

On a single node, where app_a, app_b, and app_c are all running on the same
VM, messages pass between them using the registered name.

If you later split them out across multiple nodes, spanning multiple
machines (such that app_a gets its own box, app_b gets its own box, and
app_c gets its own box, or any combination of sharing between them), you
can just ping the different nodes together, still globally register them,
and messages still pass between them using the registered name; nothing has
to change in the code. That's one of the nice things about Erlang's
distribution mechanism.

Now, if those servers need to be geographically distant, and/or netsplits
are going to be a common occurrence (and you need things to be partition
tolerant), etc, it gets tricky, and will require domain specific
considerations and implementations. But that's true regardless of what
technology or architecture you pick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From g@REDACTED  Sat May 16 22:34:19 2015
From: g@REDACTED (Garrett Smith)
Date: Sat, 16 May 2015 15:34:19 -0500
Subject: [erlang-questions] Erlang microservice architecture ?
In-Reply-To: 
References: 
 <20150516181242.GB1700@ferdmbp.local>
 
 
Message-ID: 

On Sat, May 16, 2015 at 2:17 PM, Thomas Elsgaard
 wrote:
>
>>
>> Except Thomas asked about micro services. I think you're talking about
>> nano services. That hasn't been discovered yet.
>>
>> Garrett
>
>
> I should maybe add that this is for low volume but complex service
> activation in the telco domain, the current approach has been the
> traditional SOA solution with an ESB and process orchestration (all Java
> based), but the number of services has increased, and the ESB (service bus +
> message queue) is suffering from memory leaks and other stability issues
> which is hard to troubleshoot, and in general, the solution is just complex.
>
> Now the buzz is microservices (someone might call it SOA 2.0) but i would
> like to split the architecture up in smaller isolated services, and of
> course Erlang based. My original idea was to split the services up on
> separate servers, install yaws on each server, and then install Erlang
> applications as yapps on each server. This means that i can autoinstall the
> servers just with yaws, build Erlang apps and deployed them on the servers
> as needed. I could host several yapps on the same server, and if needed,
> move individual yapps to another server.

I'd go with Fred's suggestion with 100% certainty that it's the right course.

Wait until you are driven with real problems before separating anything.

You can always, trivially separate OTP processes and put them on
separate servers, either by using Erlang distribution or wrapping each
in some socket interface. Some have suggest Thrift or other
frameworks. Why? What's driving this?

> The communication between services would then be http based.
>
> But is there an smarter solution ? Chandru had some good input for why not
> using RPC (back pressure issue). Protobuffs via sockets could also work, but
> requires some more work.

Why? Is it working, in production with known problems that you can point to?

> Fred is defiantly having some good input about just having all the
> applications on the same VM, I could do that, it's better than the current
> Java ESB monster, but if I then decide to split the services to different
> servers, it will require some re-write.

No - it will require some wrapping (additional code). But you're
proposing to do this *now* rather than put it off until you you know
you really need it. Again, why?

> Anyone who has done anything similar ? I do not need high performance, but i
> need to handle complicated business logic between many services, and of
> course keeping it fault tolerant ;-)

I'd get it working in the most direct possible way. If you're writing
OTP compliant systems, your system will be fault tolerant, at least
reasonably so out-of-the-box without any thought at all.

Once it's working and running in production in some capacity - note
the *actual* problems, then tackle each one.

Not having microservices is not an actual problem.

Garrett


From lloyd@REDACTED  Sat May 16 22:47:03 2015
From: lloyd@REDACTED (Lloyd R. Prentice)
Date: Sat, 16 May 2015 16:47:03 -0400
Subject: [erlang-questions] Erlang on Arduino - ArduinoBoardYun?
Message-ID: 

Is it possible, or has anyone installed Erlang on the Arduino Yun? And if so, how would it be done?

http://www.arduino.cc/en/Main/ArduinoBoardYun?from=Main.ArduinoYUN

If possible, it seems that this would be a cool way to get sensor data into an Erlang system.

All the best,

LRP


Sent from my iPad

From chandrashekhar.mullaparthi@REDACTED  Sat May 16 23:00:00 2015
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Sat, 16 May 2015 22:00:00 +0100
Subject: [erlang-questions] Erlang microservice architecture ?
In-Reply-To: <20150516181242.GB1700@ferdmbp.local>
References: 
 <20150516181242.GB1700@ferdmbp.local>
Message-ID: 

On 16 May 2015 at 19:12, Fred Hebert  wrote:

> On 05/15, Thomas Elsgaard wrote:
>
>> If you should recommend the architecture for an microservice
>> implementatio with around 20 Erlang microservices implemented on separate
>> servers, how would you implement it ?
>>
>>
> I would implement 20 standalone OTP applications and put them onto the
> same server, same VM. Move them out of the VM and to a different server
> if/when the need arises.
>
>
This can work. We did this with great success in one of my previous jobs.
About 20 different erlang applications, were sharing the same VM and
reusing many of the protocol interfaces. All of them were running on a
500MHz Sparc server handling about 5 million SMS a day. The best part was
that deploying a new (micro) service was simply a matter of adding a new
application into the lib directory, tweak a little bit of config and we had
a new service live. No servers to setup, firewalls to open...

cheers
Chandru
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From pierrefenoll@REDACTED  Sat May 16 23:05:12 2015
From: pierrefenoll@REDACTED (Pierre Fenoll)
Date: Sat, 16 May 2015 14:05:12 -0700
Subject: [erlang-questions] =?utf-8?q?=5BANN=5D_Other_Erldocs_=CE=B1?=
In-Reply-To: <20150408122556.GA4884@ferdair.local>
References: 
 <20150408122556.GA4884@ferdair.local>
Message-ID: 

After some more work and fixes, the index of 17+k Erlang projects is
updated daily.
Please try to find your own projects over http://other.erldocs.com/
If you get a 404, click on the "file an issue" link and Travis will try and
add your app to the index!

Thanks for all your comments


Cheers,
-- 
Pierre Fenoll
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jesper.louis.andersen@REDACTED  Sun May 17 00:19:28 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Sun, 17 May 2015 00:19:28 +0200
Subject: [erlang-questions] Erlang microservice architecture ?
In-Reply-To: 
References: 
 <20150516181242.GB1700@ferdmbp.local>
 
 
Message-ID: 

On Sat, May 16, 2015 at 9:17 PM, Thomas Elsgaard 
wrote:

> I should maybe add that this is for low volume but complex service
> activation in the telco domain, the current approach has been the
> traditional SOA solution with an ESB and process orchestration (all Java
> based), but the number of services has increased, and the ESB (service bus
> + message queue) is suffering from memory leaks and other stability
> issues which is hard to troubleshoot, and in general, the solution is just
> complex.


To me, this suggests a solution based on RabbitMQ+Protobufs, or
RabbitMQ+Transit/Msgpack. This allows you to slowly lift services out of
the clutches of Java and into Erlang one service at a time. Make a number
of Erlang nodes able to handle any kind of service task by making each
service a separate application, and you also have some added fault
tolerance. Messaging is also asynchronous if you want it, so you can avoid
some of the weaknesses of RPC.

Later on, you can probably just consider distribution of the Erlang nodes
and then send messages directly, but don't underestimate RabbitMQ as a
mediation layer.

For low volume, the flexibility of RabbitMQ is really good.


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From radek@REDACTED  Sun May 17 03:24:08 2015
From: radek@REDACTED (Rad Gruchalski)
Date: Sun, 17 May 2015 03:24:08 +0200
Subject: [erlang-questions] UDP multicast on Raspberry Pi
In-Reply-To: <174797C397A2405AB67088C4468977C2@gruchalski.com>
References: 
 
 <699999C48B374C0FA700EE718B50197B@gruchalski.com>
 <286369735.1072.1431785116362.JavaMail.zimbra@membrey.hk>
 <061B13BCD4054C90B6B49F9ED7256E07@gruchalski.com>
 <174797C397A2405AB67088C4468977C2@gruchalski.com>
Message-ID: <90E9C97D214147E291A0EE73034F73F6@gruchalski.com>

After hours of going through my few lines of code, I?m happy to report that there?s no problem with Erlang here.
The problem was between the chair and the screen.
Applying the solution suggested by Max, replacing the bind IP with interface IP instead on multicast address solves the problem.

Max, Peter, thank you for help.










Kind regards,

Radek Gruchalski

radek@REDACTED (mailto:radek@REDACTED)
 (mailto:radek@REDACTED)
de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)

Confidentiality:
This communication is intended for the above-named person and may be confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.



On Saturday, 16 May 2015 at 19:57, Rad Gruchalski wrote:

> Additionally, even when both machines have the udp_multicast program running and both were recorded by tcpdump (RPi 3 times), doing:  
>  
> $ ping 239.0.0.251
>  
> Results in:
>  
> PING 239.0.0.251 (239.0.0.251): 56 data bytes
> 64 bytes from 10.128.30.24: icmp_seq=0 ttl=64 time=0.101 ms
> 64 bytes from 10.128.30.24: icmp_seq=1 ttl=64 time=0.079 ms
> 64 bytes from 10.128.30.24: icmp_seq=2 ttl=64 time=0.125 ms
> 64 bytes from 10.128.30.24: icmp_seq=3 ttl=64 time=0.200 ms
> 64 bytes from 10.128.30.24: icmp_seq=4 ttl=64 time=0.047 ms
> 64 bytes from 10.128.30.24: icmp_seq=5 ttl=64 time=0.099 ms
> 64 bytes from 10.128.30.24: icmp_seq=6 ttl=64 time=0.087 ms
>  
>  
> Only one host is replying. RPi isn?t in that group.  
>  
>  
>  
>  
>  
>  
>  
>  
>  
>  
> Kind regards,

> Radek Gruchalski
> 
radek@REDACTED (mailto:radek@REDACTED)
 (mailto:radek@REDACTED)
> de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)
>  
> Confidentiality:
> This communication is intended for the above-named person and may be confidential and/or legally privileged.
> If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.
>  
>  
>  
> On Saturday, 16 May 2015 at 19:31, Rad Gruchalski wrote:
>  
> > Hi Peter,  
> >  
> > Thank you for your suggestions. I?m getting somewhere but not exactly sure where to, yet.
> >  
> > I?m running this on RPi:
> >  
> > tcpdump -i eth0 igmp
> >  
> > When I join multicast group from my dev box, I get the following in the tcpdump output:
> >  
> > 19:16:48.370585 IP drone?..german-fiber.net (http://german-fiber.net) > 239.0.0.251: igmp v2 report 239.0.0.251
> >  
> > When I join with the RPi itself, I see:
> >  
> > 19:18:15.245935 IP raspberrypi?..german-fiber.net (http://german-fiber.net) > igmp.mcast.net (http://igmp.mcast.net): igmp v3 report, 1 group record(s)
> > 19:18:16.035929 IP raspberrypi?..german-fiber.net (http://german-fiber.net) > igmp.mcast.net (http://igmp.mcast.net): igmp v3 report, 1 group record(s)
> >  
> >  
> > I read is that Ubuntu/Debian IGMP default version is 3. OS X (which the host is), defaults to 3 as well:
> >  
> > $ sysctl net.inet.igmp.default_version
> > net.inet.igmp.default_version: 3
> >  
> >  
> > Erlang (?) for whatever reason announces multicast with version 2 from OS X but version 3 on Raspbian. The weird thing is that I also get double join.
> >  
> > I tried changing the IGMP version on Raspbian to 2 by using:
> >  
> > echo ?2? > /proc/sys/net/ipv4/conf/eth0/force_igmp_version
> >  
> > After the change, RPi join looks like this:
> >  
> > 19:24:14.115945 IP raspberrypi?..german-fiber.net (http://german-fiber.net) > 239.0.0.251: igmp v2 report 239.0.0.251
> > 19:24:14.395920 IP raspberrypi?..german-fiber.net (http://german-fiber.net) > 239.0.0.251: igmp v2 report 239.0.0.251
> > 19:24:19.185936 IP raspberrypi?..german-fiber.net (http://german-fiber.net) > 239.0.0.251: igmp v2 report 239.0.0.251
> >  
> >  
> > For whatever reason there are 3 joins. I?m not quite sure where the problem is. It would appear that Erlang add_membership is IGMPv2? Or would that be handled by the kernel? If it is handled by the kernel, why is it not v3 as suggested by OS X sysctl default? Following on that one, why would this behaviour be different on Raspbian?
> >  
> >  
> >  
> >  
> >  
> >  
> >  
> >  
> >  
> >  
> > Kind regards,

> > Radek Gruchalski
> > 
radek@REDACTED (mailto:radek@REDACTED)
 (mailto:radek@REDACTED)
> > de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)
> >  
> > Confidentiality:
> > This communication is intended for the above-named person and may be confidential and/or legally privileged.
> > If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.
> >  
> >  
> >  
> > On Saturday, 16 May 2015 at 16:05, Peter Membrey wrote:
> >  
> > > Hi Max,
> > >  
> > > I'm afraid I don't have a direct answer for your problem, but here's some things that might help you track it down.
> > >  
> > > First, after running your Erlang listener run the following command:
> > >  
> > > $ netstat -g
> > >  
> > > This will show you all of the multicast groups that you are subscribing to and on which interface. Make sure that you see the group you care about in that list.
> > >  
> > > If it is in that list, then start a tcpdump session on that interface as root and take a look at what goes on the wire i.e:
> > >  
> > > # tcpdump -i  port 
> > >  
> > > # tcpdump -i eth0 port 10000
> > >  
> > > This lot will tell you if your Pi is looking for multicast and whether or not they're hitting the Pi itself. In other words this should help you isolate the problem.
> > >  
> > > In addition you can try:
> > >  
> > > # tcpdump -i eth0 igmp
> > >  
> > > Which should (I'm going from memory) isolate all igmp traffic. You should see packets sent when you join and leave a group. So if you start this command first, then start your Erlang app, you should see an IGMP join going out.
> > >  
> > > Hope some of this helps!
> > >  
> > > Kind Regards,
> > >  
> > > Peter Membrey
> > >  
> > > From: "Rad Gruchalski" 
> > > To: "Max Lapshin" 
> > > Cc: "erlang questions" 
> > > Sent: Saturday, 16 May, 2015 21:30:13
> > > Subject: Re: [erlang-questions] UDP multicast on Raspberry Pi
> > >  
> > > Unfortunately, I get exactly the same result.  
> > > No data received on RPi when using multicast group.
> > > It?s very strange as data published over multicast from RPi arrives at the destination. Only RPi isn?t getting any data.
> > > Just to make sure I was not having any general UDP issues - this is is the no multicast server running on RPi:
> > >  
> > > -module(udp_nomulticast).
> > >  
> > > -behaviour(gen_server).
> > >  
> > > -export([start_link/0, stop/0]).
> > > -export([init/1, handle_call/3, handle_cast/2, handle_info/2, code_change/3, terminate/2]).
> > >  
> > > start_link() ->
> > >   gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
> > >  
> > > stop() -> gen_server:cast(?MODULE, stop).
> > >  
> > > init([]) ->
> > >   Port = 6666,
> > >   IfaceIp = {0,0,0,0},
> > >   {ok, OverlaySocket} = gen_udp:open(Port, [ binary,
> > >                                              {ip, IfaceIp} ] ),
> > >   { ok, OverlaySocket }.
> > >  
> > > handle_info({udp, _ClientSocket, _ClientIp, _ClientPort, Msg}, State) ->
> > >   error_logger:info_msg("Received multicast data: ~p", [ Msg ]),
> > >   {noreply, State}.
> > >  
> > > handle_call( _, _From, State) ->
> > >   { reply, ok, State }.
> > >  
> > > code_change(_OldVsn, State, _Extra) ->
> > >   {ok, State}.
> > >  
> > > terminate(_, _) ->
> > >   ok.
> > >  
> > > handle_cast(stop, LoopData) ->
> > >   {noreply, LoopData}.
> > >  
> > >  
> > > And the client sending to it from the dev box (RPi IP on the local network is 10.128.30.23):
> > > -module(nmcc).
> > >  
> > > -export([run/0]).
> > >  
> > > run() ->
> > >   Port = 6666,
> > >   IfaceIp = {0,0,0,0},
> > >   RPiIp = {10,128,30,23},
> > >   {ok, OverlaySocket} = gen_udp:open(Port, [ binary,
> > >                                              {ip, IfaceIp} ] ),
> > >   gen_udp:send( OverlaySocket, RPiIp, Port, <<"some random datas">> ),
> > >   gen_udp:close( OverlaySocket ).
> > >  
> > >  
> > > This is arriving just fine:
> > > pi@REDACTED ~ $ erl
> > > Erlang/OTP 17 [erts-6.4] [source] [smp:4:4] [async-threads:10] [kernel-poll:false]
> > >  
> > > Eshell V6.4  (abort with ^G)
> > > 1> c("udp_nomulticast").
> > > {ok,udp_nomulticast}
> > > 2> udp_nomulticast:start_link().
> > > {ok,<0.40.0>}
> > > 3>
> > > =INFO REPORT==== 16-May-2015::15:05:35 ===
> > > Received multicast data: <<"some random datas?>>
> > >  
> > >  
> > > I?m often jumping too fast to conclusions but I genuinely believe this could be some Erlang related problem but I am not sure how to diagnose it.
> > > I tried multiple combinations, binding the server to {0,0,0,0} or multicast IP, different multicast groups. Result is always the same, traffic not arriving at RPi when using multicast UDP. What?s the best way to elevate? File a bug?
> > >  
> > >  
> > >  
> > >  
> > >  
> > >  
> > >  
> > >  
> > >  
> > >  
> > > Kind regards,

> > > Radek Gruchalski
> > > 
radek@REDACTED (mailto:radek@REDACTED)
 (mailto:radek@REDACTED)
> > > de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)
> > >  
> > > Confidentiality:
> > > This communication is intended for the above-named person and may be confidential and/or legally privileged.
> > > If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.
> > >  
> > >  
> > >  
> > > On Saturday, 16 May 2015 at 08:25, Max Lapshin wrote:
> > >  
> > > > Replace  {ip, MulticastIp},  with {ip, IfaceIp},
> > > >  
> > > >  
> > > >  
> > >  
> > >  
> > >  
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED)
> > > http://erlang.org/mailman/listinfo/erlang-questions
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED)
> > > http://erlang.org/mailman/listinfo/erlang-questions
> > >  
> > >  
> > >  
> >  
> >  
>  

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From bchesneau@REDACTED  Sun May 17 11:28:24 2015
From: bchesneau@REDACTED (Benoit Chesneau)
Date: Sun, 17 May 2015 09:28:24 +0000
Subject: [erlang-questions] Retrieving "semi-constant" data from a
 function versus Mnesia
In-Reply-To: 
References: 
 
 
 
Message-ID: 

On Mon, May 11, 2015 at 2:32 PM Joe Armstrong  wrote:

> On Sun, May 10, 2015 at 10:32 PM, Benoit Chesneau 
> wrote:
> >
> >
> > On Sun, May 10, 2015 at 10:23 PM Joe Armstrong  wrote:
> >>
> >> How large is the total data?
> >>
> >> If it's small you could define this in a module and not use a database
> >> or process at all.
> >
> >
> > How small should it be? What if the compiled bean is about 888 kilobytes?
>
> It depends :-)
>
> There are many factors involved:
>
>    a) How much memory do you have
>    b) How often do you want to change the configuration data
>    c) How quick do you the change to be
>
> Assume
>
>   a) is a smallish number of GBytes (normal these days)
>   b) is "relatively rarely" (I don't know what this means - (aside - this
> is why
>       I always ask people to provide numbers in their questions))
>      Say once a day
>   c) is a few seconds
>
> Then it should be perfectly doable. Making a module containing all the
> config data
> and recompiling when necessary is certainly the fastest way to access the
> data -
> this has very fast reads but very slow writes - you could think of a
> module as a database
> optimized for reads but not writes :-)
>
> /Joe
>
>
>
I see :) Thanks for the answer!

I'm already using such technic to speed access to unicode data in my IDNA
module [1] but always wondered if there is not a more efficient approach
since the compiled beam is quite big though less than 1MB. In term of speed
it's better that using ETS though I don't have the exact figures right now.

- benoit
[1]
https://github.com/benoitc/erlang-idna/blob/master/src/idna_unicode_data.erl




>
> >
> > - benoit
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From rickard@REDACTED  Sun May 17 20:31:36 2015
From: rickard@REDACTED (Rickard Green)
Date: Sun, 17 May 2015 20:31:36 +0200
Subject: [erlang-questions] Ets read concurrency tweak
In-Reply-To: 
References: 
Message-ID: 

On Thu, May 14, 2015 at 12:29 PM, Viacheslav V. Kovalev
 wrote:
> Hi, List!
>
> I'm playing with ets tweaks and specifically with read_concurrency.
> I've written simple test to measure how this tweak impacts on read
> performance. Test implementations is here
> https://gist.github.com/kovyl2404/826a51b27ba869527910
>
> Briefly, this test sequentially creates three [public, set] ets tables
> with different read_concurrency options (without any tweaks, with
> {read_concurrency, true} and with {read_concurrency, false}). After
> one table created, test pupulates table with some random data and runs
> N readers (N is power of 2 from 4 to 1024). Readers performs K random
> reads and exists.
>
> Result is quite surprising for me. There absolutely no difference
> between 3 these tests. I have run this test on Erlang 17 on 2-, 8-,
> and 64-core machines and could not find any significant performance
> impact from this tweak.
>
> Could anybody explain usecase of this tweak? What should I do to see
> any difference and understand when to use this option?
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

I haven't had the time to look at your code, so I cannot tell you why
you are not getting the results you expected. Here is some information
about the option, though.

When the {read_concurrency, true} option is passed, reader optimized
rwlocks are used instead of ordinary rwlocks. When reader optimized
rwlocks are used, threads performing read-locking notify about their
presence in separate cache lines, and by this avoid ping-ponging of a
common cache-line between caches.

Write-locking of a reader optimized rwlock is more expensive than
write-locking an ordinary rwlock, so if you have a large amount of
write operation you don't want to use the read_concurrency option. The
largest performance improvement will be seen when there are no
write-locking at all.

In order to determine if it is beneficial to use the option in your
use-case, you need to observe your system when it is executing under
expected load and without effecting it too much while observing it. In
your case it might be that eprof is effecting the execution too much,
but that is just a guess.

The improvement varies a lot depending on hardware. The more expensive
it is to keep a common cache line up to date in all involved caches,
the larger the performance improvement will be. It will typically be
more expensive, the further away cores are from each other and the
more cores that are involved.

I've attached a small benchmark that illustrates the effect. When run on:

Intel(R) Core(TM) i7-4600U CPU @ 2.10GHz

Eshell V6.3  (abort with ^G)
1> erlang:system_info(cpu_topology).
[{processor,[{core,[{thread,{logical,0}},{thread,{logical,2}}]},
             {core,[{thread,{logical,1}},{thread,{logical,3}}]}]}]

Without read_concurrency an execution time of about 0.85-1.0 seconds.
With read_concurrency 0.75-0.8 seconds.


When run on:

AMD Opteron(tm) Processor 4376 HE

Eshell V6.4.1  (abort with ^G)
1> erlang:system_info(cpu_topology).
[{node,[{processor,[{core,{logical,0}},
                    {core,{logical,1}},
                    {core,{logical,2}},
                    {core,{logical,3}},
                    {core,{logical,4}},
                    {core,{logical,5}},
                    {core,{logical,6}},
                    {core,{logical,7}}]}]},
 {node,[{processor,[{core,{logical,8}},
                    {core,{logical,9}},
                    {core,{logical,10}},
                    {core,{logical,11}},
                    {core,{logical,12}},
                    {core,{logical,13}},
                    {core,{logical,14}},
                    {core,{logical,15}}]}]}]

Without read_concurrency an execution time of about 39-41 seconds.
With read_concurrency 1.1-1.2 seconds.


Regards,
Rickard
-- 
Rickard Green, Erlang/OTP, Ericsson AB
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ets_rc_test.erl
Type: text/x-erlang
Size: 708 bytes
Desc: not available
URL: 

From yueyoum@REDACTED  Mon May 18 08:01:10 2015
From: yueyoum@REDACTED (=?UTF-8?B?5pyI5b+n6IyX?=)
Date: Mon, 18 May 2015 14:01:10 +0800
Subject: [erlang-questions]  Can Pid be maps key ?
Message-ID: 

Can Pid be maps key ?

Build maps from #{} syntax, The error says Pid can not be key.

Bug build with maps module, Pid can be key.

18> 18> Pid = self().
<0.39.0>19> #{Pid => 1}.
* 1: illegal use of variable 'Pid' in map20> 20> M1 =
maps:from_list([{Pid, 1}]).
#{<0.39.0> => 1}21> 21> #{Pid := V} = M1.
* 2: illegal use of variable 'Pid' in map22>                  22>
maps:get(Pid, M1).1






-- 
My GitHub
https://github.com/yueyoum
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From vychodil.hynek@REDACTED  Mon May 18 08:24:49 2015
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Mon, 18 May 2015 08:24:49 +0200
Subject: [erlang-questions] Can Pid be maps key ?
In-Reply-To: 
References: 
Message-ID: 

Variable map key syntax is not supported in R17. It is scheduled for R18.

On Mon, May 18, 2015 at 8:01 AM, ???  wrote:

>
> Can Pid be maps key ?
>
> Build maps from #{} syntax, The error says Pid can not be key.
>
> Bug build with maps module, Pid can be key.
>
> 18> 18> Pid = self().
> <0.39.0>19> #{Pid => 1}.
> * 1: illegal use of variable 'Pid' in map20> 20> M1 = maps:from_list([{Pid, 1}]).
> #{<0.39.0> => 1}21> 21> #{Pid := V} = M1.
> * 2: illegal use of variable 'Pid' in map22>                  22> maps:get(Pid, M1).1
>
>
>
>
>
>
> --
> My GitHub
> https://github.com/yueyoum
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From bchesneau@REDACTED  Mon May 18 09:33:30 2015
From: bchesneau@REDACTED (Benoit Chesneau)
Date: Mon, 18 May 2015 07:33:30 +0000
Subject: [erlang-questions] how are implemented lists?
Message-ID: 

how are implemented lists? What is the algorithm behind them? Especially
for key* functions? I am asking myself if i need to implement a skip-list
or if the lists module would fit my needs. I indeed need a fast and
concurrent data structure that allows me to retrieve the items in order and
do pop/tail, while still being abble to remove them by key. Any idea?

- beno?t
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From sergej.jurecko@REDACTED  Mon May 18 09:36:59 2015
From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=)
Date: Mon, 18 May 2015 09:36:59 +0200
Subject: [erlang-questions] how are implemented lists?
In-Reply-To: 
References: 
Message-ID: 

Pretty sure thats just a linear search but implemented in c. sorted_set ets
would probably be better?

Sergej
On May 18, 2015 9:33 AM, "Benoit Chesneau"  wrote:

> how are implemented lists? What is the algorithm behind them? Especially
> for key* functions? I am asking myself if i need to implement a skip-list
> or if the lists module would fit my needs. I indeed need a fast and
> concurrent data structure that allows me to retrieve the items in order and
> do pop/tail, while still being abble to remove them by key. Any idea?
>
> - beno?t
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From bchesneau@REDACTED  Mon May 18 10:11:08 2015
From: bchesneau@REDACTED (Benoit Chesneau)
Date: Mon, 18 May 2015 08:11:08 +0000
Subject: [erlang-questions] how are implemented lists?
In-Reply-To: 
References: 
 
Message-ID: 

On Mon, May 18, 2015 at 9:37 AM Sergej Jure?ko 
wrote:

> Pretty sure thats just a linear search but implemented in c. sorted_set
> ets would probably be better?
>
>>
>>
I will have a lot of different list, so it would require to share an ETS
table between people I guess since the number of ETS is limited, not sure
how much it would be efficient in that case but that a good idea, will see
if it can go for such implementation for now, thanks :)

- benoit
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From vychodil.hynek@REDACTED  Mon May 18 10:29:36 2015
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Mon, 18 May 2015 10:29:36 +0200
Subject: [erlang-questions] how are implemented lists?
In-Reply-To: 
References: 
 
 
Message-ID: 

You can increase an amount of ETS tables. Only your memory is the limit.
Keep in mind, that all structures are copied in and out of ETS table.

Hynek

On Mon, May 18, 2015 at 10:11 AM, Benoit Chesneau 
wrote:

>
>
> On Mon, May 18, 2015 at 9:37 AM Sergej Jure?ko 
> wrote:
>
>> Pretty sure thats just a linear search but implemented in c. sorted_set
>> ets would probably be better?
>>
>>>
>>>
> I will have a lot of different list, so it would require to share an ETS
> table between people I guess since the number of ETS is limited, not sure
> how much it would be efficient in that case but that a good idea, will see
> if it can go for such implementation for now, thanks :)
>
> - benoit
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From kostis@REDACTED  Mon May 18 10:48:12 2015
From: kostis@REDACTED (Kostis Sagonas)
Date: Mon, 18 May 2015 11:48:12 +0300
Subject: [erlang-questions] how are implemented lists?
In-Reply-To: 
References: 
Message-ID: <5559A74C.80709@cs.ntua.gr>

On 05/18/2015 10:33 AM, Benoit Chesneau wrote:
> how are implemented lists? What is the algorithm behind them? Especially
> for key* functions? I am asking myself if i need to implement a
> skip-list or if the lists module would fit my needs. I indeed need a
> fast and concurrent data structure that allows me to retrieve the items
> in order and do pop/tail, while still being abble to remove them by key.

To avoid confusion, let me remind you that in Erlang the only data 
structures that are shared between processes, and could thus be 
classified as "concurrent", are ETS tables.  There are no others.  So 
plain lists do not fit your needs, independently of how they may be 
implemented.

Given that you want "ordered traversal/retrieval", ETS tables of type 
ordered_set are the data structures that would fit your need.  Alas, 
their current performance in the presence of concurrent accesses is not 
so great.  The following publication describes some of the issues that 
are involved:

 
http://www.it.uu.se/research/group/languages/software/ca_tree/erlang_paper.pdf

We have done quite a bit of work in extending CA trees, the data 
structure described in this publication, to e.g. provide efficient 
support for range queries and range updates, but this work is not (yet) 
part of Erlang/OTP.  More information can be found at:

   http://www.it.uu.se/research/group/languages/software/ca_tree

Kostis


From zandra@REDACTED  Mon May 18 10:56:54 2015
From: zandra@REDACTED (Zandra Hird)
Date: Mon, 18 May 2015 10:56:54 +0200
Subject: [erlang-questions] Patch Package OTP 17.5.4 Released
Message-ID: <5559A956.3060007@erlang.org>

Patch Package:           OTP 17.5.4
Git Tag:                 OTP-17.5.4

  Check out the git tag OTP-17.5.4, and build a full OTP system
  including documentation. Apply one or more applications from this
  build as patches to your installation using the 'otp_patch_apply'
  tool. For information on install requirements, see descriptions for
  each application version below.

  ---------------------------------------------------------------------
  --- inets-5.10.8 ----------------------------------------------------
  ---------------------------------------------------------------------

  The inets-5.10.8 application can be applied independently of other
  applications on a full OTP 17 installation.

  --- Fixed Bugs and Malfunctions ---

   OTP-12739    Application(s): inets
                Related Id(s): seq12860

                Reject messages with a Content-Length less than 0


  Full runtime dependencies of inets-5.10.8: erts-6.0, kernel-3.0,
  mnesia-4.12, runtime_tools-1.8.14, ssl-5.3.4, stdlib-2.0


  ---------------------------------------------------------------------
  --- ssh-3.2.3 -------------------------------------------------------
  ---------------------------------------------------------------------

  Note! The ssh-3.2.3 application can *not* be applied independently of
        other applications on an arbitrary OTP 17 installation.

        On a full OTP 17 installation, also the following runtime
        dependency has to be satisfied:
        -- stdlib-2.3 (first satisfied in OTP 17.4)


  --- Fixed Bugs and Malfunctions ---

   OTP-12738    Application(s): ssh
                Related Id(s): seq12860

                A new option for handling the SSH_MSG_DEBUG message's
                printouts. A fun could be given in the options that
                will be called whenever the SSH_MSG_DEBUG message
                arrives. This enables the user to format the printout
                or just discard it.


  Full runtime dependencies of ssh-3.2.3: crypto-3.3, erts-6.0,
  kernel-3.0, public_key-0.22, stdlib-2.3


  ---------------------------------------------------------------------
  ---------------------------------------------------------------------
  ---------------------------------------------------------------------


From kenneth@REDACTED  Mon May 18 15:53:18 2015
From: kenneth@REDACTED (Kenneth Lundin)
Date: Mon, 18 May 2015 15:53:18 +0200
Subject: [erlang-questions] =?utf-8?q?=5BANN=5D_Other_Erldocs_=CE=B1?=
In-Reply-To: 
References: 
Message-ID: 

I like the format generated by erldocs but why are you generating
documentation for all modules?
The majority of modules are not intended to be used outside their
applications and thus they have no public API. I think it is a problem with
erldocs that there are so many uninteresting modules that show up inter
mixed  with the real interesting ones.

For example in OTP all source for documentation is mentioned in the make
files and other modules are uninteresting to try to generate documentation
for.

/Kenneth , Erlang/OTP, ERICSSON

On Wed, Apr 1, 2015 at 9:19 PM, Pierre Fenoll 
wrote:

> Like erldocs.com?
>
> Now you can access a regularly-updated repository of your favorite Erlang
> projects'
> documentation over at http://other.erldocs.com/
>
> You can search for a repository on the home page
> or just paste a repo's URL in the OpenSearch bar.
>
>
> Example: https://github.com/comtihon/mongodb-erlang
>
> ? A redirection to the other.erldocs.com page:
>
> http://other.erldocs.com/opensearch.html?q=https://github.com/comtihon/mongodb-erlang
>
> ? The other.erldocs.com page:
> http://other.erldocs.com/github.com/comtihon/mongodb-erlang/
>
> ? Project's documentation for tag "v0.3.1":
> http://other.erldocs.com/github.com/comtihon/mongodb-erlang/v0.3.1/
>
> ? Project's documentation for branch "devbranch":
> http://other.erldocs.com/github.com/comtihon/mongodb-erlang/devbranch/
>
> ? Erlang repository information:
> http://other.erldocs.com/github.com/comtihon/mongodb-erlang/meta.txt
>
> ? Log of repo's build
> http://other.erldocs.com/github.com/comtihon/mongodb-erlang/_.txt
>
>
> Code for generating repos' erldocs:
> https://github.com/erldocs/erldocs_other
> Repo hosting the full webiste:
> https://github.com/erldocs/other.erldocs.com
>
>
> Notes:
> ? If an Erlang repo is not listed please submit an issue.
> ? meta.txt does not hold too much information for now, but should allow
> for statistical analysis later on.
> ? The website should be updated often enough (more than once a week)
> ? You'll notice some 404s: keep in mind this is a work in progress!
>
> Cheers
>
> --
> Pierre Fenoll
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From elbrujohalcon@REDACTED  Mon May 18 16:49:28 2015
From: elbrujohalcon@REDACTED (Fernando 'Brujo' Benavides)
Date: Mon, 18 May 2015 11:49:28 -0300
Subject: [erlang-questions] [ANN] ErlangBA 2015 - Final date confirmed
Message-ID: <04744285-ED1C-4AE2-91F4-486D898251BA@inaka.net>

Hi everybody,

ErlangBA 2015  has a date!
It will be held on Wednesday June 17th at 7PM in Inaka 's Offices.
To participate in the event, please register at Meetup .
We'll be posting the full address for the event, the talks and other news in that website soon.
Cheers!
 

Fernando "Brujo" Benavides
about.me/elbrujohalcon

  				
 



-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From sean@REDACTED  Mon May 18 18:00:01 2015
From: sean@REDACTED (Functional Jobs)
Date: Mon, 18 May 2015 12:00:01 -0400
Subject: [erlang-questions] New Functional Programming Job Opportunities
Message-ID: <555a0c844c317@functionaljobs.com>

Here are some functional programming job opportunities that were posted
recently:

Senior Software Engineer (Functional Programming/Scala) at AdAgility
http://functionaljobs.com/jobs/8826-senior-software-engineer-functional-programming-scala-at-adagility

Cheers,
Sean Murphy
FunctionalJobs.com



From matthias@REDACTED  Mon May 18 22:28:31 2015
From: matthias@REDACTED (Matthias Lang)
Date: Mon, 18 May 2015 22:28:31 +0200
Subject: [erlang-questions] how are implemented lists?
In-Reply-To: 
References: 
Message-ID: <20150518202831.GA15935@corelatus.se>

Hi,

I've read the answers to your question that have come in so far, and
I'm left wondering if the answer you're looking for is: "a list in
Erlang is a singly-linked list". But maybe that's obvious.

If you haven't already come across it, an interesting book about the
sorts of data structures you can implement in languages like Erlang is

   "Purely Functional Data structures" by Chris Okasaki

a subset of the ideas in that book are in a paper by the same author:

   http://www-2.cs.cmu.edu/~rwh/theses/okasaki.pdf

Keep in mind that some/many of Okasaki's structures require a lazy
language, which Erlang isn't.

I think a skip list is something you can only (directly) implement in
a language which allows destructive updates, i.e. not as an Erlang
program.

A _quick_ glance at what's happened with skip lists since the early
90s suggests that you can implement them as trees, which in turn
_might_ mean it's possible directly in Erlang, contrary to what I claim
in the previous paragraph. E.g.:

  http://www.cs.clemson.edu/~bcdean/paper11.html

Maybe someone up to speed on this can comment; I'm a bit curious, but
not curious enough to do more than leaf through that paper.

Matthias

----------------------------------------------------------------------
Date: 18. May 2015
From: Benoit Chesneau 
To erlang-questions@REDACTED
Subject: [erlang-questions] how are implemented lists?


> how are implemented lists? What is the algorithm behind them? Especially
> for key* functions? I am asking myself if i need to implement a skip-list
> or if the lists module would fit my needs. I indeed need a fast and
> concurrent data structure that allows me to retrieve the items in order and
> do pop/tail, while still being abble to remove them by key. Any idea?
>
> - beno?t

> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions


From ok@REDACTED  Tue May 19 05:27:24 2015
From: ok@REDACTED (Richard A. O'Keefe)
Date: Tue, 19 May 2015 15:27:24 +1200
Subject: [erlang-questions] how are implemented lists?
In-Reply-To: 
References: 
Message-ID: 


On 18/05/2015, at 7:33 pm, Benoit Chesneau  wrote:

> how are implemented lists? What is the algorithm behind them? Especially for key* functions? I am asking myself if i need to implement a skip-list or if the lists module would fit my needs. I indeed need a fast and concurrent data structure that allows me to retrieve the items in order and do pop/tail, while still being abble to remove them by key. Any idea?

It's not clear how a data structure that relies on being
both shared, mutable, and cross-linked, fits into Erlang,
where data structures are (logically) unshared and immutable.

Erlang lists are simple read-only singly-linked lists, and
the key* functions in the lists.erl library file are just
linear searches.

If it weren't for the "concurrent" bit I'd suggest that
one of the set or dictionary data structures in stdlib
might be useful, or one of the Okasaki purely functional
data structures that I thought had been adapted to
Erlang, but can't find.

It looks as though ETS may be the closest thing to what you
need at the moment.

http://stackoverflow.com/questions/3489560/purely-functional-concurrent-skip-list
may be helpful, in a negative sort of way.




From norberto.ortigoza@REDACTED  Tue May 19 01:55:46 2015
From: norberto.ortigoza@REDACTED (Norberto Ortigoza)
Date: Mon, 18 May 2015 18:55:46 -0500
Subject: [erlang-questions] XML validation using XSD
Message-ID: <5F1622DF-53A9-48B2-B3DE-9C9C30D697F1@gmail.com>

Hi everyone,

I'm using xmerl_xsd to validate an XML file against a XSD schema.
The problem I have is that when the document have errors it doesn't return the field names.

For example:
[{:minInclusive, '2010', :not_greater_than_or_equal_with, '2015?},

It has the constraint and the value, but it doesn?t include the field that is wrong

I need to know the field name in order to return a message error to the user.
I can't find in the documentation any flag o information about this.

Is it possible to get that info?

Thanks in advance

Norberto

From luca.finzicontini@REDACTED  Tue May 19 10:30:52 2015
From: luca.finzicontini@REDACTED (Luca Finzi Contini)
Date: Tue, 19 May 2015 10:30:52 +0200
Subject: [erlang-questions] ErlIDE and rebar integration
Message-ID: 

Hi all,
I am trying some development environments for Erlang, since I come from
Eclipse;
I foud that IntelliJ Idea has quite a nice integration with Erlang via a
specific plugin that has some integration with rebar.
Is there any plan (if any of you know about it) to have also ErlIDE
integrate with rebar?

Thank you in advance.
Luca.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From vladdu55@REDACTED  Tue May 19 11:18:09 2015
From: vladdu55@REDACTED (Vlad Dumitrescu)
Date: Tue, 19 May 2015 11:18:09 +0200
Subject: [erlang-questions] ErlIDE and rebar integration
In-Reply-To: 
References: 
Message-ID: 

Hi!

Yes, the integration is ongoing work. I'm not sure how long it will take to
have it ready, as it is intermingled with other work.

Actually, the nightly builds already have a partial implementation
working. With
the big warning that *this is provisional, will be different when released
and might not work as intended*, you can go to your project's directory,
open .settings/org.erlide.model.prefs and edit the settings by replacing
INTERNAL with REBAR (in two places). Keep the rest intact. To restore, edit
back to to INTERNAL.

If you have questions or problems, you can contact me directly or by
opening a github issue https://github.com/erlide/erlide/issues.

best regards,
Vlad


On Tue, May 19, 2015 at 10:30 AM, Luca Finzi Contini <
luca.finzicontini@REDACTED> wrote:

> Hi all,
> I am trying some development environments for Erlang, since I come from
> Eclipse;
> I foud that IntelliJ Idea has quite a nice integration with Erlang via a
> specific plugin that has some integration with rebar.
> Is there any plan (if any of you know about it) to have also ErlIDE
> integrate with rebar?
>
> Thank you in advance.
> Luca.
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From joaohf@REDACTED  Tue May 19 21:09:10 2015
From: joaohf@REDACTED (=?windows-1252?Q?Jo=E3o_Henrique_Ferreira_de_Freitas?=)
Date: Tue, 19 May 2015 16:09:10 -0300
Subject: [erlang-questions] Erlang on Arduino - ArduinoBoardYun?
In-Reply-To: 
References: 
Message-ID: <555B8A56.1050903@gmail.com>


Hi,

You could try Yocto Project (1) and meta-erlang (2).

1 https://www.yoctoproject.org/
2 http://layers.openembedded.org/layerindex/branch/master/layer/meta-erlang/

On 16/05/2015 17:47, Lloyd R. Prentice wrote:
> Is it possible, or has anyone installed Erlang on the Arduino Yun? And if so, how would it be done?
>
> http://www.arduino.cc/en/Main/ArduinoBoardYun?from=Main.ArduinoYUN
>
> If possible, it seems that this would be a cool way to get sensor data into an Erlang system.
>
> All the best,
>
> LRP
>
>
> Sent from my iPad
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>



From lloyd@REDACTED  Tue May 19 22:30:37 2015
From: lloyd@REDACTED (Lloyd R. Prentice)
Date: Tue, 19 May 2015 16:30:37 -0400
Subject: [erlang-questions] Erlang on Arduino - ArduinoBoardYun?
In-Reply-To: <555B8A56.1050903@gmail.com>
References: 
 <555B8A56.1050903@gmail.com>
Message-ID: <058E86AD-6AFD-4A44-A159-5BEA9D98F7C6@writersglen.com>

Many thanks, Jo?o,

I see a number of Yocto tutorials, but not much on meta-erlang. Could you please give me or point to a short, sweet, high-level overview?

All the best,

Lloyd

Sent from my iPad

> On May 19, 2015, at 3:09 PM, Jo?o Henrique Ferreira de Freitas  wrote:
> 
> 
> Hi,
> 
> You could try Yocto Project (1) and meta-erlang (2).
> 
> 1 https://www.yoctoproject.org/
> 2 http://layers.openembedded.org/layerindex/branch/master/layer/meta-erlang/
> 
>> On 16/05/2015 17:47, Lloyd R. Prentice wrote:
>> Is it possible, or has anyone installed Erlang on the Arduino Yun? And if so, how would it be done?
>> 
>> http://www.arduino.cc/en/Main/ArduinoBoardYun?from=Main.ArduinoYUN
>> 
>> If possible, it seems that this would be a cool way to get sensor data into an Erlang system.
>> 
>> All the best,
>> 
>> LRP
>> 
>> 
>> Sent from my iPad
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions


From stas@REDACTED  Tue May 19 22:42:57 2015
From: stas@REDACTED (Stanislav Sedov)
Date: Tue, 19 May 2015 13:42:57 -0700
Subject: [erlang-questions] Erlang on Arduino - ArduinoBoardYun?
In-Reply-To: <058E86AD-6AFD-4A44-A159-5BEA9D98F7C6@writersglen.com>
References: 
 <555B8A56.1050903@gmail.com>
 <058E86AD-6AFD-4A44-A159-5BEA9D98F7C6@writersglen.com>
Message-ID: 

It looks like meta-erlang is an erlang package for OpenEmbedded distribution.

I don?t have an Arduino and never ran Erlang on AR9331, but I?d imagine the
process is no different from installing Erlang onto any other system:
1) Get an operating system up and running;
2) Use the OS package manager to install Erlang, or compile it from source.
   All major Linux distributions have Erlang packaged.  Erlang itself does
   run on MIPS as far as I know which AR9331 is an example of.

> On May 19, 2015, at 1:30 PM, Lloyd R. Prentice  wrote:
> 
> Many thanks, Jo?o,
> 
> I see a number of Yocto tutorials, but not much on meta-erlang. Could you please give me or point to a short, sweet, high-level overview?
> 
> All the best,
> 
> Lloyd
> 
> Sent from my iPad
> 
>> On May 19, 2015, at 3:09 PM, Jo?o Henrique Ferreira de Freitas  wrote:
>> 
>> 
>> Hi,
>> 
>> You could try Yocto Project (1) and meta-erlang (2).
>> 
>> 1 https://www.yoctoproject.org/
>> 2 http://layers.openembedded.org/layerindex/branch/master/layer/meta-erlang/
>> 
>>> On 16/05/2015 17:47, Lloyd R. Prentice wrote:
>>> Is it possible, or has anyone installed Erlang on the Arduino Yun? And if so, how would it be done?
>>> 
>>> http://www.arduino.cc/en/Main/ArduinoBoardYun?from=Main.ArduinoYUN
>>> 
>>> If possible, it seems that this would be a cool way to get sensor data into an Erlang system.
>>> 
>>> All the best,
>>> 
>>> LRP
>>> 
>>> 
>>> Sent from my iPad
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>> 
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
> 
> !DSPAM:555b9d80560099430187992!
> 
> 

--
ST4096-RIPE





From rvirding@REDACTED  Wed May 20 01:24:53 2015
From: rvirding@REDACTED (Robert Virding)
Date: Wed, 20 May 2015 01:24:53 +0200
Subject: [erlang-questions] how are implemented lists?
In-Reply-To: <5559A74C.80709@cs.ntua.gr>
References: 
 <5559A74C.80709@cs.ntua.gr>
Message-ID: 

On 18 May 2015 at 10:48, Kostis Sagonas  wrote:

>
> To avoid confusion, let me remind you that in Erlang the only data
> structures that are shared between processes, and could thus be classified
> as "concurrent", are ETS tables.  There are no others.  So plain lists do
> not fit your needs, independently of how they may be implemented.
>

Actually ETS tables and the data in them aren't shared between processes,
they are only globally accessible by processes. As soon as you access an
ETS table you are copying data to/from the table and your process. Every
write copies the data to the table and every reads copies the data from the
table to the process. This is something to be aware of when working with
ETS.

Robert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From joaohf@REDACTED  Wed May 20 02:12:16 2015
From: joaohf@REDACTED (=?UTF-8?B?Sm/Do28gSGVucmlxdWUgRmVycmVpcmEgZGUgRnJlaXRhcw==?=)
Date: Tue, 19 May 2015 21:12:16 -0300
Subject: [erlang-questions] Erlang on Arduino - ArduinoBoardYun?
In-Reply-To: <058E86AD-6AFD-4A44-A159-5BEA9D98F7C6@writersglen.com>
References: 
 <555B8A56.1050903@gmail.com>
 <058E86AD-6AFD-4A44-A159-5BEA9D98F7C6@writersglen.com>
Message-ID: <555BD160.6050803@gmail.com>

Hi,



Follow any Yocto tutorials and then:

1 - clone the meta-erlang repo

2 - so you have to add meta-erlang in conf/bblayer.conf:

BBLAYERS ?= " \
    ...
   path_to_source/sources/meta-erlang \
   "

3- the easy way is to add 'IMAGE_INSTALL_append = " erlang"' in 
conf/local.conf and run 'bitbake core-image-minimal'.

This will setup a basic image with erlang inside.

Don't forget to setup the correct MACHINE in conf/local.conf. I usually 
test with qemux86 and qemuarm.

meta-erlang layer adds Erlang to Yocto/OpenEmbedded build system. There 
is no trick to get it working. The erlang bitbake recipe splits erlang 
in many packages so you can add only ones that you will use.

Thanks.


On 19/05/2015 17:30, Lloyd R. Prentice wrote:
> Many thanks, Jo?o,
>
> I see a number of Yocto tutorials, but not much on meta-erlang. Could you please give me or point to a short, sweet, high-level overview?
>
> All the best,
>
> Lloyd
>
> Sent from my iPad
>
>> On May 19, 2015, at 3:09 PM, Jo?o Henrique Ferreira de Freitas  wrote:
>>
>>
>> Hi,
>>
>> You could try Yocto Project (1) and meta-erlang (2).
>>
>> 1 https://www.yoctoproject.org/
>> 2 http://layers.openembedded.org/layerindex/branch/master/layer/meta-erlang/
>>
>>> On 16/05/2015 17:47, Lloyd R. Prentice wrote:
>>> Is it possible, or has anyone installed Erlang on the Arduino Yun? And if so, how would it be done?
>>>
>>> http://www.arduino.cc/en/Main/ArduinoBoardYun?from=Main.ArduinoYUN
>>>
>>> If possible, it seems that this would be a cool way to get sensor data into an Erlang system.
>>>
>>> All the best,
>>>
>>> LRP
>>>
>>>
>>> Sent from my iPad
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions



From lloyd@REDACTED  Wed May 20 05:04:36 2015
From: lloyd@REDACTED (Lloyd R. Prentice)
Date: Tue, 19 May 2015 23:04:36 -0400
Subject: [erlang-questions] Erlang on Arduino - ArduinoBoardYun?
In-Reply-To: <555BD160.6050803@gmail.com>
References: 
 <555B8A56.1050903@gmail.com>
 <058E86AD-6AFD-4A44-A159-5BEA9D98F7C6@writersglen.com>
 <555BD160.6050803@gmail.com>
Message-ID: <7F0A14F8-5B87-48EF-9971-C6F319800141@writersglen.com>

Thanks guys,

It'll be fun to see if I can expand my Erlang skills, such as they are, beyond the web.

All the best,

Lloyd

Sent from my iPad

> On May 19, 2015, at 8:12 PM, Jo?o Henrique Ferreira de Freitas  wrote:
> 
> Hi,
> 
> 
> 
> Follow any Yocto tutorials and then:
> 
> 1 - clone the meta-erlang repo
> 
> 2 - so you have to add meta-erlang in conf/bblayer.conf:
> 
> BBLAYERS ?= " \
>   ...
>  path_to_source/sources/meta-erlang \
>  "
> 
> 3- the easy way is to add 'IMAGE_INSTALL_append = " erlang"' in conf/local.conf and run 'bitbake core-image-minimal'.
> 
> This will setup a basic image with erlang inside.
> 
> Don't forget to setup the correct MACHINE in conf/local.conf. I usually test with qemux86 and qemuarm.
> 
> meta-erlang layer adds Erlang to Yocto/OpenEmbedded build system. There is no trick to get it working. The erlang bitbake recipe splits erlang in many packages so you can add only ones that you will use.
> 
> Thanks.
> 
> 
>> On 19/05/2015 17:30, Lloyd R. Prentice wrote:
>> Many thanks, Jo?o,
>> 
>> I see a number of Yocto tutorials, but not much on meta-erlang. Could you please give me or point to a short, sweet, high-level overview?
>> 
>> All the best,
>> 
>> Lloyd
>> 
>> Sent from my iPad
>> 
>>> On May 19, 2015, at 3:09 PM, Jo?o Henrique Ferreira de Freitas  wrote:
>>> 
>>> 
>>> Hi,
>>> 
>>> You could try Yocto Project (1) and meta-erlang (2).
>>> 
>>> 1 https://www.yoctoproject.org/
>>> 2 http://layers.openembedded.org/layerindex/branch/master/layer/meta-erlang/
>>> 
>>>> On 16/05/2015 17:47, Lloyd R. Prentice wrote:
>>>> Is it possible, or has anyone installed Erlang on the Arduino Yun? And if so, how would it be done?
>>>> 
>>>> http://www.arduino.cc/en/Main/ArduinoBoardYun?from=Main.ArduinoYUN
>>>> 
>>>> If possible, it seems that this would be a cool way to get sensor data into an Erlang system.
>>>> 
>>>> All the best,
>>>> 
>>>> LRP
>>>> 
>>>> 
>>>> Sent from my iPad
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
> 


From hm@REDACTED  Wed May 20 09:58:16 2015
From: hm@REDACTED (=?UTF-8?Q?H=C3=A5kan_Mattsson?=)
Date: Wed, 20 May 2015 09:58:16 +0200
Subject: [erlang-questions] how are implemented lists?
Message-ID: 

On Tue, May 19, 2015 at 5:27 AM, Richard A. O'Keefe 
wrote:

>
> If it weren't for the "concurrent" bit I'd suggest that
> one of the set or dictionary data structures in stdlib
> might be useful, or one of the Okasaki purely functional
> data structures that I thought had been adapted to
> Erlang, but can't find.
>

Perhaps you ar looking for the queue module?

It is quite useful when you want to have a list where popping elements from
the tail is efficient.

/H?kan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From hm@REDACTED  Wed May 20 09:58:17 2015
From: hm@REDACTED (=?UTF-8?Q?H=C3=A5kan_Mattsson?=)
Date: Wed, 20 May 2015 09:58:17 +0200
Subject: [erlang-questions] how are implemented lists?
Message-ID: 

On Tue, May 19, 2015 at 5:27 AM, Richard A. O'Keefe 
wrote:

>
> If it weren't for the "concurrent" bit I'd suggest that
> one of the set or dictionary data structures in stdlib
> might be useful, or one of the Okasaki purely functional
> data structures that I thought had been adapted to
> Erlang, but can't find.
>

Perhaps you ar looking for the queue module?

It is quite useful when you want to have a list where popping elements from
the tail is efficient.

/H?kan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From essen@REDACTED  Wed May 20 10:33:36 2015
From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=)
Date: Wed, 20 May 2015 11:33:36 +0300
Subject: [erlang-questions] Windows
Message-ID: <555C46E0.4060000@ninenines.eu>

Hi,

Yesterday the topic of Windows development was brought up on Twitter. I 
looked into it and it's clearly an issue. At best you can compile 
projects that don't involve shell scripts, makefiles or C code, so you 
can't really work on anything significant as a large number of open 
source projects involve one or the other (yes, even when using rebar).

Now there's three ways Erlang could be more friendly to Windows developers:

To use Windows tools imply making open source projects work for Windows. 
Sure perhaps it's not an issue for the simplest NIFs but some projects 
require C libraries which themselves aren't always easy to make work on 
Windows.

MingW has the same issues. You need to fix everything to work on Windows.

Cygwin makes this much easier. Everything seems to work... except 
Erlang. Erlang on Cygwin thinks it runs on win32 and, while true to a 
certain extent, it fails badly when dealing with files (cygwin has / as 
root while win32 has c:\).

Now you might say Cygwin is massive. That's true, it takes some time to 
install and some space on the disk. But it's very easy to install and 
can be almost entirely automated using chocolatey and apt-cyg. And 
adding an SSH server on top of that is a couple commands away, so it is 
a very interesting solution if you ask me.

Now Erlang compiles on Cygwin... but produces the win32 target. Which 
brings me to the main subject of this email: what would it involve to 
make Erlang work on Cygwin? Considering Cygwin is a mix of Windows and 
Linux it's probably just about enabling one or another piece of code, or 
is it?

It could be a fine side project but I don't really know where to start. 
Any tips appreciated.

PS: tried just patching os:type()... that didn't work. Ended up with 
Erlang not finding a file that existed on disk when running common_test.

PPS: I know "real" developers use Unix. But the more numerous "fake" 
developers might be more interested in Erlang if they can keep their OS.

PPPS: I know this isn't a good option if you're going to create 
Photoshoperl, but it's still a good option for server application 
development.

-- 
Lo?c Hoguin
http://ninenines.eu


From sergej.jurecko@REDACTED  Wed May 20 10:41:10 2015
From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=)
Date: Wed, 20 May 2015 10:41:10 +0200
Subject: [erlang-questions] Windows
In-Reply-To: <555C46E0.4060000@ninenines.eu>
References: <555C46E0.4060000@ninenines.eu>
Message-ID: 

If you have git installed on windows, it comes with a bash shell. Git
works, rebar works, C drivers work with visual studio, which those "fake"
developers are probably using anyway.


Sergej

On Wed, May 20, 2015 at 10:33 AM, Lo?c Hoguin  wrote:

> Hi,
>
> Yesterday the topic of Windows development was brought up on Twitter. I
> looked into it and it's clearly an issue. At best you can compile projects
> that don't involve shell scripts, makefiles or C code, so you can't really
> work on anything significant as a large number of open source projects
> involve one or the other (yes, even when using rebar).
>
> Now there's three ways Erlang could be more friendly to Windows developers:
>
> To use Windows tools imply making open source projects work for Windows.
> Sure perhaps it's not an issue for the simplest NIFs but some projects
> require C libraries which themselves aren't always easy to make work on
> Windows.
>
> MingW has the same issues. You need to fix everything to work on Windows.
>
> Cygwin makes this much easier. Everything seems to work... except Erlang.
> Erlang on Cygwin thinks it runs on win32 and, while true to a certain
> extent, it fails badly when dealing with files (cygwin has / as root while
> win32 has c:\).
>
> Now you might say Cygwin is massive. That's true, it takes some time to
> install and some space on the disk. But it's very easy to install and can
> be almost entirely automated using chocolatey and apt-cyg. And adding an
> SSH server on top of that is a couple commands away, so it is a very
> interesting solution if you ask me.
>
> Now Erlang compiles on Cygwin... but produces the win32 target. Which
> brings me to the main subject of this email: what would it involve to make
> Erlang work on Cygwin? Considering Cygwin is a mix of Windows and Linux
> it's probably just about enabling one or another piece of code, or is it?
>
> It could be a fine side project but I don't really know where to start.
> Any tips appreciated.
>
> PS: tried just patching os:type()... that didn't work. Ended up with
> Erlang not finding a file that existed on disk when running common_test.
>
> PPS: I know "real" developers use Unix. But the more numerous "fake"
> developers might be more interested in Erlang if they can keep their OS.
>
> PPPS: I know this isn't a good option if you're going to create
> Photoshoperl, but it's still a good option for server application
> development.
>
> --
> Lo?c Hoguin
> http://ninenines.eu
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From essen@REDACTED  Wed May 20 10:54:33 2015
From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=)
Date: Wed, 20 May 2015 11:54:33 +0300
Subject: [erlang-questions] Windows
In-Reply-To: 
References: <555C46E0.4060000@ninenines.eu>
 
Message-ID: <555C4BC9.6050100@ninenines.eu>

Not out of the box, though. Or at least not for those that require a 
Makefile or shell script at some point. I haven't seen a single project 
include Visual Studio project files. Some have a .bat script instead, 
most don't.

If you try to compile you first end up with a compile error (or worse, I 
just tried with one that runs a Makefile and the win32 GNU make I have 
just threw up an exception ahah) and then you have to fix it yourself?

It's not a smooth experience and I really can't tell people that's what 
they should do. Either the experience is smooth, or it's not and you're 
better off telling people to install Linux.

On 05/20/2015 11:41 AM, Sergej Jure?ko wrote:
> If you have git installed on windows, it comes with a bash shell. Git
> works, rebar works, C drivers work with visual studio, which those
> "fake" developers are probably using anyway.
>
>
> Sergej
>
> On Wed, May 20, 2015 at 10:33 AM, Lo?c Hoguin  > wrote:
>
>     Hi,
>
>     Yesterday the topic of Windows development was brought up on
>     Twitter. I looked into it and it's clearly an issue. At best you can
>     compile projects that don't involve shell scripts, makefiles or C
>     code, so you can't really work on anything significant as a large
>     number of open source projects involve one or the other (yes, even
>     when using rebar).
>
>     Now there's three ways Erlang could be more friendly to Windows
>     developers:
>
>     To use Windows tools imply making open source projects work for
>     Windows. Sure perhaps it's not an issue for the simplest NIFs but
>     some projects require C libraries which themselves aren't always
>     easy to make work on Windows.
>
>     MingW has the same issues. You need to fix everything to work on
>     Windows.
>
>     Cygwin makes this much easier. Everything seems to work... except
>     Erlang. Erlang on Cygwin thinks it runs on win32 and, while true to
>     a certain extent, it fails badly when dealing with files (cygwin has
>     / as root while win32 has c:\).
>
>     Now you might say Cygwin is massive. That's true, it takes some time
>     to install and some space on the disk. But it's very easy to install
>     and can be almost entirely automated using chocolatey and apt-cyg.
>     And adding an SSH server on top of that is a couple commands away,
>     so it is a very interesting solution if you ask me.
>
>     Now Erlang compiles on Cygwin... but produces the win32 target.
>     Which brings me to the main subject of this email: what would it
>     involve to make Erlang work on Cygwin? Considering Cygwin is a mix
>     of Windows and Linux it's probably just about enabling one or
>     another piece of code, or is it?
>
>     It could be a fine side project but I don't really know where to
>     start. Any tips appreciated.
>
>     PS: tried just patching os:type()... that didn't work. Ended up with
>     Erlang not finding a file that existed on disk when running common_test.
>
>     PPS: I know "real" developers use Unix. But the more numerous "fake"
>     developers might be more interested in Erlang if they can keep their OS.
>
>     PPPS: I know this isn't a good option if you're going to create
>     Photoshoperl, but it's still a good option for server application
>     development.
>
>     --
>     Lo?c Hoguin
>     http://ninenines.eu
>     _______________________________________________
>     erlang-questions mailing list
>     erlang-questions@REDACTED 
>     http://erlang.org/mailman/listinfo/erlang-questions
>
>

-- 
Lo?c Hoguin
http://ninenines.eu


From zxq9@REDACTED  Wed May 20 11:01:31 2015
From: zxq9@REDACTED (zxq9)
Date: Wed, 20 May 2015 18:01:31 +0900
Subject: [erlang-questions] Windows
In-Reply-To: <555C46E0.4060000@ninenines.eu>
References: <555C46E0.4060000@ninenines.eu>
Message-ID: <5574780.Rb8JpxtEvz@changa>

On 2015?5?20? ??? 11:33:36 Lo?c Hoguin wrote:
> Hi,
> 
> Yesterday the topic of Windows development was brought up on Twitter. I 
> looked into it and it's clearly an issue. ...

Indeed, it is.

The solution to this that we've been toying with (well, deploying in limited cases, but still not polished) is a from-source updater/builder that wraps an Erlang program. It checks a source repository (if available at the moment, or at all), validates the source package signature, and either runs the last built version of a program, or pulls-and-builds the latest version of a program at the time it is called.

This is all in Erlang, and so works the same way on Linux and Windows.

The use case, though, is probably an unusual one in the Erlang world: client applications, and *not* building releases (running everything with the host's Erlang runtime, the way one does Python/Perl/Ruby/Java programs).

The abnormality of the use case relative to Erlangdom explains why we couldn't find any tools for this sort of thing, but implementing them wasn't impossibly hard. The abnormality of the use case also indicates that probably nobody else has this use case or cares about it.

In our case using this method the real difference between Linux and Windows environments is how the initial program is installed -- in Windows a typical GUI click-to-install executable that does some registry magic and sets up batch files and icons is necessary to kick things off (well, necessary if you want a typical Windows user to actually use your programs).

The good news is that all the underlying stuff that is already included in a stock Erlang environment in the interest of building releases provides a complete base from which to build a from-source self-updating executor, which makes it easy to support Linux and Windows equally from the point of initial installation.

Hopefully we can polish up the tool a bit (its still horribly ugly), document the process of rolling a Windows installer, and open source it.

-Craig


From jose.valim@REDACTED  Wed May 20 11:21:54 2015
From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=)
Date: Wed, 20 May 2015 11:21:54 +0200
Subject: [erlang-questions] Windows
In-Reply-To: <555C46E0.4060000@ninenines.eu>
References: <555C46E0.4060000@ninenines.eu>
Message-ID: 

>
> Now there's three ways Erlang could be more friendly to Windows developers:
>

Actually, if we really want to make Erlang more friendly to Windows
developers, the best way is to NOT finish the e-mail where we discuss
Windows support with "I know "real" developers use Unix". Even if "real"
and "fake" are in quotes. How do you expect all Windows users to join the
discussion if you have just called them "fake" developers?

In my experience, Cygwin makes things easier if you are a Unix developer
and you have to use Windows OR if you are a Unix developer and you don't
care much about Windows, so telling people to use Cygwin is the easiest
option for *you*. However, most Windows developers prefer to not use Cygwin.

I said "in my experience" because we ran two polls via Google Forms in the
Elixir mailing list targeted for Windows developers asking about their tool
chain, how they installed erlang/elixir, what could be improved, etc.

Since your mileage may vary, I recommend doing the same here. Open up a
form, ask questions and collect feedback. It is going to yield a much
better result than having a bunch of Unix folks deciding what is best for
Windows.

Some may not like the answer though. My biggest complaint about erlang.mk,
for example, has been exactly it is Unix centric. In Windows, if you want
to provide good experience, you may need to provide a Makefile.win. In most
cases though, Rebar works fine or with little adjustment. Here is one
example of each case:

* https://github.com/devinus/markdown
* https://github.com/synrc/fs

On the other hand, we could do a poll and conclude Cygwin support is the
biggest problem, then that should be our focus. But without asking around
and gathering information, it is all just shots in the dark.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jani.j.hakala@REDACTED  Wed May 20 11:58:25 2015
From: jani.j.hakala@REDACTED (Jani Hakala)
Date: Wed, 20 May 2015 12:58:25 +0300
Subject: [erlang-questions] Windows
In-Reply-To: <555C46E0.4060000@ninenines.eu> (=?iso-8859-1?Q?=22Lo=EFc?=
 Hoguin"'s message of "Wed, 20 May 2015 11:33:36 +0300")
References: <555C46E0.4060000@ninenines.eu>
Message-ID: <877fs3muym.fsf@pingviini.dy.fi>

Lo?c Hoguin  writes:

> Now Erlang compiles on Cygwin... but produces the win32 target. Which
> brings me to the main subject of this email: what would it involve to
> make Erlang work on Cygwin? Considering Cygwin is a mix of Windows and
> Linux it's probably just about enabling one or another piece of code,
> or is it?
>
Cygwin API Licensing Terms (https://cygwin.com/licensing.html) might be
an issue. Is linking to cygwin.dll required, and if yes, is it possible
to load only NIFs and port drivers that are distributed with a license
that satisfies the Open Source Definition?

I think GPLv3 cygwin.dll license could be one the top reasons why
windows developers stay away from cygwin.

Jani Hakala


From essen@REDACTED  Wed May 20 11:58:27 2015
From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=)
Date: Wed, 20 May 2015 12:58:27 +0300
Subject: [erlang-questions] Windows
In-Reply-To: 
References: <555C46E0.4060000@ninenines.eu>
 
Message-ID: <555C5AC3.4020503@ninenines.eu>

On 05/20/2015 12:21 PM, Jos? Valim wrote:
>     Now there's three ways Erlang could be more friendly to Windows
>     developers:
>
>
> Actually, if we really want to make Erlang more friendly to Windows
> developers, the best way is to NOT finish the e-mail where we discuss
> Windows support with "I know "real" developers use Unix". Even if "real"
> and "fake" are in quotes. How do you expect all Windows users to join
> the discussion if you have just called them "fake" developers?

Well I'm not calling them that, hence the quotes. I was actually trying 
to avoid debates on that, but I guess that was premature optimization.

> In my experience, Cygwin makes things easier if you are a Unix developer
> and you have to use Windows OR if you are a Unix developer and you don't
> care much about Windows, so telling people to use Cygwin is the easiest
> option for *you*. However, most Windows developers prefer to not use Cygwin.

It's the easiest option for developers who are targetting Linux servers, 
as it is an environment on Windows that mirrors best the server. 
Everyone I have come in contact with that stayed on Windows just ended 
up installing a Linux VM. Cygwin is a lightweight alternative compared 
to that.

> I said "in my experience" because we ran two polls via Google Forms in
> the Elixir mailing list targeted for Windows developers asking about
> their tool chain, how they installed erlang/elixir, what could be
> improved, etc.
>
> Since your mileage may vary, I recommend doing the same here. Open up a
> form, ask questions and collect feedback. It is going to yield a much
> better result than having a bunch of Unix folks deciding what is best
> for Windows.

My goal is to improve the lives of Windows developers who then deploy on 
Linux servers. This is something I can do.

I can't improve the lives of Windows developers who then deploy on 
Windows servers (or clients). I can't fix and maintain all the projects 
that don't work on Windows. That's something Windows developers should 
do if they want to.

As an added benefit of having Erlang work on Cygwin, you can just give a 
5 steps automated Cygwin installation instructions to people coming to 
your tutorial and then you don't have to worry about people using 
Windows machines having issues.

> Some may not like the answer though. My biggest complaint about
> erlang.mk , for example, has been exactly it is Unix
> centric.

It is also mine, and at some point in time it won't be Unix centric 
anymore. GNU make 4 gives me everything I need to not require a Unix 
environment. But it's probably still a little early to require make 4 so 
I am waiting for now.

However this wouldn't fix all the third party projects. Nor should it.

There will always be the problem of projects depending on libs not 
easily available on Windows. So the Windows/Windows developers have to 
step up at some point if they want nice things.

However this is not true for the Windows/Linux developers, as those libs 
are on Cygwin and so they can work in a Cygwin environment and then 
deploy to Linux.

> In Windows, if you want to provide good experience, you may
> need to provide a Makefile.win. In most cases though, Rebar works fine
> or with little adjustment. Here is one example of each case:
>
> * https://github.com/devinus/markdown
> * https://github.com/synrc/fs

Try to make this one work:

* https://github.com/basho/eleveldb

It's nowhere near close to compile with no or little adjustment. The 
same is true of the C library it uses, leveldb.

It's also one of the most used dependency in the nearly 380 packages in 
the erlang.mk package index. (Come to think of it, I will make some 
stats the next time I do a full run.)

> On the other hand, we could do a poll and conclude Cygwin support is the
> biggest problem, then that should be our focus. But without asking
> around and gathering information, it is all just shots in the dark.

It's not so much about what the biggest problem is, but more about 
what's the biggest problem I can fix. One step at a time...

-- 
Lo?c Hoguin
http://ninenines.eu


From essen@REDACTED  Wed May 20 12:05:47 2015
From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=)
Date: Wed, 20 May 2015 13:05:47 +0300
Subject: [erlang-questions] Windows
In-Reply-To: <877fs3muym.fsf@pingviini.dy.fi>
References: <555C46E0.4060000@ninenines.eu> <877fs3muym.fsf@pingviini.dy.fi>
Message-ID: <555C5C7B.4060006@ninenines.eu>

On 05/20/2015 12:58 PM, Jani Hakala wrote:
> Lo?c Hoguin  writes:
>
>> Now Erlang compiles on Cygwin... but produces the win32 target. Which
>> brings me to the main subject of this email: what would it involve to
>> make Erlang work on Cygwin? Considering Cygwin is a mix of Windows and
>> Linux it's probably just about enabling one or another piece of code,
>> or is it?
>>
> Cygwin API Licensing Terms (https://cygwin.com/licensing.html) might be
> an issue. Is linking to cygwin.dll required, and if yes, is it possible
> to load only NIFs and port drivers that are distributed with a license
> that satisfies the Open Source Definition?
>
> I think GPLv3 cygwin.dll license could be one the top reasons why
> windows developers stay away from cygwin.

This is largely a non-issue.

The GPL only applies if you distribute the source code and/or the 
resulting build that links to Cygwin.

The use case I am trying to improve is providing a development 
environment for people who deploy to Linux servers. Cygwin is not part 
of the resulting software.

See my email to Jose Valim for more details.

-- 
Lo?c Hoguin
http://ninenines.eu


From dangud@REDACTED  Wed May 20 12:30:52 2015
From: dangud@REDACTED (Dan Gudmundsson)
Date: Wed, 20 May 2015 10:30:52 +0000
Subject: [erlang-questions] Windows
In-Reply-To: <555C5C7B.4060006@ninenines.eu>
References: <555C46E0.4060000@ninenines.eu> <877fs3muym.fsf@pingviini.dy.fi>
 <555C5C7B.4060006@ninenines.eu>
Message-ID: 

Does not cygwin still produce executables that depend on cygwin.dll, i.e.
non native windows programs
that will not work if you do not have cygwin (I know you can crosscompile
with mingw compilers but that does not make
it easier).

I have ditched cygwin since long ago, and I prefer mingw and msys, it works
and you can use Microsoft compiler or mingw compilers.
rebar works and even erlang.mk worked as far as I tried it.

IMO cygwin is a rabbit hole, do not go there it just makes it worse.


On Wed, May 20, 2015 at 12:05 PM Lo?c Hoguin  wrote:

> On 05/20/2015 12:58 PM, Jani Hakala wrote:
> > Lo?c Hoguin  writes:
> >
> >> Now Erlang compiles on Cygwin... but produces the win32 target. Which
> >> brings me to the main subject of this email: what would it involve to
> >> make Erlang work on Cygwin? Considering Cygwin is a mix of Windows and
> >> Linux it's probably just about enabling one or another piece of code,
> >> or is it?
> >>
> > Cygwin API Licensing Terms (https://cygwin.com/licensing.html) might be
> > an issue. Is linking to cygwin.dll required, and if yes, is it possible
> > to load only NIFs and port drivers that are distributed with a license
> > that satisfies the Open Source Definition?
> >
> > I think GPLv3 cygwin.dll license could be one the top reasons why
> > windows developers stay away from cygwin.
>
> This is largely a non-issue.
>
> The GPL only applies if you distribute the source code and/or the
> resulting build that links to Cygwin.
>
> The use case I am trying to improve is providing a development
> environment for people who deploy to Linux servers. Cygwin is not part
> of the resulting software.
>
> See my email to Jose Valim for more details.
>
> --
> Lo?c Hoguin
> http://ninenines.eu
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jose.valim@REDACTED  Wed May 20 12:35:18 2015
From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=)
Date: Wed, 20 May 2015 12:35:18 +0200
Subject: [erlang-questions] Trying to understand the performance impact of
	binary:split/3
Message-ID: 

Hello folks,

At the beginning of the month, someone wrote a blog post comparing data
processing between different platforms and languages, one of them being
Erlang VM/Elixir:

http://blog.dimroc.com/2015/05/07/etl-language-showdown-pt2/

After running the experiments, I thought we could do much better. To my
surprise, our biggest performance hit was when calling binary:split/3. I
have rewritten the code to use only Erlang function calls (to make it
clearer for this discussion):

https://github.com/josevalim/etl-language-comparison/blob/jv-erl/elixir/lib/map_actor.ex

The performance in both Erlang and Elixir variants are the same (rewritten
in Erlang is also the same result). This line is the bottleneck:

https://github.com/josevalim/etl-language-comparison/blob/jv-erl/elixir/lib/map_actor.ex#L11

In fact, if we move the regular expression check to before the
binary:split/3 call, we get the same performance as Go in my machine.
Meaning that binary:split/3 is making the code at least twice slower.

The binary:split/3 implementation is broken in two pieces: first we find
all matches via binary:matches/3 and then we traverse the matches
converting them to binaries with binary:part/3. The binary:part/3 call is
the slow piece here.

*My question is:* is this expected? Why binary:split/3 (and binary:part/3)
is affecting performance so drastically? How can I investigate/understand
this further?

## Other bottlenecks

The other two immediate bottlenecks are the use of regular expressions and
the use of file:read_line/3 instead of loading the whole file into memory.
Those were given as hard requirements by the author. None the less, someone
wrote an Erlang implementation that removes those bottlenecks too (along
binary:split/3) and the performance is outstanding:

https://github.com/dimroc/etl-language-comparison/pull/10/files

I have since then rewritten the Elixir one and got a similar result.
However I am still puzzled because using binary:split/3 would have been my
first try (instead of relying on match+part) as it leads to cleaner code
(imo).

Thanks.

*Jos? Valim*
www.plataformatec.com.br
Skype: jv.ptec
Founder and Lead Developer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From essen@REDACTED  Wed May 20 12:35:47 2015
From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=)
Date: Wed, 20 May 2015 13:35:47 +0300
Subject: [erlang-questions] Windows
In-Reply-To: 
References: <555C46E0.4060000@ninenines.eu> <877fs3muym.fsf@pingviini.dy.fi>
 <555C5C7B.4060006@ninenines.eu>
 
Message-ID: <555C6383.20508@ninenines.eu>

I will have to try again, I was not able to make it work yesterday.

On 05/20/2015 01:30 PM, Dan Gudmundsson wrote:
> Does not cygwin still produce executables that depend on cygwin.dll,
> i.e. non native windows programs
> that will not work if you do not have cygwin (I know you can
> crosscompile with mingw compilers but that does not make
> it easier).
>
> I have ditched cygwin since long ago, and I prefer mingw and msys, it
> works and you can use Microsoft compiler or mingw compilers.
> rebar works and even erlang.mk  worked as far as I
> tried it.
>
> IMO cygwin is a rabbit hole, do not go there it just makes it worse.
>
>
> On Wed, May 20, 2015 at 12:05 PM Lo?c Hoguin  > wrote:
>
>     On 05/20/2015 12:58 PM, Jani Hakala wrote:
>      > Lo?c Hoguin > writes:
>      >
>      >> Now Erlang compiles on Cygwin... but produces the win32 target.
>     Which
>      >> brings me to the main subject of this email: what would it
>     involve to
>      >> make Erlang work on Cygwin? Considering Cygwin is a mix of
>     Windows and
>      >> Linux it's probably just about enabling one or another piece of
>     code,
>      >> or is it?
>      >>
>      > Cygwin API Licensing Terms (https://cygwin.com/licensing.html)
>     might be
>      > an issue. Is linking to cygwin.dll required, and if yes, is it
>     possible
>      > to load only NIFs and port drivers that are distributed with a
>     license
>      > that satisfies the Open Source Definition?
>      >
>      > I think GPLv3 cygwin.dll license could be one the top reasons why
>      > windows developers stay away from cygwin.
>
>     This is largely a non-issue.
>
>     The GPL only applies if you distribute the source code and/or the
>     resulting build that links to Cygwin.
>
>     The use case I am trying to improve is providing a development
>     environment for people who deploy to Linux servers. Cygwin is not part
>     of the resulting software.
>
>     See my email to Jose Valim for more details.
>
>     --
>     Lo?c Hoguin
>     http://ninenines.eu
>     _______________________________________________
>     erlang-questions mailing list
>     erlang-questions@REDACTED 
>     http://erlang.org/mailman/listinfo/erlang-questions
>

-- 
Lo?c Hoguin
http://ninenines.eu


From zxq9@REDACTED  Wed May 20 12:55:34 2015
From: zxq9@REDACTED (zxq9)
Date: Wed, 20 May 2015 19:55:34 +0900
Subject: [erlang-questions] Windows
In-Reply-To: 
References: <555C46E0.4060000@ninenines.eu> <555C5C7B.4060006@ninenines.eu>
 
Message-ID: <1557319.02jgbjD5y7@changa>

On 2015?5?20? ??? 10:30:52 Dan Gudmundsson wrote:
> Does not cygwin still produce executables that depend on cygwin.dll, i.e.
> non native windows programs
> that will not work if you do not have cygwin (I know you can crosscompile
> with mingw compilers but that does not make
> it easier).
> 
> I have ditched cygwin since long ago, and I prefer mingw and msys, it works
> and you can use Microsoft compiler or mingw compilers.
> rebar works and even erlang.mk worked as far as I tried it.
> 
> IMO cygwin is a rabbit hole, do not go there it just makes it worse.

I think there are three categories where cygwin works:

1. You are providing a platform, so you can statically compile certain things and include the necessary cygwin elements with your platform (have the installer do it -- cygwin's own installer was even developed in a way that makes creating your own environment installer based on it easyish). It seems that this is the case with Erlang Solution's own installer package (the mingw build process for Erlang appears to still be black magic, unless that has changed since I last checked).

2. You are developing for yourself and for whatever reason the overhead of maintaining a cygwin environment seems like a good tradeoff compared to just running a different OS either natively or within a VM.

3. You have a paid support structure that takes care of the cygwin stuff for you.

All of the above cases are developer or server-side situations. As far as "average Windows user, client-side target audience" type things go, mingw is so vastly superior that it is difficult to even imagine a point of comparison: one builds programs that can work within a Unixy environment on Windows, and the other builds actual Windows programs.

I did not address MS's Visual Foo environments because including a compiler with an installation in a separate installer isn't straightforward as licenses go, so I have no experience in that area, but assuming the proper money changes hands and rituals attended I can only imagine that the result would be *very* much like mingw, and very unlike cygwin.

Once again, though, Erlang is used almost exclusively on the server-side, so it may not be unreasonable to accept some level of administrative overhead to maintain an environment -- especially since it *is* possible to distribute cygwin-built binaries, just not very straightforward compared to real windows binaries. In addition, while this or that Erlang release-building tool may or may not take certain Unix utilities for granted, the built-in utilites (which very few use) do not. If there is no C code involved, rebar tends to work on Windows -- but a tool based on gnu make won't. That's an issue with the tool, not Erlang, its runtime, or anything else in the core distribution.

Perhaps that is a big part of the problem here. Most server development for web/game/chat/mobile app backends target Linux, not Windows. Quite a lot of client-side software targets Windows, Linux and OSX all at once as a matter of course (if not mobile platforms as well). The expectations are radically different between the two development cases. So when one says "we need to support Foo OS development environments better" you really have to include "... in the context of [client || server || wev] development" or the statement is meaningless.

Erlang doesn't support developing client software very well at all and Windows just happens to get pinched in the crossfire there, with client software being its most common case. Its not just a Windows problem, Erlang is a challenging environment within which to develop Linux, BSD and OSX client software as well. The focus on server-side and tiny-service-device releases skews the entire community's approach to development toward server code, not client code.

-Craig


From sergej.jurecko@REDACTED  Wed May 20 12:56:02 2015
From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=)
Date: Wed, 20 May 2015 12:56:02 +0200
Subject: [erlang-questions] Trying to understand the performance impact
	of binary:split/3
In-Reply-To: 
References: 
Message-ID: 

binary:split is not fast and unfortunately many people do not realize that.
If you want speed, here is an implementation that is made for speed:
https://github.com/biokoda/bkdcore/blob/master/src/butil.erl#L2359
https://github.com/biokoda/bkdcore/blob/master/src/butil.erl#L2373

Sergej

On Wed, May 20, 2015 at 12:35 PM, Jos? Valim <
jose.valim@REDACTED> wrote:

> Hello folks,
>
> At the beginning of the month, someone wrote a blog post comparing data
> processing between different platforms and languages, one of them being
> Erlang VM/Elixir:
>
> http://blog.dimroc.com/2015/05/07/etl-language-showdown-pt2/
>
> After running the experiments, I thought we could do much better. To my
> surprise, our biggest performance hit was when calling binary:split/3. I
> have rewritten the code to use only Erlang function calls (to make it
> clearer for this discussion):
>
>
> https://github.com/josevalim/etl-language-comparison/blob/jv-erl/elixir/lib/map_actor.ex
>
> The performance in both Erlang and Elixir variants are the same (rewritten
> in Erlang is also the same result). This line is the bottleneck:
>
>
> https://github.com/josevalim/etl-language-comparison/blob/jv-erl/elixir/lib/map_actor.ex#L11
>
> In fact, if we move the regular expression check to before the
> binary:split/3 call, we get the same performance as Go in my machine.
> Meaning that binary:split/3 is making the code at least twice slower.
>
> The binary:split/3 implementation is broken in two pieces: first we find
> all matches via binary:matches/3 and then we traverse the matches
> converting them to binaries with binary:part/3. The binary:part/3 call is
> the slow piece here.
>
> *My question is:* is this expected? Why binary:split/3 (and
> binary:part/3) is affecting performance so drastically? How can I
> investigate/understand this further?
>
> ## Other bottlenecks
>
> The other two immediate bottlenecks are the use of regular expressions and
> the use of file:read_line/3 instead of loading the whole file into memory.
> Those were given as hard requirements by the author. None the less, someone
> wrote an Erlang implementation that removes those bottlenecks too (along
> binary:split/3) and the performance is outstanding:
>
> https://github.com/dimroc/etl-language-comparison/pull/10/files
>
> I have since then rewritten the Elixir one and got a similar result.
> However I am still puzzled because using binary:split/3 would have been my
> first try (instead of relying on match+part) as it leads to cleaner code
> (imo).
>
> Thanks.
>
> *Jos? Valim*
> www.plataformatec.com.br
> Skype: jv.ptec
> Founder and Lead Developer
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From darach@REDACTED  Wed May 20 13:07:30 2015
From: darach@REDACTED (Darach Ennis)
Date: Wed, 20 May 2015 12:07:30 +0100
Subject: [erlang-questions] Trying to understand the performance impact
	of binary:split/3
In-Reply-To: 
References: 
 
Message-ID: 

Hi Sergej,

Have you any rough benchmark numbers? Perhaps this is worth
contributing as a replacement to binary:split?

Cheers,

Darach.

On Wed, May 20, 2015 at 11:56 AM, Sergej Jure?ko 
wrote:

> binary:split is not fast and unfortunately many people do not realize
> that.
> If you want speed, here is an implementation that is made for speed:
> https://github.com/biokoda/bkdcore/blob/master/src/butil.erl#L2359
> https://github.com/biokoda/bkdcore/blob/master/src/butil.erl#L2373
>
> Sergej
>
> On Wed, May 20, 2015 at 12:35 PM, Jos? Valim <
> jose.valim@REDACTED> wrote:
>
>> Hello folks,
>>
>> At the beginning of the month, someone wrote a blog post comparing data
>> processing between different platforms and languages, one of them being
>> Erlang VM/Elixir:
>>
>> http://blog.dimroc.com/2015/05/07/etl-language-showdown-pt2/
>>
>> After running the experiments, I thought we could do much better. To my
>> surprise, our biggest performance hit was when calling binary:split/3. I
>> have rewritten the code to use only Erlang function calls (to make it
>> clearer for this discussion):
>>
>>
>> https://github.com/josevalim/etl-language-comparison/blob/jv-erl/elixir/lib/map_actor.ex
>>
>> The performance in both Erlang and Elixir variants are the same
>> (rewritten in Erlang is also the same result). This line is the bottleneck:
>>
>>
>> https://github.com/josevalim/etl-language-comparison/blob/jv-erl/elixir/lib/map_actor.ex#L11
>>
>> In fact, if we move the regular expression check to before the
>> binary:split/3 call, we get the same performance as Go in my machine.
>> Meaning that binary:split/3 is making the code at least twice slower.
>>
>> The binary:split/3 implementation is broken in two pieces: first we find
>> all matches via binary:matches/3 and then we traverse the matches
>> converting them to binaries with binary:part/3. The binary:part/3 call is
>> the slow piece here.
>>
>> *My question is:* is this expected? Why binary:split/3 (and
>> binary:part/3) is affecting performance so drastically? How can I
>> investigate/understand this further?
>>
>> ## Other bottlenecks
>>
>> The other two immediate bottlenecks are the use of regular expressions
>> and the use of file:read_line/3 instead of loading the whole file into
>> memory. Those were given as hard requirements by the author. None the less,
>> someone wrote an Erlang implementation that removes those bottlenecks too
>> (along binary:split/3) and the performance is outstanding:
>>
>> https://github.com/dimroc/etl-language-comparison/pull/10/files
>>
>> I have since then rewritten the Elixir one and got a similar result.
>> However I am still puzzled because using binary:split/3 would have been my
>> first try (instead of relying on match+part) as it leads to cleaner code
>> (imo).
>>
>> Thanks.
>>
>> *Jos? Valim*
>> www.plataformatec.com.br
>> Skype: jv.ptec
>> Founder and Lead Developer
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From sergej.jurecko@REDACTED  Wed May 20 13:13:23 2015
From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=)
Date: Wed, 20 May 2015 13:13:23 +0200
Subject: [erlang-questions] Trying to understand the performance impact
	of binary:split/3
In-Reply-To: 
References: 
 
 
Message-ID: 

Sorry I don't. It's been a while since I wrote it. It should be pretty easy
to test though.

Sergej

On Wed, May 20, 2015 at 1:07 PM, Darach Ennis  wrote:

> Hi Sergej,
>
> Have you any rough benchmark numbers? Perhaps this is worth
> contributing as a replacement to binary:split?
>
> Cheers,
>
> Darach.
>
> On Wed, May 20, 2015 at 11:56 AM, Sergej Jure?ko  > wrote:
>
>> binary:split is not fast and unfortunately many people do not realize
>> that.
>> If you want speed, here is an implementation that is made for speed:
>> https://github.com/biokoda/bkdcore/blob/master/src/butil.erl#L2359
>> https://github.com/biokoda/bkdcore/blob/master/src/butil.erl#L2373
>>
>> Sergej
>>
>> On Wed, May 20, 2015 at 12:35 PM, Jos? Valim <
>> jose.valim@REDACTED> wrote:
>>
>>> Hello folks,
>>>
>>> At the beginning of the month, someone wrote a blog post comparing data
>>> processing between different platforms and languages, one of them being
>>> Erlang VM/Elixir:
>>>
>>> http://blog.dimroc.com/2015/05/07/etl-language-showdown-pt2/
>>>
>>> After running the experiments, I thought we could do much better. To my
>>> surprise, our biggest performance hit was when calling binary:split/3. I
>>> have rewritten the code to use only Erlang function calls (to make it
>>> clearer for this discussion):
>>>
>>>
>>> https://github.com/josevalim/etl-language-comparison/blob/jv-erl/elixir/lib/map_actor.ex
>>>
>>> The performance in both Erlang and Elixir variants are the same
>>> (rewritten in Erlang is also the same result). This line is the bottleneck:
>>>
>>>
>>> https://github.com/josevalim/etl-language-comparison/blob/jv-erl/elixir/lib/map_actor.ex#L11
>>>
>>> In fact, if we move the regular expression check to before the
>>> binary:split/3 call, we get the same performance as Go in my machine.
>>> Meaning that binary:split/3 is making the code at least twice slower.
>>>
>>> The binary:split/3 implementation is broken in two pieces: first we find
>>> all matches via binary:matches/3 and then we traverse the matches
>>> converting them to binaries with binary:part/3. The binary:part/3 call is
>>> the slow piece here.
>>>
>>> *My question is:* is this expected? Why binary:split/3 (and
>>> binary:part/3) is affecting performance so drastically? How can I
>>> investigate/understand this further?
>>>
>>> ## Other bottlenecks
>>>
>>> The other two immediate bottlenecks are the use of regular expressions
>>> and the use of file:read_line/3 instead of loading the whole file into
>>> memory. Those were given as hard requirements by the author. None the less,
>>> someone wrote an Erlang implementation that removes those bottlenecks too
>>> (along binary:split/3) and the performance is outstanding:
>>>
>>> https://github.com/dimroc/etl-language-comparison/pull/10/files
>>>
>>> I have since then rewritten the Elixir one and got a similar result.
>>> However I am still puzzled because using binary:split/3 would have been my
>>> first try (instead of relying on match+part) as it leads to cleaner code
>>> (imo).
>>>
>>> Thanks.
>>>
>>> *Jos? Valim*
>>> www.plataformatec.com.br
>>> Skype: jv.ptec
>>> Founder and Lead Developer
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jose.valim@REDACTED  Wed May 20 13:14:41 2015
From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=)
Date: Wed, 20 May 2015 13:14:41 +0200
Subject: [erlang-questions] Trying to understand the performance impact
	of binary:split/3
In-Reply-To: 
References: 
 
Message-ID: 

Thank you Sergej.

I have created a branch that uses the split version you mentioned and it is
4x times slower than using binary:split/3. Here is the commit that added
the new implementation:

https://github.com/josevalim/etl-language-comparison/commit/e6cf0a35700cef751b1052083ccec5a3c0394648

Thoughts?



*Jos? Valim*
www.plataformatec.com.br
Skype: jv.ptec
Founder and Lead Developer

On Wed, May 20, 2015 at 12:56 PM, Sergej Jure?ko 
wrote:

> binary:split is not fast and unfortunately many people do not realize
> that.
> If you want speed, here is an implementation that is made for speed:
> https://github.com/biokoda/bkdcore/blob/master/src/butil.erl#L2359
> https://github.com/biokoda/bkdcore/blob/master/src/butil.erl#L2373
>
> Sergej
>
> On Wed, May 20, 2015 at 12:35 PM, Jos? Valim <
> jose.valim@REDACTED> wrote:
>
>> Hello folks,
>>
>> At the beginning of the month, someone wrote a blog post comparing data
>> processing between different platforms and languages, one of them being
>> Erlang VM/Elixir:
>>
>> http://blog.dimroc.com/2015/05/07/etl-language-showdown-pt2/
>>
>> After running the experiments, I thought we could do much better. To my
>> surprise, our biggest performance hit was when calling binary:split/3. I
>> have rewritten the code to use only Erlang function calls (to make it
>> clearer for this discussion):
>>
>>
>> https://github.com/josevalim/etl-language-comparison/blob/jv-erl/elixir/lib/map_actor.ex
>>
>> The performance in both Erlang and Elixir variants are the same
>> (rewritten in Erlang is also the same result). This line is the bottleneck:
>>
>>
>> https://github.com/josevalim/etl-language-comparison/blob/jv-erl/elixir/lib/map_actor.ex#L11
>>
>> In fact, if we move the regular expression check to before the
>> binary:split/3 call, we get the same performance as Go in my machine.
>> Meaning that binary:split/3 is making the code at least twice slower.
>>
>> The binary:split/3 implementation is broken in two pieces: first we find
>> all matches via binary:matches/3 and then we traverse the matches
>> converting them to binaries with binary:part/3. The binary:part/3 call is
>> the slow piece here.
>>
>> *My question is:* is this expected? Why binary:split/3 (and
>> binary:part/3) is affecting performance so drastically? How can I
>> investigate/understand this further?
>>
>> ## Other bottlenecks
>>
>> The other two immediate bottlenecks are the use of regular expressions
>> and the use of file:read_line/3 instead of loading the whole file into
>> memory. Those were given as hard requirements by the author. None the less,
>> someone wrote an Erlang implementation that removes those bottlenecks too
>> (along binary:split/3) and the performance is outstanding:
>>
>> https://github.com/dimroc/etl-language-comparison/pull/10/files
>>
>> I have since then rewritten the Elixir one and got a similar result.
>> However I am still puzzled because using binary:split/3 would have been my
>> first try (instead of relying on match+part) as it leads to cleaner code
>> (imo).
>>
>> Thanks.
>>
>> *Jos? Valim*
>> www.plataformatec.com.br
>> Skype: jv.ptec
>> Founder and Lead Developer
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From sergej.jurecko@REDACTED  Wed May 20 13:29:07 2015
From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=)
Date: Wed, 20 May 2015 13:29:07 +0200
Subject: [erlang-questions] Trying to understand the performance impact
	of binary:split/3
In-Reply-To: 
References: 
 
 
Message-ID: 

Well I'll be damned. I thought I read somewhere that binary module was
implemented in erlang (not bifs) which was why it was slow. I guess I never
checked. I take my statements back :)

Sergej

On Wed, May 20, 2015 at 1:14 PM, Jos? Valim  wrote:

> Thank you Sergej.
>
> I have created a branch that uses the split version you mentioned and it
> is 4x times slower than using binary:split/3. Here is the commit that added
> the new implementation:
>
>
> https://github.com/josevalim/etl-language-comparison/commit/e6cf0a35700cef751b1052083ccec5a3c0394648
>
> Thoughts?
>
>
>
> *Jos? Valim*
> www.plataformatec.com.br
> Skype: jv.ptec
> Founder and Lead Developer
>
> On Wed, May 20, 2015 at 12:56 PM, Sergej Jure?ko  > wrote:
>
>> binary:split is not fast and unfortunately many people do not realize
>> that.
>> If you want speed, here is an implementation that is made for speed:
>> https://github.com/biokoda/bkdcore/blob/master/src/butil.erl#L2359
>> https://github.com/biokoda/bkdcore/blob/master/src/butil.erl#L2373
>>
>> Sergej
>>
>> On Wed, May 20, 2015 at 12:35 PM, Jos? Valim <
>> jose.valim@REDACTED> wrote:
>>
>>> Hello folks,
>>>
>>> At the beginning of the month, someone wrote a blog post comparing data
>>> processing between different platforms and languages, one of them being
>>> Erlang VM/Elixir:
>>>
>>> http://blog.dimroc.com/2015/05/07/etl-language-showdown-pt2/
>>>
>>> After running the experiments, I thought we could do much better. To my
>>> surprise, our biggest performance hit was when calling binary:split/3. I
>>> have rewritten the code to use only Erlang function calls (to make it
>>> clearer for this discussion):
>>>
>>>
>>> https://github.com/josevalim/etl-language-comparison/blob/jv-erl/elixir/lib/map_actor.ex
>>>
>>> The performance in both Erlang and Elixir variants are the same
>>> (rewritten in Erlang is also the same result). This line is the bottleneck:
>>>
>>>
>>> https://github.com/josevalim/etl-language-comparison/blob/jv-erl/elixir/lib/map_actor.ex#L11
>>>
>>> In fact, if we move the regular expression check to before the
>>> binary:split/3 call, we get the same performance as Go in my machine.
>>> Meaning that binary:split/3 is making the code at least twice slower.
>>>
>>> The binary:split/3 implementation is broken in two pieces: first we find
>>> all matches via binary:matches/3 and then we traverse the matches
>>> converting them to binaries with binary:part/3. The binary:part/3 call is
>>> the slow piece here.
>>>
>>> *My question is:* is this expected? Why binary:split/3 (and
>>> binary:part/3) is affecting performance so drastically? How can I
>>> investigate/understand this further?
>>>
>>> ## Other bottlenecks
>>>
>>> The other two immediate bottlenecks are the use of regular expressions
>>> and the use of file:read_line/3 instead of loading the whole file into
>>> memory. Those were given as hard requirements by the author. None the less,
>>> someone wrote an Erlang implementation that removes those bottlenecks too
>>> (along binary:split/3) and the performance is outstanding:
>>>
>>> https://github.com/dimroc/etl-language-comparison/pull/10/files
>>>
>>> I have since then rewritten the Elixir one and got a similar result.
>>> However I am still puzzled because using binary:split/3 would have been my
>>> first try (instead of relying on match+part) as it leads to cleaner code
>>> (imo).
>>>
>>> Thanks.
>>>
>>> *Jos? Valim*
>>> www.plataformatec.com.br
>>> Skype: jv.ptec
>>> Founder and Lead Developer
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jose.valim@REDACTED  Wed May 20 13:31:44 2015
From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=)
Date: Wed, 20 May 2015 13:31:44 +0200
Subject: [erlang-questions] Trying to understand the performance impact
	of binary:split/3
In-Reply-To: 
References: 
 
 
 
Message-ID: 

Most of the binary module is implemented as BIFs with the exception of
binary:split/3 and binary:replace/4. So binary:split/3 is indeed written in
pure Erlang although the matches are found with binary:matches/3 (which is
a BIF).



*Jos? Valim*
www.plataformatec.com.br
Skype: jv.ptec
Founder and Lead Developer

On Wed, May 20, 2015 at 1:29 PM, Sergej Jure?ko 
wrote:

> Well I'll be damned. I thought I read somewhere that binary module was
> implemented in erlang (not bifs) which was why it was slow. I guess I never
> checked. I take my statements back :)
>
> Sergej
>
> On Wed, May 20, 2015 at 1:14 PM, Jos? Valim <
> jose.valim@REDACTED> wrote:
>
>> Thank you Sergej.
>>
>> I have created a branch that uses the split version you mentioned and it
>> is 4x times slower than using binary:split/3. Here is the commit that added
>> the new implementation:
>>
>>
>> https://github.com/josevalim/etl-language-comparison/commit/e6cf0a35700cef751b1052083ccec5a3c0394648
>>
>> Thoughts?
>>
>>
>>
>> *Jos? Valim*
>> www.plataformatec.com.br
>> Skype: jv.ptec
>> Founder and Lead Developer
>>
>> On Wed, May 20, 2015 at 12:56 PM, Sergej Jure?ko <
>> sergej.jurecko@REDACTED> wrote:
>>
>>> binary:split is not fast and unfortunately many people do not realize
>>> that.
>>> If you want speed, here is an implementation that is made for speed:
>>> https://github.com/biokoda/bkdcore/blob/master/src/butil.erl#L2359
>>> https://github.com/biokoda/bkdcore/blob/master/src/butil.erl#L2373
>>>
>>> Sergej
>>>
>>> On Wed, May 20, 2015 at 12:35 PM, Jos? Valim <
>>> jose.valim@REDACTED> wrote:
>>>
>>>> Hello folks,
>>>>
>>>> At the beginning of the month, someone wrote a blog post comparing data
>>>> processing between different platforms and languages, one of them being
>>>> Erlang VM/Elixir:
>>>>
>>>> http://blog.dimroc.com/2015/05/07/etl-language-showdown-pt2/
>>>>
>>>> After running the experiments, I thought we could do much better. To my
>>>> surprise, our biggest performance hit was when calling binary:split/3. I
>>>> have rewritten the code to use only Erlang function calls (to make it
>>>> clearer for this discussion):
>>>>
>>>>
>>>> https://github.com/josevalim/etl-language-comparison/blob/jv-erl/elixir/lib/map_actor.ex
>>>>
>>>> The performance in both Erlang and Elixir variants are the same
>>>> (rewritten in Erlang is also the same result). This line is the bottleneck:
>>>>
>>>>
>>>> https://github.com/josevalim/etl-language-comparison/blob/jv-erl/elixir/lib/map_actor.ex#L11
>>>>
>>>> In fact, if we move the regular expression check to before the
>>>> binary:split/3 call, we get the same performance as Go in my machine.
>>>> Meaning that binary:split/3 is making the code at least twice slower.
>>>>
>>>> The binary:split/3 implementation is broken in two pieces: first we
>>>> find all matches via binary:matches/3 and then we traverse the matches
>>>> converting them to binaries with binary:part/3. The binary:part/3 call is
>>>> the slow piece here.
>>>>
>>>> *My question is:* is this expected? Why binary:split/3 (and
>>>> binary:part/3) is affecting performance so drastically? How can I
>>>> investigate/understand this further?
>>>>
>>>> ## Other bottlenecks
>>>>
>>>> The other two immediate bottlenecks are the use of regular expressions
>>>> and the use of file:read_line/3 instead of loading the whole file into
>>>> memory. Those were given as hard requirements by the author. None the less,
>>>> someone wrote an Erlang implementation that removes those bottlenecks too
>>>> (along binary:split/3) and the performance is outstanding:
>>>>
>>>> https://github.com/dimroc/etl-language-comparison/pull/10/files
>>>>
>>>> I have since then rewritten the Elixir one and got a similar result.
>>>> However I am still puzzled because using binary:split/3 would have been my
>>>> first try (instead of relying on match+part) as it leads to cleaner code
>>>> (imo).
>>>>
>>>> Thanks.
>>>>
>>>> *Jos? Valim*
>>>> www.plataformatec.com.br
>>>> Skype: jv.ptec
>>>> Founder and Lead Developer
>>>>
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From hanssv@REDACTED  Wed May 20 13:21:06 2015
From: hanssv@REDACTED (Hans Svensson)
Date: Wed, 20 May 2015 13:21:06 +0200
Subject: [erlang-questions] Erlang Workshop 2015: deadline extension
Message-ID: <555C6E22.9070803@gmail.com>

Hi everyone,

The PC has decided to extend the paper submission deadline for the 14th 
ACM SIGPLAN Erlang Workshop.

New final dates are as follows, there will not be any further extensions

Submissions due: Friday, 29 May, 2015 [extended]
Author notification: Friday, 26 June, 2015 [unchanged]
Final copy due: Sunday, 19 July, 2015 [unchanged]
Workshop date: September 4, 2015 [unchanged]

Apologies for any duplicates you may receive.

CALL FOR PAPERS
===============

Fourteenth ACM SIGPLAN Erlang Workshop
-----------------------------------------------------------

Vancouver, Canada, September 4, 2015
Satellite event of the 20th ACM SIGPLAN International Conference on
Functional Programming (ICFP 2015)
August 30 - September 5, 2015

http://www.erlang.org/workshop/2015/ErlangWorkshop2015.html

Erlang is a concurrent, distributed functional programming language
aimed at systems with requirements of massive concurrency, soft real
time response, fault tolerance, and high availability. It has been
available as open source for 16 years, creating a community that
actively contributes to its already existing rich set of libraries and
applications. Originally created for telecom applications, its usage
has spread to other domains including e-commerce, banking, databases,
and computer telephony and messaging.

Erlang programs are today among the largest applications written in
any functional programming language. These applications offer new
opportunities to evaluate functional programming and functional
programming methods on a very large scale and suggest new problems for
the research community to solve.

This workshop will bring together the open source, academic, and
industrial programming communities of Erlang. It will enable
participants to familiarize themselves with recent developments on new
techniques and tools tailored to Erlang, novel applications, draw
lessons from users' experiences and identify research problems and
common areas relevant to the practice of Erlang and functional
programming.

We invite three types of submissions.

1. Technical papers describing language extensions, critical
discussions of the status quo, formal semantics of language
constructs, program analysis and transformation, virtual machine
extensions and compilation techniques, implementations and interfaces
of Erlang in/with other languages, and new tools (profilers, tracers,
debuggers, testing frameworks, etc.). The maximum length for technical
papers is restricted to 12 pages.

2. Practice and application papers describing uses of Erlang in the
"real-world", Erlang libraries for specific tasks, experiences from
using Erlang in specific application domains, reusable programming
idioms and elegant new ways of using Erlang to approach or solve a
particular problem. The maximum length for the practice and
application papers is restricted to 12 pages. Note that this is a
maximum length; we welcome shorter papers also, and the program
committee will evaluate all papers on an equal basis independent of
their lengths.

3. Poster presentations describing topics related to the workshop
goals. Each includes a maximum of 2 pages of the abstract and summary.
Presentations in this category will be given an hour of shared
simultaneous demonstration time.

Workshop Co-Chairs
------------------
Hans Svensson, QuviQ AB, Sweden
Melinda T?th, E?tv?s Lor?nd University, Hungary

Program Committee
-----------------------------
(Note: the Workshop Co-Chairs are also committee members)

Jesper L. Andersen, Independent
Clara Benac Earle, Technical University of Madrid, Spain
Laura M. Castro, University of  A Coru?a, Spain
Christopher Meiklejohn, Basho Technologies, Inc., US
Samuel Rivas, Klarna AB, Sweden
Tee Teoh, Erlang Solutions Ltd, UK
Simon Thompson, University of Kent, UK


Important Dates
-----------------------
Submissions due: Friday, 29 May, 2015 [extended]
Author notification: Friday, 26 June, 2015 [unchanged]
Final copy due: Sunday, 19 July, 2015 [unchanged]
Workshop date: September 4, 2015 [unchanged]

Instructions to authors
--------------------------------
Papers must be submitted online via EasyChair (via the "Erlang2015"
event). The submission page is
https://www.easychair.org/conferences/?conf=erlang2015

Submitted papers should be in portable document format (PDF),
formatted using the ACM SIGPLAN style guidelines.

Each submission must adhere to SIGPLAN's republication policy.
Violation risks summary rejection of the offending submission.
Accepted papers will be published by the ACM and will appear in the
ACM Digital Library.

The proceedings will be freely available for download from the ACM Digital
Library from one week before the start of the conference until two weeks
after the conference.

Paper submissions will be considered for poster submission in the case
they are not accepted as full papers.

Venue & Registration Details
------------------------------------------
For registration, please see the ICFP 2015 web site at:
http://icfpconference.org/icfp2015/

Related Links
--------------------
ICFP 2015 web site: http://www.icfpconference.org/icfp2015/
Past ACM SIGPLAN Erlang workshops: http://www.erlang.org/workshop/
Open Source Erlang: http://www.erlang.org/
EasyChair submission site:
https://www.easychair.org/conferences/?conf=erlang2015
Author Information for SIGPLAN Conferences:
http://www.sigplan.org/authorInformation.htm
Atendee Information for SIGPLAN Events:
http://www.sigplan.org/Resources/Policies/Anti-harassment


From jesper.louis.andersen@REDACTED  Wed May 20 14:28:52 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Wed, 20 May 2015 14:28:52 +0200
Subject: [erlang-questions] Trying to understand the performance impact
	of binary:split/3
In-Reply-To: 
References: 
 
 
Message-ID: 

On Wed, May 20, 2015 at 1:14 PM, Jos? Valim  wrote:

> Thoughts?


Matching:

The structure of the Needle and the Haystack matter:

1) In a search for Supercalifragilisticexpialidocious, search algorithms
such as Boyer-Moore wins. GNU Grep even unrolls the inner BM loop to make
it even faster.
2) If the needle is a $, separator in a CSV file, then naive string seach
can do surprisingly well.
3) If you are searching for DNA sequence matches, the low alphabet of GACT
means you have lots of partial matches. In that case the "standard"
algorithm is Knuth-Morris-Pratt.

Part'ing:

running part is O(1), so constant factors will play an ever increasing
role. Importantly in a call binary:part(Bin, Pos), the part/2 BIF can't
know a priori that Pos is valid for the binary. So it has to run a bounds
check, even though binary:matches/3 will always provide valid positions.
Chances are that a low-level naive approach written in a statically typed
language is able to eliminate those bounds checks because it knows stuff.

Fusion:

Erlang can't fuse the construction of the MatchList with the reduction on
the part call. So you build up a list of matches and then you consume that
list right after. Compilers with a fuse-pass can eliminate these
intermediate lists.


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jose.valim@REDACTED  Wed May 20 14:41:41 2015
From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=)
Date: Wed, 20 May 2015 14:41:41 +0200
Subject: [erlang-questions] Trying to understand the performance impact
	of binary:split/3
In-Reply-To: 
References: 
 
 
 
Message-ID: 

Thanks Jesper, breaking the algorithm into steps really helps evaluating
where we can optimize.

In this particular case, matching is fast (I have measured independently)
and the generated list contains always 4 elements. That leaves us with
part/3.

I wonder how fast it would be if binary:split/3 was written as a BIF.





*Jos? Valim*
www.plataformatec.com.br
Skype: jv.ptec
Founder and Lead Developer

On Wed, May 20, 2015 at 2:28 PM, Jesper Louis Andersen <
jesper.louis.andersen@REDACTED> wrote:

>
> On Wed, May 20, 2015 at 1:14 PM, Jos? Valim <
> jose.valim@REDACTED> wrote:
>
>> Thoughts?
>
>
> Matching:
>
> The structure of the Needle and the Haystack matter:
>
> 1) In a search for Supercalifragilisticexpialidocious, search algorithms
> such as Boyer-Moore wins. GNU Grep even unrolls the inner BM loop to make
> it even faster.
> 2) If the needle is a $, separator in a CSV file, then naive string seach
> can do surprisingly well.
> 3) If you are searching for DNA sequence matches, the low alphabet of GACT
> means you have lots of partial matches. In that case the "standard"
> algorithm is Knuth-Morris-Pratt.
>
> Part'ing:
>
> running part is O(1), so constant factors will play an ever increasing
> role. Importantly in a call binary:part(Bin, Pos), the part/2 BIF can't
> know a priori that Pos is valid for the binary. So it has to run a bounds
> check, even though binary:matches/3 will always provide valid positions.
> Chances are that a low-level naive approach written in a statically typed
> language is able to eliminate those bounds checks because it knows stuff.
>
> Fusion:
>
> Erlang can't fuse the construction of the MatchList with the reduction on
> the part call. So you build up a list of matches and then you consume that
> list right after. Compilers with a fuse-pass can eliminate these
> intermediate lists.
>
>
> --
> J.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From t@REDACTED  Wed May 20 17:07:45 2015
From: t@REDACTED (Tristan Sloughter)
Date: Wed, 20 May 2015 10:07:45 -0500
Subject: [erlang-questions] Windows
In-Reply-To: <555C46E0.4060000@ninenines.eu>
References: <555C46E0.4060000@ninenines.eu>
Message-ID: <1432134465.1930071.273847977.36D7136C@webmail.messagingengine.com>

Shouldn't C code be fine when built with rebar? Makefiles are an issue
only if the project doesn't also support rebar.

That said, with rebar3 we've recently been debating what to do about
this and the most annoying thing in my opinion is Travis-CI's lack of
Windows support. Not that I blame them, Microsoft has made their money
making something like that hard and is only now attempting to reverse
decades of screw ups -- Windows containers coming in 2016, which
hopefully don't require some crazy license to run for building open
source projects like Travis does.

If anyone has a suggestion for free Windows CI it would be much
appreciated. 

But Windows support will always have trouble as long as a developer
can't easily (and freely) spin up a test environment -- and varying test
environments, some locked down Windows free VM doesn't cut it. The only
reason this isn't the case for OSX is so many devs use it, so I can just
wait for them to find the bugs :)

-- 
  Tristan Sloughter
  t@REDACTED

On Wed, May 20, 2015, at 03:33 AM, Lo?c Hoguin wrote:
> Hi,
> 
> Yesterday the topic of Windows development was brought up on Twitter. I 
> looked into it and it's clearly an issue. At best you can compile 
> projects that don't involve shell scripts, makefiles or C code, so you 
> can't really work on anything significant as a large number of open 
> source projects involve one or the other (yes, even when using rebar).
> 
> Now there's three ways Erlang could be more friendly to Windows
> developers:
> 
> To use Windows tools imply making open source projects work for Windows. 
> Sure perhaps it's not an issue for the simplest NIFs but some projects 
> require C libraries which themselves aren't always easy to make work on 
> Windows.
> 
> MingW has the same issues. You need to fix everything to work on Windows.
> 
> Cygwin makes this much easier. Everything seems to work... except 
> Erlang. Erlang on Cygwin thinks it runs on win32 and, while true to a 
> certain extent, it fails badly when dealing with files (cygwin has / as 
> root while win32 has c:\).
> 
> Now you might say Cygwin is massive. That's true, it takes some time to 
> install and some space on the disk. But it's very easy to install and 
> can be almost entirely automated using chocolatey and apt-cyg. And 
> adding an SSH server on top of that is a couple commands away, so it is 
> a very interesting solution if you ask me.
> 
> Now Erlang compiles on Cygwin... but produces the win32 target. Which 
> brings me to the main subject of this email: what would it involve to 
> make Erlang work on Cygwin? Considering Cygwin is a mix of Windows and 
> Linux it's probably just about enabling one or another piece of code, or 
> is it?
> 
> It could be a fine side project but I don't really know where to start. 
> Any tips appreciated.
> 
> PS: tried just patching os:type()... that didn't work. Ended up with 
> Erlang not finding a file that existed on disk when running common_test.
> 
> PPS: I know "real" developers use Unix. But the more numerous "fake" 
> developers might be more interested in Erlang if they can keep their OS.
> 
> PPPS: I know this isn't a good option if you're going to create 
> Photoshoperl, but it's still a good option for server application 
> development.
> 
> -- 
> Lo?c Hoguin
> http://ninenines.eu
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions


From bchesneau@REDACTED  Wed May 20 17:10:55 2015
From: bchesneau@REDACTED (Benoit Chesneau)
Date: Wed, 20 May 2015 15:10:55 +0000
Subject: [erlang-questions] Windows
In-Reply-To: <1432134465.1930071.273847977.36D7136C@webmail.messagingengine.com>
References: <555C46E0.4060000@ninenines.eu>
 <1432134465.1930071.273847977.36D7136C@webmail.messagingengine.com>
Message-ID: 

On Wed, May 20, 2015 at 5:07 PM Tristan Sloughter  wrote:

>
> If anyone has a suggestion for free Windows CI it would be much
> appreciated.
>

appveyor should do the trick, it's free for opensources projects
http://www.appveyor.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From bchesneau@REDACTED  Wed May 20 17:12:15 2015
From: bchesneau@REDACTED (Benoit Chesneau)
Date: Wed, 20 May 2015 15:12:15 +0000
Subject: [erlang-questions] Windows
In-Reply-To: <1432134465.1930071.273847977.36D7136C@webmail.messagingengine.com>
References: <555C46E0.4060000@ninenines.eu>
 <1432134465.1930071.273847977.36D7136C@webmail.messagingengine.com>
Message-ID: 

On Wed, May 20, 2015 at 5:07 PM Tristan Sloughter  wrote:

>
> If anyone has a suggestion for free Windows CI it would be much
> appreciated.
>

Last mail went wrong. Appveyor should do the trick it's free for
opensources projects:

http://www.appveyor.com/

Benoit

>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From essen@REDACTED  Wed May 20 18:20:01 2015
From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=)
Date: Wed, 20 May 2015 19:20:01 +0300
Subject: [erlang-questions] Windows
In-Reply-To: <1432134465.1930071.273847977.36D7136C@webmail.messagingengine.com>
References: <555C46E0.4060000@ninenines.eu>
 <1432134465.1930071.273847977.36D7136C@webmail.messagingengine.com>
Message-ID: <555CB431.4060601@ninenines.eu>

Like I said before, try eleveldb, or any other project that depends on 
another lib that's not so friendly with Windows. The Twitter discussion 
I was referring to also mentioned Yaws, though that's probably not a 
permanent issue. I could find you more, there's no shortage of NIFs and 
ports in the open source projects.

To make things work in a proper Windows environment you have to rely on 
every project to be targetting Windows and that's no small matter, 
especially for those that use a library that's hard to use on Windows.

At least with Cygwin there's no added burden on open source project 
developers. And that's very important, considering how little most 
projects are willing to deal with.

Making everything work in a proper Windows environment is a *gigantic* 
task and involves a lot more than just dealing with Erlang projects 
themselves. I don't see that happening anytime soon.

Rebar is fine for small things but can't deal with everything. Rebar3 is 
about the same but requires the project to also target Rebar3 (unless 
that changed). erlang.mk will eventually get at the same level as rebar 
once it switches to Make 4. But the experience is not smooth with any of 
them.

Cygwin can deal with a lot more and doesn't require effort on the part 
of the open source project developer or on the part of the user 
(everyone can run one setup command in a cmd shell). That's a win win to me.

Adding a Cygwin target to Erlang should not be too hard. After all it's 
already possible to compile Erlang/win32 on Cygwin. I imagine the Cygwin 
build would just require different defines enabled, little else.

On 05/20/2015 06:07 PM, Tristan Sloughter wrote:
> Shouldn't C code be fine when built with rebar? Makefiles are an issue
> only if the project doesn't also support rebar.
>
> That said, with rebar3 we've recently been debating what to do about
> this and the most annoying thing in my opinion is Travis-CI's lack of
> Windows support. Not that I blame them, Microsoft has made their money
> making something like that hard and is only now attempting to reverse
> decades of screw ups -- Windows containers coming in 2016, which
> hopefully don't require some crazy license to run for building open
> source projects like Travis does.
>
> If anyone has a suggestion for free Windows CI it would be much
> appreciated.
>
> But Windows support will always have trouble as long as a developer
> can't easily (and freely) spin up a test environment -- and varying test
> environments, some locked down Windows free VM doesn't cut it. The only
> reason this isn't the case for OSX is so many devs use it, so I can just
> wait for them to find the bugs :)
>

-- 
Lo?c Hoguin
http://ninenines.eu


From sdl.web@REDACTED  Wed May 20 18:34:52 2015
From: sdl.web@REDACTED (Leo Liu)
Date: Thu, 21 May 2015 00:34:52 +0800
Subject: [erlang-questions] how search and search_quit are handled by
	user_drv?
Message-ID: 

Hi there,

Press key C-r and then TAB in the eshell one can observe the prompt
changes to history `search' and back to normal again.

I try to understand how it works in details but my understanding does
not match what I observe above.

The following is what I think should have happened where `|' indicates
the cursor and indented lines are messages the user_drv process receives
which cause the prompt to change from state 1 to 5.

--------------------------------

1.
(devel@REDACTED)3> |

    {trace,<0.28.0>,'receive',{#Port<0.349>,{data,[18]}}}
    {trace,<0.28.0>,'receive', {<0.30.0>,{requests,[{delete_chars,0},{delete_chars,-20}]}}}
    {trace,<0.28.0>,'receive', {<0.30.0>,{requests,[{put_chars,unicode,"(search)`': "}]}}}
    {trace,<0.28.0>,'receive',{<0.30.0>,beep}}

2.
(search)`': |

    {trace,<0.28.0>,'receive',{#Port<0.349>,{data,"\t"}}}
    {trace,<0.28.0>,'receive', {<0.30.0>,{requests,[{move_rel,-3},{put_chars,unicode,"\n"}]}}}

3.
(search)`
|': 

    {trace,<0.28.0>,'receive', {<0.30.0>,{requests,[{delete_chars,0},{delete_chars,-12}]}}}

4.
(search)`
|': 

    {trace,<0.28.0>,'receive', {<0.30.0>,
                                {requests,[{put_chars,unicode,"(devel@REDACTED)3> "},
                                {put_chars,unicode,[]},
                                {move_rel,0}]}}}
5.
(search)`
(devel@REDACTED)3> |

--------------------------------

State 1 and 2 are as observed. State 3 is NOT though it seems to agree
with {requests,[{move_rel,-3},{put_chars,unicode,"\n"}]}.

Could someone explain the discrepancy? Many thanks.

Leo



From ddosia@REDACTED  Wed May 20 19:30:28 2015
From: ddosia@REDACTED (Daniil Churikov)
Date: Wed, 20 May 2015 18:30:28 +0100
Subject: [erlang-questions] HEART
In-Reply-To: 
References: 
 
 
 
 <1431449900.2713951.266897849.0D1D4324@webmail.messagingengine.com>
 <9B082D3C-0E45-4D21-B914-4AD4CECE03C3@widetag.com>
 <1431452073.2721867.266937257.35C09C47@webmail.messagingengine.com>
 
 
 
 <1431525293.3002069.267735585.0D9F5F7F@webmail.messagingengine.com>
 
Message-ID: 

Sorry Roberto, forgot to cc the list.

If you will use heart be aware of OOM. We had some issues with it,
in essence OOM kills both heart and VM.

2015-05-13 15:08 GMT+01:00 Roberto Ostinelli :

> Ok,
> After a while of experimenting I want to provide some feedback to the
> great help that was provided here.
> I've come up with a init.d script (currently running on Ubuntu) that
> provides me with what I needed.
>
> As a reminder:
>
>    - I have an Erlang 17.4 release generated with rebar.
>    - I want to have the release started on system boot.
>    - I want to the VM monitored and restarted it if it crashes.
>
>
> First, ensure that the `-heart` option is used in your `vm.args` file.
> Heart will monitor the VM and restart it if needed.
>
> Second, create the file `/etc/init.d/myapp`:
>
>
> ########################################################################
>
> #!/usr/bin/env bash
> # myapp daemon
> # chkconfig: 345 20 80
> # description: myapp daemon
> # processname: myapp
>
> NAME=myapp
> PROJECT_ROOT_PATH=/usr/local/$NAME
> APP_SCRIPT="bin/$NAME"
>
> # export
> export HOME=/root
>
> case "$1" in
> start)
>     printf "%-50s" "Starting $NAME..."
>
>     # start
>     cd $PROJECT_ROOT_PATH
>     $APP_SCRIPT start > /dev/null 2>&1;
>
>     # wait for pid
>     for (( i=0; i<10; ++i )); do
>         OUT=`$APP_SCRIPT getpid`;
>         if [ $? == 0 ]; then PID=$OUT; break; fi
>         sleep 1;
>     done
>
>     if [ -z "$PID" ]; then
>         printf "%s\n" "Failsd"
>     else
>         printf "%s\n" "Ok"
>     fi
> ;;
> status)
>     printf "%-50s" "Checking $NAME..."
>
>     # wait for pid
>     cd $PROJECT_ROOT_PATH
>     $APP_SCRIPT getpid > /dev/null 2>&1;
>
>     if [ $? != 0 ]; then
>         printf "%s\n" "Node is not running!"
>     else
>         printf "%s\n" "Ok"
>     fi
> ;;
> stop)
>     printf "%-50s" "Stopping $NAME..."
>
>     # cd and stop
>     cd $PROJECT_ROOT_PATH
>     $APP_SCRIPT stop > /dev/null 2>&1;
>
>     if [ $? != 0 ]; then
>         printf "%s\n" "Node is not running!"
>     else
>         printf "%s\n" "Ok"
>     fi
> ;;
>
> restart)
>     $0 stop
>     $0 start
> ;;
>
> *)
>     echo "Usage: $0 {status|start|stop|restart}"
>     exit 1
> esac
>
>
> ########################################################################
>
> You can use this file as normal services:
>
> ```
> $ sudo service cometa start
> Starting myapp...                                Ok
> ```
>
>
> Third, ensure this script is used at boot time:
>
> `sudo update-rc.d myapp defaults`
>
>
>
> Side note: you can see that the script waits to exit the start function
> until the PID is retrieved from the VM.
> This is not strictly necessary, although in this way you can even consider
> dumping it into PID files or perform other types of monitoring actions
> instead of using HEART.
>
>
> Hope this helps someone in my same spot.
>
> Best,
> r.
>
>
>
>
> On Wed, May 13, 2015 at 3:54 PM, Tristan Sloughter 
> wrote:
>
>>  Use remote_console instead of attach.
>>
>> --
>> Tristan Sloughter
>> t@REDACTED
>>
>>
>>
>> On Wed, May 13, 2015, at 06:41 AM, Roberto Ostinelli wrote:
>>
>> Hi,
>> Still experiencing weirdnesses though.
>>
>> My upstart script is:
>>
>> script
>> export HOME=/root
>> cd /usr/local/myapp
>>     exec bin/myapp foreground > /dev/null 2>&1
>> end script
>>
>>
>> When I start to attach to the app's node, I get:
>>
>> $ /usr/local/myapp/bin/myapp attach
>> pong
>> Can't access pipe directory /tmp//usr/local/myapp/: No such file or
>> directory
>>
>>
>> However, if I start my app manually:
>>
>> $ /usr/local/myapp/bin/myapp start
>>
>> Then everything works fine:
>>
>> $ /usr/local/cometa/bin/cometa attach
>> pong
>> Attaching to /tmp//usr/local/myapp/erlang.pipe.1 (^D to exit)
>>
>> (myapp@REDACTED)1>
>>
>>
>> Can some kind soul explain to me what is going on?
>>
>> Thank you,
>> r.
>>
>>
>>
>>
>>
>>
>>
>> On Tue, May 12, 2015 at 8:44 PM, Roger Lipscombe 
>> wrote:
>>
>> On 12 May 2015 at 18:45, Roberto Ostinelli
>>  wrote:
>> > Right. Unfortunately I can't find a way to oass this pid to the
>> original script that starts it (using upstart).
>>
>> We use relx-generated releases with upstart. Simply run "bin/myapp
>> foreground" from the upstart script.
>>
>>
>>
>>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From t@REDACTED  Wed May 20 20:11:34 2015
From: t@REDACTED (Tristan Sloughter)
Date: Wed, 20 May 2015 13:11:34 -0500
Subject: [erlang-questions] Windows
In-Reply-To: 
References: <555C46E0.4060000@ninenines.eu>
 <1432134465.1930071.273847977.36D7136C@webmail.messagingengine.com>
 
Message-ID: <1432145494.1973460.274037513.00585572@webmail.messagingengine.com>

Thanks! appveyor was surprisingly easy to start building Erlang projects
with. But wow, Windows issues seem to run deep. It builds rebar3 but
running tests fails right away:

c:/projects/rebar3/test/rebar_xref_SUITE.erl:1: no module definition

Hopefully I can get someone with Windows to help figure out what
is going on.

--
Tristan Sloughter t@REDACTED



On Wed, May 20, 2015, at 10:12 AM, Benoit Chesneau wrote:
>
>
> On Wed, May 20, 2015 at 5:07 PM Tristan Sloughter
>  wrote:
>>
>>
If anyone has a suggestion for free Windows CI it would be much
>>
appreciated.
>
> Last mail went wrong. Appveyor should do the trick it's free for
> opensources projects:
>
> http://www.appveyor.com/
>
> Benoit
>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From boozelclark@REDACTED  Wed May 20 21:02:28 2015
From: boozelclark@REDACTED (Chris Clark)
Date: Wed, 20 May 2015 19:02:28 +0000
Subject: [erlang-questions] Distributed Erlang Architecture
Message-ID: 

Hi

I have an erlang application that is made up of 3 applications and their
dependencies. I am trying to decide on how to distribute these so that I
can scale them by running additional instance of which ever app start to
reach its performance limits.

I've managed to find lot of info of how to connect erlang nodes into a
cluster but nothing on how to structure a distributed application in
practice. Does anyone know where I can find some info on this?

If for example I have three hosts in my cluster. Should I
A:
Compile these all into one release and the not start the apps automatically
but just start the number of required instances within the single erlang
nodes each running on a host

B:
Create each app into a separate release and then just start a node for each
instance of each app I want resulting in multiple erlang nodes per host. If
for example I just wanted one host then I would run three nodes on that
host. One for each app.

Any guidance or info on the pros and cons of each approach would be
appreciated.

Thanks,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From g@REDACTED  Thu May 21 01:15:04 2015
From: g@REDACTED (Garrett Smith)
Date: Wed, 20 May 2015 18:15:04 -0500
Subject: [erlang-questions] Distributed Erlang Architecture
In-Reply-To: 
References: 
Message-ID: 

Hi Chris,

On Wed, May 20, 2015 at 2:02 PM, Chris Clark  wrote:
> Hi
>
> I have an erlang application that is made up of 3 applications and their
> dependencies. I am trying to decide on how to distribute these so that I can
> scale them by running additional instance of which ever app start to reach
> its performance limits.
>
> I've managed to find lot of info of how to connect erlang nodes into a
> cluster but nothing on how to structure a distributed application in
> practice. Does anyone know where I can find some info on this?
>
> If for example I have three hosts in my cluster. Should I
> A:
> Compile these all into one release and the not start the apps automatically
> but just start the number of required instances within the single erlang
> nodes each running on a host

A release corresponds to a node and what's in that release should be
driven by the behavior of the node - there may be some linkage with so
called scalability problems in specific cases, but certainly not in
the general case.

As for not starting "all" - this sounds like a misunderstanding - on
your part or my part. Maybe I'm misreading you but it sounds like
you're viewing your "apps" as units of power - like octane - that you
might want to hold in reserve for the time you need to go super fast.

An app is just a single supervisory tree that provides some set of
functionality within your system (i.e. a running release).

If an "app" in this case can "scale" - then you ought to know how. Is
the constraint CPU, disk, IO - or a host of complex interactions that
you'd never guess in a million years? If and when you understand how
it can scale, look to deploy multiple instances of it per unit of
actual power (CPU, spindle, network interface, etc) - if that would
even work. Chances are you'll have to rethink something fundamental in
the way you're doing things. That's okay though - it's the sign of a
system evolving under pressure.

Erlang is outstanding for this.

> B:
> Create each app into a separate release and then just start a node for each
> instance of each app I want resulting in multiple erlang nodes per host. If
> for example I just wanted one host then I would run three nodes on that
> host. One for each app.

Releases are just ways to package apps and run them in concert in a
VM. If you want one app running on a VM, then do this. Otherwise don't
do this.

> Any guidance or info on the pros and cons of each approach would be
> appreciated.

Knowing nothing about your application, I'd just start by creating one
release with a complete working system (user facing functionality).
Then see how it behaves at various levels of load (request frequency,
concurrency, volumes, etc.)

Personally, I wouldn't think about scalability at all - and not just
to make a social statement. I'd put the system into production and see
what happens. You'll need to monitor things, but start by monitoring
things from the end user perspective - this will force you to define
meaningful performance targets. When you start to see something bad
happening, study it carefully and take steps to fix the problem.

When you're at the point you have actual problems (rather than
architectural, which is not an actual thing, much less a problem) you
can post some detailed information here and get some actual help.

Hope that helps ;)

Garrett


From daniel.widgren@REDACTED  Thu May 21 08:10:11 2015
From: daniel.widgren@REDACTED (Daniel Hallin Widgren)
Date: Thu, 21 May 2015 08:10:11 +0200
Subject: [erlang-questions] Windows
In-Reply-To: <1432145494.1973460.274037513.00585572@webmail.messagingengine.com>
References: <555C46E0.4060000@ninenines.eu>
 <1432134465.1930071.273847977.36D7136C@webmail.messagingengine.com>
 
 <1432145494.1973460.274037513.00585572@webmail.messagingengine.com>
Message-ID: 

Tristan i can try to look at it tonight after work.

Any specific windows or just any to get it to work?
Den 20 maj 2015 20:11 skrev "Tristan Sloughter" :

>  Thanks! appveyor was surprisingly easy to start building Erlang projects
> with. But wow, Windows issues seem to run deep. It builds rebar3 but
> running tests fails right away:
>
> c:/projects/rebar3/test/rebar_xref_SUITE.erl:1: no module definition
>
> Hopefully I can get someone with Windows to help figure out what is going
> on.
>
> --
> Tristan Sloughter
> t@REDACTED
>
>
>
> On Wed, May 20, 2015, at 10:12 AM, Benoit Chesneau wrote:
>
>
>
> On Wed, May 20, 2015 at 5:07 PM Tristan Sloughter  wrote:
>
>
> If anyone has a suggestion for free Windows CI it would be much
> appreciated.
>
>
> Last mail went wrong. Appveyor should do the trick it's free for
> opensources projects:
>
> http://www.appveyor.com/
>
> Benoit
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From mononcqc@REDACTED  Thu May 21 13:46:21 2015
From: mononcqc@REDACTED (Fred Hebert)
Date: Thu, 21 May 2015 07:46:21 -0400
Subject: [erlang-questions] Windows
In-Reply-To: 
References: <555C46E0.4060000@ninenines.eu>
 <1432134465.1930071.273847977.36D7136C@webmail.messagingengine.com>
 
 <1432145494.1973460.274037513.00585572@webmail.messagingengine.com>
 
Message-ID: <20150521114620.GD2572@ferdmbp.local>

On 05/21, Daniel Hallin Widgren wrote:
>Tristan i can try to look at it tonight after work.
>
>Any specific windows or just any to get it to work?

Neither of us is a heavy windows user, but I'm guessing that if 
something like Windows 7 gets to work, then the latter versions should 
be fine too?

But any Windows to get it to work would be nice at first as it would 
probably filter out a lot of the weirdness we have that makes it not 
work at all right now.


From pablo.platt@REDACTED  Thu May 21 15:04:36 2015
From: pablo.platt@REDACTED (pablo platt)
Date: Thu, 21 May 2015 16:04:36 +0300
Subject: [erlang-questions] SO_REUSEPORT with UDP
Message-ID: 

Hi,

How can I have multiple workers listen on the same UDP port on Linux?
SO_REUSEADDR or SO_REUSEPORT are suggested in several places but I couldn't
find how to use it.

What's the difference between SO_REUSEADDR and SO_REUSEPORT?

Do I just listen on the same IP/Port with gen_udp several times with the
option {reuseaddr, true}?
Is there an equivalent reuseport option?

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roger@REDACTED  Thu May 21 16:11:04 2015
From: roger@REDACTED (Roger Lipscombe)
Date: Thu, 21 May 2015 15:11:04 +0100
Subject: [erlang-questions] Why have a supervisor behaviour?
Message-ID: 

I find myself writing a custom supervisor (because I need restarts to
be delayed[1]), and I find myself wondering why OTP has a supervisor
behaviour?

That is: why does it require us to provide the Module:init/1 function?
Surely we could just pass the restart strategy, child specs, etc. to
supervisor:start_link directly?

Is there something I'm missing, that I'm going to regret if I don't do
it the same way in my custom supervisor?

Regards,
Roger.

[1] http://erlang.org/pipermail/erlang-patches/2012-January/002575.html
discusses a supervisor with delayed child restart (and has some code),
but (a) it seems to have died a death, and (b) it's not exactly what I
need.


From radek@REDACTED  Thu May 21 16:15:36 2015
From: radek@REDACTED (Rad Gruchalski)
Date: Thu, 21 May 2015 16:15:36 +0200
Subject: [erlang-questions] SO_REUSEPORT with UDP
In-Reply-To: 
References: 
Message-ID: <08BB3F0CBB7C470AB27D84A2AEDCA51C@gruchalski.com>

Hi Pablo,  

I found a perfect explanation of the difference between SO_REUSEADDR and SO_REUSEPORT:
http://stackoverflow.com/a/14388707/56250
It goes a long way into explaining the implementation on different platforms and the differences between these two.
I might be wrong but I would imagine that reuseaddr means SO_REUSEADDR and I do not think Erlang does SO_REUSEPORT in any way. If I?m wrong, please, somebody correct me.










Kind regards,

Radek Gruchalski
de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)

Confidentiality:
This communication is intended for the above-named person and may be confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.



On Thursday, 21 May 2015 at 15:04, pablo platt wrote:

> Hi,
>  
> How can I have multiple workers listen on the same UDP port on Linux?
> SO_REUSEADDR or SO_REUSEPORT are suggested in several places but I couldn't find how to use it.
>  
> What's the difference between SO_REUSEADDR and SO_REUSEPORT?
>  
> Do I just listen on the same IP/Port with gen_udp several times with the option {reuseaddr, true}?
> Is there an equivalent reuseport option?
>  
> Thanks
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED (mailto:erlang-questions@REDACTED)
> http://erlang.org/mailman/listinfo/erlang-questions
>  
>  


-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From bchesneau@REDACTED  Thu May 21 16:18:54 2015
From: bchesneau@REDACTED (Benoit Chesneau)
Date: Thu, 21 May 2015 14:18:54 +0000
Subject: [erlang-questions] SO_REUSEPORT with UDP
In-Reply-To: 
References: 
Message-ID: 

On Thu, May 21, 2015 at 3:04 PM pablo platt  wrote:

> Hi,
>
> How can I have multiple workers listen on the same UDP port on Linux?
> SO_REUSEADDR or SO_REUSEPORT are suggested in several places but I
> couldn't find how to use it.
>
>
In Erlang you can use the raw option to use REUSE_PORT on your platform:

https://github.com/refuge/rbeacon/blob/master/src/rbeacon.erl#L414-L425

and:

https://github.com/refuge/rbeacon/blob/master/src/rbeacon.erl#L400-L412

- benoit


> What's the difference between SO_REUSEADDR and SO_REUSEPORT?
>
> Do I just listen on the same IP/Port with gen_udp several times with the
> option {reuseaddr, true}?
> Is there an equivalent reuseport option?
>
> Thanks
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From radek@REDACTED  Thu May 21 16:20:10 2015
From: radek@REDACTED (Rad Gruchalski)
Date: Thu, 21 May 2015 16:20:10 +0200
Subject: [erlang-questions] SO_REUSEPORT with UDP
In-Reply-To: 
References: 
 
Message-ID: <9BF17FBFE6E14E96AA96622C1C822C25@gruchalski.com>

Perfect.










Kind regards,

Radek Gruchalski
de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)

Confidentiality:
This communication is intended for the above-named person and may be confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.



On Thursday, 21 May 2015 at 16:18, Benoit Chesneau wrote:

>  
>  
> On Thu, May 21, 2015 at 3:04 PM pablo platt  wrote:
> > Hi,
> >  
> > How can I have multiple workers listen on the same UDP port on Linux?
> > SO_REUSEADDR or SO_REUSEPORT are suggested in several places but I couldn't find how to use it.
> >  
>  
> In Erlang you can use the raw option to use REUSE_PORT on your platform:
>  
> https://github.com/refuge/rbeacon/blob/master/src/rbeacon.erl#L414-L425
>  
> and:
>  
> https://github.com/refuge/rbeacon/blob/master/src/rbeacon.erl#L400-L412
>  
> - benoit
>   
> > What's the difference between SO_REUSEADDR and SO_REUSEPORT?
> >  
> > Do I just listen on the same IP/Port with gen_udp several times with the option {reuseaddr, true}?
> > Is there an equivalent reuseport option?
> >  
> > Thanks
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED)
> > http://erlang.org/mailman/listinfo/erlang-questions
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED (mailto:erlang-questions@REDACTED)
> http://erlang.org/mailman/listinfo/erlang-questions
>  
>  


-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jesper.louis.andersen@REDACTED  Thu May 21 16:23:53 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Thu, 21 May 2015 16:23:53 +0200
Subject: [erlang-questions] Why have a supervisor behaviour?
In-Reply-To: 
References: 
Message-ID: 

On Thu, May 21, 2015 at 4:11 PM, Roger Lipscombe 
wrote:

> I find myself writing a custom supervisor (because I need restarts to
> be delayed[1]), and I find myself wondering why OTP has a supervisor
> behaviour?
>

Another way around this is to let the supervisor just start the server and
then let the server itself handle the delay by querying a delay_manager, or
something such.


> That is: why does it require us to provide the Module:init/1 function?
> Surely we could just pass the restart strategy, child specs, etc. to
> supervisor:start_link directly?
>

Because init/1 runs in the context of the supervisor process, not the
invoker of start_link/1. If you create an ETS table in the supervisor, for
instance, its protection is relative to who created it. And so is its
lifetime. You can't easily do that if you pass in the data in start_link/1,
since you would have to pass a fun anyway.



-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From boozelclark@REDACTED  Thu May 21 16:48:25 2015
From: boozelclark@REDACTED (Chris Clark)
Date: Thu, 21 May 2015 14:48:25 +0000
Subject: [erlang-questions] Distributed Erlang Architecture
In-Reply-To: 
References: 
 
Message-ID: 

Hi Garrett

Thanks very much for the information that does make a lot of sense.

I am trying to wrap my head around how large scale Erlang systems are
structured in practice but I see that it will depend on the application and
is only something worth thinking about once/if in becomes a real problem.

What initially had me thinking about it was availability rather than scale.
A lot of what I have read and seen in videos suggests avoiding hot code
loading where possible and rather having multiple independent app instances
that can just be swapped, one at a time with new instances (provided the
app allows it).

Thanks for your assistance

Regards,

On Thu, May 21, 2015 at 1:15 AM Garrett Smith  wrote:

> Hi Chris,
>
> On Wed, May 20, 2015 at 2:02 PM, Chris Clark 
> wrote:
> > Hi
> >
> > I have an erlang application that is made up of 3 applications and their
> > dependencies. I am trying to decide on how to distribute these so that I
> can
> > scale them by running additional instance of which ever app start to
> reach
> > its performance limits.
> >
> > I've managed to find lot of info of how to connect erlang nodes into a
> > cluster but nothing on how to structure a distributed application in
> > practice. Does anyone know where I can find some info on this?
> >
> > If for example I have three hosts in my cluster. Should I
> > A:
> > Compile these all into one release and the not start the apps
> automatically
> > but just start the number of required instances within the single erlang
> > nodes each running on a host
>
> A release corresponds to a node and what's in that release should be
> driven by the behavior of the node - there may be some linkage with so
> called scalability problems in specific cases, but certainly not in
> the general case.
>
> As for not starting "all" - this sounds like a misunderstanding - on
> your part or my part. Maybe I'm misreading you but it sounds like
> you're viewing your "apps" as units of power - like octane - that you
> might want to hold in reserve for the time you need to go super fast.
>
> An app is just a single supervisory tree that provides some set of
> functionality within your system (i.e. a running release).
>
> If an "app" in this case can "scale" - then you ought to know how. Is
> the constraint CPU, disk, IO - or a host of complex interactions that
> you'd never guess in a million years? If and when you understand how
> it can scale, look to deploy multiple instances of it per unit of
> actual power (CPU, spindle, network interface, etc) - if that would
> even work. Chances are you'll have to rethink something fundamental in
> the way you're doing things. That's okay though - it's the sign of a
> system evolving under pressure.
>
> Erlang is outstanding for this.
>
> > B:
> > Create each app into a separate release and then just start a node for
> each
> > instance of each app I want resulting in multiple erlang nodes per host.
> If
> > for example I just wanted one host then I would run three nodes on that
> > host. One for each app.
>
> Releases are just ways to package apps and run them in concert in a
> VM. If you want one app running on a VM, then do this. Otherwise don't
> do this.
>
> > Any guidance or info on the pros and cons of each approach would be
> > appreciated.
>
> Knowing nothing about your application, I'd just start by creating one
> release with a complete working system (user facing functionality).
> Then see how it behaves at various levels of load (request frequency,
> concurrency, volumes, etc.)
>
> Personally, I wouldn't think about scalability at all - and not just
> to make a social statement. I'd put the system into production and see
> what happens. You'll need to monitor things, but start by monitoring
> things from the end user perspective - this will force you to define
> meaningful performance targets. When you start to see something bad
> happening, study it carefully and take steps to fix the problem.
>
> When you're at the point you have actual problems (rather than
> architectural, which is not an actual thing, much less a problem) you
> can post some detailed information here and get some actual help.
>
> Hope that helps ;)
>
> Garrett
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roger@REDACTED  Thu May 21 16:58:06 2015
From: roger@REDACTED (Roger Lipscombe)
Date: Thu, 21 May 2015 15:58:06 +0100
Subject: [erlang-questions] Why have a supervisor behaviour?
In-Reply-To: 
References: 
 
Message-ID: 

On 21 May 2015 at 15:23, Jesper Louis Andersen
 wrote:
> Another way around this is to let the supervisor just start the server and
> then let the server itself handle the delay by querying a delay_manager, or
> something such.

I considered that, but I have a small wrinkle: I need find_or_start
semantics, such that if the child is already started, it's returned;
if the child doesn't exist, it's started and returned (so far this is
just one_for_one, ish); but -- if it's in the sin bin -- it needs to
be restarted immediately. The caller _needs_ a valid pid. Having a
delay_manager might complicate that?

>> That is: why does it require us to provide the Module:init/1 function?
>> Surely we could just pass the restart strategy, child specs, etc. to
>> supervisor:start_link directly?
>
> Because init/1 runs in the context of the supervisor process, not the
> invoker of start_link/1. If you create an ETS table in the supervisor, for
> instance, its protection is relative to who created it. And so is its
> lifetime. You can't easily do that if you pass in the data in start_link/1,
> since you would have to pass a fun anyway.

But the supervisor's not supposed to _do_ anything, right? It only has
Mod:init. If you want an ETS table, you should have it owned by the
supervisor's first child, right?

Cheers,
Roger.


From roger@REDACTED  Thu May 21 17:00:04 2015
From: roger@REDACTED (Roger Lipscombe)
Date: Thu, 21 May 2015 16:00:04 +0100
Subject: [erlang-questions] Why have a supervisor behaviour?
In-Reply-To: 
References: 
 
Message-ID: 

On 21 May 2015 at 15:56, Vance Shipley  wrote:
> Return {ok, State, Timeout) from your gen_server:init/1 callback. Continue
> initialization after the timeout in your handle_info/2 handler.

I need delayed _restart_. Is this what Jesper refers to when he talks
about "a delay_manager"? Such that init queries that and then
might/might not delay?


From g@REDACTED  Thu May 21 17:11:07 2015
From: g@REDACTED (Garrett Smith)
Date: Thu, 21 May 2015 10:11:07 -0500
Subject: [erlang-questions] Distributed Erlang Architecture
In-Reply-To: 
References: 
 
 
Message-ID: 

On Thu, May 21, 2015 at 9:48 AM, Chris Clark  wrote:
> Hi Garrett
>
> Thanks very much for the information that does make a lot of sense.
>
> I am trying to wrap my head around how large scale Erlang systems are
> structured in practice but I see that it will depend on the application and
> is only something worth thinking about once/if in becomes a real problem.
>
> What initially had me thinking about it was availability rather than scale.
> A lot of what I have read and seen in videos suggests avoiding hot code
> loading where possible and rather having multiple independent app instances
> that can just be swapped, one at a time with new instances (provided the app
> allows it).

On that point, if you can rid your application of state, it's a
trivial problem - you can have multiple services running that are
behind a router. The router becomes your point of failure, but that
role is presumably much simpler than your application and less likely
to be down, in theory.

Your best bet is to build failover logic into clients directly and let
them handle outages and re-routing.

If you have state to manage you have a harder problem - I can't say
anything specific other than it's hard.

To start though, I'd build your app without concern for availability
and get it doing something useful. To handle outages, focus on a fast
restore strategy to get your server back and running as quickly as
possible. If you focus on that problem you should be able to get a
server restored in under a minute. For many many many applications, a
minute of downtime in the event of a server failure is Outstanding.

The advantage of this approach is that it's generally acceptable and
avoids very complicated routing/replication/failover strategies needed
in faster recovery scenarios.

If you absolutely can't tolerate that sort of outage you'll need to go
down the hard road of building a more complex system. But you'll still
need to understand the specifics of your app and what it needs to run,
so I'd get it working first, then deal with advanced recovery
scenarios.

All that punting said, OTP has some recovery features that ostensibly
can be used for near-real-time service recovery, but I don't have any
experience with them. Others can weigh in.

Garrett


From roger@REDACTED  Thu May 21 17:49:05 2015
From: roger@REDACTED (Roger Lipscombe)
Date: Thu, 21 May 2015 16:49:05 +0100
Subject: [erlang-questions] Why have a supervisor behaviour?
In-Reply-To: 
References: 
 
 
Message-ID: 

On 21 May 2015 at 15:58, Roger Lipscombe  wrote:
> On 21 May 2015 at 15:23, Jesper Louis Andersen
>  wrote:
>> Because init/1 runs in the context of the supervisor process, not the
>> invoker of start_link/1. If you create an ETS table in the supervisor, for
>> instance, its protection is relative to who created it. And so is its
>> lifetime.
>
> But the supervisor's not supposed to _do_ anything, right? It only has
> Mod:init. If you want an ETS table, you should have it owned by the
> supervisor's first child, right?

OK. On thinking about this more, it makes sense: if you've got an ETS
table that every child needs to access, having it owned by the
supervisor might make sense in some scenarios. Gotcha.


From jesper.louis.andersen@REDACTED  Thu May 21 17:58:39 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Thu, 21 May 2015 17:58:39 +0200
Subject: [erlang-questions] Why have a supervisor behaviour?
In-Reply-To: 
References: 
 
 
Message-ID: 

On Thu, May 21, 2015 at 5:00 PM, Roger Lipscombe 
wrote:

> I need delayed _restart_. Is this what Jesper refers to when he talks
> about "a delay_manager"? Such that init queries that and then
> might/might not delay?
>

%% The delay manager decouples delay policy from a worker by tracking
delays in one place.
%% As such, it has global knowledge and can opt to delay registered
processes more or less
%% depending on current load.
-module(delay_mgr).
-behaviour(gen_server).

[..]

%% Call this from a newly started worker, but not in it's init/1 callback
since that blocks the supervisor
%% Send the process itself a message in init/1 and do it in that state.
delay(Reg) ->
    gen_server:call(?MODULE, {delay, Reg}, infinity).

[..]

handle_call({delay, Reg}, From, #state { conf = Conf }) ->
    Delay = maps:get(Reg, Conf),
    erlang:send_after(Delay, self(), {go, From}),
    {noreply, State};

handle_info({go, Reg}, State) ->
    gen_server:reply(From, ok),
    {noreply, State};

[..]

This is static skeleton, but:

* Add monitoring of delayed processes. Increase the delay for processes
that respawn too often
* Decay delays for processes which operates as they should
* Add metrics and stats

-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From bchesneau@REDACTED  Thu May 21 18:00:36 2015
From: bchesneau@REDACTED (Benoit Chesneau)
Date: Thu, 21 May 2015 16:00:36 +0000
Subject: [erlang-questions] how are implemented lists?
In-Reply-To: 
References: 
Message-ID: 

On Wed, May 20, 2015 at 9:58 AM H?kan Mattsson  wrote:

> On Tue, May 19, 2015 at 5:27 AM, Richard A. O'Keefe 
> wrote:
>
>>
>> If it weren't for the "concurrent" bit I'd suggest that
>> one of the set or dictionary data structures in stdlib
>> might be useful, or one of the Okasaki purely functional
>> data structures that I thought had been adapted to
>> Erlang, but can't find.
>>
>
> Perhaps you ar looking for the queue module?
>
> It is quite useful when you want to have a list where popping elements
> from the tail is efficient.
>
> /H?kan
>
>
Thanks all for your answers, it's really useful

In fact I wasn't thinking directly to use ETS at first. The geneeral idea
was indeed to have a sort of queue (pop or tail functions) but with the
ability to remove one item from it if needed with a performance better than
O(N). Which is if I understand correctly the case with  a list. In my
understanding a skip list algorithm would do the trick. Maybe there are
another data-structure for it that would work better with Erlang?

- benoit


>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From k.petrauskas@REDACTED  Thu May 21 19:08:38 2015
From: k.petrauskas@REDACTED (Karolis Petrauskas)
Date: Thu, 21 May 2015 20:08:38 +0300
Subject: [erlang-questions] Why have a supervisor behaviour?
In-Reply-To: 
References: 
 
 
 
Message-ID: 

Another reason for supervisor to be a behaviour is the code upgrades.
The init/1 function is called on process startup and on code upgrades.

Karolis

On Thu, May 21, 2015 at 6:58 PM, Jesper Louis Andersen
 wrote:
>
> On Thu, May 21, 2015 at 5:00 PM, Roger Lipscombe 
> wrote:
>>
>> I need delayed _restart_. Is this what Jesper refers to when he talks
>> about "a delay_manager"? Such that init queries that and then
>> might/might not delay?
>
>
> %% The delay manager decouples delay policy from a worker by tracking delays
> in one place.
> %% As such, it has global knowledge and can opt to delay registered
> processes more or less
> %% depending on current load.
> -module(delay_mgr).
> -behaviour(gen_server).
>
> [..]
>
> %% Call this from a newly started worker, but not in it's init/1 callback
> since that blocks the supervisor
> %% Send the process itself a message in init/1 and do it in that state.
> delay(Reg) ->
>     gen_server:call(?MODULE, {delay, Reg}, infinity).
>
> [..]
>
> handle_call({delay, Reg}, From, #state { conf = Conf }) ->
>     Delay = maps:get(Reg, Conf),
>     erlang:send_after(Delay, self(), {go, From}),
>     {noreply, State};
>
> handle_info({go, Reg}, State) ->
>     gen_server:reply(From, ok),
>     {noreply, State};
>
> [..]
>
> This is static skeleton, but:
>
> * Add monitoring of delayed processes. Increase the delay for processes that
> respawn too often
> * Decay delays for processes which operates as they should
> * Add metrics and stats
>
> --
> J.
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>


From jay@REDACTED  Thu May 21 21:56:58 2015
From: jay@REDACTED (Jay Nelson)
Date: Thu, 21 May 2015 12:56:58 -0700
Subject: [erlang-questions] Trying to understand the performance
	impact	of binary:split/3
Message-ID: 

Jose wrote:

> I wonder how fast it would be if binary:split/3 was written as a BIF.

I wrote a BIF in 2005 for manipulating binaries. It was documented in
an ICFP presentation and ACM paper. You can read about it, the
erlang code, and more importantly the actual BIF code at my website
http://duomark.com/erlang/index.html  listed under the title
?A Stream Library Using Erlang Binaries?. There are separate links
for the HTML Briefing and Code.

http://duomark.com/erlang/publications/acm2005.pdf 

From the abstract:

"The new BIFs are shown to improve performance as much as 250
times over native erlang functions. The reduction in memory usage
caused by the BIFs also allows successful processing in situations
that crashed the runtime as application functions."

There is a gzip tar file that you used to be able to overlay on R10B(!!)
to build the BIFs. But it is all contained in a single stream.c file and the
corresponding BIF table entries are in the other parts of the erts and
beam directories there, so it should be easy to adapt to the current
release you are working with. You should be able
to quickly get into the source and see if it meets any of your needs.
It will require some modification, and there are manual loop unrolling
and so forth that I did for performance, but as a rough quick test of
whether a BIF would be helpful this might be the quickest way to get
started.

If you do achieve interesting results, be sure to add reduction count
bumping in the BIF. This code predates the option of doing that.

jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From mononcqc@REDACTED  Thu May 21 22:32:40 2015
From: mononcqc@REDACTED (Fred Hebert)
Date: Thu, 21 May 2015 16:32:40 -0400
Subject: [erlang-questions] Why have a supervisor behaviour?
In-Reply-To: 
References: 
 
 
Message-ID: <20150521203238.GA52846@ferdair.local>

On 05/21, Roger Lipscombe wrote:
>I need delayed _restart_. Is this what Jesper refers to when he talks
>about "a delay_manager"? Such that init queries that and then
>might/might not delay?

That's a classic question, and one I started answering differently.  
Requiring a timeout in your supervisor rebooting function means that you 
are letting things crash or restart for the wrong reason.

The thing is, it's all about the guarantees[1]. In a nutshell, a 
supervisor should exit on any error, and ideally bring you back to a 
known, stable state.

So of course all expected or unexpected errors should be able to bring 
you back to that state properly, specifically transient errors.

But the distinction is that because supervisors boot synchronously for 
all apps, they also represent a chain of dependencies of what should be 
available to all processes started *after* them.

That's why failure modes such as 'one for all' or 'rest for one' exist.  
They allow you to specify that the processes there are related to each 
other in ways that their death violates some guarantee of invariant in 
the system and that the only good way to restart is by refreshing all of 
them.

In a nutshell, if you expect disconnections or event that require a 
backoff to happen frequently enough they are to be expected by the 
processes depending on yours, then that connection or that event is not 
a thing that should take place in your process' init function. Otherwise 
you're indirectly stating that without this thing working, the system 
should just not boot.

See the example in [2] for an idea of how to respect this. This does not 
change the code in any major way, but moves function calls around to 
properly respect these semantics.

My position is that this isn't a problem with supervisors' interface, 
but in how they are being use and what they mean for your system. I know 
this is not the most helpful response, but oh well.


[1]: http://ferd.ca/it-s-about-the-guarantees.html
[2]: http://www.erlang-in-anger.com, section 2.2.3


From trapexit@REDACTED  Fri May 22 02:04:46 2015
From: trapexit@REDACTED (Antonio SJ Musumeci)
Date: Thu, 21 May 2015 20:04:46 -0400
Subject: [erlang-questions] inet options / getopts
Message-ID: 

I've got a situation where I'd like to in effect duplicate a socket. Look
at it's options, peername, etc. and make another connection. Looking over
inet.erl I discovered inet:options/0 which provides a list of possible
options which can/could be used with inet:getopts/2 but I noticed a few
missing keys such as packet_size. Further in the module you'll see
connection_options/0 and listen_options/0 which would be perfect but they
aren't exported.

Anyone know why these aren't exported or why there isn't a shortcut to the
option list to getopts/2 like 'all'? Would there be any interest in a patch
providing some of this functionality?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From raimo+erlang-questions@REDACTED  Fri May 22 09:04:21 2015
From: raimo+erlang-questions@REDACTED (Raimo Niskanen)
Date: Fri, 22 May 2015 09:04:21 +0200
Subject: [erlang-questions] how are implemented lists?
In-Reply-To: 
References: 
 
Message-ID: <20150522070421.GA4414@erix.ericsson.se>

On Thu, May 21, 2015 at 04:00:36PM +0000, Benoit Chesneau wrote:
> On Wed, May 20, 2015 at 9:58 AM H?kan Mattsson  wrote:
> 
> > On Tue, May 19, 2015 at 5:27 AM, Richard A. O'Keefe 
> > wrote:
> >
> >>
> >> If it weren't for the "concurrent" bit I'd suggest that
> >> one of the set or dictionary data structures in stdlib
> >> might be useful, or one of the Okasaki purely functional
> >> data structures that I thought had been adapted to
> >> Erlang, but can't find.
> >>
> >
> > Perhaps you ar looking for the queue module?
> >
> > It is quite useful when you want to have a list where popping elements
> > from the tail is efficient.
> >
> > /H?kan
> >
> >
> Thanks all for your answers, it's really useful
> 
> In fact I wasn't thinking directly to use ETS at first. The geneeral idea
> was indeed to have a sort of queue (pop or tail functions) but with the
> ability to remove one item from it if needed with a performance better than
> O(N). Which is if I understand correctly the case with  a list. In my
> understanding a skip list algorithm would do the trick. Maybe there are
> another data-structure for it that would work better with Erlang?

Have a look at the brand new map data type.

> 
> - benoit
> 
> 
> >

> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions


-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB


From eriksoe@REDACTED  Fri May 22 09:09:42 2015
From: eriksoe@REDACTED (=?UTF-8?Q?Erik_S=C3=B8e_S=C3=B8rensen?=)
Date: Fri, 22 May 2015 09:09:42 +0200
Subject: [erlang-questions] how are implemented lists?
In-Reply-To: 
References: 
 
Message-ID: 

The "Erlangic" solution would, I think, be to
- have a process which holds the data;
- have one table (sorted_set) which contains {Time, Key}, where Time is
some unique monotonously increasing value (now(), or the recently
introduced replacement);
- have another table (set) which contains {Key, Time, Value}.
The process owns the tables, and inserts/deletes from both on each update.
In-process data structures could be used instead of tables.
This gives O(log(N)) update time.
Optimizations can be made, but I'd have to know more about the usage
pattern before I'd go into that...
/Erik
Den 21/05/2015 18.00 skrev "Benoit Chesneau" :

>
>
> On Wed, May 20, 2015 at 9:58 AM H?kan Mattsson  wrote:
>
>> On Tue, May 19, 2015 at 5:27 AM, Richard A. O'Keefe 
>> wrote:
>>
>>>
>>> If it weren't for the "concurrent" bit I'd suggest that
>>> one of the set or dictionary data structures in stdlib
>>> might be useful, or one of the Okasaki purely functional
>>> data structures that I thought had been adapted to
>>> Erlang, but can't find.
>>>
>>
>> Perhaps you ar looking for the queue module?
>>
>> It is quite useful when you want to have a list where popping elements
>> from the tail is efficient.
>>
>> /H?kan
>>
>>
> Thanks all for your answers, it's really useful
>
> In fact I wasn't thinking directly to use ETS at first. The geneeral idea
> was indeed to have a sort of queue (pop or tail functions) but with the
> ability to remove one item from it if needed with a performance better than
> O(N). Which is if I understand correctly the case with  a list. In my
> understanding a skip list algorithm would do the trick. Maybe there are
> another data-structure for it that would work better with Erlang?
>
> - benoit
>
>
>>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From bchesneau@REDACTED  Fri May 22 09:33:24 2015
From: bchesneau@REDACTED (Benoit Chesneau)
Date: Fri, 22 May 2015 07:33:24 +0000
Subject: [erlang-questions] how are implemented lists?
In-Reply-To: 
References: 
 
 
Message-ID: 

Hi,

On Fri, May 22, 2015 at 9:09 AM Erik S?e S?rensen  wrote:

> The "Erlangic" solution would, I think, be to
> - have a process which holds the data;
> - have one table (sorted_set) which contains {Time, Key}, where Time is
> some unique monotonously increasing value (now(), or the recently
> introduced replacement);
> - have another table (set) which contains {Key, Time, Value}.
> The process owns the tables, and inserts/deletes from both on each update.
> In-process data structures could be used instead of tables.
> This gives O(log(N)) update time.
>
Indeed I thought about it, but was afraid it would require to increst the
limit of the number of ETS tables in my case and I am not sure what does it
imply at the moment. The number of queues can be indeed higher than the
current limit.

> Optimizations can be made, but I'd have to know more about the usage
> pattern before I'd go into that...
>

For now I have a process that maintains a queue of items on which I am
doing selective receive to fetch from and append to some items to a queue
using the queue module.  Due to some events an item could have been already
handled on another node I need to discard it in the queue.

Right now to handle that I keep a list of the discarded items apart and
when the discarded item is fetched on queue I ignore it and ask for a new
item. With the drawback that I have to keep a place of where the items are
queued. So i was thinking to have a data structure that would allows me to
quickly check a if the item to discard is queued on that node/process and
then remove it or mark it as removed (The other way to do it would be
filtering the queue  and return the new filtered queue but It would be
quite inefficient imo). Not sure what's the best data structure for it if
it exists.

I guess having an ordset + dict (or maps) - or something like it - could
also do the trick you define above without requiring an ETS but I not sure
if it will perform the same.

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roberto@REDACTED  Fri May 22 10:10:20 2015
From: roberto@REDACTED (Roberto Ostinelli)
Date: Fri, 22 May 2015 10:10:20 +0200
Subject: [erlang-questions] Ranch listeners exiting
Message-ID: 

Dear list,
I keep on seeing various entries in my logs like this:

```
Ranch listener cometa_listener had connection process started with
cowboy_protocol:start_link/4 at <0.6039.0> exit with reason:
{closed,[{ranch_ssl,accept_ack,2,[{file,"src/ranch_ssl.erl"},{line,115}]},{cowboy_protocol,init,4,[{file,"src/cowboy_protocol.erl"},{line,91}]}]}
```

No other entries are set. Can someone tell me how I can trace down the
cause for this?

Thank you in advance,
r.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roberto@REDACTED  Fri May 22 11:09:52 2015
From: roberto@REDACTED (Roberto Ostinelli)
Date: Fri, 22 May 2015 11:09:52 +0200
Subject: [erlang-questions] Ranch listeners exiting
In-Reply-To: 
References: 
 
Message-ID: 

On Fri, May 22, 2015 at 10:18 AM, ?????? <236227381@REDACTED> wrote:

> 
> case ssl:ssl_accept(CSocket, Timeout) of
> ok ->
> ok;
> %% Garbage was most likely sent to the socket, don't error out.
> {error, {tls_alert, _}} ->
> ok = close(CSocket),
> exit(normal);
> %% Socket most likely stopped responding, don't error out.
> {error, timeout} ->
> ok = close(CSocket),
> exit(normal);
> {error, Reason} ->
> ok = close(CSocket),
> error(Reason)
> end.
> 
> as you can see,your socket connection is not accept through ssl
>


Not sure I understand. Do you mean that a non-SSL connection was requested
on the socket, which therefore considers it as "garbage"?

Thank you,
r.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From essen@REDACTED  Fri May 22 11:14:32 2015
From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=)
Date: Fri, 22 May 2015 12:14:32 +0300
Subject: [erlang-questions] Ranch listeners exiting
In-Reply-To: 
References: 
Message-ID: <555EF378.5070208@ninenines.eu>

It's harmless. The connection was closed by the client during SSL handshake.

Sometimes Cowboy or Ranch will spew some messages that should not be 
considered errors. We may or may not have a ticket for this, and it's 
something we improve on slowly.

On 05/22/2015 11:10 AM, Roberto Ostinelli wrote:
> Dear list,
> I keep on seeing various entries in my logs like this:
>
> ```
> Ranch listener cometa_listener had connection process started with
> cowboy_protocol:start_link/4 at <0.6039.0> exit with reason:
> {closed,[{ranch_ssl,accept_ack,2,[{file,"src/ranch_ssl.erl"},{line,115}]},{cowboy_protocol,init,4,[{file,"src/cowboy_protocol.erl"},{line,91}]}]}
> ```
>
> No other entries are set. Can someone tell me how I can trace down the
> cause for this?
>
> Thank you in advance,
> r.
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>

-- 
Lo?c Hoguin
http://ninenines.eu


From roberto@REDACTED  Fri May 22 11:16:50 2015
From: roberto@REDACTED (Roberto Ostinelli)
Date: Fri, 22 May 2015 11:16:50 +0200
Subject: [erlang-questions] Ranch listeners exiting
In-Reply-To: <555EF378.5070208@ninenines.eu>
References: 
 <555EF378.5070208@ninenines.eu>
Message-ID: 

On Fri, May 22, 2015 at 11:14 AM, Lo?c Hoguin  wrote:

> It's harmless. The connection was closed by the client during SSL
> handshake.
>
> Sometimes Cowboy or Ranch will spew some messages that should not be
> considered errors. We may or may not have a ticket for this, and it's
> something we improve on slowly.


Thank you Lo?c. Will add it to my ignore list.

Best,
r.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From 236227381@REDACTED  Fri May 22 10:18:53 2015
From: 236227381@REDACTED (=?gb18030?B?ydmyqLKosqiyqMnZ?=)
Date: Fri, 22 May 2015 16:18:53 +0800
Subject: [erlang-questions] =?gb18030?b?u9i4tKO6IFJhbmNoIGxpc3RlbmVycyBl?=
 =?gb18030?q?xiting?=
In-Reply-To: 
References: 
Message-ID: 


case ssl:ssl_accept(CSocket, Timeout) of
		ok ->
			ok;
		%% Garbage was most likely sent to the socket, don't error out.
		{error, {tls_alert, _}} ->
			ok = close(CSocket),
			exit(normal);
		%% Socket most likely stopped responding, don't error out.
		{error, timeout} ->
			ok = close(CSocket),
			exit(normal);
		{error, Reason} ->
			ok = close(CSocket),
			error(Reason)
	end.

as you can see,your socket connection is not accept through ssl




------------------ ???? ------------------
???: "Roberto Ostinelli"; 
????: 2015?5?22?(???) ??4:10
???: "Erlang"; 
??: [erlang-questions] Ranch listeners exiting



Dear list,I keep on seeing various entries in my logs like this:


```
Ranch listener cometa_listener had connection process started with cowboy_protocol:start_link/4 at <0.6039.0> exit with reason: {closed,[{ranch_ssl,accept_ack,2,[{file,"src/ranch_ssl.erl"},{line,115}]},{cowboy_protocol,init,4,[{file,"src/cowboy_protocol.erl"},{line,91}]}]}

```



No other entries are set. Can someone tell me how I can trace down the cause for this?


Thank you in advance,
r.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jose.valim@REDACTED  Fri May 22 12:41:30 2015
From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=)
Date: Fri, 22 May 2015 12:41:30 +0200
Subject: [erlang-questions] Trying to understand the performance impact
	of binary:split/3
In-Reply-To: 
References: 
Message-ID: 

Thanks for the fantastic reply Jay, I will look into this stuff carefully.



*Jos? Valim*
www.plataformatec.com.br
Skype: jv.ptec
Founder and Lead Developer

On Thu, May 21, 2015 at 9:56 PM, Jay Nelson  wrote:

> Jose wrote:
>
> > I wonder how fast it would be if binary:split/3 was written as a BIF.
>
> I wrote a BIF in 2005 for manipulating binaries. It was documented in
> an ICFP presentation and ACM paper. You can read about it, the
> erlang code, and more importantly the actual BIF code at my website
> http://duomark.com/erlang/index.html listed under the title
> ?A Stream Library Using Erlang Binaries?. There are separate links
> for the HTML Briefing and Code.
>
> http://duomark.com/erlang/publications/acm2005.pdf
>
> From the abstract:
>
> "The new BIFs are shown to improve performance as much as 250
> times over native erlang functions. The reduction in memory usage
> caused by the BIFs also allows successful processing in situations
> that crashed the runtime as application functions."
>
> There is a gzip tar file that you used to be able to overlay on R10B(!!)
> to build the BIFs. But it is all contained in a single stream.c file and
> the
> corresponding BIF table entries are in the other parts of the erts and
> beam directories there, so it should be easy to adapt to the current
> release you are working with. You should be able
> to quickly get into the source and see if it meets any of your needs.
> It will require some modification, and there are manual loop unrolling
> and so forth that I did for performance, but as a rough quick test of
> whether a BIF would be helpful this might be the quickest way to get
> started.
>
> If you do achieve interesting results, be sure to add reduction count
> bumping in the BIF. This code predates the option of doing that.
>
> jay
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From lostcolony@REDACTED  Fri May 22 14:08:43 2015
From: lostcolony@REDACTED (Christopher Phillips)
Date: Fri, 22 May 2015 08:08:43 -0400
Subject: [erlang-questions] Why have a supervisor behaviour?
Message-ID: 

>
>
> Message: 17
> Date: Thu, 21 May 2015 16:32:40 -0400
> From: Fred Hebert 
> To: Roger Lipscombe 
> Cc: "erlang-questions@REDACTED" 
> Subject: Re: [erlang-questions] Why have a supervisor behaviour?
> Message-ID: <20150521203238.GA52846@REDACTED>
> Content-Type: text/plain; charset=us-ascii; format=flowed
>
> On 05/21, Roger Lipscombe wrote:
> >I need delayed _restart_. Is this what Jesper refers to when he talks
> >about "a delay_manager"? Such that init queries that and then
> >might/might not delay?
>
> That's a classic question, and one I started answering differently.
> Requiring a timeout in your supervisor rebooting function means that you
> are letting things crash or restart for the wrong reason.
>
> The thing is, it's all about the guarantees[1]. In a nutshell, a
> supervisor should exit on any error, and ideally bring you back to a
> known, stable state.
>
> So of course all expected or unexpected errors should be able to bring
> you back to that state properly, specifically transient errors.
>
> But the distinction is that because supervisors boot synchronously for
> all apps, they also represent a chain of dependencies of what should be
> available to all processes started *after* them.
>
> That's why failure modes such as 'one for all' or 'rest for one' exist.
> They allow you to specify that the processes there are related to each
> other in ways that their death violates some guarantee of invariant in
> the system and that the only good way to restart is by refreshing all of
> them.
>
> In a nutshell, if you expect disconnections or event that require a
> backoff to happen frequently enough they are to be expected by the
> processes depending on yours, then that connection or that event is not
> a thing that should take place in your process' init function. Otherwise
> you're indirectly stating that without this thing working, the system
> should just not boot.
>
> See the example in [2] for an idea of how to respect this. This does not
> change the code in any major way, but moves function calls around to
> properly respect these semantics.
>
> My position is that this isn't a problem with supervisors' interface,
> but in how they are being use and what they mean for your system. I know
> this is not the most helpful response, but oh well.
>
>
> [1]: http://ferd.ca/it-s-about-the-guarantees.html
> [2]: http://www.erlang-in-anger.com, section 2.2.3
>
>
> I wanted to add, every time I've seen this pattern brought up (a
supervisor with a delayed restart), it's been due to something on the
network has become unavailable, or overloaded to where it can't respond in
a reasonable amount of time, that sort of thing, and the developers
involved were seeing restart limits being hit and the system coming down,
even though there was every reason to expect the resource to become
available again, in time.

If that's the case for you, those types of issues would likely be better
addressed using a circuit breaker or similar[1] than a custom supervisor,
for the reasons Fred mentions.

In general, having this thing be unavailable is either due to bad internal
state (in which case a supervisor can help you), or something external to
your program (in which case the supervisor can't), and in the latter case
you should be thinking about how the system should behave when it's
unavailable (since it would be unavailable even with a delayed supervisor
in any case). Making it a clean response that means "this resource is not
available" allows you to address it in a well defined way, rather than
having a child process that may or may not be there being called from
elsewhere in your system.

[1]: https://github.com/jlouis/fuse
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From roger@REDACTED  Fri May 22 15:51:38 2015
From: roger@REDACTED (Roger Lipscombe)
Date: Fri, 22 May 2015 14:51:38 +0100
Subject: [erlang-questions] Why have a supervisor behaviour?
In-Reply-To: <20150521203238.GA52846@ferdair.local>
References: 
 
 
 <20150521203238.GA52846@ferdair.local>
Message-ID: 

On 21 May 2015 at 21:32, Fred Hebert  wrote:
> [...a bunch of useful stuff, but most importantly this next bit...]
> My position is that this isn't a problem with supervisors' interface, but in
> how they are being use and what they mean for your system. I know this is
> not the most helpful response, but oh well.

It turns out that I probably don't need a supervisor at all, then.

Currently I have:

- A supervisor, which supervises:
- A collection of host processes, each of which owns:
- An instance of a Squirrel VM [1] implemented in a C++ NIF.
- When the Squirrel VM wishes to communicate with its host process, it
sends it a message.
- For some of those messages (divide by zero exception, syntax error,
etc.), my host process responds via
  handle_info({exception, Whatever}, State) -> {stop, {shutdown,
Whatever}, State}.
- This causes the supervisor to restart the host, which fires up a
fresh instance of the Squirrel VM.

Because the Squirrel VM is running arbitrary code, it can get itself
into a state where that code constantly crashes, so the host kills
itself, and the supervisor restarts it constantly. My existing custom
supervisor doesn't handle restart intensity, for precisely this
reason. If this happens really quickly, it can lead to bad effects
downstream (log spamming, etc.). Hence the business requirement to
delay the restart to give the rest of the system a breather.

It seems, however, that I *don't* really want a supervisor to handle
restarting the Squirrel VM; it looks like the host should do it, and I
might be able to remove my custom supervisor in favour of a standard
'simple_one_for_one' supervisor to handle crashes in the host process.
Not sure about that last -- I don't want one process hitting max
restart intensity to bring down the other host processes.

[1] http://www.squirrel-lang.org/


From mononcqc@REDACTED  Fri May 22 16:19:12 2015
From: mononcqc@REDACTED (Fred Hebert)
Date: Fri, 22 May 2015 10:19:12 -0400
Subject: [erlang-questions] Why have a supervisor behaviour?
In-Reply-To: 
References: 
 
 
 <20150521203238.GA52846@ferdair.local>
 
Message-ID: <20150522141911.GC52846@ferdair.local>

On 05/22, Roger Lipscombe wrote:
>It turns out that I probably don't need a supervisor at all, then.
>
> [project description]
>
>It seems, however, that I *don't* really want a supervisor to handle
>restarting the Squirrel VM; it looks like the host should do it, and I
>might be able to remove my custom supervisor in favour of a standard
>'simple_one_for_one' supervisor to handle crashes in the host process.
>Not sure about that last -- I don't want one process hitting max
>restart intensity to bring down the other host processes.
>

Ah that's interesting. To reason about this, one question to ask is: 
what is it that your system guarantees to its subsequent processes. So 
if you have some form of front-end or client handling the order of 
spawning and restarting a VM (who do you do it on behalf of?), there's 
likely a restricted set of operations you provide, right?

Something like:

- Run task
- Interrupt task
- Get task status or state report
- Has the task completed?

Or possibly, if you're going event-based, the following events are to be 
expected:

- Task accepted
- VM booted
- VM failed
- Task aborted
- Task completion

Those are probably things you expect to provide and should work fine, 
because those are the kinds of failures you do expect all the time from 
the Squirrel VM itself. Furthermore, it's possible you'd eventually add 
in a backpressure mechanism ("only 10 VMs can run at a time for a user") 
or something like that. This means what you might want is the host 
process to always be able to provide that information, and isolate your 
user from the VM process' fickle behaviour.

So what does this tell us? What you guarantee when the supervision tree 
is booted is therefore:

- I can contact the system to know if I can host a VM and run it
- Once I am given a process, there's a manager (the host process) I can 
  talk to or expect to get information from.

There is no guarantee about the Squirrel VM being up and running and 
available; there's a good likelihood it's gonna be there, but in 
reality, it can go terribly bad and we just can't pretend it's not gonna 
take place.

This means that these two types of processes are those you want to be 
ready and available as soon as 'init/1' has been executed. That a VM is 
available or not is not core functionality; what's core is that you can 
ask to get one, and know if it didn't work.

To really help figure this out, simply ask "Can my system still run if X 
is not there?" If it can run without it, then your main recovery 
mechanism should probably not be the supervisor through failed `init/1` 
calls; it's a thing that likely becomes your responsibility as a 
developer because it's a common event. It might need to move to 
`handle_info/2`; If the system can't run without it, encode it in the 
`init/1` function. It's a guarantee you have to make.

You'll find out that for some database connections, it's true. For some 
it's not and the DB *needs* to be there for the system to make sense.  
The supervisors then let you encode these requirements in your program 
structure, and their boot and shutdown sequences. Same for anything you 
may depend on.

Does this make sense?

Then to pick the exact supervision strategy and error handling 
mechanism, you can ask yourself what do you do when the host process 
dies. Can a new one take its place seemlessly? If not, then it's 
possible the error needs to bubble up (through a monitor or some 
message) to the caller so *they* decide whether to give up or try again.  
If you can make it transparently or it's a best effort mechanism, then 
yeah, just restarting the worker is enough.

"Let it crash" is a fun fun way to get going and to grow a system, but 
when it has reached some level of growth, we can't avoid starting to 
really reason about how we want things to fail; It lets us slowly 
discover the properties we want to expose to our users, and after a few 
solid crashes, it's entirely fine to reorganize a few bits of code to 
reflect the real world and its constraints.

What's great is that we've goot all the building blocks and tools to 
reason about it and implement the solution properly.

Regards,
Fred.


From roger@REDACTED  Fri May 22 17:02:43 2015
From: roger@REDACTED (Roger Lipscombe)
Date: Fri, 22 May 2015 16:02:43 +0100
Subject: [erlang-questions] Why have a supervisor behaviour?
In-Reply-To: <20150522141911.GC52846@ferdair.local>
References: 
 
 
 <20150521203238.GA52846@ferdair.local>
 
 <20150522141911.GC52846@ferdair.local>
Message-ID: 

On 22 May 2015 at 15:19, Fred Hebert  wrote:
> On 05/22, Roger Lipscombe wrote:
>>
>> It turns out that I probably don't need a supervisor at all, then.
>
> Ah that's interesting. To reason about this, one question to ask is: what is
> it that your system guarantees to its subsequent processes. So if you have
> some form of front-end or client handling the order of spawning and
> restarting a VM (who do you do it on behalf of?), there's likely a
> restricted set of operations you provide, right?

Yes:
- Find (or start) a VM by ID.
- Stop a VM, either by ID or by sending it a message.
- Send a message to a VM. Some of these are gen_server:call, because
either we need back-pressure, or we need a response; some are
gen_server:cast (or Pid ! foo), because we don't.

> To really help figure this out, simply ask "Can my system still run if X is
> not there?"

In this case, yes. If we get into a situation where we're consistently
failing to start the squirrel VM, or if they're _all_ consistently
failing, we'll spot that through metrics or another form of alerting.

> Does this make sense?

Yes. Absolutely. I need the squirrel_vm_manager (to give it a name) to
be up, so that I can find_or_start and stop the VMs. Whether a
particular VM is up or not is not a problem for *my* application as a
whole.

Thanks,
Roger.


From max.lapshin@REDACTED  Fri May 22 19:16:34 2015
From: max.lapshin@REDACTED (Max Lapshin)
Date: Fri, 22 May 2015 20:16:34 +0300
Subject: [erlang-questions] SO_REUSEPORT with UDP
In-Reply-To: <9BF17FBFE6E14E96AA96622C1C822C25@gruchalski.com>
References: 
 
 <9BF17FBFE6E14E96AA96622C1C822C25@gruchalski.com>
Message-ID: 

Wow!

Thanks for this thing!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From kennethlakin@REDACTED  Sat May 23 07:42:02 2015
From: kennethlakin@REDACTED (Kenneth Lakin)
Date: Fri, 22 May 2015 22:42:02 -0700
Subject: [erlang-questions] Mnesia doesn't report insertion errors when
 underlying DETS file is full?
Message-ID: <5560132A.4040107@gmail.com>

Hi, all. Here's hoping that I'm just misunderstanding how things work.
I've done some searching, but it doesn't seem that anyone has asked this
question on the list before.

I'm testing with Erlang 17.3 (erts 6.2) on AMD64 Kubuntu 15.04.

I wrote a little test program that uses mnesia:write/1 inside of a
mnesia:transaction/1 to write a single integer into a Mnesia
disc_only_copies table. Once the table grows to ~2,097,000 bytes -give
or take-, the the on-disk table file stops growing, and mnesia:info/0
reports that the number of records contained in the table remains constant.

It's obvious that *something* is wrong, as attempts to use mnesia:first
or mnesia:last return {aborted, {badarg, [table]}}. However, the write
operations wrapped in mnesia:transaction keep returning {atomic,
Result}, and comparison between a couple of calls of mnesia:info reveals
that transactions continue to be committed and logged to disk (even
though no new data *sticks around* on disk). I also tested this with
mnesia:activity/2. When the DETS table hits 2GB, mnesia:activity keeps
on trucking: it doesn't throw or exit when called with either
transaction, or async_dirty.

I would have expected mnesia:transaction to return {aborted, table_full}
or something, rather than lying that my data was committed. Is this an
unreasonable expectation? Did the Ubuntu or Debian folks mispackage
Erlang in some way? Is this an oversight on the part of the OTP folks?

For the curious, my program is at the end of this message. (Run it with
test/0.)

Thanks,
-Kenneth Lakin
%%%%%
-module(test).
-compile(export_all).
-record(rec, {key, val}).

test() ->
  %Default to signed intmax iterations.
  test(math:pow(2, 32)/2).
test(Iters) ->
  mnesia:create_schema([node()]),
  mnesia:start(),
  mnesia:wait_for_tables([test], timer:seconds(30)),
  mnesia:clear_table(test),
  mnesia:create_table(test, [{attributes, record_info(fields, rec)},
                             {type, bag},
                             {disc_only_copies, [node()]}
                            ]),
  insertData(Iters).

insertData(Iters) ->
  insertData(0, Iters).
insertData(Step, Iters) when Iters == Step ->
  ok;
insertData(Step, Iters) ->
  F=fun() -> mnesia:write({test, Step, 0}) end,
  %This doesn't seem to throw or exit when
  %inserts fail. Trying mnesia:transaction.
  %mnesia:activity(transaction, F),
  {atomic, _} = mnesia:transaction(F),
  insertData(Step+1, Iters).

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

From ulf@REDACTED  Sat May 23 10:20:21 2015
From: ulf@REDACTED (Ulf Wiger)
Date: Sat, 23 May 2015 10:20:21 +0200
Subject: [erlang-questions] Mnesia doesn't report insertion errors when
	underlying DETS file is full?
In-Reply-To: <5560132A.4040107@gmail.com>
References: <5560132A.4040107@gmail.com>
Message-ID: <730EBF82-9E07-4CBD-A087-A580A2DF18C9@feuerlabs.com>


> On 23 May 2015, at 07:42, Kenneth Lakin  wrote:
> 
> When the DETS table hits 2GB, mnesia:activity keeps
> on trucking: it doesn't throw or exit when called with either
> transaction, or async_dirty.
> 
> I would have expected mnesia:transaction to return {aborted, table_full}
> or something, rather than lying that my data was committed. Is this an
> unreasonable expectation? Did the Ubuntu or Debian folks mispackage
> Erlang in some way? Is this an oversight on the part of the OTP folks?

This is a known problem. Dets returns an error when the table is full, but this error cannot be handled by mnesia, since it occurs *after* commit - mnesia simply expects the backend store to work.

There are a number of ways to address the issue:

* Use mnesia_frag to fragment disc_only tables. This is no guarantee that you will avoid the issue, but by continuously monitoring the number of objects vs number of fragments, you can minimize the risk. I consider this a pretty ugly solution, but it has been used in commercial systems. One can of course also manually ?fragment? the data, but this has the same issues, and then some.

* Use disc_copies, ensure that you have a big enough box to keep all data in RAM. This has been used to great lengths by some commercial companies. The solution is not that bad, although eventually, you end up with some pretty ridiculous hardware, and pretty long startup times, since all data must be bulk-loaded into memory. One advantage is that reads and match operations become wicked fast.

* Fix dets to support larger data sets. Richard Carlsson once wrote a 64-bit dets version, but it has not been adopted by OTP. One could argue that the underlying dets design is unsuitable for very large data sets, e.g. due to its recovery semantics.

* Use an alternative backend. This has been done a few times, e.g. with ?mnesiaex? [1], but AFAIK, it is nowadays obsolete. Klarna has presented a similar approach, using the eleveldb backend [2], but has not (unless I missed it) made any recent announcement regarding status or availability of this solution.

* Use another database solution. Many projects do this. You will forfeit some of the unique characteristics of mnesia - in particular, the extremely close integration with the runtime environment - but obviously gain other things.

BR,
Ulf W

[1] https://www.openhub.net/p/mnesiaex
[2] https://erlangcentral.org/mnesia-backend-plugin-framework-and-a-leveldb-based-plugin/#.VWA08FmeDGc

Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
http://feuerlabs.com





From ulf@REDACTED  Sat May 23 10:30:31 2015
From: ulf@REDACTED (Ulf Wiger)
Date: Sat, 23 May 2015 10:30:31 +0200
Subject: [erlang-questions] Mnesia doesn't report insertion errors when
	underlying DETS file is full?
In-Reply-To: <730EBF82-9E07-4CBD-A087-A580A2DF18C9@feuerlabs.com>
References: <5560132A.4040107@gmail.com>
 <730EBF82-9E07-4CBD-A087-A580A2DF18C9@feuerlabs.com>
Message-ID: 


> On 23 May 2015, at 10:20, Ulf Wiger  wrote:
> 
> * Use an alternative backend. This has been done a few times, e.g. with ?mnesiaex? [1], but AFAIK, it is nowadays obsolete. Klarna has presented a similar approach, using the eleveldb backend [2], but has not (unless I missed it) made any recent announcement regarding status or availability of this solution.

Ah, I did miss that Mikael Pettersson of Klarna will talk about this at the upcoming EUC:

http://www.erlang-factory.com/euc2015/mikael-pettersson
?mnesia + leveldb: liberating mnesia from the limitations of DETS
Mnesia offers various database features, but restricts users to a few storage engines with substantial limitations. This talk describes mnesia_ext, an extension which allows arbitrary storage engines to be plugged into mnesia, and how Klarna used this to migrate parts of its database to LevelDB. We will also talk about our experiences with LevelDB, and some improvements we have made.

Talk objectives:

- Present mnesia_ext. Show that LevelDB is a production-grade storage engine for Erlang (with or without mnesia on top).

Target audience:

- Backend developers. Erlang/OTP maintainers.?

BR,
Ulf W

Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
http://feuerlabs.com





From eric.pailleau@REDACTED  Sat May 23 10:38:56 2015
From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=)
Date: Sat, 23 May 2015 10:38:56 +0200
Subject: [erlang-questions] Mnesia doesn't report insertion errors
	when	underlying DETS file is full?
In-Reply-To: 
Message-ID: 

Hi,
Just wanted to know if mnesia subscribe on system events can raise a

{mnesia_error, Format, Args}

In such case ?

Regards


Le?23 mai 2015 10:30, Ulf Wiger  a ?crit?:
>
>
> > On 23 May 2015, at 10:20, Ulf Wiger  wrote:
> > 
> > * Use an alternative backend. This has been done a few times, e.g. with ?mnesiaex? [1], but AFAIK, it is nowadays obsolete. Klarna has presented a similar approach, using the eleveldb backend [2], but has not (unless I missed it) made any recent announcement regarding status or availability of this solution.
>
> Ah, I did miss that Mikael Pettersson of Klarna will talk about this at the upcoming EUC:
>
> http://www.erlang-factory.com/euc2015/mikael-pettersson
> ?mnesia + leveldb: liberating mnesia from the limitations of DETS
> Mnesia offers various database features, but restricts users to a few storage engines with substantial limitations. This talk describes mnesia_ext, an extension which allows arbitrary storage engines to be plugged into mnesia, and how Klarna used this to migrate parts of its database to LevelDB. We will also talk about our experiences with LevelDB, and some improvements we have made.
>
> Talk objectives:
>
> - Present mnesia_ext. Show that LevelDB is a production-grade storage engine for Erlang (with or without mnesia on top).
>
> Target audience:
>
> - Backend developers. Erlang/OTP maintainers.?
>
> BR,
> Ulf W
>
> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
> http://feuerlabs.com
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

From ulf@REDACTED  Sat May 23 13:02:43 2015
From: ulf@REDACTED (Ulf Wiger)
Date: Sat, 23 May 2015 13:02:43 +0200
Subject: [erlang-questions] Mnesia doesn't report insertion errors when
	underlying DETS file is full?
In-Reply-To: 
References: 
Message-ID: <826DC5E6-B902-411D-8F35-D17027D147F7@feuerlabs.com>


> On 23 May 2015, at 10:38, ?ric Pailleau  wrote:
> 
> Just wanted to know if mnesia subscribe on system events can raise a
> 
> {mnesia_error, Format, Args}
> 
> In such case ?

It *almost* does, if you set mnesia?s debug level to ?verbose?. ;-)

The following call is made from mnesia_tm if do_update_op/3 crashes:

            verbose("do_update in ~w failed: ~p -> {'EXIT', ~p}~n",
                    [Tid, Op, Reason]),

?which, in its turn ends up calling the following in mnesia_lib:

important(Format, Args) ->
    save({Format, Args}),
    report_system_event({mnesia_info, Format, Args}).

The only problem is that the dets operation in question doesn?t crash, but instead returns {error, Reason}, so that particular issue won?t be reported. The value *is* captured by mnesia_tm:do_update/4, and it will return the last non-ok value from the commit. However, this return value is discarded, except in a few cases where it?s captured as debug info.

Of course, even if it were reported, it would be of the category ?oops - your database was just corrupted!?, which is good to know, but not so easy to handle.

BR,
Ulf W

Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
http://feuerlabs.com





From eric.pailleau@REDACTED  Sat May 23 14:25:20 2015
From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=)
Date: Sat, 23 May 2015 14:25:20 +0200
Subject: [erlang-questions] Mnesia doesn't report insertion errors when
 underlying DETS file is full?
In-Reply-To: <826DC5E6-B902-411D-8F35-D17027D147F7@feuerlabs.com>
Message-ID: <29229e97-d62e-447f-91ba-2b33377db33b@email.android.com>

Hi, 
Thanks Ulf.
It would be probably more smart to warn before to loose data...
A system event 'almost_full' should be more convenient. It is sometime better to stop the application instead loose data.
An event on each last 5 percent change could allow to handle different actions. Mail, SMS, or ultimately stop the application on almost_full_99. 
There is already an event on database inconsistency, it would be a nice enhancement.
Regards 

Le?23 mai 2015 13:02, Ulf Wiger  a ?crit?:
>
>
> > On 23 May 2015, at 10:38, ?ric Pailleau  wrote: 
> > 
> > Just wanted to know if mnesia subscribe on system events can raise a 
> > 
> > {mnesia_error, Format, Args} 
> > 
> > In such case ? 
>
> It *almost* does, if you set mnesia?s debug level to ?verbose?. ;-) 
>
> The following call is made from mnesia_tm if do_update_op/3 crashes: 
>
> ??????????? verbose("do_update in ~w failed: ~p -> {'EXIT', ~p}~n", 
> ??????????????????? [Tid, Op, Reason]), 
>
> ?which, in its turn ends up calling the following in mnesia_lib: 
>
> important(Format, Args) -> 
> ??? save({Format, Args}), 
> ??? report_system_event({mnesia_info, Format, Args}). 
>
> The only problem is that the dets operation in question doesn?t crash, but instead returns {error, Reason}, so that particular issue won?t be reported. The value *is* captured by mnesia_tm:do_update/4, and it will return the last non-ok value from the commit. However, this return value is discarded, except in a few cases where it?s captured as debug info. 
>
> Of course, even if it were reported, it would be of the category ?oops - your database was just corrupted!?, which is good to know, but not so easy to handle. 
>
> BR, 
> Ulf W 
>
> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. 
> http://feuerlabs.com 
>
>
>

From kgd130kgd@REDACTED  Sat May 23 15:31:51 2015
From: kgd130kgd@REDACTED (kgd130kgd)
Date: Sat, 23 May 2015 21:31:51 +0800 (CST)
Subject: [erlang-questions] =?gbk?q?=BB=D8=B8=B4=3Aerlang-questions_Digest?=
	=?gbk?q?=2C_Vol_218=2C_Issue_8?=
In-Reply-To: 
References: 
Message-ID: <2b34de46.7680.14d80f8feb1.Coremail.kgd130kgd@126.com>

i use redhat6.5 make install ejabberd15.2(github version) report error like this.
i not change anything code
-c -m 644 deps/*/lib/*/ebin/*.app //lib/ejabberd/ebin
/usr/bin/install: ????"deps/*/lib/*/ebin/*.app" ?????(stat): ?????????
make: [install] ?? 1 (??)
/usr/bin/install -c -m 644 deps/*/lib/*/ebin/*.beam //lib/ejabberd/ebin
/usr/bin/install: ????"deps/*/lib/*/ebin/*.beam" ?????(stat): ?????????









At 2015-05-23 18:00:00, ""  wrote:
>Send erlang-questions mailing list submissions to
>	erlang-questions@REDACTED
>
>To subscribe or unsubscribe via the World Wide Web, visit
>	http://erlang.org/mailman/listinfo/erlang-questions
>or, via email, send a message with subject or body 'help' to
>	erlang-questions-request@REDACTED
>
>You can reach the person managing the list at
>	erlang-questions-owner@REDACTED
>
>When replying, please edit your Subject line so it is more specific
>than "Re: Contents of erlang-questions digest..."
>
>
>Today's Topics:
>
>   1. Re:  Trying to understand the performance impact	of
>      binary:split/3 (Jos? Valim)
>   2. Re:  Why have a supervisor behaviour? (Christopher Phillips)
>   3. Re:  Why have a supervisor behaviour? (Roger Lipscombe)
>   4. Re:  Why have a supervisor behaviour? (Fred Hebert)
>   5. Re:  Why have a supervisor behaviour? (Roger Lipscombe)
>   6. Re:  SO_REUSEPORT with UDP (Max Lapshin)
>   7.  Mnesia doesn't report insertion errors when underlying DETS
>      file is full? (Kenneth Lakin)
>   8. Re:  Mnesia doesn't report insertion errors when	underlying
>      DETS file is full? (Ulf Wiger)
>   9. Re:  Mnesia doesn't report insertion errors when	underlying
>      DETS file is full? (Ulf Wiger)
>  10. Re:  Mnesia doesn't report insertion errors	when	underlying
>      DETS file is full? (?ric Pailleau)
>
>
>----------------------------------------------------------------------
>
>Message: 1
>Date: Fri, 22 May 2015 12:41:30 +0200
>From: Jos? Valim 
>To: Jay Nelson 
>Cc: erlang-questions 
>Subject: Re: [erlang-questions] Trying to understand the performance
>	impact	of binary:split/3
>Message-ID:
>	
>Content-Type: text/plain; charset="utf-8"
>
>Thanks for the fantastic reply Jay, I will look into this stuff carefully.
>
>
>
>*Jos? Valim*
>www.plataformatec.com.br
>Skype: jv.ptec
>Founder and Lead Developer
>
>On Thu, May 21, 2015 at 9:56 PM, Jay Nelson  wrote:
>
>> Jose wrote:
>>
>> > I wonder how fast it would be if binary:split/3 was written as a BIF.
>>
>> I wrote a BIF in 2005 for manipulating binaries. It was documented in
>> an ICFP presentation and ACM paper. You can read about it, the
>> erlang code, and more importantly the actual BIF code at my website
>> http://duomark.com/erlang/index.html listed under the title
>> ?A Stream Library Using Erlang Binaries?. There are separate links
>> for the HTML Briefing and Code.
>>
>> http://duomark.com/erlang/publications/acm2005.pdf
>>
>> From the abstract:
>>
>> "The new BIFs are shown to improve performance as much as 250
>> times over native erlang functions. The reduction in memory usage
>> caused by the BIFs also allows successful processing in situations
>> that crashed the runtime as application functions."
>>
>> There is a gzip tar file that you used to be able to overlay on R10B(!!)
>> to build the BIFs. But it is all contained in a single stream.c file and
>> the
>> corresponding BIF table entries are in the other parts of the erts and
>> beam directories there, so it should be easy to adapt to the current
>> release you are working with. You should be able
>> to quickly get into the source and see if it meets any of your needs.
>> It will require some modification, and there are manual loop unrolling
>> and so forth that I did for performance, but as a rough quick test of
>> whether a BIF would be helpful this might be the quickest way to get
>> started.
>>
>> If you do achieve interesting results, be sure to add reduction count
>> bumping in the BIF. This code predates the option of doing that.
>>
>> jay
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: 
>
>------------------------------
>
>Message: 2
>Date: Fri, 22 May 2015 08:08:43 -0400
>From: Christopher Phillips 
>To: Roger Lipscombe 
>Cc: "erlang-questions@REDACTED" 
>Subject: Re: [erlang-questions] Why have a supervisor behaviour?
>Message-ID:
>	
>Content-Type: text/plain; charset="utf-8"
>
>>
>>
>> Message: 17
>> Date: Thu, 21 May 2015 16:32:40 -0400
>> From: Fred Hebert 
>> To: Roger Lipscombe 
>> Cc: "erlang-questions@REDACTED" 
>> Subject: Re: [erlang-questions] Why have a supervisor behaviour?
>> Message-ID: <20150521203238.GA52846@REDACTED>
>> Content-Type: text/plain; charset=us-ascii; format=flowed
>>
>> On 05/21, Roger Lipscombe wrote:
>> >I need delayed _restart_. Is this what Jesper refers to when he talks
>> >about "a delay_manager"? Such that init queries that and then
>> >might/might not delay?
>>
>> That's a classic question, and one I started answering differently.
>> Requiring a timeout in your supervisor rebooting function means that you
>> are letting things crash or restart for the wrong reason.
>>
>> The thing is, it's all about the guarantees[1]. In a nutshell, a
>> supervisor should exit on any error, and ideally bring you back to a
>> known, stable state.
>>
>> So of course all expected or unexpected errors should be able to bring
>> you back to that state properly, specifically transient errors.
>>
>> But the distinction is that because supervisors boot synchronously for
>> all apps, they also represent a chain of dependencies of what should be
>> available to all processes started *after* them.
>>
>> That's why failure modes such as 'one for all' or 'rest for one' exist.
>> They allow you to specify that the processes there are related to each
>> other in ways that their death violates some guarantee of invariant in
>> the system and that the only good way to restart is by refreshing all of
>> them.
>>
>> In a nutshell, if you expect disconnections or event that require a
>> backoff to happen frequently enough they are to be expected by the
>> processes depending on yours, then that connection or that event is not
>> a thing that should take place in your process' init function. Otherwise
>> you're indirectly stating that without this thing working, the system
>> should just not boot.
>>
>> See the example in [2] for an idea of how to respect this. This does not
>> change the code in any major way, but moves function calls around to
>> properly respect these semantics.
>>
>> My position is that this isn't a problem with supervisors' interface,
>> but in how they are being use and what they mean for your system. I know
>> this is not the most helpful response, but oh well.
>>
>>
>> [1]: http://ferd.ca/it-s-about-the-guarantees.html
>> [2]: http://www.erlang-in-anger.com, section 2.2.3
>>
>>
>> I wanted to add, every time I've seen this pattern brought up (a
>supervisor with a delayed restart), it's been due to something on the
>network has become unavailable, or overloaded to where it can't respond in
>a reasonable amount of time, that sort of thing, and the developers
>involved were seeing restart limits being hit and the system coming down,
>even though there was every reason to expect the resource to become
>available again, in time.
>
>If that's the case for you, those types of issues would likely be better
>addressed using a circuit breaker or similar[1] than a custom supervisor,
>for the reasons Fred mentions.
>
>In general, having this thing be unavailable is either due to bad internal
>state (in which case a supervisor can help you), or something external to
>your program (in which case the supervisor can't), and in the latter case
>you should be thinking about how the system should behave when it's
>unavailable (since it would be unavailable even with a delayed supervisor
>in any case). Making it a clean response that means "this resource is not
>available" allows you to address it in a well defined way, rather than
>having a child process that may or may not be there being called from
>elsewhere in your system.
>
>[1]: https://github.com/jlouis/fuse
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: 
>
>------------------------------
>
>Message: 3
>Date: Fri, 22 May 2015 14:51:38 +0100
>From: Roger Lipscombe 
>To: Fred Hebert 
>Cc: "erlang-questions@REDACTED" 
>Subject: Re: [erlang-questions] Why have a supervisor behaviour?
>Message-ID:
>	
>Content-Type: text/plain; charset=UTF-8
>
>On 21 May 2015 at 21:32, Fred Hebert  wrote:
>> [...a bunch of useful stuff, but most importantly this next bit...]
>> My position is that this isn't a problem with supervisors' interface, but in
>> how they are being use and what they mean for your system. I know this is
>> not the most helpful response, but oh well.
>
>It turns out that I probably don't need a supervisor at all, then.
>
>Currently I have:
>
>- A supervisor, which supervises:
>- A collection of host processes, each of which owns:
>- An instance of a Squirrel VM [1] implemented in a C++ NIF.
>- When the Squirrel VM wishes to communicate with its host process, it
>sends it a message.
>- For some of those messages (divide by zero exception, syntax error,
>etc.), my host process responds via
>  handle_info({exception, Whatever}, State) -> {stop, {shutdown,
>Whatever}, State}.
>- This causes the supervisor to restart the host, which fires up a
>fresh instance of the Squirrel VM.
>
>Because the Squirrel VM is running arbitrary code, it can get itself
>into a state where that code constantly crashes, so the host kills
>itself, and the supervisor restarts it constantly. My existing custom
>supervisor doesn't handle restart intensity, for precisely this
>reason. If this happens really quickly, it can lead to bad effects
>downstream (log spamming, etc.). Hence the business requirement to
>delay the restart to give the rest of the system a breather.
>
>It seems, however, that I *don't* really want a supervisor to handle
>restarting the Squirrel VM; it looks like the host should do it, and I
>might be able to remove my custom supervisor in favour of a standard
>'simple_one_for_one' supervisor to handle crashes in the host process.
>Not sure about that last -- I don't want one process hitting max
>restart intensity to bring down the other host processes.
>
>[1] http://www.squirrel-lang.org/
>
>
>------------------------------
>
>Message: 4
>Date: Fri, 22 May 2015 10:19:12 -0400
>From: Fred Hebert 
>To: Roger Lipscombe 
>Cc: "erlang-questions@REDACTED" 
>Subject: Re: [erlang-questions] Why have a supervisor behaviour?
>Message-ID: <20150522141911.GC52846@REDACTED>
>Content-Type: text/plain; charset=us-ascii; format=flowed
>
>On 05/22, Roger Lipscombe wrote:
>>It turns out that I probably don't need a supervisor at all, then.
>>
>> [project description]
>>
>>It seems, however, that I *don't* really want a supervisor to handle
>>restarting the Squirrel VM; it looks like the host should do it, and I
>>might be able to remove my custom supervisor in favour of a standard
>>'simple_one_for_one' supervisor to handle crashes in the host process.
>>Not sure about that last -- I don't want one process hitting max
>>restart intensity to bring down the other host processes.
>>
>
>Ah that's interesting. To reason about this, one question to ask is: 
>what is it that your system guarantees to its subsequent processes. So 
>if you have some form of front-end or client handling the order of 
>spawning and restarting a VM (who do you do it on behalf of?), there's 
>likely a restricted set of operations you provide, right?
>
>Something like:
>
>- Run task
>- Interrupt task
>- Get task status or state report
>- Has the task completed?
>
>Or possibly, if you're going event-based, the following events are to be 
>expected:
>
>- Task accepted
>- VM booted
>- VM failed
>- Task aborted
>- Task completion
>
>Those are probably things you expect to provide and should work fine, 
>because those are the kinds of failures you do expect all the time from 
>the Squirrel VM itself. Furthermore, it's possible you'd eventually add 
>in a backpressure mechanism ("only 10 VMs can run at a time for a user") 
>or something like that. This means what you might want is the host 
>process to always be able to provide that information, and isolate your 
>user from the VM process' fickle behaviour.
>
>So what does this tell us? What you guarantee when the supervision tree 
>is booted is therefore:
>
>- I can contact the system to know if I can host a VM and run it
>- Once I am given a process, there's a manager (the host process) I can 
>  talk to or expect to get information from.
>
>There is no guarantee about the Squirrel VM being up and running and 
>available; there's a good likelihood it's gonna be there, but in 
>reality, it can go terribly bad and we just can't pretend it's not gonna 
>take place.
>
>This means that these two types of processes are those you want to be 
>ready and available as soon as 'init/1' has been executed. That a VM is 
>available or not is not core functionality; what's core is that you can 
>ask to get one, and know if it didn't work.
>
>To really help figure this out, simply ask "Can my system still run if X 
>is not there?" If it can run without it, then your main recovery 
>mechanism should probably not be the supervisor through failed `init/1` 
>calls; it's a thing that likely becomes your responsibility as a 
>developer because it's a common event. It might need to move to 
>`handle_info/2`; If the system can't run without it, encode it in the 
>`init/1` function. It's a guarantee you have to make.
>
>You'll find out that for some database connections, it's true. For some 
>it's not and the DB *needs* to be there for the system to make sense.  
>The supervisors then let you encode these requirements in your program 
>structure, and their boot and shutdown sequences. Same for anything you 
>may depend on.
>
>Does this make sense?
>
>Then to pick the exact supervision strategy and error handling 
>mechanism, you can ask yourself what do you do when the host process 
>dies. Can a new one take its place seemlessly? If not, then it's 
>possible the error needs to bubble up (through a monitor or some 
>message) to the caller so *they* decide whether to give up or try again.  
>If you can make it transparently or it's a best effort mechanism, then 
>yeah, just restarting the worker is enough.
>
>"Let it crash" is a fun fun way to get going and to grow a system, but 
>when it has reached some level of growth, we can't avoid starting to 
>really reason about how we want things to fail; It lets us slowly 
>discover the properties we want to expose to our users, and after a few 
>solid crashes, it's entirely fine to reorganize a few bits of code to 
>reflect the real world and its constraints.
>
>What's great is that we've goot all the building blocks and tools to 
>reason about it and implement the solution properly.
>
>Regards,
>Fred.
>
>
>------------------------------
>
>Message: 5
>Date: Fri, 22 May 2015 16:02:43 +0100
>From: Roger Lipscombe 
>To: Fred Hebert 
>Cc: "erlang-questions@REDACTED" 
>Subject: Re: [erlang-questions] Why have a supervisor behaviour?
>Message-ID:
>	
>Content-Type: text/plain; charset=UTF-8
>
>On 22 May 2015 at 15:19, Fred Hebert  wrote:
>> On 05/22, Roger Lipscombe wrote:
>>>
>>> It turns out that I probably don't need a supervisor at all, then.
>>
>> Ah that's interesting. To reason about this, one question to ask is: what is
>> it that your system guarantees to its subsequent processes. So if you have
>> some form of front-end or client handling the order of spawning and
>> restarting a VM (who do you do it on behalf of?), there's likely a
>> restricted set of operations you provide, right?
>
>Yes:
>- Find (or start) a VM by ID.
>- Stop a VM, either by ID or by sending it a message.
>- Send a message to a VM. Some of these are gen_server:call, because
>either we need back-pressure, or we need a response; some are
>gen_server:cast (or Pid ! foo), because we don't.
>
>> To really help figure this out, simply ask "Can my system still run if X is
>> not there?"
>
>In this case, yes. If we get into a situation where we're consistently
>failing to start the squirrel VM, or if they're _all_ consistently
>failing, we'll spot that through metrics or another form of alerting.
>
>> Does this make sense?
>
>Yes. Absolutely. I need the squirrel_vm_manager (to give it a name) to
>be up, so that I can find_or_start and stop the VMs. Whether a
>particular VM is up or not is not a problem for *my* application as a
>whole.
>
>Thanks,
>Roger.
>
>
>------------------------------
>
>Message: 6
>Date: Fri, 22 May 2015 20:16:34 +0300
>From: Max Lapshin 
>To: Rad Gruchalski 
>Cc: erlang questions 
>Subject: Re: [erlang-questions] SO_REUSEPORT with UDP
>Message-ID:
>	
>Content-Type: text/plain; charset="utf-8"
>
>Wow!
>
>Thanks for this thing!
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: 
>
>------------------------------
>
>Message: 7
>Date: Fri, 22 May 2015 22:42:02 -0700
>From: Kenneth Lakin 
>To: erlang-questions@REDACTED
>Subject: [erlang-questions] Mnesia doesn't report insertion errors
>	when underlying DETS file is full?
>Message-ID: <5560132A.4040107@REDACTED>
>Content-Type: text/plain; charset="utf-8"
>
>Hi, all. Here's hoping that I'm just misunderstanding how things work.
>I've done some searching, but it doesn't seem that anyone has asked this
>question on the list before.
>
>I'm testing with Erlang 17.3 (erts 6.2) on AMD64 Kubuntu 15.04.
>
>I wrote a little test program that uses mnesia:write/1 inside of a
>mnesia:transaction/1 to write a single integer into a Mnesia
>disc_only_copies table. Once the table grows to ~2,097,000 bytes -give
>or take-, the the on-disk table file stops growing, and mnesia:info/0
>reports that the number of records contained in the table remains constant.
>
>It's obvious that *something* is wrong, as attempts to use mnesia:first
>or mnesia:last return {aborted, {badarg, [table]}}. However, the write
>operations wrapped in mnesia:transaction keep returning {atomic,
>Result}, and comparison between a couple of calls of mnesia:info reveals
>that transactions continue to be committed and logged to disk (even
>though no new data *sticks around* on disk). I also tested this with
>mnesia:activity/2. When the DETS table hits 2GB, mnesia:activity keeps
>on trucking: it doesn't throw or exit when called with either
>transaction, or async_dirty.
>
>I would have expected mnesia:transaction to return {aborted, table_full}
>or something, rather than lying that my data was committed. Is this an
>unreasonable expectation? Did the Ubuntu or Debian folks mispackage
>Erlang in some way? Is this an oversight on the part of the OTP folks?
>
>For the curious, my program is at the end of this message. (Run it with
>test/0.)
>
>Thanks,
>-Kenneth Lakin
>%%%%%
>-module(test).
>-compile(export_all).
>-record(rec, {key, val}).
>
>test() ->
>  %Default to signed intmax iterations.
>  test(math:pow(2, 32)/2).
>test(Iters) ->
>  mnesia:create_schema([node()]),
>  mnesia:start(),
>  mnesia:wait_for_tables([test], timer:seconds(30)),
>  mnesia:clear_table(test),
>  mnesia:create_table(test, [{attributes, record_info(fields, rec)},
>                             {type, bag},
>                             {disc_only_copies, [node()]}
>                            ]),
>  insertData(Iters).
>
>insertData(Iters) ->
>  insertData(0, Iters).
>insertData(Step, Iters) when Iters == Step ->
>  ok;
>insertData(Step, Iters) ->
>  F=fun() -> mnesia:write({test, Step, 0}) end,
>  %This doesn't seem to throw or exit when
>  %inserts fail. Trying mnesia:transaction.
>  %mnesia:activity(transaction, F),
>  {atomic, _} = mnesia:transaction(F),
>  insertData(Step+1, Iters).
>
>-------------- next part --------------
>A non-text attachment was scrubbed...
>Name: signature.asc
>Type: application/pgp-signature
>Size: 819 bytes
>Desc: OpenPGP digital signature
>URL: 
>
>------------------------------
>
>Message: 8
>Date: Sat, 23 May 2015 10:20:21 +0200
>From: Ulf Wiger 
>To: Kenneth Lakin 
>Cc: erlang questions 
>Subject: Re: [erlang-questions] Mnesia doesn't report insertion errors
>	when	underlying DETS file is full?
>Message-ID: <730EBF82-9E07-4CBD-A087-A580A2DF18C9@REDACTED>
>Content-Type: text/plain; charset=utf-8
>
>
>> On 23 May 2015, at 07:42, Kenneth Lakin  wrote:
>> 
>> When the DETS table hits 2GB, mnesia:activity keeps
>> on trucking: it doesn't throw or exit when called with either
>> transaction, or async_dirty.
>> 
>> I would have expected mnesia:transaction to return {aborted, table_full}
>> or something, rather than lying that my data was committed. Is this an
>> unreasonable expectation? Did the Ubuntu or Debian folks mispackage
>> Erlang in some way? Is this an oversight on the part of the OTP folks?
>
>This is a known problem. Dets returns an error when the table is full, but this error cannot be handled by mnesia, since it occurs *after* commit - mnesia simply expects the backend store to work.
>
>There are a number of ways to address the issue:
>
>* Use mnesia_frag to fragment disc_only tables. This is no guarantee that you will avoid the issue, but by continuously monitoring the number of objects vs number of fragments, you can minimize the risk. I consider this a pretty ugly solution, but it has been used in commercial systems. One can of course also manually ?fragment? the data, but this has the same issues, and then some.
>
>* Use disc_copies, ensure that you have a big enough box to keep all data in RAM. This has been used to great lengths by some commercial companies. The solution is not that bad, although eventually, you end up with some pretty ridiculous hardware, and pretty long startup times, since all data must be bulk-loaded into memory. One advantage is that reads and match operations become wicked fast.
>
>* Fix dets to support larger data sets. Richard Carlsson once wrote a 64-bit dets version, but it has not been adopted by OTP. One could argue that the underlying dets design is unsuitable for very large data sets, e.g. due to its recovery semantics.
>
>* Use an alternative backend. This has been done a few times, e.g. with ?mnesiaex? [1], but AFAIK, it is nowadays obsolete. Klarna has presented a similar approach, using the eleveldb backend [2], but has not (unless I missed it) made any recent announcement regarding status or availability of this solution.
>
>* Use another database solution. Many projects do this. You will forfeit some of the unique characteristics of mnesia - in particular, the extremely close integration with the runtime environment - but obviously gain other things.
>
>BR,
>Ulf W
>
>[1] https://www.openhub.net/p/mnesiaex
>[2] https://erlangcentral.org/mnesia-backend-plugin-framework-and-a-leveldb-based-plugin/#.VWA08FmeDGc
>
>Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
>http://feuerlabs.com
>
>
>
>
>
>------------------------------
>
>Message: 9
>Date: Sat, 23 May 2015 10:30:31 +0200
>From: Ulf Wiger 
>To: Kenneth Lakin 
>Cc: erlang questions 
>Subject: Re: [erlang-questions] Mnesia doesn't report insertion errors
>	when	underlying DETS file is full?
>Message-ID: 
>Content-Type: text/plain; charset=utf-8
>
>
>> On 23 May 2015, at 10:20, Ulf Wiger  wrote:
>> 
>> * Use an alternative backend. This has been done a few times, e.g. with ?mnesiaex? [1], but AFAIK, it is nowadays obsolete. Klarna has presented a similar approach, using the eleveldb backend [2], but has not (unless I missed it) made any recent announcement regarding status or availability of this solution.
>
>Ah, I did miss that Mikael Pettersson of Klarna will talk about this at the upcoming EUC:
>
>http://www.erlang-factory.com/euc2015/mikael-pettersson
>?mnesia + leveldb: liberating mnesia from the limitations of DETS
>Mnesia offers various database features, but restricts users to a few storage engines with substantial limitations. This talk describes mnesia_ext, an extension which allows arbitrary storage engines to be plugged into mnesia, and how Klarna used this to migrate parts of its database to LevelDB. We will also talk about our experiences with LevelDB, and some improvements we have made.
>
>Talk objectives:
>
>- Present mnesia_ext. Show that LevelDB is a production-grade storage engine for Erlang (with or without mnesia on top).
>
>Target audience:
>
>- Backend developers. Erlang/OTP maintainers.?
>
>BR,
>Ulf W
>
>Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
>http://feuerlabs.com
>
>
>
>
>
>------------------------------
>
>Message: 10
>Date: Sat, 23 May 2015 10:38:56 +0200
>From: ?ric Pailleau 
>To: Ulf Wiger 
>Cc: Erlang Questions 
>Subject: Re: [erlang-questions] Mnesia doesn't report insertion errors
>	when	underlying DETS file is full?
>Message-ID: 
>Content-Type: text/plain; charset=utf-8
>
>Hi,
>Just wanted to know if mnesia subscribe on system events can raise a
>
>{mnesia_error, Format, Args}
>
>In such case ?
>
>Regards
>
>
>Le?23 mai 2015 10:30, Ulf Wiger  a ?crit?:
>>
>>
>> > On 23 May 2015, at 10:20, Ulf Wiger  wrote:
>> > 
>> > * Use an alternative backend. This has been done a few times, e.g. with ?mnesiaex? [1], but AFAIK, it is nowadays obsolete. Klarna has presented a similar approach, using the eleveldb backend [2], but has not (unless I missed it) made any recent announcement regarding status or availability of this solution.
>>
>> Ah, I did miss that Mikael Pettersson of Klarna will talk about this at the upcoming EUC:
>>
>> http://www.erlang-factory.com/euc2015/mikael-pettersson
>> ?mnesia + leveldb: liberating mnesia from the limitations of DETS
>> Mnesia offers various database features, but restricts users to a few storage engines with substantial limitations. This talk describes mnesia_ext, an extension which allows arbitrary storage engines to be plugged into mnesia, and how Klarna used this to migrate parts of its database to LevelDB. We will also talk about our experiences with LevelDB, and some improvements we have made.
>>
>> Talk objectives:
>>
>> - Present mnesia_ext. Show that LevelDB is a production-grade storage engine for Erlang (with or without mnesia on top).
>>
>> Target audience:
>>
>> - Backend developers. Erlang/OTP maintainers.?
>>
>> BR,
>> Ulf W
>>
>> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
>> http://feuerlabs.com
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>
>------------------------------
>
>_______________________________________________
>erlang-questions mailing list
>erlang-questions@REDACTED
>http://erlang.org/mailman/listinfo/erlang-questions
>
>
>End of erlang-questions Digest, Vol 218, Issue 8
>************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From ulf@REDACTED  Sat May 23 18:39:12 2015
From: ulf@REDACTED (Ulf Wiger)
Date: Sat, 23 May 2015 18:39:12 +0200
Subject: [erlang-questions] Mnesia doesn't report insertion errors when
	underlying DETS file is full?
In-Reply-To: <29229e97-d62e-447f-91ba-2b33377db33b@email.android.com>
References: <29229e97-d62e-447f-91ba-2b33377db33b@email.android.com>
Message-ID: 

May I suggest trying out Jobs [1]?

There, you can create a sampler which e.g. checks the size of critical tables, and generates overload events, which can automatically throttle incoming traffic, as well as notify subscribers [2].

[1] https://github.com/uwiger/jobs
[2] https://github.com/uwiger/jobs/blob/master/doc/jobs_sampler.md#subscribe-0

BR,
Ulf W


> On 23 May 2015, at 14:25, ?ric Pailleau  wrote:
> 
> Hi, 
> Thanks Ulf.
> It would be probably more smart to warn before to loose data...
> A system event 'almost_full' should be more convenient. It is sometime better to stop the application instead loose data.
> An event on each last 5 percent change could allow to handle different actions. Mail, SMS, or ultimately stop the application on almost_full_99. 
> There is already an event on database inconsistency, it would be a nice enhancement.
> Regards 
> 
> Le 23 mai 2015 13:02, Ulf Wiger  a ?crit :
>> 
>> 
>>> On 23 May 2015, at 10:38, ?ric Pailleau  wrote: 
>>> 
>>> Just wanted to know if mnesia subscribe on system events can raise a 
>>> 
>>> {mnesia_error, Format, Args} 
>>> 
>>> In such case ? 
>> 
>> It *almost* does, if you set mnesia?s debug level to ?verbose?. ;-) 
>> 
>> The following call is made from mnesia_tm if do_update_op/3 crashes: 
>> 
>>             verbose("do_update in ~w failed: ~p -> {'EXIT', ~p}~n", 
>>                     [Tid, Op, Reason]), 
>> 
>> ?which, in its turn ends up calling the following in mnesia_lib: 
>> 
>> important(Format, Args) -> 
>>     save({Format, Args}), 
>>     report_system_event({mnesia_info, Format, Args}). 
>> 
>> The only problem is that the dets operation in question doesn?t crash, but instead returns {error, Reason}, so that particular issue won?t be reported. The value *is* captured by mnesia_tm:do_update/4, and it will return the last non-ok value from the commit. However, this return value is discarded, except in a few cases where it?s captured as debug info. 
>> 
>> Of course, even if it were reported, it would be of the category ?oops - your database was just corrupted!?, which is good to know, but not so easy to handle. 
>> 
>> BR, 
>> Ulf W 
>> 
>> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. 
>> http://feuerlabs.com 
>> 
>> 
>> 

Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
http://feuerlabs.com





From pablo.platt@REDACTED  Sat May 23 18:50:29 2015
From: pablo.platt@REDACTED (pablo platt)
Date: Sat, 23 May 2015 19:50:29 +0300
Subject: [erlang-questions] SO_REUSEPORT with UDP
In-Reply-To: 
References: 
 
 <9BF17FBFE6E14E96AA96622C1C822C25@gruchalski.com>
 
Message-ID: 

>From the stackoverflow link, SO_REUSEADDR let you use the same source
address when a previous socket is closing (TIME_WAIT).
SO_REUSEPORT let you use the same source address and port as long as the
socket tuple is different.
{, , , , }
On some systems (Linux =< 3.9, Windos) SO_REUSEADDR is the same as the
combination of SO_REUSEADDR and SO_REUSEPORT.

It seems that Linux and BSD use a slightly different raw setting for
SO_REUSEPORT:
https://github.com/aetrion/erl-dns/commit/0c8d768ae69773d2163ea0741125471983b9a57d#diff-06495a7d8430a91558d99be5f1209c32R83
[{raw, 1, 15, <<1:32/native>>}];

How can I build a UDP server that listen on a fixed {Addr, Port} and
creates a new process for a new client connection?
Does this suppose to work?

1. Open a UDP socket and listen on a fixed port with SO_REUSEPORT set.
gen_udp:open(Port, Opts)
2. When we get a UDP packet, connect to the client and create a new process
and pass the existing socket to it.
gen_udp:connect(Socket, Address, Port)
3. goto 1.


On Fri, May 22, 2015 at 8:16 PM, Max Lapshin  wrote:

> Wow!
>
> Thanks for this thing!
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From matthias@REDACTED  Sat May 23 22:33:05 2015
From: matthias@REDACTED (Matthias Lang)
Date: Sat, 23 May 2015 22:33:05 +0200
Subject: [erlang-questions] HEART
In-Reply-To: <5552A20B.7090104@gmail.com>
References: 
 
 
 
 <08E64BBA-163C-4CFD-885C-0F2FC9BAD36B@widetag.com>
 
 <555280BE.7030605@meetinghouse.net> <5552A20B.7090104@gmail.com>
Message-ID: <20150523203304.GA6840@corelatus.se>

On 13. May 2015, Michael L Martin wrote:

> But who watches the watchdog?

Late reply. One approach is to have a hardware watchdog.

On the embedded system I work on, Erlang kicks a custom 'heart'
program.  The heart program kicks the hardware watchdog.

Hangs in Erlang code are dealt with by timeouts and supervisors.
Hangs in the VM are dealt with by 'heart'.
Hangs in 'heart' are dealt with by the hardware watchdog.

This approach is sufficient to make hangs an insignificant contributor
to downtime in a five-nines environment (signalling in the SS7
network), in my experience.

Matt


From bchesneau@REDACTED  Sat May 23 22:46:09 2015
From: bchesneau@REDACTED (Benoit Chesneau)
Date: Sat, 23 May 2015 20:46:09 +0000
Subject: [erlang-questions] SO_REUSEPORT with UDP
In-Reply-To: 
References: 
 
 <9BF17FBFE6E14E96AA96622C1C822C25@gruchalski.com>
 
 
Message-ID: 

On Sat, May 23, 2015 at 6:50 PM pablo platt  wrote:

> From the stackoverflow link, SO_REUSEADDR let you use the same source
> address when a previous socket is closing (TIME_WAIT).
> SO_REUSEPORT let you use the same source address and port as long as the
> socket tuple is different.
> {, , , , }
> On some systems (Linux =< 3.9, Windos) SO_REUSEADDR is the same as the
> combination of SO_REUSEADDR and SO_REUSEPORT.
>
> It seems that Linux and BSD use a slightly different raw setting for
> SO_REUSEPORT:
>
> https://github.com/aetrion/erl-dns/commit/0c8d768ae69773d2163ea0741125471983b9a57d#diff-06495a7d8430a91558d99be5f1209c32R83
> [{raw, 1, 15, <<1:32/native>>}];
>
> How can I build a UDP server that listen on a fixed {Addr, Port} and
> creates a new process for a new client connection?
> Does this suppose to work?
>

Of course it is:
https://github.com/refuge/rbeacon/blob/master/src/rbeacon.erl#L564-L599

- benoit

>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From tony@REDACTED  Sun May 24 10:41:06 2015
From: tony@REDACTED (Tony Rogvall)
Date: Sun, 24 May 2015 16:41:06 +0800
Subject: [erlang-questions] Chengdu, meetup
Message-ID: <078D48CB-1A57-4CCB-9A80-7F600A59DF56@rogvall.se>

Hi!
I will be in Chengdu, China from 25/5 to 4/6. Are there any programmers and/or hackers that want to have a beer or two and chat about tech and Erlang ?
You can respond privatly if you like.

See you!

/Tony



From ok@REDACTED  Mon May 25 03:36:30 2015
From: ok@REDACTED (Richard A. O'Keefe)
Date: Mon, 25 May 2015 13:36:30 +1200
Subject: [erlang-questions] how are implemented lists?
In-Reply-To: <20150522070421.GA4414@erix.ericsson.se>
References: 
 
 <20150522070421.GA4414@erix.ericsson.se>
Message-ID: <9AD1EC29-6622-4857-848A-036C5712FE04@cs.otago.ac.nz>


On 22/05/2015, at 7:04 pm, Raimo Niskanen  wrote:
> 
> 
> Have a look at the brand new map data type.

It's not clear how that would help.
The original request was for

> a fast and concurrent data structure that allows me to retrieve the items
> in order and do pop/tail, while still being abble to remove them by key.

Maps are no more concurrent than lists are.

When requests like this come along, it would be nice to have a
specification that's a bit more precise.  I read this as

typeclass Useful_Key key
  -- at a minimum, support equality, maybe support comparison,
  -- possibly equality and hashing.
data Useful_Key key => Thingy key val

empty :: Thingy key val

add :: key -> val -> Thingy key val -> Thingy key val

remove :: key -> Thingy key val -> Thingy key val

pop :: Thingy key val -> Maybe (key, val, Thingy key val)

so a sort of dictionary/list hybrid.




From bchesneau@REDACTED  Mon May 25 09:14:59 2015
From: bchesneau@REDACTED (Benoit Chesneau)
Date: Mon, 25 May 2015 07:14:59 +0000
Subject: [erlang-questions] how are implemented lists?
In-Reply-To: <9AD1EC29-6622-4857-848A-036C5712FE04@cs.otago.ac.nz>
References: 
 
 <20150522070421.GA4414@erix.ericsson.se>
 <9AD1EC29-6622-4857-848A-036C5712FE04@cs.otago.ac.nz>
Message-ID: 

Richard,

Indeed; sorry for that: So indeed that is this kind of data structure that
I would like to have. Was thinking to use a skip-list data-structure as a
nif but that may be overkill..

Thinking more about it using a process for it may be more efficient than
using an ETS. Especially that I need to be handle many "queue". As I
understand using many ETS tables i not really advised, and I'm worried
about the performance if I start to store multiple queues per ETS table.

- benoit

On Mon, May 25, 2015 at 3:36 AM Richard A. O'Keefe 
wrote:

>
> On 22/05/2015, at 7:04 pm, Raimo Niskanen <
> raimo+erlang-questions@REDACTED> wrote:
> >
> >
> > Have a look at the brand new map data type.
>
> It's not clear how that would help.
> The original request was for
>
> > a fast and concurrent data structure that allows me to retrieve the items
> > in order and do pop/tail, while still being abble to remove them by key.
>
> Maps are no more concurrent than lists are.
>
> When requests like this come along, it would be nice to have a
> specification that's a bit more precise.  I read this as
>
> typeclass Useful_Key key
>   -- at a minimum, support equality, maybe support comparison,
>   -- possibly equality and hashing.
> data Useful_Key key => Thingy key val
>
> empty :: Thingy key val
>
> add :: key -> val -> Thingy key val -> Thingy key val
>
> remove :: key -> Thingy key val -> Thingy key val
>
> pop :: Thingy key val -> Maybe (key, val, Thingy key val)
>
> so a sort of dictionary/list hybrid.
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From rickard@REDACTED  Mon May 25 09:30:37 2015
From: rickard@REDACTED (Rickard Green)
Date: Mon, 25 May 2015 09:30:37 +0200
Subject: [erlang-questions] Erlang/OTP 18.0-rc2 is available for testing
In-Reply-To: 
References: 
 
 
 
 
 <555389EB.5070102@gmail.com>
 
 
 
Message-ID: 

On Fri, May 15, 2015 at 8:18 PM, Steve Vinoski  wrote:
> Yes, the patch you're referring to merged a few weeks ago, but note that
> 18.0-rc2 doesn't work for dirty schedulers as discussed earlier in the
> thread due to an issue related to timer wheel changes. Rickard and I will
> have a fix soon.
>
> --steve
>
> On Fri, May 15, 2015 at 1:03 PM, Max Lapshin  wrote:
>>
>> Steve, are your dirty nif patches included?
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>

There is now a fix (b7c9657) for this in the master branch. Thanks to
Steve Vinoski!

Regards,
Rickard
-- 
Rickard Green, Erlang/OTP, Ericsson AB


From vychodil.hynek@REDACTED  Mon May 25 17:11:34 2015
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Mon, 25 May 2015 17:11:34 +0200
Subject: [erlang-questions] PropEr after test clean up
Message-ID: 

Hi,

I bumped in the problem how clean up after property in PropEr. Let's have
simple property where I make ets table:

prop_ets() ->
    ?FORALL(V, integer(),
        begin
            E = ets:new(),
            true = ets:insert(T, {V, ok}),
            equals([{V, ok}], ets:lookup(T, V))
        end
    ).

How am I supposed to delete ets table? It is trickier than looks like. The
problem is when I want use another ?FORALL inside my property. The
conjunction/1 is the same problem. You can`t write

prop_ets() ->
    ?FORALL(V, integer(),
        begin
            E = ets:new(),
            true = ets:insert(T, {V, ok}),
            Res = conjunction([{lookup, equals([{V, ok}], ets:lookup(T,
V))},
                              {lookup_element,
equals(ok, ets:lookup_element(T, V, 2))}]),
            ets:delete(T),
            Res
        end
    ).

Because Res is test(). In this case, you can make calls to the ets:lookup/2
and the ets:lookup_element/3 before conjunction/1 but it doesn't solve
properties like

?FORALL(L, list(...),
    begin
        T = ets:new(),
        ...
        ?FORALL(V, oneof(L),
              ...
        )
     end
)

The solution would be simple is every test case would be run in a separate
process, but it is not! It is very unusual in Erlang word to make such
thing. Processes are cheap and I can they are in a defined state each run
for free. Why is it all running in the same process?

Has anybody solved this problem somehouw?

Hynek Vychodil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From trapexit@REDACTED  Mon May 25 17:28:47 2015
From: trapexit@REDACTED (Antonio SJ Musumeci)
Date: Mon, 25 May 2015 11:28:47 -0400
Subject: [erlang-questions] atoms in iodata() / iolist()
Message-ID: 

Does anyone know of any backwards incompatibility which could arise if
atoms were added to the iodata() and iolist() definitions?

iolist() = maybe_improper_list(byte() | binary() | atom() | iolist(),
binary() | [])
iodata() = iolist() | atom() | binary()
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From sean@REDACTED  Mon May 25 18:00:01 2015
From: sean@REDACTED (Functional Jobs)
Date: Mon, 25 May 2015 12:00:01 -0400
Subject: [erlang-questions] New Functional Programming Job Opportunities
Message-ID: <55634703b2e6a@functionaljobs.com>

Here are some functional programming job opportunities that were posted
recently:

Clojure/Erlang backend engineer/architect at Zoomo
http://functionaljobs.com/jobs/8831-clojure-erlang-backend-engineer-architect-at-zoomo

Cheers,
Sean Murphy
FunctionalJobs.com



From eriksoe@REDACTED  Mon May 25 18:34:40 2015
From: eriksoe@REDACTED (=?UTF-8?Q?Erik_S=C3=B8e_S=C3=B8rensen?=)
Date: Mon, 25 May 2015 18:34:40 +0200
Subject: [erlang-questions] PropEr after test clean up
In-Reply-To: 
References: 
Message-ID: 

Would a try...after...end be any help?
Den 25/05/2015 17.11 skrev "Hynek Vychodil" :

> Hi,
>
> I bumped in the problem how clean up after property in PropEr. Let's have
> simple property where I make ets table:
>
> prop_ets() ->
>     ?FORALL(V, integer(),
>         begin
>             E = ets:new(),
>             true = ets:insert(T, {V, ok}),
>             equals([{V, ok}], ets:lookup(T, V))
>         end
>     ).
>
> How am I supposed to delete ets table? It is trickier than looks like. The
> problem is when I want use another ?FORALL inside my property. The
> conjunction/1 is the same problem. You can`t write
>
> prop_ets() ->
>     ?FORALL(V, integer(),
>         begin
>             E = ets:new(),
>             true = ets:insert(T, {V, ok}),
>             Res = conjunction([{lookup, equals([{V, ok}], ets:lookup(T,
> V))},
>                               {lookup_element,
> equals(ok, ets:lookup_element(T, V, 2))}]),
>             ets:delete(T),
>             Res
>         end
>     ).
>
> Because Res is test(). In this case, you can make calls to
> the ets:lookup/2 and the ets:lookup_element/3 before conjunction/1 but it
> doesn't solve properties like
>
> ?FORALL(L, list(...),
>     begin
>         T = ets:new(),
>         ...
>         ?FORALL(V, oneof(L),
>               ...
>         )
>      end
> )
>
> The solution would be simple is every test case would be run in a separate
> process, but it is not! It is very unusual in Erlang word to make such
> thing. Processes are cheap and I can they are in a defined state each run
> for free. Why is it all running in the same process?
>
> Has anybody solved this problem somehouw?
>
> Hynek Vychodil
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From vychodil.hynek@REDACTED  Mon May 25 18:40:32 2015
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Mon, 25 May 2015 18:40:32 +0200
Subject: [erlang-questions] PropEr after test clean up
In-Reply-To: 
References: 
 
Message-ID: 

It is exactly how I discovered this behaviour. See
https://github.com/pichi/erlgt/blob/ff3249893464f19765c74db2eae24273f15f66fd/src/gen_digraph.erl#L153

When I implemented otp_digraph as a wrapper for digraph module from stdlib
I realised it doesn't work.

On Mon, May 25, 2015 at 6:34 PM, Erik S?e S?rensen 
wrote:

> Would a try...after...end be any help?
> Den 25/05/2015 17.11 skrev "Hynek Vychodil" :
>
>> Hi,
>>
>> I bumped in the problem how clean up after property in PropEr. Let's have
>> simple property where I make ets table:
>>
>> prop_ets() ->
>>     ?FORALL(V, integer(),
>>         begin
>>             E = ets:new(),
>>             true = ets:insert(T, {V, ok}),
>>             equals([{V, ok}], ets:lookup(T, V))
>>         end
>>     ).
>>
>> How am I supposed to delete ets table? It is trickier than looks like.
>> The problem is when I want use another ?FORALL inside my property. The
>> conjunction/1 is the same problem. You can`t write
>>
>> prop_ets() ->
>>     ?FORALL(V, integer(),
>>         begin
>>             E = ets:new(),
>>             true = ets:insert(T, {V, ok}),
>>             Res = conjunction([{lookup, equals([{V, ok}], ets:lookup(T,
>> V))},
>>                               {lookup_element,
>> equals(ok, ets:lookup_element(T, V, 2))}]),
>>             ets:delete(T),
>>             Res
>>         end
>>     ).
>>
>> Because Res is test(). In this case, you can make calls to
>> the ets:lookup/2 and the ets:lookup_element/3 before conjunction/1 but it
>> doesn't solve properties like
>>
>> ?FORALL(L, list(...),
>>     begin
>>         T = ets:new(),
>>         ...
>>         ?FORALL(V, oneof(L),
>>               ...
>>         )
>>      end
>> )
>>
>> The solution would be simple is every test case would be run in a
>> separate process, but it is not! It is very unusual in Erlang word to make
>> such thing. Processes are cheap and I can they are in a defined state each
>> run for free. Why is it all running in the same process?
>>
>> Has anybody solved this problem somehouw?
>>
>> Hynek Vychodil
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From mononcqc@REDACTED  Mon May 25 18:43:51 2015
From: mononcqc@REDACTED (Fred Hebert)
Date: Mon, 25 May 2015 12:43:51 -0400
Subject: [erlang-questions] atoms in iodata() / iolist()
In-Reply-To: 
References: 
Message-ID: <20150525164349.GB75968@ferdair.local>

On 05/25, Antonio SJ Musumeci wrote:
>Does anyone know of any backwards incompatibility which could arise if
>atoms were added to the iodata() and iolist() definitions?
>
>iolist() = maybe_improper_list(byte() | binary() | atom() | iolist(),
>binary() | [])
>iodata() = iolist() | atom() | binary()

Why would you do that?

But yes, there could be incompatibilities. For example, the `iodata()` 
type is defined in terms of `iolist()`, and operations such as 
`erlang:port_control()` return `iodata()` as a type. As such, operations 
to port programs could now technically start returning atoms to code 
that won't expect such things.


From vychodil.hynek@REDACTED  Mon May 25 18:45:08 2015
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Mon, 25 May 2015 18:45:08 +0200
Subject: [erlang-questions] PropEr after test clean up
In-Reply-To: 
References: 
 
 
Message-ID: 

There is commit showing how ugly is resulting code
https://github.com/pichi/erlgt/commit/47aa088658ca6ca1e39e9630cd188f46e685e27a

On Mon, May 25, 2015 at 6:40 PM, Hynek Vychodil 
wrote:

> It is exactly how I discovered this behaviour. See
> https://github.com/pichi/erlgt/blob/ff3249893464f19765c74db2eae24273f15f66fd/src/gen_digraph.erl#L153
>
> When I implemented otp_digraph as a wrapper for digraph module from stdlib
> I realised it doesn't work.
>
> On Mon, May 25, 2015 at 6:34 PM, Erik S?e S?rensen 
> wrote:
>
>> Would a try...after...end be any help?
>> Den 25/05/2015 17.11 skrev "Hynek Vychodil" :
>>
>>> Hi,
>>>
>>> I bumped in the problem how clean up after property in PropEr. Let's
>>> have simple property where I make ets table:
>>>
>>> prop_ets() ->
>>>     ?FORALL(V, integer(),
>>>         begin
>>>             E = ets:new(),
>>>             true = ets:insert(T, {V, ok}),
>>>             equals([{V, ok}], ets:lookup(T, V))
>>>         end
>>>     ).
>>>
>>> How am I supposed to delete ets table? It is trickier than looks like.
>>> The problem is when I want use another ?FORALL inside my property. The
>>> conjunction/1 is the same problem. You can`t write
>>>
>>> prop_ets() ->
>>>     ?FORALL(V, integer(),
>>>         begin
>>>             E = ets:new(),
>>>             true = ets:insert(T, {V, ok}),
>>>             Res = conjunction([{lookup, equals([{V, ok}], ets:lookup(T,
>>> V))},
>>>                               {lookup_element,
>>> equals(ok, ets:lookup_element(T, V, 2))}]),
>>>             ets:delete(T),
>>>             Res
>>>         end
>>>     ).
>>>
>>> Because Res is test(). In this case, you can make calls to
>>> the ets:lookup/2 and the ets:lookup_element/3 before conjunction/1 but it
>>> doesn't solve properties like
>>>
>>> ?FORALL(L, list(...),
>>>     begin
>>>         T = ets:new(),
>>>         ...
>>>         ?FORALL(V, oneof(L),
>>>               ...
>>>         )
>>>      end
>>> )
>>>
>>> The solution would be simple is every test case would be run in a
>>> separate process, but it is not! It is very unusual in Erlang word to make
>>> such thing. Processes are cheap and I can they are in a defined state each
>>> run for free. Why is it all running in the same process?
>>>
>>> Has anybody solved this problem somehouw?
>>>
>>> Hynek Vychodil
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From trapexit@REDACTED  Mon May 25 18:59:06 2015
From: trapexit@REDACTED (Antonio SJ Musumeci)
Date: Mon, 25 May 2015 12:59:06 -0400
Subject: [erlang-questions] atoms in iodata() / iolist()
In-Reply-To: <20150525164349.GB75968@ferdair.local>
References: 
 <20150525164349.GB75968@ferdair.local>
Message-ID: 

There are situations where atoms are mixed with strings such as filenames
and it'd be nice to not need to flatten and convert them. There are other
situations where knowing an atom would be converted to it's string form
would be more efficient and convenient than the alternative.
On May 25, 2015 12:44 PM, "Fred Hebert"  wrote:

> On 05/25, Antonio SJ Musumeci wrote:
>
>> Does anyone know of any backwards incompatibility which could arise if
>> atoms were added to the iodata() and iolist() definitions?
>>
>> iolist() = maybe_improper_list(byte() | binary() | atom() | iolist(),
>> binary() | [])
>> iodata() = iolist() | atom() | binary()
>>
>
> Why would you do that?
>
> But yes, there could be incompatibilities. For example, the `iodata()`
> type is defined in terms of `iolist()`, and operations such as
> `erlang:port_control()` return `iodata()` as a type. As such, operations to
> port programs could now technically start returning atoms to code that
> won't expect such things.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From carlsson.richard@REDACTED  Mon May 25 19:16:35 2015
From: carlsson.richard@REDACTED (Richard Carlsson)
Date: Mon, 25 May 2015 19:16:35 +0200
Subject: [erlang-questions] atoms in iodata() / iolist()
In-Reply-To: 
References: 
 <20150525164349.GB75968@ferdair.local>
 
Message-ID: 

Yes, I think it was a mistake not to include atoms in iolists to begin with
(funnily, filenames can be deep and include atoms, but iolists can't). It
might be pretty hard to change though, since there is so much existing code
that might crash if it stumbled over an atom in a list that was produced by
more modern code.

Several functions in the standard libraries become needlessly expensive
because of this need to expand atoms before they can be included in
iolists, even when the result is just going to be sent directly to a file
or port.


        /Richard

On Mon, May 25, 2015 at 6:59 PM, Antonio SJ Musumeci 
wrote:

> There are situations where atoms are mixed with strings such as filenames
> and it'd be nice to not need to flatten and convert them. There are other
> situations where knowing an atom would be converted to it's string form
> would be more efficient and convenient than the alternative.
> On May 25, 2015 12:44 PM, "Fred Hebert"  wrote:
>
>> On 05/25, Antonio SJ Musumeci wrote:
>>
>>> Does anyone know of any backwards incompatibility which could arise if
>>> atoms were added to the iodata() and iolist() definitions?
>>>
>>> iolist() = maybe_improper_list(byte() | binary() | atom() | iolist(),
>>> binary() | [])
>>> iodata() = iolist() | atom() | binary()
>>>
>>
>> Why would you do that?
>>
>> But yes, there could be incompatibilities. For example, the `iodata()`
>> type is defined in terms of `iolist()`, and operations such as
>> `erlang:port_control()` return `iodata()` as a type. As such, operations to
>> port programs could now technically start returning atoms to code that
>> won't expect such things.
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From vychodil.hynek@REDACTED  Mon May 25 19:23:03 2015
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Mon, 25 May 2015 19:23:03 +0200
Subject: [erlang-questions] PropEr after test clean up
In-Reply-To: 
References: 
 
 
 
Message-ID: 

Well, this
https://github.com/pichi/erlgt/commit/d470433dfb8859eaa05381eeba74511a2ff4f6e5
is a little bit better but not a general solution at all. It is fragile and
complicated.

On Mon, May 25, 2015 at 6:45 PM, Hynek Vychodil 
wrote:

> There is commit showing how ugly is resulting code
> https://github.com/pichi/erlgt/commit/47aa088658ca6ca1e39e9630cd188f46e685e27a
>
> On Mon, May 25, 2015 at 6:40 PM, Hynek Vychodil 
> wrote:
>
>> It is exactly how I discovered this behaviour. See
>> https://github.com/pichi/erlgt/blob/ff3249893464f19765c74db2eae24273f15f66fd/src/gen_digraph.erl#L153
>>
>> When I implemented otp_digraph as a wrapper for digraph module
>> from stdlib I realised it doesn't work.
>>
>> On Mon, May 25, 2015 at 6:34 PM, Erik S?e S?rensen 
>> wrote:
>>
>>> Would a try...after...end be any help?
>>> Den 25/05/2015 17.11 skrev "Hynek Vychodil" :
>>>
>>>> Hi,
>>>>
>>>> I bumped in the problem how clean up after property in PropEr. Let's
>>>> have simple property where I make ets table:
>>>>
>>>> prop_ets() ->
>>>>     ?FORALL(V, integer(),
>>>>         begin
>>>>             E = ets:new(),
>>>>             true = ets:insert(T, {V, ok}),
>>>>             equals([{V, ok}], ets:lookup(T, V))
>>>>         end
>>>>     ).
>>>>
>>>> How am I supposed to delete ets table? It is trickier than looks like.
>>>> The problem is when I want use another ?FORALL inside my property. The
>>>> conjunction/1 is the same problem. You can`t write
>>>>
>>>> prop_ets() ->
>>>>     ?FORALL(V, integer(),
>>>>         begin
>>>>             E = ets:new(),
>>>>             true = ets:insert(T, {V, ok}),
>>>>             Res = conjunction([{lookup, equals([{V, ok}], ets:lookup(T,
>>>> V))},
>>>>                               {lookup_element,
>>>> equals(ok, ets:lookup_element(T, V, 2))}]),
>>>>             ets:delete(T),
>>>>             Res
>>>>         end
>>>>     ).
>>>>
>>>> Because Res is test(). In this case, you can make calls to
>>>> the ets:lookup/2 and the ets:lookup_element/3 before conjunction/1 but it
>>>> doesn't solve properties like
>>>>
>>>> ?FORALL(L, list(...),
>>>>     begin
>>>>         T = ets:new(),
>>>>         ...
>>>>         ?FORALL(V, oneof(L),
>>>>               ...
>>>>         )
>>>>      end
>>>> )
>>>>
>>>> The solution would be simple is every test case would be run in a
>>>> separate process, but it is not! It is very unusual in Erlang word to make
>>>> such thing. Processes are cheap and I can they are in a defined state each
>>>> run for free. Why is it all running in the same process?
>>>>
>>>> Has anybody solved this problem somehouw?
>>>>
>>>> Hynek Vychodil
>>>>
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>
>>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From mononcqc@REDACTED  Mon May 25 20:07:08 2015
From: mononcqc@REDACTED (Fred Hebert)
Date: Mon, 25 May 2015 14:07:08 -0400
Subject: [erlang-questions] atoms in iodata() / iolist()
In-Reply-To: 
References: 
 <20150525164349.GB75968@ferdair.local>
 
 
Message-ID: <20150525180706.GC75968@ferdair.local>

On 05/25, Richard Carlsson wrote:
>Yes, I think it was a mistake not to include atoms in iolists to begin with
>(funnily, filenames can be deep and include atoms, but iolists can't). It
>might be pretty hard to change though, since there is so much existing code
>that might crash if it stumbled over an atom in a list that was produced by
>more modern code.
>
>Several functions in the standard libraries become needlessly expensive
>because of this need to expand atoms before they can be included in
>iolists, even when the result is just going to be sent directly to a file
>or port.
>
I would have expected it more reasonable to remove atoms from these than 
to add them everywhere.


From rtrlists@REDACTED  Mon May 25 23:27:50 2015
From: rtrlists@REDACTED (Robert Raschke)
Date: Mon, 25 May 2015 22:27:50 +0100
Subject: [erlang-questions] PropEr after test clean up
In-Reply-To: 
References: 
Message-ID: 

Hello Hynek,

since you are testing something that has state, I think you probably need
to look into the state based testing approach that is possible with PropEr.
It's been a while since I looked into that, so I don't have a handy link at
hand, sorry :-(

Hope this helps a bit,
Robby
On May 25, 2015 4:11 PM, "Hynek Vychodil"  wrote:

> Hi,
>
> I bumped in the problem how clean up after property in PropEr. Let's have
> simple property where I make ets table:
>
> prop_ets() ->
>     ?FORALL(V, integer(),
>         begin
>             E = ets:new(),
>             true = ets:insert(T, {V, ok}),
>             equals([{V, ok}], ets:lookup(T, V))
>         end
>     ).
>
> How am I supposed to delete ets table? It is trickier than looks like. The
> problem is when I want use another ?FORALL inside my property. The
> conjunction/1 is the same problem. You can`t write
>
> prop_ets() ->
>     ?FORALL(V, integer(),
>         begin
>             E = ets:new(),
>             true = ets:insert(T, {V, ok}),
>             Res = conjunction([{lookup, equals([{V, ok}], ets:lookup(T,
> V))},
>                               {lookup_element,
> equals(ok, ets:lookup_element(T, V, 2))}]),
>             ets:delete(T),
>             Res
>         end
>     ).
>
> Because Res is test(). In this case, you can make calls to
> the ets:lookup/2 and the ets:lookup_element/3 before conjunction/1 but it
> doesn't solve properties like
>
> ?FORALL(L, list(...),
>     begin
>         T = ets:new(),
>         ...
>         ?FORALL(V, oneof(L),
>               ...
>         )
>      end
> )
>
> The solution would be simple is every test case would be run in a separate
> process, but it is not! It is very unusual in Erlang word to make such
> thing. Processes are cheap and I can they are in a defined state each run
> for free. Why is it all running in the same process?
>
> Has anybody solved this problem somehouw?
>
> Hynek Vychodil
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From vychodil.hynek@REDACTED  Tue May 26 00:10:51 2015
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Tue, 26 May 2015 00:10:51 +0200
Subject: [erlang-questions] PropEr after test clean up
In-Reply-To: 
References: 
 
Message-ID: 

Hi Robert,

I have used proper_statem and it has the same issue. You are not able to
make reliable clean up because there is zero support for it. Statefullness
has nothing with it. If there would be some hook or even if it does not run
all in one process. For example, property fragment from my
proper_statem module:

prop_xad_cpu() ->
    ?FORALL(Cmds, commands(?MODULE),
            ?TRAPEXIT(
            begin
                {ok, Pid1} = xad_cpu:start_link(),
                {ok, Pid2} = ?MODULE:start_link(),
                Mrefs = [monitor(process, X) || X <- [Pid1, Pid2]],
                Free = get_schedulers(),
                {History, State, Result} = run_commands(?MODULE, Cmds),
                Msgs = collect_messages(collect_processes(History)),
                timer:sleep(1),
                Free2 = get_schedulers(),
                xad_cpu:stop(),
                [receive {'DOWN', X, process, _, _} -> ok end || X <-
Mrefs],
                ?WHENFAIL(
                    io:format(
                        "History: ~p\nState: ~w\nResult: ~w\nMessages: ~w\n"
                        "Free: ~p =?= ~p~n",
                        [pp_history(Cmds, History), pp_state(State), Result,
                         Msgs, Free, Free2]),
                          aggregate(command_names(Cmds), Result =:= ok
                                    andalso Free =:= Free2))
            end
            )).

You can see, you have to make do all cleanup inside property but explicitly
before test() type itself as ?WHENFAIL in this case. It is defacto same
solution as I have used in
https://github.com/pichi/erlgt/commit/d470433dfb8859eaa05381eeba74511a2ff4f6e5
It doesn't allow me make any abstraction when writing property. See how I
had to move ?FORALL outside of ?WITH_G macro. If you would be able to make
something like proper:add_cleanup(fun()->ok) or if each test will run in
separate process, ets would clean up itself. It is erlang idiom: make
process and catch its exit.

On Mon, May 25, 2015 at 11:27 PM, Robert Raschke 
wrote:

> Hello Hynek,
>
> since you are testing something that has state, I think you probably need
> to look into the state based testing approach that is possible with PropEr.
> It's been a while since I looked into that, so I don't have a handy link at
> hand, sorry :-(
>
> Hope this helps a bit,
> Robby
> On May 25, 2015 4:11 PM, "Hynek Vychodil" 
> wrote:
>
>> Hi,
>>
>> I bumped in the problem how clean up after property in PropEr. Let's have
>> simple property where I make ets table:
>>
>> prop_ets() ->
>>     ?FORALL(V, integer(),
>>         begin
>>             E = ets:new(),
>>             true = ets:insert(T, {V, ok}),
>>             equals([{V, ok}], ets:lookup(T, V))
>>         end
>>     ).
>>
>> How am I supposed to delete ets table? It is trickier than looks like.
>> The problem is when I want use another ?FORALL inside my property. The
>> conjunction/1 is the same problem. You can`t write
>>
>> prop_ets() ->
>>     ?FORALL(V, integer(),
>>         begin
>>             E = ets:new(),
>>             true = ets:insert(T, {V, ok}),
>>             Res = conjunction([{lookup, equals([{V, ok}], ets:lookup(T,
>> V))},
>>                               {lookup_element,
>> equals(ok, ets:lookup_element(T, V, 2))}]),
>>             ets:delete(T),
>>             Res
>>         end
>>     ).
>>
>> Because Res is test(). In this case, you can make calls to
>> the ets:lookup/2 and the ets:lookup_element/3 before conjunction/1 but it
>> doesn't solve properties like
>>
>> ?FORALL(L, list(...),
>>     begin
>>         T = ets:new(),
>>         ...
>>         ?FORALL(V, oneof(L),
>>               ...
>>         )
>>      end
>> )
>>
>> The solution would be simple is every test case would be run in a
>> separate process, but it is not! It is very unusual in Erlang word to make
>> such thing. Processes are cheap and I can they are in a defined state each
>> run for free. Why is it all running in the same process?
>>
>> Has anybody solved this problem somehouw?
>>
>> Hynek Vychodil
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From trapexit@REDACTED  Tue May 26 00:53:48 2015
From: trapexit@REDACTED (Antonio SJ Musumeci)
Date: Mon, 25 May 2015 18:53:48 -0400
Subject: [erlang-questions] atoms in iodata() / iolist()
In-Reply-To: 
References: 
 <20150525164349.GB75968@ferdair.local>
 
 
 <20150525180706.GC75968@ferdair.local>
 
Message-ID: 

Likely though it'd probably be nicer to add atoms to the definition or
create a new type.

I created a patch for iolist_to_binary and iolist_size and was looking at
ports and files next but was wondering how practical it was more generally.
On May 25, 2015 14:07, "Fred Hebert"  wrote:

On 05/25, Richard Carlsson wrote:

> Yes, I think it was a mistake not to include atoms in iolists to begin with
> (funnily, filenames can be deep and include atoms, but iolists can't). It
> might be pretty hard to change though, since there is so much existing code
> that might crash if it stumbled over an atom in a list that was produced by
> more modern code.
>
> Several functions in the standard libraries become needlessly expensive
> because of this need to expand atoms before they can be included in
> iolists, even when the result is just going to be sent directly to a file
> or port.
>
>  I would have expected it more reasonable to remove atoms from these than
to add them everywhere.

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://erlang.org/mailman/listinfo/erlang-questions
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From kostis@REDACTED  Tue May 26 01:44:16 2015
From: kostis@REDACTED (Kostis Sagonas)
Date: Tue, 26 May 2015 01:44:16 +0200
Subject: [erlang-questions] PropEr after test clean up
In-Reply-To: 
References: 
Message-ID: <5563B3D0.9080006@cs.ntua.gr>

On 05/25/2015 05:11 PM, Hynek Vychodil wrote:
>
> I bumped in the problem how clean up after property in PropEr. Let's
> have simple property where I make ets table:
>
> prop_ets() ->
>      ?FORALL(V, integer(),
>          begin
>              E = ets:new(),
>              true = ets:insert(T, {V, ok}),
>              equals([{V, ok}], ets:lookup(T, V))
>          end
>      ).
>
> How am I supposed to delete ets table? It is trickier than looks like.
> ... 

I am not sure I understand what it is exactly that you are finding 
difficult to do in the simple example above.  Here is how:

=====================================================================
-module(pets).
-include_lib("proper/include/proper.hrl").

-export_type([key/0, val/0]). % shut off compiler warning

-type key() :: a | b | c.
-type val() :: 0..42.

prop_ets() ->
   ?FORALL({K,V}, {key(),val()},
	  begin
	      T = setup(),
	      true = ets:insert(T, {K, V}),
	      Res = ets:lookup(T, K),
	      cleanup(T),
	      [{K, V}] =:= Res
	  end).

setup() ->
    ets:new(?MODULE, [public, named_table]).

cleanup(T) ->
    ets:delete(T).
========================================================================

The pattern of doing a setup, running some commands and getting some 
Result, doing the cleanup you want to do, and the check that the result 
is the one that you expect is a general pattern in property-based testing.

As you can see, the above works:

kostis@REDACTED:~/proper$ erl -pa ~/proper
Erlang/OTP 17 [erts-6.4.1] [source] [64-bit] [smp:8:8] 
[async-threads:10] [hipe] [kernel-poll:false]

Eshell V6.4.1  (abort with ^G)
1> c(pets, [debug_info]).
{ok,pets}
2> proper:quickcheck(pets:prop_ets()).
....................................................................................................
OK: Passed 100 test(s).
true


Change the property to one that does not hold (e.g. [{K, V}] =/= Res) 
and you will also see that shrinking works.


I am not sure I follow what you mean by your nested ?FORALL properties 
but note that, as shown above, you can easily imitate multiple ?FORALLs 
by using complex terms (tuples, lists, etc.).

Hope this helps,
Kostis


From sdl.web@REDACTED  Tue May 26 07:24:48 2015
From: sdl.web@REDACTED (Leo Liu)
Date: Tue, 26 May 2015 13:24:48 +0800
Subject: [erlang-questions] how search and search_quit are handled by
	user_drv?
References: 
Message-ID: 

On 2015-05-21 00:34 +0800, Leo Liu wrote:
> 1.
> (devel@REDACTED)3> |
>
>     {trace,<0.28.0>,'receive',{#Port<0.349>,{data,[18]}}}
>     {trace,<0.28.0>,'receive', {<0.30.0>,{requests,[{delete_chars,0},{delete_chars,-20}]}}}
>     {trace,<0.28.0>,'receive', {<0.30.0>,{requests,[{put_chars,unicode,"(search)`': "}]}}}
>     {trace,<0.28.0>,'receive',{<0.30.0>,beep}}
>
> 2.
> (search)`': |
>
>     {trace,<0.28.0>,'receive',{#Port<0.349>,{data,"\t"}}}
>     {trace,<0.28.0>,'receive', {<0.30.0>,{requests,[{move_rel,-3},{put_chars,unicode,"\n"}]}}}
>
> 3.
> (search)`
> |': 
>
>     {trace,<0.28.0>,'receive', {<0.30.0>,{requests,[{delete_chars,0},{delete_chars,-12}]}}}
>
> 4.
> (search)`
> |': 
>
>     {trace,<0.28.0>,'receive', {<0.30.0>,
>                                 {requests,[{put_chars,unicode,"(devel@REDACTED)3> "},
>                                 {put_chars,unicode,[]},
>                                 {move_rel,0}]}}}
> 5.
> (search)`
> (devel@REDACTED)3> |

What's wrong with my interpretation of the edlin.erl messages? Could
someone shed some light on it? Thanks.

Leo



From knutin@REDACTED  Tue May 26 10:01:16 2015
From: knutin@REDACTED (Knut Nesheim)
Date: Tue, 26 May 2015 10:01:16 +0200
Subject: [erlang-questions] Using maps from NIFs
Message-ID: 

Hey,

I would like to work with maps from within NIFs. Looking around the
documentation (also for 18-rc2), I can't find anything about such an
API. Did I miss it? If not, are there any plans for introducing an API
usable from NIFs?

Regards
Knut


From sergej.jurecko@REDACTED  Tue May 26 10:13:58 2015
From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=)
Date: Tue, 26 May 2015 10:13:58 +0200
Subject: [erlang-questions] Using maps from NIFs
In-Reply-To: 
References: 
Message-ID: 

It's not in the documentation yet, but there is an api if you look at
erl_nif.h

Check for functions with _map in the name.
https://github.com/erlang/otp/blob/743ed31108ee555db18d9833186865e85e34333e/erts/emulator/beam/erl_nif_api_funcs.h


Sergej

On Tue, May 26, 2015 at 10:01 AM, Knut Nesheim  wrote:

> Hey,
>
> I would like to work with maps from within NIFs. Looking around the
> documentation (also for 18-rc2), I can't find anything about such an
> API. Did I miss it? If not, are there any plans for introducing an API
> usable from NIFs?
>
> Regards
> Knut
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From knutin@REDACTED  Tue May 26 10:29:27 2015
From: knutin@REDACTED (Knut Nesheim)
Date: Tue, 26 May 2015 10:29:27 +0200
Subject: [erlang-questions] Using maps from NIFs
In-Reply-To: 
References: 
 
Message-ID: 

Thanks! :-)

On Tue, May 26, 2015 at 10:13 AM, Sergej Jure?ko
 wrote:
> It's not in the documentation yet, but there is an api if you look at
> erl_nif.h
>
> Check for functions with _map in the name.
> https://github.com/erlang/otp/blob/743ed31108ee555db18d9833186865e85e34333e/erts/emulator/beam/erl_nif_api_funcs.h
>
>
> Sergej
>
> On Tue, May 26, 2015 at 10:01 AM, Knut Nesheim  wrote:
>>
>> Hey,
>>
>> I would like to work with maps from within NIFs. Looking around the
>> documentation (also for 18-rc2), I can't find anything about such an
>> API. Did I miss it? If not, are there any plans for introducing an API
>> usable from NIFs?
>>
>> Regards
>> Knut
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>
>


From vychodil.hynek@REDACTED  Tue May 26 11:34:41 2015
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Tue, 26 May 2015 11:34:41 +0200
Subject: [erlang-questions] PropEr after test clean up
In-Reply-To: <5563B3D0.9080006@cs.ntua.gr>
References: 
 <5563B3D0.9080006@cs.ntua.gr>
Message-ID: 

I think you are missing the point. Let's demonstrate the problem in this
way. There is natural but not working way how to do it

-define(WITH_G(L, Do),
        begin
            G = Module:from_edgelist(L),
            try Do after delete(G)
            end
        end
       ).

-define(WITH_G(L, DoEmpty, DoOther),
        ?WITH_G(L,
                case L of
                    [] -> DoEmpty;
                    _ -> DoOther
                end
               )
       ).

prop_vertices(Module) ->
    ?FORALL(
       L, digraph(),
       ?WITH_G(
          L, equals([], vertices(G)),
          ?FORALL(
             {V1, V2}, oneof(L),
             conjunction(
               [{source,     lists:member(V1, vertices(G))},
                {sink,       lists:member(V2, vertices(G))},
                {in_sources, lists:member(V1, sources(G))},
                {in_sinks,   lists:member(V2, sinks(G))}
               ]
              )
            )
         )
      ).

And how to make it work:

prop_vertices(Module) ->
    ?FORALL(
       L, digraph(),
       case L of
           [] -> ?WITH_G(L, equals([], vertices(G)));
           _  ->
               ?FORALL(
                  {V1, V2}, oneof(L),
                  ?WITH_G(L,
                          begin
                              Vs  = vertices(G),
                              Ses = sources(G),
                              Sks = sinks(G),
                              conjunction(
                                [{source,     lists:member(V1, Vs)},
                                 {sink,       lists:member(V2, Vs)},
                                 {in_sources, lists:member(V1, Ses)},
                                 {in_sinks,   lists:member(V2, Sks)}
                                ]
                               )
                          end
                         )
                 )
       end
      ).

There is the problem. There is no way you can write ?WITH_G macro working
with any of PropEr macros as a parameter. And it would have a simple
solution. Just spawn the process or add hooks for clean up.

I know it is possible to write it in a way it will work, but it prevents to
write it reliably and in structured way.

On Tue, May 26, 2015 at 1:44 AM, Kostis Sagonas  wrote:

> On 05/25/2015 05:11 PM, Hynek Vychodil wrote:
>
>>
>> I bumped in the problem how clean up after property in PropEr. Let's
>> have simple property where I make ets table:
>>
>> prop_ets() ->
>>      ?FORALL(V, integer(),
>>          begin
>>              E = ets:new(),
>>              true = ets:insert(T, {V, ok}),
>>              equals([{V, ok}], ets:lookup(T, V))
>>          end
>>      ).
>>
>> How am I supposed to delete ets table? It is trickier than looks like.
>> ... 
>>
>
> I am not sure I understand what it is exactly that you are finding
> difficult to do in the simple example above.  Here is how:
>
> =====================================================================
> -module(pets).
> -include_lib("proper/include/proper.hrl").
>
> -export_type([key/0, val/0]). % shut off compiler warning
>
> -type key() :: a | b | c.
> -type val() :: 0..42.
>
> prop_ets() ->
>   ?FORALL({K,V}, {key(),val()},
>           begin
>               T = setup(),
>               true = ets:insert(T, {K, V}),
>               Res = ets:lookup(T, K),
>               cleanup(T),
>               [{K, V}] =:= Res
>           end).
>
> setup() ->
>    ets:new(?MODULE, [public, named_table]).
>
> cleanup(T) ->
>    ets:delete(T).
> ========================================================================
>
> The pattern of doing a setup, running some commands and getting some
> Result, doing the cleanup you want to do, and the check that the result is
> the one that you expect is a general pattern in property-based testing.
>
> As you can see, the above works:
>
> kostis@REDACTED:~/proper$ erl -pa ~/proper
> Erlang/OTP 17 [erts-6.4.1] [source] [64-bit] [smp:8:8] [async-threads:10]
> [hipe] [kernel-poll:false]
>
> Eshell V6.4.1  (abort with ^G)
> 1> c(pets, [debug_info]).
> {ok,pets}
> 2> proper:quickcheck(pets:prop_ets()).
>
> ....................................................................................................
> OK: Passed 100 test(s).
> true
>
>
> Change the property to one that does not hold (e.g. [{K, V}] =/= Res) and
> you will also see that shrinking works.
>
>
> I am not sure I follow what you mean by your nested ?FORALL properties but
> note that, as shown above, you can easily imitate multiple ?FORALLs by
> using complex terms (tuples, lists, etc.).
>
> Hope this helps,
> Kostis
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From mononcqc@REDACTED  Tue May 26 14:46:31 2015
From: mononcqc@REDACTED (Fred Hebert)
Date: Tue, 26 May 2015 08:46:31 -0400
Subject: [erlang-questions] how search and search_quit are handled by
 user_drv?
In-Reply-To: 
References: 
Message-ID: <20150526124630.GH75968@ferdair.local>

On 05/21, Leo Liu wrote:
>Press key C-r and then TAB in the eshell one can observe the prompt
>changes to history `search' and back to normal again.
>
>I try to understand how it works in details but my understanding does
>not match what I observe above.
>
>The following is what I think should have happened where `|' indicates
>the cursor and indented lines are messages the user_drv process receives
>which cause the prompt to change from state 1 to 5.
>

Hi,

I implemented the search functionality so I'm in a good position to 
explain how it works.

The thing to understand about the Erlang shell is how it is being split 
up [1]:
- user.erl (or a tty program) ends up hadnling stdio
- user_sup.erl handles initialization of the whole system
- user_drv handles translation between the internal Erlang IO protocol 
  and the driver (tty) stuff. It also handles job management (^G) and 
  choosing which shell is active.
- group.erl has an instance for each individual job/shell, does line 
  buffering (until they're ready to be evaluated), handles up/down 
  arrows (to switch lines), and so on.
- edlin.erl handles each individual *line* for cursor movement, 
  character deletion, and so on. It also handles things like 
  interpreting keyboard shortcuts (^A, ^H, etc.)
- shell.erl handles evaluation of Erlang code

So when functionality like ^r search gets implemented, it needs to work 
both with group.erl (to cycle through lines, buffer the input, etc.) and 
edlin.erl (to interpret ^R and ^S). The stuff in user_drv takes place at 
an earlier time, and shell.erl at a latter time; both aren't connected 
to search.

The other thing to understand is that while vi and vim are the most 
known modal editors in the wild, the Erlang shell uses similar modes to 
understand how to handle its stuff internally, even through emacs mode; 
as such, +B (backwards word) is represented in two characters: $\e 
(escape), which switches the shell in 'meta' mode, and 'B', which is 
interpreted as a special sequence under that mode rather than regular 
ASCII input.

So the implementation works a bit as follows:

1. When ^R (backward search) or ^S (forward search) is seen in group.erl 
and we're in the normal mode, we store the current line prompt ('1>') 
and create a new one, then set edlin into the 'search' mode[2]

2. The search mode is enabled in edlin, which will handle interpreting 
what to do in there. Edlin has a character mapping table[3] that lets 
search interpret which mode the shell should be in. This means special 
sequences to mean quitting search mode, ways to specify whether to look 
back up or down again (^S or ^R has been pressed a second time), or 
whether the character is just valid. This portion is really about 
selecting the right *mode*.

3. The mapping is used to switch on operations [4] and identify proper 
transformations to the visual representation of the buffer than needs to 
be done [5]. This is because there's a difference between what we have 
as an internal state, and how we represent it. This bit is about 
*representation*.

4. Whenever the shell is in search mode, group.erl sends its data 
through edlin, handles the result, and looks at characters that came in, 
the current line buffer, and performs a search on each line, either 
backwards or forwards according to the search direction [6]. It then 
takes whatever line it has found (if any) and sends it to the driver to 
output (bypassing or adding to what edlin had done before) to get the 
right *representation* for the user.

This bit is tricky because while we display old user text in the line, 
this text never makes it to edlin.erl -- what we do is tell the driver 
"show this", while edlin holds the current search buffer in memory. The 
trick here is that group.erl stores the current line stack position 
(where in the list of line is the cursor?) so that user search doesn't 
conflict with what we show on screen.

5. Whenever the user is satisfied with a result, they press , 
which edlin interprets as the 'search_found' state; group.erl interprets 
this to fetch the current line buffer (which should be in a position 
where a satisfactory entry was found!), copy it in the current topmost 
position in the line buffer, and revert the prompt to normal with the 
input ready to go [7].

6. If on the other hand the search is cancelled, the shell falls back 
into edit mode, but at the current stack position [8] to replicate 
existing bash/zsh behaviour. This lets you edit a result you had found 
previously, in place.

7. Any other special meta modes (like search_meta_left_sq_bracket, which 
handles fancy escape sequences while in shell mode) are tunneled through 
to edlin because they have nothing to do with multiline business 
group.erl handles, and everything to do with mode switching.

This is all rather complex, but respects (as well as possible) the 
previously established separation of concerns of the Erlang shell 
between line editing and buffer management.

Regards,
Fred.

[1]: http://ferd.ca/repl-a-bit-more-and-less-than-that.html
[2]: 
https://github.com/erlang/otp/blob/maint/lib/stdlib/src/edlin.erl#L234
[3]: 
https://github.com/erlang/otp/blob/maint/lib/stdlib/src/edlin.erl#L234
[4]: 
https://github.com/erlang/otp/blob/maint/lib/stdlib/src/edlin.erl#L131
[5]: 
https://github.com/erlang/otp/blob/maint/lib/stdlib/src/edlin.erl#L268-L318
[6]: 
https://github.com/erlang/otp/blob/maint/lib/kernel/src/group.erl#L581-L605
[7]: 
https://github.com/erlang/otp/blob/maint/lib/kernel/src/group.erl#L558-L564
[8]: 
https://github.com/erlang/otp/blob/maint/lib/kernel/src/group.erl#L565-L580


From roger@REDACTED  Tue May 26 15:03:47 2015
From: roger@REDACTED (Roger Lipscombe)
Date: Tue, 26 May 2015 14:03:47 +0100
Subject: [erlang-questions] how are implemented lists?
In-Reply-To: 
References: 
 
 <20150522070421.GA4414@erix.ericsson.se>
 <9AD1EC29-6622-4857-848A-036C5712FE04@cs.otago.ac.nz>
 
Message-ID: 

On 25 May 2015 at 08:14, Benoit Chesneau  wrote:
> Indeed; sorry for that: So indeed that is this kind of data structure that I
> would like to have. Was thinking to use a skip-list data-structure as a nif
> but that may be overkill..

You want:

> a fast and concurrent data structure that allows me to retrieve the items
> in order and do pop/tail, while still being abble to remove them by key.

When you say "remove by key", do you need to retrieve the value, or do
you need to simply discard it?

I'm thinking that an ordinary list, coupled with an ordinary set,
might be what you want. When you remove the value, you _don't_ remove
it; you put it in the dictionary. Then, when it comes time to pop the
head, just look to see if the value is in the dictionary, and should
be ignored. Drop it on the floor, remove it from the dictionary. Done.

Obviously this depends on your exact use case, and the frequency of
the different operations. It also won't necessarily work if you want
to reuse a key, but you might be able to fudge that with some kind of
generation ID.

Cheers,
Roger.


From andra.dinu@REDACTED  Tue May 26 16:01:16 2015
From: andra.dinu@REDACTED (Andra Dinu)
Date: Tue, 26 May 2015 15:01:16 +0100
Subject: [erlang-questions] Call for Talks - Code Mesh London 3-5 Nov
Message-ID: 

Hi all,

Code Mesh is back! Like every year, our aim is to bring together a wide
range of alternative technologies and programming languages and the
wonderful crazy people who use them to solve real-world problems in
software industry.

With the aim of bringing together as many new users and inventors of
non-mainstream tech as possible, we are opening our first Call for Talks
(which closes on 26 June).

http://www.codemesh.io/#call-for-talks


*Andra Dinu*
Community  & Social

Check out the Erlang User Conference 
in Stockholm on 11-12 June!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From pdiwadkar@REDACTED  Tue May 26 16:34:11 2015
From: pdiwadkar@REDACTED (prasanna diwadkar)
Date: Tue, 26 May 2015 20:04:11 +0530
Subject: [erlang-questions] Using erlang instead of jmx
Message-ID: 

Hello all,I am new to erlang and trying to learn. One of the community project i am working on involves multiple java based distributed systems. Each of these system exposes its functionality through MBeans(JMX) which runs on JBoss instances. So the entire communication happens through JMX channel.I am trying to understand if Erlang is fit as communication layer here instead of JMX? (We don't face concurrency issues  here(except each MBean contains code containing locks,threads etc so lots of threads run on each JVM)! but will erlang make this kind of communication better or cleaner? 
RegardsC 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From kostis@REDACTED  Tue May 26 18:15:18 2015
From: kostis@REDACTED (Kostis Sagonas)
Date: Tue, 26 May 2015 18:15:18 +0200
Subject: [erlang-questions] PropEr after test clean up
In-Reply-To: 
References: 	<5563B3D0.9080006@cs.ntua.gr>
 
Message-ID: <55649C16.9070802@cs.ntua.gr>

On 05/26/2015 11:34 AM, Hynek Vychodil wrote:
> I think you are missing the point. Let's demonstrate the problem in this
> way. There is natural but not working way how to do it
>  ...  ...
> There is the problem. There is no way you can write ?WITH_G macro
> working with any of PropEr macros as a parameter. And it would have a
> simple solution. Just spawn the process or add hooks for clean up.
>
> I know it is possible to write it in a way it will work, but it prevents
> to write it reliably and in structured way.

It's quite possible I am missing the point -- the set of complicated 
macros you show do not make it easy (for me) to understand what the 
problem is and sadly I do not have any time to devote to deciphering 
them.  Perhaps a natural language description of the property you are 
trying to check would help (me) more than the macros you show.

Anyway, all I wrote in my previous mail was that the pattern of first 
doing a set up action, then running a test that returns some result that 
you save in a variable, then performing a cleanup action and finally 
checking whether the result satisfies or falsifies the property you want 
your test to satisfy is so natural --to me at least-- and general that I 
cannot possibly see why it does not work in what you want to do.  But of 
course what's "natural" is not defined objectively.

Kostis


From vychodil.hynek@REDACTED  Tue May 26 20:02:00 2015
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Tue, 26 May 2015 20:02:00 +0200
Subject: [erlang-questions] PropEr after test clean up
In-Reply-To: <55649C16.9070802@cs.ntua.gr>
References: 
 <5563B3D0.9080006@cs.ntua.gr>
 
 <55649C16.9070802@cs.ntua.gr>
Message-ID: 

It is pretty simple. You can't make macro like

-define(FOO(Data, Operation),
    (fun(Obj) ->
        try Operation after delete(Obj) end
    end)(new(Data))
  ).

which will work with Operation containing ?FORALL, ?WHENFAIL, ... etc. That
is it. It is very simple. You can't even use Erlang idiom.

do_after(F) ->
    Pid = self(),
    spawn_link(fun() -> process_flag(trap_exit, true),
            receive {'EXIT', Pid, _} -> F() end
        end).

because it all runs in one process. You just can't make simple and reliable
clean up embedded in wrapper macro or function. You always have to make
explicit ugly procedural code

Objs = init(),
run(),
delete(Objs),
test()

You simply can't make nice clean functional or function like wrapper. It is
ugly, repetitive, error-prone, procedural code. This is wrong. This is the
reason why we have
http://www.erlang.org/doc/apps/eunit/chapter.html#Fixtures in eunit!

On Tue, May 26, 2015 at 6:15 PM, Kostis Sagonas  wrote:

> On 05/26/2015 11:34 AM, Hynek Vychodil wrote:
>
>> I think you are missing the point. Let's demonstrate the problem in this
>> way. There is natural but not working way how to do it
>>  ...  ...
>> There is the problem. There is no way you can write ?WITH_G macro
>> working with any of PropEr macros as a parameter. And it would have a
>> simple solution. Just spawn the process or add hooks for clean up.
>>
>> I know it is possible to write it in a way it will work, but it prevents
>> to write it reliably and in structured way.
>>
>
> It's quite possible I am missing the point -- the set of complicated
> macros you show do not make it easy (for me) to understand what the problem
> is and sadly I do not have any time to devote to deciphering them.  Perhaps
> a natural language description of the property you are trying to check
> would help (me) more than the macros you show.
>
> Anyway, all I wrote in my previous mail was that the pattern of first
> doing a set up action, then running a test that returns some result that
> you save in a variable, then performing a cleanup action and finally
> checking whether the result satisfies or falsifies the property you want
> your test to satisfy is so natural --to me at least-- and general that I
> cannot possibly see why it does not work in what you want to do.  But of
> course what's "natural" is not defined objectively.
>
> Kostis
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From felixgallo@REDACTED  Tue May 26 20:14:24 2015
From: felixgallo@REDACTED (Felix Gallo)
Date: Tue, 26 May 2015 11:14:24 -0700
Subject: [erlang-questions] lowering jitter: best practices?
Message-ID: 

For a game server, I have a large number (~3000) of concurrent gen_fsm
processes which need to send out UDP packets every 30 milliseconds.  Each
individual UDP-sending function can be presumed to be very short (<1ms) and
not dominate or otherwise affect the scheduling.

I've tested a number of different scenarios:

1.  Each gen_fsm schedules itself via the gen_fsm timeout mechanism.  This
is the easiest and most natural way, but jitter can be +7ms in the 95%
case, and I occasionally see unusual events (e.g. timeout event happens
when only 25-28ms of real time have elapsed, despite 30ms being scheduled).


2.  One gen_fsm 'god timer' process schedules itself and sends out messages
to all of the concurrent gen_fsms to trigger them to send out their UDP
packets upon its own timeout.  Jitter is more variable, probably because
the BEAM decides that the god timer is being too chatty, and sometimes the
gen_fsms overwhelm the god timer in the schedulers.

3.  One port 'god timer' process (written in C, emitting a byte every
30ms).  Jitter is significantly reduced over #2 because apparently port
processes get better scheduling and BEAM doesn't get as angry at the
chattiness.  About on par with #1, maybe a little better, but the design is
unpretty owing to the increased complexity.

Additionally, I've heard of someone using a c-written port-process
god-timer per-fsm, which sounds to me like it would have the absolute best
latency scenario but would involve several thousand external processes, all
competing with erlang for thread execution, which sounds even unprettier.
Haven't tested that one.

Ideally I'd get as close to isochronous as possible, because the game
'feels' right if the stream of incoming packets is uniform.

None of the solutions gets me quite where I wanted; it's possible that any
of the above are the optimal available path and I just don't know that yet,
but I'm wondering if anyone has traveled down this same road and has
advice, tuning secrets, or other plans to share.

F.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From sergej.jurecko@REDACTED  Tue May 26 20:20:24 2015
From: sergej.jurecko@REDACTED (Sergej Jurecko)
Date: Tue, 26 May 2015 20:20:24 +0200
Subject: [erlang-questions] lowering jitter: best practices?
In-Reply-To: 
References: 
Message-ID: <9A0AA733-F61A-459F-84DB-BABA1F8490C6@gmail.com>

What about 2. with timer process having high priority?


Sergej

On 26 May 2015, at 20:14, Felix Gallo  wrote:

> For a game server, I have a large number (~3000) of concurrent gen_fsm processes which need to send out UDP packets every 30 milliseconds.  Each individual UDP-sending function can be presumed to be very short (<1ms) and not dominate or otherwise affect the scheduling.
> 
> I've tested a number of different scenarios:
> 
> 1.  Each gen_fsm schedules itself via the gen_fsm timeout mechanism.  This is the easiest and most natural way, but jitter can be +7ms in the 95% case, and I occasionally see unusual events (e.g. timeout event happens when only 25-28ms of real time have elapsed, despite 30ms being scheduled).  
> 
> 2.  One gen_fsm 'god timer' process schedules itself and sends out messages to all of the concurrent gen_fsms to trigger them to send out their UDP packets upon its own timeout.  Jitter is more variable, probably because the BEAM decides that the god timer is being too chatty, and sometimes the gen_fsms overwhelm the god timer in the schedulers.
> 
> 3.  One port 'god timer' process (written in C, emitting a byte every 30ms).  Jitter is significantly reduced over #2 because apparently port processes get better scheduling and BEAM doesn't get as angry at the chattiness.  About on par with #1, maybe a little better, but the design is unpretty owing to the increased complexity.
> 
> Additionally, I've heard of someone using a c-written port-process god-timer per-fsm, which sounds to me like it would have the absolute best latency scenario but would involve several thousand external processes, all competing with erlang for thread execution, which sounds even unprettier.  Haven't tested that one.
> 
> Ideally I'd get as close to isochronous as possible, because the game 'feels' right if the stream of incoming packets is uniform.
> 
> None of the solutions gets me quite where I wanted; it's possible that any of the above are the optimal available path and I just don't know that yet, but I'm wondering if anyone has traveled down this same road and has advice, tuning secrets, or other plans to share.
> 
> F.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions



From dmkolesnikov@REDACTED  Tue May 26 20:42:14 2015
From: dmkolesnikov@REDACTED (Dmitry Kolesnikov)
Date: Tue, 26 May 2015 21:42:14 +0300
Subject: [erlang-questions] lowering jitter: best practices?
In-Reply-To: 
References: 
Message-ID: <530DBD34-06F2-4489-8B19-C6715BF3F763@gmail.com>

hello, 

I am curious about #1. Are you using timer or send_after?

- Dmitry 

> On 26 May 2015, at 21:14, Felix Gallo  wrote:
> 
> For a game server, I have a large number (~3000) of concurrent gen_fsm processes which need to send out UDP packets every 30 milliseconds.  Each individual UDP-sending function can be presumed to be very short (<1ms) and not dominate or otherwise affect the scheduling.
> 
> I've tested a number of different scenarios:
> 
> 1.  Each gen_fsm schedules itself via the gen_fsm timeout mechanism.  This is the easiest and most natural way, but jitter can be +7ms in the 95% case, and I occasionally see unusual events (e.g. timeout event happens when only 25-28ms of real time have elapsed, despite 30ms being scheduled).  
> 
> 2.  One gen_fsm 'god timer' process schedules itself and sends out messages to all of the concurrent gen_fsms to trigger them to send out their UDP packets upon its own timeout.  Jitter is more variable, probably because the BEAM decides that the god timer is being too chatty, and sometimes the gen_fsms overwhelm the god timer in the schedulers.
> 
> 3.  One port 'god timer' process (written in C, emitting a byte every 30ms).  Jitter is significantly reduced over #2 because apparently port processes get better scheduling and BEAM doesn't get as angry at the chattiness.  About on par with #1, maybe a little better, but the design is unpretty owing to the increased complexity.
> 
> Additionally, I've heard of someone using a c-written port-process god-timer per-fsm, which sounds to me like it would have the absolute best latency scenario but would involve several thousand external processes, all competing with erlang for thread execution, which sounds even unprettier.  Haven't tested that one.
> 
> Ideally I'd get as close to isochronous as possible, because the game 'feels' right if the stream of incoming packets is uniform.
> 
> None of the solutions gets me quite where I wanted; it's possible that any of the above are the optimal available path and I just don't know that yet, but I'm wondering if anyone has traveled down this same road and has advice, tuning secrets, or other plans to share.
> 
> F.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions



From felixgallo@REDACTED  Tue May 26 20:52:14 2015
From: felixgallo@REDACTED (Felix Gallo)
Date: Tue, 26 May 2015 11:52:14 -0700
Subject: [erlang-questions] lowering jitter: best practices?
In-Reply-To: <530DBD34-06F2-4489-8B19-C6715BF3F763@gmail.com>
References: 
 <530DBD34-06F2-4489-8B19-C6715BF3F763@gmail.com>
Message-ID: 

For #1 and #2, I'm using the 4-item tuple return value for state calls in
gen_fsm:StateName/2:

{next_state,NextStateName,NewStateData,Timeout}

e.g.:

{ next_state, s_firststate, StateData#state{ last_time = NewTime, count =
Count + 1, pid = NotificationPid, jitters = [Jitter | Jitters] },
TickInterval };



On Tue, May 26, 2015 at 11:42 AM, Dmitry Kolesnikov 
wrote:

> hello,
>
> I am curious about #1. Are you using timer or send_after?
>
> - Dmitry
>
> > On 26 May 2015, at 21:14, Felix Gallo  wrote:
> >
> > For a game server, I have a large number (~3000) of concurrent gen_fsm
> processes which need to send out UDP packets every 30 milliseconds.  Each
> individual UDP-sending function can be presumed to be very short (<1ms) and
> not dominate or otherwise affect the scheduling.
> >
> > I've tested a number of different scenarios:
> >
> > 1.  Each gen_fsm schedules itself via the gen_fsm timeout mechanism.
> This is the easiest and most natural way, but jitter can be +7ms in the 95%
> case, and I occasionally see unusual events (e.g. timeout event happens
> when only 25-28ms of real time have elapsed, despite 30ms being scheduled).
> >
> > 2.  One gen_fsm 'god timer' process schedules itself and sends out
> messages to all of the concurrent gen_fsms to trigger them to send out
> their UDP packets upon its own timeout.  Jitter is more variable, probably
> because the BEAM decides that the god timer is being too chatty, and
> sometimes the gen_fsms overwhelm the god timer in the schedulers.
> >
> > 3.  One port 'god timer' process (written in C, emitting a byte every
> 30ms).  Jitter is significantly reduced over #2 because apparently port
> processes get better scheduling and BEAM doesn't get as angry at the
> chattiness.  About on par with #1, maybe a little better, but the design is
> unpretty owing to the increased complexity.
> >
> > Additionally, I've heard of someone using a c-written port-process
> god-timer per-fsm, which sounds to me like it would have the absolute best
> latency scenario but would involve several thousand external processes, all
> competing with erlang for thread execution, which sounds even unprettier.
> Haven't tested that one.
> >
> > Ideally I'd get as close to isochronous as possible, because the game
> 'feels' right if the stream of incoming packets is uniform.
> >
> > None of the solutions gets me quite where I wanted; it's possible that
> any of the above are the optimal available path and I just don't know that
> yet, but I'm wondering if anyone has traveled down this same road and has
> advice, tuning secrets, or other plans to share.
> >
> > F.
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From kostis@REDACTED  Tue May 26 21:03:06 2015
From: kostis@REDACTED (Kostis Sagonas)
Date: Tue, 26 May 2015 21:03:06 +0200
Subject: [erlang-questions] PropEr after test clean up
In-Reply-To: 
References: 	<5563B3D0.9080006@cs.ntua.gr>		<55649C16.9070802@cs.ntua.gr>
 
Message-ID: <5564C36A.2050400@cs.ntua.gr>

On 05/26/2015 08:02 PM, Hynek Vychodil wrote:
> You just can't make simple and reliable clean up embedded in wrapper
> macro or function. You always have to make explicit ugly procedural code
>
> Objs = init(),
> run(),
> delete(Objs),
> test()
>
> You simply can't make nice clean functional or function like wrapper. It
> is ugly, repetitive, error-prone, procedural code. This is wrong.

I will not comment on the "ugly", "repetitive", "error-prone" and 
"wrong" characterizations.  As I wrote in my previous message these are 
not defined objectively.

However, please realize that the whole issue is about something which is 
not functional in the first place!  Under this prism, I am not very 
surprised you cannot, or at least find it difficult to, find a "clean 
functional" solution...  I would even argue that it is better this way. 
(I.e., that there is a "penalty" for wanting to test non-functional 
code.  If you find it disturbing, then simply write functional code.)

Kostis


From jesper.louis.andersen@REDACTED  Tue May 26 21:09:13 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Tue, 26 May 2015 21:09:13 +0200
Subject: [erlang-questions] lowering jitter: best practices?
In-Reply-To: 
References: 
 <530DBD34-06F2-4489-8B19-C6715BF3F763@gmail.com>
 
Message-ID: 

On Tue, May 26, 2015 at 8:52 PM, Felix Gallo  wrote:

> {next_state,NextStateName,NewStateData,Timeout}


This explains why you sometimes get less than 30ms sleep times. If an event
reaches the process before Timeout, then the timeout is not triggered.
Also, it may explain the jitter you are seeing, because an early event will
reset the timeout. Try using gen_fsm:start_timer/2 or erlang:send_after...

If the problem persists, check lcnt. If you are locked on the timer wheel,
then consider release 18 :)


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From felixgallo@REDACTED  Tue May 26 23:03:41 2015
From: felixgallo@REDACTED (Felix Gallo)
Date: Tue, 26 May 2015 14:03:41 -0700
Subject: [erlang-questions] lowering jitter: best practices?
In-Reply-To: 
References: 
 <530DBD34-06F2-4489-8B19-C6715BF3F763@gmail.com>
 
 
Message-ID: 

Innovative thinking, Jesper!  But in this case, in this testbed, the fsms
aren't getting any messages other than those which they are delivering to
themselves.  Which adds to the intrigue.

I took your suggestion and tried using gen_fsm:start_timer/2.
Interestingly it slightly increased the jitter variance and the negative
jitter issue is still present.  It's possible that my, ah,
rapidly-and-pragmatically-built testbed suffers from some flaw, but I'm not
seeing it.

Here's my code:

https://gist.github.com/anonymous/47cde5e60a619319053f

Here's sample output on this small but moderately modern non-cloud osx
machine:

> test_fsm5:go(1000,40,40,10).
waiting for 1000 FSMs, tickrate 40
avg: 1324.1012703862662
max: 50219
min: -184
median: 1018
95th: 2615
99th: 9698

note that the max is 50ms of jitter; the min is negative 184 us jitter, and
the median jitter is about 1ms, which correlates well with my beliefs about
scheduler wakeup timers...

F.


On Tue, May 26, 2015 at 12:09 PM, Jesper Louis Andersen <
jesper.louis.andersen@REDACTED> wrote:

>
> On Tue, May 26, 2015 at 8:52 PM, Felix Gallo  wrote:
>
>> {next_state,NextStateName,NewStateData,Timeout}
>
>
> This explains why you sometimes get less than 30ms sleep times. If an
> event reaches the process before Timeout, then the timeout is not
> triggered. Also, it may explain the jitter you are seeing, because an early
> event will reset the timeout. Try using gen_fsm:start_timer/2 or
> erlang:send_after...
>
> If the problem persists, check lcnt. If you are locked on the timer wheel,
> then consider release 18 :)
>
>
> --
> J.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From g@REDACTED  Wed May 27 00:08:08 2015
From: g@REDACTED (Garrett Smith)
Date: Tue, 26 May 2015 17:08:08 -0500
Subject: [erlang-questions] Erlang meetup this Wednesday night - Chicago
Message-ID: 

If you're in the Chicago area tomorrow night and have some spare time,
come on in - the Erlang's fine!

http://www.meetup.com/ErlangChicago/

Garrett


From chandrashekhar.mullaparthi@REDACTED  Wed May 27 08:38:25 2015
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Wed, 27 May 2015 07:38:25 +0100
Subject: [erlang-questions] Using erlang instead of jmx
In-Reply-To: 
References: 
Message-ID: 

Hi,

Without knowing much more detail, at the very least you will get more
expressive and concise code. Multithreaded Java code is not a pretty sight.
At the risk of sounding like a salesman, things you will gain by using
erlang in this context are:

- Smaller, more expressive code base
- No need to worry about GC pauses/tuning
- Easier troubleshooting using the runtime trace capabilities of Erlang
- You will enjoy your coding!

I think the best way to find out for yourself is to prototype this. You may
be surprised at how quickly you can build reasonably complex systems.

cheers
Chandru


On 26 May 2015 at 15:34, prasanna diwadkar  wrote:

> Hello all,
> I am new to erlang and trying to learn. One of the community project i am
> working on involves multiple java based distributed systems. Each of these
> system exposes its functionality through MBeans(JMX) which runs on JBoss
> instances. So the entire communication happens through JMX channel.
> I am trying to understand if Erlang is fit as communication layer here
> instead of JMX? (We don't face concurrency issues  here(except each MBean
> contains code containing locks,threads etc so lots of threads run on each
> JVM)! but will erlang make this kind of communication better or cleaner?
>
> Regards
> C
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From chandrashekhar.mullaparthi@REDACTED  Wed May 27 08:48:09 2015
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Wed, 27 May 2015 07:48:09 +0100
Subject: [erlang-questions] lowering jitter: best practices?
In-Reply-To: 
References: 
Message-ID: 

On 26 May 2015 at 19:14, Felix Gallo  wrote:

> For a game server, I have a large number (~3000) of concurrent gen_fsm
> processes which need to send out UDP packets every 30 milliseconds.  Each
> individual UDP-sending function can be presumed to be very short (<1ms) and
> not dominate or otherwise affect the scheduling.
>
> I've tested a number of different scenarios:
>
> 1.  Each gen_fsm schedules itself via the gen_fsm timeout mechanism.  This
> is the easiest and most natural way, but jitter can be +7ms in the 95%
> case, and I occasionally see unusual events (e.g. timeout event happens
> when only 25-28ms of real time have elapsed, despite 30ms being scheduled).
>
>

Here are a few ideas, obviously all untested.

* How about setting the timer to fire initially at 10-15ms, and adjust the
next timer interval based on observed drift?
* Let all the gen_fsm processes insert the packets they have to send into
an ordered_set (ordered by time) ets table and have a single process which
is checking the ETS table for messages to send at a certain point in time?
* Do you have any control on the receiving end? Can some smoothing of this
jitter be done there?

Chandru
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From dmkolesnikov@REDACTED  Wed May 27 11:07:36 2015
From: dmkolesnikov@REDACTED (Dmitry Kolesnikov)
Date: Wed, 27 May 2015 12:07:36 +0300
Subject: [erlang-questions] lowering jitter: best practices?
In-Reply-To: 
References: 
 <530DBD34-06F2-4489-8B19-C6715BF3F763@gmail.com>
 
 
 
Message-ID: <743E2D6A-7346-4059-A24B-0048149D2DE6@gmail.com>

Hello,

This appears to be an interesting issue. First of all I?ve not seen a negative jitter and my impression was that your FSM loop suffer from measurement error. I?ve changed the measurement loop to be very-very tiny but there was not significant gain on jitter it is on par with your original code.

...
T0 = os:timestamp(),
receive
...
after TickInterval ->
   T1     = os:timestamp(),
   Jitter = timer:now_diff(T1, T0) - (TickInterval * 1000)
   ...
end
...

All-in-all, I?ve run the measurement both on my laptop Mac Book (Intel i5) and Amazon (cr1.8xlarge). The results was steady. You can find them below. I?ve seen jitter over 50ms for 10K process on my laptop only.
I am tend to thing that you are experience high jitter due to excessive CPU utilisation in your test bed. However, you can run the test with single FSM, the jitter is far from 0. May be it is time look VM internal. 

You have proposed three possible solutions in your earlier email, I am afraid all of them will suffer from ?jitter? due to multiple reasons including the overhead in network communication. The option #1 (timer per fsm) still looks more feasible from my perspective. You might implement adaptive timer to minimise experienced errors. 

Laptop:
kolesnik@REDACTED:tmp$ erl -sbt ts -sws very_eager -swt high
Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V6.2  (abort with ^G)
1> test_fsm5:go(1000, 50, 50, 1).
waiting for 1000 FSMs, tickrate 50
avg: 3164.69932160804
max: 13372
min: 4
median: 2426
95th: 8339
99th: 10552
all_done

AWS:

[ec2-user@REDACTED ~]$ /usr/local/xxx/erts-6.2/bin/erl -sbt ts -sws very_eager -swt high
Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:32:32] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V6.2  (abort with ^G)
1> test_fsm5:go(1000, 50, 50, 1).
waiting for 1000 FSMs, tickrate 50
avg: 998.798351758794
max: 1926
min: 82
median: 998
95th: 1152
99th: 1204
all_done

- Dmitry


> On 27 May 2015, at 00:03, Felix Gallo  wrote:
> 
> Innovative thinking, Jesper!  But in this case, in this testbed, the fsms aren't getting any messages other than those which they are delivering to themselves.  Which adds to the intrigue.  
> 
> I took your suggestion and tried using gen_fsm:start_timer/2.  Interestingly it slightly increased the jitter variance and the negative jitter issue is still present.  It's possible that my, ah, rapidly-and-pragmatically-built testbed suffers from some flaw, but I'm not seeing it.
> 
> Here's my code:
> 
> https://gist.github.com/anonymous/47cde5e60a619319053f 
> 
> Here's sample output on this small but moderately modern non-cloud osx machine:
> 
> > test_fsm5:go(1000,40,40,10).
> waiting for 1000 FSMs, tickrate 40
> avg: 1324.1012703862662
> max: 50219
> min: -184
> median: 1018
> 95th: 2615
> 99th: 9698
> 
> note that the max is 50ms of jitter; the min is negative 184 us jitter, and the median jitter is about 1ms, which correlates well with my beliefs about scheduler wakeup timers...
> 
> F.
> 
> 
> On Tue, May 26, 2015 at 12:09 PM, Jesper Louis Andersen > wrote:
> 
> On Tue, May 26, 2015 at 8:52 PM, Felix Gallo > wrote:
> {next_state,NextStateName,NewStateData,Timeout}
> 
> This explains why you sometimes get less than 30ms sleep times. If an event reaches the process before Timeout, then the timeout is not triggered. Also, it may explain the jitter you are seeing, because an early event will reset the timeout. Try using gen_fsm:start_timer/2 or erlang:send_after...
> 
> If the problem persists, check lcnt. If you are locked on the timer wheel, then consider release 18 :)
> 
> 
> -- 
> J.
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From vychodil.hynek@REDACTED  Wed May 27 11:25:55 2015
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Wed, 27 May 2015 11:25:55 +0200
Subject: [erlang-questions] PropEr after test clean up
In-Reply-To: <5564C36A.2050400@cs.ntua.gr>
References: 
 <5563B3D0.9080006@cs.ntua.gr>
 
 <55649C16.9070802@cs.ntua.gr>
 
 <5564C36A.2050400@cs.ntua.gr>
Message-ID: 

Well is so uncommon and subjective that this approach even make it to the
http://www.erlang.se/doc/programming_rules.shtml#HDR13

Making fixtures, keeping related things together and using processes for an
initiating clean state are best practices in Erlang. PropEr doesn't follow
those principles. Sentences like " the whole issue is about something which
is not functional in the first place" is not helping. "If you find it
disturbing, then simply write functional code." I would like to, but I
found this issue dealing with digraph module form OTP stdlib!

On Tue, May 26, 2015 at 9:03 PM, Kostis Sagonas  wrote:

> On 05/26/2015 08:02 PM, Hynek Vychodil wrote:
>
>> You just can't make simple and reliable clean up embedded in wrapper
>> macro or function. You always have to make explicit ugly procedural code
>>
>> Objs = init(),
>> run(),
>> delete(Objs),
>> test()
>>
>> You simply can't make nice clean functional or function like wrapper. It
>> is ugly, repetitive, error-prone, procedural code. This is wrong.
>>
>
> I will not comment on the "ugly", "repetitive", "error-prone" and "wrong"
> characterizations.  As I wrote in my previous message these are not defined
> objectively.
>
> However, please realize that the whole issue is about something which is
> not functional in the first place!  Under this prism, I am not very
> surprised you cannot, or at least find it difficult to, find a "clean
> functional" solution...  I would even argue that it is better this way.
> (I.e., that there is a "penalty" for wanting to test non-functional code.
> If you find it disturbing, then simply write functional code.)
>
> Kostis
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From pdiwadkar@REDACTED  Wed May 27 09:13:19 2015
From: pdiwadkar@REDACTED (prasanna diwadkar)
Date: Wed, 27 May 2015 12:43:19 +0530
Subject: [erlang-questions] Using erlang instead of jmx
In-Reply-To: 
References: ,
 
Message-ID: 

Thanks Chandru. I am in process of doing prototype.As underlying functionality is primarily Java(for e.g. DatabaseMBean exposes methods to get database details), how easy is to switch context from Erlang to Java and vice versa?  i guess  there won't me much latency issues.

Regards
P




From: chandrashekhar.mullaparthi@REDACTED
Date: Wed, 27 May 2015 07:38:25 +0100
Subject: Re: [erlang-questions] Using erlang instead of jmx
To: pdiwadkar@REDACTED
CC: erlang-questions@REDACTED

Hi,

Without knowing much more detail, at the very least you will get more expressive and concise code. Multithreaded Java code is not a pretty sight. At the risk of sounding like a salesman, things you will gain by using erlang in this context are:

- Smaller, more expressive code base 
- No need to worry about GC pauses/tuning
- Easier troubleshooting using the runtime trace capabilities of Erlang
- You will enjoy your coding!

I think the best way to find out for yourself is to prototype this. You may be surprised at how quickly you can build reasonably complex systems.

cheers
Chandru


On 26 May 2015 at 15:34, prasanna diwadkar  wrote:



Hello all,I am new to erlang and trying to learn. One of the community project i am working on involves multiple java based distributed systems. Each of these system exposes its functionality through MBeans(JMX) which runs on JBoss instances. So the entire communication happens through JMX channel.I am trying to understand if Erlang is fit as communication layer here instead of JMX? (We don't face concurrency issues  here(except each MBean contains code containing locks,threads etc so lots of threads run on each JVM)! but will erlang make this kind of communication better or cleaner? 
RegardsC 		 	   		  

_______________________________________________

erlang-questions mailing list

erlang-questions@REDACTED

http://erlang.org/mailman/listinfo/erlang-questions



 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jesper.louis.andersen@REDACTED  Wed May 27 14:17:35 2015
From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen)
Date: Wed, 27 May 2015 14:17:35 +0200
Subject: [erlang-questions] lowering jitter: best practices?
In-Reply-To: 
References: 
 <530DBD34-06F2-4489-8B19-C6715BF3F763@gmail.com>
 
 
 
Message-ID: 

Hi,

I applied the following patch:

; diff -u test_fsm5.orig test_fsm5.erl
--- test_fsm5.orig 2015-05-27 13:54:34.381978128 +0200
+++ test_fsm5.erl 2015-05-27 13:51:38.521826422 +0200
@@ -10,7 +10,7 @@
 -define(MINCOUNT, 300). % iters.  Will not record jitter until this many
timeouts have passed.  Crude attempt to give schedulers settle time.

 init([TickInterval,NotificationPid]) ->
-  State = #state{ last_time = get_os_time(), tickinterval = TickInterval,
pid = NotificationPid },
+  State = #state{ last_time = erlang:monotonic_time(), tickinterval =
TickInterval, pid = NotificationPid },
   {ok, s_firststate, State, TickInterval}.

 handle_event(_Event, StateName, StateData) ->
@@ -29,10 +29,10 @@
   {ok, StateName, StateData}.

 s_firststate(timeout, #state{ last_time = LastTime, count = Count ,
tickinterval = TickInterval, pid = NotificationPid, jitters = Jitters } =
StateData) ->
-  NewTime = get_os_time(),
-  TimeDiff = NewTime - LastTime,
-  Jitter = TimeDiff - (TickInterval * 1000), % microseconds
+  NewTime = erlang:monotonic_time(),
   gen_fsm:start_timer(TickInterval, timeout),
+  TimeDiff = erlang:convert_time_unit(NewTime - LastTime, native,
micro_seconds),
+  Jitter = TimeDiff - (TickInterval * 1000), % microseconds
   case {(Count > ?MINCOUNT), (Count < ?MAXCOUNT)} of
     {false, true} ->
       { next_state, s_firststate, StateData#state{ last_time = NewTime,
count = Count + 1, pid = NotificationPid, jitters = Jitters } };
@@ -81,10 +81,6 @@
   report(TickFrom,NumFSMs),
   go_run(NumFSMs, TickFrom + TickStep, TickTo, TickStep).

-get_os_time() ->
-  {MegaS, S, MicroS} = os:timestamp(),
-  (MegaS * 1000000 * 1000000 + S * 1000000 + MicroS).
-
 await(0) -> ok;
 await(N) ->
   receive _ ->
@@ -93,6 +89,7 @@

 report(Tick, NumFSMs) ->
   X = lists:sort([A || {_, A} <- ets:lookup(metrics,jitter)]),
+  file:write_file("observations.txt", [[integer_to_binary(E), $\n] || E <-
X]),
   Avg = lists:sum(X)/length(X),
   Max = lists:max(X),
   Min = lists:min(X),

which switches to Erlang/OTP 18-rc2 and also uses the new time API. Machine
is a Core i7-4900MQ CPU, ArchLinux, fairly recent linux kernel (Linux
lady-of-pain 4.0.4-2-ARCH #1 SMP PREEMPT Fri May 22 03:05:23 UTC 2015
x86_64 GNU/Linux). It also dumps the jitter to "observations.txt" so we can
look at the observations.

We run `erl +sbt db +C multi_time_warp` to bind schedulers to cores and
request a timing mode which is able to maximize precise/accurate monotonic
time.

Then I ran the data through a bit of R:

> x <- read.csv("observations.txt", header=FALSE)
> require(ggplot2)
> p <- ggplot(x, aes(x = V1))
> png("observations.png")
> p + geom_density()
> dev.off()
X11cairo
       2
> summary(x)
       V1
 Min.   : 153
 1st Qu.: 973
 Median : 998
 Mean   :1001
 3rd Qu.:1028
 Max.   :8018

One may wonder what the spread is of the upper quantiles:

> quantile(x$V1,  c(.9, .95, .99, .999, .9999, .99999))
     90%      95%      99%    99.9%   99.99%  99.999%
1065.000 1083.000 1142.000 1818.001 5045.000 7945.010

The kernel density plot is attached, and it places itself around 1000 quite
precisely.

This is actually somewhat to be expected. When we request a timeout of
40ms, we are somewhere inside a milli-second T. We can be close to the
"start", i.e., T.001, or close to the "end", i.e., T.997. If we define T =
0 as our start point for our time, then we clearly can't wake up at P = 40,
because 40 - 0.997 is 39.003, which would violate our wait time and wake up
too early. Hence, we round our wake-time up to the next full milli-second,
which is 41. This tracks extremely well with our numbers.

But it is highly unlikely that our 1000 processes would all trigger in the
*same* millisecond, which would make a few of them round up to a different
timepoint for wakeup. At least this would be a plausible explanation for
jitter less than 1000 micro-seconds.

As for the 99.99th and 99.999th percentile, I think these can be attributed
to something else happening in the system: OS, Garbage Collection, etc. I'm
not sure the culprit there is the timer wheels.

Another point worth mentioning is that if 1000 processes wake up at the
same millisecond edge, then they will queue over N cores. So invariably,
this means some of the processes will have jitter. The workload you are
working with is very "spiky" in this case, even if the average load on the
system is very low. Do the math, assuming perfect spread, each core gets
1000 div 8 = 125 processes. They all awaken at the same time. Even if we
can handle each process in 1 micro-second, there will be a process with a
125 micro-second latency. That is, since we pack so many processes in a
short window, the system becomes more sensitive to small fluctuations. The
numbers suggest we handle each process in less than a microsecond.

Going even faster, in the low nanoseconds, requires a change of system,
since Erlang isn't the right tool for the job at that scale. You need to
pack data in arrays to get better cache-access patterns at that scale since
a DRAM hit is roughly 30ns (or more!). The functional nature of Erlang will
hurt here. This is usually a job for OCaml, C, Go, GPUs, FPGAs or ASICs.


On Tue, May 26, 2015 at 11:03 PM, Felix Gallo  wrote:

> Innovative thinking, Jesper!  But in this case, in this testbed, the fsms
> aren't getting any messages other than those which they are delivering to
> themselves.  Which adds to the intrigue.
>
> I took your suggestion and tried using gen_fsm:start_timer/2.
> Interestingly it slightly increased the jitter variance and the negative
> jitter issue is still present.  It's possible that my, ah,
> rapidly-and-pragmatically-built testbed suffers from some flaw, but I'm not
> seeing it.
>
> Here's my code:
>
> https://gist.github.com/anonymous/47cde5e60a619319053f
>
> Here's sample output on this small but moderately modern non-cloud osx
> machine:
>
> > test_fsm5:go(1000,40,40,10).
> waiting for 1000 FSMs, tickrate 40
> avg: 1324.1012703862662
> max: 50219
> min: -184
> median: 1018
> 95th: 2615
> 99th: 9698
>
> note that the max is 50ms of jitter; the min is negative 184 us jitter,
> and the median jitter is about 1ms, which correlates well with my beliefs
> about scheduler wakeup timers...
>
> F.
>
>
> On Tue, May 26, 2015 at 12:09 PM, Jesper Louis Andersen <
> jesper.louis.andersen@REDACTED> wrote:
>
>>
>> On Tue, May 26, 2015 at 8:52 PM, Felix Gallo 
>> wrote:
>>
>>> {next_state,NextStateName,NewStateData,Timeout}
>>
>>
>> This explains why you sometimes get less than 30ms sleep times. If an
>> event reaches the process before Timeout, then the timeout is not
>> triggered. Also, it may explain the jitter you are seeing, because an early
>> event will reset the timeout. Try using gen_fsm:start_timer/2 or
>> erlang:send_after...
>>
>> If the problem persists, check lcnt. If you are locked on the timer
>> wheel, then consider release 18 :)
>>
>>
>> --
>> J.
>>
>
>


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: observations.png
Type: image/png
Size: 10253 bytes
Desc: not available
URL: 

From serge@REDACTED  Wed May 27 22:39:31 2015
From: serge@REDACTED (Serge Aleynikov)
Date: Wed, 27 May 2015 16:39:31 -0400
Subject: [erlang-questions] Application environment
Message-ID: 

I run into an issue while trying to fix a test case in this pull request:
https://github.com/erlang/otp/pull/713

Is it possible to override application's startup environment dynamically if
its *.app file contains an {env, [...]} clause?

I had the following in the test case, in which my {file, File} setting
would get discarded:

    SaslEnv = application:get_all_env(sasl),
    lists:foreach(fun({E,_V}) -> application:unset_env(sasl,E) end, SaslEnv),

    ok = application:set_env(sasl,sasl_error_logger,{file, File}),
    ok = application:start(sasl),

Serge
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jose.valim@REDACTED  Wed May 27 22:49:02 2015
From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=)
Date: Wed, 27 May 2015 22:49:02 +0200
Subject: [erlang-questions] Application environment
In-Reply-To: 
References: 
Message-ID: 

You can call set_env with the [{persistent, true}] option.

More information about it in the docs:
http://www.erlang.org/doc/apps/kernel/application.html#set_env-3



*Jos? Valim*
www.plataformatec.com.br
Skype: jv.ptec
Founder and Lead Developer

On Wed, May 27, 2015 at 10:39 PM, Serge Aleynikov 
wrote:

> I run into an issue while trying to fix a test case in this pull request:
> https://github.com/erlang/otp/pull/713
>
> Is it possible to override application's startup environment dynamically
> if its *.app file contains an {env, [...]} clause?
>
> I had the following in the test case, in which my {file, File} setting
> would get discarded:
>
>     SaslEnv = application:get_all_env(sasl),
>     lists:foreach(fun({E,_V}) -> application:unset_env(sasl,E) end, SaslEnv),
>
>     ok = application:set_env(sasl,sasl_error_logger,{file, File}),
>     ok = application:start(sasl),
>
> Serge
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From eric.pailleau@REDACTED  Wed May 27 22:55:18 2015
From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=)
Date: Wed, 27 May 2015 22:55:18 +0200
Subject: [erlang-questions] Application environment
In-Reply-To: 
Message-ID: <6d015851-be24-4c2f-8273-29d06792a6ee@email.android.com>

Hi,
See process_win function in this file :
https://github.com/erlang/otp/blob/master/lib/observer/test/observer_SUITE.erl

An example of temporary app env variable overriding... 
Regards


Le?27 mai 2015 22:39, Serge Aleynikov  a ?crit?:
>
> I run into an issue while trying to fix a test case in this pull request:
> https://github.com/erlang/otp/pull/713
>
> Is it possible to override application's startup environment dynamically if its *.app file contains an {env, [...]} clause?
>
> I had the following in the test case, in which my {file, File} setting would get discarded:
>
>     SaslEnv = application:get_all_env(sasl),
>
>     lists:foreach(fun({E,_V}) -> application:unset_env(sasl,E) end, SaslEnv),
>
>
>
>     ok = application:set_env(sasl,sasl_error_logger,{file, File}),
>
>     ok = application:start(sasl),
>
> Serge

From serge@REDACTED  Wed May 27 23:37:40 2015
From: serge@REDACTED (Serge Aleynikov)
Date: Wed, 27 May 2015 17:37:40 -0400
Subject: [erlang-questions] Application environment
In-Reply-To: 
References: 
 
Message-ID: 

That's it! Thank you very much!

On Wed, May 27, 2015 at 4:49 PM, Jos? Valim  wrote:

> You can call set_env with the [{persistent, true}] option.
>
> More information about it in the docs:
> http://www.erlang.org/doc/apps/kernel/application.html#set_env-3
>
>
>
> *Jos? Valim*
> www.plataformatec.com.br
> Skype: jv.ptec
> Founder and Lead Developer
>
> On Wed, May 27, 2015 at 10:39 PM, Serge Aleynikov 
> wrote:
>
>> I run into an issue while trying to fix a test case in this pull request:
>> https://github.com/erlang/otp/pull/713
>>
>> Is it possible to override application's startup environment dynamically
>> if its *.app file contains an {env, [...]} clause?
>>
>> I had the following in the test case, in which my {file, File} setting
>> would get discarded:
>>
>>     SaslEnv = application:get_all_env(sasl),
>>     lists:foreach(fun({E,_V}) -> application:unset_env(sasl,E) end, SaslEnv),
>>
>>     ok = application:set_env(sasl,sasl_error_logger,{file, File}),
>>     ok = application:start(sasl),
>>
>> Serge
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From felixgallo@REDACTED  Thu May 28 01:25:08 2015
From: felixgallo@REDACTED (Felix Gallo)
Date: Wed, 27 May 2015 16:25:08 -0700
Subject: [erlang-questions] lowering jitter: best practices?
In-Reply-To: 
References: 
 <530DBD34-06F2-4489-8B19-C6715BF3F763@gmail.com>
 
 
 
 
Message-ID: 

Jesper, thanks for the analysis.

Since your results suggested that R17 os:timestamp was heavily implicated
in my jitter error calculations (all too typical a result for
microbenchmarking, but I had to trust something), I tried again on a
variety of cloud linux boxes and got similar results, suggesting that
something may be up with R17.5.  Since you're further not seeing that with
R18rc2, I have to guess that either the new time API or the timer wheel
rework are improving matters.  Which is excellent news, even if it
currently doesn't compile on my box. :)

I'll do some more jitter exploration with R18rc2 in the coming weeks; if
anyone is reading this in the future and would like to know how it all went
down, please feel free to contact me for details.  Thanks to all for the
help.

F.



On Wed, May 27, 2015 at 5:17 AM, Jesper Louis Andersen <
jesper.louis.andersen@REDACTED> wrote:

> Hi,
>
> I applied the following patch:
>
> ; diff -u test_fsm5.orig test_fsm5.erl
> --- test_fsm5.orig 2015-05-27 13:54:34.381978128 +0200
> +++ test_fsm5.erl 2015-05-27 13:51:38.521826422 +0200
> @@ -10,7 +10,7 @@
>  -define(MINCOUNT, 300). % iters.  Will not record jitter until this many
> timeouts have passed.  Crude attempt to give schedulers settle time.
>
>  init([TickInterval,NotificationPid]) ->
> -  State = #state{ last_time = get_os_time(), tickinterval = TickInterval,
> pid = NotificationPid },
> +  State = #state{ last_time = erlang:monotonic_time(), tickinterval =
> TickInterval, pid = NotificationPid },
>    {ok, s_firststate, State, TickInterval}.
>
>  handle_event(_Event, StateName, StateData) ->
> @@ -29,10 +29,10 @@
>    {ok, StateName, StateData}.
>
>  s_firststate(timeout, #state{ last_time = LastTime, count = Count ,
> tickinterval = TickInterval, pid = NotificationPid, jitters = Jitters } =
> StateData) ->
> -  NewTime = get_os_time(),
> -  TimeDiff = NewTime - LastTime,
> -  Jitter = TimeDiff - (TickInterval * 1000), % microseconds
> +  NewTime = erlang:monotonic_time(),
>    gen_fsm:start_timer(TickInterval, timeout),
> +  TimeDiff = erlang:convert_time_unit(NewTime - LastTime, native,
> micro_seconds),
> +  Jitter = TimeDiff - (TickInterval * 1000), % microseconds
>    case {(Count > ?MINCOUNT), (Count < ?MAXCOUNT)} of
>      {false, true} ->
>        { next_state, s_firststate, StateData#state{ last_time = NewTime,
> count = Count + 1, pid = NotificationPid, jitters = Jitters } };
> @@ -81,10 +81,6 @@
>    report(TickFrom,NumFSMs),
>    go_run(NumFSMs, TickFrom + TickStep, TickTo, TickStep).
>
> -get_os_time() ->
> -  {MegaS, S, MicroS} = os:timestamp(),
> -  (MegaS * 1000000 * 1000000 + S * 1000000 + MicroS).
> -
>  await(0) -> ok;
>  await(N) ->
>    receive _ ->
> @@ -93,6 +89,7 @@
>
>  report(Tick, NumFSMs) ->
>    X = lists:sort([A || {_, A} <- ets:lookup(metrics,jitter)]),
> +  file:write_file("observations.txt", [[integer_to_binary(E), $\n] || E
> <- X]),
>    Avg = lists:sum(X)/length(X),
>    Max = lists:max(X),
>    Min = lists:min(X),
>
> which switches to Erlang/OTP 18-rc2 and also uses the new time API.
> Machine is a Core i7-4900MQ CPU, ArchLinux, fairly recent linux kernel
> (Linux lady-of-pain 4.0.4-2-ARCH #1 SMP PREEMPT Fri May 22 03:05:23 UTC
> 2015 x86_64 GNU/Linux). It also dumps the jitter to "observations.txt" so
> we can look at the observations.
>
> We run `erl +sbt db +C multi_time_warp` to bind schedulers to cores and
> request a timing mode which is able to maximize precise/accurate monotonic
> time.
>
> Then I ran the data through a bit of R:
>
> > x <- read.csv("observations.txt", header=FALSE)
> > require(ggplot2)
> > p <- ggplot(x, aes(x = V1))
> > png("observations.png")
> > p + geom_density()
> > dev.off()
> X11cairo
>        2
> > summary(x)
>        V1
>  Min.   : 153
>  1st Qu.: 973
>  Median : 998
>  Mean   :1001
>  3rd Qu.:1028
>  Max.   :8018
>
> One may wonder what the spread is of the upper quantiles:
>
> > quantile(x$V1,  c(.9, .95, .99, .999, .9999, .99999))
>      90%      95%      99%    99.9%   99.99%  99.999%
> 1065.000 1083.000 1142.000 1818.001 5045.000 7945.010
>
> The kernel density plot is attached, and it places itself around 1000
> quite precisely.
>
> This is actually somewhat to be expected. When we request a timeout of
> 40ms, we are somewhere inside a milli-second T. We can be close to the
> "start", i.e., T.001, or close to the "end", i.e., T.997. If we define T =
> 0 as our start point for our time, then we clearly can't wake up at P = 40,
> because 40 - 0.997 is 39.003, which would violate our wait time and wake up
> too early. Hence, we round our wake-time up to the next full milli-second,
> which is 41. This tracks extremely well with our numbers.
>
> But it is highly unlikely that our 1000 processes would all trigger in the
> *same* millisecond, which would make a few of them round up to a different
> timepoint for wakeup. At least this would be a plausible explanation for
> jitter less than 1000 micro-seconds.
>
> As for the 99.99th and 99.999th percentile, I think these can be
> attributed to something else happening in the system: OS, Garbage
> Collection, etc. I'm not sure the culprit there is the timer wheels.
>
> Another point worth mentioning is that if 1000 processes wake up at the
> same millisecond edge, then they will queue over N cores. So invariably,
> this means some of the processes will have jitter. The workload you are
> working with is very "spiky" in this case, even if the average load on the
> system is very low. Do the math, assuming perfect spread, each core gets
> 1000 div 8 = 125 processes. They all awaken at the same time. Even if we
> can handle each process in 1 micro-second, there will be a process with a
> 125 micro-second latency. That is, since we pack so many processes in a
> short window, the system becomes more sensitive to small fluctuations. The
> numbers suggest we handle each process in less than a microsecond.
>
> Going even faster, in the low nanoseconds, requires a change of system,
> since Erlang isn't the right tool for the job at that scale. You need to
> pack data in arrays to get better cache-access patterns at that scale since
> a DRAM hit is roughly 30ns (or more!). The functional nature of Erlang will
> hurt here. This is usually a job for OCaml, C, Go, GPUs, FPGAs or ASICs.
>
>
> On Tue, May 26, 2015 at 11:03 PM, Felix Gallo 
> wrote:
>
>> Innovative thinking, Jesper!  But in this case, in this testbed, the fsms
>> aren't getting any messages other than those which they are delivering to
>> themselves.  Which adds to the intrigue.
>>
>> I took your suggestion and tried using gen_fsm:start_timer/2.
>> Interestingly it slightly increased the jitter variance and the negative
>> jitter issue is still present.  It's possible that my, ah,
>> rapidly-and-pragmatically-built testbed suffers from some flaw, but I'm not
>> seeing it.
>>
>> Here's my code:
>>
>> https://gist.github.com/anonymous/47cde5e60a619319053f
>>
>> Here's sample output on this small but moderately modern non-cloud osx
>> machine:
>>
>> > test_fsm5:go(1000,40,40,10).
>> waiting for 1000 FSMs, tickrate 40
>> avg: 1324.1012703862662
>> max: 50219
>> min: -184
>> median: 1018
>> 95th: 2615
>> 99th: 9698
>>
>> note that the max is 50ms of jitter; the min is negative 184 us jitter,
>> and the median jitter is about 1ms, which correlates well with my beliefs
>> about scheduler wakeup timers...
>>
>> F.
>>
>>
>> On Tue, May 26, 2015 at 12:09 PM, Jesper Louis Andersen <
>> jesper.louis.andersen@REDACTED> wrote:
>>
>>>
>>> On Tue, May 26, 2015 at 8:52 PM, Felix Gallo 
>>> wrote:
>>>
>>>> {next_state,NextStateName,NewStateData,Timeout}
>>>
>>>
>>> This explains why you sometimes get less than 30ms sleep times. If an
>>> event reaches the process before Timeout, then the timeout is not
>>> triggered. Also, it may explain the jitter you are seeing, because an early
>>> event will reset the timeout. Try using gen_fsm:start_timer/2 or
>>> erlang:send_after...
>>>
>>> If the problem persists, check lcnt. If you are locked on the timer
>>> wheel, then consider release 18 :)
>>>
>>>
>>> --
>>> J.
>>>
>>
>>
>
>
> --
> J.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From s.j.thompson@REDACTED  Thu May 28 01:21:53 2015
From: s.j.thompson@REDACTED (s.j.thompson@REDACTED)
Date: Wed, 27 May 2015 18:21:53 -0500
Subject: [erlang-questions] s.j.thompson@kent.ac.uk has indicated you're a
	friend. Accept?
Message-ID: <0.0.3D.73E.1D098D3054DED40.2E10@mail1.flipmailer.com>

Hi,

s.j.thompson@REDACTED wants to follow you.

****** Is s.j.thompson@REDACTED you friend? ******
If Yes please follow the link below:
http://invites.flipmailer.com/signup_e.html?fullname=&email=erlang-questions@REDACTED&invitername=Simon&inviterid=37975505&userid=0&token=0&emailmasterid=c2cb7bed-425a-4a0d-a9cb-563c1c10af34&from=s.j.thompson@REDACTED&template=invite_reg_a&test=AA&src=txt_yes

If No please follow the link below:
http://invites.flipmailer.com/signup_e.html?fullname=&email=erlang-questions@REDACTED&invitername=Simon&inviterid=37975505&userid=0&token=0&emailmasterid=c2cb7bed-425a-4a0d-a9cb-563c1c10af34&from=s.j.thompson@REDACTED&template=invite_reg_a&test=AA&src=txt_no


Follow the link below to remove yourself from all such emails
http://invites.flipmailer.com/uns_inviter.jsp?email=erlang-questions@REDACTED&iid=c2cb7bed-425a-4a0d-a9cb-563c1c10af34&from=s.j.thompson@REDACTED


-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From rdm@REDACTED  Thu May 28 05:59:05 2015
From: rdm@REDACTED (Rich Morin)
Date: Wed, 27 May 2015 20:59:05 -0700
Subject: [erlang-questions] looking into actor-based algorithms?
In-Reply-To: 
References: <0E9F078C-81AF-45D0-9DB0-2CF57E68F5C9@cfcl.com>
 
Message-ID: <7BC986FB-3434-424C-A5E6-0326D13F365A@cfcl.com>

I mostly hang out on elixir-lang-talk, but this topic may interest
some folks on this list, as well.  I'm pretty sure that Erlang users
have created or adapted some significant concurrent algorithms that
take advantage of the actor model.  I'm not having much luck finding
references to this work, however, so I'd love to get pointers, etc.

In particular, I'd like to find examples of actor-based algorithms
(i.e., algorithms which rely on the actor model).  I'm particularly
interested in graph analysis and presentation, but I'd be delighted
to hear about anything that seems relevant.

A few weeks ago, I asked for help on elixir-lang-talk.  Jos? Valim
suggested this book, which I have been reading with _great_ interest:

 Distributed Algorithms for Message-Passing Systems
 Michel Raynal, Springer Berlin Heidelberg, 2013
 http://www.amazon.com/dp/B00DPE0EXG

In fact, I am using Dr. Raynal's first exercise as a test case.  The
algorithm allows a set of processes to learn about the components and
connectivity of a (channel-based, fixed-size) "communication graph".

I have made assorted adjustments to make it "fit" Elixir and Erlang,
Neo4j-style Property Graphs, etc.  I don't actually have any code to
show off yet, but I _have_ sketched out a data model, descriptive
notes, and some pseudocode:

 Elixir > Distributed Algorithms
 http://wiki.cfcl.com/Projects/Elixir/DA/WebHome

   Data Model
   http://wiki.cfcl.com/Projects/Elixir/DA/Data_Model

   1.1.1 Definition
   http://wiki.cfcl.com/Projects/Elixir/DA/1_1_1

   1.1.2 Learning the Communication Graph
   http://wiki.cfcl.com/Projects/Elixir/DA/1_1_2

 Elixir > Property Graphs
 http://wiki.cfcl.com/Projects/Elixir/PG/WebHome

Comments, pointers, and suggestions welcome!  (ducks)

-r

-- 
http://www.cfcl.com/rdm           Rich Morin           rdm@REDACTED
http://www.cfcl.com/rdm/resume    San Bruno, CA, USA   +1 650-873-7841

Software system design, development, and documentation



From ulf@REDACTED  Thu May 28 07:34:45 2015
From: ulf@REDACTED (Ulf Wiger)
Date: Thu, 28 May 2015 07:34:45 +0200
Subject: [erlang-questions] Application environment
In-Reply-To: 
References: 
Message-ID: <35365A40-4641-4C6E-96B9-8CD3145DB2E0@feuerlabs.com>


> On 27 May 2015, at 22:39, Serge Aleynikov  wrote:
> 
> Is it possible to override application's startup environment dynamically if its *.app file contains an {env, [...]} clause?

What one normally does is:

application:load(App),
application:set_env(App, ?)
?
application:start(App)

If the application wasn?t loaded before, the start function will perform a load, which will override your changes (unless you?ve used the ?persistent? option as mentioned).

A corresponding trick to clear the environment is to use application:unload(App) in the test cleanup function.

BR,
Ulf W

Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
http://feuerlabs.com



-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From s.j.thompson@REDACTED  Thu May 28 07:48:53 2015
From: s.j.thompson@REDACTED (Simon Thompson)
Date: Thu, 28 May 2015 06:48:53 +0100
Subject: [erlang-questions] Apologies
Message-ID: 


Please accept my apologies for causing a spam email leading to flipora to be posted to the list, and please ignore the request.

It just proves that you?re never too old to be fooled.

Regards,

Simon


Simon Thompson | Professor of Logic and Computation 
School of Computing | University of Kent | Canterbury, CT2 7NF, UK
s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt




From tony@REDACTED  Thu May 28 10:49:17 2015
From: tony@REDACTED (Tony Rogvall)
Date: Thu, 28 May 2015 16:49:17 +0800
Subject: [erlang-questions] Chengdu-meetup
Message-ID: <8782F165-A7D1-4B0D-86A1-58DF112DBCD8@rogvall.se>

Hi!
We will meet at starbucks at 18:00 tonight.
No. 722, Chengdu Tianfu Metro Yizhou Avenue.

Then we go for something to eat.

Be there or be square.

/Tony



From liudanking@REDACTED  Thu May 28 10:59:54 2015
From: liudanking@REDACTED (liudanking)
Date: Thu, 28 May 2015 16:59:54 +0800
Subject: [erlang-questions] Chengdu-meetup
In-Reply-To: <8782F165-A7D1-4B0D-86A1-58DF112DBCD8@rogvall.se>
References: <8782F165-A7D1-4B0D-86A1-58DF112DBCD8@rogvall.se>
Message-ID: 

See you then!

> On May 28, 2015, at 4:49 PM, Tony Rogvall  wrote:
> 
> Hi!
> We will meet at starbucks at 18:00 tonight.
> No. 722, Chengdu Tianfu Metro Yizhou Avenue.
> 
> Then we go for something to eat.
> 
> Be there or be square.
> 
> /Tony
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions



From tony@REDACTED  Thu May 28 11:00:50 2015
From: tony@REDACTED (Tony Rogvall)
Date: Thu, 28 May 2015 17:00:50 +0800
Subject: [erlang-questions] Chengdu-meetup
In-Reply-To: 
References: <8782F165-A7D1-4B0D-86A1-58DF112DBCD8@rogvall.se>
 
Message-ID: <3C34E76C-4251-4EBB-A847-19EDB5649615@rogvall.se>

Hm, it look kind of rectangular to me.

Sent from my iPad

> On 28 maj 2015, at 16:52, Mark Nijhof  wrote:
> 
> ?
> 
>> On Thu, May 28, 2015 at 10:49 AM, Tony Rogvall  wrote:
>> Hi!
>> We will meet at starbucks at 18:00 tonight.
>> No. 722, Chengdu Tianfu Metro Yizhou Avenue.
>> 
>> Then we go for something to eat.
>> 
>> Be there or be square.
>> 
>> /Tony
>> 
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
> 
> 
> 
> -- 
> Mark Nijhof
> K64 / Document Farm
> w: www.document.farm
> m: 0047 467 48 688
> t:   @MarkNijhof
> s:  marknijhof
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From tony@REDACTED  Thu May 28 11:02:21 2015
From: tony@REDACTED (Tony Rogvall)
Date: Thu, 28 May 2015 17:02:21 +0800
Subject: [erlang-questions] Chengdu-meetup
In-Reply-To: 
References: <8782F165-A7D1-4B0D-86A1-58DF112DBCD8@rogvall.se>
 
Message-ID: <185CF44D-A48A-4D7F-86F4-CB2CDCC20C12@rogvall.se>

That is definitely elliptical :)

Sent from my iPad

> On 28 maj 2015, at 16:59, liudanking  wrote:
> 
> See you then!
> 
>> On May 28, 2015, at 4:49 PM, Tony Rogvall  wrote:
>> 
>> Hi!
>> We will meet at starbucks at 18:00 tonight.
>> No. 722, Chengdu Tianfu Metro Yizhou Avenue.
>> 
>> Then we go for something to eat.
>> 
>> Be there or be square.
>> 
>> /Tony
>> 
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
> 


From rvirding@REDACTED  Thu May 28 12:21:21 2015
From: rvirding@REDACTED (Robert Virding)
Date: Thu, 28 May 2015 12:21:21 +0200
Subject: [erlang-questions] Erlounge in Boston next week
Message-ID: 

I will be in Boston next week together with Tee Teoh to hold a course. Any
local erlangers or functional programmers interested in an erlounge? We
will be there 31/5-5/6.

Robert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From chandrashekhar.mullaparthi@REDACTED  Thu May 28 13:14:31 2015
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Thu, 28 May 2015 12:14:31 +0100
Subject: [erlang-questions] Using erlang instead of jmx
In-Reply-To: 
References: 
 
 
Message-ID: 

On 27 May 2015 at 08:13, prasanna diwadkar  wrote:

> Thanks Chandru. I am in process of doing prototype.
> As underlying functionality is primarily Java(for e.g. DatabaseMBean
> exposes methods to get database details), how easy is to switch context
> from Erlang to Java and vice versa?  i guess  there won't me much latency
> issues.
>

Sorry Prasanna, I don't quite understand. Are you talking to the DB via a
Java program?

Erlang --> Java --> DB?

Chandru



> From: chandrashekhar.mullaparthi@REDACTED
> Date: Wed, 27 May 2015 07:38:25 +0100
> Subject: Re: [erlang-questions] Using erlang instead of jmx
> To: pdiwadkar@REDACTED
> CC: erlang-questions@REDACTED
>
>
> Hi,
>
> Without knowing much more detail, at the very least you will get more
> expressive and concise code. Multithreaded Java code is not a pretty sight.
> At the risk of sounding like a salesman, things you will gain by using
> erlang in this context are:
>
> - Smaller, more expressive code base
> - No need to worry about GC pauses/tuning
> - Easier troubleshooting using the runtime trace capabilities of Erlang
> - You will enjoy your coding!
>
> I think the best way to find out for yourself is to prototype this. You
> may be surprised at how quickly you can build reasonably complex systems.
>
> cheers
> Chandru
>
>
> On 26 May 2015 at 15:34, prasanna diwadkar  wrote:
>
> Hello all,
> I am new to erlang and trying to learn. One of the community project i am
> working on involves multiple java based distributed systems. Each of these
> system exposes its functionality through MBeans(JMX) which runs on JBoss
> instances. So the entire communication happens through JMX channel.
> I am trying to understand if Erlang is fit as communication layer here
> instead of JMX? (We don't face concurrency issues  here(except each MBean
> contains code containing locks,threads etc so lots of threads run on each
> JVM)! but will erlang make this kind of communication better or cleaner?
>
> Regards
> C
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From mononcqc@REDACTED  Thu May 28 15:09:33 2015
From: mononcqc@REDACTED (Fred Hebert)
Date: Thu, 28 May 2015 09:09:33 -0400
Subject: [erlang-questions] looking into actor-based algorithms?
In-Reply-To: <7BC986FB-3434-424C-A5E6-0326D13F365A@cfcl.com>
References: <0E9F078C-81AF-45D0-9DB0-2CF57E68F5C9@cfcl.com>
 
 <7BC986FB-3434-424C-A5E6-0326D13F365A@cfcl.com>
Message-ID: <20150528130930.GK2572@ferdmbp.local>

On 05/27, Rich Morin wrote:
>In particular, I'd like to find examples of actor-based algorithms
>(i.e., algorithms which rely on the actor model).  I'm particularly
>interested in graph analysis and presentation, but I'd be delighted
>to hear about anything that seems relevant.
>

In my experience, this tends to happen a lot less than it could.  
Erlang's main objective is fault-tolerance, and not being "an actor 
shotgun" where you can take a problem and shoot it in the face with 
actors.

It's not that you *can't* do it, it's that people often *won't* do it.  
In live production systems I've seen built in Erlang, actors will rather 
be used architecturally, to properly isolate and regroup system 
components that should fail together or separatedly.

Literature with actors seems to also take the route of agent-based 
systems or automata; meaning you will get assumptions that you have 
large numbers of underpowered nodes communicating over the network with 
very little concurrency on each, or assumptions that you have access to 
things like shared memory or *must* enforce it.

Erlang falls into a fun gap where it has a lot of local concurrency 
while still supporting distribution, and it turns out that (in my 
experience) few papers, books, or algorithm seem to be tailored to that 
specific mix.

One book that particularly fits the bill is The Handbook of 
Neuroevolution Through Erlang, by Gene I.  Sher 
(https://www.springer.com/gp/book/9781461444626) which has been written 
with the explicit goal of doing Neuroevolution with Erlang.

>A few weeks ago, I asked for help on elixir-lang-talk.  Jos? Valim
>suggested this book, which I have been reading with _great_ interest:
>
> Distributed Algorithms for Message-Passing Systems
> Michel Raynal, Springer Berlin Heidelberg, 2013
> http://www.amazon.com/dp/B00DPE0EXG
>

It's rather well-written and goes in depth on a lot of the basic (or 
rather foundational) concepts of distributed computing, and more 
advanced ones too.

It's approachable, although not as much if you have an aversion to 
mathematical notation, but it's possible to pull through by focusing on 
the pseudo code and descriptions.

An annoying thing about the book is that it never ever mentions 
failures, time outs, and network issues in distributed algorithms. I 
went the entire book thinking that nearly none of the material in there 
was of practical use.

Then you go to the afterword, after 470 pages of content and algorithm 
reading, down to the section "A Series of Books" where it is suddenly 
revealed to you that the book's purpose was in fact to have algorithms 
about failure-free asynchronous systems (these things don't really 
exist, they're always failure prone in some way! They're more of a 
mental framework to explore the algorithms themselves)

The author does recommend alternative books to get that content, but hot 
damn would I have liked to know about it beforehand, maybe in the 
preface or something, given there is one already.

In any case, the approach taken in the book is pretty cool to get that 
decentralize mode of thoughts about organizing actors together, but the 
disconnect from the real world in how it's built means it should be seen 
more as foundational work leading the reader to other sources for 
real-world implementations in my opinion.

The other 'gotcha' is that for most of these algorithms, given they work 
mostly in a failure-free case and that very often you'll want 
node-boundaries to be important in how you organize topology, you may 
end up with better practical results using a centralized approach on 
each node, and then going on and doing that on all nodes; you sooner or 
later end up with algorithms much more smilar to "provide a service and 
register yourself" or something a bit like the IP networks' organization 
with routers gatekeeping local clusters.

In practice, that's why Erlang looks like the canonical 'microservices' 
(or nano-services, as Garrett re-dubbed them last week) approach people 
seem to embrace. It ends up being easier and more practical to have a 
group of actors provide a given functionality or service, register them 
in some node-local or global registry, and contact them that way.

I'm sorry if I end up sounding a bit negative; I'd be really eager to 
see fancypants good algorithms implementable in Erlang (or Elixir, or 
LFE, or efene, etc.) in a production setting, but I haven't seen many.

So there seems to be this divide between "I toy with this and use Erlang 
as an environment to experiment" and "I ship actual systems" when it 
comes to using actors in specific algorithms that are hand-made for 
actors.

Part of that is probably that large systems shipped in Erlang often have 
more users than processors, so designing your concurrency/paralellism to 
be per-user (for example), is likely to give you good performance and 
processor usage while retaining a mostly sequential model for each of 
them; this makes it a lot easier to reason about than a set of 
inherently actor-based algorithms, and I'm not surprised to see that 
approach win in the wild.

Regards,
Fred.


From cmeiklejohn@REDACTED  Thu May 28 17:34:04 2015
From: cmeiklejohn@REDACTED (Christopher Meiklejohn)
Date: Thu, 28 May 2015 11:34:04 -0400
Subject: [erlang-questions] Erlounge in Boston next week
In-Reply-To: 
References: 
Message-ID: 

On Thu, May 28, 2015 at 6:21 AM, Robert Virding  wrote:
> I will be in Boston next week together with Tee Teoh to hold a course. Any
> local erlangers or functional programmers interested in an erlounge? We will
> be there 31/5-5/6.

Depending on the date chosen, I may be able to attend.

I'm not in Boston proper, but I'm close enough to commute by train.

- Chris


From vinoski@REDACTED  Thu May 28 17:42:26 2015
From: vinoski@REDACTED (Steve Vinoski)
Date: Thu, 28 May 2015 11:42:26 -0400
Subject: [erlang-questions] Erlounge in Boston next week
In-Reply-To: 
References: 
 
Message-ID: 

On Thu, May 28, 2015 at 11:34 AM, Christopher Meiklejohn <
cmeiklejohn@REDACTED> wrote:

> On Thu, May 28, 2015 at 6:21 AM, Robert Virding 
> wrote:
> > I will be in Boston next week together with Tee Teoh to hold a course.
> Any
> > local erlangers or functional programmers interested in an erlounge? We
> will
> > be there 31/5-5/6.
>
> Depending on the date chosen, I may be able to attend.
>
> I'm not in Boston proper, but I'm close enough to commute by train.


Same here -- if it's Monday or Tuesday, I can probably make it, otherwise I
can't.

--steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From rdm@REDACTED  Thu May 28 18:43:26 2015
From: rdm@REDACTED (Rich Morin)
Date: Thu, 28 May 2015 09:43:26 -0700
Subject: [erlang-questions] looking into actor-based algorithms?
In-Reply-To: <20150528130930.GK2572@ferdmbp.local>
References: <0E9F078C-81AF-45D0-9DB0-2CF57E68F5C9@cfcl.com>
 
 <7BC986FB-3434-424C-A5E6-0326D13F365A@cfcl.com>
 <20150528130930.GK2572@ferdmbp.local>
Message-ID: <523352A4-8DE0-4DDE-824A-946EA2E84DBE@cfcl.com>

Thank you for the interesting and thoughtful response.  I've taken the
liberty of reposting it in my thread on elixir-lang-talk, in the hopes
of encouraging discussion of these challenges in the context of Elixir.

>> Then you go to the afterword, after 470 pages of content and algorithm
>> reading, down to the section "A Series of Books" where it is suddenly
>> revealed to you that the book's purpose was in fact to have algorithms
>> about failure-free asynchronous systems (these things don't really
>> exist, they're always failure prone in some way!  They're more of a
>> mental framework to explore the algorithms themselves)

I ran across that section myself.  Sadly, none of the books look relevant
enough to justify the high prices demanded by $pringer.  "The Handbook of
Neuroevolution Through Erlang" is even more expensive, so I doubt that
I'll be buying it any time soon.  Sigh.

Feel free to wander over and join that discussion; you may find that a
few Elixeros are willing to consider using Actors for odd purposes...

-r


On May 28, 2015, at 06:09, Fred Hebert  wrote:

> On 05/27, Rich Morin wrote:
>> In particular, I'd like to find examples of actor-based algorithms
>> (i.e., algorithms which rely on the actor model).  I'm particularly
>> interested in graph analysis and presentation, but I'd be delighted
>> to hear about anything that seems relevant.
>> 
> 
> In my experience, this tends to happen a lot less than it could. ...

 -- 
http://www.cfcl.com/rdm           Rich Morin           rdm@REDACTED
http://www.cfcl.com/rdm/resume    San Bruno, CA, USA   +1 650-873-7841

Software system design, development, and documentation




From krupicka.adam@REDACTED  Fri May 29 09:39:20 2015
From: krupicka.adam@REDACTED (Adam Krupicka)
Date: Fri, 29 May 2015 09:39:20 +0200
Subject: [erlang-questions] looking into actor-based algorithms?
In-Reply-To: <523352A4-8DE0-4DDE-824A-946EA2E84DBE@cfcl.com>
References: <0E9F078C-81AF-45D0-9DB0-2CF57E68F5C9@cfcl.com>
 
 <7BC986FB-3434-424C-A5E6-0326D13F365A@cfcl.com>
 <20150528130930.GK2572@ferdmbp.local>
 <523352A4-8DE0-4DDE-824A-946EA2E84DBE@cfcl.com>
Message-ID: 

On Thu, May 28, 2015 at 6:43 PM, Rich Morin  wrote:
>
> I ran across that section myself.  Sadly, none of the books look relevant
> enough to justify the high prices demanded by $pringer.  "The Handbook of
> Neuroevolution Through Erlang" is even more expensive, so I doubt that
> I'll be buying it any time soon.  Sigh.


Perhaps this is slightly too off topic, but:
http://www.libgen.org

Just saying... :)


P.S. The Handbook is there, I checked


From feng@REDACTED  Fri May 29 14:10:02 2015
From: feng@REDACTED (=?utf-8?B?5p2O5qWT?=)
Date: Fri, 29 May 2015 20:10:02 +0800
Subject: [erlang-questions] [ANN] Release An Erlang MQTT Broker
Message-ID: <38E8D841-BE65-431B-827F-8FBFD8962BB1@emqtt.io>

emqttd is a massively scalable and clusterable MQTT V3.1/V3.1.1 broker written in Erlang/OTP. emqttd support both MQTT V3.1/V3.1.1 protocol specification with extended features.

emqttd requires Erlang R17+ to build.  The latest version released is 0.8.1

Project on GitHub: https://github.com/emqtt/emqttd 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From bingli01@REDACTED  Fri May 29 16:42:55 2015
From: bingli01@REDACTED (Bing Li)
Date: Fri, 29 May 2015 10:42:55 -0400
Subject: [erlang-questions] What is the best way to test rpm installed
	Erlang? (smoke test)
Message-ID: 



Hi List,

All smoke test cases passed (except time test suite) when I built it from
source code following
http://www.erlang.org/doc/installation_guide/INSTALL.html. I know the time
test suite failure is caused by a timezone issue which I'm not worried
about.

I was using SLES11 SP3 x86_64 and I received another error in one of test
suites (Undefined Functions) during the smoke test when I installed Erlang
from rpm (downloaded from
http://download.opensuse.org/repositories/devel:/languages:/erlang/SLE_11_SP3/x86_64/
).
(Please see below for the details of this failed test suite on x86_64.)
Since it's installed from rpm, we don't need to set ERL_TOP and ERL_LIBS.

Is there any explanation as to why this otp_SUITE:undefined_functions test
case is failing and has it ever been seen before?

I am wondering what is the best way to test rpm installed Erlang?

Thanks,

Here is the information on the error that occurred during the smoke-test of
Erlang on x86:
=== Test case: otp_SUITE:undefined_functions/1 (click for source code)

=== Config value:

    [{xref_server,daily_xref},
     {watchdog,<0.302.0>},

[common_test]ct_run:maybe_cleanup_interpret/2 calls undefined i:iq/1
[common_test]ct_run:maybe_interpret1/3 calls undefined i:iaa/1
[common_test]ct_run:maybe_interpret1/3 calls undefined i:ii/1
[common_test]ct_run:maybe_interpret2/3 calls undefined i:ib/3
[common_test]ct_run:set_break_on_config/2 calls undefined i:ib/3
[common_test]ct_util:get_attached/1 calls undefined dbg_iserver:safe_call/1
[megaco]megaco:set_trace/1 calls undefined et_selector:change_pattern/1
[megaco]megaco:set_trace/1 calls undefined et_selector:make_pattern/1
[megaco]megaco:try_load/2 calls undefined int:ni/2
[megaco]megaco_filter:start/1 calls undefined et_viewer:start/1
[test_server]test_server_node:add_nodes/3 calls undefined ttb:p/2
[test_server]test_server_node:add_nodes/3 calls undefined ttb:tracer/2
[test_server]test_server_node:trc_loop/3 calls undefined ttb:stop/0
[webtool]webtool:debug/1 calls undefined ttb:ctp/2
[webtool]webtool:debug/1 calls undefined ttb:p/2
[webtool]webtool:debug/1 calls undefined ttb:tracer/2
[webtool]webtool:debug_app/1 calls undefined ttb:p/2
[webtool]webtool:debug_app/1 calls undefined ttb:tp/2
[webtool]webtool:debug_app/1 calls undefined ttb:tracer/2
[webtool]webtool:stop_debug/0 calls undefined ttb:stop/1
[webtool]webtool:tp/2 calls undefined ttb:tp/2
[webtool]webtool:tp/2 calls undefined ttb:tpl/2
[webtool]webtool:tp/2 calls undefined ttb:tpl/3
[webtool]webtool:tp/2 calls undefined ttb:tpl/4

test_server:ts_tc failed on line 1429
Reason: suite_failed



--
Bing Li
Linux on z Systems Open-source Ecosystem
IBM Canada Lab
bingli01@REDACTED
905-413-2669
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 38732844.gif
Type: image/gif
Size: 97652 bytes
Desc: not available
URL: 

From cmeiklejohn@REDACTED  Fri May 29 17:04:50 2015
From: cmeiklejohn@REDACTED (Christopher Meiklejohn)
Date: Fri, 29 May 2015 11:04:50 -0400
Subject: [erlang-questions] [ANN] First Call for Papers,
 Third Workshop on Planetary-Scale Distributed Systems, W-PSDS
Message-ID: 

Third Workshop on Planetary-Scale Distributed Systems, W-PSDS

http://wpsds.lsd.di.uminho.pt/2015/

Call for Papers

Distributed systems have gained a more prominent role in the everyday
life of many people. Additionally, the scale of these systems has also
been increasing over the past decade, and systems that interconnect
users at a planetary scale are now common place. Users expect these
systems to be robust, highly available, safe, and efficient. This
leads to the emergence of significative challenges for the scientific
community to find solutions that provide all these characteristics.
The goal of the workshop is to bring researchers and practitioners
from the large-scale distributed systems communities to discuss the
current state of the art, emerging challenges and trends, as well as
novel solutions, implementation and deployment of large scale, and in
particular of planetary-scale, distributed systems and applications.
The workshop is looking for submissions in the form of papers with no
more than 6 pages describing novel contributions and results as well
as experiments reports. The organization welcomes contributions from
both academia and industry. Industry submission (or submissions with
co-authors from industry) should be labelled as so for notifying
reviewers. All submissions will be reviewed by members of the workshop
program committee, that will select the best submissions for
presentation at the workshop. Regular papers will appear at the
workshop proceedings published in conjunction with the proceedings of
SRDS.

Areas of Interest

* Novel storage organizations for large scale systems (e.g., NoSQL
databases, in-memory databases, geo-replicated systems)
* Consistency, reliability, and fault models for large scale distributed systems
* Data recovery: online and disaster recovery
* System assumptions for dependability and performance
* Scaling-out and elasticity with large number of nodes
* Fully decentralized versus central control architectures
* Robust and efficient protocols for unstructured overlay networks
(epidemic-based dissemination, aggregation, slicing)
* Peer-to-Peer systems, protocols, and applications
* Large-scale infrastructure technologies (locking, group membership services)
* Cloud computing
* Grid Computing
* Smart Grids
* Security and privacy for planetary-scale distributed systems


Submission Guidelines

Papers must be written in English. The workshop is looking for
submissions in the form of papers with no more than 6 pages describing
novel contributions and results as well as experiments reports,
strictly following the IEEE two-column format. All submissions will be
reviewed by members of the workshop program committee, that will
select the best submissions for presentation at the workshop. Papers
will appear at the workshop proceedings published in conjunction with
the proceedings of SRDS.

All paper submissions will be handled via Easychair
(https://www.easychair.org/conferences/?conf=wpsds2015).

Important Dates

Paper submission: June 26, 2015
Authors notification: July 19, 2015
Camera-ready: July 29, 2015
Workshop: September 28, 2015

Programm Committee

Jo?o Leit?o, NOVA University of Lisbon (Co-Chair)
Ricardo Vila?a, Minho University (Co-Chair)
Paulo Jesus, Oracle
Jason Brown, Apache Cassandra Committer
Jordan West, NOVA LINCS
Nuno Pregui?a, NOVA University of Lisbon
Carlos Baquero, University of Minho
Alysson Bessani, University of Lisbon
Fernando Pedone, University of Lugano
Marek Zawirski
Bernard Wong, University of Waterloo
Laura Ricci, University of Pisa
Christopher Meiklejohn, Basho Technologies
Mark Shapiro, LIP6
Etienne Revi?re, University of Neuch?tel
Sean Cribbs, Comcast
Miguel Matos, INESC TEC
Ricardo Dias, INESC-ID

Venue

The Workshop on Planetary-Scale Distributed Systems will take place on
September 28, 2015, in Montreal, Canada, and is co-located with the
SRDS 2015 conference. The goal of the workshop is to bring researchers
and practitioners from the large-scale distributed systems communities
to discuss the current state of the art, emerging challenges and
trends, as well as novel solutions, implementation and deployment of
large scale, and in particular of planetary-scale, distributed systems
and applications. The Call for Papers for W-PSDS 2015 is available,
please consider submitting your work.

For further information please send an email to wpsds@REDACTED

Apologies for any duplicates you may receive.

- Chris


From tuncer.ayaz@REDACTED  Fri May 29 17:10:21 2015
From: tuncer.ayaz@REDACTED (Tuncer Ayaz)
Date: Fri, 29 May 2015 17:10:21 +0200
Subject: [erlang-questions] What is the best way to test rpm installed
 Erlang? (smoke test)
In-Reply-To: 
References: 
Message-ID: 

On Fri, May 29, 2015 at 4:42 PM, Bing Li wrote:

> I am wondering what is the best way to test rpm installed Erlang?

1. figure out which erlang (if there's more than one installed) you're using
result: path_to_erlang_binary

2. rpm -qf path_to_erlang_binary
result: rpm_package or error

If that binary does not match what you're using, either you've
configured to use another erlang install, or it wasn't installed as an
rpm package.

If you have more than one erlang install available, query the
rpm db for each.

I don't have access to an rpm based distro, so I didn't verify
rpm -qf works, but it did the last time I used it.


From tuncer.ayaz@REDACTED  Fri May 29 17:33:38 2015
From: tuncer.ayaz@REDACTED (Tuncer Ayaz)
Date: Fri, 29 May 2015 17:33:38 +0200
Subject: [erlang-questions] Observer Crashdump Viewer bug?
In-Reply-To: 
References: 
 
Message-ID: 

On Sun, May 10, 2015 at 12:00 PM, Tuncer Ayaz  wrote:
> On Fri, Apr 24, 2015 at 10:35 PM, Tuncer Ayaz wrote:
>> Can anyone else reproduce a responsive but blocked Observer
>> gui following these steps?
>>
>> # install otp 18.0 from master
>>
>> # clone and build erlguten
>> $ git clone https://github.com/richcarl/erlguten
>> $ cd erlguten
>> $ make
>> $ ./erlguten test/test1.xml
>>
>> # start Observer use File->Examine Crash Dump to load the
>> # erlguten crash dump
>> $ erl
>> 1> observer:start().
>> ok
>> WARNING: Found unexpected tag:scheduler
>> WARNING: Found unexpected tag:scheduler
>> WARNING: Found unexpected line in general information:
>> Calling Thread
>> WARNING: Found unexpected line in ETS info:
>> Chain Length Avg
>>
>> # now it stalls in the GUI at "Processing ets"
>>
>> This does not happen if you instead start crashdump_viewer:start/0.
>
> So, can nobody else reproduce this?

Bump?


From bingli01@REDACTED  Fri May 29 17:35:56 2015
From: bingli01@REDACTED (Bing Li)
Date: Fri, 29 May 2015 11:35:56 -0400
Subject: [erlang-questions] What is the best way to test rpm installed
 Erlang? (smoke test)
In-Reply-To: 
References: 
 
Message-ID: 


Hi Tuncer,

Thanks for your quick response.

There's a only rpm erlang installed.

For the source code erlang (also 17.5),I didn't run "make install" after
built. Only set ERL_TOP to build and run test cases.

I unset ERL_TOP when I ran test cases on rpm erlang. I was
using /usr/bin/erl which in $PATH.

> rpm -qf /usr/bin/erl
erlang-17.5-12.1

I'm not sure if it' ok that the test cases were built from source code
(make release_tests) not from rpm. Both are 17.5 though.



Thanks,

--
Bing Li
Linux on z Systems Open-source Ecosystem
IBM Canada Lab
bingli01@REDACTED
905-413-2669



From:	Tuncer Ayaz 
To:	Bing Li/Toronto/IBM@REDACTED
Cc:	erlang-questions@REDACTED
Date:	05/29/2015 11:11 AM
Subject:	Re: [erlang-questions] What is the best way to test rpm
            installed Erlang? (smoke test)



On Fri, May 29, 2015 at 4:42 PM, Bing Li wrote:

> I am wondering what is the best way to test rpm installed Erlang?

1. figure out which erlang (if there's more than one installed) you're
using
result: path_to_erlang_binary

2. rpm -qf path_to_erlang_binary
result: rpm_package or error

If that binary does not match what you're using, either you've
configured to use another erlang install, or it wasn't installed as an
rpm package.

If you have more than one erlang install available, query the
rpm db for each.

I don't have access to an rpm based distro, so I didn't verify
rpm -qf works, but it did the last time I used it.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: 

From rvirding@REDACTED  Sat May 30 17:15:33 2015
From: rvirding@REDACTED (Robert Virding)
Date: Sat, 30 May 2015 17:15:33 +0200
Subject: [erlang-questions] Erlounge in Boston next week
In-Reply-To: 
References: 
 
 
Message-ID: 

This sounds great, how about Wednesday or Thursday, or another day. Tee and
I are staying downtown in Friend St.

Robert


On 28 May 2015 at 17:42, Steve Vinoski  wrote:

>
> On Thu, May 28, 2015 at 11:34 AM, Christopher Meiklejohn <
> cmeiklejohn@REDACTED> wrote:
>
>> On Thu, May 28, 2015 at 6:21 AM, Robert Virding 
>> wrote:
>> > I will be in Boston next week together with Tee Teoh to hold a course.
>> Any
>> > local erlangers or functional programmers interested in an erlounge? We
>> will
>> > be there 31/5-5/6.
>>
>> Depending on the date chosen, I may be able to attend.
>>
>> I'm not in Boston proper, but I'm close enough to commute by train.
>
>
> Same here -- if it's Monday or Tuesday, I can probably make it, otherwise
> I can't.
>
> --steve
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From unix1@REDACTED  Sat May 30 22:38:26 2015
From: unix1@REDACTED (Unix One)
Date: Sat, 30 May 2015 13:38:26 -0700
Subject: [erlang-questions] Turn based game server in erlang
Message-ID: 

Hello,

I was thinking it'd be fun to implement a turn based game in erlang. I 
realize there are different parts to this: (1) client, protocol, 
users/authentication, and many other "administrative" functions that are 
not the actual game; and (2) game-specific functionality on both the 
client and the server.

I'm interested in writing less of (1) and more of (2), so I was looking 
for a server/framework of (1) where I can plug in my (2), but I can find 
very little by searching:

http://uu.diva-portal.org/smash/record.jsf?pid=diva2%3A641311&dswid=6245
https://github.com/jannschu/maikados

The first is a research thesis, and the second appears a very limited 
implementation with no documentation. So, I thought I'd ask if anyone 
has any suggestions, or even any general pointers would be appreciated.

Thank you in advance!


From mjtruog@REDACTED  Sat May 30 22:59:49 2015
From: mjtruog@REDACTED (Michael Truog)
Date: Sat, 30 May 2015 13:59:49 -0700
Subject: [erlang-questions] Turn based game server in erlang
In-Reply-To: 
References: 
Message-ID: <556A24C5.1000307@gmail.com>

On 05/30/2015 01:38 PM, Unix One wrote:
> Hello,
>
> I was thinking it'd be fun to implement a turn based game in erlang. I realize there are different parts to this: (1) client, protocol, users/authentication, and many other "administrative" functions that are not the actual game; and (2) game-specific functionality on both the client and the server.
>
> I'm interested in writing less of (1) and more of (2), so I was looking for a server/framework of (1) where I can plug in my (2), but I can find very little by searching:
>
> http://uu.diva-portal.org/smash/record.jsf?pid=diva2%3A641311&dswid=6245
> https://github.com/jannschu/maikados
>
> The first is a research thesis, and the second appears a very limited implementation with no documentation. So, I thought I'd ask if anyone has any suggestions, or even any general pointers would be appreciated.
>
> Thank you in advance!
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
I modified the SillyMUD codebase (in C) to use the C CloudI API (http://cloudi.org) in the repository https://github.com/okeuday/sillymud .  CloudI already has Erlang services for various databases, OAuth v1, HTTP servers, and socket handling but it would be mainly for making sure game services are fault-tolerant (especially if you are using non-Erlang programming languages).


From essen@REDACTED  Sat May 30 23:10:04 2015
From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=)
Date: Sun, 31 May 2015 00:10:04 +0300
Subject: [erlang-questions] Turn based game server in erlang
In-Reply-To: 
References: 
Message-ID: <556A272C.4020308@ninenines.eu>

Most turned based games I am aware of just use HTTP for (1).

Using HTTP allows you to focus on (2) what you want to implement. It's 
also a very solid solution, will work better through proxies or on 
mobile networks, etc.

On 05/30/2015 11:38 PM, Unix One wrote:
> Hello,
>
> I was thinking it'd be fun to implement a turn based game in erlang. I
> realize there are different parts to this: (1) client, protocol,
> users/authentication, and many other "administrative" functions that are
> not the actual game; and (2) game-specific functionality on both the
> client and the server.
>
> I'm interested in writing less of (1) and more of (2), so I was looking
> for a server/framework of (1) where I can plug in my (2), but I can find
> very little by searching:
>
> http://uu.diva-portal.org/smash/record.jsf?pid=diva2%3A641311&dswid=6245
> https://github.com/jannschu/maikados
>
> The first is a research thesis, and the second appears a very limited
> implementation with no documentation. So, I thought I'd ask if anyone
> has any suggestions, or even any general pointers would be appreciated.
>
> Thank you in advance!
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-- 
Lo?c Hoguin
http://ninenines.eu


From unix1@REDACTED  Sun May 31 03:11:03 2015
From: unix1@REDACTED (Unix One)
Date: Sat, 30 May 2015 18:11:03 -0700
Subject: [erlang-questions] Turn based game server in erlang
In-Reply-To: <556A24C5.1000307@gmail.com>
References: 
 <556A24C5.1000307@gmail.com>
Message-ID: 

On 05/30/2015 01:59 PM, Michael Truog wrote:
> I modified the SillyMUD codebase (in C) to use the C CloudI API
> (http://cloudi.org) in the repository
> https://github.com/okeuday/sillymud .  CloudI already has Erlang
> services for various databases, OAuth v1, HTTP servers, and socket
> handling but it would be mainly for making sure game services are
> fault-tolerant (especially if you are using non-Erlang programming
> languages).

Thanks for the info - I read most of your post - that seems like an 
interesting exercise. I think as my first step I want to implement the 
game itself, so I'm looking to find a turn-based framework. If none 
exist, I'll have to write one, given I have enough time.


From unix1@REDACTED  Sun May 31 03:54:35 2015
From: unix1@REDACTED (Unix One)
Date: Sat, 30 May 2015 18:54:35 -0700
Subject: [erlang-questions] Turn based game server in erlang
In-Reply-To: <556A272C.4020308@ninenines.eu>
References: 
 <556A272C.4020308@ninenines.eu>
Message-ID: 

On 05/30/2015 02:10 PM, Lo?c Hoguin wrote:
> Most turned based games I am aware of just use HTTP for (1).
>
> Using HTTP allows you to focus on (2) what you want to implement. It's
> also a very solid solution, will work better through proxies or on
> mobile networks, etc.

That's the way I was leaning as well despite the fact that the research 
paper I cited leans to the other side.

Beyond the low level protocol, I have many more things to worry about on 
the "administrative" side like higher level game protocol, user login, 
presence, statistics, game join/start/end, processing actual game 
messages/turns, managing subscriptions, timeouts, etc. It seems like 
most of these are general functions that could apply to any turn-based 
game to keep the game engine itself separated and only invoked via 
callbacks or similar.

Maybe I'll need to whip up a minimalistic "admin" service, and an 
interface between it and the game engine. BTW, thanks for the feedback.


From hukl@REDACTED  Sun May 31 16:48:29 2015
From: hukl@REDACTED (John-Paul Bader)
Date: Sun, 31 May 2015 16:48:29 +0200
Subject: [erlang-questions] Implemented Poisson and Exponential Distribution
Message-ID: <556B1F3D.1000703@berlin.ccc.de>

Hey guys,


I've spent the last couple of days with re-implementing the algorithms 
for taking random samples from the poisson and exponential distribution 
from R's native C implementation.

It was quite the challenge to translate the imperative C code to Erlangs 
functional style, especially the one goto statement.

The Erlang code might still be subject for optimization and I would 
appreciate any feedback on how to improve it.

The C code is included in the Repo for reference and it is implementing 
after two papers which presented highly efficient algorithms for both 
problems.

The code can be found here:

https://github.com/hukl/rstats

Kind regards,

~ John


From jay@REDACTED  Sun May 31 19:16:41 2015
From: jay@REDACTED (Jay Nelson)
Date: Sun, 31 May 2015 10:16:41 -0700
Subject: [erlang-questions] Implemented Poisson and Exponential
	Distribution
Message-ID: <43F3CF0B-2EAB-4FBE-ABE6-8E036EA43A90@duomark.com>

Your slowest things are going to be lists:append and lists:nth.
You should use lists:sublist instead of a filter with lists:nth.
And see if there is any way to avoid appending to the end of
a list, or walking to a specific position in the other cases.

jay