From alexakarpov@REDACTED Sun Oct 1 05:22:30 2017 From: alexakarpov@REDACTED (Alexandre Karpov) Date: Sat, 30 Sep 2017 23:22:30 -0400 Subject: [erlang-questions] nodes active/inactive, but why? Message-ID: Hi all, (Forgive me if this mailing list is not a best place for newbie question (y'all seem to be way above that level, based on what I've seen in the last couple days). Working through Joe's book, I've got to the exciting bits in "Distributed Programming" (chapter 14), which has examples for setting up rpc calls between nodes on the same machine (trivial) and different machines (not as trivial). I've got two boxes up for this exercise, got Erlang installed, and set up my DNS - I can ping both. But one of them acts like an "inactive node", so I don't think my setup is ready for playing: Node 1 (started with: *erl -name little setcookie abc):* (little@REDACTED)1> node(). 'little@REDACTED' Node 2 (started with *erl -name yyz setcookie abc):* 1> node(). nonode@REDACTED Um... what did I do wrong? running 'hostname' gives expected results: yyz.alexakarpov.xyz and little.alexakarpov.xyz respectively, and they're started the same way... both are running same Erlang versions, of course. Thank you for your time! Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Sun Oct 1 06:18:50 2017 From: zxq9@REDACTED (zxq9) Date: Sun, 01 Oct 2017 13:18:50 +0900 Subject: [erlang-questions] nodes active/inactive, but why? In-Reply-To: References: Message-ID: <1834087.UV7QUuv1vL@changa> Hi, Alexandre! On 2017?09?30? ??? 23:22:30 Alexandre Karpov wrote: > (Forgive me if this mailing list is not a best place for newbie question > y'all seem to be way above that level, based on what I've seen in the last > couple days). Don't worry! You've got a question about Erlang. You've come to the right place. > Working through Joe's book, I've got to the exciting bits in "Distributed > Programming" (chapter 14), which has examples for setting up rpc calls > between nodes on the same machine (trivial) and different machines (not as > trivial). I've got two boxes up for this exercise, got Erlang installed, > and set up my DNS - I can ping both. But one of them acts like an "inactive > node", so I don't think my setup is ready for playing: > > Node 1 (started with: *erl -name little setcookie abc):* > > (little@REDACTED)1> node(). > > 'little@REDACTED' > > Node 2 (started with *erl -name yyz setcookie abc):* > > 1> node(). > > nonode@REDACTED > > Um... what did I do wrong? running 'hostname' gives expected results: > yyz.alexakarpov.xyz and little.alexakarpov.xyz respectively, and they're > started the same way... both are running same Erlang versions, of course. I'm not sure from this what went wrong. Try doing it manually from within each node node to see what is going on: ceverett@REDACTED:~/vcs/zomp/src$ erl Erlang/OTP 20 [RELEASE CANDIDATE 2] [erts-9.0] [source] [64-bit] [smp:4:4] [ds:2:2:10] [async-threads:10] [hipe] [kernel-poll:false] Eshell V9.0 (abort with ^G) 1> net_kernel:start([foo]). {ok,<0.63.0>} (foo@REDACTED)2> erlang:set_cookie(node(), 'abc'). true (foo@REDACTED)3> Give that a shot and see what happens. Maybe you can start finding a new direction to troubleshoot. I don't use disterl very often, actually, so I'm not familiar with what might be going on here with epmd and net_kernel. I usually use node names to enforce that only a single instance of a program is running on a client system, and occasionally that they can talk on the same system... which isn't really what epmd is for. -Craig From v.levytskyy@REDACTED Sun Oct 1 10:13:44 2017 From: v.levytskyy@REDACTED (Vyacheslav Levytskyy) Date: Sun, 1 Oct 2017 10:13:44 +0200 Subject: [erlang-questions] nodes active/inactive, but why? In-Reply-To: References: Message-ID: <2c78ba82-a865-375d-fbb7-bb6b92504e02@yahoo.com> Hi, you have an error around 'setcookie', prepend it with - pls: erl -sname yyz -setcookie abc Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false] Eshell V9.0? (abort with ^G) (yyz@REDACTED)1> node(). yyz@REDACTED (yyz@REDACTED)2> erlang:get_cookie(). abc Best regards, Vyacheslav On 01.10.2017 05:22, Alexandre Karpov wrote: > Hi all, > > (Forgive me if this mailing list is not a best place for newbie > question (y'all seem to be way above that level, based on what I've > seen in the last couple days). > > Working through Joe's book, I've got to the exciting bits in > "Distributed Programming" (chapter 14), which has examples for setting > up rpc calls between nodes on the same machine (trivial) and different > machines (not as trivial). I've got two boxes up for this exercise, > got Erlang installed, and set up my DNS - I can ping both. But one of > them acts like an "inactive node", so I don't think my setup is ready > for playing: > > Node 1 (started with: *erl -name little setcookie abc):* > > (little@REDACTED > )1> node(). > > 'little@REDACTED ' > > Node 2 (started with *erl -name yyz setcookie abc):* > > 1> node(). > > nonode@REDACTED > > Um... what did I do wrong? running 'hostname' gives expected results: > yyz.alexakarpov.xyz and > little.alexakarpov.xyz respectively, > and they're started the same way... both are running same Erlang > versions, of course. > > Thank you for your time! > Alex > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexakarpov@REDACTED Sun Oct 1 12:48:31 2017 From: alexakarpov@REDACTED (Alexandre Karpov) Date: Sun, 1 Oct 2017 06:48:31 -0400 Subject: [erlang-questions] nodes active/inactive, but why? In-Reply-To: References: Message-ID: Oh my. As usual, the problem had to be that trivial, eh. Thank you (and everyone else who responded). On Sun, Oct 1, 2017 at 6:03 AM, bengt e wrote: > Greetings, > > Is there a typo in the command 'erl -name little setcookie abc):' ? I that > setcookie is a flag, ie -setcookie. > > Also, after starting distributed Erlang you have epmd up and running. You > can ask for other nodes 'epmd -names' > > > bengt > > On Sun, Oct 1, 2017 at 5:22 AM, Alexandre Karpov > wrote: > >> Hi all, >> >> (Forgive me if this mailing list is not a best place for newbie question >> (y'all seem to be way above that level, based on what I've seen in the last >> couple days). >> >> Working through Joe's book, I've got to the exciting bits in "Distributed >> Programming" (chapter 14), which has examples for setting up rpc calls >> between nodes on the same machine (trivial) and different machines (not as >> trivial). I've got two boxes up for this exercise, got Erlang installed, >> and set up my DNS - I can ping both. But one of them acts like an "inactive >> node", so I don't think my setup is ready for playing: >> >> Node 1 (started with: *erl -name little setcookie abc):* >> >> (little@REDACTED)1> node(). >> >> 'little@REDACTED' >> >> Node 2 (started with *erl -name yyz setcookie abc):* >> >> 1> node(). >> >> nonode@REDACTED >> >> Um... what did I do wrong? running 'hostname' gives expected results: >> yyz.alexakarpov.xyz and little.alexakarpov.xyz respectively, and they're >> started the same way... both are running same Erlang versions, of course. >> >> Thank you for your time! >> Alex >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Sun Oct 1 13:28:48 2017 From: zxq9@REDACTED (zxq9) Date: Sun, 01 Oct 2017 20:28:48 +0900 Subject: [erlang-questions] nodes active/inactive, but why? In-Reply-To: References: Message-ID: <12882308.yBUbU7XADl@changa> On 2017?10?01? ??? 06:48:31 Alexandre Karpov wrote: > Oh my. As usual, the problem had to be that trivial, eh. Thank you (and > everyone else who responded). Hah! But of course it was a missing dash. ;-) -Craig From ulf@REDACTED Sun Oct 1 14:09:35 2017 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 1 Oct 2017 14:09:35 +0200 Subject: [erlang-questions] Small performance hint: avoid abusing module_info(attributes) In-Reply-To: References: Message-ID: At the very least, one should exercise caution on some particular modules, and perhaps generally modules that have lots of type specs. Interestingly, when fetching attributes from beam_lib, dialyzer-generated attributes are moved to a subgroup, so at least in OTP 20.0 code, attribute lists are no longer than 4. Fetching via M:module_info/1, however, the list is flat, and can be quite long, mostly depending on the number of types. Looking again at OTP, but loading each module and fetching the attributes via M:module_info/1 (the example below lists all modules with at least 20 attributes): 19> lists:foldr(fun(D,Acc) -> filelib:fold_files(D, ".*", false, fun(F,A) -> M = list_to_atom(filename:basename(F,".beam")), try M:module_info(attributes) of As -> case length(As) of L when L >= 20 -> [{M,L}|A]; _ -> A end catch error:_ -> A end end, []) ++ Acc end, [], code:get_path()). [{gen_fsm,20}, {erl_parse,208}, {xmerl_xpath_parse,45}, {xref_parser,34}, {snmpc_mib_gram,232}, {megaco_text_parser_v3,413}, {megaco_text_parser_v2,378}, {megaco_text_parser_v1,325}, {megaco_text_parser_prev3c,410}, {megaco_text_parser_prev3b,396}, {megaco_text_parser_prev3a,389}, {megaco_text_mini_parser,25}, {icparse,185}, {edoc_parser,94}, {diameter_dict_parser,76}, {cosNotification_Grammar,36}, {core_parse,224}] BR, Ulf W 2017-09-28 8:56 GMT+02:00 Max Lapshin : > Hi. > > We have moved from plain text lager logging to our own event system that > has main difference: we have limited amount of possible events that are > compiled in code. > > In known_events.erl I have: > > > -event({stream_started, info, ["stream started"]}). > > and then I write in code: > > > events:stream_started([{media,Name}]). > > Later all this is printed (using lager formatting, it is cool). > > > > So we are close to the problem. My code that was handling such: > events:stream_started was searching for predefined event via: > known_events:module_info(attributes). > > Suddenly eprof told that I need to look carefully at this place and I > wrote parse_transform that takes all attributes from module AST and > compiles them into code. Also I've generated function > known_events:find(Name) with all possible clauses. > > On single core of my macbook I got 4000 times speedup. 100 000 of fetches > known_events:all() took about 8 seconds with module_info(attributes) and > about 2 milliseconds with generated and compiled code. > > > So lesson from here is: try to avoid calling module_info(attributes) in > innner loops. It may be suprisingly expensive. > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris@REDACTED Sun Oct 1 19:22:34 2017 From: chris@REDACTED (Chris Waymire) Date: Sun, 1 Oct 2017 10:22:34 -0700 Subject: [erlang-questions] gen_server & wait/notify In-Reply-To: <2279123.ehtPpbLAke@changa> References: <4042515.OzFQNc8idd@changa> <2279123.ehtPpbLAke@changa> Message-ID: I'll try to provide some context around what I am doing. Basically I am writing a client for a custom MQTT network that in addition to the basic MQTT functionality also provides a request/response system. So the idea here is that when a user of the app executes the "request" function a uuid will be attached to their message payload and published onto the network. Another endpoint that is subscribed to that topic will receive the message, process it, and publish a response with a reference to the original message id. The original sender then picks up this message and accepts it as the response to their request. There's more going on there but that's the gist of it. So which the request function is called I need the caller to block, but not the MQTT client itself as it will still need to be processing messages on the network. On Fri, Sep 29, 2017 at 12:40 AM, zxq9 wrote: > On 2017?09?29? ??? 16:23:42 zxq9 wrote: > > On 2017?09?28? ??? 17:58:08 Chris Waymire wrote: > > > handle_call({request, Data#data{uid=UID}}, _From, State) -> > > > make_asycnc_req(Data), > > > receive > > > {UID, Response} -> {reply, Response, State} > > > end. > > > > > > handle_info({response, Response#response{uid=UID}}, State) -> > > > self() ! {UID, Response}, > > > {noreply, State}. > > > > > > Your naked `receive` will work just fine. The flow of the process has > never > > returned to the gen_server module yet to await another message to > dispatch, > > so when you write a naked `receive` in some handling code that is exactly > > where the process will block, and you can receive any arbitrary thing you > > want there. Just be careful not to match on any system or gen_server > message > > types and you'll get the behavior you expect (though your mailbox may be > > filling up with other stuff in the meantime). > > One note of caution is that you might actually be fighting against the > natural order of things. Consider carefully whether you need this and what > it is achieving for you. You may really be better off with casts. See if > there is a way for you to design the system to be entirely async, and if > not, why not. What state is being async going to threaten? Is it an > ordering issue? Are you relaying a message and waiting for a response > that will be returned? If so, why is the originating process not just > receiving the response directly? (Do you really need a middle-man?) > > Another way to achieve this without blocking is to use references to > tag messages and keep a digest of them. That prevents your gen_server > from becoming unresponsive to system messages or blocking indefinitely > in the event the sender of the message you are waiting on crashes. > > This is actually what the `From` argument to handle_call/3 is for: > > handle_call({request, Data = data{uid = UID}}, > {Sender, Tag}, > State = #s{queue = Q}) -> > ok = make_async_request(Data, Tag), > {noreply, State#s{queue = [{{Tag, UID, Sender} | Queue]}}; > % ... > > handle_cast({response, {Tag, UID}, Message}, State = #s{queue = Queue}) -> > case lists:keyfind(Tag, 1, Queue) of > Ticket = {Tag, UID, Sender} -> > NewState = State#s{queue = lists:delete(Ticket, Queue)}, > {reply, Message, NewState}; > false -> > LogString = "Received bad message: ~tp", > ok = log(warning, LogString, [{{Tag, UID}, Message}]), > {noreply, State} > end; > % ... > > There may be any number of ways you might want to phrase that or structure > it, but basically that's how you can safely stow such a pending response > value and sender, and get the response back out to them without any weird > blockages or unresponsiveness. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Sun Oct 1 23:44:35 2017 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sun, 1 Oct 2017 23:44:35 +0200 Subject: [erlang-questions] gen_server & wait/notify In-Reply-To: References: <4042515.OzFQNc8idd@changa> <2279123.ehtPpbLAke@changa> Message-ID: See Vlad's reply a few days ago. What you're describing is easily solvable by: 1. having the gen_server's handle_call for this API call stash away the >From and uuid in its internal state and return noreply as that blocks the caller, and 2. when the response comes in (tagged with that uuid) match it to the corresponding From and gen_server:reply to that one, which unblocks the caller. On Sun, Oct 1, 2017 at 7:22 PM, Chris Waymire wrote: > I'll try to provide some context around what I am doing. Basically I am > writing a client for a custom MQTT network that in addition to the basic > MQTT functionality also provides a request/response system. So the idea > here is that when a user of the app executes the "request" function a uuid > will be attached to their message payload and published onto the network. > Another endpoint that is subscribed to that topic will receive the message, > process it, and publish a response with a reference to the original message > id. The original sender then picks up this message and accepts it as the > response to their request. There's more going on there but that's the gist > of it. So which the request function is called I need the caller to block, > but not the MQTT client itself as it will still need to be processing > messages on the network. > > On Fri, Sep 29, 2017 at 12:40 AM, zxq9 wrote: > >> On 2017?09?29? ??? 16:23:42 zxq9 wrote: >> > On 2017?09?28? ??? 17:58:08 Chris Waymire wrote: >> > > handle_call({request, Data#data{uid=UID}}, _From, State) -> >> > > make_asycnc_req(Data), >> > > receive >> > > {UID, Response} -> {reply, Response, State} >> > > end. >> > > >> > > handle_info({response, Response#response{uid=UID}}, State) -> >> > > self() ! {UID, Response}, >> > > {noreply, State}. >> > >> > >> > Your naked `receive` will work just fine. The flow of the process has >> never >> > returned to the gen_server module yet to await another message to >> dispatch, >> > so when you write a naked `receive` in some handling code that is >> exactly >> > where the process will block, and you can receive any arbitrary thing >> you >> > want there. Just be careful not to match on any system or gen_server >> message >> > types and you'll get the behavior you expect (though your mailbox may be >> > filling up with other stuff in the meantime). >> >> One note of caution is that you might actually be fighting against the >> natural order of things. Consider carefully whether you need this and what >> it is achieving for you. You may really be better off with casts. See if >> there is a way for you to design the system to be entirely async, and if >> not, why not. What state is being async going to threaten? Is it an >> ordering issue? Are you relaying a message and waiting for a response >> that will be returned? If so, why is the originating process not just >> receiving the response directly? (Do you really need a middle-man?) >> >> Another way to achieve this without blocking is to use references to >> tag messages and keep a digest of them. That prevents your gen_server >> from becoming unresponsive to system messages or blocking indefinitely >> in the event the sender of the message you are waiting on crashes. >> >> This is actually what the `From` argument to handle_call/3 is for: >> >> handle_call({request, Data = data{uid = UID}}, >> {Sender, Tag}, >> State = #s{queue = Q}) -> >> ok = make_async_request(Data, Tag), >> {noreply, State#s{queue = [{{Tag, UID, Sender} | Queue]}}; >> % ... >> >> handle_cast({response, {Tag, UID}, Message}, State = #s{queue = Queue}) -> >> case lists:keyfind(Tag, 1, Queue) of >> Ticket = {Tag, UID, Sender} -> >> NewState = State#s{queue = lists:delete(Ticket, Queue)}, >> {reply, Message, NewState}; >> false -> >> LogString = "Received bad message: ~tp", >> ok = log(warning, LogString, [{{Tag, UID}, Message}]), >> {noreply, State} >> end; >> % ... >> >> There may be any number of ways you might want to phrase that or structure >> it, but basically that's how you can safely stow such a pending response >> value and sender, and get the response back out to them without any weird >> blockages or unresponsiveness. >> >> -Craig >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris@REDACTED Mon Oct 2 00:49:21 2017 From: chris@REDACTED (Chris Waymire) Date: Sun, 1 Oct 2017 15:49:21 -0700 Subject: [erlang-questions] gen_server & wait/notify In-Reply-To: References: <4042515.OzFQNc8idd@changa> <2279123.ehtPpbLAke@changa> Message-ID: Ahhh.. I was not familiar with how the noreply response works from handle_call. If I'm understanding it correctly though, in the snippet he provided, I would need to change {reply, Message, NewState} to instead call gen_server:reply/2 which then unblocks the caller. That is exactly what I am needing. I figured there had to be something to do that without having to cobble together some cludgy code. Thanks. -- Chris On Sun, Oct 1, 2017 at 2:44 PM, Mikael Pettersson wrote: > See Vlad's reply a few days ago. What you're describing is easily > solvable by: > 1. having the gen_server's handle_call for this API call stash away the > From and uuid in its internal state and return noreply as that blocks the > caller, and > 2. when the response comes in (tagged with that uuid) match it to the > corresponding From and gen_server:reply to that one, which unblocks the > caller. > > > On Sun, Oct 1, 2017 at 7:22 PM, Chris Waymire wrote: > >> I'll try to provide some context around what I am doing. Basically I am >> writing a client for a custom MQTT network that in addition to the basic >> MQTT functionality also provides a request/response system. So the idea >> here is that when a user of the app executes the "request" function a uuid >> will be attached to their message payload and published onto the network. >> Another endpoint that is subscribed to that topic will receive the message, >> process it, and publish a response with a reference to the original message >> id. The original sender then picks up this message and accepts it as the >> response to their request. There's more going on there but that's the gist >> of it. So which the request function is called I need the caller to block, >> but not the MQTT client itself as it will still need to be processing >> messages on the network. >> >> On Fri, Sep 29, 2017 at 12:40 AM, zxq9 wrote: >> >>> On 2017?09?29? ??? 16:23:42 zxq9 wrote: >>> > On 2017?09?28? ??? 17:58:08 Chris Waymire wrote: >>> > > handle_call({request, Data#data{uid=UID}}, _From, State) -> >>> > > make_asycnc_req(Data), >>> > > receive >>> > > {UID, Response} -> {reply, Response, State} >>> > > end. >>> > > >>> > > handle_info({response, Response#response{uid=UID}}, State) -> >>> > > self() ! {UID, Response}, >>> > > {noreply, State}. >>> > >>> > >>> > Your naked `receive` will work just fine. The flow of the process has >>> never >>> > returned to the gen_server module yet to await another message to >>> dispatch, >>> > so when you write a naked `receive` in some handling code that is >>> exactly >>> > where the process will block, and you can receive any arbitrary thing >>> you >>> > want there. Just be careful not to match on any system or gen_server >>> message >>> > types and you'll get the behavior you expect (though your mailbox may >>> be >>> > filling up with other stuff in the meantime). >>> >>> One note of caution is that you might actually be fighting against the >>> natural order of things. Consider carefully whether you need this and >>> what >>> it is achieving for you. You may really be better off with casts. See if >>> there is a way for you to design the system to be entirely async, and if >>> not, why not. What state is being async going to threaten? Is it an >>> ordering issue? Are you relaying a message and waiting for a response >>> that will be returned? If so, why is the originating process not just >>> receiving the response directly? (Do you really need a middle-man?) >>> >>> Another way to achieve this without blocking is to use references to >>> tag messages and keep a digest of them. That prevents your gen_server >>> from becoming unresponsive to system messages or blocking indefinitely >>> in the event the sender of the message you are waiting on crashes. >>> >>> This is actually what the `From` argument to handle_call/3 is for: >>> >>> handle_call({request, Data = data{uid = UID}}, >>> {Sender, Tag}, >>> State = #s{queue = Q}) -> >>> ok = make_async_request(Data, Tag), >>> {noreply, State#s{queue = [{{Tag, UID, Sender} | Queue]}}; >>> % ... >>> >>> handle_cast({response, {Tag, UID}, Message}, State = #s{queue = Queue}) >>> -> >>> case lists:keyfind(Tag, 1, Queue) of >>> Ticket = {Tag, UID, Sender} -> >>> NewState = State#s{queue = lists:delete(Ticket, Queue)}, >>> {reply, Message, NewState}; >>> false -> >>> LogString = "Received bad message: ~tp", >>> ok = log(warning, LogString, [{{Tag, UID}, Message}]), >>> {noreply, State} >>> end; >>> % ... >>> >>> There may be any number of ways you might want to phrase that or >>> structure >>> it, but basically that's how you can safely stow such a pending response >>> value and sender, and get the response back out to them without any weird >>> blockages or unresponsiveness. >>> >>> -Craig >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aschultz@REDACTED Mon Oct 2 09:34:08 2017 From: aschultz@REDACTED (Andreas Schultz) Date: Mon, 2 Oct 2017 09:34:08 +0200 (CEST) Subject: [erlang-questions] binding for hyperscan Message-ID: <262622585.3047406.1506929648247.JavaMail.zimbra@tpip.net> Hi, Does anyone know of a Erlang binding for the hyperscan [1] multiple regex matching library? Many thanks Andreas 1: https://01.org/hyperscan From ok@REDACTED Tue Oct 3 07:02:32 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 3 Oct 2017 18:02:32 +1300 Subject: [erlang-questions] McErlang Message-ID: <2ace7d67-390c-9d61-7cb1-66a990f49096@cs.otago.ac.nz> I just downloaded McErlang today. I note that - the last entry in NEWS is from 2011 - the last change in GitHub is 2 years ago - I had to change the type :: record() to :: any() to get it to compile, and there are numerous warnings from the compiler, including an ambiguous call, and of course lots of erlang:now() complaints. - it was produced as part of an EU project which appears to have come to an end, so there may not be money to keep it up to date. I have got it to the point where it compiles, but given the date, I haven't dared to see what it does with maps. Now this looked like a very useful tool, but before I spend any time getting a clean compilation, - is anyone else (still) using McErlang? - has McErlang been replaced by something better that I can afford? - or should I just forget all about it? From quantumpotato@REDACTED Tue Oct 3 07:05:53 2017 From: quantumpotato@REDACTED (qp) Date: Tue, 3 Oct 2017 01:05:53 -0400 Subject: [erlang-questions] No Makefile Found Message-ID: When I copy a folder of my erlang project on my local machine, I am able to cd into the copied directory and gmake run successfully. When I upload that folder to github and download it, I get "no makefile found" ~/tdep/gmake run Error: No Makefile to build dependency /Users/quantum/tdep/deps/cowboy. gmake: *** [erlang.mk:4182: deps] Error 2 from https://github.com/QuantumProductions/tunnel-deploy There is a Makefile present: ~/cd tdep ~/tdep/ls Makefile deps erlang.mk rel src tunnel.d _rel ebin oldh relx.config static ~/tdep/cat Makefile PROJECT = tunnel PROJECT_DESCRIPTION = Find?k PROJECT_VERSION = 0.1.0 DEPS = cowboy jiffy dep_cowboy_commit = master dep_jiffy = git https://github.com/davisp/jiffy include erlang.mk What is going on here? Could there be some setting on my machine that's affecting how gmake works? I expected to be able to clone this repo to my server and run it there. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Tue Oct 3 07:13:49 2017 From: zxq9@REDACTED (zxq9) Date: Tue, 03 Oct 2017 14:13:49 +0900 Subject: [erlang-questions] McErlang In-Reply-To: <2ace7d67-390c-9d61-7cb1-66a990f49096@cs.otago.ac.nz> References: <2ace7d67-390c-9d61-7cb1-66a990f49096@cs.otago.ac.nz> Message-ID: <5647409.nM4Hs5uE01@changa> On 2017?10?03? ??? 18:02:32 Richard A. O'Keefe wrote: > I just downloaded McErlang today. > I note that > - the last entry in NEWS is from 2011 > - the last change in GitHub is 2 years ago > - I had to change the type :: record() to :: any() to > get it to compile, and there are numerous warnings > from the compiler, including an ambiguous call, and > of course lots of erlang:now() complaints. > - it was produced as part of an EU project which > appears to have come to an end, so there may not be > money to keep it up to date. > > I have got it to the point where it compiles, but given > the date, I haven't dared to see what it does with maps. > > Now this looked like a very useful tool, but before I > spend any time getting a clean compilation, > - is anyone else (still) using McErlang? > - has McErlang been replaced by something better that > I can afford? > - or should I just forget all about it? I think this is *great*, at least in concept, if not in execution. I had never heard of this before. It looks pretty dead, though. Depending on the condition of the code I would be interested in patching it to bring the thing up to date with changes in Dialyzer and Erlang, but I don't have much time to contribute to that sort of thing for at least a few more months. Concurrent model testing in Erlang is a rather large gap we have that tends to get covered by actual runs and post-mortem analysis. Which is actually fine for the majority of cases (clearly; we've gotten this far) but having a tool that can do for concurrent sequence testing what tools like PropER can do for functional testing would be quite useful. Definitely putting it on my list of things to sort... -Craig From kostis@REDACTED Tue Oct 3 08:27:06 2017 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 3 Oct 2017 08:27:06 +0200 Subject: [erlang-questions] McErlang In-Reply-To: <5647409.nM4Hs5uE01@changa> References: <2ace7d67-390c-9d61-7cb1-66a990f49096@cs.otago.ac.nz> <5647409.nM4Hs5uE01@changa> Message-ID: <36f1c261-36c2-1782-71cb-07b50b12c3d6@cs.ntua.gr> On 10/03/2017 07:13 AM, zxq9 wrote: >> Now this looked like a very useful tool, but before I >> spend any time getting a clean compilation, >> - is anyone else (still) using McErlang? >> - has McErlang been replaced by something better that >> I can afford? >> - or should I just forget all about it? > ... ... > > Concurrent model testing in Erlang is a rather large gap we have that > tends to get covered by actual runs and post-mortem analysis. Which is > actually fine for the majority of cases (clearly; we've gotten this far) > but having a tool that can do for concurrent sequence testing what tools > like PropER can do for functional testing would be quite useful. I think that you should really take a look at Concuerror. http://concuerror.com/ Besides various tutorials, which are quite comprehensive, it also comes with (not always easy to read) papers that explain its technology and its differences from McErlang. I am of course very partial, since I'm heavily involved in its design and development, but it's in many ways a superior tool. Hope that others find it useful; some already have. Kostis From fritchie@REDACTED Tue Oct 3 08:25:01 2017 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 03 Oct 2017 06:25:01 +0000 Subject: [erlang-questions] McErlang In-Reply-To: <2ace7d67-390c-9d61-7cb1-66a990f49096@cs.otago.ac.nz> References: <2ace7d67-390c-9d61-7cb1-66a990f49096@cs.otago.ac.nz> Message-ID: Hi, Richard. It is too late in the middle of the US for a great answer (zzzzz...), but since it isn?t such a crazy hour for you (probably), a terrible answer might be helpful? The Concuerror tool at http://concuerror.com/ is under active development today by Stavros Aronis. Its DPOR techniques are likely far more efficient than McErlang?s search strategies, even if McErlang were cleaned up for a newer Erlang release. I?ve used it on a couple of verification projects with good results. -Scott -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Tue Oct 3 09:40:31 2017 From: zxq9@REDACTED (zxq9) Date: Tue, 03 Oct 2017 16:40:31 +0900 Subject: [erlang-questions] McErlang In-Reply-To: <36f1c261-36c2-1782-71cb-07b50b12c3d6@cs.ntua.gr> References: <2ace7d67-390c-9d61-7cb1-66a990f49096@cs.otago.ac.nz> <5647409.nM4Hs5uE01@changa> <36f1c261-36c2-1782-71cb-07b50b12c3d6@cs.ntua.gr> Message-ID: <3023259.7ibsdADjjD@changa> On 2017?10?03? ??? 08:27:06 Kostis Sagonas wrote: > On 10/03/2017 07:13 AM, zxq9 wrote: > >> Now this looked like a very useful tool, but before I > >> spend any time getting a clean compilation, > >> - is anyone else (still) using McErlang? > >> - has McErlang been replaced by something better that > >> I can afford? > >> - or should I just forget all about it? > > ... ... > > > > Concurrent model testing in Erlang is a rather large gap we have that > > tends to get covered by actual runs and post-mortem analysis. Which is > > actually fine for the majority of cases (clearly; we've gotten this far) > > but having a tool that can do for concurrent sequence testing what tools > > like PropER can do for functional testing would be quite useful. > > I think that you should really take a look at Concuerror. > > http://concuerror.com/ > > Besides various tutorials, which are quite comprehensive, it also comes > with (not always easy to read) papers that explain its technology and > its differences from McErlang. > > I am of course very partial, since I'm heavily involved in its design > and development, but it's in many ways a superior tool. > > Hope that others find it useful; some already have. Wow, awesome! This is an area I haven't had a chance to get on board with yet (limited time) but I'm very glad that ROK's email wound up pointing me toward both McErlang and inadvertently led me to Concuerror. Thank you guys for the link! -Craig From hans.r.nilsson@REDACTED Tue Oct 3 10:28:49 2017 From: hans.r.nilsson@REDACTED (Hans Nilsson R) Date: Tue, 3 Oct 2017 10:28:49 +0200 Subject: [erlang-questions] Patch package OTP 20.1.1 released Message-ID: <76f14d77-bef8-0198-f076-c1523bb4fb0e@ericsson.com> Patch Package: OTP 20.1.1 Git Tag: OTP-20.1.1 Date: 2017-10-03 Trouble Report Id: OTP-14638, OTP-14640, OTP-14645, OTP-14648, OTP-14649 Seq num: ERL-490 System: OTP Release: 20 Application: compiler-7.1.3, erts-9.1.1, ssh-4.6.1 Predecessor: OTP 20.1 Check out the git tag OTP-20.1.1, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- compiler-7.1.3 -------------------------------------------------- --------------------------------------------------------------------- The compiler-7.1.3 application can be applied independently of other applications on a full OTP 20 installation. --- Fixed Bugs and Malfunctions --- OTP-14640 Application(s): compiler Related Id(s): ERL-490 The compiler could issue an incorrect internal consistency failure diagnostic for some complicated bit syntax maches. Full runtime dependencies of compiler-7.1.3: crypto-3.6, erts-9.0, hipe-3.12, kernel-4.0, stdlib-2.5 --------------------------------------------------------------------- --- erts-9.1.1 ------------------------------------------------------ --------------------------------------------------------------------- The erts-9.1.1 application can be applied independently of other applications on a full OTP 20 installation. --- Fixed Bugs and Malfunctions --- OTP-14648 Application(s): erts The new zlib module returned a data_error when inflating concatenated streams, which was incompatible with the old module's behavior of returning the uncompressed data up to the end of the first stream. OTP-14649 Application(s): erts zlib:gunzip/1 will no longer stop at the end of the first stream when decompressing concatenated gzip files. Full runtime dependencies of erts-9.1.1: kernel-5.0, sasl-3.0.1, stdlib-3.0 --------------------------------------------------------------------- --- ssh-4.6.1 ------------------------------------------------------- --------------------------------------------------------------------- The ssh-4.6.1 application can be applied independently of other applications on a full OTP 20 installation. --- Fixed Bugs and Malfunctions --- OTP-14645 Application(s): ssh Fixed broken printout --- Improvements and New Features --- OTP-14638 Application(s): ssh Disable aes_gcm ciphers if peer is OpenSSH 6.2 which is known to have trouble with them in some cases. Full runtime dependencies of ssh-4.6.1: crypto-3.7.3, erts-6.0, kernel-3.0, public_key-1.4, stdlib-3.3 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4101 bytes Desc: S/MIME Cryptographic Signature URL: From lfredlund@REDACTED Tue Oct 3 11:05:01 2017 From: lfredlund@REDACTED (=?UTF-8?Q?Lars-=c3=85ke_Fredlund?=) Date: Tue, 3 Oct 2017 11:05:01 +0200 Subject: [erlang-questions] McErlang In-Reply-To: <3023259.7ibsdADjjD@changa> References: <2ace7d67-390c-9d61-7cb1-66a990f49096@cs.otago.ac.nz> <5647409.nM4Hs5uE01@changa> <36f1c261-36c2-1782-71cb-07b50b12c3d6@cs.ntua.gr> <3023259.7ibsdADjjD@changa> Message-ID: Hello all, as you have noticed we haven't really been working on McErlang for some time. There is no support for some modern features of Erlang, and frankly I doubt we will have time to fix things in the short term. McErlang was a cool project though. Is McErlang better or worse than Concuerror? Well, it is always better to use a maintained tool than a tool that nobody has time to fix. The tools are based on different internal ideas, with different search strategies, which at least in theory can cause some errors to be detected by one tool to fail to be detected by the other tool, and vice versa. Clearly applying Concuerror to a program is easier than applying McErlang, and so I would definitely try using Concuerror first. Another tool worth trying is QuickCheck with the pulse random scheduler; it can be surprisingly good at detecting concurrency bugs with a very moderate effort. Lars-Ake On 03/10/17 09:40, zxq9 wrote: > On 2017?10?03? ??? 08:27:06 Kostis Sagonas wrote: >> On 10/03/2017 07:13 AM, zxq9 wrote: >>>> Now this looked like a very useful tool, but before I >>>> spend any time getting a clean compilation, >>>> - is anyone else (still) using McErlang? >>>> - has McErlang been replaced by something better that >>>> I can afford? >>>> - or should I just forget all about it? >>> ... ... >>> >>> Concurrent model testing in Erlang is a rather large gap we have that >>> tends to get covered by actual runs and post-mortem analysis. Which is >>> actually fine for the majority of cases (clearly; we've gotten this far) >>> but having a tool that can do for concurrent sequence testing what tools >>> like PropER can do for functional testing would be quite useful. >> I think that you should really take a look at Concuerror. >> >> http://concuerror.com/ >> >> Besides various tutorials, which are quite comprehensive, it also comes >> with (not always easy to read) papers that explain its technology and >> its differences from McErlang. >> >> I am of course very partial, since I'm heavily involved in its design >> and development, but it's in many ways a superior tool. >> >> Hope that others find it useful; some already have. > Wow, awesome! > > This is an area I haven't had a chance to get on board with yet (limited > time) but I'm very glad that ROK's email wound up pointing me toward both > McErlang and inadvertently led me to Concuerror. > > Thank you guys for the link! > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From kostis@REDACTED Tue Oct 3 11:57:23 2017 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 3 Oct 2017 11:57:23 +0200 Subject: [erlang-questions] McErlang In-Reply-To: References: <2ace7d67-390c-9d61-7cb1-66a990f49096@cs.otago.ac.nz> <5647409.nM4Hs5uE01@changa> <36f1c261-36c2-1782-71cb-07b50b12c3d6@cs.ntua.gr> <3023259.7ibsdADjjD@changa> Message-ID: On 10/03/2017 11:05 AM, Lars-?ke Fredlund wrote: > > McErlang was a cool project though. Is McErlang better or worse than > Concuerror? Well, it is always better to use a maintained tool than a > tool that nobody has time to fix. The tools are based on different > internal ideas, with different search strategies, which at least in > theory can cause some errors to be detected by one tool to fail to be > detected by the other tool, and vice versa. I would be interested in seeing a simple example program containing a _concurrency_ error that can be detected by McErlang but not by Concuerror. AFAIK, Concuerror finds *all* concurrency errors (for a particular way of using/running a program), where the set of concurrency errors contains program crashes, assertion violations and global deadlocks that happen in some (and not necessarily in all) process schedulings of an Erlang program. Thus Concuerror, in principle at least (i.e., modulo bugs in its code), actually *verifies*, using stateless model checking, that a particular (part of a) program is free of concurrency errors, in contrast to e.g. QuickCheck/PULSE which fundamentally cannot provide any correctness guarantees (other than probabilistic ones). Best, Kostis From lfredlund@REDACTED Tue Oct 3 12:16:04 2017 From: lfredlund@REDACTED (=?UTF-8?Q?Lars-=c3=85ke_Fredlund?=) Date: Tue, 3 Oct 2017 12:16:04 +0200 Subject: [erlang-questions] McErlang In-Reply-To: References: <2ace7d67-390c-9d61-7cb1-66a990f49096@cs.otago.ac.nz> <5647409.nM4Hs5uE01@changa> <36f1c261-36c2-1782-71cb-07b50b12c3d6@cs.ntua.gr> <3023259.7ibsdADjjD@changa> Message-ID: <67c4b688-d80b-7e2a-4718-052d257b94e1@fi.upm.es> I am very impressed that Concuerror is able to find all concurrency errors, for an arbitrary complex program, given a finite amount of time, and a finite amount of memory... :-) McErlang can find all errors too, of course, but in practice for many (complex) programs it will fail to do so given the practical limitations (memory, time). Concurerror has a different search strategy, and in many/most cases will suffer less than McErlang from the inherent complex behaviour of the program under analysis, but it will suffer too. /Lars-Ake On 03/10/17 11:57, Kostis Sagonas wrote: > On 10/03/2017 11:05 AM, Lars-?ke Fredlund wrote: >> >> McErlang was a cool project though. Is McErlang better or worse than >> Concuerror? Well, it is always better to use a maintained tool than a >> tool that nobody has time to fix. The tools are based on different >> internal ideas, with different search strategies, which at least in >> theory can cause some errors to be detected by one tool to fail to be >> detected by the other tool, and vice versa. > > I would be interested in seeing a simple example program containing a > _concurrency_ error that can be detected by McErlang but not by > Concuerror. > > AFAIK, Concuerror finds *all* concurrency errors (for a particular way > of using/running a program), where the set of concurrency errors > contains program crashes, assertion violations and global deadlocks > that happen in some (and not necessarily in all) process schedulings > of an Erlang program. > > Thus Concuerror, in principle at least (i.e., modulo bugs in its > code), actually *verifies*, using stateless model checking, that a > particular (part of a) program is free of concurrency errors, in > contrast to e.g. QuickCheck/PULSE which fundamentally cannot provide > any correctness guarantees (other than probabilistic ones). > > Best, > Kostis From arunp@REDACTED Tue Oct 3 15:10:54 2017 From: arunp@REDACTED (Arun) Date: Tue, 3 Oct 2017 18:40:54 +0530 Subject: [erlang-questions] How RPC works in erlang ? Message-ID: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> Hi all, I have two distributed applications running on two different nodes. And from one node I tried to call the function on the remote node by doing RPC, and its been observed that all the debug prints given in the application running on the remote node is getting printed on the host node's console. Can somebody please tell me in detail why this is happening and how the RPC in erlang works. ? Thanks in advance. Arun P. From aronisstav@REDACTED Tue Oct 3 15:11:39 2017 From: aronisstav@REDACTED (Stavros Aronis) Date: Tue, 03 Oct 2017 13:11:39 +0000 Subject: [erlang-questions] McErlang In-Reply-To: <67c4b688-d80b-7e2a-4718-052d257b94e1@fi.upm.es> References: <2ace7d67-390c-9d61-7cb1-66a990f49096@cs.otago.ac.nz> <5647409.nM4Hs5uE01@changa> <36f1c261-36c2-1782-71cb-07b50b12c3d6@cs.ntua.gr> <3023259.7ibsdADjjD@changa> <67c4b688-d80b-7e2a-4718-052d257b94e1@fi.upm.es> Message-ID: Concuerror is indeed an actively maintained tool that is a suitable replacement for McErlang. I spent the morning updating some of the pages on http://concuerror.com. :-) /Stavros -------------- next part -------------- An HTML attachment was scrubbed... URL: From cean.ebengt@REDACTED Tue Oct 3 15:19:46 2017 From: cean.ebengt@REDACTED (bengt e) Date: Tue, 3 Oct 2017 15:19:46 +0200 Subject: [erlang-questions] How RPC works in erlang ? In-Reply-To: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> Message-ID: Greetings, Yes, that is a feature. All prints go back to the host node. bengt On Tue, Oct 3, 2017 at 3:10 PM, Arun wrote: > Hi all, > > I have two distributed applications running on two different nodes. And > from one node I tried to call the function on the remote node by doing RPC, > and its been observed that all the debug prints given in the application > running on the remote node is getting printed on the host node's console. > > Can somebody please tell me in detail why this is happening and how the > RPC in erlang works. ? > > Thanks in advance. > Arun P. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From elbrujohalcon@REDACTED Tue Oct 3 15:20:43 2017 From: elbrujohalcon@REDACTED (Brujo Benavides) Date: Tue, 3 Oct 2017 10:20:43 -0300 Subject: [erlang-questions] How RPC works in erlang ? In-Reply-To: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> Message-ID: <2086D046-061B-4AC3-B063-0F2B3E7C22F7@gmail.com> Hi Arun, That?s intended. It happens because of how Erlang redirects all IO through the process group_leader. This is the best explanatory link I could find googling quickly. Hope it helps. Brujo Benavides > On Oct 3, 2017, at 10:10, Arun wrote: > > Hi all, > > I have two distributed applications running on two different nodes. And from one node I tried to call the function on the remote node by doing RPC, and its been observed that all the debug prints given in the application running on the remote node is getting printed on the host node's console. > > Can somebody please tell me in detail why this is happening and how the RPC in erlang works. ? > > Thanks in advance. > Arun P. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From cbenac@REDACTED Tue Oct 3 16:08:59 2017 From: cbenac@REDACTED (Clara Benac Earle) Date: Tue, 3 Oct 2017 16:08:59 +0200 Subject: [erlang-questions] McErlang In-Reply-To: References: <2ace7d67-390c-9d61-7cb1-66a990f49096@cs.otago.ac.nz> <5647409.nM4Hs5uE01@changa> <36f1c261-36c2-1782-71cb-07b50b12c3d6@cs.ntua.gr> <3023259.7ibsdADjjD@changa> Message-ID: <78ecc4b8-b417-ea24-081e-5aaa411edebe@fi.upm.es> Dear Kostis, > I would be interested in seeing a simple example program containing a > _concurrency_ error that can be detected by McErlang but not by > Concuerror. Please have a look at the locker case-study presented in several of our papers (see below). This case-study contains a server that grants exclusive/shared access to resources. It is vaguely inspired on some code in the AXD301. Starvation of a client requesting shared access to one resource occurs when there are two clients requesting exclusive access to the same resource, since exclusive access has priority over shared access. We were able to find this already in 2004, and later on with McErlang. Concuerror has been running now for more than 20 minutes and still no answer so I cannot tell you whether it can find the starvation problem or not. Best regards Clara Thomas Arts , Clara Benac Earle, John Derrick : Development of a verified Erlang program for resource locking. STTT 5(2-3) : 205-220 (2004) Lars-?ke Fredlund , Clara Benac Earle: Model checking erlang programs: the functional approach. Erlang Workshop 2006 : 11-19 -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.hughes@REDACTED Tue Oct 3 19:08:14 2017 From: john.hughes@REDACTED (John Hughes) Date: Tue, 3 Oct 2017 19:08:14 +0200 Subject: [erlang-questions] McErlang In-Reply-To: <78ecc4b8-b417-ea24-081e-5aaa411edebe@fi.upm.es> References: <2ace7d67-390c-9d61-7cb1-66a990f49096@cs.otago.ac.nz> <5647409.nM4Hs5uE01@changa> <36f1c261-36c2-1782-71cb-07b50b12c3d6@cs.ntua.gr> <3023259.7ibsdADjjD@changa> <78ecc4b8-b417-ea24-081e-5aaa411edebe@fi.upm.es> Message-ID: <326501d33c6a$336e6870$9a4b3950$@quviq.com> Concuerror will guarantee to find bugs? with relatively short runs, relatively few processes, and a small number of preemptions. It?s searching very thoroughly in a small part of the space of possible executions?and it?s very good at it, but there is still an exponential blowup that will strike eventually. Random sampling, such as QuickCheck and PULSE do, can sample from a much larger part of the space? and if that?s where the bugs are, then that?s where you have to look for them! So Concuerror/McErlang and random tools are really complementary?either one may find bugs faster, depending on the kind of bug. There is plenty of evidence that sampling larger random tests, and shrinking them, can be a much faster way to find faults than enumerating smaller tests. John Fr?n: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] F?r Clara Benac Earle Skickat: den 3 oktober 2017 16:09 Till: Kostis Sagonas ; erlang-questions@REDACTED; Lars-?ke Fredlund ?mne: Re: [erlang-questions] McErlang Dear Kostis, I would be interested in seeing a simple example program containing a _concurrency_ error that can be detected by McErlang but not by Concuerror. Please have a look at the locker case-study presented in several of our papers (see below). This case-study contains a server that grants exclusive/shared access to resources. It is vaguely inspired on some code in the AXD301. Starvation of a client requesting shared access to one resource occurs when there are two clients requesting exclusive access to the same resource, since exclusive access has priority over shared access. We were able to find this already in 2004, and later on with McErlang. Concuerror has been running now for more than 20 minutes and still no answer so I cannot tell you whether it can find the starvation problem or not. Best regards Clara Thomas Arts , Clara Benac Earle, John Derrick : Development of a verified Erlang program for resource locking. STTT 5(2-3) : 205-220 (2004) Lars-?ke Fredlund , Clara Benac Earle: Model checking erlang programs: the functional approach. Erlang Workshop 2006 : 11-19 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Tue Oct 3 21:08:09 2017 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 3 Oct 2017 21:08:09 +0200 Subject: [erlang-questions] No Makefile Found In-Reply-To: References: Message-ID: It looks a bit like your cowboy dependency isn't building correctly. Try doing a gmake distclean and then the gmake run. That should redo the deps. HTH, Robby On 3 Oct 2017 07:05, "qp" wrote: When I copy a folder of my erlang project on my local machine, I am able to cd into the copied directory and gmake run successfully. When I upload that folder to github and download it, I get "no makefile found" ~/tdep/gmake run Error: No Makefile to build dependency /Users/quantum/tdep/deps/cowboy. gmake: *** [erlang.mk:4182: deps] Error 2 from https://github.com/QuantumProductions/tunnel-deploy There is a Makefile present: ~/cd tdep ~/tdep/ls Makefile deps erlang.mk rel src tunnel.d _rel ebin oldh relx.config static ~/tdep/cat Makefile PROJECT = tunnel PROJECT_DESCRIPTION = Find?k PROJECT_VERSION = 0.1.0 DEPS = cowboy jiffy dep_cowboy_commit = master dep_jiffy = git https://github.com/davisp/jiffy include erlang.mk What is going on here? Could there be some setting on my machine that's affecting how gmake works? I expected to be able to clone this repo to my server and run it there. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Wed Oct 4 01:57:12 2017 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 4 Oct 2017 01:57:12 +0200 Subject: [erlang-questions] McErlang In-Reply-To: <326501d33c6a$336e6870$9a4b3950$@quviq.com> References: <2ace7d67-390c-9d61-7cb1-66a990f49096@cs.otago.ac.nz> <5647409.nM4Hs5uE01@changa> <36f1c261-36c2-1782-71cb-07b50b12c3d6@cs.ntua.gr> <3023259.7ibsdADjjD@changa> <78ecc4b8-b417-ea24-081e-5aaa411edebe@fi.upm.es> <326501d33c6a$336e6870$9a4b3950$@quviq.com> Message-ID: <538db898-8af8-d434-29b7-35571b5f30fb@cs.ntua.gr> On 10/03/2017 07:08 PM, John Hughes wrote: > Concuerror will guarantee to find bugs? with relatively short runs, > relatively few processes, and a small number of preemptions. It?s > searching very thoroughly in a small part of the space of possible > executions?and it?s very good at it, but there is still an exponential > blowup that will strike eventually. Random sampling, such as QuickCheck > and PULSE do, can sample from a much larger part of the space? and if > that?s where the bugs are, then that?s where you have to look for them! > So Concuerror/McErlang and random tools are really complementary?either > one may find bugs faster, depending on the kind of bug. There is plenty > of evidence that sampling larger random tests, and shrinking them, can > be a much faster way to find faults than enumerating smaller tests. I very much agree that systematic and random-based tools are really complementary: both have their pros and cons and, for that reason, their proper place in a programmer's tool suite. Also, for testing purposes, there is a lot to be said for finding bugs sooner rather than later and perhaps one can detect bugs faster with random sampling if the bug occurs sufficiently frequently. Still, some of my recent experiences [1] [2] are different: a sufficiently powerful stateless model checking tool, such as Concuerror (or a similar such tool for C), can find concurrency bugs pretty fast, _provided_ that one pays attention to the careful *modeling* of the system. This is where one of the difficulties of using systematic concurrency testing tools lies: how to "minimize the model"(*) in order to fight the combinatorial explosion. Once this is done properly, finding concurrency bugs is easy, because most such bugs require very few preemptions/context switches to manifest themselves. (*) In many cases the part that needs to be minimized consists of the components of the system that are not tested; in some others it's the complete program under test. Unfortunately, this manual minimization process that tools which test very thoroughly require (e,g., in a program with N reader processes and M writer processes, all concurrent, one does not need to use large values of N and M to find a data race; using just one of each kind will suffice for read/write races and two writers for write/write races or immutability violations) goes very much against the idea of "you need to stress-test to increase your likelihood of finding concurrency errors" that many programmers have been brainwashed to use, which may or may not be related to why random sampling often works well. But if one is interested just in "finding bugs", bounding techniques can often do wonders. [1] reports by that by using bounding, Concuerror finds a bug in some particular method/program (Method 1 in that paper) in less than a minute, while without bounding the tool needs 144 hours to detect the bug. (Later, the same paper shows that, even without bounding, the bugs in that method are detected very fast if one pays attention to the model [1, Table 2].) An interesting (to me at least) question is what should happen after all "shallow" bugs have been fixed and a particular tool does NOT detect a bug anymore. What sort of guarantees are provided by the tool? Isn't it time to use a systematic instead of a random-based tool? This is exactly what I was referring to in my previous mail where I wrote: Thus Concuerror, in principle at least (i.e., modulo bugs in its code), actually *verifies*, using stateless model checking, that a particular (part of a) program is free of concurrency errors, in contrast to e.g. QuickCheck/PULSE which fundamentally cannot provide any correctness guarantees (other than probabilistic ones). As both [1] and [2] report, it's possible to get actual guarantees about the absence of certain classes of (concurrency) errors using a stateless model checking tool, even in code bases of significant size such as portions of the Linux kernel. This guarantee exists _because of_ the systematic exploration and the fact that the tools are sound (i.e., can in principle detect all errors). Kostis [1] Testing and Verifying Chain Repair Methods for Corfu Using Stateless Model Checking (http://concuerror.com/assets/pdf/iFM2017.pdf) [2] Stateless model checking of the Linux kernel's hierarchical read-copy-update (Tree RCU) - https://dl.acm.org/citation.cfm?id=3092287 From quantumpotato@REDACTED Wed Oct 4 02:13:15 2017 From: quantumpotato@REDACTED (qp) Date: Tue, 3 Oct 2017 20:13:15 -0400 Subject: [erlang-questions] No Makefile Found In-Reply-To: References: Message-ID: Hi, that changed the error I got! Now I get a kernel pid crash: https://gist.github.com/quantumproducer/44ed820a42a02435beab413d4177c33d Again, I can cd into my original directory and run fine. On Tue, Oct 3, 2017 at 3:08 PM, Robert Raschke wrote: > It looks a bit like your cowboy dependency isn't building correctly. Try > doing a gmake distclean and then the gmake run. That should redo the deps. > > HTH, > Robby > > > On 3 Oct 2017 07:05, "qp" wrote: > > When I copy a folder of my erlang project on my local machine, I am able > to cd into the copied directory and gmake run successfully. > When I upload that folder to github and download it, I get "no makefile > found" > > ~/tdep/gmake run > Error: No Makefile to build dependency /Users/quantum/tdep/deps/cowboy. > gmake: *** [erlang.mk:4182: deps] Error 2 > > from https://github.com/QuantumProductions/tunnel-deploy > > There is a Makefile present: > > ~/cd tdep > ~/tdep/ls > Makefile deps erlang.mk rel src tunnel.d > _rel ebin oldh relx.config static > ~/tdep/cat Makefile > PROJECT = tunnel > PROJECT_DESCRIPTION = Find?k > PROJECT_VERSION = 0.1.0 > > DEPS = cowboy jiffy > dep_cowboy_commit = master > > dep_jiffy = git https://github.com/davisp/jiffy > > include erlang.mk > > What is going on here? Could there be some setting on my machine that's > affecting how gmake works? > > I expected to be able to clone this repo to my server and run it there. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arunp@REDACTED Wed Oct 4 08:09:02 2017 From: arunp@REDACTED (Arun) Date: Wed, 4 Oct 2017 11:39:02 +0530 Subject: [erlang-questions] ssh_sftp start_channel function call is getting failed Message-ID: Hi all, We are running a distributed system with one server node and hundreds of client nodes(the systems are distributed over the intra-net). We have a requirement that all the client nodes will connect to the server node and try to download some file(mostly the same file will be accessed by all client nodes) *simultaneously* by using *sftp. *The steps we follow for downloading the file is: 1. Establish ssh sftp connection between the server node and client node using ssh_sftp:start_channel/2 function call. 2. Then reads the file by doing ssh_sftp:read_file/2 function call. The problem what we are facing is that when the number of clients are more then it is observed that few client nodes are failing to establish connection between server node. i.e. the ssh_sftp:start_channel/2 function call is failing. Can somebody please explain me; 1. Is there any limitation for the number of sftp session what we can establish in a single system ? 2. What are the possible reason because of which the connection request fails ? 3. Is there anything wrong we are doing in this approach ? 4. Is there any better solution by which we can guarantee that all client nodes will be able to connect to server and will be able to download the file. *Observation:* We tried to connect 25 client nodes to the server; during the first try only 2 nodes failed to connect and on the second try 5 nodes failed to connect. Why this random behavior ?. Thanks in advance, Arun P. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Wed Oct 4 08:18:12 2017 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 4 Oct 2017 08:18:12 +0200 Subject: [erlang-questions] No Makefile Found In-Reply-To: References: Message-ID: The tunnel app is trying to call a function called cowboy:start_clear which can't be found. I think that's a cowboy v2 function and you are pulling v1 as a dependency. On 4 Oct 2017 02:13, "qp" wrote: > Hi, that changed the error I got! Now I get a kernel pid crash: > > https://gist.github.com/quantumproducer/44ed820a42a02435beab413d4177c33d > > Again, I can cd into my original directory and run fine. > > On Tue, Oct 3, 2017 at 3:08 PM, Robert Raschke > wrote: > >> It looks a bit like your cowboy dependency isn't building correctly. Try >> doing a gmake distclean and then the gmake run. That should redo the deps. >> >> HTH, >> Robby >> >> >> On 3 Oct 2017 07:05, "qp" wrote: >> >> When I copy a folder of my erlang project on my local machine, I am able >> to cd into the copied directory and gmake run successfully. >> When I upload that folder to github and download it, I get "no makefile >> found" >> >> ~/tdep/gmake run >> Error: No Makefile to build dependency /Users/quantum/tdep/deps/cowboy. >> gmake: *** [erlang.mk:4182: deps] Error 2 >> >> from https://github.com/QuantumProductions/tunnel-deploy >> >> There is a Makefile present: >> >> ~/cd tdep >> ~/tdep/ls >> Makefile deps erlang.mk rel src tunnel.d >> _rel ebin oldh relx.config static >> ~/tdep/cat Makefile >> PROJECT = tunnel >> PROJECT_DESCRIPTION = Find?k >> PROJECT_VERSION = 0.1.0 >> >> DEPS = cowboy jiffy >> dep_cowboy_commit = master >> >> dep_jiffy = git https://github.com/davisp/jiffy >> >> include erlang.mk >> >> What is going on here? Could there be some setting on my machine that's >> affecting how gmake works? >> >> I expected to be able to clone this repo to my server and run it there. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fchschneider@REDACTED Wed Oct 4 11:03:33 2017 From: fchschneider@REDACTED (Frans Schneider) Date: Wed, 4 Oct 2017 11:03:33 +0200 Subject: [erlang-questions] gen_statem next_event and appending to the event queue Message-ID: <0b7881b5-b776-8e71-9402-7be2f2e5cabd@gmail.com> Dear list, I have a state machine which implements the RTPS reader locator [1]. Basically, it has a list of unsent changes which should be send one by one to some transport. If the list is empty, the state becomes idle, otherwise it keeps the state pushing and uses the state callback action {next_event, internal, can_send} to process the next change from the list. However, the state machine also has to respond to some other events like the one which will replace the list with unsent changes with a new one (this is according to the specs.) But, since the callback action next_event inserts events in front of the event queue, an event like (re)setting the unsent changes list is only processed after all internal events have been processed. Adding events to the back of the queue can of course be done using a gen_statem:cast or a simple '!', but I would like to use a callback action for this as well, which looks tidier and feels better. What about adding an action {add_event, EventType, EventContent}? Frans [1] http://www.omg.org/spec/DDSI-RTPS/2.2/ From raimo+erlang-questions@REDACTED Wed Oct 4 11:20:09 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 4 Oct 2017 11:20:09 +0200 Subject: [erlang-questions] gen_statem next_event and appending to the event queue In-Reply-To: <0b7881b5-b776-8e71-9402-7be2f2e5cabd@gmail.com> References: <0b7881b5-b776-8e71-9402-7be2f2e5cabd@gmail.com> Message-ID: <20171004092009.GA20378@erix.ericsson.se> On Wed, Oct 04, 2017 at 11:03:33AM +0200, Frans Schneider wrote: > Dear list, > > I have a state machine which implements the RTPS reader locator [1]. > Basically, it has a list of unsent changes which should be send one by > one to some transport. If the list is empty, the state becomes idle, > otherwise it keeps the state pushing and uses the state callback action > {next_event, internal, can_send} to process the next change from the list. > However, the state machine also has to respond to some other events like > the one which will replace the list with unsent changes with a new one > (this is according to the specs.) But, since the callback action > next_event inserts events in front of the event queue, an event like > (re)setting the unsent changes list is only processed after all internal > events have been processed. > Adding events to the back of the queue can of course be done using a > gen_statem:cast or a simple '!', but I would like to use a callback > action for this as well, which looks tidier and feels better. What about > adding an action {add_event, EventType, EventContent}? The gen_statem {next_event,,} event queue is not intended to solve all possible event queue problems, and in particular not intended to be a complete event execution model which would create a language within the language. One may handle this with a specific queue in the state e.g with the module 'queue'. That said, there is a trick that might solve this problem, and that is to use a timeout 0. It is guaranteed to be handled by the state machine before all external events. > > Frans > > [1] http://www.omg.org/spec/DDSI-RTPS/2.2/ -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From essen@REDACTED Wed Oct 4 12:36:27 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Wed, 4 Oct 2017 12:36:27 +0200 Subject: [erlang-questions] No Makefile Found In-Reply-To: References: Message-ID: <49f2a298-2d9e-4b6b-71a3-39a812eeff4a@ninenines.eu> Hello, You have empty deps/cowboy and other directories, why? If the directory exists, Erlang.mk assumes the dependency is there. Things work fine after rm -rf deps I also don't recommend committing the .erlang.mk directory. You might also want to update Erlang.mk via "make erlang-mk". Cheers, On 10/03/2017 07:05 AM, qp wrote: > When I copy a folder of my erlang project on my local machine, I am able > to cd into the copied directory and gmake run successfully. > When I upload that folder to github and download it, I get "no makefile > found" > > ~/tdep/gmake run > Error: No Makefile to build dependency /Users/quantum/tdep/deps/cowboy. > gmake: *** [erlang.mk:4182 : deps] Error 2 > > from https://github.com/QuantumProductions/tunnel-deploy > > There is a Makefile present: > > ~/cd tdep > ~/tdep/ls > Makefile??? deps erlang.mk ??? rel??? ??? src > tunnel.d > _rel??? ??? ebin??? ??? oldh??? ??? relx.config??? static > ~/tdep/cat Makefile > PROJECT = tunnel > PROJECT_DESCRIPTION = Find?k > PROJECT_VERSION = 0.1.0 > > DEPS = cowboy jiffy > dep_cowboy_commit = master > > dep_jiffy = git https://github.com/davisp/jiffy > > include erlang.mk > > What is going on here? Could there be some setting on my machine that's > affecting how gmake works? > > I expected to be able to clone this repo to my server and run it there. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From fchschneider@REDACTED Wed Oct 4 13:28:59 2017 From: fchschneider@REDACTED (Frans Schneider) Date: Wed, 4 Oct 2017 13:28:59 +0200 Subject: [erlang-questions] gen_statem next_event and appending to the event queue Message-ID: <127a87b7-7bb4-c750-6bc0-2673075fcf06@gmail.com> > On Wed, Oct 04, 2017 at 11:03:33AM +0200, Frans Schneider wrote: >> Dear list, >> >> I have a state machine which implements the RTPS reader locator [1]. >> Basically, it has a list of unsent changes which should be send one by >> one to some transport. If the list is empty, the state becomes idle, >> otherwise it keeps the state pushing and uses the state callback action >> {next_event, internal, can_send} to process the next change from the list. >> However, the state machine also has to respond to some other events like >> the one which will replace the list with unsent changes with a new one >> (this is according to the specs.) But, since the callback action >> next_event inserts events in front of the event queue, an event like >> (re)setting the unsent changes list is only processed after all internal >> events have been processed. >> Adding events to the back of the queue can of course be done using a >> gen_statem:cast or a simple '!', but I would like to use a callback >> action for this as well, which looks tidier and feels better. What about >> adding an action {add_event, EventType, EventContent}? > > The gen_statem {next_event,,} event queue is not intended to solve all > possible event queue problems, and in particular not intended to be a > complete event execution model which would create a language within > the language. One may handle this with a specific queue in the state > e.g with the module 'queue'. > > That said, there is a trick that might solve this problem, and that is to > use a timeout 0. It is guaranteed to be handled by the state machine > before all external events. Actualy, I don't want the event _before_ the external events but just added at the _end_ of the event queue. If I put it before external events, it would still process the can_send events before the (re)set events which are external. > >> >> Frans >> >> [1] http://www.omg.org/spec/DDSI-RTPS/2.2/ > From essen@REDACTED Wed Oct 4 14:19:10 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Wed, 4 Oct 2017 14:19:10 +0200 Subject: [erlang-questions] [ANN] Cowboy 2.0 Message-ID: Hello, I am glad to announce that the new stable version of Cowboy has been released! You can find more details about the release at https://ninenines.eu/articles/cowboy-2.0.0/ I will be happy to answer any question you may have. Cheers! -- Lo?c Hoguin https://ninenines.eu From essen@REDACTED Wed Oct 4 14:28:20 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Wed, 4 Oct 2017 14:28:20 +0200 Subject: [erlang-questions] [ANN] Looking Glass profiler Message-ID: <5e1067c3-d74d-3760-6ee2-6aaa7eb23a13@ninenines.eu> Hello again, I would also like to introduce you to the Looking Glass profiler that I have been working on for RabbitMQ for most of the year. It currently comes with callgrind output, can profile which processes deal with messages the most and can output a sequential diagram of the messages exchanged between processes. You can find it at https://github.com/rabbitmq/looking_glass I am still working on improving the current features but also the performance of the tracer (which has room for improvement still) and will be added flame graphs output next. It requires Erlang/OTP 19+ because it is using the new NIF tracing interface for better performance. It also compresses the trace files using LZ4 (which is available as a separate NIF). It should work with both rebar and erlang.mk projects. The user guide is available at https://github.com/rabbitmq/looking_glass/blob/master/doc/src/guide/book.asciidoc I am currently at a point where feedback would be great so please forward my way any comments you may have. Thanks in advance! Enjoy! -- Lo?c Hoguin https://ninenines.eu From raimo+erlang-questions@REDACTED Wed Oct 4 15:23:26 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 4 Oct 2017 15:23:26 +0200 Subject: [erlang-questions] gen_statem next_event and appending to the event queue In-Reply-To: <127a87b7-7bb4-c750-6bc0-2673075fcf06@gmail.com> References: <127a87b7-7bb4-c750-6bc0-2673075fcf06@gmail.com> Message-ID: <20171004132326.GB20378@erix.ericsson.se> On Wed, Oct 04, 2017 at 01:28:59PM +0200, Frans Schneider wrote: > > On Wed, Oct 04, 2017 at 11:03:33AM +0200, Frans Schneider wrote: > >> Dear list, > >> > >> I have a state machine which implements the RTPS reader locator [1]. I do not have the time to read that. > >> Basically, it has a list of unsent changes which should be send one by > >> one to some transport. If the list is empty, the state becomes idle, > >> otherwise it keeps the state pushing and uses the state callback action > >> {next_event, internal, can_send} to process the next change from the > list. > >> However, the state machine also has to respond to some other events like > >> the one which will replace the list with unsent changes with a new one > >> (this is according to the specs.) But, since the callback action > >> next_event inserts events in front of the event queue, an event like > >> (re)setting the unsent changes list is only processed after all internal > >> events have been processed. > >> Adding events to the back of the queue can of course be done using a > >> gen_statem:cast or a simple '!', but I would like to use a callback That would be processed after all queued events, and after all other messages in the message queue. > >> action for this as well, which looks tidier and feels better. What about > >> adding an action {add_event, EventType, EventContent}? > > > > The gen_statem {next_event,,} event queue is not intended to solve all > > possible event queue problems, and in particular not intended to be a > > complete event execution model which would create a language within > > the language. One may handle this with a specific queue in the state > > e.g with the module 'queue'. > > > > That said, there is a trick that might solve this problem, and that is to > > use a timeout 0. It is guaranteed to be handled by the state machine > > before all external events. That would be processed after all other queued events and before all messages in the message queue. > > Actualy, I don't want the event _before_ the external events but just > added at the _end_ of the event queue. If I put it before external Yes. That is what will happen. > events, it would still process the can_send events before the (re)set > events which are external. ... You lost me there ... All events in the internal queue will be processed before the gen_statem goes out and receives a new message. And the last of these internal events to process will be a timeout 0 event if there is any. The message queue is not part of the event queue. > > > > >> > >> Frans > >> > >> [1] http://www.omg.org/spec/DDSI-RTPS/2.2/ > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From daniel.goertzen@REDACTED Wed Oct 4 17:01:27 2017 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 04 Oct 2017 15:01:27 +0000 Subject: [erlang-questions] [ANN] Cowboy 2.0 In-Reply-To: References: Message-ID: Congrats! I've used the pre-releases some and map pattern matching on the requests makes for very pleasant code. On Wed, Oct 4, 2017 at 7:19 AM Lo?c Hoguin wrote: > Hello, > > I am glad to announce that the new stable version of Cowboy has been > released! > > You can find more details about the release at > https://ninenines.eu/articles/cowboy-2.0.0/ > > I will be happy to answer any question you may have. > > Cheers! > > -- > Lo?c Hoguin > https://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ali.sabil@REDACTED Wed Oct 4 20:23:30 2017 From: ali.sabil@REDACTED (Ali Sabil) Date: Wed, 4 Oct 2017 20:23:30 +0200 Subject: [erlang-questions] [ANN] Cowboy 2.0 In-Reply-To: References: Message-ID: Yay! Congrats :) On Wed, Oct 4, 2017 at 5:01 PM, Daniel Goertzen wrote: > Congrats! I've used the pre-releases some and map pattern matching on the > requests makes for very pleasant code. > > On Wed, Oct 4, 2017 at 7:19 AM Lo?c Hoguin wrote: > >> Hello, >> >> I am glad to announce that the new stable version of Cowboy has been >> released! >> >> You can find more details about the release at >> https://ninenines.eu/articles/cowboy-2.0.0/ >> >> I will be happy to answer any question you may have. >> >> Cheers! >> >> -- >> Lo?c Hoguin >> https://ninenines.eu >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexakarpov@REDACTED Wed Oct 4 21:19:08 2017 From: alexakarpov@REDACTED (Alexandre Karpov) Date: Wed, 4 Oct 2017 15:19:08 -0400 Subject: [erlang-questions] How RPC works in erlang ? In-Reply-To: <2086D046-061B-4AC3-B063-0F2B3E7C22F7@gmail.com> References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> <2086D046-061B-4AC3-B063-0F2B3E7C22F7@gmail.com> Message-ID: Thank Brujo for the answer; and thanks, Arun, for the question - I was thinking about asking the same; though I've already discovered the concepts of group_leader and io redirection, I needed just the S/O answer from above to settle the matter. Cheers! On Tue, Oct 3, 2017 at 9:20 AM, Brujo Benavides wrote: > Hi Arun, > > That?s *intended*. It happens because of how Erlang redirects all IO > through the process group_leader. > This > is > the best explanatory link I could find googling quickly. > Hope it helps. > > ------------------------------ > *Brujo Benavides * > > > > On Oct 3, 2017, at 10:10, Arun wrote: > > Hi all, > > I have two distributed applications running on two different nodes. And > from one node I tried to call the function on the remote node by doing RPC, > and its been observed that all the debug prints given in the application > running on the remote node is getting printed on the host node's console. > > Can somebody please tell me in detail why this is happening and how the > RPC in erlang works. ? > > Thanks in advance. > Arun P. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Oct 4 23:08:55 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 5 Oct 2017 00:08:55 +0300 Subject: [erlang-questions] [ANN] Looking Glass profiler In-Reply-To: <5e1067c3-d74d-3760-6ee2-6aaa7eb23a13@ninenines.eu> References: <5e1067c3-d74d-3760-6ee2-6aaa7eb23a13@ninenines.eu> Message-ID: Wow, need to look at it! -------------- next part -------------- An HTML attachment was scrubbed... URL: From arunp@REDACTED Thu Oct 5 06:51:33 2017 From: arunp@REDACTED (Arun) Date: Thu, 5 Oct 2017 10:21:33 +0530 Subject: [erlang-questions] How RPC works in erlang ? In-Reply-To: <2086D046-061B-4AC3-B063-0F2B3E7C22F7@gmail.com> References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> <2086D046-061B-4AC3-B063-0F2B3E7C22F7@gmail.com> Message-ID: <9427e5a8-6e85-f824-7623-f1e92956cbdc@utl.in> Hi Brujo, Thanks for the information, but i have one more doubt; is this behavior only for io:format function calls or the execution of the entire function also will be happening in the host node. Regards, Arun P. On Tuesday 03 October 2017 06:50 PM, Brujo Benavides wrote: > Hi Arun, > > That?s /intended/. It happens because of how Erlang redirects all IO > through the process group_leader. > This > is > the best explanatory link I could find googling quickly. > Hope it helps. > ------------------------------------------------------------------------ > */Brujo Benavides /* > ** >> On Oct 3, 2017, at 10:10, Arun > >> wrote: >> Hi all, I have two distributed applications running on two different >> nodes. And from one node I tried to call the function on the remote >> node by doing RPC, and its been observed that all the debug prints >> given in the application running on the remote node is getting >> printed on the host node's console. Can somebody please tell me in >> detail why this is happening and how the RPC in erlang works. ? >> Thanks in advance. Arun P. >> _______________________________________________ erlang-questions >> mailing list erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From elbrujohalcon@REDACTED Thu Oct 5 12:02:57 2017 From: elbrujohalcon@REDACTED (Brujo Benavides) Date: Thu, 05 Oct 2017 10:02:57 +0000 Subject: [erlang-questions] How RPC works in erlang ? In-Reply-To: <9427e5a8-6e85-f824-7623-f1e92956cbdc@utl.in> References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> <2086D046-061B-4AC3-B063-0F2B3E7C22F7@gmail.com> <9427e5a8-6e85-f824-7623-f1e92956cbdc@utl.in> Message-ID: Only for io, AFAIK. On Thu, 5 Oct 2017 at 01:51 Arun wrote: > Hi Brujo, > > Thanks for the information, but i have one more doubt; is this behavior > only for io:format function calls or the execution of the entire function > also will be happening in the host node. > Regards, > Arun P. > > > On Tuesday 03 October 2017 06:50 PM, Brujo Benavides wrote: > > Hi Arun, > > That?s *intended*. It happens because of how Erlang redirects all IO > through the process group_leader. > This > is > the best explanatory link I could find googling quickly. > Hope it helps. > ------------------------------ > *Brujo Benavides * > > On Oct 3, 2017, at 10:10, Arun wrote: > Hi all, I have two distributed applications running on two different > nodes. And from one node I tried to call the function on the remote node by > doing RPC, and its been observed that all the debug prints given in the > application running on the remote node is getting printed on the host > node's console. Can somebody please tell me in detail why this is happening > and how the RPC in erlang works. ? Thanks in advance. Arun P. > _______________________________________________ erlang-questions mailing > list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- elbrujohalcon @ iPhone -------------- next part -------------- An HTML attachment was scrubbed... URL: From silver.surfertab@REDACTED Thu Oct 5 12:57:16 2017 From: silver.surfertab@REDACTED (Greg) Date: Thu, 5 Oct 2017 16:27:16 +0530 Subject: [erlang-questions] Why the output differs in these two erlang expression sequence in shell? Message-ID: In Erlang shell why the following produces different result? 1> Total=15. 2> Calculate=fun(Number)-> Total=2*Number end. 3> Calculate(6). **exception error: no match of right hand side value 12** 1> Calculate=fun(Number)-> Total=2*Number end. 2> Total=15. 3> Calculate(6). 12 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo@REDACTED Thu Oct 5 13:17:17 2017 From: hugo@REDACTED (Hugo Mills) Date: Thu, 5 Oct 2017 11:17:17 +0000 Subject: [erlang-questions] Why the output differs in these two erlang expression sequence in shell? In-Reply-To: References: Message-ID: <20171005111717.GH3293@carfax.org.uk> On Thu, Oct 05, 2017 at 04:27:16PM +0530, Greg wrote: > In Erlang shell why the following produces different result? It's about scope of variables, closures, and the fact that the = operator isn't an assignment operator but a matching operator. > 1> Total=15. Match the unbound "Total" against value 15: Total is bound to the value "15". At this point, "Total" is now a bound variable in the outer scope (the shell). > 2> Calculate=fun(Number)-> Total=2*Number end. "Total" is already bound, so you're using the _bound_ variable in the fun, so it has a value (which is immutable) in the context of the fun. The fun here actually consists of a function of one variable, plus a context which contains the bound variable Total, which is the context in which that fun is evaluated. This combination of a fun+context is usually called a closure. > 3> Calculate(6). Total is bound to the value 15 in the context of the closure. 2*Number is 12. 15 does not match 12, and so: > **exception error: no match of right hand side value 12** Observe that if you ran f(Total) to make the shell forget about the value of Total, the original value of Total would still be bound into the Calculate closure, and you get the same result: 1> Total=15. 15 2> Calculate = fun(Number) -> Total = 2*Number end. #Fun 3> f(Total). ok 4> Calculate(6). ** exception error: no match of right hand side value 12 > 1> Calculate=fun(Number)-> Total=2*Number end. Defines a fun where Total is _unbound_ in the fun, and is local to the fun, so the closure doesn't contain "Total" (or at least, contains an unbound "Total"). > 2> Total=15. Defines a different binding of Total, in the scope of the shell. This can't affect the local variable inside the fun at this point. > 3> Calculate(6). Computes 2*Number, which is 12, and matches it against the unbound local variable Total, which binds Total to the value 12. The value of the function is then: > 12 Note that the fun definition (i.e. the closure) isn't modified by this operation. So you can safely run the fun again, and the Total value is still unbound, because it's only bound to a value within the context of a function call. Each call to the function uses the original context which was attached to the fun (thus making a closure) when the fun was defined. I'm not sure if that actually helps make things any clearer. I hope so. Basically, if you bind a variable to a value and then, in the same context, define a fun which uses that variable, the value of the variable at that moment gets included in the fun definition. Hugo. -- Hugo Mills | Prof Brain had been in search of The Truth for 25 hugo@REDACTED carfax.org.uk | years, with the intention of putting it under house http://carfax.org.uk/ | arrest. PGP: E2AB1DE4 | Zeb Carter, The Number of the Beast -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From rickard@REDACTED Thu Oct 5 19:41:29 2017 From: rickard@REDACTED (Rickard Green) Date: Thu, 5 Oct 2017 19:41:29 +0200 Subject: [erlang-questions] Patch package OTP 19.3.6.3 released Message-ID: Patch Package: OTP 19.3.6.3 Git Tag: OTP-19.3.6.3 Date: 2017-10-05 Trouble Report Id: OTP-14522, OTP-14548, OTP-14590, OTP-14612, OTP-14661 Seq num: ERIERL-48, ERIERL-80, ERL-468, ERL-474 System: OTP Release: 19 Application: compiler-7.0.4.1, erts-8.3.5.3 Predecessor: OTP 19.3.6.2 Check out the git tag OTP-19.3.6.3, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- compiler-7.0.4.1 ------------------------------------------------ --------------------------------------------------------------------- The compiler-7.0.4.1 application can be applied independently of other applications on a full OTP 19 installation. --- Fixed Bugs and Malfunctions --- OTP-14522 Application(s): compiler Related Id(s): ERIERL-48 Fail labels on guard BIFs weren't taken into account during an optimization pass, and a bug in the validation pass sometimes prevented this from being noticed when a fault occurred. Full runtime dependencies of compiler-7.0.4.1: crypto-3.6, erts-7.0, hipe-3.12, kernel-4.0, stdlib-2.5 --------------------------------------------------------------------- --- erts-8.3.5.3 ---------------------------------------------------- --------------------------------------------------------------------- Note! The erts-8.3.5.3 application can *not* be applied independently of other applications on an arbitrary OTP 19 installation. On a full OTP 19 installation, also the following runtime dependency has to be satisfied: -- sasl-3.0.1 (first satisfied in OTP 19.1) --- Fixed Bugs and Malfunctions --- OTP-14548 Application(s): erts Related Id(s): ERL-468, OTP-11997 A timer internal bit-field used for storing scheduler id was too small. As a result, VM internal timer data structures could become inconsistent when using 1024 schedulers on the system. Note that systems with less than 1024 schedulers are not effected by this bug. This bug was introduced in ERTS version 7.0 (OTP 18.0). OTP-14590 Application(s): erts Related Id(s): ERL-474 Fixed bug in binary_to_term and binary_to_atom that could cause VM crash. Typically happens when the last character of an UTF8 string is in the range 128 to 255, but truncated to only one byte. Bug exists in binary_to_term since ERTS version 5.10.2 (OTP_R16B01) and binary_to_atom since ERTS version 9.0 (OTP-20.0). OTP-14612 Application(s): erts Fix bug causing VM crash when a module with -on_load directive is loaded while erlang:trace(on_load, ...) is enabled. OTP-14661 Application(s): erts Related Id(s): ERIERL-80 Fixed bug that could cause a VM crash when a corrupt message is received on distribution channel from other node. Full runtime dependencies of erts-8.3.5.3: kernel-5.0, sasl-3.0.1, stdlib-3.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From mfidelman@REDACTED Thu Oct 5 19:50:41 2017 From: mfidelman@REDACTED (Miles Fidelman) Date: Thu, 5 Oct 2017 10:50:41 -0700 Subject: [erlang-questions] Why RPC in Erlang? [was: How RPC works in erlang?] In-Reply-To: References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> <2086D046-061B-4AC3-B063-0F2B3E7C22F7@gmail.com> <9427e5a8-6e85-f824-7623-f1e92956cbdc@utl.in> Message-ID: <34426b55-3455-2c53-96fb-5809abe57b12@meetinghouse.net> I guess an obvious question is why does Erlang have an RPC mechanism at all?? It's so at odds with the basic architecture of message-passing concurrency. Miles Fidelman On 10/5/17 3:02 AM, Brujo Benavides wrote: > Only for io, AFAIK. > On Thu, 5 Oct 2017 at 01:51 Arun > > wrote: > > Hi Brujo, > > Thanks for the information, but i have one more doubt; is this > behavior only for io:format function calls or the execution of the > entire function also will be happening in the host node. > > Regards, > Arun P. > > > On Tuesday 03 October 2017 06:50 PM, Brujo Benavides wrote: >> Hi Arun, >> >> That?s /intended/. It happens because of how Erlang redirects all >> IO through the process group_leader. >> This >> ?is >> the best explanatory link I could find googling quickly. >> Hope it helps. >> ------------------------------------------------------------------------ >> */Brujo Benavides /* >> ** >>> On Oct 3, 2017, at 10:10, Arun >> > wrote: >>> Hi all, I have two distributed applications running on two >>> different nodes. And from one node I tried to call the function >>> on the remote node by doing RPC, and its been observed that all >>> the debug prints given in the application running on the remote >>> node is getting printed on the host node's console. Can somebody >>> please tell me in detail why this is happening and how the RPC >>> in erlang works. ? Thanks in advance. Arun P. >>> _______________________________________________ erlang-questions >>> mailing list erlang-questions@REDACTED >>> >>> http://erlang.org/mailman/listinfo/erlang-questions > > -- > elbrujohalcon @ iPhone > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra -------------- next part -------------- An HTML attachment was scrubbed... URL: From elbrujohalcon@REDACTED Thu Oct 5 19:55:33 2017 From: elbrujohalcon@REDACTED (Brujo Benavides) Date: Thu, 5 Oct 2017 14:55:33 -0300 Subject: [erlang-questions] Why RPC in Erlang? [was: How RPC works in erlang?] In-Reply-To: <34426b55-3455-2c53-96fb-5809abe57b12@meetinghouse.net> References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> <2086D046-061B-4AC3-B063-0F2B3E7C22F7@gmail.com> <9427e5a8-6e85-f824-7623-f1e92956cbdc@utl.in> <34426b55-3455-2c53-96fb-5809abe57b12@meetinghouse.net> Message-ID: Hi Miles, I would go with: because someone needed it at some point. And now if you need it, too? you don?t have to reinvent it now. And to be super-precise (and a bit pedantic, I know): RPC is not in Erlang (i.e. in the VM), it comes (and is implemented using) OTP. In other words: it?s a tool built with the language, it?s not a part of the language like the basic architecture of message-passing concurrency. Cheers! Brujo Benavides > On Oct 5, 2017, at 14:50, Miles Fidelman wrote: > > I guess an obvious question is why does Erlang have an RPC mechanism at all? It's so at odds with the basic architecture of message-passing concurrency. > > Miles Fidelman > > On 10/5/17 3:02 AM, Brujo Benavides wrote: >> Only for io, AFAIK. >> On Thu, 5 Oct 2017 at 01:51 Arun > wrote: >> Hi Brujo, >> >> Thanks for the information, but i have one more doubt; is this behavior only for io:format function calls or the execution of the entire function also will be happening in the host node. >> >> Regards, >> Arun P. >> >> >> On Tuesday 03 October 2017 06:50 PM, Brujo Benavides wrote: >>> Hi Arun, >>> >>> That?s intended. It happens because of how Erlang redirects all IO through the process group_leader. >>> This is the best explanatory link I could find googling quickly. >>> Hope it helps. >>> >>> Brujo Benavides >>>> On Oct 3, 2017, at 10:10, Arun > wrote: >>>> Hi all, I have two distributed applications running on two different nodes. And from one node I tried to call the function on the remote node by doing RPC, and its been observed that all the debug prints given in the application running on the remote node is getting printed on the host node's console. Can somebody please tell me in detail why this is happening and how the RPC in erlang works. ? Thanks in advance. Arun P. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions >> -- >> elbrujohalcon @ iPhone >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Thu Oct 5 21:37:25 2017 From: vans_163@REDACTED (Vans S) Date: Thu, 5 Oct 2017 19:37:25 +0000 (UTC) Subject: [erlang-questions] Why RPC in Erlang? [was: How RPC works in erlang?] In-Reply-To: References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> <2086D046-061B-4AC3-B063-0F2B3E7C22F7@gmail.com> <9427e5a8-6e85-f824-7623-f1e92956cbdc@utl.in> <34426b55-3455-2c53-96fb-5809abe57b12@meetinghouse.net> Message-ID: <17826829.2140387.1507232245409@mail.yahoo.com> I agree here. I think RPC and gen_server calls which are the same thing really should not have been included/baked in. Instead there should of been a separate OTP behavior called async/wait or something. ?This behavior can turn any async message into a sync procedure.? Instead of doing ?gen_server:call(name, {msg}), we do ?{reply, AsyncAwait} = erlang:send(Pid, {self(), msg}, {reply, '_'}, 5000), what this means, send this message to the pid, and wait until a {reply, '_'} message comes, after 5000 ms timeout and error. ? This way is much more generic, and simple. On Thursday, October 5, 2017 1:55 PM, Brujo Benavides wrote: Hi Miles, I would go with: because someone needed it at some point. And now if you need it, too? you don?t have to reinvent it now. And to be super-precise (and a bit pedantic, I know): RPC is not in Erlang (i.e. in the VM), it comes (and is implemented using) OTP. In other words: it?s a tool built with the language, it?s not a part of the language like the basic architecture of message-passing concurrency. Cheers! Brujo Benavides On Oct 5, 2017, at 14:50, Miles Fidelman wrote: I guess an obvious question is why does Erlang have an RPC mechanism at all?? It's so at odds with the basic architecture of message-passing concurrency.Miles Fidelman On 10/5/17 3:02 AM, Brujo Benavides wrote: Only for io, AFAIK. On Thu, 5 Oct 2017 at 01:51 Arun wrote: Hi Brujo,Thanks for the information, but i have one more doubt; is this behavior only for io:format function calls or the execution of the entire function also will be happening in the host node. Regards, Arun P. On Tuesday 03 October 2017 06:50 PM, Brujo Benavides wrote: Hi Arun, That?s intended. It happens because of how Erlang redirects all IO through the process group_leader. This?is the best explanatory link I could find googling quickly. Hope it helps. Brujo Benavides On Oct 3, 2017, at 10:10, Arun wrote: Hi all, I have two distributed applications running on two different nodes. And from one node I tried to call the function on the remote node by doing RPC, and its been observed that all the debug prints given in the application running on the remote node is getting printed on the host node's console. Can somebody please tell me in detail why this is happening and how the RPC in erlang works. ? Thanks in advance. Arun P. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -- elbrujohalcon @ iPhone _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Oct 6 04:03:11 2017 From: zxq9@REDACTED (zxq9) Date: Fri, 06 Oct 2017 11:03:11 +0900 Subject: [erlang-questions] Why RPC in Erlang? [was: How RPC works in erlang?] In-Reply-To: <34426b55-3455-2c53-96fb-5809abe57b12@meetinghouse.net> References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> <34426b55-3455-2c53-96fb-5809abe57b12@meetinghouse.net> Message-ID: <1670503.cvL8OcreNN@changa> On 2017?10?05? ??? 10:50:41 Miles Fidelman wrote: > I guess an obvious question is why does Erlang have an RPC mechanism at > all? It's so at odds with the basic architecture of message-passing > concurrency. To be more precise, "Why is there an RPC module in the standard library?" RPC is not part of the language or runtime, of course, it is just a module written on top of the bits that already exist in Erlang. Any message that is sent might initiate an RPC -- or might not. The term "RPC" doesn't even make much sense to use in the context of Erlang, and one might notice that this term is actually never used by long-time Erlangers. I don't think I've ever seen any code that uses the RPC module, either. Maybe at some point someone thought they needed it, or there may have been pressure to include that to comply with someone's expectations -- but I've never found it useful or seen anyone other than beginning Erlangers even read the docs or ask questions about it (and they stop once they find out how gen_server:call/2 works over disterl). "Why is there BERT?", on the other hand, is a question with a ready answer. -Craig From heinz@REDACTED Fri Oct 6 04:18:22 2017 From: heinz@REDACTED (Heinz N. Gies) Date: Fri, 6 Oct 2017 04:18:22 +0200 Subject: [erlang-questions] Why RPC in Erlang? [was: How RPC works in erlang?] In-Reply-To: <1670503.cvL8OcreNN@changa> References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> <34426b55-3455-2c53-96fb-5809abe57b12@meetinghouse.net> <1670503.cvL8OcreNN@changa> Message-ID: RPC is used for tests (when you want to control another node) and CLI wrappers, (when you want to execute commands on a remote node from a script). In both cases it makes sense and isn?t at odds with the erlang principles. > On 6. Oct 2017, at 04:03, zxq9 wrote: > > On 2017?10?05? ??? 10:50:41 Miles Fidelman wrote: >> I guess an obvious question is why does Erlang have an RPC mechanism at >> all? It's so at odds with the basic architecture of message-passing >> concurrency. > > To be more precise, "Why is there an RPC module in the standard library?" > > RPC is not part of the language or runtime, of course, it is just a module > written on top of the bits that already exist in Erlang. Any message that > is sent might initiate an RPC -- or might not. The term "RPC" doesn't even > make much sense to use in the context of Erlang, and one might notice that > this term is actually never used by long-time Erlangers. > > I don't think I've ever seen any code that uses the RPC module, either. > Maybe at some point someone thought they needed it, or there may have been > pressure to include that to comply with someone's expectations -- but I've > never found it useful or seen anyone other than beginning Erlangers even > read the docs or ask questions about it (and they stop once they find out > how gen_server:call/2 works over disterl). > > "Why is there BERT?", on the other hand, is a question with a ready answer. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From zxq9@REDACTED Fri Oct 6 04:37:02 2017 From: zxq9@REDACTED (zxq9) Date: Fri, 06 Oct 2017 11:37:02 +0900 Subject: [erlang-questions] Why RPC in Erlang? [was: How RPC works in erlang?] In-Reply-To: References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> <1670503.cvL8OcreNN@changa> Message-ID: <5078126.RgNHhUtOgi@changa> On 2017?10?06? ??? 04:18:22 Heinz N. Gies wrote: > RPC is used for tests (when you want to control another node) and > CLI wrappers, (when you want to execute commands on a remote node > from a script). In both cases it makes sense and isn?t at odds with > the erlang principles. I didn't say it was at odds with Erlang principles. I said the rpc module is essentially useless. Thinking that the rpc module is specially useful for tests, CLI wrappers, or when you want to execute commands on a remote node (from a script or otherwise) is a bit of an odd conclusion to draw, given the way that the even-arity spawn_* functions work. If you mean "the IO redirect thing" then sure, the group leader is switched if you pass the call through rex so calls to an IO function on the remote node will turn up on the local shell. But that prompts the question: Why aren't you returning a value in the first place? Asking for a remote note to cause a side effect on the local node violates the concept of RPC, actually. Which leads me to think that this module is inappropriately named. When people want to offload an expensive procedure (that is, actually remotely call a procedure) they either send it over some protocol unrelated to disterl or use spawn_monitor/2,4 directly -- they don't tend pass it through rex. When people want a special IO side effect so that remote calls to IO functions magically redirect themselves to the calling node they use the rpc module -- which is not at all what the term RPC means. The semantics surrounding this trips up quite a few beginners I've noticed (noticed a lot more this month as a new school session has really gotten in gear). -Craig From roger@REDACTED Fri Oct 6 11:43:48 2017 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 6 Oct 2017 10:43:48 +0100 Subject: [erlang-questions] Why RPC in Erlang? [was: How RPC works in erlang?] In-Reply-To: <5078126.RgNHhUtOgi@changa> References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> <1670503.cvL8OcreNN@changa> <5078126.RgNHhUtOgi@changa> Message-ID: On 6 October 2017 at 03:37, zxq9 wrote: > Thinking that the rpc module is specially useful for tests, CLI > wrappers, or when you want to execute commands on a remote node (from > a script or otherwise) is a bit of an odd conclusion to draw, given > the way that the even-arity spawn_* functions work. On occasion, we've needed to (e.g.) tweak application settings in a remote node, at runtime, during system tests, synchronously. Sure, I could do that with spawn_*, but that's a lot of plumbing I'd need to write. Hey, look: someone already did; it's in the rpc module. We *also* use spawn_*; we've even got some code that pushes a local module into the remote node, and spawns a process running *that*, but sometimes: you just want what the rpc module offers. I also note that erl_call uses the "rex" server... From zxq9@REDACTED Fri Oct 6 16:47:45 2017 From: zxq9@REDACTED (zxq9) Date: Fri, 06 Oct 2017 23:47:45 +0900 Subject: [erlang-questions] Why RPC in Erlang? [was: How RPC works in erlang?] In-Reply-To: References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> <5078126.RgNHhUtOgi@changa> Message-ID: <6643279.YoUT2Rh97o@changa> On 2017?10?06? ??? 10:43:48 Roger Lipscombe wrote: > On 6 October 2017 at 03:37, zxq9 wrote: > > Thinking that the rpc module is specially useful for tests, CLI > > wrappers, or when you want to execute commands on a remote node (from > > a script or otherwise) is a bit of an odd conclusion to draw, given > > the way that the even-arity spawn_* functions work. > > On occasion, we've needed to (e.g.) tweak application settings in a > remote node, at runtime, during system tests, synchronously. Sure, I > could do that with spawn_*, but that's a lot of plumbing I'd need to > write. Hey, look: someone already did; it's in the rpc module. Sure. I'm not disagreeing with that. I'm disagreeing with the false notion that this is what "remote procedure call" actually means. -Craig From vans_163@REDACTED Fri Oct 6 17:24:14 2017 From: vans_163@REDACTED (Vans S) Date: Fri, 6 Oct 2017 15:24:14 +0000 (UTC) Subject: [erlang-questions] Why RPC in Erlang? [was: How RPC works in erlang?] In-Reply-To: <6643279.YoUT2Rh97o@changa> References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> <5078126.RgNHhUtOgi@changa> <6643279.YoUT2Rh97o@changa> Message-ID: <722213155.2281466.1507303454406@mail.yahoo.com> > I'm disagreeing with the false notion that this is what "remote > procedure call" actually means. RPC practically (not theoretically) means to send a message and get a reply back to the sender for that particular message. ?It is different from the request/response model because it allows non-blocking multiplexing on 1 data channel.? Usually this is implemented by passing a unique ID in the message mapping to a callback on the sender, in erlangs case this is the pid of the sender, but it can be a service dedicated to RPC. On Friday, October 6, 2017 10:48 AM, zxq9 wrote: On 2017?10?06? ??? 10:43:48 Roger Lipscombe wrote: > On 6 October 2017 at 03:37, zxq9 wrote: > > Thinking that the rpc module is specially useful for tests, CLI > > wrappers, or when you want to execute commands on a remote node (from > > a script or otherwise) is a bit of an odd conclusion to draw, given > > the way that the even-arity spawn_* functions work. > > On occasion, we've needed to (e.g.) tweak application settings in a > remote node, at runtime, during system tests, synchronously. Sure, I > could do that with spawn_*, but that's a lot of plumbing I'd need to > write. Hey, look: someone already did; it's in the rpc module. Sure. I'm not disagreeing with that. I'm disagreeing with the false notion that this is what "remote procedure call" actually means. -Craig _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From silver.surfertab@REDACTED Fri Oct 6 18:43:24 2017 From: silver.surfertab@REDACTED (Greg) Date: Fri, 6 Oct 2017 22:13:24 +0530 Subject: [erlang-questions] Why RPC in Erlang? [was: How RPC works in erlang?] In-Reply-To: <6643279.YoUT2Rh97o@changa> References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> <5078126.RgNHhUtOgi@changa> <6643279.YoUT2Rh97o@changa> Message-ID: So when to use rpc:call and when to send a message by RemotePID ! {call, procedure}? Is there any general guidelines? There was these long standing debate over this http://armstrongonsoftware.blogspot.in/2008/05/road-we-didnt-go-down.html; On Fri, Oct 6, 2017 at 8:17 PM, zxq9 wrote: > On 2017?10?06? ??? 10:43:48 Roger Lipscombe wrote: > > On 6 October 2017 at 03:37, zxq9 wrote: > > > Thinking that the rpc module is specially useful for tests, CLI > > > wrappers, or when you want to execute commands on a remote node (from > > > a script or otherwise) is a bit of an odd conclusion to draw, given > > > the way that the even-arity spawn_* functions work. > > > > On occasion, we've needed to (e.g.) tweak application settings in a > > remote node, at runtime, during system tests, synchronously. Sure, I > > could do that with spawn_*, but that's a lot of plumbing I'd need to > > write. Hey, look: someone already did; it's in the rpc module. > > Sure. I'm not disagreeing with that. > > I'm disagreeing with the false notion that this is what "remote > procedure call" actually means. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Oct 6 20:07:59 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 06 Oct 2017 18:07:59 +0000 Subject: [erlang-questions] Erlang, mini-erlang, and TLA+ In-Reply-To: References: <27f5ddbe-f2e3-ccd5-4081-dd06bae13f2b@gmail.com> <842384618.10511969.1506450165630@mail.yahoo.com> Message-ID: General rule when trying to build language cores formally: Every construction which can be written in terms of another construction should be. * ETS tables are really just processes * The Process dictionary is just a state monad in the language * You need monads for message passing side effects anyway, so you can probably reuse the work on top * The whole exception system can be cast inside the monad as well * Ports can be reimagined as special processes and you can use messaging in order to handle them The reason is that any construction is a formal liability: it needs special handling. It is often more beneficial to generalize the system a little bit in order to vastly simplify the number of cases you have to handle in proofs. On Wed, Sep 27, 2017 at 7:20 AM Karlo Kuna wrote: > > No - write a new runtime but inspired by and stealing good bits from the > old > > what would you consider good bits here? > > i also suspect that mini erlang would be better off without process > dictionaries? > > > > On Tue, Sep 26, 2017 at 10:59 PM, Joe Armstrong wrote: > >> On Tue, Sep 26, 2017 at 8:22 PM, Brady Powers >> wrote: >> > If we're talking about a new instruction set, >> >> Yes >> >> > and removing pieces from the >> > runtime, >> >> No - write a new runtime but inspired by and stealing good bits from the >> old >> >> then are we still talking about a minimal Erlang, >> or are we talking >> >> It would be a subset of Erlang - so strictly not Erlang >> >> > about something new? >> >> Yes >> >> >> > Because I'm sure we could all think of thing's we'd change given the >> > opportunity. Joe's already written Erl2. >> >> No - erl2 just compiles to erl >> >> /Joe >> >> > -Brady >> > On Tuesday, September 26, 2017, 12:36:04 PM EDT, Joe Armstrong >> > wrote: >> > >> > >> > On Tue, Sep 26, 2017 at 5:47 PM, Todd Greenwood-Geer >> > wrote: >> >> Hi, >> >> >> >> 1. Are there existing TLA+ specifications [1] for Erlang? >> > >> > Not that I know of. >> > >> >> >> >> Recent discussions regarding the behavior of messages for local and >> remote >> >> processes, the behavior of data in ETS, and the discussion about a new >> >> mini-erlang all make me think there would be great benefit in formal >> >> specifications of Erlang behaviors. >> >> >> >> For instance, I've been thinking about writing various widgets for >> >> distributed applications, each with a well defined TLA+ specification, >> and >> >> then starting to compose larger and more interesting applications from >> >> them. >> >> >> >> My workflow would look like this: >> >> Write widget spec -> impl widget -> Write application spec (composed of >> >> previously spec'd widgets) -> impl application -> profit! >> > >> > Now sure about the profit bit at the end. >> > >> > (write crap, sell it, sell thousands of consulting hours to fix it = >> profit >> > :-) >> > >> >> >> >> The runtime behaviors such as message passing, process linking, etc. >> would >> >> be the first level that would need specifications... so has this ever >> been >> >> tackled? >> > >> > Years ago - but not in phase with the distribution. >> > >> > There has always been a bit of gap between reality (C code, and what >> > it does) and specifications (math, and what it should do). The two are >> > not the same. >> > >> >> >> >> 2. Regarding the mini-erlang, I'd really like to get involved in this >> >> project. Perhaps I should be replying to a separate thread, but my >> >> questions >> >> are: >> > >> > There is no project - just a few people discussion what they would like >> to >> > see >> > in a new system. >> > >> > >> >> >> >> * What are the goals of mini-erlang? >> > >> > My goal would be a tiny kernel - performance is >> > of secondary importance. Tiny run-time footprint. >> > >> >> * How are these goals different from Erlang? >> > >> > The current Erlang is not designed for a small memory footprint. >> > >> >> * What needs to be removed/added from Erlang to make mini-erlang? >> > >> > Remove NIFS binarys ets tables (possibly) >> > >> >> * What is the current VM instruction set, and what would be the Mini >> >> instruction set? >> > >> > Current is BEAM - Mini instruction set unknown. >> > >> >> (my reason for posing these questions in this thread) >> >> *** Is there any interest in modeling/specifying the behaviors prior to >> >> implementing? >> > >> > I dont' think so - the main problem is figuring out how to make >> something >> > with tiny footprint - so for example GC strategy and memory layout is >> > very important - I don't think modelling helps. Counting bytes on >> squared >> > paper >> > seems the best method (really) >> > >> >> >> >> Regarding implementation, I'm fairly agnostic. One thought is that a >> >> prototype could be written in anything (python, bash, java, go, c, >> etc.), >> > >> > I disagree - I'd like the final memory footprint to be small. >> > c (and friends) are OK but I don't want to have to include an entire JVM >> > (or whatever). >> > >> > Could implement it in Forth though ... >> > >> >> and once the design has settled down, then anyone should be able to go >> >> back >> >> and re-write in their language of choice. Should be fairly simple since >> >> there would be design docs, formal specifications, test suites, etc. >> Then >> >> we >> >> could have mini-erlang runtime competitions/shootouts ;-). >> > >> > That would be fun. >> > >> > Cheers >> > >> > /Joe >> > >> > >> >> >> >> [1] http://lamport.azurewebsites.net/tla/tla.html >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Fri Oct 6 23:56:09 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 6 Oct 2017 23:56:09 +0200 Subject: [erlang-questions] Erlang, mini-erlang, and TLA+ In-Reply-To: References: <27f5ddbe-f2e3-ccd5-4081-dd06bae13f2b@gmail.com> <842384618.10511969.1506450165630@mail.yahoo.com> Message-ID: Also, it is important that all BIFs that have to do with state outside of the process itself (as opposed to e.g. the process dictionary functions) - such as the process registry functions, system time, timers, etc. - can and should be modelled as messages to and from the corresponding VM subsystem. Yes, this implies a lot of possible race conditions - but these already exist in a modern Beam, where multiple threads are working simultaneously and trying to avoid global locks. Best to get those out in the open right away, so you don't paint yourself in a corner by assuming that certain BIFs always will take effect globally, atomically and instantaneously. I think this is an underappreciated (or at least not widely understood) property of Erlang and its implementation. You could in fact have large parts of its runtime system execute on a physically separate machine from the processes that run your Beam code, without having to change a line in your programs. /Richard 2017-10-06 20:07 GMT+02:00 Jesper Louis Andersen < jesper.louis.andersen@REDACTED>: > General rule when trying to build language cores formally: > > Every construction which can be written in terms of another construction > should be. > > * ETS tables are really just processes > * The Process dictionary is just a state monad in the language > * You need monads for message passing side effects anyway, so you can > probably reuse the work on top > * The whole exception system can be cast inside the monad as well > * Ports can be reimagined as special processes and you can use messaging > in order to handle them > > The reason is that any construction is a formal liability: it needs > special handling. It is often more beneficial to generalize the system a > little bit in order to vastly simplify the number of cases you have to > handle in proofs. > > > On Wed, Sep 27, 2017 at 7:20 AM Karlo Kuna wrote: > >> > No - write a new runtime but inspired by and stealing good bits from >> the old >> >> what would you consider good bits here? >> >> i also suspect that mini erlang would be better off without process >> dictionaries? >> >> >> >> On Tue, Sep 26, 2017 at 10:59 PM, Joe Armstrong wrote: >> >>> On Tue, Sep 26, 2017 at 8:22 PM, Brady Powers >>> wrote: >>> > If we're talking about a new instruction set, >>> >>> Yes >>> >>> > and removing pieces from the >>> > runtime, >>> >>> No - write a new runtime but inspired by and stealing good bits from the >>> old >>> >>> then are we still talking about a minimal Erlang, >>> or are we talking >>> >>> It would be a subset of Erlang - so strictly not Erlang >>> >>> > about something new? >>> >>> Yes >>> >>> >>> > Because I'm sure we could all think of thing's we'd change given the >>> > opportunity. Joe's already written Erl2. >>> >>> No - erl2 just compiles to erl >>> >>> /Joe >>> >>> > -Brady >>> > On Tuesday, September 26, 2017, 12:36:04 PM EDT, Joe Armstrong >>> > wrote: >>> > >>> > >>> > On Tue, Sep 26, 2017 at 5:47 PM, Todd Greenwood-Geer >>> > wrote: >>> >> Hi, >>> >> >>> >> 1. Are there existing TLA+ specifications [1] for Erlang? >>> > >>> > Not that I know of. >>> > >>> >> >>> >> Recent discussions regarding the behavior of messages for local and >>> remote >>> >> processes, the behavior of data in ETS, and the discussion about a new >>> >> mini-erlang all make me think there would be great benefit in formal >>> >> specifications of Erlang behaviors. >>> >> >>> >> For instance, I've been thinking about writing various widgets for >>> >> distributed applications, each with a well defined TLA+ >>> specification, and >>> >> then starting to compose larger and more interesting applications from >>> >> them. >>> >> >>> >> My workflow would look like this: >>> >> Write widget spec -> impl widget -> Write application spec (composed >>> of >>> >> previously spec'd widgets) -> impl application -> profit! >>> > >>> > Now sure about the profit bit at the end. >>> > >>> > (write crap, sell it, sell thousands of consulting hours to fix it = >>> profit >>> > :-) >>> > >>> >> >>> >> The runtime behaviors such as message passing, process linking, etc. >>> would >>> >> be the first level that would need specifications... so has this ever >>> been >>> >> tackled? >>> > >>> > Years ago - but not in phase with the distribution. >>> > >>> > There has always been a bit of gap between reality (C code, and what >>> > it does) and specifications (math, and what it should do). The two are >>> > not the same. >>> > >>> >> >>> >> 2. Regarding the mini-erlang, I'd really like to get involved in this >>> >> project. Perhaps I should be replying to a separate thread, but my >>> >> questions >>> >> are: >>> > >>> > There is no project - just a few people discussion what they would >>> like to >>> > see >>> > in a new system. >>> > >>> > >>> >> >>> >> * What are the goals of mini-erlang? >>> > >>> > My goal would be a tiny kernel - performance is >>> > of secondary importance. Tiny run-time footprint. >>> > >>> >> * How are these goals different from Erlang? >>> > >>> > The current Erlang is not designed for a small memory footprint. >>> > >>> >> * What needs to be removed/added from Erlang to make mini-erlang? >>> > >>> > Remove NIFS binarys ets tables (possibly) >>> > >>> >> * What is the current VM instruction set, and what would be the Mini >>> >> instruction set? >>> > >>> > Current is BEAM - Mini instruction set unknown. >>> > >>> >> (my reason for posing these questions in this thread) >>> >> *** Is there any interest in modeling/specifying the behaviors prior >>> to >>> >> implementing? >>> > >>> > I dont' think so - the main problem is figuring out how to make >>> something >>> > with tiny footprint - so for example GC strategy and memory layout is >>> > very important - I don't think modelling helps. Counting bytes on >>> squared >>> > paper >>> > seems the best method (really) >>> > >>> >> >>> >> Regarding implementation, I'm fairly agnostic. One thought is that a >>> >> prototype could be written in anything (python, bash, java, go, c, >>> etc.), >>> > >>> > I disagree - I'd like the final memory footprint to be small. >>> > c (and friends) are OK but I don't want to have to include an entire >>> JVM >>> > (or whatever). >>> > >>> > Could implement it in Forth though ... >>> > >>> >> and once the design has settled down, then anyone should be able to go >>> >> back >>> >> and re-write in their language of choice. Should be fairly simple >>> since >>> >> there would be design docs, formal specifications, test suites, etc. >>> Then >>> >> we >>> >> could have mini-erlang runtime competitions/shootouts ;-). >>> > >>> > That would be fun. >>> > >>> > Cheers >>> > >>> > /Joe >>> > >>> > >>> >> >>> >> [1] http://lamport.azurewebsites.net/tla/tla.html >>> >> _______________________________________________ >>> >> erlang-questions mailing list >>> >> erlang-questions@REDACTED >>> >> http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Sat Oct 7 03:50:32 2017 From: vans_163@REDACTED (Vans S) Date: Sat, 7 Oct 2017 01:50:32 +0000 (UTC) Subject: [erlang-questions] Why RPC in Erlang? [was: How RPC works in erlang?] In-Reply-To: References: <434c8093-d424-e4a3-cb98-d136ec916962@utl.in> <5078126.RgNHhUtOgi@changa> <6643279.YoUT2Rh97o@changa> Message-ID: <1197073383.2967763.1507341032720@mail.yahoo.com> > So when to use rpc:call and when to send a message by RemotePID ! {call, procedure}? Is there any general guidelines? No guidelines pretty much if you need code that requires responses from different endpoints, use RPC/custom_async_await, if you need to pass a message do that. {:ok, ICareReply} = rpc:call(bid_node, bidder, sendBidToPolonix, [155000, 100]) bid_entered_in_books DontCareReply = MarketScannerPid ! scan_market_price_and_enter_into_db On Friday, October 6, 2017 12:43 PM, Greg wrote: So when to use rpc:call and when to send a message by RemotePID ! {call, procedure}? Is there any general guidelines? There was these long standing debate over this?http://armstrongonsoftware.blogspot.in/2008/05/road-we-didnt-go-down.html; ? On Fri, Oct 6, 2017 at 8:17 PM, zxq9 wrote: On 2017?10?06? ??? 10:43:48 Roger Lipscombe wrote: > On 6 October 2017 at 03:37, zxq9 wrote: > > Thinking that the rpc module is specially useful for tests, CLI > > wrappers, or when you want to execute commands on a remote node (from > > a script or otherwise) is a bit of an odd conclusion to draw, given > > the way that the even-arity spawn_* functions work. > > On occasion, we've needed to (e.g.) tweak application settings in a > remote node, at runtime, during system tests, synchronously. Sure, I > could do that with spawn_*, but that's a lot of plumbing I'd need to > write. Hey, look: someone already did; it's in the rpc module. Sure. I'm not disagreeing with that. I'm disagreeing with the false notion that this is what "remote procedure call" actually means. -Craig ______________________________ _________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/ listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Sat Oct 7 04:39:49 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Sat, 07 Oct 2017 02:39:49 +0000 Subject: [erlang-questions] Erlang, mini-erlang, and TLA+ In-Reply-To: References: <27f5ddbe-f2e3-ccd5-4081-dd06bae13f2b@gmail.com> <842384618.10511969.1506450165630@mail.yahoo.com> Message-ID: Hi Jesper, Can you please enlighten me (Erlang newbie here) about this ?Monad? things. I?ve heard this word multiple times and still can?t get it. To me, It?s kind of magic word you say to impress people, and let them think you know functional programming very well ?? A simple, concrete example will be more than welcome. Thanks /Frank Oct 6. 2017 at 20:08, Jesper Louis Andersen wrote : > General rule when trying to build language cores formally: > > Every construction which can be written in terms of another construction > should be. > > * ETS tables are really just processes > * The Process dictionary is just a state monad in the language > * You need monads for message passing side effects anyway, so you can > probably reuse the work on top > * The whole exception system can be cast inside the monad as well > * Ports can be reimagined as special processes and you can use messaging > in order to handle them > > The reason is that any construction is a formal liability: it needs > special handling. It is often more beneficial to generalize the system a > little bit in order to vastly simplify the number of cases you have to > handle in proofs. > > > On Wed, Sep 27, 2017 at 7:20 AM Karlo Kuna wrote: > >> > No - write a new runtime but inspired by and stealing good bits from >> the old >> >> what would you consider good bits here? >> >> i also suspect that mini erlang would be better off without process >> dictionaries? >> >> >> >> On Tue, Sep 26, 2017 at 10:59 PM, Joe Armstrong wrote: >> >>> On Tue, Sep 26, 2017 at 8:22 PM, Brady Powers >>> wrote: >>> > If we're talking about a new instruction set, >>> >>> Yes >>> >>> > and removing pieces from the >>> > runtime, >>> >>> No - write a new runtime but inspired by and stealing good bits from the >>> old >>> >>> then are we still talking about a minimal Erlang, >>> or are we talking >>> >>> It would be a subset of Erlang - so strictly not Erlang >>> >>> > about something new? >>> >>> Yes >>> >>> >>> > Because I'm sure we could all think of thing's we'd change given the >>> > opportunity. Joe's already written Erl2. >>> >>> No - erl2 just compiles to erl >>> >>> /Joe >>> >>> > -Brady >>> > On Tuesday, September 26, 2017, 12:36:04 PM EDT, Joe Armstrong >>> > wrote: >>> > >>> > >>> > On Tue, Sep 26, 2017 at 5:47 PM, Todd Greenwood-Geer >>> > wrote: >>> >> Hi, >>> >> >>> >> 1. Are there existing TLA+ specifications [1] for Erlang? >>> > >>> > Not that I know of. >>> > >>> >> >>> >> Recent discussions regarding the behavior of messages for local and >>> remote >>> >> processes, the behavior of data in ETS, and the discussion about a new >>> >> mini-erlang all make me think there would be great benefit in formal >>> >> specifications of Erlang behaviors. >>> >> >>> >> For instance, I've been thinking about writing various widgets for >>> >> distributed applications, each with a well defined TLA+ >>> specification, and >>> >> then starting to compose larger and more interesting applications from >>> >> them. >>> >> >>> >> My workflow would look like this: >>> >> Write widget spec -> impl widget -> Write application spec (composed >>> of >>> >> previously spec'd widgets) -> impl application -> profit! >>> > >>> > Now sure about the profit bit at the end. >>> > >>> > (write crap, sell it, sell thousands of consulting hours to fix it = >>> profit >>> > :-) >>> > >>> >> >>> >> The runtime behaviors such as message passing, process linking, etc. >>> would >>> >> be the first level that would need specifications... so has this ever >>> been >>> >> tackled? >>> > >>> > Years ago - but not in phase with the distribution. >>> > >>> > There has always been a bit of gap between reality (C code, and what >>> > it does) and specifications (math, and what it should do). The two are >>> > not the same. >>> > >>> >> >>> >> 2. Regarding the mini-erlang, I'd really like to get involved in this >>> >> project. Perhaps I should be replying to a separate thread, but my >>> >> questions >>> >> are: >>> > >>> > There is no project - just a few people discussion what they would >>> like to >>> > see >>> > in a new system. >>> > >>> > >>> >> >>> >> * What are the goals of mini-erlang? >>> > >>> > My goal would be a tiny kernel - performance is >>> > of secondary importance. Tiny run-time footprint. >>> > >>> >> * How are these goals different from Erlang? >>> > >>> > The current Erlang is not designed for a small memory footprint. >>> > >>> >> * What needs to be removed/added from Erlang to make mini-erlang? >>> > >>> > Remove NIFS binarys ets tables (possibly) >>> > >>> >> * What is the current VM instruction set, and what would be the Mini >>> >> instruction set? >>> > >>> > Current is BEAM - Mini instruction set unknown. >>> > >>> >> (my reason for posing these questions in this thread) >>> >> *** Is there any interest in modeling/specifying the behaviors prior >>> to >>> >> implementing? >>> > >>> > I dont' think so - the main problem is figuring out how to make >>> something >>> > with tiny footprint - so for example GC strategy and memory layout is >>> > very important - I don't think modelling helps. Counting bytes on >>> squared >>> > paper >>> > seems the best method (really) >>> > >>> >> >>> >> Regarding implementation, I'm fairly agnostic. One thought is that a >>> >> prototype could be written in anything (python, bash, java, go, c, >>> etc.), >>> > >>> > I disagree - I'd like the final memory footprint to be small. >>> > c (and friends) are OK but I don't want to have to include an entire >>> JVM >>> > (or whatever). >>> > >>> > Could implement it in Forth though ... >>> > >>> >> and once the design has settled down, then anyone should be able to go >>> >> back >>> >> and re-write in their language of choice. Should be fairly simple >>> since >>> >> there would be design docs, formal specifications, test suites, etc. >>> Then >>> >> we >>> >> could have mini-erlang runtime competitions/shootouts ;-). >>> > >>> > That would be fun. >>> > >>> > Cheers >>> > >>> > /Joe >>> > >>> > >>> >> >>> >> [1] http://lamport.azurewebsites.net/tla/tla.html >>> >> _______________________________________________ >>> >> erlang-questions mailing list >>> >> erlang-questions@REDACTED >>> >> http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Sat Oct 7 09:02:10 2017 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Sat, 7 Oct 2017 03:02:10 -0400 Subject: [erlang-questions] Erlang web applications + security Message-ID: <9318B441-488C-45D9-9DE6-07FE8C00BC2A@writersglen.com> Hello, When I put an Erlang web application on line, what security issues do I need to address and what are recommended best practices to address them? Thanks, LRP Sent from my iPad From arunp@REDACTED Sat Oct 7 09:03:29 2017 From: arunp@REDACTED (Arun) Date: Sat, 7 Oct 2017 12:33:29 +0530 Subject: [erlang-questions] System getting hang after running normal recursion program Message-ID: <6c3e693c-b48c-cefd-8a6d-bb9684e25d4f@utl.in> Hi all, I tried to run normal recursion program from erlang shell, the program runs infinitely and it is doing addition of numbers. But its been observed that soon after starting the program entire PC is getting hang. My doubt is that, why my PC is getting hang. ? , if something is going wrong, the erlang vm should be able to handle it, like in C if we run some recursion program infinity, it will crash by throwing segmentation fault error. What will be happening in the VM if I run a normal recursion program in erlang infinitely ? What will be the behavior of erlang scheduler in this scenario ? Can somebody please assist me on this issue. Thanks in advance Arun P. From ameretat.reith@REDACTED Sat Oct 7 13:34:50 2017 From: ameretat.reith@REDACTED (Ameretat Reith) Date: Sat, 7 Oct 2017 15:04:50 +0330 Subject: [erlang-questions] System getting hang after running normal recursion program In-Reply-To: <6c3e693c-b48c-cefd-8a6d-bb9684e25d4f@utl.in> References: <6c3e693c-b48c-cefd-8a6d-bb9684e25d4f@utl.in> Message-ID: In C your infinite recursion will cause infinite increase in stack size (infinite number of return addresses and local variables that should be remembered). There are some languages coming with optimizations making you able to call functions infinitely as long as the last line of code is same function call [1] or another function call [2]. Erlang is of latter type so if last code of your recursion function is another function call, there is no reason to terminate it. Stack won't grow infinitely and Is perfectly fine to keep running. 1: https://en.wikipedia.org/wiki/Tail_call 2: http://erlang.org/pipermail/erlang-questions/2016-October/090663.html From arunp@REDACTED Sat Oct 7 14:14:00 2017 From: arunp@REDACTED (Arun) Date: Sat, 7 Oct 2017 17:44:00 +0530 Subject: [erlang-questions] System getting hang after running normal recursion program In-Reply-To: References: <6c3e693c-b48c-cefd-8a6d-bb9684e25d4f@utl.in> Message-ID: Hi Leandro, Am aware about tail recursion in erlang, but here I am talking about recursion which is not tail recursive. Here is the code snippet; start(N) -> N*start(N-1). Here the function is not tail recursive and it is in an infinite loop. If run this function my system is getting hang. That why i got the doubt why it is making my system to hang and how the erlang scheduler will behave in this scenario.? Regards, Arun P. On Saturday 07 October 2017 04:07 PM, Leandro Ostera wrote: > Hej Arun, hi. > > Depending on your recursion, you may or may not be building up a stack > that will at some point crash. I am unsure about how C compilers > typically deal with tail-recursion, but tail call optimization is > automatically performed in the Erlang VM for you. > > From the Learn You Some Erlang book chapter on Recursion > (http://learnyousomeerlang.com/recursion): > > *Note:* tail recursion as seen here is not making the memory grow > because when the virtual machine sees a function calling itself in a > tail position (the last expression to be evaluated in a function), it > eliminates the current stack frame. This is called tail-call > optimisation (TCO) and it is a special case of a more general > optimisation named /Last Call Optimisation/ (LCO). > > LCO is done whenever the last expression to be evaluated in a function > body is another function call. When that happens, as with TCO, the > Erlang VM avoids storing the stack frame. As such tail recursion is > also possible between multiple functions. As an example, the chain of > functions |a() -> b(). b() -> c(). c() -> a().| will effectively > create an infinite loop that won't go out of memory as LCO avoids > overflowing the stack. This principle, combined with our use of > accumulators is what makes tail recursion useful. > > > Hope this helps! > > On Sat, Oct 7, 2017 at 9:03 AM Arun > wrote: > > Hi all, > > I tried to run normal recursion program from erlang shell, the program > runs infinitely and it is doing addition of numbers. But its been > observed that soon after starting the program entire PC is getting > hang. > > My doubt is that, why my PC is getting hang. ? , if something is going > wrong, the erlang vm should be able to handle it, like in C if we run > some recursion program infinity, it will crash by throwing > segmentation > fault error. > > What will be happening in the VM if I run a normal recursion > program in > erlang infinitely ? > > What will be the behavior of erlang scheduler in this scenario ? > > Can somebody please assist me on this issue. > > Thanks in advance > Arun P. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From olopierpa@REDACTED Sat Oct 7 14:25:15 2017 From: olopierpa@REDACTED (Pierpaolo Bernardi) Date: Sat, 07 Oct 2017 14:25:15 +0200 Subject: [erlang-questions] System getting hang after running normal recursion program Message-ID: Il giorno 7 ottobre 2017, alle ore 14:14, Arun ha scritto: > > >Hi Leandro, > >Am aware about tail recursion in erlang, but here I am talking about recursion which is not tail recursive. Here is the code snippet; > >start(N) -> >??? N*start(N-1). > >Here the function is not tail recursive and it is in an infinite loop. If run this function my system is getting hang. That why i got the doubt why it is making my system to hang and how the erlang scheduler will behave in this scenario.? The numbers involved here grow very quickly. Your system is boggled down by the integer multiplications after a few hundred functions calls, much before the depth of recursion is a problem at all. If you want to stress recursion depth, try 1+start(N-1), for example. > >Regards, >Arun P. > >On Saturday 07 October 2017 04:07 PM, Leandro Ostera wrote: > >Hej Arun, hi. > >Depending on your recursion, you may or may not be building up a stack that will at some point crash. I am unsure about how C compilers typically deal with tail-recursion, but tail call optimization is automatically performed in the Erlang VM for you. > >From the Learn You Some Erlang book chapter on Recursion (http://learnyousomeerlang.com/recursion): > >Note:?tail recursion as seen here is not making the memory grow because when the virtual machine sees a function calling itself in a tail position (the last expression to be evaluated in a function), it eliminates the current stack frame. This is called tail-call optimisation (TCO) and it is a special case of a more general optimisation named?Last Call Optimisation?(LCO). > >LCO is done whenever the last expression to be evaluated in a function body is another function call. When that happens, as with TCO, the Erlang VM avoids storing the stack frame. As such tail recursion is also possible between multiple functions. As an example, the chain of functions?a() -> b(). b() -> c(). c() -> a().?will effectively create an infinite loop that won't go out of memory as LCO avoids overflowing the stack. This principle, combined with our use of accumulators is what makes tail recursion useful. > >Hope this helps! > >On Sat, Oct 7, 2017 at 9:03 AM Arun wrote: > >Hi all, >I tried to run normal recursion program from erlang shell, the program >runs infinitely and it is doing addition of numbers. But its been >observed that soon after starting the program entire PC is getting hang. >My doubt is that, why my PC is getting hang. ? , if something is going >wrong, the erlang vm should be able to handle it, like in C if we run >some recursion program infinity, it will crash by throwing segmentation >fault error. >What will be happening in the VM if I run a normal recursion program in >erlang infinitely ? >What will be the behavior of erlang scheduler in this scenario ? >Can somebody please assist me on this issue. >Thanks in advance >Arun P. >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Sat Oct 7 15:04:36 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Sat, 7 Oct 2017 15:04:36 +0200 Subject: [erlang-questions] System getting hang after running normal recursion program In-Reply-To: References: <6c3e693c-b48c-cefd-8a6d-bb9684e25d4f@utl.in> Message-ID: Your small function has no way to terminate, and very quickly eats up your system RAM with its ever growing stack. The CPU load is not the problem. /Richard 2017-10-07 14:14 GMT+02:00 Arun : > Hi Leandro, > > Am aware about tail recursion in erlang, but here I am talking about > recursion which is not tail recursive. Here is the code snippet; > > start(N) -> > N*start(N-1). > > Here the function is not tail recursive and it is in an infinite loop. If > run this function my system is getting hang. That why i got the doubt why > it is making my system to hang and how the erlang scheduler will behave in > this scenario.? > Regards, > Arun P. > > > On Saturday 07 October 2017 04:07 PM, Leandro Ostera wrote: > > Hej Arun, hi. > > Depending on your recursion, you may or may not be building up a stack > that will at some point crash. I am unsure about how C compilers typically > deal with tail-recursion, but tail call optimization is automatically > performed in the Erlang VM for you. > > From the Learn You Some Erlang book chapter on Recursion ( > http://learnyousomeerlang.com/recursion): > > *Note:* tail recursion as seen here is not making the memory grow because > when the virtual machine sees a function calling itself in a tail position > (the last expression to be evaluated in a function), it eliminates the > current stack frame. This is called tail-call optimisation (TCO) and it is > a special case of a more general optimisation named *Last Call > Optimisation* (LCO). > > LCO is done whenever the last expression to be evaluated in a function > body is another function call. When that happens, as with TCO, the Erlang > VM avoids storing the stack frame. As such tail recursion is also possible > between multiple functions. As an example, the chain of functions a() -> > b(). b() -> c(). c() -> a(). will effectively create an infinite loop > that won't go out of memory as LCO avoids overflowing the stack. This > principle, combined with our use of accumulators is what makes tail > recursion useful. > > Hope this helps! > > On Sat, Oct 7, 2017 at 9:03 AM Arun wrote: > >> Hi all, >> >> I tried to run normal recursion program from erlang shell, the program >> runs infinitely and it is doing addition of numbers. But its been >> observed that soon after starting the program entire PC is getting hang. >> >> My doubt is that, why my PC is getting hang. ? , if something is going >> wrong, the erlang vm should be able to handle it, like in C if we run >> some recursion program infinity, it will crash by throwing segmentation >> fault error. >> >> What will be happening in the VM if I run a normal recursion program in >> erlang infinitely ? >> >> What will be the behavior of erlang scheduler in this scenario ? >> >> Can somebody please assist me on this issue. >> >> Thanks in advance >> Arun P. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sat Oct 7 16:54:15 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 07 Oct 2017 14:54:15 +0000 Subject: [erlang-questions] Erlang, mini-erlang, and TLA+ In-Reply-To: References: <27f5ddbe-f2e3-ccd5-4081-dd06bae13f2b@gmail.com> <842384618.10511969.1506450165630@mail.yahoo.com> Message-ID: In this setting, Monads are used subtly different from the more "normal" setting, but I'll try to draw the connection anyway. When you are trying to formalize a programming language, there is a list of effectful constructions we may want to add to the language: * Exceptions or Process Crashes * State or the Process Dictionary * Communication patterns * Hibernation (or more generally: continuation constructs) * Non-deterministic computation * ... The list goes on and also includes more "esoteric" constructions such as variables which has a certain value with a probability, quantum computations, computations where state flows in the reverse order of the program and so on. If we add these constructions to a programming language and wish to formalize the language so we can prove theorems about it, then the language can easily become quite large very quickly. This hampers our proof-effort since we need to address the added complexity in all proofs we are doing. Exceptions will mean we have to address the concepts of "throw" and "handle". State will mean we have to address what "put" and "get" does and so on. More importantly, we will have to alter the whole semantics since now there are state and added control flow to keep track of! But there is hope. A non-trivial insight is that all the above constructions have something in common. We can, so to speak, extract a more general notion of the above. This is good, because if we can prove something in generality, we can "plug in" the specifics--- exceptions, the state and so on---into our proof. We don't have to handle everything up front if this works out. The price is that the general notion might be harder to work with in the first place. You can then ask: what should the rules of such a general notion be? It turns out mathematicians working in the area of Category Theory already had a definition which fit: the Monad. Remarkably, this creates a link, so to speak, between mathematics and programming languages. We can capture the above needed generality by observing that what they have in common are the "Monad Laws". Hence our task, as formal semanticists, can now be changed and we can attack the problem in another way: 1. Show that we can add an abstract Monad to our language. Since the abstraction must hold for every possible monad, we are thus only allowed the use of the Monad Laws when dealing with proofs about the language. 2. Show that Exceptions, State, ... obeys the Monad laws. 3. Plug the things together to obtain a language which is extended with the effects of our desire. The hope is that dealing with 1 and 2 separately is easier than dealing with both at the same time. ------------------------- The other common use of a "Monad" is to embed effects into a pure language. This idea goes back to Moggi (I believe) and was popularized by Phil Wadler for Haskell[0]. Here the Monad acts like a stratification: computations are put into classes depending on what effects they have. E.g., "exceptions goes here", "io computation goes there" and so on. The key insight is that we can thus keep computations with effects apart from (pure) computations without. Even more interesting, because Haskell is a statically typed language, we can enforce that computations with different effects cannot accidentally be mixed in wrong ways. This helps with correctness. What makes the monad construction hard to grasp initially is that it is an abstract notion in line with recursion. One has to work with it for a while to build up a certain amount of familiarity with the concept. This is also why it is so hard to explain: most tutorials are giving you a glimpse of the real deal through metaphor and analogy in the hope that it can make your mind think in the right way. But in reality it is just a definition of a mathematical object with certain laws. Why exactly *those* laws and not something else? Well it turned out that these laws were the useful ones[1]. Why the link to computer science? Because the universe is a remarkable place of systems. [0] Phil's original paper is one of the best expositions of the idea and still to this day is one of the most readable. [1] Mathematicians some times call such laws the "natural laws" in the sense that they afford you the most power from a perspective of using the definition as a building block for further work. On Sat, Oct 7, 2017 at 4:40 AM Frank Muller wrote: > Hi Jesper, > > Can you please enlighten me (Erlang newbie here) about this ?Monad? > things. I?ve heard this word multiple times and still can?t get it. > > To me, It?s kind of magic word you say to impress people, and let them > think you know functional programming very well ?? > > A simple, concrete example will be more than welcome. > > Thanks > /Frank > > Oct 6. 2017 at 20:08, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote : > >> General rule when trying to build language cores formally: >> >> Every construction which can be written in terms of another construction >> should be. >> >> * ETS tables are really just processes >> * The Process dictionary is just a state monad in the language >> * You need monads for message passing side effects anyway, so you can >> probably reuse the work on top >> * The whole exception system can be cast inside the monad as well >> * Ports can be reimagined as special processes and you can use messaging >> in order to handle them >> >> The reason is that any construction is a formal liability: it needs >> special handling. It is often more beneficial to generalize the system a >> little bit in order to vastly simplify the number of cases you have to >> handle in proofs. >> >> >> On Wed, Sep 27, 2017 at 7:20 AM Karlo Kuna wrote: >> >>> > No - write a new runtime but inspired by and stealing good bits from >>> the old >>> >>> what would you consider good bits here? >>> >>> i also suspect that mini erlang would be better off without process >>> dictionaries? >>> >>> >>> >>> On Tue, Sep 26, 2017 at 10:59 PM, Joe Armstrong >>> wrote: >>> >>>> On Tue, Sep 26, 2017 at 8:22 PM, Brady Powers >>>> wrote: >>>> > If we're talking about a new instruction set, >>>> >>>> Yes >>>> >>>> > and removing pieces from the >>>> > runtime, >>>> >>>> No - write a new runtime but inspired by and stealing good bits from >>>> the old >>>> >>>> then are we still talking about a minimal Erlang, >>>> or are we talking >>>> >>>> It would be a subset of Erlang - so strictly not Erlang >>>> >>>> > about something new? >>>> >>>> Yes >>>> >>>> >>>> > Because I'm sure we could all think of thing's we'd change given the >>>> > opportunity. Joe's already written Erl2. >>>> >>>> No - erl2 just compiles to erl >>>> >>>> /Joe >>>> >>>> > -Brady >>>> > On Tuesday, September 26, 2017, 12:36:04 PM EDT, Joe Armstrong >>>> > wrote: >>>> > >>>> > >>>> > On Tue, Sep 26, 2017 at 5:47 PM, Todd Greenwood-Geer >>>> > wrote: >>>> >> Hi, >>>> >> >>>> >> 1. Are there existing TLA+ specifications [1] for Erlang? >>>> > >>>> > Not that I know of. >>>> > >>>> >> >>>> >> Recent discussions regarding the behavior of messages for local and >>>> remote >>>> >> processes, the behavior of data in ETS, and the discussion about a >>>> new >>>> >> mini-erlang all make me think there would be great benefit in formal >>>> >> specifications of Erlang behaviors. >>>> >> >>>> >> For instance, I've been thinking about writing various widgets for >>>> >> distributed applications, each with a well defined TLA+ >>>> specification, and >>>> >> then starting to compose larger and more interesting applications >>>> from >>>> >> them. >>>> >> >>>> >> My workflow would look like this: >>>> >> Write widget spec -> impl widget -> Write application spec (composed >>>> of >>>> >> previously spec'd widgets) -> impl application -> profit! >>>> > >>>> > Now sure about the profit bit at the end. >>>> > >>>> > (write crap, sell it, sell thousands of consulting hours to fix it = >>>> profit >>>> > :-) >>>> > >>>> >> >>>> >> The runtime behaviors such as message passing, process linking, etc. >>>> would >>>> >> be the first level that would need specifications... so has this >>>> ever been >>>> >> tackled? >>>> > >>>> > Years ago - but not in phase with the distribution. >>>> > >>>> > There has always been a bit of gap between reality (C code, and what >>>> > it does) and specifications (math, and what it should do). The two are >>>> > not the same. >>>> > >>>> >> >>>> >> 2. Regarding the mini-erlang, I'd really like to get involved in this >>>> >> project. Perhaps I should be replying to a separate thread, but my >>>> >> questions >>>> >> are: >>>> > >>>> > There is no project - just a few people discussion what they would >>>> like to >>>> > see >>>> > in a new system. >>>> > >>>> > >>>> >> >>>> >> * What are the goals of mini-erlang? >>>> > >>>> > My goal would be a tiny kernel - performance is >>>> > of secondary importance. Tiny run-time footprint. >>>> > >>>> >> * How are these goals different from Erlang? >>>> > >>>> > The current Erlang is not designed for a small memory footprint. >>>> > >>>> >> * What needs to be removed/added from Erlang to make mini-erlang? >>>> > >>>> > Remove NIFS binarys ets tables (possibly) >>>> > >>>> >> * What is the current VM instruction set, and what would be the Mini >>>> >> instruction set? >>>> > >>>> > Current is BEAM - Mini instruction set unknown. >>>> > >>>> >> (my reason for posing these questions in this thread) >>>> >> *** Is there any interest in modeling/specifying the behaviors prior >>>> to >>>> >> implementing? >>>> > >>>> > I dont' think so - the main problem is figuring out how to make >>>> something >>>> > with tiny footprint - so for example GC strategy and memory layout is >>>> > very important - I don't think modelling helps. Counting bytes on >>>> squared >>>> > paper >>>> > seems the best method (really) >>>> > >>>> >> >>>> >> Regarding implementation, I'm fairly agnostic. One thought is that a >>>> >> prototype could be written in anything (python, bash, java, go, c, >>>> etc.), >>>> > >>>> > I disagree - I'd like the final memory footprint to be small. >>>> > c (and friends) are OK but I don't want to have to include an entire >>>> JVM >>>> > (or whatever). >>>> > >>>> > Could implement it in Forth though ... >>>> > >>>> >> and once the design has settled down, then anyone should be able to >>>> go >>>> >> back >>>> >> and re-write in their language of choice. Should be fairly simple >>>> since >>>> >> there would be design docs, formal specifications, test suites, etc. >>>> Then >>>> >> we >>>> >> could have mini-erlang runtime competitions/shootouts ;-). >>>> > >>>> > That would be fun. >>>> > >>>> > Cheers >>>> > >>>> > /Joe >>>> > >>>> > >>>> >> >>>> >> [1] http://lamport.azurewebsites.net/tla/tla.html >>>> >> _______________________________________________ >>>> >> erlang-questions mailing list >>>> >> erlang-questions@REDACTED >>>> >> http://erlang.org/mailman/listinfo/erlang-questions >>>> > >>>> > _______________________________________________ >>>> > erlang-questions mailing list >>>> > erlang-questions@REDACTED >>>> > http://erlang.org/mailman/listinfo/erlang-questions >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Sat Oct 7 17:16:34 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Sat, 07 Oct 2017 15:16:34 +0000 Subject: [erlang-questions] Erlang, mini-erlang, and TLA+ In-Reply-To: References: <27f5ddbe-f2e3-ccd5-4081-dd06bae13f2b@gmail.com> <842384618.10511969.1506450165630@mail.yahoo.com> Message-ID: Jesper: many thanks again for taking time explaining the Monad concept in this specific case and in general. I found this module and I?ll try to play with it: https://github.com/rabbitmq/erlando /Frank In this setting, Monads are used subtly different from the more "normal" > setting, but I'll try to draw the connection anyway. > > When you are trying to formalize a programming language, there is a list > of effectful constructions we may want to add to the language: > > * Exceptions or Process Crashes > * State or the Process Dictionary > * Communication patterns > * Hibernation (or more generally: continuation constructs) > * Non-deterministic computation > * ... > > The list goes on and also includes more "esoteric" constructions such as > variables which has a certain value with a probability, quantum > computations, computations where state flows in the reverse order of the > program and so on. > > If we add these constructions to a programming language and wish to > formalize the language so we can prove theorems about it, then the language > can easily become quite large very quickly. This hampers our proof-effort > since we need to address the added complexity in all proofs we are doing. > Exceptions will mean we have to address the concepts of "throw" and > "handle". State will mean we have to address what "put" and "get" does and > so on. More importantly, we will have to alter the whole semantics since > now there are state and added control flow to keep track of! > > But there is hope. A non-trivial insight is that all the above > constructions have something in common. We can, so to speak, extract a more > general notion of the above. This is good, because if we can prove > something in generality, we can "plug in" the specifics--- exceptions, the > state and so on---into our proof. We don't have to handle everything up > front if this works out. The price is that the general notion might be > harder to work with in the first place. > > You can then ask: what should the rules of such a general notion be? It > turns out mathematicians working in the area of Category Theory already had > a definition which fit: the Monad. Remarkably, this creates a link, so to > speak, between mathematics and programming languages. We can capture the > above needed generality by observing that what they have in common are the > "Monad Laws". > > Hence our task, as formal semanticists, can now be changed and we can > attack the problem in another way: > > 1. Show that we can add an abstract Monad to our language. Since the > abstraction must hold for every possible monad, we are thus only allowed > the use of the Monad Laws when dealing with proofs about the language. > > 2. Show that Exceptions, State, ... obeys the Monad laws. > > 3. Plug the things together to obtain a language which is extended with > the effects of our desire. > > The hope is that dealing with 1 and 2 separately is easier than dealing > with both at the same time. > > ------------------------- > > The other common use of a "Monad" is to embed effects into a pure > language. This idea goes back to Moggi (I believe) and was popularized by > Phil Wadler for Haskell[0]. Here the Monad acts like a stratification: > computations are put into classes depending on what effects they have. > E.g., "exceptions goes here", "io computation goes there" and so on. The > key insight is that we can thus keep computations with effects apart from > (pure) computations without. Even more interesting, because Haskell is a > statically typed language, we can enforce that computations with different > effects cannot accidentally be mixed in wrong ways. This helps with > correctness. > > What makes the monad construction hard to grasp initially is that it is an > abstract notion in line with recursion. One has to work with it for a while > to build up a certain amount of familiarity with the concept. This is also > why it is so hard to explain: most tutorials are giving you a glimpse of > the real deal through metaphor and analogy in the hope that it can make > your mind think in the right way. But in reality it is just a definition of > a mathematical object with certain laws. Why exactly *those* laws and not > something else? Well it turned out that these laws were the useful ones[1]. > Why the link to computer science? Because the universe is a remarkable > place of systems. > > [0] Phil's original paper is one of the best expositions of the idea and > still to this day is one of the most readable. > > [1] Mathematicians some times call such laws the "natural laws" in the > sense that they afford you the most power from a perspective of using the > definition as a building block for further work. > > > On Sat, Oct 7, 2017 at 4:40 AM Frank Muller > wrote: > >> Hi Jesper, >> >> Can you please enlighten me (Erlang newbie here) about this ?Monad? >> things. I?ve heard this word multiple times and still can?t get it. >> >> To me, It?s kind of magic word you say to impress people, and let them >> think you know functional programming very well ?? >> >> A simple, concrete example will be more than welcome. >> >> Thanks >> /Frank >> >> Oct 6. 2017 at 20:08, Jesper Louis Andersen < >> jesper.louis.andersen@REDACTED> wrote : >> >>> General rule when trying to build language cores formally: >>> >>> Every construction which can be written in terms of another construction >>> should be. >>> >>> * ETS tables are really just processes >>> * The Process dictionary is just a state monad in the language >>> * You need monads for message passing side effects anyway, so you can >>> probably reuse the work on top >>> * The whole exception system can be cast inside the monad as well >>> * Ports can be reimagined as special processes and you can use messaging >>> in order to handle them >>> >>> The reason is that any construction is a formal liability: it needs >>> special handling. It is often more beneficial to generalize the system a >>> little bit in order to vastly simplify the number of cases you have to >>> handle in proofs. >>> >>> >>> On Wed, Sep 27, 2017 at 7:20 AM Karlo Kuna wrote: >>> >>>> > No - write a new runtime but inspired by and stealing good bits from >>>> the old >>>> >>>> what would you consider good bits here? >>>> >>>> i also suspect that mini erlang would be better off without process >>>> dictionaries? >>>> >>>> >>>> >>>> On Tue, Sep 26, 2017 at 10:59 PM, Joe Armstrong >>>> wrote: >>>> >>>>> On Tue, Sep 26, 2017 at 8:22 PM, Brady Powers >>>>> wrote: >>>>> > If we're talking about a new instruction set, >>>>> >>>>> Yes >>>>> >>>>> > and removing pieces from the >>>>> > runtime, >>>>> >>>>> No - write a new runtime but inspired by and stealing good bits from >>>>> the old >>>>> >>>>> then are we still talking about a minimal Erlang, >>>>> or are we talking >>>>> >>>>> It would be a subset of Erlang - so strictly not Erlang >>>>> >>>>> > about something new? >>>>> >>>>> Yes >>>>> >>>>> >>>>> > Because I'm sure we could all think of thing's we'd change given the >>>>> > opportunity. Joe's already written Erl2. >>>>> >>>>> No - erl2 just compiles to erl >>>>> >>>>> /Joe >>>>> >>>>> > -Brady >>>>> > On Tuesday, September 26, 2017, 12:36:04 PM EDT, Joe Armstrong >>>>> > wrote: >>>>> > >>>>> > >>>>> > On Tue, Sep 26, 2017 at 5:47 PM, Todd Greenwood-Geer >>>>> > wrote: >>>>> >> Hi, >>>>> >> >>>>> >> 1. Are there existing TLA+ specifications [1] for Erlang? >>>>> > >>>>> > Not that I know of. >>>>> > >>>>> >> >>>>> >> Recent discussions regarding the behavior of messages for local and >>>>> remote >>>>> >> processes, the behavior of data in ETS, and the discussion about a >>>>> new >>>>> >> mini-erlang all make me think there would be great benefit in formal >>>>> >> specifications of Erlang behaviors. >>>>> >> >>>>> >> For instance, I've been thinking about writing various widgets for >>>>> >> distributed applications, each with a well defined TLA+ >>>>> specification, and >>>>> >> then starting to compose larger and more interesting applications >>>>> from >>>>> >> them. >>>>> >> >>>>> >> My workflow would look like this: >>>>> >> Write widget spec -> impl widget -> Write application spec >>>>> (composed of >>>>> >> previously spec'd widgets) -> impl application -> profit! >>>>> > >>>>> > Now sure about the profit bit at the end. >>>>> > >>>>> > (write crap, sell it, sell thousands of consulting hours to fix it = >>>>> profit >>>>> > :-) >>>>> > >>>>> >> >>>>> >> The runtime behaviors such as message passing, process linking, >>>>> etc. would >>>>> >> be the first level that would need specifications... so has this >>>>> ever been >>>>> >> tackled? >>>>> > >>>>> > Years ago - but not in phase with the distribution. >>>>> > >>>>> > There has always been a bit of gap between reality (C code, and what >>>>> > it does) and specifications (math, and what it should do). The two >>>>> are >>>>> > not the same. >>>>> > >>>>> >> >>>>> >> 2. Regarding the mini-erlang, I'd really like to get involved in >>>>> this >>>>> >> project. Perhaps I should be replying to a separate thread, but my >>>>> >> questions >>>>> >> are: >>>>> > >>>>> > There is no project - just a few people discussion what they would >>>>> like to >>>>> > see >>>>> > in a new system. >>>>> > >>>>> > >>>>> >> >>>>> >> * What are the goals of mini-erlang? >>>>> > >>>>> > My goal would be a tiny kernel - performance is >>>>> > of secondary importance. Tiny run-time footprint. >>>>> > >>>>> >> * How are these goals different from Erlang? >>>>> > >>>>> > The current Erlang is not designed for a small memory footprint. >>>>> > >>>>> >> * What needs to be removed/added from Erlang to make mini-erlang? >>>>> > >>>>> > Remove NIFS binarys ets tables (possibly) >>>>> > >>>>> >> * What is the current VM instruction set, and what would be the Mini >>>>> >> instruction set? >>>>> > >>>>> > Current is BEAM - Mini instruction set unknown. >>>>> > >>>>> >> (my reason for posing these questions in this thread) >>>>> >> *** Is there any interest in modeling/specifying the behaviors >>>>> prior to >>>>> >> implementing? >>>>> > >>>>> > I dont' think so - the main problem is figuring out how to make >>>>> something >>>>> > with tiny footprint - so for example GC strategy and memory layout is >>>>> > very important - I don't think modelling helps. Counting bytes on >>>>> squared >>>>> > paper >>>>> > seems the best method (really) >>>>> > >>>>> >> >>>>> >> Regarding implementation, I'm fairly agnostic. One thought is that a >>>>> >> prototype could be written in anything (python, bash, java, go, c, >>>>> etc.), >>>>> > >>>>> > I disagree - I'd like the final memory footprint to be small. >>>>> > c (and friends) are OK but I don't want to have to include an entire >>>>> JVM >>>>> > (or whatever). >>>>> > >>>>> > Could implement it in Forth though ... >>>>> > >>>>> >> and once the design has settled down, then anyone should be able to >>>>> go >>>>> >> back >>>>> >> and re-write in their language of choice. Should be fairly simple >>>>> since >>>>> >> there would be design docs, formal specifications, test suites, >>>>> etc. Then >>>>> >> we >>>>> >> could have mini-erlang runtime competitions/shootouts ;-). >>>>> > >>>>> > That would be fun. >>>>> > >>>>> > Cheers >>>>> > >>>>> > /Joe >>>>> > >>>>> > >>>>> >> >>>>> >> [1] http://lamport.azurewebsites.net/tla/tla.html >>>>> >> _______________________________________________ >>>>> >> erlang-questions mailing list >>>>> >> erlang-questions@REDACTED >>>>> >> http://erlang.org/mailman/listinfo/erlang-questions >>>>> > >>>>> > _______________________________________________ >>>>> > erlang-questions mailing list >>>>> > erlang-questions@REDACTED >>>>> > http://erlang.org/mailman/listinfo/erlang-questions >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Sat Oct 7 17:44:29 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Sat, 7 Oct 2017 17:44:29 +0200 Subject: [erlang-questions] System getting hang after running normal recursion program In-Reply-To: References: <6c3e693c-b48c-cefd-8a6d-bb9684e25d4f@utl.in> Message-ID: Exactly. Erlang processes manage their own stacks and reallocate and grow them as needed. This means that stacks can start out very small (like 2-4 k) and you can have millions of processes without running out of address space for stacks. /Richard 2017-10-07 15:23 GMT+02:00 Leandro Ostera : > @Richard, so there's no static stack limit, but rather the stack grows > dynamically and the evaluation will be RAM-bound? > > On Sat, Oct 7, 2017 at 3:04 PM Richard Carlsson < > carlsson.richard@REDACTED> wrote: > >> Your small function has no way to terminate, and very quickly eats up >> your system RAM with its ever growing stack. The CPU load is not the >> problem. >> >> >> /Richard >> >> 2017-10-07 14:14 GMT+02:00 Arun : >> >>> Hi Leandro, >>> >>> Am aware about tail recursion in erlang, but here I am talking about >>> recursion which is not tail recursive. Here is the code snippet; >>> >>> start(N) -> >>> N*start(N-1). >>> >>> Here the function is not tail recursive and it is in an infinite loop. >>> If run this function my system is getting hang. That why i got the doubt >>> why it is making my system to hang and how the erlang scheduler will behave >>> in this scenario.? >>> Regards, >>> Arun P. >>> >>> >>> On Saturday 07 October 2017 04:07 PM, Leandro Ostera wrote: >>> >>> Hej Arun, hi. >>> >>> Depending on your recursion, you may or may not be building up a stack >>> that will at some point crash. I am unsure about how C compilers typically >>> deal with tail-recursion, but tail call optimization is automatically >>> performed in the Erlang VM for you. >>> >>> From the Learn You Some Erlang book chapter on Recursion ( >>> http://learnyousomeerlang.com/recursion): >>> >>> *Note:* tail recursion as seen here is not making the memory grow >>> because when the virtual machine sees a function calling itself in a tail >>> position (the last expression to be evaluated in a function), it eliminates >>> the current stack frame. This is called tail-call optimisation (TCO) and it >>> is a special case of a more general optimisation named *Last Call >>> Optimisation* (LCO). >>> >>> LCO is done whenever the last expression to be evaluated in a function >>> body is another function call. When that happens, as with TCO, the Erlang >>> VM avoids storing the stack frame. As such tail recursion is also possible >>> between multiple functions. As an example, the chain of functions a() >>> -> b(). b() -> c(). c() -> a(). will effectively create an infinite >>> loop that won't go out of memory as LCO avoids overflowing the stack. This >>> principle, combined with our use of accumulators is what makes tail >>> recursion useful. >>> >>> Hope this helps! >>> >>> On Sat, Oct 7, 2017 at 9:03 AM Arun wrote: >>> >>>> Hi all, >>>> >>>> I tried to run normal recursion program from erlang shell, the program >>>> runs infinitely and it is doing addition of numbers. But its been >>>> observed that soon after starting the program entire PC is getting hang. >>>> >>>> My doubt is that, why my PC is getting hang. ? , if something is going >>>> wrong, the erlang vm should be able to handle it, like in C if we run >>>> some recursion program infinity, it will crash by throwing segmentation >>>> fault error. >>>> >>>> What will be happening in the VM if I run a normal recursion program in >>>> erlang infinitely ? >>>> >>>> What will be the behavior of erlang scheduler in this scenario ? >>>> >>>> Can somebody please assist me on this issue. >>>> >>>> Thanks in advance >>>> Arun P. >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From leandro.21.2008@REDACTED Sat Oct 7 19:27:56 2017 From: leandro.21.2008@REDACTED (Leandro David Cacciagioni) Date: Sat, 7 Oct 2017 19:27:56 +0200 Subject: [erlang-questions] Erlang web applications + security In-Reply-To: <9318B441-488C-45D9-9DE6-07FE8C00BC2A@writersglen.com> References: <9318B441-488C-45D9-9DE6-07FE8C00BC2A@writersglen.com> Message-ID: Hi Lloyd, I would say that for any web app (Not only in erlang) you must start at least for securing the issues named in the OWASP 10 ( https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project) that will gonna give you a good coverage for the most basic and common sec issues in the world wide web of today. Thanks, Leandro.- On Oct 7, 2017 09:02, "Lloyd R. Prentice" wrote: > Hello, > > When I put an Erlang web application on line, what security issues do I > need to address and what are recommended best practices to address them? > > Thanks, > > LRP > > Sent from my iPad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Sun Oct 8 03:49:28 2017 From: zxq9@REDACTED (zxq9) Date: Sun, 08 Oct 2017 10:49:28 +0900 Subject: [erlang-questions] System getting hang after running normal recursion program In-Reply-To: References: <6c3e693c-b48c-cefd-8a6d-bb9684e25d4f@utl.in> Message-ID: <9855652.BeiGhhoJ8l@changa> On 2017?10?07? ??? 17:44:00 Arun wrote: > Hi Leandro, > > Am aware about tail recursion in erlang, but here I am talking about > recursion which is not tail recursive. Here is the code snippet; > > start(N) -> > N*start(N-1). > > Here the function is not tail recursive and it is in an infinite loop. > If run this function my system is getting hang. That why i got the doubt > why it is making my system to hang and how the erlang scheduler will > behave in this scenario.? Infinite depth recursion will make your system run out of memory, not CPU. The call stack is just growing and growing and growing... This has nothing to do with the schedulers at all -- they will behave the way they always would, but will be starved for things to do because your system will the thrashing (swapping in and out of virtual memory on disk in the common case). The reason this slows the entire system down is because the WHOLE SYSTEM is now starved for memory and is thrashing, including your OS and anything else running. Eventually the size of the Erlang runtime will grow large enough that the system's OOM killer will terminate it and things will return to normal. If you want to see this effect firsthand but much more quickly, turn off swap. That said, deep recursion is occasionally the correct approach to take, particularly in problems where the natural solution is iterative and deep, and the depth is known to be finite (the shape of some trees, like filesystems, and certain cases of document or string processing). -Craig From alexakarpov@REDACTED Sun Oct 8 04:54:19 2017 From: alexakarpov@REDACTED (Alexandre Karpov) Date: Sat, 7 Oct 2017 22:54:19 -0400 Subject: [erlang-questions] erlang to browsers ("sessions" question, I suppose) Message-ID: Hi all, I've just started on Cowboy (which I've discovered through this same mailing list, thanks y'all), and the introductory article resonated perfectly and expressed exactly the reasons for me quitting my last job to go on a "sabbatical" studying Erlang (which I've first found out back in 2010 from a talk by an Amazon's Principal Engineer in Seattle, who was leagues ahead of me in all things software, and was in love with the language and environment). Aside from the goal of studying the language and OTP, there were a couple of questions which really interested me; hope they're not too hand-wavy and general to raise up: 1) Will there be a BEAM emulator in Javascript? Joe chuckled in one of his talks how back when Erlang was being created, the computers it was to run on were weaker than our modern cell phones =) But, as I can see, the state of the art is still not quite there yet. What are your thoughts on it? Will this ever come? If no, why not? - and if yes, then what are the major problems? Because I'm sure you can see why this sounds like a majestic possible universe - imagine each browser a potential Erlang node... 2) Aside from that phantasy, having browser acting as a client of some Erlang system is obviously a reality long time ago. There are, well, web servers, and Cowboy with Websockets is an obvious point of interest. Sorry if I am using improper terminology here - is Cowboy server a good fit for creating and maintaining "sessions", which is to say, connections which have some sort of a pair of stateful process on the server side and a client side? Normally I'd think of databases when I think of such "persistent connections" - connection is actually stateless, but every request from a client is accompanied by database queries, and so we're faking the "session"... with WebSockets+Erlang, is there a more real session option? Is there already a setup where I get an Erlang process waiting for a next message from a browser, replying to the browser with a message? Cheers, Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfrench@REDACTED Sun Oct 8 08:57:23 2017 From: mfrench@REDACTED (Mike French) Date: Sun, 8 Oct 2017 06:57:23 +0000 Subject: [erlang-questions] erlang to browsers ("sessions" question, I suppose) In-Reply-To: References: Message-ID: <4abcb12990e34602be57dc8a279e2065@TCC-EXCH-01.tcclimited1.com> (1) I assume you want BEAM on Web Assembly these days, not JavaScript ? http://webassembly.org/ Mike From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Alexandre Karpov Sent: Sunday, October 08, 2017 5:54 AM To: erlang-questions@REDACTED Subject: [erlang-questions] erlang to browsers ("sessions" question, I suppose) Hi all, I've just started on Cowboy (which I've discovered through this same mailing list, thanks y'all), and the introductory article resonated perfectly and expressed exactly the reasons for me quitting my last job to go on a "sabbatical" studying Erlang (which I've first found out back in 2010 from a talk by an Amazon's Principal Engineer in Seattle, who was leagues ahead of me in all things software, and was in love with the language and environment). Aside from the goal of studying the language and OTP, there were a couple of questions which really interested me; hope they're not too hand-wavy and general to raise up: 1) Will there be a BEAM emulator in Javascript? Joe chuckled in one of his talks how back when Erlang was being created, the computers it was to run on were weaker than our modern cell phones =) But, as I can see, the state of the art is still not quite there yet. What are your thoughts on it? Will this ever come? If no, why not? - and if yes, then what are the major problems? Because I'm sure you can see why this sounds like a majestic possible universe - imagine each browser a potential Erlang node... 2) Aside from that phantasy, having browser acting as a client of some Erlang system is obviously a reality long time ago. There are, well, web servers, and Cowboy with Websockets is an obvious point of interest. Sorry if I am using improper terminology here - is Cowboy server a good fit for creating and maintaining "sessions", which is to say, connections which have some sort of a pair of stateful process on the server side and a client side? Normally I'd think of databases when I think of such "persistent connections" - connection is actually stateless, but every request from a client is accompanied by database queries, and so we're faking the "session"... with WebSockets+Erlang, is there a more real session option? Is there already a setup where I get an Erlang process waiting for a next message from a browser, replying to the browser with a message? Cheers, Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: From jamhedd@REDACTED Sun Oct 8 09:26:27 2017 From: jamhedd@REDACTED (Roman Galeev) Date: Sun, 8 Oct 2017 09:26:27 +0200 Subject: [erlang-questions] erlang to browsers ("sessions" question, I suppose) In-Reply-To: <4abcb12990e34602be57dc8a279e2065@TCC-EXCH-01.tcclimited1.com> References: <4abcb12990e34602be57dc8a279e2065@TCC-EXCH-01.tcclimited1.com> Message-ID: > Is there already a setup where I get an Erlang process waiting for a next message from a browser, replying to the browser with a message? Well, you don't need BEAM on a browser to have it. If you use web sockets, then Cowboy will spawn an Erlang process per each connection and manage its state. On a client, you need to have a singleton (or whatever) and do the same state management. Then you code a line protocol, and voila. And the web socket connection is persistent. This approach has an advantage: you can delegate the development of a browser UI to 'front-end' developers. On Sun, Oct 8, 2017 at 8:57 AM, Mike French wrote: > (1) I assume you want BEAM on Web Assembly these days, not JavaScript > ? > > > > http://webassembly.org/ > > > > Mike > > > > *From:* erlang-questions-bounces@REDACTED [mailto:erlang-questions- > bounces@REDACTED] *On Behalf Of *Alexandre Karpov > *Sent:* Sunday, October 08, 2017 5:54 AM > *To:* erlang-questions@REDACTED > *Subject:* [erlang-questions] erlang to browsers ("sessions" question, I > suppose) > > > > Hi all, > > > > I've just started on Cowboy (which I've discovered through this same > mailing list, thanks y'all), and the introductory article resonated > perfectly and expressed exactly the reasons for me quitting my last job to > go on a "sabbatical" studying Erlang (which I've first found out back in > 2010 from a talk by an Amazon's Principal Engineer in Seattle, who was > leagues ahead of me in all things software, and was in love with the > language and environment). > > > > Aside from the goal of studying the language and OTP, there were a couple > of questions which really interested me; hope they're not too hand-wavy and > general to raise up: > > > > 1) Will there be a BEAM emulator in Javascript? > > Joe chuckled in one of his talks how back when Erlang was being created, > the computers it was to run on were weaker than our modern cell phones =) > But, as I can see, the state of the art is still not quite there yet. What > are your thoughts on it? Will this ever come? If no, why not? - and if yes, > then what are the major problems? Because I'm sure you can see why this > sounds like a majestic possible universe - imagine each browser a potential > Erlang node... > > > > 2) Aside from that phantasy, having browser acting as a client of some > Erlang system is obviously a reality long time ago. There are, well, web > servers, and Cowboy with Websockets is an obvious point of interest. Sorry > if I am using improper terminology here - is Cowboy server a good fit for > creating and maintaining "sessions", which is to say, connections which > have some sort of a pair of stateful process on the server side and a > client side? Normally I'd think of databases when I think of such > "persistent connections" - connection is actually stateless, but every > request from a client is accompanied by database queries, and so we're > faking the "session"... with WebSockets+Erlang, is there a more real > session option? Is there already a setup where I get an Erlang process > waiting for a next message from a browser, replying to the browser with a > message? > > > > Cheers, > > Alex > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- With best regards, Roman Galeev, +420 702 817 968 -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Sun Oct 8 12:34:51 2017 From: serge@REDACTED (Serge Aleynikov) Date: Sun, 8 Oct 2017 06:34:51 -0400 Subject: [erlang-questions] Erlang/OTP 20.1 has been released, In-Reply-To: <1506502877.3411.5.camel@ericsson.com> References: <1506502877.3411.5.camel@ericsson.com> Message-ID: Please note that the source tarball http://erlang.org/download/otp_src_20.1.tar.gz and the associated manual http://erlang.org/download/otp_doc_man_20.1.tar.gz are not stored in the compressed format despite having the ".gz" extension. On Wed, Sep 27, 2017 at 5:01 AM, Henrik Nord X wrote: > Erlang/OTP 20.1 is the first service release for the 20 major release. > The service release contains mostly bug fixes and characteristics > improvements but also some new features. > > Some highlights for 20.1 > > crypto, public_key: Extend crypto and public_key functions sign and > verify with: > support for RSASSA-PS padding for signatures and for > saltlength setting > X9.31 RSA padding. > sha, sha224, sha256, sha384, and sha512 for dss > signatures as mentioned in NIST SP 800-57 Part 1. > ripemd160 to be used for rsa signatures. > > A new tuple in crypto:supports/0 reports supported MAC > algorithms. > > diameter: > Add service option decode_format to allow incoming messages to > be decoded into maps > instead of records. > Decode performance has been improved. > Add service/transport option avp_dictionaries to give better > support for dictionaries only defining AVPs. > > erts: Upgraded the ERTS internal PCRE library from version 8.40 to > version 8.41. > > erts, kernel, tools: Profiling with lock counting can now be fully > toggled at runtime in > the lock counting emulator (-emu_type lcnt). Everything > is enabled by default to match the old behavior, but > specific categories can be toggled at will with minimal > runtime overhead when disabled. Refer to the > documentation on lcnt:rt_mask/1 for details. > > lcnt:collect and lcnt:clear will no longer block all > other threads in the runtime system. This makes it possible to run > the lock counting > under heavier load. > > erts: The zlib module has been refactored and all its operations > will now yield appropriately, > allowing them to be used freely in concurrent applications. > > erts, tools: Add erlang:iolist_to_iovec/1, which converts an > iolist() to an erlang:iovec(), which is suitable for use > with enif_inspect_iovec(). > > erts: Add new nif API functions for managing an I/O Queue. > The added functions are: enif_ioq_create(), enif_ioq_destroy(), > enif_ioq_enq_binary(), > enif_ioq_enqv(), enif_ioq_deq(), enif_ioq_peek(), > enif_inspect_iovec(), enif_free_iovec() > > observer/crashdump_viewer: > Reading of crash dumps with many binaries is optimized. > A progress bar is shown when the detail view for a process is > opened. > The cdv script now sets ERL_CRASH_DUMP_SECONDS=0 to > avoid generating a new crash dump from the node running the > Crashdump Viewer. > > observer: > Add system statistics and limits to frontpage in > observer. > > public_key, ssl**: > Improved error propagation and reports > > ssh: A new option modify_algorithms is implemented. It > enables specifying changes on the default algorithms > list. See the reference manual and the SSH User's Guide > chapter "Configuring algorithms in SSH". > > tools/xref: The predefined Xref analysis locals_not_used now > understands > the -on_load() attribute and does not report unused functions. > > tools/fprof: When sampling multiple processes and analyzing > with totals set to true, the output now sums together > all caller and callee entries which concerns the same > function. Previous behaviour was to report each > contributing entry separately. > > You can find the README and the full listing of changes for this > service release at > > http://www.erlang.org/download/otp_src_20.1.readme > > The source distribution and binary distributions for Windows can be > downloaded from > > http://www.erlang.org/download/otp_src_20.1.tar.gz > http://www.erlang.org/download/otp_win32_20.1.exe > http://www.erlang.org/download/otp_win64_20.1.exe > > Note: To unpack the TAR archive you need a GNU TAR compatible program. > > For installation instructions please consult the README file that is > part > of the distribution. > > The Erlang/OTP source can also be found at GitHub on the official > Erlang > repository, https://github.com/erlang/otp with tag OTP-20.1 > > The on-line documentation can be found at: http://www.erlang.org/doc/ > You can also download the complete HTML documentation or the Unix > manual files > > http://www.erlang.org/download/otp_doc_html_20.1.tar.gz > http://www.erlang.org/download/otp_doc_man_20.1.tar.gz > > Please report any new issues via Erlang/OTPs public issue tracker > > https://bugs.erlang.org > > We want to thank all of those who sent us patches, suggestions and bug > reports! > > Thank you! > > The Erlang/OTP Team at Ericsson > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sun Oct 8 12:40:16 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sun, 8 Oct 2017 12:40:16 +0200 Subject: [erlang-questions] Erlang/OTP 20.1 has been released, In-Reply-To: References: <1506502877.3411.5.camel@ericsson.com> Message-ID: They are. The program you are using to download them must decompress them automatically without removing the extension. $ file otp_src_20.1.tar.gz otp_src_20.1.tar.gz: gzip compressed data, last modified: Tue Sep 26 09:39:57 2017, from Unix Probably because of the content-encoding header: $ curl -I http://erlang.org/download/otp_src_20.1.tar.gz HTTP/1.1 200 OK Date: Sun, 08 Oct 2017 10:38:50 GMT Server: Apache/1.3.42 (Unix) Last-Modified: Tue, 26 Sep 2017 10:17:28 GMT ETag: "22319d-534bcd8-59ca2938" Accept-Ranges: bytes Content-Length: 87342296 Content-Type: application/x-tar Content-Encoding: x-gzip On 10/08/2017 12:34 PM, Serge Aleynikov wrote: > Please note that the source tarball > http://erlang.org/download/otp_src_20.1.tar.gz and the associated manual > http://erlang.org/download/otp_doc_man_20.1.tar.gz are not stored in the > compressed format despite having the ".gz" extension. > > On Wed, Sep 27, 2017 at 5:01 AM, Henrik Nord X > > wrote: > > Erlang/OTP 20.1 is the first service release for the 20 major release. > The service release contains mostly bug fixes and characteristics > improvements but also some new features. > > Some highlights for 20.1 > > ????crypto, public_key: Extend crypto and public_key functions sign and > verify with: > ????????support for RSASSA-PS padding for signatures and for > ????????saltlength setting > ????????X9.31 RSA padding. > ????????sha, sha224, sha256, sha384, and sha512 for dss > ????????signatures as mentioned in NIST SP 800-57 Part 1. > ????????ripemd160 to be used for rsa signatures. > > ????A new tuple in crypto:supports/0 reports supported MAC > ????algorithms. > > ????diameter: > ????????Add service option decode_format to allow incoming messages to > be decoded into maps > ????????instead of records. > ????????Decode performance has been improved. > ????????Add service/transport option avp_dictionaries to give better > support for dictionaries only defining AVPs. > > ????erts: Upgraded the ERTS internal PCRE library from version 8.40 to > version 8.41. > > ????erts, kernel, tools: Profiling with lock counting can now be fully > toggled at runtime in > ????the lock counting emulator (-emu_type lcnt). Everything > ????is enabled by default to match the old behavior, but > ????specific categories can be toggled at will with minimal > ????runtime overhead when disabled. Refer to the > ????documentation on lcnt:rt_mask/1 for details. > > ????lcnt:collect and lcnt:clear will no longer block all > ????other threads in the runtime system. This makes it possible to run > the lock counting > ????under heavier load. > > ????erts: The zlib module has been refactored and all its operations > will now yield appropriately, > ????allowing them to be used freely in concurrent applications. > > ????erts, tools: Add erlang:iolist_to_iovec/1, which converts an > ????iolist() to an erlang:iovec(), which is suitable for use > ????with enif_inspect_iovec(). > > ????erts: Add new nif API functions for managing an I/O Queue. > ????The added functions are: enif_ioq_create(), enif_ioq_destroy(), > enif_ioq_enq_binary(), > ????enif_ioq_enqv(), enif_ioq_deq(), enif_ioq_peek(), > enif_inspect_iovec(), enif_free_iovec() > > ????observer/crashdump_viewer: > ????????Reading of crash dumps with many binaries is optimized. > ????????A progress bar is shown when the detail view for a process is > opened. > ????????The cdv script now sets ERL_CRASH_DUMP_SECONDS=0 to > ????????avoid generating a new crash dump from the node running the > Crashdump Viewer. > > ????observer: > ????Add system statistics and limits to frontpage in > ????observer. > > ????public_key, ssl**: > ????Improved error propagation and reports > > ????ssh: A new option modify_algorithms is implemented. It > ????enables specifying changes on the default algorithms > ????list. See the reference manual and the SSH User's Guide > ????chapter "Configuring algorithms in SSH". > > ????tools/xref: The predefined Xref analysis locals_not_used now > understands > ????the -on_load() attribute and does not report unused functions. > > ????tools/fprof: When sampling multiple processes and analyzing > ????with totals set to true, the output now sums together > ????all caller and callee entries which concerns the same > ????function. Previous behaviour was to report each > ????contributing entry separately. > > You can find the README and the full listing of changes for this > service release at > > http://www.erlang.org/download/otp_src_20.1.readme > > > The source distribution and binary distributions for Windows can be > downloaded from > > http://www.erlang.org/download/otp_src_20.1.tar.gz > > http://www.erlang.org/download/otp_win32_20.1.exe > > http://www.erlang.org/download/otp_win64_20.1.exe > > > Note: To unpack the TAR archive you need a GNU TAR compatible program. > > For installation instructions please consult the README file that is > part > of the distribution. > > The Erlang/OTP source can also be found at GitHub on the official > Erlang > repository, https://github.com/erlang/otp with tag OTP-20.1 > > The on-line documentation can be found at: http://www.erlang.org/doc/ > You can also download the complete HTML documentation or the Unix > manual files > > http://www.erlang.org/download/otp_doc_html_20.1.tar.gz > > http://www.erlang.org/download/otp_doc_man_20.1.tar.gz > > > Please report any new issues via Erlang/OTPs public issue tracker > > https://bugs.erlang.org > > We want to thank all of those who sent us patches, suggestions and bug > reports! > > Thank you! > > The Erlang/OTP Team at Ericsson > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From bchesneau@REDACTED Sun Oct 8 19:14:10 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sun, 8 Oct 2017 19:14:10 +0200 Subject: [erlang-questions] data structure like skips list in Erlang Message-ID: Hi all, I?m looking for a datastructure like a skiplist to maintain an ordered set of Key/Values. It has to have the following properties: * allows custom compare function to order the data in a specific order * allows the user to iterate forward (next) and backward (prev) the data Has someone already written a lib that offers such datastructure? Or maybe we can add a `prev(Key) function to gb_tree? Any idea i welcome :) - benoit From dmkolesnikov@REDACTED Sun Oct 8 19:33:44 2017 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sun, 8 Oct 2017 20:33:44 +0300 Subject: [erlang-questions] data structure like skips list in Erlang In-Reply-To: References: Message-ID: Hello, It looks like you are looking for data structure called zipper. I?ve not used any of these but there is at least two versions of zippers for Erlang. https://github.com/ferd/zippers https://github.com/inaka/zipper - Dmitry > On 8 Oct 2017, at 20.14, Benoit Chesneau wrote: > > Hi all, > > I?m looking for a datastructure like a skiplist to maintain an ordered set of Key/Values. It has to have the following properties: > > * allows custom compare function to order the data in a specific order > * allows the user to iterate forward (next) and backward (prev) the data > > Has someone already written a lib that offers such datastructure? Or maybe we can add a `prev(Key) function to gb_tree? Any idea i welcome :) > > - benoit > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Sun Oct 8 20:21:43 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sun, 8 Oct 2017 20:21:43 +0200 Subject: [erlang-questions] data structure like skips list in Erlang In-Reply-To: References: Message-ID: <881BB116-193D-4CE9-94FC-C9B4ECFA1A0F@gmail.com> hmm i forgot to mention i need to also lookup for the data so not sure if a zipper is designed for it. I will have a loom kn the libs anyway, thanks for the links :) - beno?t > On 8 Oct 2017, at 19:33, Dmitry Kolesnikov wrote: > > Hello, > > It looks like you are looking for data structure called zipper. I?ve not used any of these but there is at least two versions of zippers for Erlang. > > https://github.com/ferd/zippers > https://github.com/inaka/zipper > > - Dmitry > > >> On 8 Oct 2017, at 20.14, Benoit Chesneau wrote: >> >> Hi all, >> >> I?m looking for a datastructure like a skiplist to maintain an ordered set of Key/Values. It has to have the following properties: >> >> * allows custom compare function to order the data in a specific order >> * allows the user to iterate forward (next) and backward (prev) the data >> >> Has someone already written a lib that offers such datastructure? Or maybe we can add a `prev(Key) function to gb_tree? Any idea i welcome :) >> >> - benoit >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Mon Oct 9 00:49:14 2017 From: serge@REDACTED (Serge Aleynikov) Date: Sun, 8 Oct 2017 18:49:14 -0400 Subject: [erlang-questions] Erlang/OTP 20.1 has been released, In-Reply-To: References: <1506502877.3411.5.camel@ericsson.com> Message-ID: Yes, that program happens to be the Chrome browser, which when clicked, saves the file in decompressed form: $ ls -l otp_src_20.1.tar.gz -rw-r--r-- 1 serge mqt 237137920 Oct 8 18:45 otp_src_20.1.tar.gz $ file otp_src_20.1.tar.gz otp_src_20.1.tar.gz: POSIX tar archive (GNU) I don't think it's the Chrome issue - probably as you mentioned one of the headers messes it up. On Sun, Oct 8, 2017 at 6:40 AM, Lo?c Hoguin wrote: > They are. The program you are using to download them must decompress them > automatically without removing the extension. > > $ file otp_src_20.1.tar.gz > otp_src_20.1.tar.gz: gzip compressed data, last modified: Tue Sep 26 > 09:39:57 2017, from Unix > > Probably because of the content-encoding header: > > $ curl -I http://erlang.org/download/otp_src_20.1.tar.gz > HTTP/1.1 200 OK > Date: Sun, 08 Oct 2017 10:38:50 GMT > Server: Apache/1.3.42 (Unix) > Last-Modified: Tue, 26 Sep 2017 10:17:28 GMT > ETag: "22319d-534bcd8-59ca2938" > Accept-Ranges: bytes > Content-Length: 87342296 > Content-Type: application/x-tar > Content-Encoding: x-gzip > > On 10/08/2017 12:34 PM, Serge Aleynikov wrote: > >> Please note that the source tarball http://erlang.org/download/otp >> _src_20.1.tar.gz and the associated manual http://erlang.org/download/otp >> _doc_man_20.1.tar.gz are not stored in the compressed format despite >> having the ".gz" extension. >> >> On Wed, Sep 27, 2017 at 5:01 AM, Henrik Nord X < >> henrik.x.nord@REDACTED > wrote: >> >> Erlang/OTP 20.1 is the first service release for the 20 major release. >> The service release contains mostly bug fixes and characteristics >> improvements but also some new features. >> >> Some highlights for 20.1 >> >> crypto, public_key: Extend crypto and public_key functions sign >> and >> verify with: >> support for RSASSA-PS padding for signatures and for >> saltlength setting >> X9.31 RSA padding. >> sha, sha224, sha256, sha384, and sha512 for dss >> signatures as mentioned in NIST SP 800-57 Part 1. >> ripemd160 to be used for rsa signatures. >> >> A new tuple in crypto:supports/0 reports supported MAC >> algorithms. >> >> diameter: >> Add service option decode_format to allow incoming messages >> to >> be decoded into maps >> instead of records. >> Decode performance has been improved. >> Add service/transport option avp_dictionaries to give better >> support for dictionaries only defining AVPs. >> >> erts: Upgraded the ERTS internal PCRE library from version 8.40 >> to >> version 8.41. >> >> erts, kernel, tools: Profiling with lock counting can now be >> fully >> toggled at runtime in >> the lock counting emulator (-emu_type lcnt). Everything >> is enabled by default to match the old behavior, but >> specific categories can be toggled at will with minimal >> runtime overhead when disabled. Refer to the >> documentation on lcnt:rt_mask/1 for details. >> >> lcnt:collect and lcnt:clear will no longer block all >> other threads in the runtime system. This makes it possible to >> run >> the lock counting >> under heavier load. >> >> erts: The zlib module has been refactored and all its operations >> will now yield appropriately, >> allowing them to be used freely in concurrent applications. >> >> erts, tools: Add erlang:iolist_to_iovec/1, which converts an >> iolist() to an erlang:iovec(), which is suitable for use >> with enif_inspect_iovec(). >> >> erts: Add new nif API functions for managing an I/O Queue. >> The added functions are: enif_ioq_create(), enif_ioq_destroy(), >> enif_ioq_enq_binary(), >> enif_ioq_enqv(), enif_ioq_deq(), enif_ioq_peek(), >> enif_inspect_iovec(), enif_free_iovec() >> >> observer/crashdump_viewer: >> Reading of crash dumps with many binaries is optimized. >> A progress bar is shown when the detail view for a process is >> opened. >> The cdv script now sets ERL_CRASH_DUMP_SECONDS=0 to >> avoid generating a new crash dump from the node running the >> Crashdump Viewer. >> >> observer: >> Add system statistics and limits to frontpage in >> observer. >> >> public_key, ssl**: >> Improved error propagation and reports >> >> ssh: A new option modify_algorithms is implemented. It >> enables specifying changes on the default algorithms >> list. See the reference manual and the SSH User's Guide >> chapter "Configuring algorithms in SSH". >> >> tools/xref: The predefined Xref analysis locals_not_used now >> understands >> the -on_load() attribute and does not report unused functions. >> >> tools/fprof: When sampling multiple processes and analyzing >> with totals set to true, the output now sums together >> all caller and callee entries which concerns the same >> function. Previous behaviour was to report each >> contributing entry separately. >> >> You can find the README and the full listing of changes for this >> service release at >> >> http://www.erlang.org/download/otp_src_20.1.readme >> >> >> The source distribution and binary distributions for Windows can be >> downloaded from >> >> http://www.erlang.org/download/otp_src_20.1.tar.gz >> >> http://www.erlang.org/download/otp_win32_20.1.exe >> >> http://www.erlang.org/download/otp_win64_20.1.exe >> >> >> Note: To unpack the TAR archive you need a GNU TAR compatible program. >> >> For installation instructions please consult the README file that is >> part >> of the distribution. >> >> The Erlang/OTP source can also be found at GitHub on the official >> Erlang >> repository, https://github.com/erlang/otp with tag OTP-20.1 >> >> The on-line documentation can be found at: http://www.erlang.org/doc/ >> You can also download the complete HTML documentation or the Unix >> manual files >> >> http://www.erlang.org/download/otp_doc_html_20.1.tar.gz >> >> http://www.erlang.org/download/otp_doc_man_20.1.tar.gz >> >> >> Please report any new issues via Erlang/OTPs public issue tracker >> >> https://bugs.erlang.org >> >> We want to thank all of those who sent us patches, suggestions and bug >> reports! >> >> Thank you! >> >> The Erlang/OTP Team at Ericsson >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- > Lo?c Hoguin > https://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmorneau@REDACTED Mon Oct 9 05:12:38 2017 From: dmorneau@REDACTED (Dominic Morneau) Date: Mon, 9 Oct 2017 12:12:38 +0900 Subject: [erlang-questions] System getting hang after running normal recursion program In-Reply-To: References: <6c3e693c-b48c-cefd-8a6d-bb9684e25d4f@utl.in> Message-ID: To add to the other replies, you can also set a heap size limit and have the VM kill your process if it grows too much: 1> erlang:process_flag(max_heap_size, 10000000). #{error_logger => true,kill => true,size => 0} 2> Start = fun Start(N) -> N * Start(N - 1) end. #Fun 3> Start(1). =ERROR REPORT==== 9-Oct-2017::11:59:38 === Process: <0.64.0> Context: maximum heap size reached Max Heap Size: 10000000 Total Heap Size: 15420698 Kill: true Error Logger: true GC Info: [{old_heap_block_size,5157867}, {heap_block_size,11347307}, {mbuf_size,0}, {recent_size,1308970}, {stack_size,1084478}, {old_heap_size,0}, {heap_size,4073388}, {bin_vheap_size,4}, {bin_vheap_block_size,46422}, {bin_old_vheap_size,0}, {bin_old_vheap_block_size,46422}] ** exception exit: killed 4> Dominic 2017-10-07 21:14 GMT+09:00 Arun : > Hi Leandro, > > Am aware about tail recursion in erlang, but here I am talking about > recursion which is not tail recursive. Here is the code snippet; > > start(N) -> > N*start(N-1). > > Here the function is not tail recursive and it is in an infinite loop. If > run this function my system is getting hang. That why i got the doubt why > it is making my system to hang and how the erlang scheduler will behave in > this scenario.? > Regards, > Arun P. > > > On Saturday 07 October 2017 04:07 PM, Leandro Ostera wrote: > > Hej Arun, hi. > > Depending on your recursion, you may or may not be building up a stack > that will at some point crash. I am unsure about how C compilers typically > deal with tail-recursion, but tail call optimization is automatically > performed in the Erlang VM for you. > > From the Learn You Some Erlang book chapter on Recursion ( > http://learnyousomeerlang.com/recursion): > > *Note:* tail recursion as seen here is not making the memory grow because > when the virtual machine sees a function calling itself in a tail position > (the last expression to be evaluated in a function), it eliminates the > current stack frame. This is called tail-call optimisation (TCO) and it is > a special case of a more general optimisation named *Last Call > Optimisation* (LCO). > > LCO is done whenever the last expression to be evaluated in a function > body is another function call. When that happens, as with TCO, the Erlang > VM avoids storing the stack frame. As such tail recursion is also possible > between multiple functions. As an example, the chain of functions a() -> > b(). b() -> c(). c() -> a(). will effectively create an infinite loop > that won't go out of memory as LCO avoids overflowing the stack. This > principle, combined with our use of accumulators is what makes tail > recursion useful. > > Hope this helps! > > On Sat, Oct 7, 2017 at 9:03 AM Arun wrote: > >> Hi all, >> >> I tried to run normal recursion program from erlang shell, the program >> runs infinitely and it is doing addition of numbers. But its been >> observed that soon after starting the program entire PC is getting hang. >> >> My doubt is that, why my PC is getting hang. ? , if something is going >> wrong, the erlang vm should be able to handle it, like in C if we run >> some recursion program infinity, it will crash by throwing segmentation >> fault error. >> >> What will be happening in the VM if I run a normal recursion program in >> erlang infinitely ? >> >> What will be the behavior of erlang scheduler in this scenario ? >> >> Can somebody please assist me on this issue. >> >> Thanks in advance >> Arun P. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From billy.svensson@REDACTED Mon Oct 9 09:52:52 2017 From: billy.svensson@REDACTED (Billy Svensson) Date: Mon, 9 Oct 2017 09:52:52 +0200 Subject: [erlang-questions] data structure like skips list in Erlang In-Reply-To: <881BB116-193D-4CE9-94FC-C9B4ECFA1A0F@gmail.com> References: <881BB116-193D-4CE9-94FC-C9B4ECFA1A0F@gmail.com> Message-ID: I've needed this in the past too. Couldn't find any lib that could do it so I just ported the floor and ceiling methods of the TreeMap class in Java to an Erlang module that operated on a gb_tree. It's a bit ugly though since you have to work on the "hidden" data structure of the gb_trees module, so would be great if this could be a part of the standard module instead. Not sure I still have that code, but could try to find it if you want. You would probably need to test it a bit better though since I only used it for a small hobby project. On Sun, Oct 8, 2017 at 8:21 PM, Benoit Chesneau wrote: > hmm i forgot to mention i need to also lookup for the data so not sure if a > zipper is designed for it. > > I will have a loom kn the libs anyway, thanks for the links :) > > - beno?t > > On 8 Oct 2017, at 19:33, Dmitry Kolesnikov wrote: > > Hello, > > It looks like you are looking for data structure called zipper. I?ve not > used any of these but there is at least two versions of zippers for Erlang. > > https://github.com/ferd/zippers > https://github.com/inaka/zipper > > - Dmitry > > > On 8 Oct 2017, at 20.14, Benoit Chesneau wrote: > > Hi all, > > I?m looking for a datastructure like a skiplist to maintain an ordered set > of Key/Values. It has to have the following properties: > > * allows custom compare function to order the data in a specific order > * allows the user to iterate forward (next) and backward (prev) the data > > Has someone already written a lib that offers such datastructure? Or maybe > we can add a `prev(Key) function to gb_tree? Any idea i welcome :) > > - benoit > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From lutz.behnke@REDACTED Mon Oct 9 10:24:32 2017 From: lutz.behnke@REDACTED (Lutz Behnke) Date: Mon, 9 Oct 2017 10:24:32 +0200 Subject: [erlang-questions] How handle Error in Dialyzer Message-ID: <41aa63a0-0ebd-c67f-fac8-522e18281bff@informatik.haw-hamburg.de> Hi there, I am experiencing a bug in Dialyzer (see log trace below), I have checked https://github.com/erlang/otp/wiki/Bug-reports, which tells me, I should prepare a minimal programm to show the error. Unfortunately I see the error only on a fairly large package. I have no idea how to start trimming my code without changing/removing the cause of the bug. Could anybody give me some ideas what is causing the error, so that I can trimm only the blubber and not the meat? Side note: The dialyzer is executing in a Docker environment running the curated erlang:latest image from Docker Hub (currently 20.1.1). I have tried with erlang:19, but got the same result. TIA mfg lutz ---------------------------log----------------------------------------- ===> Dialyzer starting, this may take a while... ===> Updating plt... ===> Resolving files... ===> Updating base plt... ===> Resolving files... ===> Building with 166 files in "/root/.cache/rebar3/rebar3_20.1.1_plt"... ===> Copying "/root/.cache/rebar3/rebar3_20.1.1_plt" to "/sage/LoadBalancingServer/_build/test/rebar3_20.1.1_plt"... ===> Checking 166 files in "/sage/LoadBalancingServer/_build/test/rebar3_20.1.1_plt"... ===> Adding 11 files to "/sage/LoadBalancingServer/_build/test/rebar3_20.1.1_plt"... ===> Doing success typing analysis... ===> Resolving files... ===> Analyzing 33 files with "/sage/LoadBalancingServer/_build/test/rebar3_20.1.1_plt"... =ERROR REPORT==== 9-Oct-2017::07:34:49 === Error in process <0.25464.0> with exit value: {{nocatch,{error,["dim_3_tree_map.erl",58,"14",58,32, [73,108,108,101,103,97,108,32,100,101,99,108,97,114,97,116, 105,111,110,32,111,102,32,35,"dim_3_map",123,"map",125, 10]]}}, [{dialyzer_utils,msg_with_position,2, [{file,"dialyzer_utils.erl"},{line,451}]}, {lists,foldl,3,[{file,"lists.erl"},{line,1263}]}, {lists,foreach,2,[{file,"lists.erl"},{line,1338}]}, {dialyzer_utils,process_record_remote_types,1, [{file,"dialyzer_utils.erl"},{line,365}]}, {dialyzer_analysis_callgraph,'-remote_type_postproc/2-fun-1-',4, [{file,"dialyzer_analysis_callgraph.erl"}, {line,206}]}, {dialyzer_analysis_callgraph,'-remote_type_postprocessing/2-fun-0-',2, [{file,"dialyzer_analysis_callgraph.erl"}, {line,168}]}]} =ERROR REPORT==== 9-Oct-2017::07:34:49 === Error in process <0.25428.0> with exit value: {function_clause, [{dialyzer_codeserver,get_callbacks, [{{nocatch, {error, ["dim_3_tree_map.erl",58,"14",58,32, [73,108,108,101,103,97,108,32,100,101,99,108,97,114,97, 116,105,111,110,32,111,102,32,35,"dim_3_map",123,"map", 125,10]]}}, [{dialyzer_utils,msg_with_position,2, [{file,"dialyzer_utils.erl"},{line,451}]}, {lists,foldl,3,[{file,"lists.erl"},{line,1263}]}, {lists,foreach,2,[{file,"lists.erl"},{line,1338}]}, {dialyzer_utils,process_record_remote_types,1, [{file,"dialyzer_utils.erl"},{line,365}]}, {dialyzer_analysis_callgraph,'-remote_type_postproc/2-fun-1-',4, [{file,"dialyzer_analysis_callgraph.erl"},{line,206}]}, {dialyzer_analysis_callgraph, '-remote_type_postprocessing/2-fun-0-',2, [{file,"dialyzer_analysis_callgraph.erl"},{line,168}]}]}], [{file,"dialyzer_codeserver.erl"},{line,368}]}, {dialyzer_plt,insert_callbacks,2,[{file,"dialyzer_plt.erl"},{line,149}]}, {dialyzer_analysis_callgraph,analysis_start,3, [{file,"dialyzer_analysis_callgraph.erl"},{line,143}]}]} ===> Error in dialyzing apps: Analysis failed with error: {function_clause, [{dialyzer_codeserver,get_callbacks, [{{nocatch, {error, ["dim_3_tree_map.erl",58,"14",58,32, [73,108,108,101,103,97,108,32,100,101,99,108,97,114,97, 116,105,111,110,32,111,102,32,35,"dim_3_map",123,"map", 125,10]]}}, [{dialyzer_utils,msg_with_position,2, [{file,"dialyzer_utils.erl"},{line,451}]}, {lists,foldl,3,[{file,"lists.erl"},{line,1263}]}, {lists,foreach,2,[{file,"lists.erl"},{line,1338}]}, {dialyzer_utils,process_record_remote_types,1, [{file,"dialyzer_utils.erl"},{line,365}]}, {dialyzer_analysis_callgraph,'-remote_type_postproc/2-fun-1-',4, [{file,"dialyzer_analysis_callgraph.erl"},{line,206}]}, {dialyzer_analysis_callgraph, '-remote_type_postprocessing/2-fun-0-',2, [{file,"dialyzer_analysis_callgraph.erl"},{line,168}]}]}], [{file,"dialyzer_codeserver.erl"},{line,368}]}, {dialyzer_plt,insert_callbacks,2,[{file,"dialyzer_plt.erl"},{line,149}]}, {dialyzer_analysis_callgraph,analysis_start,3, [{file,"dialyzer_analysis_callgraph.erl"},{line,143}]}]} Last messages in the log cache: Reading files and computing callgraph... done in 0.14 secs Removing edges... done in 0.01 secs -- Lutz Behnke Hochschule f?r Angewandte Wissenschaften Hamburg, Labor f?r Allgemeine Informatik, phone: +49 40 42875-8156 mailto:lutz.behnke@REDACTED fax : +49 40 2803770 http://users.informatik.haw-hamburg.de/~sage Berliner Tor 7, 20099 Hamburg, Germany -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5006 bytes Desc: S/MIME Cryptographic Signature URL: From kostis@REDACTED Mon Oct 9 10:57:27 2017 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 9 Oct 2017 10:57:27 +0200 Subject: [erlang-questions] How handle Error in Dialyzer In-Reply-To: <41aa63a0-0ebd-c67f-fac8-522e18281bff@informatik.haw-hamburg.de> References: <41aa63a0-0ebd-c67f-fac8-522e18281bff@informatik.haw-hamburg.de> Message-ID: <48cc22d7-cf13-f2a8-6100-913815cf5d6c@cs.ntua.gr> On 10/09/2017 10:24 AM, Lutz Behnke wrote: > Hi there, > > I am experiencing a bug in Dialyzer (see log trace below), I have > checked https://github.com/erlang/otp/wiki/Bug-reports, which tells me, > I should prepare a minimal programm to show the error. Unfortunately I > see the error only on a fairly large package. I have no idea how to > start trimming my code without changing/removing the cause of the bug. > > Could anybody give me some ideas what is causing the error, so that I > can trimm only the blubber and not the meat? As it's pretty clear from the line below, the crash happens when analyzing one of these 33 modules: > ===> Analyzing 33 files with > "/sage/LoadBalancingServer/_build/test/rebar3_20.1.1_plt"... It's not a very small number, but it's not a very big one either. I would start with "some educated guess" on a subset of them. Also, the lines below refer to a file "dim_3_tree_map.erl", which is most likely involved in the problem (if not the culprit) and to a module that is supposed to contain -- if not actually contains -- callbacks. So, I think you have a starting point. Hope this helps, Kostis > =ERROR REPORT==== 9-Oct-2017::07:34:49 === > Error in process <0.25464.0> with exit value: > {{nocatch,{error,["dim_3_tree_map.erl",58,"14",58,32, > > [73,108,108,101,103,97,108,32,100,101,99,108,97,114,97,116, > ?????????????????? 105,111,110,32,111,102,32,35,"dim_3_map",123,"map",125, > ?????????????????? 10]]}}, > ===> Error in dialyzing apps: Analysis failed with error: > {function_clause, > ??? [{dialyzer_codeserver,get_callbacks, > ???????? [{{nocatch, > ?????????????? {error, > ?????????????????? ["dim_3_tree_map.erl",58,"14",58,32, > > [73,108,108,101,103,97,108,32,100,101,99,108,97,114,97, From bchesneau@REDACTED Mon Oct 9 11:07:11 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 9 Oct 2017 11:07:11 +0200 Subject: [erlang-questions] data structure like skips list in Erlang In-Reply-To: References: <881BB116-193D-4CE9-94FC-C9B4ECFA1A0F@gmail.com> Message-ID: <086FCCAD-84CF-46B1-B263-9B39DA933683@gmail.com> I?m didn?t find anything that fit my need. There are some btree libs but that doesn?t fit my needs. Also a tree is not that fast for memory imo. I?m thinking to build a red-black tree lib that support iterations for it. Red-black trees look simple enough to be build as a pure functional data structure. Still contemplating though. Benoit > On 9 Oct 2017, at 09:52, Billy Svensson wrote: > > I've needed this in the past too. Couldn't find any lib that could do > it so I just ported the floor and ceiling methods of the TreeMap class > in Java to an Erlang module that operated on a gb_tree. > It's a bit ugly though since you have to work on the "hidden" data > structure of the gb_trees module, so would be great if this could be a > part of the standard module instead. > > Not sure I still have that code, but could try to find it if you want. > You would probably need to test it a bit better though since I only > used it for a small hobby project. > > On Sun, Oct 8, 2017 at 8:21 PM, Benoit Chesneau wrote: >> hmm i forgot to mention i need to also lookup for the data so not sure if a >> zipper is designed for it. >> >> I will have a loom kn the libs anyway, thanks for the links :) >> >> - beno?t >> >> On 8 Oct 2017, at 19:33, Dmitry Kolesnikov wrote: >> >> Hello, >> >> It looks like you are looking for data structure called zipper. I?ve not >> used any of these but there is at least two versions of zippers for Erlang. >> >> https://github.com/ferd/zippers >> https://github.com/inaka/zipper >> >> - Dmitry >> >> >> On 8 Oct 2017, at 20.14, Benoit Chesneau wrote: >> >> Hi all, >> >> I?m looking for a datastructure like a skiplist to maintain an ordered set >> of Key/Values. It has to have the following properties: >> >> * allows custom compare function to order the data in a specific order >> * allows the user to iterate forward (next) and backward (prev) the data >> >> Has someone already written a lib that offers such datastructure? Or maybe >> we can add a `prev(Key) function to gb_tree? Any idea i welcome :) >> >> - benoit >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From hans.bolinder@REDACTED Mon Oct 9 11:08:30 2017 From: hans.bolinder@REDACTED (Hans Bolinder) Date: Mon, 9 Oct 2017 09:08:30 +0000 Subject: [erlang-questions] How handle Error in Dialyzer In-Reply-To: <41aa63a0-0ebd-c67f-fac8-522e18281bff@informatik.haw-hamburg.de> References: <41aa63a0-0ebd-c67f-fac8-522e18281bff@informatik.haw-hamburg.de> Message-ID: Hi, The reason is "dim_3_tree_map.erl:14: Illegal declaration of #dim_3_map{map}\n" This is possibly due to the 'map' field having a type that is not a subtype of the field's declared type. I'm not sure why the error is reported in such a harsh way, or if it's indented to be that way. It may take a while before I have the opportunity to check the details, though. Best regards, Hans Bolinder, Erlang/OTP team, Ericsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From lutz.behnke@REDACTED Mon Oct 9 12:16:12 2017 From: lutz.behnke@REDACTED (Lutz Behnke) Date: Mon, 9 Oct 2017 12:16:12 +0200 Subject: [erlang-questions] How handle Error in Dialyzer In-Reply-To: <48cc22d7-cf13-f2a8-6100-913815cf5d6c@cs.ntua.gr> References: <41aa63a0-0ebd-c67f-fac8-522e18281bff@informatik.haw-hamburg.de> <48cc22d7-cf13-f2a8-6100-913815cf5d6c@cs.ntua.gr> Message-ID: Hi, I meant 'large' when compared to a piece of code of maybe 1-3 modules that would be ideal to provoke the error for further analysis. You are right regarding the module dim_3_tree_map. I had not identified it as the culprit, as it does not follow the naming convention of the application and did not recognized it as one of ours. And due to the unfamiliar form of the error messages, I did not initially consider the code under test to be the culprit. mfg lutz Am 09.10.2017 um 10:57 schrieb Kostis Sagonas: > On 10/09/2017 10:24 AM, Lutz Behnke wrote: >> Hi there, >> >> I am experiencing a bug in Dialyzer (see log trace below), I have >> checked https://github.com/erlang/otp/wiki/Bug-reports, which tells >> me, I should prepare a minimal programm to show the error. >> Unfortunately I see the error only on a fairly large package. I have >> no idea how to start trimming my code without changing/removing the >> cause of the bug. >> >> Could anybody give me some ideas what is causing the error, so that I >> can trimm only the blubber and not the meat? > > As it's pretty clear from the line below, the crash happens when > analyzing one of these 33 modules: > >> ===> Analyzing 33 files with >> "/sage/LoadBalancingServer/_build/test/rebar3_20.1.1_plt"... > > It's not a very small number, but it's not a very big one either.? I > would start with "some educated guess" on a subset of them. > > Also, the lines below refer to a file "dim_3_tree_map.erl", which is > most likely involved in the problem (if not the culprit) and to a module > that is supposed to contain -- if not actually contains -- callbacks. > > So, I think you have a starting point. > > Hope this helps, > > Kostis > >> =ERROR REPORT==== 9-Oct-2017::07:34:49 === >> Error in process <0.25464.0> with exit value: >> {{nocatch,{error,["dim_3_tree_map.erl",58,"14",58,32, >> >> [73,108,108,101,103,97,108,32,100,101,99,108,97,114,97,116, >> >> 105,111,110,32,111,102,32,35,"dim_3_map",123,"map",125, >> ??????????????????? 10]]}}, >> ===> Error in dialyzing apps: Analysis failed with error: >> {function_clause, >> ???? [{dialyzer_codeserver,get_callbacks, >> ????????? [{{nocatch, >> ??????????????? {error, >> ??????????????????? ["dim_3_tree_map.erl",58,"14",58,32, >> [73,108,108,101,103,97,108,32,100,101,99,108,97,114,97, > -- Lutz Behnke Hochschule f?r Angewandte Wissenschaften Hamburg, Labor f?r Allgemeine Informatik, phone: +49 40 42875-8156 mailto:lutz.behnke@REDACTED fax : +49 40 2803770 http://users.informatik.haw-hamburg.de/~sage Berliner Tor 7, 20099 Hamburg, Germany -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5006 bytes Desc: S/MIME Cryptographic Signature URL: From frank.muller.erl@REDACTED Mon Oct 9 12:24:24 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 09 Oct 2017 10:24:24 +0000 Subject: [erlang-questions] data structure like skips list in Erlang In-Reply-To: <086FCCAD-84CF-46B1-B263-9B39DA933683@gmail.com> References: <881BB116-193D-4CE9-94FC-C9B4ECFA1A0F@gmail.com> <086FCCAD-84CF-46B1-B263-9B39DA933683@gmail.com> Message-ID: Benoit, It?s called ?Judy Array? and Erlang bindings do exist: https://github.com/knutin/judy You can search for next/previous elements and do stuffs no other hashing libs support. /Frank I?m didn?t find anything that fit my need. There are some btree libs but > that doesn?t fit my needs. Also a tree is not that fast for memory imo. I?m > thinking to build a red-black tree lib that support iterations for it. > Red-black trees look simple enough to be build as a pure functional data > structure. Still contemplating though. > > Benoit > > > On 9 Oct 2017, at 09:52, Billy Svensson > wrote: > > > > I've needed this in the past too. Couldn't find any lib that could do > > it so I just ported the floor and ceiling methods of the TreeMap class > > in Java to an Erlang module that operated on a gb_tree. > > It's a bit ugly though since you have to work on the "hidden" data > > structure of the gb_trees module, so would be great if this could be a > > part of the standard module instead. > > > > Not sure I still have that code, but could try to find it if you want. > > You would probably need to test it a bit better though since I only > > used it for a small hobby project. > > > > On Sun, Oct 8, 2017 at 8:21 PM, Benoit Chesneau > wrote: > >> hmm i forgot to mention i need to also lookup for the data so not sure > if a > >> zipper is designed for it. > >> > >> I will have a loom kn the libs anyway, thanks for the links :) > >> > >> - beno?t > >> > >> On 8 Oct 2017, at 19:33, Dmitry Kolesnikov > wrote: > >> > >> Hello, > >> > >> It looks like you are looking for data structure called zipper. I?ve not > >> used any of these but there is at least two versions of zippers for > Erlang. > >> > >> https://github.com/ferd/zippers > >> https://github.com/inaka/zipper > >> > >> - Dmitry > >> > >> > >> On 8 Oct 2017, at 20.14, Benoit Chesneau wrote: > >> > >> Hi all, > >> > >> I?m looking for a datastructure like a skiplist to maintain an ordered > set > >> of Key/Values. It has to have the following properties: > >> > >> * allows custom compare function to order the data in a specific order > >> * allows the user to iterate forward (next) and backward (prev) the data > >> > >> Has someone already written a lib that offers such datastructure? Or > maybe > >> we can add a `prev(Key) function to gb_tree? Any idea i welcome :) > >> > >> - benoit > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Mon Oct 9 14:24:29 2017 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 9 Oct 2017 14:24:29 +0200 Subject: [erlang-questions] How handle Error in Dialyzer In-Reply-To: References: <41aa63a0-0ebd-c67f-fac8-522e18281bff@informatik.haw-hamburg.de> <48cc22d7-cf13-f2a8-6100-913815cf5d6c@cs.ntua.gr> Message-ID: On 10/09/2017 12:16 PM, Lutz Behnke wrote: > You are right regarding the module dim_3_tree_map. I had not identified > it as the culprit, as it does not follow the naming convention of the > application and did not recognized it as one of ours. > > And due to the unfamiliar form of the error messages, ... Error messages generally have that property. They only start to look mildly familiar only once you have seen them for a number of times already. What is more surprising is that you may be using some software successfully for quite some time before you see the first error being printed from it. Based on your experience, I think that Dialyzer's error messages can be improved -- please do send the offending module. Somebody thought it's a good idea to create an io_list() instead of a string() expecting that the printing routine at the end will always be doing the flattening. Apparently, this is not always the case when things go wrong... Kostis From lutz.behnke@REDACTED Mon Oct 9 16:17:24 2017 From: lutz.behnke@REDACTED (Lutz Behnke) Date: Mon, 9 Oct 2017 16:17:24 +0200 Subject: [erlang-questions] How handle Error in Dialyzer In-Reply-To: References: <41aa63a0-0ebd-c67f-fac8-522e18281bff@informatik.haw-hamburg.de> Message-ID: <00ebebe4-7e6e-b42d-8dd8-f0c45eed205e@informatik.haw-hamburg.de> I am confused! Where did you pull that message from? Am 09.10.2017 um 11:08 schrieb Hans Bolinder: > Hi, > > The reason is > > "dim_3_tree_map.erl:14: Illegal declaration of #dim_3_map{map}\n" > > This is possibly due to the 'map' field having a type that is not > a subtype of the field's declared type. > > I'm not sure why the error is reported in such a harsh way, or if it's > indented to be that way. It may take a while before I have the > opportunity to check the details, though. > > Best regards, > > Hans Bolinder, Erlang/OTP team, Ericsson > > -- Lutz Behnke Hochschule f?r Angewandte Wissenschaften Hamburg, Labor f?r Allgemeine Informatik, phone: +49 40 42875-8156 mailto:lutz.behnke@REDACTED fax : +49 40 2803770 http://users.informatik.haw-hamburg.de/~sage Berliner Tor 7, 20099 Hamburg, Germany -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5006 bytes Desc: S/MIME Cryptographic Signature URL: From z@REDACTED Mon Oct 9 16:20:49 2017 From: z@REDACTED (Danil Zagoskin) Date: Mon, 9 Oct 2017 17:20:49 +0300 Subject: [erlang-questions] Proposal: process_info(Pid, start_time) Message-ID: Hi! TL;DR: Would you like to know when any process was started in little cost of memory usage on 32-bit systems and crashdump generation speed? Sometimes we have to inspect long-running systems. And if a complex enough system runs for a month, it's hard to answer these questions: - Process A should run at most 1 hour. Did it stall a week ago? - Process B is supervised and it's not good when it restarts. Did it restart last night? - Did process C exist when process D was started? - Was pid E reused by a new process since it was stored in ETS? To answer that questions we need a process start time to be stored somewhere. Given that almost every project uses third-party libraries with their own processes, the universal solution could be having process start time available in process_info. It turned out that ERTS already have a process start time, used by a crashdump writer. I proposed a PR exposing that value in process_info: https://github.com/erlang/otp/pull/1597 It's very simple ? just return a value we already have. But as Rickard Green noticed, it's better to have this time as Erlang monotonic time instead of system seconds in current implementation. This will slightly increase memory consumption on 32-bit systems and slightly slow down crashdump generation. The other option Rickard proposed is to remove that timestamp completely and save some memory. So, the questions are: - Do you find process_info(Pid, start_time) useful and worth 4 bytes/process on 32 bit systems? - Do you find process start time in a crashdump useful? -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Mon Oct 9 16:42:32 2017 From: vans_163@REDACTED (Vans S) Date: Mon, 9 Oct 2017 14:42:32 +0000 (UTC) Subject: [erlang-questions] data structure like skips list in Erlang In-Reply-To: References: <881BB116-193D-4CE9-94FC-C9B4ECFA1A0F@gmail.com> <086FCCAD-84CF-46B1-B263-9B39DA933683@gmail.com> Message-ID: <17682176.3975536.1507560152569@mail.yahoo.com> Either im completely missing the point or this has been in core erlang for a long time. ets ordered_set. > * allows custom compare function to order the data in a specific order Setup a key tuple instead of custom compare function. ?If you need to sort by values pull those out into the key tuple. > * allows the user to iterate forward (next) and backward (prev) the data> ets:next|prev > Has someone already written a lib that offers such datastructure? Or maybe we can add a `prev(Key) function to gb_tree? Any idea i welcome :) Only part that this does not fit is the requirement for a custom compare function. ?If the custom compare is very complex, itl be very expensive regardless and wont scale. ?If the custom compare is simple, a tuple key should replace it. On Monday, October 9, 2017 6:24 AM, Frank Muller wrote: Benoit, It?s called ?Judy Array? and Erlang bindings do exist:https://github.com/knutin/judy You can search for next/previous elements and do stuffs no other hashing libs support. /Frank I?m didn?t find anything that fit my need. There are some btree libs but that doesn?t fit my needs. Also a tree is not that fast for memory imo. I?m thinking to build a red-black tree lib that support iterations for it. Red-black trees look simple enough to be build as a pure functional data structure. Still contemplating though. Benoit > On 9 Oct 2017, at 09:52, Billy Svensson wrote: > > I've needed this in the past too. Couldn't find any lib that could do > it so I just ported the floor and ceiling methods of the TreeMap class > in Java to an Erlang module that operated on a gb_tree. > It's a bit ugly though since you have to work on the "hidden" data > structure of the gb_trees module, so would be great if this could be a > part of the standard module instead. > > Not sure I still have that code, but could try to find it if you want. > You would probably need to test it a bit better though since I only > used it for a small hobby project. > > On Sun, Oct 8, 2017 at 8:21 PM, Benoit Chesneau wrote: >> hmm i forgot to mention i need to also lookup for the data so not sure if a >> zipper is designed for it. >> >> I will have a loom kn the libs anyway, thanks for the links :) >> >> - beno?t >> >> On 8 Oct 2017, at 19:33, Dmitry Kolesnikov wrote: >> >> Hello, >> >> It looks like you are looking for data structure called zipper. I?ve not >> used any of these but there is at least two versions of zippers for Erlang. >> >> https://github.com/ferd/zippers >> https://github.com/inaka/zipper >> >> - Dmitry >> >> >> On 8 Oct 2017, at 20.14, Benoit Chesneau wrote: >> >> Hi all, >> >> I?m looking for a datastructure like a skiplist to maintain an ordered set >> of Key/Values. It has to have the following properties: >> >> * allows custom compare function to order the data in a specific order >> * allows the user to iterate forward (next) and backward (prev) the data >> >> Has someone already written a lib that offers such datastructure? Or maybe >> we can add a `prev(Key) function to gb_tree? Any idea i welcome :) >> >> - benoit >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Mon Oct 9 18:47:55 2017 From: lloyd@REDACTED (lloyd@REDACTED) Date: Mon, 9 Oct 2017 12:47:55 -0400 (EDT) Subject: [erlang-questions] Erlang web applications + security In-Reply-To: References: <9318B441-488C-45D9-9DE6-07FE8C00BC2A@writersglen.com> Message-ID: <1507567675.83211619@apps.rackspace.com> Thanks all, This is really helpful. These seem like tasks that invite "re-inventing the wheel" redundant effort. Anyone aware of scripts, Ansible playbooks, step-by-step checklists, etc. to help make the process more efficient and secure? Lloyd -----Original Message----- From: "Yu-ri Gordon" Sent: Monday, October 9, 2017 12:30pm To: "Leandro David Cacciagioni" Cc: "Lloyd R. Prentice" , erlang-questions@REDACTED Subject: Re: [erlang-questions] Erlang web applications + security from the OWASP list some of the high level tasks you will need to do: setting proper response headers ( cross origin, strict https, etc) input validation ( for cross site scripting) file upload scanning for viruses, etc securing authentication ( appropriate token policies, account lock against brute force attacks) update configs to remove server details from exposed via headers ( e.g. server:cowboy) you can run a vulnerability scan using tools like burp, zap to scan for holes in your web app On Sat, Oct 7, 2017 at 1:27 PM, Leandro David Cacciagioni < leandro.21.2008@REDACTED> wrote: > Hi Lloyd, > > I would say that for any web app (Not only in erlang) you must start at > least for securing the issues named in the OWASP 10 ( > https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project) that will > gonna give you a good coverage for the most basic and common sec issues in > the world wide web of today. > > Thanks, > Leandro.- > > On Oct 7, 2017 09:02, "Lloyd R. Prentice" wrote: > >> Hello, >> >> When I put an Erlang web application on line, what security issues do I >> need to address and what are recommended best practices to address them? >> >> Thanks, >> >> LRP >> >> Sent from my iPad >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From yuri.cho@REDACTED Mon Oct 9 18:30:09 2017 From: yuri.cho@REDACTED (Yu-ri Gordon) Date: Mon, 9 Oct 2017 12:30:09 -0400 Subject: [erlang-questions] Erlang web applications + security In-Reply-To: References: <9318B441-488C-45D9-9DE6-07FE8C00BC2A@writersglen.com> Message-ID: from the OWASP list some of the high level tasks you will need to do: setting proper response headers ( cross origin, strict https, etc) input validation ( for cross site scripting) file upload scanning for viruses, etc securing authentication ( appropriate token policies, account lock against brute force attacks) update configs to remove server details from exposed via headers ( e.g. server:cowboy) you can run a vulnerability scan using tools like burp, zap to scan for holes in your web app On Sat, Oct 7, 2017 at 1:27 PM, Leandro David Cacciagioni < leandro.21.2008@REDACTED> wrote: > Hi Lloyd, > > I would say that for any web app (Not only in erlang) you must start at > least for securing the issues named in the OWASP 10 ( > https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project) that will > gonna give you a good coverage for the most basic and common sec issues in > the world wide web of today. > > Thanks, > Leandro.- > > On Oct 7, 2017 09:02, "Lloyd R. Prentice" wrote: > >> Hello, >> >> When I put an Erlang web application on line, what security issues do I >> need to address and what are recommended best practices to address them? >> >> Thanks, >> >> LRP >> >> Sent from my iPad >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Mon Oct 9 19:06:11 2017 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 9 Oct 2017 19:06:11 +0200 Subject: [erlang-questions] How handle Error in Dialyzer In-Reply-To: <00ebebe4-7e6e-b42d-8dd8-f0c45eed205e@informatik.haw-hamburg.de> References: <41aa63a0-0ebd-c67f-fac8-522e18281bff@informatik.haw-hamburg.de> <00ebebe4-7e6e-b42d-8dd8-f0c45eed205e@informatik.haw-hamburg.de> Message-ID: On 10/09/2017 04:17 PM, Lutz Behnke wrote: > I am confused! > > Where did you pull that message from? Eshell V9.1.1 (abort with ^G) 1> ["dim_3_tree_map.erl",58,"14",58,32,[73,108,108,101,103,97,108,32,100,101,99,108,97,114,97,116,105,111,110,32,111,102,32,35,"dim_3_map",123,"map",125,10]]. ["dim_3_tree_map.erl",58,"14",58,32, [73,108,108,101,103,97,108,32,100,101,99,108,97,114,97,116, 105,111,110,32,111,102,32|...]] 2> io:format("~s", [v(1)]). dim_3_tree_map.erl:14: Illegal declaration of #dim_3_map{map} ok It was there all along ;) Kostis > Am 09.10.2017 um 11:08 schrieb Hans Bolinder: >> Hi, >> >> The reason is >> >> "dim_3_tree_map.erl:14: Illegal declaration of #dim_3_map{map}\n" >> >> This is possibly due to the 'map' field having a type that is not >> a subtype of the field's declared type. >> >> I'm not sure why the error is reported in such a harsh way, or if it's >> indented to be that way. It may take a while before I have the >> opportunity to check the details, though. >> >> Best regards, >> >> Hans Bolinder, Erlang/OTP team, Ericsson >> >> > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From gordeev.vladimir.v@REDACTED Mon Oct 9 20:18:49 2017 From: gordeev.vladimir.v@REDACTED (Vladimir Gordeev) Date: Mon, 9 Oct 2017 21:18:49 +0300 Subject: [erlang-questions] Proposal: process_info(Pid, start_time) In-Reply-To: References: Message-ID: Good feature. Currently I'm trying to build a trace visualization tool, and such information would be quite useful. It would allow to explore in what order different parts of the system were started. Also +1 for monotonic timestamp, that would allow to consistently put it along other trace timestamps. On Mon, Oct 9, 2017 at 5:20 PM, Danil Zagoskin wrote: > Hi! > > TL;DR: Would you like to know when any process was started in little cost > of memory usage on 32-bit systems and crashdump generation speed? > > Sometimes we have to inspect long-running systems. > And if a complex enough system runs for a month, it's hard to answer these > questions: > - Process A should run at most 1 hour. Did it stall a week ago? > - Process B is supervised and it's not good when it restarts. Did it > restart last night? > - Did process C exist when process D was started? > - Was pid E reused by a new process since it was stored in ETS? > > To answer that questions we need a process start time to be stored > somewhere. > Given that almost every project uses third-party libraries with their own > processes, the universal solution could be having process start time > available in process_info. > > It turned out that ERTS already have a process start time, used by a > crashdump writer. > > I proposed a PR exposing that value in process_info: https://github. > com/erlang/otp/pull/1597 > It's very simple ? just return a value we already have. > > But as Rickard Green noticed, it's better to have this time as Erlang > monotonic time instead of system seconds in current implementation. > This will slightly increase memory consumption on 32-bit systems and > slightly slow down crashdump generation. > > The other option Rickard proposed is to remove that timestamp completely > and save some memory. > > > So, the questions are: > > - Do you find process_info(Pid, start_time) useful and worth 4 > bytes/process on 32 bit systems? > - Do you find process start time in a crashdump useful? > > -- > Danil Zagoskin | z@REDACTED > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.c.williams10@REDACTED Tue Oct 10 21:45:42 2017 From: michael.c.williams10@REDACTED (Mike Williams) Date: Tue, 10 Oct 2017 20:45:42 +0100 Subject: [erlang-questions] WX not working! Message-ID: My old Pee Cee gave up the ghost and is now pushing up the daisies. However I had most things backed up, except the Erlang/OTP installation which I assumed would be easy to reproduce, but is wasn't . What wx packages do I need to get wx working? Despite having installed every wx package I can think off, I still get the message: "configure: WARNING: Can not link wx program are all developer packages installed?" Any helpful advice be appreciated! Thanks, mIKe -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed Oct 11 00:21:33 2017 From: zxq9@REDACTED (zxq9) Date: Wed, 11 Oct 2017 07:21:33 +0900 Subject: [erlang-questions] WX not working! In-Reply-To: References: Message-ID: <2450708.5gmhMshgK7@changa> On 2017?10?10? ??? 20:45:42 Mike Williams wrote: > My old Pee Cee gave up the ghost and is now pushing up the daisies. > However I had most things backed up, except the Erlang/OTP installation > which I assumed would be easy to reproduce, but is wasn't . > > What wx packages do I need to get wx working? Despite having installed > every wx package I can think off, I still get the message: > > "configure: WARNING: Can not link wx program are all developer packages > installed?" > > Any helpful advice be appreciated! On Ubuntu and Debian I believe the following are present before I build: Possible - lib64ncurses5-dev Installed - libncurses5-dev - libssl-dev - wx-common - libwxbase3.0-dev - flex - xsltproc - libwxgtk3.0-dev I am not quite sure what the minimum requirements are just to get wx to work, though. If you're on a different distro it will look similar to this. -Craig From attila.r.nohl@REDACTED Wed Oct 11 08:13:32 2017 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 11 Oct 2017 08:13:32 +0200 Subject: [erlang-questions] WX not working! In-Reply-To: <2450708.5gmhMshgK7@changa> References: <2450708.5gmhMshgK7@changa> Message-ID: 2017-10-11 0:21 GMT+02:00 zxq9 : > On 2017?10?10? ??? 20:45:42 Mike Williams wrote: >> My old Pee Cee gave up the ghost and is now pushing up the daisies. >> However I had most things backed up, except the Erlang/OTP installation >> which I assumed would be easy to reproduce, but is wasn't . >> >> What wx packages do I need to get wx working? Despite having installed >> every wx package I can think off, I still get the message: >> >> "configure: WARNING: Can not link wx program are all developer packages >> installed?" >> >> Any helpful advice be appreciated! > > On Ubuntu and Debian I believe the following are present before I build: On Ubuntu and Debian the apt-get build-dep erlang command will install everything required to build the package - probably that's good enough for building from source. From roger@REDACTED Wed Oct 11 10:28:38 2017 From: roger@REDACTED (Roger Lipscombe) Date: Wed, 11 Oct 2017 09:28:38 +0100 Subject: [erlang-questions] WX not working! In-Reply-To: References: Message-ID: Here are my vagrant shell-provisioning scripts for Erlang pre-reqs on Ubuntu 16.04: install-erlang-prerequisites.sh #!/bin/sh # I'm assuming that build-essential is required; it's hard to tell, because # Vagrant on VirtualBox installs a bunch of compiler-type stuff in order to # upgrade the Guest Additions. sudo apt-get -y install build-essential # assumed # These will result in the build failing if they're not present. sudo apt-get -y install autoconf m4 # ./otp_build: autoconf: not found sudo apt-get -y install libssl-dev # No usable OpenSSL found sudo apt-get -y install libncurses5-dev # configure: error: No curses library functions found # I'd consider this one "essential", because you need it to make observer work. sudo apt-get -y install libwxgtk3.0-dev # wxWidgets not found, wx will NOT be usable install-erlang-optional-prerequisites.sh #!/bin/sh # I consider these "optional", because I've never noticed them missing. sudo apt-get -y install default-jdk # jinterface : No Java compiler found sudo apt-get -y install unixodbc-dev # odbc : ODBC library - link check failed # These are for the documentation. sudo apt-get -y install xsltproc fop libxml2-utils I've tested these with Erlang/OTP 19.3.x; not with 20.x (yet). (forgot to cc the list, apologies to Mike for the duplicate...) On 10 October 2017 at 20:45, Mike Williams wrote: > My old Pee Cee gave up the ghost and is now pushing up the daisies. > However I had most things backed up, except the Erlang/OTP installation > which I assumed would be easy to reproduce, but is wasn't . > > What wx packages do I need to get wx working? Despite having installed > every wx package I can think off, I still get the message: > > "configure: WARNING: Can not link wx program are all developer packages > installed?" > > Any helpful advice be appreciated! > > Thanks, > > mIKe > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From technion@REDACTED Wed Oct 11 10:47:40 2017 From: technion@REDACTED (Technion) Date: Wed, 11 Oct 2017 08:47:40 +0000 Subject: [erlang-questions] Asserting exact maps Message-ID: Hi, I'm wondering if there is a simple process I can use to verify a map only contains valid variables. Consider the following example: 2> Checkmaps = fun(M) -> 2> #{"one" := One, "two" := _Two} = M, 2> One end. #Fun % These crash as expected 5> Checkmaps(#{"test" => one }). ** exception error: no match of right hand side value #{"test" => one} 6> Checkmaps(#{"one" => one }). ** exception error: no match of right hand side value #{"one" => one} % This works as expected 7> Checkmaps(#{"one" => one, "two" => two }). one % This however also runs - I would like it to crash like the first example 8> Checkmaps(#{"one" => one, "two" => two, "three" => test }). one The use case here is I'm pulling external data - anything I'm not expecting is not a happy path. Any assistance appreciated. -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Wed Oct 11 10:50:06 2017 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 11 Oct 2017 10:50:06 +0200 Subject: [erlang-questions] Asserting exact maps In-Reply-To: References: Message-ID: 2017-10-11 10:47 GMT+02:00 Technion : > Hi, > > > I'm wondering if there is a simple process I can use to verify a map only > contains valid variables. Consider the following example: > > > 2> Checkmaps = fun(M) -> > 2> #{"one" := One, "two" := _Two} = M, > 2> One end. > #Fun > > % These crash as expected > > 5> Checkmaps(#{"test" => one }). > ** exception error: no match of right hand side value #{"test" => one} > 6> Checkmaps(#{"one" => one }). > ** exception error: no match of right hand side value #{"one" => one} > > > % This works as expected > > 7> Checkmaps(#{"one" => one, "two" => two }). > one > > % This however also runs - I would like it to crash like the first example > > 8> Checkmaps(#{"one" => one, "two" => two, "three" => test }). > > one > > The use case here is I'm pulling external data - anything I'm not expecting > is not a happy path. Any assistance appreciated. Check also the size of the map. If it's greater than expected, it has extra elements. From zxq9@REDACTED Wed Oct 11 11:20:20 2017 From: zxq9@REDACTED (zxq9) Date: Wed, 11 Oct 2017 18:20:20 +0900 Subject: [erlang-questions] Asserting exact maps In-Reply-To: References: Message-ID: <2634448.OS7HlRA0Bn@changa> On 2017?10?11? ??? 08:47:40 Technion wrote: > Hi, > > > I'm wondering if there is a simple process I can use to verify a map > only contains valid variables. Consider the following example: > > > 2> Checkmaps = fun(M) -> > 2> #{"one" := One, "two" := _Two} = M, > 2> One end. > #Fun > > % These crash as expected > > 5> Checkmaps(#{"test" => one }). > ** exception error: no match of right hand side value #{"test" => one} > 6> Checkmaps(#{"one" => one }). > ** exception error: no match of right hand side value #{"one" => one} > > > % This works as expected > > 7> Checkmaps(#{"one" => one, "two" => two }). > one > > % This however also runs - I would like it to crash like the first example > > 8> Checkmaps(#{"one" => one, "two" => two, "three" => test }). > > one > > > The use case here is I'm pulling external data - anything I'm not > expecting is not a happy path. Any assistance appreciated. I find it easiest (both to write and to prove) to convert the map to a list and then perform a complete and exact comparison of keys over it. If I still need the map version then I'll pass the original if it survived, but almost every time I have an exact set of keys that must exist (and be the only keys that exist) I actually need a record, not a map (because the data is semantically a tuple, not a map of variable {K, V} pairs). The only exception to this is when I am passing the data on to some external lib that was written by the sort of person who believes that there is only one data type in the universe. Another easy to write and easy to prove approach is to incrementally pack a structure by using maps:take/2, and checking whether the map is empty at the end or not. If it is empty then your structure should be passed forward and life goes on as expected. Any variation of that will work fairly simply. 1> Checkmap = 1> fun(Map,Keys) -> 1> Drop = fun(K, M) -> {_, Next} = maps:take(K, M), Next end, 1> 0 = maps:size(lists:foldl(Drop, Map, Keys)) 1> end. #Fun 2> Checkmap(#{"one" => 1, "two" => 2}, ["one", "two"]). 0 3> Checkmap(#{"one" => 1, "two" => 2}, ["one", "two", "three"]). ** exception error: no match of right hand side value error 4> Checkmap(#{"one" => 1, "two" => 2}, ["one", "three"]). ** exception error: no match of right hand side value error 5> Checkmap(#{"one" => 1, "two" => 2}, ["one"]). ** exception error: no match of right hand side value 1 But I prefer the list version that operates over lists, personally. -Craig From zxq9@REDACTED Wed Oct 11 11:25:22 2017 From: zxq9@REDACTED (zxq9) Date: Wed, 11 Oct 2017 18:25:22 +0900 Subject: [erlang-questions] Asserting exact maps In-Reply-To: <2634448.OS7HlRA0Bn@changa> References: <2634448.OS7HlRA0Bn@changa> Message-ID: <1547974.34lHeh8ilg@changa> On 2017?10?11? ??? 18:20:20 zxq9 wrote: > On 2017?10?11? ??? 08:47:40 Technion wrote: > > Hi, > > > > > > I'm wondering if there is a simple process I can use to verify a map > > only contains valid variables. > > But I prefer the list version that operates over lists, personally. ... which I totally failed to provide any version of whatsoever. A super simple one: 1> Checkmap = 1> fun(Map, Keys) -> 1> lists:sort(Keys) == lists:sort(maps:keys(Map)) 1> end. #Fun 2> Checkmap(#{"one" => 1, "two" => 2}, ["one", "two"]). true 3> Checkmap(#{"one" => 1, "two" => 2}, ["two", "one"]). true 4> Checkmap(#{"one" => 1, "two" => 2}, ["two", "one", "three"]). false 5> Checkmap(#{"one" => 1, "two" => 2}, ["two"]). false Of course, here I am returning a boolean, but I prefer that for assertions because I can do: true = checkmap(Map, Keys) and move on -- and reuse that function somewhere else in a non-crashing way if I need to. -Craig From z@REDACTED Wed Oct 11 11:47:33 2017 From: z@REDACTED (Danil Zagoskin) Date: Wed, 11 Oct 2017 12:47:33 +0300 Subject: [erlang-questions] Asserting exact maps In-Reply-To: References: Message-ID: Just to add one more option. You may use maps:without to ensure the map does not have extra keys: 1> Check = fun(M) -> maps:without(["one", "two"], M) == #{} end. #Fun 2> Check(#{"one" => 1}). true 3> Check(#{"one" => 1, "three" => 3}). false On Wed, Oct 11, 2017 at 11:47 AM, Technion wrote: > Hi, > > > I'm wondering if there is a simple process I can use to verify a map only > contains valid variables. Consider the following example: > > > 2> Checkmaps = fun(M) -> > 2> #{"one" := One, "two" := _Two} = M, > 2> One end. > #Fun > > % These crash as expected > > 5> Checkmaps(#{"test" => one }). > ** exception error: no match of right hand side value #{"test" => one} > 6> Checkmaps(#{"one" => one }). > ** exception error: no match of right hand side value #{"one" => one} > > > % This works as expected > > 7> Checkmaps(#{"one" => one, "two" => two }). > one > > % This however also runs - I would like it to crash like the first example > > 8> Checkmaps(#{"one" => one, "two" => two, "three" => test }). > > one > > The use case here is I'm pulling external data - anything I'm not > expecting is not a happy path. Any assistance appreciated. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From a@REDACTED Wed Oct 11 11:05:16 2017 From: a@REDACTED (Alexander Mihajlovic) Date: Wed, 11 Oct 2017 11:05:16 +0200 Subject: [erlang-questions] Asserting exact maps In-Reply-To: References: Message-ID: <1507712716.1441281.1134971800.4ACA73CE@webmail.messagingengine.com> The function maps:with/2 [1] will get you most of the way there. It lets you filter a map to only contain keys in your given list. After that it's only a matter of matching the filtered map against the original, or you could check their lengths like Attila Rajmund Nohl wrote. 1> X = #{one => "1", two => "2"}. #{one => "1",two => "2"} 2> X = maps:with([one, two], X). #{one => "1",two => "2"} 3> Y = #{one => "1", two => "2", three => "3"}. #{one => "1",three => "3",two => "2"} 4> Y = maps:with([one, two], Y). ** exception error: no match of right hand side value #{one => "1",two => "2"} [1]: http://erlang.org/doc/man/maps.html#with-2 On Wed, Oct 11, 2017, at 10:50, Attila Rajmund Nohl wrote: > 2017-10-11 10:47 GMT+02:00 Technion : > > Hi, > > > > > > I'm wondering if there is a simple process I can use to verify a map only > > contains valid variables. Consider the following example: > > > > > > 2> Checkmaps = fun(M) -> > > 2> #{"one" := One, "two" := _Two} = M, > > 2> One end. > > #Fun > > > > % These crash as expected > > > > 5> Checkmaps(#{"test" => one }). > > ** exception error: no match of right hand side value #{"test" => one} > > 6> Checkmaps(#{"one" => one }). > > ** exception error: no match of right hand side value #{"one" => one} > > > > > > % This works as expected > > > > 7> Checkmaps(#{"one" => one, "two" => two }). > > one > > > > % This however also runs - I would like it to crash like the first example > > > > 8> Checkmaps(#{"one" => one, "two" => two, "three" => test }). > > > > one > > > > The use case here is I'm pulling external data - anything I'm not expecting > > is not a happy path. Any assistance appreciated. > > Check also the size of the map. If it's greater than expected, it has > extra elements. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From fxn@REDACTED Wed Oct 11 11:08:47 2017 From: fxn@REDACTED (Xavier Noria) Date: Wed, 11 Oct 2017 11:08:47 +0200 Subject: [erlang-questions] module autoloading and parallelism Message-ID: Say two processes running in parallel happen to trigger autoloading of the same module at the same time. How does the error handler manage this situation? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Wed Oct 11 12:35:16 2017 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Wed, 11 Oct 2017 12:35:16 +0200 Subject: [erlang-questions] module autoloading and parallelism In-Reply-To: References: Message-ID: Assuming the VM, stdlib and kernel have booted then the request goes to the code server process: https://github.com/erlang/otp/blob/ 5c917e33b53a397b30ca56a9cdd9922d9df19e7d/lib/kernel/src/ error_handler.erl#L41 Before that, it goes to the init process. In other words, the requests are serialized. *Jos? Valimwww.plataformatec.com.br Founder and Director of R&D* -------------- next part -------------- An HTML attachment was scrubbed... URL: From fxn@REDACTED Wed Oct 11 12:39:44 2017 From: fxn@REDACTED (Xavier Noria) Date: Wed, 11 Oct 2017 12:39:44 +0200 Subject: [erlang-questions] module autoloading and parallelism In-Reply-To: References: Message-ID: Awesome, thanks Jos? :). On Wed, Oct 11, 2017 at 12:35 PM, Jos? Valim wrote: > Assuming the VM, stdlib and kernel have booted then the request goes to > the code server process: https://github.com/erlang/otp/ > blob/5c917e33b53a397b30ca56a9cdd9922d9df19e7d/lib/kernel/ > src/error_handler.erl#L41 > > Before that, it goes to the init process. > > In other words, the requests are serialized. > > > > *Jos? Valimwww.plataformatec.com.br > Founder and Director of R&D* > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rickard@REDACTED Wed Oct 11 15:26:15 2017 From: rickard@REDACTED (Rickard Green) Date: Wed, 11 Oct 2017 15:26:15 +0200 Subject: [erlang-questions] Patch package OTP 20.1.2 released Message-ID: <4f9e6f29-c2df-3fe9-859e-2918b1a97507@erlang.org> Patch Package: OTP 20.1.2 Git Tag: OTP-20.1.2 Date: 2017-10-11 Trouble Report Id: OTP-14661, OTP-14668 Seq num: ERIERL-80, ERIERL-83 System: OTP Release: 20 Application: diameter-2.1.1, erts-9.1.2 Predecessor: OTP 20.1.1 Check out the git tag OTP-20.1.2, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- diameter-2.1.1 -------------------------------------------------- --------------------------------------------------------------------- The diameter-2.1.1 application can be applied independently of other applications on a full OTP 20 installation. --- Fixed Bugs and Malfunctions --- OTP-14668 Application(s): diameter Related Id(s): ERIERL-83 An inadvertently removed monitor in diameter 2.1 caused the ets table diameter_reg to leak entries, and caused service restart and more to fail. Full runtime dependencies of diameter-2.1.1: erts-6.4, kernel-3.2, ssl-6.0, stdlib-2.4 --------------------------------------------------------------------- --- erts-9.1.2 ------------------------------------------------------ --------------------------------------------------------------------- The erts-9.1.2 application can be applied independently of other applications on a full OTP 20 installation. --- Fixed Bugs and Malfunctions --- OTP-14661 Application(s): erts Related Id(s): ERIERL-80 Fixed bug that could cause a VM crash when a corrupt message is received on distribution channel from other node. Full runtime dependencies of erts-9.1.2: kernel-5.0, sasl-3.0.1, stdlib-3.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From michael.c.williams10@REDACTED Wed Oct 11 20:11:19 2017 From: michael.c.williams10@REDACTED (Mike Williams) Date: Wed, 11 Oct 2017 19:11:19 +0100 Subject: [erlang-questions] WX not working! In-Reply-To: References: Message-ID: All worked after I had installed openGL! Thanks mike On 11 October 2017 at 09:28, Roger Lipscombe wrote: > Here are my vagrant shell-provisioning scripts for Erlang pre-reqs on > Ubuntu 16.04: > > install-erlang-prerequisites.sh > > #!/bin/sh > > # I'm assuming that build-essential is required; it's hard to tell, because > # Vagrant on VirtualBox installs a bunch of compiler-type stuff in order to > # upgrade the Guest Additions. > sudo apt-get -y install build-essential # assumed > > # These will result in the build failing if they're not present. > sudo apt-get -y install autoconf m4 # ./otp_build: autoconf: not > found > sudo apt-get -y install libssl-dev # No usable OpenSSL found > sudo apt-get -y install libncurses5-dev # configure: error: No > curses library functions found > > # I'd consider this one "essential", because you need it to make observer > work. > sudo apt-get -y install libwxgtk3.0-dev # wxWidgets not found, wx > will NOT be usable > > install-erlang-optional-prerequisites.sh > > #!/bin/sh > > # I consider these "optional", because I've never noticed them missing. > sudo apt-get -y install default-jdk # jinterface : No Java > compiler found > sudo apt-get -y install unixodbc-dev # odbc : ODBC > library - link check failed > > # These are for the documentation. > sudo apt-get -y install xsltproc fop libxml2-utils > > I've tested these with Erlang/OTP 19.3.x; not with 20.x (yet). > > (forgot to cc the list, apologies to Mike for the duplicate...) > > On 10 October 2017 at 20:45, Mike Williams > wrote: > > My old Pee Cee gave up the ghost and is now pushing up the daisies. > > However I had most things backed up, except the Erlang/OTP installation > > which I assumed would be easy to reproduce, but is wasn't . > > > > What wx packages do I need to get wx working? Despite having installed > > every wx package I can think off, I still get the message: > > > > "configure: WARNING: Can not link wx program are all developer packages > > installed?" > > > > Any helpful advice be appreciated! > > > > Thanks, > > > > mIKe > > > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Thu Oct 12 10:01:57 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 12 Oct 2017 10:01:57 +0200 Subject: [erlang-questions] Cross OTP version compare-and-swap semantic on ETS Message-ID: <588F45D7-A6E1-471C-BCA9-F185A0138F14@gmail.com> Hi all, I?m looking on the best way (in term of concurrency and response speed) to provide a equivalent of compare-and-swap semantic on differs Erlang version with ETS. For now what I do is using `select_replace/2` on OTP 20 and for other versions I?m using a pen_server to serialise the writes on this values, making this gen_server a bottle neck. Which is fine for a small list of keys to update. I didn?t find any other solution to do it right now. But maybe someone has an idea? Also something I?m currently missing is the possibility do the dame for delete. `select_delete` isn?t atomic right? Beno?t From chandrashekhar.mullaparthi@REDACTED Thu Oct 12 10:17:21 2017 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Thu, 12 Oct 2017 09:17:21 +0100 Subject: [erlang-questions] Cross OTP version compare-and-swap semantic on ETS In-Reply-To: <588F45D7-A6E1-471C-BCA9-F185A0138F14@gmail.com> References: <588F45D7-A6E1-471C-BCA9-F185A0138F14@gmail.com> Message-ID: On 12 October 2017 at 09:01, Benoit Chesneau wrote: > Hi all, > > I?m looking on the best way (in term of concurrency and response speed) to > provide a equivalent of compare-and-swap semantic on differs Erlang version > with ETS. > For now what I do is using `select_replace/2` on OTP 20 and for other > versions I?m using a pen_server to serialise the writes on this values, > making this gen_server a bottle neck. Which is fine for a small list of > keys to update. I didn?t find any other solution to do it right now. But > maybe someone has an idea? > The only thing I can think of is to partition the key space and use multiple ETS tables (with a dedicated process for each). Also something I?m currently missing is the possibility do the dame for > delete. `select_delete` isn?t atomic right? > Not sure what you mean by 'atomic' here. I expect ETS will delete whatever entries have matched this specified pattern at that point in time. In the context of ETS operations, the consideration of whether it is 'atomic' or not makes sense in the 'insert', 'replace' or 'update' scenario. cheers, Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Oct 12 11:32:17 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 12 Oct 2017 12:32:17 +0300 Subject: [erlang-questions] Cross OTP version compare-and-swap semantic on ETS In-Reply-To: References: <588F45D7-A6E1-471C-BCA9-F185A0138F14@gmail.com> Message-ID: You should remember, that when you have a gen_server as a bottleneck, you can inspect his queue. When you move this bottleneck to raw access to ets, you get locks and they are very hard to introspect. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Thu Oct 12 13:15:30 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 12 Oct 2017 13:15:30 +0200 Subject: [erlang-questions] Cross OTP version compare-and-swap semantic on ETS In-Reply-To: References: <588F45D7-A6E1-471C-BCA9-F185A0138F14@gmail.com> Message-ID: <23A4A74C-6394-4749-8F94-FA05C1C32E1F@gmail.com> Thanks for the answer :) > On 12 Oct 2017, at 10:17, Chandru wrote: > > On 12 October 2017 at 09:01, Benoit Chesneau > wrote: > Hi all, > > I?m looking on the best way (in term of concurrency and response speed) to provide a equivalent of compare-and-swap semantic on differs Erlang version with ETS. > > For now what I do is using `select_replace/2` on OTP 20 and for other versions I?m using a pen_server to serialise the writes on this values, making this gen_server a bottle neck. Which is fine for a small list of keys to update. I didn?t find any other solution to do it right now. But maybe someone has an idea? > > The only thing I can think of is to partition the key space and use multiple ETS tables (with a dedicated process for each). True. I can do that indeed ? That may be the easiest thing to do in the long term. For now I I did this simple lib: https://gitlab.com/barrel-db/lab/ets_cas with limited test coverage. > > Also something I?m currently missing is the possibility do the dame for delete. `select_delete` isn?t atomic right? > > Not sure what you mean by 'atomic' here. I expect ETS will delete whatever entries have matched this specified pattern at that point in time. In the context of ETS operations, the consideration of whether it is 'atomic' or not makes sense in the 'insert', 'replace' or 'update' scenario. I?m not sure what?s the difference between a delete and a replace there with the current ETS librarie 20.1 . By atomic I mean atomic at the key level, ie 2 concurrent delete will be serialisable which seems to be the case for `select_replace` . Benoit. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Thu Oct 12 13:19:37 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 12 Oct 2017 13:19:37 +0200 Subject: [erlang-questions] Cross OTP version compare-and-swap semantic on ETS In-Reply-To: References: <588F45D7-A6E1-471C-BCA9-F185A0138F14@gmail.com> Message-ID: > On 12 Oct 2017, at 11:32, Max Lapshin wrote: > > You should remember, that when you have a gen_server as a bottleneck, you can inspect his queue. > > When you move this bottleneck to raw access to ets, you get locks and they are very hard to introspect. I hear point IMO the question is probably more to know until when it becomes a bottleneck ETS offers a simple way to maintain a fast concurrent access to data across processes. I believe it will takes time before achieving any bottleneck in term of concurrent accesses (do we have benches for it?) . But I like your conservative approach also. I need to think a little about it. Benoit From sverker.eriksson@REDACTED Thu Oct 12 13:44:32 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Thu, 12 Oct 2017 13:44:32 +0200 Subject: [erlang-questions] Cross OTP version compare-and-swap semantic on ETS In-Reply-To: <23A4A74C-6394-4749-8F94-FA05C1C32E1F@gmail.com> References: <588F45D7-A6E1-471C-BCA9-F185A0138F14@gmail.com> <23A4A74C-6394-4749-8F94-FA05C1C32E1F@gmail.com> Message-ID: <81ea8a62-db7a-7ea8-b696-ceb164f04b97@ericsson.com> On 10/12/2017 01:15 PM, Benoit Chesneau wrote: > Thanks for the answer :) > >> On 12 Oct 2017, at 10:17, Chandru >> > > wrote: >> >> On 12 October 2017 at 09:01, Benoit Chesneau > > wrote: >> >> Hi all, >> >> I?m looking on the best way (in term of concurrency and response >> speed) to provide a equivalent of compare-and-swap semantic on >> differs Erlang version with ETS. >> >> >> For now what I do is using `select_replace/2` on OTP 20 and for >> other versions I?m using a pen_server to serialise the writes on >> this values, making this gen_server a bottle neck. Which is fine >> for a small list of keys to update. I didn?t find any other >> solution to do it right now. But maybe someone has an idea? >> >> >> The only thing I can think of is to partition the key space and use >> multiple ETS tables (with a dedicated process for each). > > True. I can do that indeed ? That may be the easiest thing to do in > the long term. > > For now I I did this simple lib: > https://gitlab.com/barrel-db/lab/ets_cas?with limited test coverage. > >> >> Also something I?m currently missing is the possibility do the >> dame for delete. `select_delete` isn?t atomic right? >> >> >> Not sure what you mean by 'atomic' here. I expect ETS will delete >> whatever entries have matched this specified pattern at that point in >> time. In the context of ETS operations, the consideration of whether >> it is 'atomic' or not makes sense in the 'insert', 'replace' or >> 'update' scenario. > > > I?m not sure what?s the difference between a delete and a replace > there with the current ETS librarie 20.1 . By atomic I mean atomic at > the key level, ie 2 concurrent delete ?will be serialisable which > seems to be the case for `select_replace` . > > All ets:select* functions are atomic per table object. That is, each match-and-* operation is atomic. The iteration is not. /Sverker -------------- next part -------------- An HTML attachment was scrubbed... URL: From devangana@REDACTED Fri Oct 13 16:09:31 2017 From: devangana@REDACTED (Devangana Tarafdar) Date: Fri, 13 Oct 2017 09:09:31 -0500 Subject: [erlang-questions] Changing lager console backend log level Message-ID: Hello, I am having an issue with lager (3.4.0) and am wondering if anyone can point out if I am doing something wrong: Basically I want to change the log level for the console backend from the app config file. I am starting lager with the following inserted in the config file: {lager,[ {error_logger_whitelist,[yaws_log_file_h]}, {handlers, [ {lager_file_backend, [{file,"/tmp/lager_test.info"} ,{level,debug}]}, {lager_console_backend, [{level,info}]} ]}, {log_root, "/tmp"} ]}, Is the above syntax for lager_console_backend correct ? Lager seems to be having some problem with the console backend handler. I see this in the log file: 2017-10-12 15:34:33.405 [error] <0.60.0>@lager_handler_watcher:123 Lager fatally failed to install handler lager_console_backend into lager_event, NOT retrying: bad_log_level 2017-10-12 15:34:33.405 [debug] <0.62.0>@lager_handler_watcher:119 Lager installed handler error_logger_lager_h into error_logger 2017-10-12 15:34:33.435 [debug] <0.47.0> Supervisor gr_param_sup started gr_param:start_link(gr_lager_default_tracer_params) at pid <0.64.0> 2017-10-12 15:34:33.435 [debug] <0.46.0> Supervisor gr_counter_sup started gr_counter:start_link(gr_lager_default_tracer_counters) at pid <0.65.0> 2017-10-12 15:34:33.439 [debug] <0.48.0> Supervisor gr_manager_sup started gr_manager:start_link(gr_lager_default_tracer_params_mgr, gr_lager_default_tracer_params, []) at pid <0.66.0> 2017-10-12 15:34:33.439 [debug] <0.48.0> Supervisor gr_manager_sup started gr_manager:start_link(gr_lager_default_tracer_counters_mgr, gr_lager_default_tracer_counters, [{input,0},{filter,0},{output, 0},{job_input,0},{job_run,0},{job_time,0},{job_error,0}]) at pid <0.67.0> 2017-10-12 15:34:33.633 [info] <0.7.0> Application lager started on node ' d@REDACTED ' 2017-10-12 15:34:33.888 [debug] <0.56.0>@lager_handler_watcher:119 Lager installed handler lager_backend_throttle into lager_event I am starting up my tiny test environment as follows: [dtaraf@REDACTED lager_test_rel-1.0.0]$ erl -name d -setcookie x -config /usr/pkg/otp/rel/lager_test_rel-1.0.0/lager_test_rel.config -pz /usr/pkg/otp/lib/lager-3.4.0/ebin -pz /usr/pkg/otp/lib/goldrush-0.1.9/ebin *Erlang/OTP 17* [erts-6.3] [source] [64-bit] [async-threads:10] [kernel-poll:false] Eshell V6.3 (abort with ^G) (d@REDACTED)1> application:start(compiler),ap plication:start(syntax_tools),application:start(goldrush), application:start(lager). ok (d@REDACTED)2> lager:status(). Lager status: File /tmp/lager_test.info (lager_event) at level debug Console (lager_event) at level {error,bad_module} Active Traces: Tracing Reductions: Tracing Statistics: input: 0 output: 0 filter: 0 ok (d@REDACTED)3> application:which_applications(). [*{lager,"Erlang logging framework","3.4.0*"}, {goldrush,"Erlang event stream processor","0.1.9"}, {syntax_tools,"Syntax tools","1.6.17"}, {compiler,"ERTS CXC 138 10","5.0.3"}, {stdlib,"ERTS CXC 138 10","2.3"}, {kernel,"ERTS CXC 138 10","3.1"}] (d@REDACTED)4> Thanks ! Devangana -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Fri Oct 13 16:41:35 2017 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 13 Oct 2017 15:41:35 +0100 Subject: [erlang-questions] What is "Abandon carrier utilization limit" for? Message-ID: What does it mean for a carrier to be abandoned? The documentation says that if an allocator needs another carrier, it'll re-use an abandoned one, which doesn't sound very abandoned to me. Can the VM move blocks from an abandoned carrier to a new carrier? I assume not, which means that abandoned carriers can't (easily) be returned to the OS. Or is it that, once a carrier falls below the threshold, the VM tries not to use it, so that it's more likely (assuming that the remaining content eventually gets freed) that it'll get returned to the OS? Also: what is the default setting? The documentation says "de", but system_info is saying "0" for all of my allocators. Is "de" choosing zero for me? Cheers, Roger. From sverker.eriksson@REDACTED Fri Oct 13 19:05:43 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 13 Oct 2017 19:05:43 +0200 Subject: [erlang-questions] What is "Abandon carrier utilization limit" for? In-Reply-To: References: Message-ID: There is actually an internal doc on this subject: https://github.com/erlang/otp/blob/master/erts/emulator/internal_doc/CarrierMigration.md On 10/13/2017 04:41 PM, Roger Lipscombe wrote: > What does it mean for a carrier to be abandoned? Blocks are not allocated from abandoned carrier. An abandoned carrier is available to be employed by any thread in which case it's not abandoned anymore. > > The documentation says that if an allocator needs another carrier, > it'll re-use an abandoned one, which doesn't sound very abandoned to > me. > Can the VM move blocks from an abandoned carrier to a new carrier? I > assume not, which means that abandoned carriers can't (easily) be > returned to the OS. A carrier is a large contiguous piece of memory. A block a smaller contiguous piece of memory within a carrier. So no, moving block between carrier does not even make sense. > > Or is it that, once a carrier falls below the threshold, the VM tries > not to use it, so that it's more likely (assuming that the remaining > content eventually gets freed) that it'll get returned to the OS? Yes, that is the idea. > Also: what is the default setting? The documentation says "de", but > system_info is saying "0" for all of my allocators. Is "de" choosing > zero for me? > > Hmm.... I'll be back... From roger@REDACTED Fri Oct 13 19:33:00 2017 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 13 Oct 2017 18:33:00 +0100 Subject: [erlang-questions] What is "Abandon carrier utilization limit" for? In-Reply-To: References: Message-ID: On 13 October 2017 at 18:05, Sverker Eriksson wrote: > There is actually an internal doc on this subject: > > https://github.com/erlang/otp/blob/master/erts/emulator/internal_doc/CarrierMigration.md Thanks. I'll check that out. > A carrier is a large contiguous piece of memory. > A block a smaller contiguous piece of memory within a carrier. I watched Lukas's presentation from 2014, so I got that part :) > So no, moving block between carrier does not even make sense. I guess I meant: can the VM copy the content to another block in a "live" carrier, thus freeing up this carrier, but I guess that'd involve updating all kinds of pointers, and that stuff happens at a higher level in the VM, right? >> Or is it that, once a carrier falls below the threshold, the VM tries >> not to use it, so that it's more likely (assuming that the remaining >> content eventually gets freed) that it'll get returned to the OS? > > Yes, that is the idea. Cool. >> Also: what is the default setting? The documentation says "de", but >> system_info is saying "0" for all of my allocators. Is "de" choosing >> zero for me? >> >> > Hmm.... I'll be back... OTP 20-rc2, btw; I'll try a newer build... I'll be back... :) From roger@REDACTED Fri Oct 13 19:42:13 2017 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 13 Oct 2017 18:42:13 +0100 Subject: [erlang-questions] What is "Abandon carrier utilization limit" for? In-Reply-To: References: Message-ID: On 13 October 2017 at 18:33, Roger Lipscombe wrote: >>> Also: what is the default setting? The documentation says "de", but >>> system_info is saying "0" for all of my allocators. Is "de" choosing >>> zero for me? >>> >>> >> Hmm.... I'll be back... > > OTP 20-rc2, btw; I'll try a newer build... I'll be back... :) Still shows zero for OTP-20.1: 1> {_, _, _, A} = erlang:system_info(allocator). 2> lists:filtermap(fun({Id, Props}) when is_list(Props) -> {true, {Id, proplists:get_value(acul, Props, undefined)}}; (_) -> false end, A). [{sys_alloc,undefined}, {temp_alloc,0}, {sl_alloc,0}, {std_alloc,0}, {ll_alloc,0}, {eheap_alloc,0}, {ets_alloc,0}, {fix_alloc,0}, {literal_alloc,0}, {exec_alloc,0}, {binary_alloc,0}, {driver_alloc,0}, {test_alloc,undefined}, {mseg_alloc,undefined}, {alloc_util,undefined}, {erts_mmap,undefined}, {instr,undefined}] From sverker.eriksson@REDACTED Fri Oct 13 20:08:49 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 13 Oct 2017 20:08:49 +0200 Subject: [erlang-questions] What is "Abandon carrier utilization limit" for? In-Reply-To: References: Message-ID: On 10/13/2017 07:33 PM, Roger Lipscombe wrote: >> So no, moving block between carrier does not even make sense. > I guess I meant: can the VM copy the content to another block in a > "live" carrier, thus freeing up this carrier, but I guess that'd > involve updating all kinds of pointers, and that stuff happens at a > higher level in the VM, right? No, we don't do that. It can get very complicated in the general case as the higher level code has to update all pointers to the block in a thread-safe manner. We did some experimental hacks for binary_alloc where it could maybe be feasible as the data is read-only. >>> Or is it that, once a carrier falls below the threshold, the VM tries >>> not to use it, so that it's more likely (assuming that the remaining >>> content eventually gets freed) that it'll get returned to the OS? >> Yes, that is the idea. > Cool. > >>> Also: what is the default setting? The documentation says "de", but >>> system_info is saying "0" for all of my allocators. Is "de" choosing >>> zero for me? >>> >>> >> Hmm.... I'll be back... > OTP 20-rc2, btw; I'll try a newer build... I'll be back... :) > system_info(allocator) only gives an overview which can be a bit misleading. If you look at the individual allocator instances with for example erlang:system_info({allocator, ets_alloc}). you will see all instances except #0 have non zero values for 'acul'. Instance #0 is a lock protected allocator used only by non-scheduler threads. It disables carrier migration as only scheduler threads can use the lock-free carrier pool implementation. /Sverker From alexakarpov@REDACTED Sat Oct 14 04:21:41 2017 From: alexakarpov@REDACTED (Alexandre Karpov) Date: Fri, 13 Oct 2017 22:21:41 -0400 Subject: [erlang-questions] puzzled with this charset/encoding -related behaviour Message-ID: TL;DR: how do I run erl which understands Unicode? Or, in more detail: (Disclaimer: this official documentation got me really humbled: http://www1.erlang.org/doc/apps/stdlib/unicode_usage.html , and just a little bit scared =) ) Judging by my S/O question, which got 3 upvotes and no answers, I'm not the only one wondering: https://stackoverflow.com/questions/46735539/erlang-regexp-matching-on-chinese-characters Here's the gist of the problem: 57> "???". [1072,1073,1074] The codes are correct Unicode for the [Cyrillic] characters - which means my Terminal didn't fail to understand my keyboard's input =) but Erlang shell didn't recognize Terminal's input as printable characters. And it is my understanding that this is exactly why this call fails: 25> re:run("???.asd", xmerl_regexp:sh_to_awk("*.*"), [{capture, none}]). ** exception error: bad argument in function re:run/3 called as re:run([1081,1094,1091,46,97,115,100], "^(.*\\..*)$", [{capture,none}]) The reason why this came up is me trying the example from "Programming Erlang" where Joe gives you a lib_find module, and demonstrates reading of MP3 tags from files; because I tried looking for mp3 files on a path which had some Chinese characters in some filenames, this problem arose. I've tried finding way to run erl with a different charset (hoping for erl --charset=UTF8 or something), but only found references to file names, which in my case doesn't sound very related. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Sat Oct 14 10:12:19 2017 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Sat, 14 Oct 2017 10:12:19 +0200 Subject: [erlang-questions] puzzled with this charset/encoding -related behaviour In-Reply-To: References: Message-ID: 2017-10-14 4:21 GMT+02:00 Alexandre Karpov : > TL;DR: how do I run erl which understands Unicode? > > Or, in more detail: > > (Disclaimer: this official documentation got me really humbled: > http://www1.erlang.org/doc/apps/stdlib/unicode_usage.html > , and just a little bit scared =) ) > > Judging by my S/O question, which got 3 upvotes and no answers, I'm not the > only one wondering: > https://stackoverflow.com/questions/46735539/erlang-regexp-matching-on-chinese-characters > > Here's the gist of the problem: > > 57> "???". > > [1072,1073,1074] > > The codes are correct Unicode for the [Cyrillic] characters - which means my > Terminal didn't fail to understand my keyboard's input =) but Erlang shell > didn't recognize Terminal's input as printable characters. And it is my > understanding that this is exactly why this call fails: > > 25> re:run("???.asd", xmerl_regexp:sh_to_awk("*.*"), [{capture, none}]). ** > exception error: bad argument in function re:run/3 called as > re:run([1081,1094,1091,46,97,115,100], "^(.*\\..*)$", [{capture,none}]) Try re:run(<<"???.asd"/utf8>>, xmerl_regexp:sh_to_awk("*.*"), [{capture, none}]). From zxq9@REDACTED Sat Oct 14 10:19:08 2017 From: zxq9@REDACTED (zxq9) Date: Sat, 14 Oct 2017 17:19:08 +0900 Subject: [erlang-questions] puzzled with this charset/encoding -related behaviour In-Reply-To: References: Message-ID: <4715589.5CK81IsYuJ@changa> On 2017?10?14? ??? 10:12:19 Attila Rajmund Nohl wrote: > 2017-10-14 4:21 GMT+02:00 Alexandre Karpov : > > TL;DR: how do I run erl which understands Unicode? > > > > Or, in more detail: > > > > (Disclaimer: this official documentation got me really humbled: > > http://www1.erlang.org/doc/apps/stdlib/unicode_usage.html > > , and just a little bit scared =) ) > > > > Judging by my S/O question, which got 3 upvotes and no answers, I'm not the > > only one wondering: > > https://stackoverflow.com/questions/46735539/erlang-regexp-matching-on-chinese-characters > > > > Here's the gist of the problem: > > > > 57> "???". > > > > [1072,1073,1074] > > > > The codes are correct Unicode for the [Cyrillic] characters - which means my > > Terminal didn't fail to understand my keyboard's input =) but Erlang shell > > didn't recognize Terminal's input as printable characters. And it is my > > understanding that this is exactly why this call fails: > > > > 25> re:run("???.asd", xmerl_regexp:sh_to_awk("*.*"), [{capture, none}]). ** > > exception error: bad argument in function re:run/3 called as > > re:run([1081,1094,1091,46,97,115,100], "^(.*\\..*)$", [{capture,none}]) > > Try > > re:run(<<"???.asd"/utf8>>, xmerl_regexp:sh_to_awk("*.*"), [{capture, none}]). FYI: the SO question has an answer now. The regex execution needs to be put into unicode mode: re:run(<<"???.asd"/utf8>>, xmerl_regexp:sh_to_awk("*.*"), [unicode, {capture, none}]). -Craig From dangud@REDACTED Sat Oct 14 10:24:41 2017 From: dangud@REDACTED (Dan Gudmundsson) Date: Sat, 14 Oct 2017 08:24:41 +0000 Subject: [erlang-questions] puzzled with this charset/encoding -related behaviour In-Reply-To: References: Message-ID: re:run("???.asd", xmerl_regexp:sh_to_awk("*.*"), [{capture, none}, unicode]). The binary one matches since it works on bytes and not utf-8 characters? Also the erlang shell doesn't know if a list of integers is a list of integers or a string, since they may be represented by the same list of integers. So it tries to guess, by default it guesses that lists containing integers larger than 255 is not a string but a list of integers. You can change that with: (w)erl +pc unicode 1> "???.asd". "???.asd" /Dan On Sat, Oct 14, 2017 at 10:12 AM Attila Rajmund Nohl < attila.r.nohl@REDACTED> wrote: > 2017-10-14 4:21 GMT+02:00 Alexandre Karpov : > > TL;DR: how do I run erl which understands Unicode? > > > > Or, in more detail: > > > > (Disclaimer: this official documentation got me really humbled: > > http://www1.erlang.org/doc/apps/stdlib/unicode_usage.html > > , and just a little bit scared =) ) > > > > Judging by my S/O question, which got 3 upvotes and no answers, I'm not > the > > only one wondering: > > > https://stackoverflow.com/questions/46735539/erlang-regexp-matching-on-chinese-characters > > > > Here's the gist of the problem: > > > > 57> "???". > > > > [1072,1073,1074] > > > > The codes are correct Unicode for the [Cyrillic] characters - which > means my > > Terminal didn't fail to understand my keyboard's input =) but Erlang > shell > > didn't recognize Terminal's input as printable characters. And it is my > > understanding that this is exactly why this call fails: > > > > 25> re:run("???.asd", xmerl_regexp:sh_to_awk("*.*"), [{capture, none}]). > ** > > exception error: bad argument in function re:run/3 called as > > re:run([1081,1094,1091,46,97,115,100], "^(.*\\..*)$", [{capture,none}]) > > Try > > re:run(<<"???.asd"/utf8>>, xmerl_regexp:sh_to_awk("*.*"), [{capture, > none}]). > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexakarpov@REDACTED Sat Oct 14 18:23:19 2017 From: alexakarpov@REDACTED (Alexandre Karpov) Date: Sat, 14 Oct 2017 12:23:19 -0400 Subject: [erlang-questions] puzzled with this charset/encoding -related behaviour In-Reply-To: References: Message-ID: Thanks everyone! I didn't realize until this conversation how much more important strings-as-binaries are, compared to simple "strings". Everything _works_ now, of course, but I don't think my understanding has caught up 100% "by default it guesses that lists containing integers larger than 255 is not a string but a list of integers" <<< this really set some things straight But suffer me this follow-up question, Dan. Using +pc unicode indeed gave me a shell that represents lists of integers using the characters found in Unicode mapping; so now, in error messages, I see arguments reported more clearly: *7> re:run("???.asd", xmerl_regexp:sh_to_awk("*.*"), [{capture, none}]). * *** exception error: bad argument* * in function re:run/3* * called as re:run("???.asd","^(.*\\..*)$",[{capture,none}])* If I use the binary-string representation, it works _even_without_ /utf8, it works just fine: *3> re:run(<<"???.asd">>, xmerl_regexp:sh_to_awk("*.*"), [{capture, none}]).* *match* Note that the call above was executed in the shell started _without_ the +pc unicode, and the binary does _not_ have the /utf8>> thingy... This means my understanding is still lacking... binaries are honest and good, strings are fake and evil, but +px unicode seems to help a little with fake string... while using binary-strings doesn't _always_ require the /utf8 ... what is this sorcery?! =) On Sat, Oct 14, 2017 at 4:24 AM, Dan Gudmundsson wrote: > re:run("???.asd", xmerl_regexp:sh_to_awk("*.*"), [{capture, none}, > unicode]). > The binary one matches since it works on bytes and not utf-8 characters? > > Also the erlang shell doesn't know if a list of integers is a list of > integers or a string, > since they may be represented by the same list of integers. > > So it tries to guess, by default it guesses that lists containing integers > larger than 255 > is not a string but a list of integers. You can change that with: > > (w)erl +pc unicode > > 1> "???.asd". > "???.asd" > > /Dan > > > On Sat, Oct 14, 2017 at 10:12 AM Attila Rajmund Nohl < > attila.r.nohl@REDACTED> wrote: > >> 2017-10-14 4:21 GMT+02:00 Alexandre Karpov : >> > TL;DR: how do I run erl which understands Unicode? >> > >> > Or, in more detail: >> > >> > (Disclaimer: this official documentation got me really humbled: >> > http://www1.erlang.org/doc/apps/stdlib/unicode_usage.html >> > , and just a little bit scared =) ) >> > >> > Judging by my S/O question, which got 3 upvotes and no answers, I'm not >> the >> > only one wondering: >> > https://stackoverflow.com/questions/46735539/erlang-regexp-m >> atching-on-chinese-characters >> > >> > Here's the gist of the problem: >> > >> > 57> "???". >> > >> > [1072,1073,1074] >> > >> > The codes are correct Unicode for the [Cyrillic] characters - which >> means my >> > Terminal didn't fail to understand my keyboard's input =) but Erlang >> shell >> > didn't recognize Terminal's input as printable characters. And it is my >> > understanding that this is exactly why this call fails: >> > >> > 25> re:run("???.asd", xmerl_regexp:sh_to_awk("*.*"), [{capture, >> none}]). ** >> > exception error: bad argument in function re:run/3 called as >> > re:run([1081,1094,1091,46,97,115,100], "^(.*\\..*)$", [{capture,none}]) >> >> Try >> >> re:run(<<"???.asd"/utf8>>, xmerl_regexp:sh_to_awk("*.*"), [{capture, >> none}]). >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris@REDACTED Sat Oct 14 21:14:59 2017 From: chris@REDACTED (Chris Waymire) Date: Sat, 14 Oct 2017 12:14:59 -0700 Subject: [erlang-questions] Changing lager console backend log level In-Reply-To: References: Message-ID: Just taking a quick look it does not appear that lager_console_backend supports that argument style in 3.4.0 https://github.com/erlang-lager/lager/blob/3.4.0/src/lager_console_backend.erl Try {lager_console_backend, [info]} On Fri, Oct 13, 2017 at 7:09 AM, Devangana Tarafdar wrote: > Hello, > > I am having an issue with lager (3.4.0) and am wondering if anyone can > point out if I am doing something wrong: > > Basically I want to change the log level for the console backend from the > app config file. > > I am starting lager with the following inserted in the config file: > > {lager,[ > {error_logger_whitelist,[yaws_log_file_h]}, > {handlers, [ > {lager_file_backend, [{file,"/tmp/lager_test.info"} > ,{level,debug}]}, > > {lager_console_backend, [{level,info}]} > ]}, > > {log_root, "/tmp"} > > ]}, > > Is the above syntax for lager_console_backend correct ? Lager seems to > be having some problem with the console backend handler. I see this in the > log file: > > > 2017-10-12 15:34:33.405 [error] <0.60.0>@lager_handler_watcher:123 Lager > fatally failed to install handler lager_console_backend into lager_event, > NOT retrying: bad_log_level > > 2017-10-12 15:34:33.405 [debug] <0.62.0>@lager_handler_watcher:119 Lager > installed handler error_logger_lager_h into error_logger > 2017-10-12 15:34:33.435 [debug] <0.47.0> Supervisor gr_param_sup started > gr_param:start_link(gr_lager_default_tracer_params) at pid <0.64.0> > 2017-10-12 15:34:33.435 [debug] <0.46.0> Supervisor gr_counter_sup started > gr_counter:start_link(gr_lager_default_tracer_counters) at pid <0.65.0> > 2017-10-12 15:34:33.439 [debug] <0.48.0> Supervisor gr_manager_sup started > gr_manager:start_link(gr_lager_default_tracer_params_mgr, > gr_lager_default_tracer_params, []) at pid <0.66.0> > 2017-10-12 15:34:33.439 [debug] <0.48.0> Supervisor gr_manager_sup started > gr_manager:start_link(gr_lager_default_tracer_counters_mgr, > gr_lager_default_tracer_counters, [{input,0},{filter,0},{output, > 0},{job_input,0},{job_run,0},{job_time,0},{job_error,0}]) at pid <0.67.0> > 2017-10-12 15:34:33.633 [info] <0.7.0> Application lager started on node ' > d@REDACTED ' > 2017-10-12 15:34:33.888 [debug] <0.56.0>@lager_handler_watcher:119 Lager > installed handler lager_backend_throttle into lager_event > > I am starting up my tiny test environment as follows: > > [dtaraf@REDACTED lager_test_rel-1.0.0]$ erl -name d -setcookie x > -config /usr/pkg/otp/rel/lager_test_rel-1.0.0/lager_test_rel.config -pz > /usr/pkg/otp/lib/lager-3.4.0/ebin -pz /usr/pkg/otp/lib/goldrush-0.1.9/ebin > > *Erlang/OTP 17* [erts-6.3] [source] [64-bit] [async-threads:10] > [kernel-poll:false] > > Eshell V6.3 (abort with ^G) > > (d@REDACTED)1> application:start(compiler),ap > plication:start(syntax_tools),application:start(goldrush),ap > plication:start(lager). > ok > (d@REDACTED)2> lager:status(). > Lager status: > File /tmp/lager_test.info (lager_event) at level debug > Console (lager_event) at level {error,bad_module} > Active Traces: > Tracing Reductions: > Tracing Statistics: > input: 0 > output: 0 > filter: 0 > ok > > (d@REDACTED)3> application:which_applications(). > [*{lager,"Erlang logging framework","3.4.0*"}, > {goldrush,"Erlang event stream processor","0.1.9"}, > {syntax_tools,"Syntax tools","1.6.17"}, > {compiler,"ERTS CXC 138 10","5.0.3"}, > {stdlib,"ERTS CXC 138 10","2.3"}, > {kernel,"ERTS CXC 138 10","3.1"}] > (d@REDACTED)4> > > > Thanks ! > > Devangana > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From devangana@REDACTED Sun Oct 15 19:52:53 2017 From: devangana@REDACTED (Devangana Tarafdar) Date: Sun, 15 Oct 2017 12:52:53 -0500 Subject: [erlang-questions] Changing lager console backend log level In-Reply-To: References: Message-ID: Thank you for that suggestion, I will try that. On Oct 14, 2017 2:15 PM, "Chris Waymire" wrote: > Just taking a quick look it does not appear that lager_console_backend > supports that argument style in 3.4.0 > > https://github.com/erlang-lager/lager/blob/3.4.0/src/ > lager_console_backend.erl > > Try {lager_console_backend, [info]} > > On Fri, Oct 13, 2017 at 7:09 AM, Devangana Tarafdar > wrote: > >> Hello, >> >> I am having an issue with lager (3.4.0) and am wondering if anyone can >> point out if I am doing something wrong: >> >> Basically I want to change the log level for the console backend from the >> app config file. >> >> I am starting lager with the following inserted in the config file: >> >> {lager,[ >> {error_logger_whitelist,[yaws_log_file_h]}, >> {handlers, [ >> {lager_file_backend, [{file,"/tmp/lager_test.info"} >> ,{level,debug}]}, >> >> {lager_console_backend, [{level,info}]} >> ]}, >> >> {log_root, "/tmp"} >> >> ]}, >> >> Is the above syntax for lager_console_backend correct ? Lager seems to >> be having some problem with the console backend handler. I see this in the >> log file: >> >> >> 2017-10-12 15:34:33.405 [error] <0.60.0>@lager_handler_watcher:123 Lager >> fatally failed to install handler lager_console_backend into lager_event, >> NOT retrying: bad_log_level >> >> 2017-10-12 15:34:33.405 [debug] <0.62.0>@lager_handler_watcher:119 Lager >> installed handler error_logger_lager_h into error_logger >> 2017-10-12 15:34:33.435 [debug] <0.47.0> Supervisor gr_param_sup started >> gr_param:start_link(gr_lager_default_tracer_params) at pid <0.64.0> >> 2017-10-12 15:34:33.435 [debug] <0.46.0> Supervisor gr_counter_sup >> started gr_counter:start_link(gr_lager_default_tracer_counters) at pid >> <0.65.0> >> 2017-10-12 15:34:33.439 [debug] <0.48.0> Supervisor gr_manager_sup >> started gr_manager:start_link(gr_lager_default_tracer_params_mgr, >> gr_lager_default_tracer_params, []) at pid <0.66.0> >> 2017-10-12 15:34:33.439 [debug] <0.48.0> Supervisor gr_manager_sup >> started gr_manager:start_link(gr_lager_default_tracer_counters_mgr, >> gr_lager_default_tracer_counters, [{input,0},{filter,0},{output, >> 0},{job_input,0},{job_run,0},{job_time,0},{job_error,0}]) at pid <0.67.0> >> 2017-10-12 15:34:33.633 [info] <0.7.0> Application lager started on node ' >> d@REDACTED ' >> 2017-10-12 15:34:33.888 [debug] <0.56.0>@lager_handler_watcher:119 Lager >> installed handler lager_backend_throttle into lager_event >> >> I am starting up my tiny test environment as follows: >> >> [dtaraf@REDACTED lager_test_rel-1.0.0]$ erl -name d -setcookie x >> -config /usr/pkg/otp/rel/lager_test_rel-1.0.0/lager_test_rel.config -pz >> /usr/pkg/otp/lib/lager-3.4.0/ebin -pz /usr/pkg/otp/lib/goldrush-0.1. >> 9/ebin >> >> *Erlang/OTP 17* [erts-6.3] [source] [64-bit] [async-threads:10] >> [kernel-poll:false] >> >> Eshell V6.3 (abort with ^G) >> >> (d@REDACTED)1> application:start(compiler),ap >> plication:start(syntax_tools),application:start(goldrush),ap >> plication:start(lager). >> ok >> (d@REDACTED)2> lager:status(). >> Lager status: >> File /tmp/lager_test.info (lager_event) at level debug >> Console (lager_event) at level {error,bad_module} >> Active Traces: >> Tracing Reductions: >> Tracing Statistics: >> input: 0 >> output: 0 >> filter: 0 >> ok >> >> (d@REDACTED)3> application:which_applications(). >> [*{lager,"Erlang logging framework","3.4.0*"}, >> {goldrush,"Erlang event stream processor","0.1.9"}, >> {syntax_tools,"Syntax tools","1.6.17"}, >> {compiler,"ERTS CXC 138 10","5.0.3"}, >> {stdlib,"ERTS CXC 138 10","2.3"}, >> {kernel,"ERTS CXC 138 10","3.1"}] >> (d@REDACTED)4> >> >> >> Thanks ! >> >> Devangana >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From boyofmonk@REDACTED Mon Oct 16 10:29:22 2017 From: boyofmonk@REDACTED (Monk Boy) Date: Mon, 16 Oct 2017 16:29:22 +0800 Subject: [erlang-questions] erts dump in /usr/lib64/libcrypto.so.10 Message-ID: Hi all: When I use the ejabberd, I got a core file. I use gdb to analyzw the reasons. But I did not find the reason. The following is the process of mu analysis: ============= [/home/q/ejabberd16.09.1903]# gdb /home/q/erlang1903/lib/erlang/erts-8.3/bin/beam.smp var/lib/ejabberd/core GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /home/q/erlang1903/lib/erlang/erts-8.3/bin/beam.smp...done. [New Thread 22142] [New Thread 22144] [New Thread 22153] [New Thread 22143] [New Thread 22145] [New Thread 22130] [New Thread 22147] [New Thread 22134] [New Thread 22150] [New Thread 22128] [New Thread 22151] [New Thread 22131] [New Thread 22149] [New Thread 22148] [New Thread 22156] [New Thread 22160] [New Thread 22158] [New Thread 22136] [New Thread 22146] [New Thread 22126] [New Thread 22138] [New Thread 22141] [New Thread 22133] [New Thread 22135] [New Thread 22154] [New Thread 22152] [New Thread 22162] [New Thread 22132] [New Thread 22129] [New Thread 22140] [New Thread 22127] [New Thread 22161] [New Thread 22159] [New Thread 22163] [New Thread 22155] [New Thread 22137] [New Thread 22164] [New Thread 22157] Missing separate debuginfo for /home/q/erlang1903/lib/erlang/lib/crypto-3.7.3/priv/lib/crypto.so Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/81/bfdd1fd64e3ab54a08250be06c1206e9f58451 Missing separate debuginfo for /home/q/erlang1903/lib/erlang/lib/crypto-3.7.3/priv/lib/crypto_callback.so Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/48/3db7c6adda8b1189f736e2d6add5c27c8519cd Missing separate debuginfo for /home/q/ejabberd16.09.1903/lib/fast_yaml-1.0.6/priv/lib/fast_yaml.so Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/bd/2eda60ce2875f84049772382d146f5463a6b4f Missing separate debuginfo for /lib64/libyaml-0.so.2 Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/ae/35d9988c86d9305257e3a66c1dabf9a8d26247 Missing separate debuginfo for /home/q/ejabberd16.09.1903/lib/fast_tls-1.0.7/priv/lib/p1_sha.so Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/2f/2216be1af770efa084baf2ef9d12379bff0c28 Missing separate debuginfo for /home/q/ejabberd16.09.1903/lib/fast_tls-1.0.7/priv/lib/fast_tls_drv.so Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/19/1e48d91446e248dbff31134c9f5d1db9a5a596 Missing separate debuginfo for /home/q/ejabberd16.09.1903/lib/fast_xml-1.1.15/priv/lib/fxml.so Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/5b/8c6fd4eb179d16391929c1dcd30c6894c04154 Missing separate debuginfo for /home/q/ejabberd16.09.1903/lib/fast_xml-1.1.15/priv/lib/fxml_stream.so Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/8d/56b1d43d23899556c668377ee64129a6bc611f Missing separate debuginfo for /home/q/ejabberd16.09.1903/lib/stringprep-1.0.6/priv/lib/stringprep.so Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/58/3a0dd70b94291dfdcc8072ae0a3d0c0c5d49d2 Missing separate debuginfo for /home/q/ejabberd16.09.1903/lib/ezlib-1.0.1/priv/lib/ezlib_drv.so Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/bb/101cc9d2f28f536b249896e08eb342ed38a5c2 Missing separate debuginfo for Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/71/a96b79419fe40ff1706d59ed06d6dbe37c8e97 Reading symbols from /lib64/libutil.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libutil.so.1 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libm.so.6 Reading symbols from /lib64/libtinfo.so.5...(no debugging symbols found)...done. Loaded symbols for /lib64/libtinfo.so.5 Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /home/q/erlang1903/lib/erlang/lib/crypto-3.7.3/priv/lib/crypto.so...done. Loaded symbols for /home/q/erlang1903/lib/erlang/lib/crypto-3.7.3/priv/lib/crypto.so Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libcrypto.so.10 Reading symbols from //lib64/libz.so.1...(no debugging symbols found)...done. Loaded symbols for //lib64/libz.so.1 Reading symbols from /home/q/erlang1903/lib/erlang/lib/crypto-3.7.3/priv/lib/crypto_callback.so...done. Loaded symbols for /home/q/erlang1903/lib/erlang/lib/crypto-3.7.3/priv/lib/crypto_callback.so Reading symbols from /home/q/ejabberd16.09.1903/lib/fast_yaml-1.0.6/priv/lib/fast_yaml.so...done. Loaded symbols for /home/q/ejabberd16.09.1903/lib/fast_yaml-1.0.6/priv/lib/fast_yaml.so Reading symbols from /lib64/libyaml-0.so.2...done. Loaded symbols for /lib64/libyaml-0.so.2 Reading symbols from /home/q/ejabberd16.09.1903/lib/fast_tls-1.0.7/priv/lib/p1_sha.so...done. Loaded symbols for /home/q/ejabberd16.09.1903/lib/fast_tls-1.0.7/priv/lib/p1_sha.so Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libssl.so.10 Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libgssapi_krb5.so.2 Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols found)...done. Loaded symbols for /lib64/libkrb5.so.3 Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libcom_err.so.2 Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols found)...done. Loaded symbols for /lib64/libk5crypto.so.3 Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libkrb5support.so.0 Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libkeyutils.so.1 Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /lib64/libselinux.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libselinux.so.1 Reading symbols from /home/q/ejabberd16.09.1903/lib/fast_tls-1.0.7/priv/lib/fast_tls_drv.so...done. Loaded symbols for /home/q/ejabberd16.09.1903/lib/fast_tls-1.0.7/priv/lib/fast_tls_drv.so Reading symbols from /home/q/ejabberd16.09.1903/lib/fast_xml-1.1.15/priv/lib/fxml.so...done. Loaded symbols for /home/q/ejabberd16.09.1903/lib/fast_xml-1.1.15/priv/lib/fxml.so Reading symbols from /lib64/libexpat.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libexpat.so.1 Reading symbols from /home/q/ejabberd16.09.1903/lib/fast_xml-1.1.15/priv/lib/fxml_stream.so...done. Loaded symbols for /home/q/ejabberd16.09.1903/lib/fast_xml-1.1.15/priv/lib/fxml_stream.so Reading symbols from /home/q/ejabberd16.09.1903/lib/stringprep-1.0.6/priv/lib/stringprep.so...done. Loaded symbols for /home/q/ejabberd16.09.1903/lib/stringprep-1.0.6/priv/lib/stringprep.so Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libstdc++.so.6 Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libgcc_s.so.1 Reading symbols from /home/q/ejabberd16.09.1903/lib/ezlib-1.0.1/priv/lib/ezlib_drv.so...done. Loaded symbols for /home/q/ejabberd16.09.1903/lib/ezlib-1.0.1/priv/lib/ezlib_drv.so Core was generated by `/home/q/erlang1903/lib/erlang/erts-8.3/bin/beam.smp -K true -P 1000000 -- -root'. Program terminated with signal 11, Segmentation fault. #0 0x0000003e4f2766cb in ?? () from /usr/lib64/libcrypto.so.10 Missing separate debuginfos, use: debuginfo-install expat-2.0.1-11.el6_2.x86_64 glibc-2.12-1.149.el6_6.5.x86_64 keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-33.el6.x86_64 libcom_err-1.41.12-21.el6.x86_64 libgcc-4.4.7-11.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64 libstdc++-4.4.7-11.el6.x86_64 ncurses-libs-5.7-3.20090208.el6.x86_64 openssl-1.0.1e-30.el6_6.5.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) source /home/q/lffan.liu/otp_src_19.3/erts/etc/unix/etp-commands.in %--------------------------------------------------------------------------- % Use etp-help for a command overview and general help. % % To use the Erlang support module, the environment variable ROOTDIR % must be set to the toplevel installation directory of Erlang/OTP, % so the etp-commands file becomes: % $ROOTDIR/erts/etc/unix/etp-commands % Also, erl and erlc must be in the path. %--------------------------------------------------------------------------- etp-set-max-depth 20 etp-set-max-string-length 100 --------------- System Information --------------- OTP release: 19 ERTS version: 8.3 Compile date: Sat Sep 30 10:12:09 2017 Arch: x86_64-unknown-linux-gnu Endianness: Little Word size: 64-bit HiPE support: yes SMP support: yes Thread support: yes Kernel poll: Supported and used Debug compiled: no Lock checking: no Lock counting: no Node name: 'ejabberd@REDACTED' Number of schedulers: 24 Number of async-threads: 10 -------------------------------------------------- (gdb) bt #0 0x0000003e4f2766cb in ?? () from /usr/lib64/libcrypto.so.10 #1 0xca62c1d6ca62c1d6 in ?? () #2 0xca62c1d6ca62c1d6 in ?? () #3 0xca62c1d6ca62c1d6 in ?? () #4 0xca62c1d6ca62c1d6 in ?? () #5 0xca62c1d6ca62c1d6 in ?? () #6 0xca62c1d6ca62c1d6 in ?? () #7 0xca62c1d6ca62c1d6 in ?? () #8 0xca62c1d6ca62c1d6 in ?? () #9 0x0000003e4f5df7fc in ?? () from /usr/lib64/libcrypto.so.10 #10 0x00007f78ab21d848 in ?? () #11 0x0000000000000023 in ?? () #12 0x0000003e4f272d67 in SHA1_Update () from /usr/lib64/libcrypto.so.10 #13 0x0000003e4f2e6374 in ?? () from /usr/lib64/libcrypto.so.10 #14 0x0000003e4f2aac7a in ?? () from /usr/lib64/libcrypto.so.10 #15 0x0000003e4f2aab18 in ?? () from /usr/lib64/libcrypto.so.10 #16 0x00007f78f6fba8ec in rand_uniform_nif (env=0x7f793b5fdd80, argc=, argv=0x7f796bf48200) at crypto.c:2120 #17 0x000000000043d907 in process_main () at beam/beam_emu.c:3575 #18 0x00000000004e0cf5 in sched_thread_func (vesdp=0x7f7962c60cc0) at beam/erl_process.c:8206 #19 0x000000000062ec02 in thr_wrapper (vtwd=) at pthread/ethread.c:114 #20 0x0000003e4b6079d1 in start_thread () from /lib64/libpthread.so.0 #21 0x0000003e4b2e88fd in clone () from /lib64/libc.so.6 (gdb) f 16 #16 0x00007f78f6fba8ec in rand_uniform_nif (env=0x7f793b5fdd80, argc=, argv=0x7f796bf48200) at crypto.c:2120 2120 BN_pseudo_rand_range(bn_rand, bn_to); (gdb) l 2115 return enif_make_badarg(env); 2116 } 2117 2118 bn_to = BN_new(); 2119 BN_sub(bn_to, bn_rand, bn_from); 2120 BN_pseudo_rand_range(bn_rand, bn_to); 2121 BN_add(bn_rand, bn_rand, bn_from); 2122 dlen = BN_num_bytes(bn_rand); 2123 data = enif_make_new_binary(env, dlen+4, &ret); 2124 put_int32(data, dlen); (gdb) p *bn_from $1 = {d = 0x7f78ab1c5698, top = 0, dmax = 1, neg = 0, flags = 1} (gdb) p *bn_rand $2 = {d = 0x7f78ab21d748, top = 2, dmax = 2, neg = 0, flags = 1} (gdb) p *bn_to $3 = {d = 0x7f78ab21d7c8, top = 2, dmax = 2, neg = 0, flags = 1} (gdb) p *((*bn_to).d) $4 = 0 (gdb) p *((*bn_from).d) $5 = 0 (gdb) p *((*bn_rand).d) $6 = 0 (gdb) ptype bn_to type = struct bignum_st { long unsigned int *d; int top; int dmax; int neg; int flags; } * (gdb) f 17 #17 0x000000000043d907 in process_main () at beam/beam_emu.c:3575 3575 nif_bif_result = (*fp)(&env, bif_nif_arity, reg); (gdb) info local fp = 0x7f78f6fba850 env = {mod_nif = 0x7f796a6c2390, proc = 0x7f79383bfc70, hp = 0x7f788b7ba9b0, hp_end = 0x7f788b7bae30, heap_frag = 0x0, fpe_was_unmasked = 0, tmp_obj_list = 0x0, exception_thrown = 0, tracee = 0x0, exiting = 0} vbf = 0x7f78f6fba850 live_hf_end = 0x0 nif_bif_result = bif_nif_arity = 2 init_done = 1 c_p = 0x7f79383bfc70 reds_used = reg = 0x7f796bf48200 HTOP = 0x0 E = 0x7f788b7bae30 I = FCALLS = 1798 freg = 0x7f796bf4a280 neg_o_reds = 0 opcodes = {0x43d6bb, 0x43c050, 0x43c164, 0x43d70d, 0x43d648, 0x440afa, 0x43d7c5, 0x440a7d, 0x440f57, 0x43d926, 0x43bf32, 0x43b959, 0x44069f, 0x440865, 0x4405ac, 0x4402e0, 0x4400c0, 0x43fed0, 0x43fe9d, 0x43fe71, 0x43ef91, 0x43bb80, 0x43d76b, 0x43d8a6, 0x4407a3, 0x43bdcf, 0x43c0cf, 0x43d987, 0x43f9ab, 0x43d635, 0x43a279, 0x439d0c, 0x43e69b, 0x43f87d, 0x43e710, 0x43e761, 0x43d60d, 0x43d5e4, 0x43d5b3, 0x43d583, 0x43d55b, 0x43d52a, 0x43d4eb, 0x43d4ad, 0x43d46f, 0x43d432, 0x43d3f4, 0x43d3b7, 0x43d37a, 0x43d33e, 0x43f32a, 0x43f136, 0x43f0df, 0x43efbc, 0x43f09f, 0x43f05f, 0x43ad6e, 0x440b71, 0x43fd7d, 0x440a12, 0x43f7f4, 0x43f66a, 0x43f33a, 0x43f44b, 0x43b642, 0x43bdef, 0x43b8b5, 0x4405fa, 0x43faee, 0x43fa74, 0x440eb2, 0x43eb60, 0x43ea58, 0x440d26, 0x43ff96, 0x43ff02, 0x43fdfa, 0x44001a, 0x43e637, 0x440e87, 0x440cff, 0x43ec7d, 0x43e79a, 0x440ce4, 0x440c5c, 0x440c8c, 0x43e89c, 0x440be7, 0x440c30, 0x44089a, 0x440c47, 0x440511, 0x440780, 0x440906, 0x440493, 0x44015c, 0x43fc1d, 0x440e60, 0x43e9e0, 0x43e967, 0x43e930, 0x43e901, 0x44019c, 0x44026d, 0x440104, 0x43ebcd, 0x440060, 0x44027e, 0x43ba40, 0x43baf3, 0x43b82e, 0x439d84, 0x439dfa, 0x439db5, 0x439e37, 0x44099e, 0x43e30b, 0x439d4b, 0x439d25, 0x43f230, 0x43abf3, 0x43ab98, 0x43acb0, 0x43b9e3, 0x43c157, 0x439d0c, 0x43f1dd, 0x43f28a, 0x43f18a, 0x43f2dd, 0x43e8e6, 0x43bd31, 0x43bc59, 0x43b6fd, 0x43f918, 0x440816, 0x4407c3, 0x43d2ca, 0x43d257, 0x43d1e4, 0x43d172, 0x43d113, 0x43d0b5, 0x43d057, 0x43dceb, 0x43ae81, 0x43dbf6, 0x43ee7c, 0x43dbc4, 0x43ee56, 0x43dcb0, 0x43dc6c, 0x43dc29, 0x43f6e4, 0x43a2a2, 0x43a56c, 0x43a55f, 0x43adf3, 0x43f58b, 0x43dba6, 0x43db83, 0x43db61, 0x43a35d, 0x43ad2d, 0x43db42, 0x43db24, 0x43ad22, 0x43acfd, 0x43afb8, 0x43afdd, 0x43b934, 0x43bf25, 0x439e7c, 0x43f498, 0x43edfa, 0x43a3c9, 0x43a383, 0x439d78, 0x439dee, 0x439dad, 0x439e2f, 0x439d43, 0x439d1d, 0x43eea3, 0x43edaf, 0x43ef46, 0x43e593, 0x43fbd0, 0x43e49e, 0x43e544, 0x43ec31, 0x43a548, 0x43a489, 0x43a57a, 0x43db02, 0x43da76, 0x43d946, 0x439e61, 0x43f3c7, 0x43f381...} pt_arity = start_time = 0 start_time_i = 0x0 EBS = 0x7f7962c60cf0 [/home/q/ejabberd16.09.1903]# ls -l /usr/lib64/libcrypto.so.10 lrwxrwxrwx. 1 root root 19 May 8 2015 /usr/lib64/libcrypto.so.10 -> libcrypto.so.1.0.1e [/home/q/ejabberd16.09.1903]# ls -l /usr/lib64/libcrypto.so.1.0.1e -rwxr-xr-x 1 root root 1965856 Jan 21 2015 /usr/lib64/libcrypto.so.1.0.1e ======== crypto.c 2105 static ERL_NIF_TERM rand_uniform_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) 2106 {/* (Lo,Hi) */ 2107 BIGNUM *bn_from = NULL, *bn_to, *bn_rand; 2108 unsigned char* data; 2109 unsigned dlen; 2110 ERL_NIF_TERM ret; 2111 2112 if (!get_bn_from_mpint(env, argv[0], &bn_from) 2113 || !get_bn_from_mpint(env, argv[1], &bn_rand)) { 2114 if (bn_from) BN_free(bn_from); 2115 return enif_make_badarg(env); 2116 } 2117 2118 bn_to = BN_new(); 2119 BN_sub(bn_to, bn_rand, bn_from); 2120 BN_pseudo_rand_range(bn_rand, bn_to); 2121 BN_add(bn_rand, bn_rand, bn_from); 2122 dlen = BN_num_bytes(bn_rand); 2123 data = enif_make_new_binary(env, dlen+4, &ret); 2124 put_int32(data, dlen); 2125 BN_bn2bin(bn_rand, data+4); 2126 ERL_VALGRIND_MAKE_MEM_DEFINED(data+4, dlen); 2127 BN_free(bn_rand); 2128 BN_free(bn_from); 2129 BN_free(bn_to); 2130 return ret; 2131 } ======= Thank you for your time. Monk Boy -------------- next part -------------- An HTML attachment was scrubbed... URL: From sperber@REDACTED Mon Oct 16 10:56:46 2017 From: sperber@REDACTED (Michael Sperber) Date: Mon, 16 Oct 2017 10:56:46 +0200 Subject: [erlang-questions] 2nd Call for Contributions: BOB 2018 - Berlin, Feb 23, 2018 Message-ID: Erlang submissions are very welcome at BOB! ---- BOB Conference 2018 "What happens when we use what's best for a change?" http://bobkonf.de/2018/en/cfp.html Berlin, February 23 Call for Contributions Deadline: October 29, 2017 You are actively engaged in advanced software engineering methods, implement ambitious architectures and are open to cutting-edge innovation? Attend this conference, meet people that share your goals, and get to know the best software tools and technologies available today. We strive to offer a day full of new experiences and impressions that you can use to immediately improve your daily life as a software developer. If you share our vision and want to contribute, submit a proposal for a talk or tutorial! Topics ------ We are looking for talks about best-of-breed software technology, e.g.: - functional programming - persistent data structures and databases - types - formal methods for correctness and robustness - abstractions for concurrency and parallelism - metaprogramming - probabilistic programming - math and programming - controlled side effects - beyond REST and SOAP - effective abstractions for data analytics - ... everything really that isn?t mainstream, but you think should be. Presenters should provide the audience with information that is practically useful for software developers. We?re especially interested in experience reports. But this could also take other forms, e.g.: - introductory talks on technical background - overviews of a given field - demos and how-tos Requirements ------------ We accept proposals for presentations of 45 minutes (40 minutes talk + 5 minutes questions), as well as 90 minute tutorials for beginners. The language of presentation should be either English or German. Your proposal should include (in your presentation language of choice): - an abstract of max. 1500 characters. - a short bio/cv - contact information (including at least email address) - a list of 3-5 concrete ideas of how your work can be applied in a developer?s daily life - additional material (websites, blogs, slides, videos of past presentations, ...) Submit here: https://docs.google.com/forms/d/e/1FAIpQLSdjgwulSMpaITJ6q6cK_ndrfR1FlEs_HQlZy04LnUKC-ArCaQ/viewform?usp=sf_link Organisation - direct questions to contact at bobkonf dot de - proposal deadline: October 29, 2017 - notification: November 13, 2017 - program: December 1, 2017 NOTE: The conference fee will be waived for presenters, but travel expenses will not be covered. Speaker Grants -------------- BOB has Speaker Grants available to support speakers from groups under-represented in technology. We specifically seek women speakers and speakers who are not be able to attend the conference for financial reasons. Details are here: http://bobkonf.de/2018/en/speaker-grants.html Shepherding ----------- The program committee offers shepherding to all speakers. Shepherding provides speakers assistance with preparing their sessions, as well as a review of the talk slides. Program Committee ----------------- (more information here: http://bobkonf.de/2018/en/programmkomitee.html) - Matthias Fischmann, zerobuzz UG - Matthias Neubauer, SICK AG - Nicole Rauch, Softwareentwicklung und Entwicklungscoaching - Michael Sperber, Active Group - Stefan Wehr, factis research Scientific Advisory Board - Annette Bieniusa, TU Kaiserslautern - Torsten Grust, Uni T?bingen - Peter Thiemann, Uni Freiburg From dangud@REDACTED Mon Oct 16 11:42:10 2017 From: dangud@REDACTED (Dan Gudmundsson) Date: Mon, 16 Oct 2017 09:42:10 +0000 Subject: [erlang-questions] puzzled with this charset/encoding -related behaviour In-Reply-To: References: Message-ID: On Sat, Oct 14, 2017 at 6:23 PM Alexandre Karpov wrote: > Thanks everyone! I didn't realize until this conversation how much more > important strings-as-binaries are, compared to simple "strings". Everything > _works_ now, of course, but I don't think my understanding has caught up > 100% > > "by default it guesses that lists containing integers larger than 255 > is not a string but a list of integers" <<< this really set some things > straight > > But suffer me this follow-up question, Dan. Using +pc unicode indeed gave > me a shell that represents lists of integers using the characters found in > Unicode mapping; so now, in error messages, I see arguments reported more > clearly: > > *7> re:run("???.asd", xmerl_regexp:sh_to_awk("*.*"), [{capture, none}]). > * > > *** exception error: bad argument* > > * in function re:run/3* > > * called as re:run("???.asd","^(.*\\..*)$",[{capture,none}])* > > > If I use the binary-string representation, it works _even_without_ /utf8, > it works just fine: > > *3> re:run(<<"???.asd">>, xmerl_regexp:sh_to_awk("*.*"), [{capture, > none}]).* > > *match* > Note that the call above was executed in the shell started _without_ the > +pc unicode, and the binary does _not_ have the /utf8>> thingy... This > means my understanding is still lacking... binaries are honest and good, > strings are fake and evil, but +px unicode seems to help a little with fake > string... while using binary-strings doesn't _always_ require the /utf8 ... > what is this sorcery?! > =) > First see the difference here: *4> <<"???.asd">>.* *<<110,26,113,46,97,115,100>>* *i.e. the codepoints are just truncated to below 256* *5> <<"???.asd"/utf8>>.* *<<230,153,174,233,128,154,232,169,177,46,97,115,100>>* *6> * And the codepoints are utf8 encoded. Don't give up on lists, they can be useful and fast for some usages. And your regexp matches anything with a dot in it, so even if the string is handled as utf8 encoded binary or just plain bytes, it still works since in both representation you get a match. To understand unicode, play around and try to make it work (on both lists and binaries) with some fancier regexps try match something with a unicode sign in it and capture the result so you see what you matched. Print your input/result strings with both io:format("~ts: ~w~n",[Str, Str]). So you can see both the actual string and it's representation, test with both binaries and lists as representations. /Dan > > On Sat, Oct 14, 2017 at 4:24 AM, Dan Gudmundsson wrote: > >> re:run("???.asd", xmerl_regexp:sh_to_awk("*.*"), [{capture, none}, >> unicode]). >> The binary one matches since it works on bytes and not utf-8 characters? >> >> Also the erlang shell doesn't know if a list of integers is a list of >> integers or a string, >> since they may be represented by the same list of integers. >> >> So it tries to guess, by default it guesses that lists containing >> integers larger than 255 >> is not a string but a list of integers. You can change that with: >> >> (w)erl +pc unicode >> >> 1> "???.asd". >> "???.asd" >> >> /Dan >> >> >> On Sat, Oct 14, 2017 at 10:12 AM Attila Rajmund Nohl < >> attila.r.nohl@REDACTED> wrote: >> >>> 2017-10-14 4:21 GMT+02:00 Alexandre Karpov : >>> > TL;DR: how do I run erl which understands Unicode? >>> > >>> > Or, in more detail: >>> > >>> > (Disclaimer: this official documentation got me really humbled: >>> > http://www1.erlang.org/doc/apps/stdlib/unicode_usage.html >>> > , and just a little bit scared =) ) >>> > >>> > Judging by my S/O question, which got 3 upvotes and no answers, I'm >>> not the >>> > only one wondering: >>> > >>> https://stackoverflow.com/questions/46735539/erlang-regexp-matching-on-chinese-characters >>> > >>> > Here's the gist of the problem: >>> > >>> > 57> "???". >>> > >>> > [1072,1073,1074] >>> > >>> > The codes are correct Unicode for the [Cyrillic] characters - which >>> means my >>> > Terminal didn't fail to understand my keyboard's input =) but Erlang >>> shell >>> > didn't recognize Terminal's input as printable characters. And it is my >>> > understanding that this is exactly why this call fails: >>> > >>> > 25> re:run("???.asd", xmerl_regexp:sh_to_awk("*.*"), [{capture, >>> none}]). ** >>> > exception error: bad argument in function re:run/3 called as >>> > re:run([1081,1094,1091,46,97,115,100], "^(.*\\..*)$", [{capture,none}]) >>> >>> Try >>> >>> re:run(<<"???.asd"/utf8>>, xmerl_regexp:sh_to_awk("*.*"), [{capture, >>> none}]). >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Mon Oct 16 14:14:13 2017 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 16 Oct 2017 14:14:13 +0200 Subject: [erlang-questions] Erlang 19.3 crashes when getting sockname of stdin Message-ID: Hello! I have a oneliner that prints out which erlang process is listening on a given TCP port. Recently I upgraded from Debian 8.9 to Debian 9.2 and on Erlang/OTP 19.3.1 this oneliner no longer works. The minimal example to reproduce: Erlang/OTP 19 [erts-8.3.1] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] Eshell V8.3.1 (abort with ^G) 1> inet:sockname(self()). ** exception error: no function clause matching prim_inet:sockname(<0.58.0>) 2> inet:sockname(hd(erlang:ports())). Failed to write to erl_child_setup: 9 Crash dump is being written to: erl_crash.dump...done The (relevant?) part of the crash dump: Current Process: <0.60.0> Current Process State: Running Current Process Internal State: ACT_PRIO_NORMAL | USR_PRIO_NORMAL | PRQ_PRIO_NORMAL | ACTIVE | RUNNING | ON_HEAP_MSGQ Current Process Program counter: 0x00007f695af4b1d0 (unknown function) Current Process CP: 0x0000000000000000 (invalid) Current Process Limited Stack Trace: 0x00007f6958e16048:SReturn addr 0x5C473830 (prim_inet:ctl_cmd/3 + 64) 0x00007f6958e16070:SReturn addr 0x5C45F3F8 (prim_inet:sockname/1 + 96) 0x00007f6958e16080:SReturn addr 0x5C291510 (erl_eval:do_apply/6 + 408) 0x00007f6958e16088:SReturn addr 0x580DF1B0 (shell:exprs/7 + 632) 0x00007f6958e160a0:SReturn addr 0x580DE8D0 (shell:eval_exprs/7 + 128) 0x00007f6958e160f8:SReturn addr 0x580DE4C8 (shell:eval_loop/3 + 576) 0x00007f6958e16130:SReturn addr 0x943E78 () Erlang is built from source using kerl. This used to work on Debian 8.9 (as far as I remember, but I may be wrong here) and this also works (at least doesn't crash the VM) on Erlang 18.3: 3> inet:sockname(hd(erlang:ports())). ** exception error: no try clause matching [] in function prim_inet:ctl_cmd/3 in call from prim_inet:sockname/1 Sometimes my terminal hangs when I reproduce this bug. I just checked and I got similar crash on OS X too with Erlang/OTP 19.3. From erlang.org@REDACTED Mon Oct 16 14:31:33 2017 From: erlang.org@REDACTED (Stanislaw Klekot) Date: Mon, 16 Oct 2017 14:31:33 +0200 Subject: [erlang-questions] Erlang 19.3 crashes when getting sockname of stdin In-Reply-To: References: Message-ID: <20171016123133.GA24737@jarowit.net> On Mon, Oct 16, 2017 at 02:14:13PM +0200, Attila Rajmund Nohl wrote: > Hello! > > I have a oneliner that prints out which erlang process is listening on > a given TCP port. So basically the same thing as inet:i/0 does. Granted, it's not a documented function. > Recently I upgraded from Debian 8.9 to Debian 9.2 > and on Erlang/OTP 19.3.1 this oneliner no longer works. The minimal > example to reproduce: > > Erlang/OTP 19 [erts-8.3.1] [source] [64-bit] [smp:2:2] > [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V8.3.1 (abort with ^G) > 1> inet:sockname(self()). > ** exception error: no function clause matching prim_inet:sockname(<0.58.0>) > 2> inet:sockname(hd(erlang:ports())). > Failed to write to erl_child_setup: 9 > > Crash dump is being written to: erl_crash.dump...done inet:sockname() uses erlang:port_control(), which is a private command interface. Each port driver can interpret the command as it sees fit, without any guarantee that the result will be sane when you send a command intended for a totally different driver. In other words, don't do that even in local hacks unless you know inside out the port driver you talk to. If you need to use something that's not officially supported, use inet:i() (shorter and bundled with the distribution) or dig out where the TCP ports come from. -- Stanislaw Klekot From z@REDACTED Mon Oct 16 15:18:06 2017 From: z@REDACTED (Danil Zagoskin) Date: Mon, 16 Oct 2017 16:18:06 +0300 Subject: [erlang-questions] Erlang 19.3 crashes when getting sockname of stdin In-Reply-To: References: Message-ID: Hi! Before calling inet:sockname on a port first make sure it's inet_tcp or inet_udp. For example, [{inet:sockname(P), erlang:port_info(P, connected)} || P <- erlang:ports(), erlang:port_get_data(P) == inet_tcp]. You also can filter by name from erlang:port_info/1: [{inet:sockname(P), erlang:port_info(P, connected)} || P <- erlang:ports(), erlang:port_info(P, name) == {name,"tcp_inet"}]. On Mon, Oct 16, 2017 at 3:14 PM, Attila Rajmund Nohl < attila.r.nohl@REDACTED> wrote: > Hello! > > I have a oneliner that prints out which erlang process is listening on > a given TCP port. Recently I upgraded from Debian 8.9 to Debian 9.2 > and on Erlang/OTP 19.3.1 this oneliner no longer works. The minimal > example to reproduce: > > Erlang/OTP 19 [erts-8.3.1] [source] [64-bit] [smp:2:2] > [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V8.3.1 (abort with ^G) > 1> inet:sockname(self()). > ** exception error: no function clause matching > prim_inet:sockname(<0.58.0>) > 2> inet:sockname(hd(erlang:ports())). > Failed to write to erl_child_setup: 9 > > Crash dump is being written to: erl_crash.dump...done > > The (relevant?) part of the crash dump: > > Current Process: <0.60.0> > Current Process State: Running > Current Process Internal State: ACT_PRIO_NORMAL | USR_PRIO_NORMAL | > PRQ_PRIO_NORMAL | ACTIVE | RUNNING | ON_HEAP_MSGQ > Current Process Program counter: 0x00007f695af4b1d0 (unknown function) > Current Process CP: 0x0000000000000000 (invalid) > Current Process Limited Stack Trace: > 0x00007f6958e16048:SReturn addr 0x5C473830 (prim_inet:ctl_cmd/3 + 64) > 0x00007f6958e16070:SReturn addr 0x5C45F3F8 (prim_inet:sockname/1 + 96) > 0x00007f6958e16080:SReturn addr 0x5C291510 (erl_eval:do_apply/6 + 408) > 0x00007f6958e16088:SReturn addr 0x580DF1B0 (shell:exprs/7 + 632) > 0x00007f6958e160a0:SReturn addr 0x580DE8D0 (shell:eval_exprs/7 + 128) > 0x00007f6958e160f8:SReturn addr 0x580DE4C8 (shell:eval_loop/3 + 576) > 0x00007f6958e16130:SReturn addr 0x943E78 () > > Erlang is built from source using kerl. This used to work on Debian > 8.9 (as far as I remember, but I may be wrong here) and this also > works (at least doesn't crash the VM) on Erlang 18.3: > > 3> inet:sockname(hd(erlang:ports())). > ** exception error: no try clause matching [] > in function prim_inet:ctl_cmd/3 > in call from prim_inet:sockname/1 > > Sometimes my terminal hangs when I reproduce this bug. I just checked > and I got similar crash on OS X too with Erlang/OTP 19.3. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmartin4242@REDACTED Mon Oct 16 16:09:27 2017 From: mmartin4242@REDACTED (Michael L Martin) Date: Mon, 16 Oct 2017 10:09:27 -0400 Subject: [erlang-questions] Socket option FREEBIND Message-ID: <865fda37-c29c-89b6-a8bb-aedd2d19f344@gmail.com> Hi all, Is there a way to specify the FREEBIND option when opening a socket? I don't see any reference to it in the documentation. Thanks, From devangana@REDACTED Mon Oct 16 16:35:00 2017 From: devangana@REDACTED (Devangana Tarafdar) Date: Mon, 16 Oct 2017 09:35:00 -0500 Subject: [erlang-questions] Changing lager console backend log level In-Reply-To: References: Message-ID: That worked, thank you ! -Devangana On Sun, Oct 15, 2017 at 12:52 PM, Devangana Tarafdar wrote: > Thank you for that suggestion, I will try that. > > On Oct 14, 2017 2:15 PM, "Chris Waymire" wrote: > >> Just taking a quick look it does not appear that lager_console_backend >> supports that argument style in 3.4.0 >> >> https://github.com/erlang-lager/lager/blob/3.4.0/src/lager_ >> console_backend.erl >> >> Try {lager_console_backend, [info]} >> >> On Fri, Oct 13, 2017 at 7:09 AM, Devangana Tarafdar >> wrote: >> >>> Hello, >>> >>> I am having an issue with lager (3.4.0) and am wondering if anyone can >>> point out if I am doing something wrong: >>> >>> Basically I want to change the log level for the console backend from >>> the app config file. >>> >>> I am starting lager with the following inserted in the config file: >>> >>> {lager,[ >>> {error_logger_whitelist,[yaws_log_file_h]}, >>> {handlers, [ >>> {lager_file_backend, [{file,"/tmp/lager_test.info"} >>> ,{level,debug}]}, >>> >>> {lager_console_backend, [{level,info}]} >>> ]}, >>> >>> {log_root, "/tmp"} >>> >>> ]}, >>> >>> Is the above syntax for lager_console_backend correct ? Lager seems to >>> be having some problem with the console backend handler. I see this in the >>> log file: >>> >>> >>> 2017-10-12 15:34:33.405 [error] <0.60.0>@lager_handler_watcher:123 >>> Lager fatally failed to install handler lager_console_backend into >>> lager_event, NOT retrying: bad_log_level >>> >>> 2017-10-12 15:34:33.405 [debug] <0.62.0>@lager_handler_watcher:119 >>> Lager installed handler error_logger_lager_h into error_logger >>> 2017-10-12 15:34:33.435 [debug] <0.47.0> Supervisor gr_param_sup started >>> gr_param:start_link(gr_lager_default_tracer_params) at pid <0.64.0> >>> 2017-10-12 15:34:33.435 [debug] <0.46.0> Supervisor gr_counter_sup >>> started gr_counter:start_link(gr_lager_default_tracer_counters) at pid >>> <0.65.0> >>> 2017-10-12 15:34:33.439 [debug] <0.48.0> Supervisor gr_manager_sup >>> started gr_manager:start_link(gr_lager_default_tracer_params_mgr, >>> gr_lager_default_tracer_params, []) at pid <0.66.0> >>> 2017-10-12 15:34:33.439 [debug] <0.48.0> Supervisor gr_manager_sup >>> started gr_manager:start_link(gr_lager_default_tracer_counters_mgr, >>> gr_lager_default_tracer_counters, [{input,0},{filter,0},{output, >>> 0},{job_input,0},{job_run,0},{job_time,0},{job_error,0}]) at pid >>> <0.67.0> >>> 2017-10-12 15:34:33.633 [info] <0.7.0> Application lager started on node >>> 'd@REDACTED ' >>> 2017-10-12 15:34:33.888 [debug] <0.56.0>@lager_handler_watcher:119 >>> Lager installed handler lager_backend_throttle into lager_event >>> >>> I am starting up my tiny test environment as follows: >>> >>> [dtaraf@REDACTED lager_test_rel-1.0.0]$ erl -name d -setcookie x >>> -config /usr/pkg/otp/rel/lager_test_rel-1.0.0/lager_test_rel.config -pz >>> /usr/pkg/otp/lib/lager-3.4.0/ebin -pz /usr/pkg/otp/lib/goldrush-0.1. >>> 9/ebin >>> >>> *Erlang/OTP 17* [erts-6.3] [source] [64-bit] [async-threads:10] >>> [kernel-poll:false] >>> >>> Eshell V6.3 (abort with ^G) >>> >>> (d@REDACTED)1> application:start(compiler),ap >>> plication:start(syntax_tools),application:start(goldrush),ap >>> plication:start(lager). >>> ok >>> (d@REDACTED)2> lager:status(). >>> Lager status: >>> File /tmp/lager_test.info (lager_event) at level debug >>> Console (lager_event) at level {error,bad_module} >>> Active Traces: >>> Tracing Reductions: >>> Tracing Statistics: >>> input: 0 >>> output: 0 >>> filter: 0 >>> ok >>> >>> (d@REDACTED)3> application:which_applications(). >>> [*{lager,"Erlang logging framework","3.4.0*"}, >>> {goldrush,"Erlang event stream processor","0.1.9"}, >>> {syntax_tools,"Syntax tools","1.6.17"}, >>> {compiler,"ERTS CXC 138 10","5.0.3"}, >>> {stdlib,"ERTS CXC 138 10","2.3"}, >>> {kernel,"ERTS CXC 138 10","3.1"}] >>> (d@REDACTED)4> >>> >>> >>> Thanks ! >>> >>> Devangana >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dinushaganganath@REDACTED Mon Oct 16 17:39:34 2017 From: dinushaganganath@REDACTED (Dinusha Ganganath Perera) Date: Mon, 16 Oct 2017 21:24:34 +0545 Subject: [erlang-questions] Multiple Diameter Connection issue Message-ID: Hi All, I tried to establish two diameter connection to same diameter peer and port (used two different source port). One connection was established successfully and other connection was failed. The failed connection sent below tcp packets to diameter peer, but did not sent CER to diameter node. 1. sent [SYN] to diameter node 2. received [SYN, ACK] from diameter node 3. sent [ACK] to diameter node 4. sent [FIN, ACK] to diameter node 5. received [FIN, ACK] from diameter node If I down established connection, then failed one is successfully established. I used below application services, {application, [{alias, smsc_diameter_cli_dia_1}, {dictionary, rfc4006_cc_Gy}, {module, smsc_diameter_clii_cb}, {answer_errors, callback}, {sequence, {0,19}}, {share_peers, true}, {use_shared_peers, true}, {restrict_connections, false}]} I used diameter-1.3 erlang library. Please help me to solve this issue. Thanks & BRs, Dinusha -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Mon Oct 16 17:51:06 2017 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 16 Oct 2017 17:51:06 +0200 Subject: [erlang-questions] Erlang 19.3 crashes when getting sockname of stdin In-Reply-To: References: Message-ID: Hello, On Mon, Oct 16, 2017 at 3:18 PM, Danil Zagoskin wrote: > > Before calling inet:sockname on a port first make sure it's inet_tcp or > inet_udp. > For example, > [{inet:sockname(P), erlang:port_info(P, connected)} || P <- > erlang:ports(), erlang:port_get_data(P) == inet_tcp]. > > You also can filter by name from erlang:port_info/1: > [{inet:sockname(P), erlang:port_info(P, connected)} || P <- > erlang:ports(), erlang:port_info(P, name) == {name,"tcp_inet"}]. > > Note that even this approach is not future proof as it assumes that tcp connection are ports. Something which we are planning to change in the future. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Mon Oct 16 19:18:46 2017 From: g@REDACTED (Guilherme Andrade) Date: Mon, 16 Oct 2017 18:18:46 +0100 Subject: [erlang-questions] Socket option FREEBIND In-Reply-To: <865fda37-c29c-89b6-a8bb-aedd2d19f344@gmail.com> References: <865fda37-c29c-89b6-a8bb-aedd2d19f344@gmail.com> Message-ID: I believe you can use raw socket options[1] for that, but it won't be portable. [1]: http://erlang.org/doc/man/inet.html#setopts-2 - {raw, Protocol, OptionNum, ValueBin} On 16 October 2017 at 15:09, Michael L Martin wrote: > Hi all, > > Is there a way to specify the FREEBIND option when opening a socket? I > don't see any reference to it in the documentation. > > > Thanks, > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Guilherme -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Mon Oct 16 20:48:20 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 16 Oct 2017 18:48:20 +0000 Subject: [erlang-questions] Cowboy2: disable HTTP2 Message-ID: Hi guys, Does anyone know how to tell Cowboy2 to disable HTTP2 (+ also forbid to upgrade connection)? Thanks /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Oct 16 21:32:19 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 16 Oct 2017 20:32:19 +0100 Subject: [erlang-questions] Cowboy2: disable HTTP2 In-Reply-To: References: Message-ID: <5eeff762-2910-f8ab-9f19-e3d5750bbd1d@ninenines.eu> Copy the cowboy:start_tls function and don't include the h2 protocol in the list. I don't think there's an option to forbid upgrades yet. On 10/16/2017 07:48 PM, Frank Muller wrote: > Hi guys, > > Does anyone know how to tell Cowboy2 to disable HTTP2 (+ also forbid to > upgrade connection)? > > Thanks > /Frank > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From frank.muller.erl@REDACTED Mon Oct 16 21:56:58 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 16 Oct 2017 19:56:58 +0000 Subject: [erlang-questions] Cowboy2: disable HTTP2 In-Reply-To: <5eeff762-2910-f8ab-9f19-e3d5750bbd1d@ninenines.eu> References: <5eeff762-2910-f8ab-9f19-e3d5750bbd1d@ninenines.eu> Message-ID: Thank you. Working now. Copy the cowboy:start_tls function and don't include the h2 protocol in > the list. > > I don't think there's an option to forbid upgrades yet. > > On 10/16/2017 07:48 PM, Frank Muller wrote: > > Hi guys, > > > > Does anyone know how to tell Cowboy2 to disable HTTP2 (+ also forbid to > > upgrade connection)? > > > > Thanks > > /Frank > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Lo?c Hoguin > https://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Oct 17 01:38:21 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 17 Oct 2017 12:38:21 +1300 Subject: [erlang-questions] Erlang 19.3 crashes when getting sockname of stdin In-Reply-To: References: Message-ID: On 17/10/17 4:51 AM, Lukas Larsson wrote: > Note that even this approach is not future proof as it assumes that tcp > connection are ports. Something which we are planning to change in the > future. Would you care to say more about this? From aschultz@REDACTED Tue Oct 17 08:58:11 2017 From: aschultz@REDACTED (Andreas Schultz) Date: Tue, 17 Oct 2017 08:58:11 +0200 (CEST) Subject: [erlang-questions] Socket option FREEBIND In-Reply-To: References: <865fda37-c29c-89b6-a8bb-aedd2d19f344@gmail.com> Message-ID: <513007423.3057554.1508223491955.JavaMail.zimbra@tpip.net> ----- On Oct 16, 2017, at 7:18 PM, Guilherme Andrade wrote: > I believe you can use raw socket options[1] for that, but it won't be portable. IP_FREEBIND is Linux only. FreeBSD has IP(V6)_BINDANY, which is similar but not identical. Windows apparently has nothing alike. Don't know about the other OS's. Andreas > [1]: [ http://erlang.org/doc/man/inet.html#setopts-2 | > http://erlang.org/doc/man/inet.html#setopts-2 ] - {raw, Protocol, OptionNum, > ValueBin} > On 16 October 2017 at 15:09, Michael L Martin < [ mailto:mmartin4242@REDACTED | > mmartin4242@REDACTED ] > wrote: >> Hi all, >> Is there a way to specify the FREEBIND option when opening a socket? I don't see >> any reference to it in the documentation. >> Thanks, >> _______________________________________________ >> erlang-questions mailing list >> [ mailto:erlang-questions@REDACTED | erlang-questions@REDACTED ] >> [ http://erlang.org/mailman/listinfo/erlang-questions | >> http://erlang.org/mailman/listinfo/erlang-questions ] > -- > Guilherme > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lawrence.ansell@REDACTED Tue Oct 17 12:28:48 2017 From: lawrence.ansell@REDACTED (Lawrence Ansell) Date: Tue, 17 Oct 2017 11:28:48 +0100 Subject: [erlang-questions] [ANN] Code Mesh Early Bird Ends Today Message-ID: Lots of Erlang talks and workshops by Joe Armstrong and Robert Virding in London. Code Mesh the conference for alternative and functional programming Early Bird ends today. Details here: http://www.codemesh.io/#register -- Lawrence Ansell Conference Marketing Co-ordinator Code Mesh - London - 7-9 November EEFL India - Bangalore - 17 November Lambda Days - Krak?w - 22-23 February 2018 Code BEAM SF - San Francisco - 15-16 March 2018 ElixirConf EU - Warsaw - 16-18 April 2018 Code BEAM STO - Stockholm - 31 May-1 June 2018 Elixir.LDN - London - 16 August 2018 -------------- next part -------------- An HTML attachment was scrubbed... URL: From anders.gs.svensson@REDACTED Tue Oct 17 12:30:15 2017 From: anders.gs.svensson@REDACTED (anders.gs.svensson@REDACTED) Date: Tue, 17 Oct 2017 12:30:15 +0200 Subject: [erlang-questions] Multiple Diameter Connection issue In-Reply-To: References: Message-ID: <23013.56247.494252.724529@gargle.gargle.HOWL> diameter 1.3 is from OTP R15B03, released in November 2012, which is the release that added the restrict_connections option, which looks to be what you're having problems with. I don't recall any issues (not that I recall much of anything from 2012), and don't see anything in the release notes, but I suggest trying with a more recent release for starters. The latest is diameter 2.1.1 in OTP 20.1.2. (Although there's a fix to a problem introduced in 20.1 coming in 20.1.3, unrelated to restrict_connections.) Anders erlang-questions-request@REDACTED writes: > Date: Mon, 16 Oct 2017 21:24:34 +0545 > From: Dinusha Ganganath Perera > To: erlang-questions@REDACTED > Subject: [erlang-questions] Multiple Diameter Connection issue > Message-ID: > > Content-Type: text/plain; charset="utf-8" > > Hi All, > > I tried to establish two diameter connection to same diameter peer and port > (used two different source port). > One connection was established successfully and other connection was > failed. > The failed connection sent below tcp packets to diameter peer, but did not > sent CER to diameter node. > > 1. sent [SYN] to diameter node > 2. received [SYN, ACK] from diameter node > 3. sent [ACK] to diameter node > 4. sent [FIN, ACK] to diameter node > 5. received [FIN, ACK] from diameter node > > If I down established connection, then failed one is successfully > established. > > I used below application services, > > {application, > [{alias, smsc_diameter_cli_dia_1}, > {dictionary, rfc4006_cc_Gy}, > {module, smsc_diameter_clii_cb}, > {answer_errors, callback}, > {sequence, {0,19}}, > {share_peers, true}, > {use_shared_peers, true}, > {restrict_connections, false}]} > > > I used diameter-1.3 erlang library. > > > Please help me to solve this issue. > > > Thanks & BRs, > Dinusha From lukas@REDACTED Tue Oct 17 13:19:53 2017 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 17 Oct 2017 13:19:53 +0200 Subject: [erlang-questions] Erlang 19.3 crashes when getting sockname of stdin In-Reply-To: References: Message-ID: On Tue, Oct 17, 2017 at 1:38 AM, Richard A. O'Keefe wrote: > > > On 17/10/17 4:51 AM, Lukas Larsson wrote: > > Note that even this approach is not future proof as it assumes that tcp >> connection are ports. Something which we are planning to change in the >> future. >> > Would you care to say more about this? > > We are planning to rewrite a lot of functionality in erts that today uses linked-in driver to use nifs instead. We have already re-written the zlib driver, we are in the process of re-writing the efile driver and are planning to rewrite the inet driver in the future. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Tue Oct 17 13:29:47 2017 From: zxq9@REDACTED (zxq9) Date: Tue, 17 Oct 2017 20:29:47 +0900 Subject: [erlang-questions] Erlang 19.3 crashes when getting sockname of stdin In-Reply-To: References: Message-ID: <4587501.rhlNAjQ3aP@changa> On 2017?10?17? ??? 13:19:53 Lukas Larsson wrote: > > We are planning to rewrite a lot of functionality in erts that today uses > linked-in driver to use nifs instead. We have already re-written the zlib > driver, we are in the process of re-writing the efile driver and are > planning to rewrite the inet driver in the future. What is the rationale? -Craig From Andras.Bekes@REDACTED Tue Oct 17 16:00:41 2017 From: Andras.Bekes@REDACTED (Bekes, Andras G) Date: Tue, 17 Oct 2017 14:00:41 +0000 Subject: [erlang-questions] Where are the binaries in message queue but not part of process_info(Pid, binary) ? Message-ID: Hi All, While investigating a system that transfers lots of refc binaries, I tried measuring the amount of data in the message queue of a process. I noticed that the binaries in the message queue are not always included in the result of process_info(Pid, binary). In particular, when observing the binaries, I see time intervals when each of the new messages appear in the binary list, and time intervals when the binary list stays the same regardless of new messages in the queue. I made a simple module that demonstrates the phenomenon. The output looks like: Msgs sent: 0 Msgs in queue: 0 Binaries of receiver: 0 Msgs sent: 1 Msgs in queue: 1 Binaries of receiver: 1 Msgs sent: 2 Msgs in queue: 2 Binaries of receiver: 2 [...goes well up to 36...] Msgs sent: 36 Msgs in queue: 36 Binaries of receiver: 36 [then binary count stays 36, for thousands of messages, and suddenly starts counting again ! ] Looks like if there are some magic limits at 36 (sometimes 33), 256, 4781, 12518 and 20255 messages/binaries and so far I was unable to find any logic in which limit takes effect. I am aware that this feature of process_info is not meant for production use ("can be changed or removed without prior notice."), but this misleading behavior is maybe worse than not having it. May I kindly ask for an explanation of what I observe? Thanks! Andras G. Bekes ---------------------------------------------------------------------------------- My test module: -module(msgq_len_bin_test). -export([test_sleeper/0,test_receiver/0]). test_sleeper()-> io:format("Testing sleeper...\n",[]), PID = spawn_link(fun sleeper/0), sender(PID,0,10000,x). % no reason to test further, binary count never goes above 36 test_receiver()-> io:format("Testing receiver...\n",[]), PID = spawn_link(fun receiver/0), sender(PID,0,100000,x). % worth testing big numbers sleeper()->timer:sleep(100000000). receiver()-> timer:sleep(10), receive _ -> ok end, receiver(). sender(_,N,N,_)-> io:format("Sent ~p messages.",[N]); sender(PID,N,Max,LastBinLen)-> [{message_queue_len,MsgQLen}, {binary,BinaryInfoList}] = erlang:process_info(PID,[message_queue_len,binary]), BinLen = length(BinaryInfoList), case BinLen of LastBinLen -> io:format(".",[]); _ -> io:format("Msgs sent: ~p Msgs in queue: ~p Binaries of receiver: ~p \n",[N,MsgQLen,BinLen]) end, RefcBinary = binary:copy(<>,1000), PID ! RefcBinary, sender(PID,N+1,Max,BinLen). ________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Tue Oct 17 16:31:37 2017 From: garazdawi@REDACTED (Lukas Larsson) Date: Tue, 17 Oct 2017 16:31:37 +0200 Subject: [erlang-questions] Erlang 19.3 crashes when getting sockname of stdin In-Reply-To: <4587501.rhlNAjQ3aP@changa> References: <4587501.rhlNAjQ3aP@changa> Message-ID: On Tue, Oct 17, 2017 at 1:29 PM, zxq9 wrote: > On 2017?10?17? ??? 13:19:53 Lukas Larsson wrote: > > > > We are planning to rewrite a lot of functionality in erts that today uses > > linked-in driver to use nifs instead. We have already re-written the zlib > > driver, we are in the process of re-writing the efile driver and are > > planning to rewrite the inet driver in the future. > > What is the rationale? > It varies depending on driver. In general we want to be able to write more code in Erlang and less in C than we are able to with linked-in drivers. In zlib we wanted to improve how we handle large inputs, i.e. yield back to schedulers etc. For efile we want to start using dirty schedulers for file IO, something that is not possible with linked-in drivers. For inet the main reasons are maintainability and performance. We want to be able to add and extend functionality more easily. We also think that it should be possible to write a faster implementation using nifs because sometimes we can remove the context switch between process and port that is mandatory today. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Tue Oct 17 16:37:02 2017 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 17 Oct 2017 16:37:02 +0200 Subject: [erlang-questions] Where are the binaries in message queue but not part of process_info(Pid, binary) ? In-Reply-To: References: Message-ID: Hello, On Tue, Oct 17, 2017 at 4:00 PM, Bekes, Andras G < Andras.Bekes@REDACTED> wrote: > While investigating a system that transfers lots of refc binaries, I tried > measuring the amount of data in the message queue of a process. I noticed > that the binaries in the message queue are not always included in the > result of process_info(Pid, binary). > > In particular, when observing the binaries, I see time intervals when each > of the new messages appear in the binary list, and time intervals when the > binary list stays the same regardless of new messages in the queue. > erlang:process_info(Pid, binary) returns the binaries that are currently on the heap of the process. Binaries in messages that have not yet been received may or may not be part of the heap depending on internal emulator optimizations. The message_queue_data process flag explains this a little more and also gives a way to control it to some extent. http://erlang.org/doc/man/erlang.html#process_flag_message_queue_data Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From nbartley@REDACTED Tue Oct 17 16:47:26 2017 From: nbartley@REDACTED (nato) Date: Tue, 17 Oct 2017 07:47:26 -0700 Subject: [erlang-questions] BIFs use with care Message-ID: I recently saw an ESL video advising that one should use Erlang BIFs with care... it was a bit of an aside, so I was wondering if someone could could elaborate on this. It seemed to suggest that BIFs (the ones written in C?) could block/cause problems ... After many books, this is the first I've heard of it and would love a rundown on things to watch out for regarding BIFs. From mikpelinux@REDACTED Tue Oct 17 19:42:48 2017 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Tue, 17 Oct 2017 19:42:48 +0200 Subject: [erlang-questions] BIFs use with care In-Reply-To: References: Message-ID: On Tue, Oct 17, 2017 at 4:47 PM, nato wrote: > I recently saw an ESL video advising that one should > use Erlang BIFs with care... it was a bit of an aside, so > I was wondering if someone could could elaborate > on this. It seemed to suggest that BIFs (the ones written > in C?) could block/cause problems ... > Presumably they were referring to preemption. BEAM code is preemptive by design, meaning that the VM can suspend a process and schedule another at almost any instant. (I'm simplifying, but that's the essence of it.) BIFs, being C code inside the VM, are not preemptive by default: they have to be written in a special and potentially slower way to make them preemptive. This means that for large inputs (e.g. long lists or binaries, or deep terms) a broken BIF may block its scheduler until it's finished. That's potentially very bad, depending on your application. Most such bugs should be fixed by now (they were more common 5+ years ago), but it's possible some remain. I wouldn't avoid BIFs unless I knew for a fact that the VM I was using had such a bug in it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Andras.Bekes@REDACTED Tue Oct 17 23:38:33 2017 From: Andras.Bekes@REDACTED (Bekes, Andras G) Date: Tue, 17 Oct 2017 21:38:33 +0000 Subject: [erlang-questions] Where are the binaries in message queue but not part of process_info(Pid, binary) ? In-Reply-To: References: Message-ID: Hi Lukas, Thank you very much for the quick and detailed reply. This explains it. Now I have a few further questions. - It seems the reliable way to measure the amount of data in the message queue is setting message_queue_data process flag to off_heap beforehand (ideally at spawn time with spawn_opt), and retrieve the full message queue content with process_info(Pid,messages) ? which must be fully scanned. Is there a better solution? (I originally aimed at refc binaries only because I just wanted a quick estimation, and in my case anything else was negligible.) - May I suggest to improve the documentation of erlang:process_info(Pid, binary) by the below facts, also referring to message_queue_data process flag? - Is it possible to implement a new process_info_item() of erlang:process_info/2 that gives information about binaries currently in the message queue but off-heap? - The documentation suggests using off_heap message_queue_data ?if the process potentially can get many messages?, but this is rather vague. Does this refer to processes expected to have many *queued* messages? I guess a process receiving but not queuing many messages performs better with on_heap (?Performance of the actual message passing is however generally better when not using flag off_heap.?). Thanks, Andras From: Lukas Larsson [mailto:lukas@REDACTED] Sent: Tuesday, October 17, 2017 4:37 PM To: Bekes, Andras G (IST) Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Where are the binaries in message queue but not part of process_info(Pid, binary) ? Hello, On Tue, Oct 17, 2017 at 4:00 PM, Bekes, Andras G > wrote: While investigating a system that transfers lots of refc binaries, I tried measuring the amount of data in the message queue of a process. I noticed that the binaries in the message queue are not always included in the result of process_info(Pid, binary). In particular, when observing the binaries, I see time intervals when each of the new messages appear in the binary list, and time intervals when the binary list stays the same regardless of new messages in the queue. erlang:process_info(Pid, binary) returns the binaries that are currently on the heap of the process. Binaries in messages that have not yet been received may or may not be part of the heap depending on internal emulator optimizations. The message_queue_data process flag explains this a little more and also gives a way to control it to some extent. http://erlang.org/doc/man/erlang.html#process_flag_message_queue_data Lukas ________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Wed Oct 18 17:52:19 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 18 Oct 2017 17:52:19 +0200 Subject: [erlang-questions] monitor dirty schedulers? Message-ID: <2D770F73-CD06-4119-91A5-AF3EB18C2D0A@gmail.com> How can I monitor the usage of dirty schedulers? I would like to monitor the number of threads used, time for a dirty nif call etc. Should i monitor it in erlang or is there already some internal instrumentation I could use? Benoit From silver.surfertab@REDACTED Wed Oct 18 20:34:35 2017 From: silver.surfertab@REDACTED (Greg) Date: Thu, 19 Oct 2017 00:04:35 +0530 Subject: [erlang-questions] Netconf Support Message-ID: Does Erlang/OTP has Netconf client and server implementation? Google brings me to http://erlang.org/doc/man/ct_netconfc.html; what is it? The other project that I found seems to be not maintained https://github.com/FlowForwarding/enetconf Is there anyone who is using Netconf with Erlang and can help provide some starting point. Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Oct 19 08:26:23 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 19 Oct 2017 09:26:23 +0300 Subject: [erlang-questions] monitor dirty schedulers? In-Reply-To: <2D770F73-CD06-4119-91A5-AF3EB18C2D0A@gmail.com> References: <2D770F73-CD06-4119-91A5-AF3EB18C2D0A@gmail.com> Message-ID: Start from this: http://erlang.org/doc/man/erlang.html#statistics_scheduler_wall_time_all it will give you some numbers of aggregated scheduler load. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dinushaganganath@REDACTED Thu Oct 19 14:11:17 2017 From: dinushaganganath@REDACTED (Dinusha Ganganath Perera) Date: Thu, 19 Oct 2017 17:56:17 +0545 Subject: [erlang-questions] Multiple Diameter Connection issue In-Reply-To: <23016.36845.467485.638510@gargle.gargle.HOWL> References: <23013.56247.494252.724529@gargle.gargle.HOWL> <23016.27917.574496.812778@gargle.gargle.HOWL> <23016.36845.467485.638510@gargle.gargle.HOWL> Message-ID: Dear Anders, Please find the trace. Thanks & BRs, Dinusha On Thu, Oct 19, 2017 at 5:28 PM, wrote: > Hi Dinusha. > > Enable the following trace while your nodes are trying to connect and > it will narrow it down. > > dbg:tracer(). > dbg:p(all,[c,timestamp]). > dbg:tp(diameter_peer_fsm, start, x). > dbg:tp(diameter_peer_fsm, terminate, x). > > Send me the resulting output in the Erlang logs. You can disable the > trace with dbg:stop_clear(). > > Anders > > > > Dinusha Ganganath Perera writes: > > Dear Anders, > > > > My Application is client. > > > > I am trying to establish multiple connections with remote Diameter > Server. > > > > > > > > Thanks & BRs, > > Dinusha > > > > > > > > > > > > On Thu, Oct 19, 2017 at 2:59 PM, > wrote: > > > > Hi Dinusha. > > > > Which of the two nodes in your test is running OTP/diameter: client, > > server, or both? > > > > Anders > > > > > > > > Dinusha Ganganath Perera writes: > > > Dear Anders, > > > > > > Thanks for your reply. > > > > > > I upgraded Erlang and current version of diameter library is 2.1. > > > > > > But issue is not resolved. > > > > > > > > > Thanks & BRs, > > > Dinusha > > > > > > On Tue, Oct 17, 2017 at 4:15 PM, > wrote: > > > > > > diameter 1.3 is from OTP R15B03, released in November 2012, > which is > > > the release that added the restrict_connections option, which > looks > > to > > > be what you're having problems with. I don't recall any issues > (not > > > that I recall much of anything from 2012), and don't see > anything in > > > the release notes, but I suggest trying with a more recent > release > > for > > > starters. > > > > > > The latest is diameter 2.1.1 in OTP 20.1.2. (Although there's > a fix > > to > > > a problem introduced in 20.1 coming in 20.1.3, unrelated to > > > restrict_connections.) > > > > > > Anders > > > > > > > > > erlang-questions-request@REDACTED writes: > > > > Date: Mon, 16 Oct 2017 21:24:34 +0545 > > > > From: Dinusha Ganganath Perera > > > > To: erlang-questions@REDACTED > > > > Subject: [erlang-questions] Multiple Diameter Connection > issue > > > > Message-ID: > > > > > mail.gmail.com > > > > > > > > Content-Type: text/plain; charset="utf-8" > > > > > > > > Hi All, > > > > > > > > I tried to establish two diameter connection to same > diameter peer > > and > > > port > > > > (used two different source port). > > > > One connection was established successfully and other > connection > > was > > > > failed. > > > > The failed connection sent below tcp packets to diameter > peer, but > > did > > > not > > > > sent CER to diameter node. > > > > > > > > 1. sent [SYN] to diameter node > > > > 2. received [SYN, ACK] from diameter node > > > > 3. sent [ACK] to diameter node > > > > 4. sent [FIN, ACK] to diameter node > > > > 5. received [FIN, ACK] from diameter node > > > > > > > > If I down established connection, then failed one is > successfully > > > > established. > > > > > > > > I used below application services, > > > > > > > > {application, > > > > [{alias, smsc_diameter_cli_dia_1}, > > > > {dictionary, rfc4006_cc_Gy}, > > > > {module, smsc_diameter_clii_cb}, > > > > {answer_errors, callback}, > > > > {sequence, {0,19}}, > > > > {share_peers, true}, > > > > {use_shared_peers, true}, > > > > {restrict_connections, false}]} > > > > > > > > > > > > I used diameter-1.3 erlang library. > > > > > > > > > > > > Please help me to solve this issue. > > > > > > > > > > > > Thanks & BRs, > > > > Dinusha > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Charging2_2017-10-19_17-46-48.log Type: application/octet-stream Size: 122160 bytes Desc: not available URL: From dinushaganganath@REDACTED Thu Oct 19 15:17:59 2017 From: dinushaganganath@REDACTED (Dinusha Ganganath Perera) Date: Thu, 19 Oct 2017 19:02:59 +0545 Subject: [erlang-questions] Multiple Diameter Connection issue In-Reply-To: <23016.41297.145526.989025@gargle.gargle.HOWL> References: <23013.56247.494252.724529@gargle.gargle.HOWL> <23016.27917.574496.812778@gargle.gargle.HOWL> <23016.36845.467485.638510@gargle.gargle.HOWL> <23016.41297.145526.989025@gargle.gargle.HOWL> Message-ID: Dear Anders, Yes I override default value and set passed restrict_connections = false. To enable multiple connections. Thanks & BRs, Dinusha On Thu, Oct 19, 2017 at 6:42 PM, wrote: > What are you passing to diameter:add_transport/2? I can see from the > trace that it has restrict_connection = nodes, so it looks like you're > overriding the service configuration. > > Anders > > > > > Dinusha Ganganath Perera writes: > > Dear Anders, > > > > Please find the trace. > > > > Thanks & BRs, > > Dinusha > > > > On Thu, Oct 19, 2017 at 5:28 PM, > wrote: > > > > Hi Dinusha. > > > > Enable the following trace while your nodes are trying to connect and > > it will narrow it down. > > > > dbg:tracer(). > > dbg:p(all,[c,timestamp]). > > dbg:tp(diameter_peer_fsm, start, x). > > dbg:tp(diameter_peer_fsm, terminate, x). > > > > Send me the resulting output in the Erlang logs. You can disable the > > trace with dbg:stop_clear(). > > > > Anders > > > > > > > > Dinusha Ganganath Perera writes: > > > Dear Anders, > > > > > > My Application is client. > > > > > > I am trying to establish multiple connections with remote Diameter > > Server. > > > > > > > > > > > > Thanks & BRs, > > > Dinusha > > > > > > > > > > > > > > > > > > On Thu, Oct 19, 2017 at 2:59 PM, > wrote: > > > > > > Hi Dinusha. > > > > > > Which of the two nodes in your test is running OTP/diameter: > client, > > > server, or both? > > > > > > Anders > > > > > > > > > > > > Dinusha Ganganath Perera writes: > > > > Dear Anders, > > > > > > > > Thanks for your reply. > > > > > > > > I upgraded Erlang and current version of diameter library is > 2.1. > > > > > > > > But issue is not resolved. > > > > > > > > > > > > Thanks & BRs, > > > > Dinusha > > > > > > > > On Tue, Oct 17, 2017 at 4:15 PM, < > anders.gs.svensson@REDACTED> > > wrote: > > > > > > > > diameter 1.3 is from OTP R15B03, released in November > 2012, > > which is > > > > the release that added the restrict_connections option, > which > > looks > > > to > > > > be what you're having problems with. I don't recall any > issues > > (not > > > > that I recall much of anything from 2012), and don't see > > anything in > > > > the release notes, but I suggest trying with a more > recent > > release > > > for > > > > starters. > > > > > > > > The latest is diameter 2.1.1 in OTP 20.1.2. (Although > there's a > > fix > > > to > > > > a problem introduced in 20.1 coming in 20.1.3, unrelated > to > > > > restrict_connections.) > > > > > > > > Anders > > > > > > > > > > > > erlang-questions-request@REDACTED writes: > > > > > Date: Mon, 16 Oct 2017 21:24:34 +0545 > > > > > From: Dinusha Ganganath Perera < > dinushaganganath@REDACTED> > > > > > To: erlang-questions@REDACTED > > > > > Subject: [erlang-questions] Multiple Diameter > Connection > > issue > > > > > Message-ID: > > > > > kGDiwECMM1GECmL1uqTPyG6xRw@ > > > mail.gmail.com > > > > > > > > > > Content-Type: text/plain; charset="utf-8" > > > > > > > > > > Hi All, > > > > > > > > > > I tried to establish two diameter connection to same > diameter > > peer > > > and > > > > port > > > > > (used two different source port). > > > > > One connection was established successfully and other > > connection > > > was > > > > > failed. > > > > > The failed connection sent below tcp packets to > diameter > > peer, but > > > did > > > > not > > > > > sent CER to diameter node. > > > > > > > > > > 1. sent [SYN] to diameter node > > > > > 2. received [SYN, ACK] from diameter node > > > > > 3. sent [ACK] to diameter node > > > > > 4. sent [FIN, ACK] to diameter node > > > > > 5. received [FIN, ACK] from diameter node > > > > > > > > > > If I down established connection, then failed one is > > successfully > > > > > established. > > > > > > > > > > I used below application services, > > > > > > > > > > {application, > > > > > [{alias, smsc_diameter_cli_dia_1}, > > > > > {dictionary, rfc4006_cc_Gy}, > > > > > {module, smsc_diameter_clii_cb}, > > > > > {answer_errors, callback}, > > > > > {sequence, {0,19}}, > > > > > {share_peers, true}, > > > > > {use_shared_peers, true}, > > > > > {restrict_connections, false}]} > > > > > > > > > > > > > > > I used diameter-1.3 erlang library. > > > > > > > > > > > > > > > Please help me to solve this issue. > > > > > > > > > > > > > > > Thanks & BRs, > > > > > Dinusha > > > > > > > > > > > > > > > > > > > > x[DELETED ATTACHMENT Charging2_2017-10-19_17-46-48.log, Untyped binary > data] > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dinushaganganath@REDACTED Thu Oct 19 15:25:58 2017 From: dinushaganganath@REDACTED (Dinusha Ganganath Perera) Date: Thu, 19 Oct 2017 19:10:58 +0545 Subject: [erlang-questions] Multiple Diameter Connection issue In-Reply-To: References: <23013.56247.494252.724529@gargle.gargle.HOWL> <23016.27917.574496.812778@gargle.gargle.HOWL> <23016.36845.467485.638510@gargle.gargle.HOWL> <23016.41297.145526.989025@gargle.gargle.HOWL> Message-ID: Dear Anders, I made a mistake. Actually I use restrict_connections when starting service only. I will use it in add_transport and check. Thanks & BRs, Dinusha On Thu, Oct 19, 2017 at 7:02 PM, Dinusha Ganganath Perera < dinushaganganath@REDACTED> wrote: > Dear Anders, > > Yes I override default value and set passed restrict_connections = false. > > To enable multiple connections. > > > Thanks & BRs, > Dinusha > > > > > > On Thu, Oct 19, 2017 at 6:42 PM, wrote: > >> What are you passing to diameter:add_transport/2? I can see from the >> trace that it has restrict_connection = nodes, so it looks like you're >> overriding the service configuration. >> >> Anders >> >> >> >> >> Dinusha Ganganath Perera writes: >> > Dear Anders, >> > >> > Please find the trace. >> > >> > Thanks & BRs, >> > Dinusha >> > >> > On Thu, Oct 19, 2017 at 5:28 PM, >> wrote: >> > >> > Hi Dinusha. >> > >> > Enable the following trace while your nodes are trying to connect >> and >> > it will narrow it down. >> > >> > dbg:tracer(). >> > dbg:p(all,[c,timestamp]). >> > dbg:tp(diameter_peer_fsm, start, x). >> > dbg:tp(diameter_peer_fsm, terminate, x). >> > >> > Send me the resulting output in the Erlang logs. You can disable the >> > trace with dbg:stop_clear(). >> > >> > Anders >> > >> > >> > >> > Dinusha Ganganath Perera writes: >> > > Dear Anders, >> > > >> > > My Application is client. >> > > >> > > I am trying to establish multiple connections with remote Diameter >> > Server. >> > > >> > > >> > > >> > > Thanks & BRs, >> > > Dinusha >> > > >> > > >> > > >> > > >> > > >> > > On Thu, Oct 19, 2017 at 2:59 PM, >> wrote: >> > > >> > > Hi Dinusha. >> > > >> > > Which of the two nodes in your test is running OTP/diameter: >> client, >> > > server, or both? >> > > >> > > Anders >> > > >> > > >> > > >> > > Dinusha Ganganath Perera writes: >> > > > Dear Anders, >> > > > >> > > > Thanks for your reply. >> > > > >> > > > I upgraded Erlang and current version of diameter library >> is 2.1. >> > > > >> > > > But issue is not resolved. >> > > > >> > > > >> > > > Thanks & BRs, >> > > > Dinusha >> > > > >> > > > On Tue, Oct 17, 2017 at 4:15 PM, < >> anders.gs.svensson@REDACTED> >> > wrote: >> > > > >> > > > diameter 1.3 is from OTP R15B03, released in November >> 2012, >> > which is >> > > > the release that added the restrict_connections option, >> which >> > looks >> > > to >> > > > be what you're having problems with. I don't recall any >> issues >> > (not >> > > > that I recall much of anything from 2012), and don't see >> > anything in >> > > > the release notes, but I suggest trying with a more >> recent >> > release >> > > for >> > > > starters. >> > > > >> > > > The latest is diameter 2.1.1 in OTP 20.1.2. (Although >> there's a >> > fix >> > > to >> > > > a problem introduced in 20.1 coming in 20.1.3, >> unrelated to >> > > > restrict_connections.) >> > > > >> > > > Anders >> > > > >> > > > >> > > > erlang-questions-request@REDACTED writes: >> > > > > Date: Mon, 16 Oct 2017 21:24:34 +0545 >> > > > > From: Dinusha Ganganath Perera < >> dinushaganganath@REDACTED> >> > > > > To: erlang-questions@REDACTED >> > > > > Subject: [erlang-questions] Multiple Diameter >> Connection >> > issue >> > > > > Message-ID: >> > > > > > iwECMM1GECmL1uqTPyG6xRw@ >> > > mail.gmail.com >> > > > > >> > > > > Content-Type: text/plain; charset="utf-8" >> > > > > >> > > > > Hi All, >> > > > > >> > > > > I tried to establish two diameter connection to same >> diameter >> > peer >> > > and >> > > > port >> > > > > (used two different source port). >> > > > > One connection was established successfully and other >> > connection >> > > was >> > > > > failed. >> > > > > The failed connection sent below tcp packets to >> diameter >> > peer, but >> > > did >> > > > not >> > > > > sent CER to diameter node. >> > > > > >> > > > > 1. sent [SYN] to diameter node >> > > > > 2. received [SYN, ACK] from diameter node >> > > > > 3. sent [ACK] to diameter node >> > > > > 4. sent [FIN, ACK] to diameter node >> > > > > 5. received [FIN, ACK] from diameter node >> > > > > >> > > > > If I down established connection, then failed one is >> > successfully >> > > > > established. >> > > > > >> > > > > I used below application services, >> > > > > >> > > > > {application, >> > > > > [{alias, smsc_diameter_cli_dia_1}, >> > > > > {dictionary, rfc4006_cc_Gy}, >> > > > > {module, smsc_diameter_clii_cb}, >> > > > > {answer_errors, callback}, >> > > > > {sequence, {0,19}}, >> > > > > {share_peers, true}, >> > > > > {use_shared_peers, true}, >> > > > > {restrict_connections, false}]} >> > > > > >> > > > > >> > > > > I used diameter-1.3 erlang library. >> > > > > >> > > > > >> > > > > Please help me to solve this issue. >> > > > > >> > > > > >> > > > > Thanks & BRs, >> > > > > Dinusha >> > > > >> > > > >> > > >> > > >> > >> > >> > x[DELETED ATTACHMENT Charging2_2017-10-19_17-46-48.log, Untyped binary >> data] >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dinushaganganath@REDACTED Thu Oct 19 16:26:26 2017 From: dinushaganganath@REDACTED (Dinusha Ganganath Perera) Date: Thu, 19 Oct 2017 20:11:26 +0545 Subject: [erlang-questions] Multiple Diameter Connection issue In-Reply-To: References: <23013.56247.494252.724529@gargle.gargle.HOWL> <23016.27917.574496.812778@gargle.gargle.HOWL> <23016.36845.467485.638510@gargle.gargle.HOWL> <23016.41297.145526.989025@gargle.gargle.HOWL> Message-ID: Dear Anders, After adding {restrict_connections, false} to add_transport, we can establish multiple connections. Thanks for your support. Thanks & BRs, Dinusha On Thu, Oct 19, 2017 at 7:31 PM, Dinusha Ganganath Perera < dinushaganganath@REDACTED> wrote: > Dear Anders, > > I added below parameters to add_transport and please find the trace. > > {pool_size, 2}, > {share_peers, true}, > {use_shared_peers,true}, > {restrict_connections, false}, > > Thanks & BRs, > Dinusha > > On Thu, Oct 19, 2017 at 7:10 PM, Dinusha Ganganath Perera < > dinushaganganath@REDACTED> wrote: > >> Dear Anders, >> >> I made a mistake. >> >> Actually I use restrict_connections when starting service only. >> >> I will use it in add_transport and check. >> >> Thanks & BRs, >> Dinusha >> >> On Thu, Oct 19, 2017 at 7:02 PM, Dinusha Ganganath Perera < >> dinushaganganath@REDACTED> wrote: >> >>> Dear Anders, >>> >>> Yes I override default value and set passed restrict_connections = false. >>> >>> To enable multiple connections. >>> >>> >>> Thanks & BRs, >>> Dinusha >>> >>> >>> >>> >>> >>> On Thu, Oct 19, 2017 at 6:42 PM, >>> wrote: >>> >>>> What are you passing to diameter:add_transport/2? I can see from the >>>> trace that it has restrict_connection = nodes, so it looks like you're >>>> overriding the service configuration. >>>> >>>> Anders >>>> >>>> >>>> >>>> >>>> Dinusha Ganganath Perera writes: >>>> > Dear Anders, >>>> > >>>> > Please find the trace. >>>> > >>>> > Thanks & BRs, >>>> > Dinusha >>>> > >>>> > On Thu, Oct 19, 2017 at 5:28 PM, >>>> wrote: >>>> > >>>> > Hi Dinusha. >>>> > >>>> > Enable the following trace while your nodes are trying to connect >>>> and >>>> > it will narrow it down. >>>> > >>>> > dbg:tracer(). >>>> > dbg:p(all,[c,timestamp]). >>>> > dbg:tp(diameter_peer_fsm, start, x). >>>> > dbg:tp(diameter_peer_fsm, terminate, x). >>>> > >>>> > Send me the resulting output in the Erlang logs. You can disable >>>> the >>>> > trace with dbg:stop_clear(). >>>> > >>>> > Anders >>>> > >>>> > >>>> > >>>> > Dinusha Ganganath Perera writes: >>>> > > Dear Anders, >>>> > > >>>> > > My Application is client. >>>> > > >>>> > > I am trying to establish multiple connections with remote >>>> Diameter >>>> > Server. >>>> > > >>>> > > >>>> > > >>>> > > Thanks & BRs, >>>> > > Dinusha >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > On Thu, Oct 19, 2017 at 2:59 PM, >>> om> wrote: >>>> > > >>>> > > Hi Dinusha. >>>> > > >>>> > > Which of the two nodes in your test is running >>>> OTP/diameter: client, >>>> > > server, or both? >>>> > > >>>> > > Anders >>>> > > >>>> > > >>>> > > >>>> > > Dinusha Ganganath Perera writes: >>>> > > > Dear Anders, >>>> > > > >>>> > > > Thanks for your reply. >>>> > > > >>>> > > > I upgraded Erlang and current version of diameter library >>>> is 2.1. >>>> > > > >>>> > > > But issue is not resolved. >>>> > > > >>>> > > > >>>> > > > Thanks & BRs, >>>> > > > Dinusha >>>> > > > >>>> > > > On Tue, Oct 17, 2017 at 4:15 PM, < >>>> anders.gs.svensson@REDACTED> >>>> > wrote: >>>> > > > >>>> > > > diameter 1.3 is from OTP R15B03, released in November >>>> 2012, >>>> > which is >>>> > > > the release that added the restrict_connections >>>> option, which >>>> > looks >>>> > > to >>>> > > > be what you're having problems with. I don't recall >>>> any issues >>>> > (not >>>> > > > that I recall much of anything from 2012), and don't >>>> see >>>> > anything in >>>> > > > the release notes, but I suggest trying with a more >>>> recent >>>> > release >>>> > > for >>>> > > > starters. >>>> > > > >>>> > > > The latest is diameter 2.1.1 in OTP 20.1.2. (Although >>>> there's a >>>> > fix >>>> > > to >>>> > > > a problem introduced in 20.1 coming in 20.1.3, >>>> unrelated to >>>> > > > restrict_connections.) >>>> > > > >>>> > > > Anders >>>> > > > >>>> > > > >>>> > > > erlang-questions-request@REDACTED writes: >>>> > > > > Date: Mon, 16 Oct 2017 21:24:34 +0545 >>>> > > > > From: Dinusha Ganganath Perera < >>>> dinushaganganath@REDACTED> >>>> > > > > To: erlang-questions@REDACTED >>>> > > > > Subject: [erlang-questions] Multiple Diameter >>>> Connection >>>> > issue >>>> > > > > Message-ID: >>>> > > > > >>> iwECMM1GECmL1uqTPyG6xRw@ >>>> > > mail.gmail.com >>>> > > > > >>>> > > > > Content-Type: text/plain; charset="utf-8" >>>> > > > > >>>> > > > > Hi All, >>>> > > > > >>>> > > > > I tried to establish two diameter connection to >>>> same diameter >>>> > peer >>>> > > and >>>> > > > port >>>> > > > > (used two different source port). >>>> > > > > One connection was established successfully and >>>> other >>>> > connection >>>> > > was >>>> > > > > failed. >>>> > > > > The failed connection sent below tcp packets to >>>> diameter >>>> > peer, but >>>> > > did >>>> > > > not >>>> > > > > sent CER to diameter node. >>>> > > > > >>>> > > > > 1. sent [SYN] to diameter node >>>> > > > > 2. received [SYN, ACK] from diameter node >>>> > > > > 3. sent [ACK] to diameter node >>>> > > > > 4. sent [FIN, ACK] to diameter node >>>> > > > > 5. received [FIN, ACK] from diameter node >>>> > > > > >>>> > > > > If I down established connection, then failed one is >>>> > successfully >>>> > > > > established. >>>> > > > > >>>> > > > > I used below application services, >>>> > > > > >>>> > > > > {application, >>>> > > > > [{alias, smsc_diameter_cli_dia_1}, >>>> > > > > {dictionary, rfc4006_cc_Gy}, >>>> > > > > {module, smsc_diameter_clii_cb}, >>>> > > > > {answer_errors, callback}, >>>> > > > > {sequence, {0,19}}, >>>> > > > > {share_peers, true}, >>>> > > > > {use_shared_peers, true}, >>>> > > > > {restrict_connections, false}]} >>>> > > > > >>>> > > > > >>>> > > > > I used diameter-1.3 erlang library. >>>> > > > > >>>> > > > > >>>> > > > > Please help me to solve this issue. >>>> > > > > >>>> > > > > >>>> > > > > Thanks & BRs, >>>> > > > > Dinusha >>>> > > > >>>> > > > >>>> > > >>>> > > >>>> > >>>> > >>>> > x[DELETED ATTACHMENT Charging2_2017-10-19_17-46-48.log, Untyped >>>> binary data] >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nistrigunya@REDACTED Fri Oct 20 14:00:47 2017 From: nistrigunya@REDACTED (Avinash Dhumane) Date: Fri, 20 Oct 2017 17:30:47 +0530 Subject: [erlang-questions] What does "soft" real-time mean? Message-ID: I have programmed high-frequency trading application in Erlang and comparing its reaction performance in the live market with other competing applications (mostly, in C). It has been a consistent observation that my Erlang application lags substantially behind the competing applications. In fact, it never matched the reaction time of the so-called "slowest" C application in the market. The benchmark reaction time is about 5 microseconds. It is the time elapsed between tick to order. That is, the difference between the timestamps when I receive the tick (market event) from the mailbox of my process and when I submit the TCP send call on my order. My process maintains the market depths of the securities and when my desired price (as a function of best buyers and/or sellers on multiple legs of the order) is observed, I compute the order terms and submit. The entire computation and communication (with the market) takes place within the single process and there are no collaborating processes (hence, no inter-process messaging; not even ETS). Still, the reaction time is in the order of 20 to 60 microseconds - that is, nowhere closer to the benchmark reaction time (5 microseconds). What I wish to hear from the experienced people on this forum here is whether I am missing on something in my Erlang code, or should I just go ahead and program in C. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Aleksander.Nycz@REDACTED Fri Oct 20 15:25:15 2017 From: Aleksander.Nycz@REDACTED (Aleksander Nycz) Date: Fri, 20 Oct 2017 15:25:15 +0200 Subject: [erlang-questions] What does "soft" real-time mean? In-Reply-To: References: Message-ID: <92b20510-42e4-09ad-4a03-26362553a941@comarch.pl> Hello, My private definition: Hard real time -> 100% requests/jobs/processes must be completed before deadline Soft real time -> N% requests/jobs/processes must be completed before deadline, where N < 100%, ??? so we can/must accept that some jobs will not be finished on time. And few question: 1. Erlang VM is running on Virtual Machine on bare metal? 2. What you erlang process really do? Can you show us some code? Regards Aleksander Nycz W dniu 2017-10-20 o?14:00, Avinash Dhumane pisze: > I have programmed high-frequency trading application in Erlang and > comparing its reaction performance in the live market with other > competing applications (mostly, in C). > > It has been a consistent observation that my Erlang application lags > substantially behind the competing applications. In fact, it never > matched the reaction time of the so-called "slowest" C application in > the market. > > The benchmark reaction time is about 5 microseconds. It is the time > elapsed between tick to order. That is, the difference between the > timestamps when I receive the tick (market event) from the mailbox of > my process and when I submit the TCP send call on my order. > > My process maintains the market depths of the securities and when my > desired price (as a function of best buyers and/or sellers on multiple > legs of the order) is observed, I compute the order terms and submit. > > The entire computation and communication (with the market) takes place > within the single process and there are no collaborating processes > (hence, no inter-process messaging; not even ETS). Still, the reaction > time is in the order of 20 to 60 microseconds - that is, nowhere > closer to the benchmark reaction time (5 microseconds). > > What I wish to hear from the experienced people on this forum here is > whether I am missing on something in my Erlang code, or should I just > go ahead and program in C. > > Thanks. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Aleksander Nycz Chief Designer Telco_021 BSS R&D Comarch SA Phone: +48 17 785 5909 Mobile: +48 691 464 275 website: www.comarch.pl -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4936 bytes Desc: Kryptograficzna sygnatura S/MIME URL: From sergej.jurecko@REDACTED Fri Oct 20 15:46:27 2017 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Fri, 20 Oct 2017 15:46:27 +0200 Subject: [erlang-questions] What does "soft" real-time mean? In-Reply-To: References: Message-ID: <46AF6606-0D5A-4DCB-A0B4-9C2BD1219ABD@gmail.com> Erlang is the right tool for the job for a majority of server workloads. However when your requirements are a reaction time in microseconds, it most surely is not the right tool. I would use Rust in that scenario if you?re picking languages. Regards, Sergej > On 20 Oct 2017, at 14:00, Avinash Dhumane wrote: > > I have programmed high-frequency trading application in Erlang and comparing its reaction performance in the live market with other competing applications (mostly, in C). > > It has been a consistent observation that my Erlang application lags substantially behind the competing applications. In fact, it never matched the reaction time of the so-called "slowest" C application in the market. > > The benchmark reaction time is about 5 microseconds. It is the time elapsed between tick to order. That is, the difference between the timestamps when I receive the tick (market event) from the mailbox of my process and when I submit the TCP send call on my order. > > My process maintains the market depths of the securities and when my desired price (as a function of best buyers and/or sellers on multiple legs of the order) is observed, I compute the order terms and submit. > > The entire computation and communication (with the market) takes place within the single process and there are no collaborating processes (hence, no inter-process messaging; not even ETS). Still, the reaction time is in the order of 20 to 60 microseconds - that is, nowhere closer to the benchmark reaction time (5 microseconds). > > What I wish to hear from the experienced people on this forum here is whether I am missing on something in my Erlang code, or should I just go ahead and program in C. > > Thanks. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From hugo@REDACTED Fri Oct 20 15:47:38 2017 From: hugo@REDACTED (Hugo Mills) Date: Fri, 20 Oct 2017 13:47:38 +0000 Subject: [erlang-questions] What does "soft" real-time mean? In-Reply-To: References: Message-ID: <20171020134738.GB27985@carfax.org.uk> On Fri, Oct 20, 2017 at 05:30:47PM +0530, Avinash Dhumane wrote: > I have programmed high-frequency trading application in Erlang and > comparing its reaction performance in the live market with other competing > applications (mostly, in C). > > It has been a consistent observation that my Erlang application lags > substantially behind the competing applications. In fact, it never matched > the reaction time of the so-called "slowest" C application in the market. > > The benchmark reaction time is about 5 microseconds. It is the time elapsed > between tick to order. That is, the difference between the timestamps when > I receive the tick (market event) from the mailbox of my process and when I > submit the TCP send call on my order. You're combining two measures here: - How fast does the process react once the message arrives in its mailbox? Not the time to getting a response out, but the delay between the message getting to the process, and the process being scheduled to actually start work on processing it. - Once the process starts work on it, how fast can it get the result out? Erlang's reputation is that's it's very good at the former measure, even when there are many thousands of processes competing to be scheduled. On the second measure, that's more about the efficiency of executing the code once it starts. Depending on what you're doing in the process, it will almost certainly be slower to do the actual work than equivalent code in C, possibly _much_ slower. Note, however, that sometimes you can end up writing very poor (inefficient) Erlang by following an efficient C implementation too closely, and that a different implementation which takes account of the functional nature of Erlang will run much faster. So the first thing I'd look at here would be benchmarking the service code first (standalone, in a tight loop, without the message passing part), to see how fast that actually runs. That should then at least give you a good idea of which of the two measures above is the slow one. Hugo. > My process maintains the market depths of the securities and when my > desired price (as a function of best buyers and/or sellers on multiple legs > of the order) is observed, I compute the order terms and submit. > > The entire computation and communication (with the market) takes place > within the single process and there are no collaborating processes (hence, > no inter-process messaging; not even ETS). Still, the reaction time is in > the order of 20 to 60 microseconds - that is, nowhere closer to the > benchmark reaction time (5 microseconds). > > What I wish to hear from the experienced people on this forum here is > whether I am missing on something in my Erlang code, or should I just go > ahead and program in C. > > Thanks. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hugo Mills | Great films about cricket: The Fantastic Four hugo@REDACTED carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From santif@REDACTED Fri Oct 20 15:48:13 2017 From: santif@REDACTED (Santiago Fernandez) Date: Fri, 20 Oct 2017 10:48:13 -0300 Subject: [erlang-questions] What does "soft" real-time mean? In-Reply-To: <46AF6606-0D5A-4DCB-A0B4-9C2BD1219ABD@gmail.com> References: <46AF6606-0D5A-4DCB-A0B4-9C2BD1219ABD@gmail.com> Message-ID: In general Erlang is not the best option to do CPU intensive computations, especially with a single process. I use Elixir/Erlang in financial apps to route incoming messages to websocket clients, process concurrent orders, etc. But, in my own benchmarks, it takes 5-10 microseconds only to parse a FIX incoming message (same process, no inter process comunication). https://github.com/santif/ex_fix#benchmarks I think if you need Erlang for concurrency/scalability *and* you need to do some kind of HFT, so the way to go is by using a NIF or linked in driver (C, Rust?) If you only need to do HFT, I think there is no reason to stick with Erlang, since there is no competition with the slowest C app. HTH -- Santiago On Fri, Oct 20, 2017 at 10:46 AM, Sergej Jure?ko wrote: > Erlang is the right tool for the job for a majority of server workloads. > However when your requirements are a reaction time in microseconds, it most > surely is not the right tool. > > I would use Rust in that scenario if you?re picking languages. > > > Regards, > Sergej > > > On 20 Oct 2017, at 14:00, Avinash Dhumane wrote: > > > > I have programmed high-frequency trading application in Erlang and > comparing its reaction performance in the live market with other competing > applications (mostly, in C). > > > > It has been a consistent observation that my Erlang application lags > substantially behind the competing applications. In fact, it never matched > the reaction time of the so-called "slowest" C application in the market. > > > > The benchmark reaction time is about 5 microseconds. It is the time > elapsed between tick to order. That is, the difference between the > timestamps when I receive the tick (market event) from the mailbox of my > process and when I submit the TCP send call on my order. > > > > My process maintains the market depths of the securities and when my > desired price (as a function of best buyers and/or sellers on multiple legs > of the order) is observed, I compute the order terms and submit. > > > > The entire computation and communication (with the market) takes place > within the single process and there are no collaborating processes (hence, > no inter-process messaging; not even ETS). Still, the reaction time is in > the order of 20 to 60 microseconds - that is, nowhere closer to the > benchmark reaction time (5 microseconds). > > > > What I wish to hear from the experienced people on this forum here is > whether I am missing on something in my Erlang code, or should I just go > ahead and program in C. > > > > Thanks. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nistrigunya@REDACTED Fri Oct 20 15:58:18 2017 From: nistrigunya@REDACTED (Avinash Dhumane) Date: Fri, 20 Oct 2017 19:28:18 +0530 Subject: [erlang-questions] What does "soft" real-time mean? In-Reply-To: <92b20510-42e4-09ad-4a03-26362553a941@comarch.pl> References: <92b20510-42e4-09ad-4a03-26362553a941@comarch.pl> Message-ID: Hello! The application runs directly on latest Intel Xeon. The source code of specific module in question is emailed to you separately. There is a process per instance of "algorithm trading strategy", but no two processes share anything except the ordering socket to the market, which is accessed through synchronization (ETS). But, during the market test, we run only one or two strategies so there is no synchronization overhead. The major computation is market depth update and price matching. On Fri, Oct 20, 2017 at 6:55 PM, Aleksander Nycz wrote: > Hello, > > > My private definition: > > Hard real time -> 100% requests/jobs/processes must be completed before > deadline > > Soft real time -> N% requests/jobs/processes must be completed before > deadline, where N < 100%, > > so we can/must accept that some jobs will not be finished on time. > > > And few question: > > 1. Erlang VM is running on Virtual Machine on bare metal? > > 2. What you erlang process really do? Can you show us some code? > > > Regards > > Aleksander Nycz > > W dniu 2017-10-20 o 14:00, Avinash Dhumane pisze: > > I have programmed high-frequency trading application in Erlang and > comparing its reaction performance in the live market with other competing > applications (mostly, in C). > > It has been a consistent observation that my Erlang application lags > substantially behind the competing applications. In fact, it never matched > the reaction time of the so-called "slowest" C application in the market. > > The benchmark reaction time is about 5 microseconds. It is the time > elapsed between tick to order. That is, the difference between the > timestamps when I receive the tick (market event) from the mailbox of my > process and when I submit the TCP send call on my order. > > My process maintains the market depths of the securities and when my > desired price (as a function of best buyers and/or sellers on multiple legs > of the order) is observed, I compute the order terms and submit. > > The entire computation and communication (with the market) takes place > within the single process and there are no collaborating processes (hence, > no inter-process messaging; not even ETS). Still, the reaction time is in > the order of 20 to 60 microseconds - that is, nowhere closer to the > benchmark reaction time (5 microseconds). > > What I wish to hear from the experienced people on this forum here is > whether I am missing on something in my Erlang code, or should I just go > ahead and program in C. > > Thanks. > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > -- > Aleksander Nycz > Chief Designer > Telco_021 BSS R&D > Comarch SA > Phone: +48 17 785 5909 > Mobile: +48 691 464 275 > website: www.comarch.pl > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmartin4242@REDACTED Fri Oct 20 17:10:37 2017 From: mmartin4242@REDACTED (Michael L Martin) Date: Fri, 20 Oct 2017 11:10:37 -0400 Subject: [erlang-questions] Socket option FREEBIND In-Reply-To: References: <865fda37-c29c-89b6-a8bb-aedd2d19f344@gmail.com> Message-ID: Raw socket options looks like it should be the answer, but I'm not getting good results: iex(worker@REDACTED)28> {:ok, s} = :gen_tcp.connect('localhost', 80, [{:raw, 6, 15, <<1>>}, {:ip, {10, 138, 69, 63}}]) {:ok, #Port<0.12110>} iex(worker@REDACTED)29> :inet.getopts(s, [{:raw, 6, 15, 32}]) {:ok, []}??? <== option not set? iex(worker@REDACTED)29> {:ok, s} = :gen_tcp.connect('localhost', 80, [{:raw, 6, 15, <<1>>}, {:ip, {10, 138, 69, 64}}]) ** (MatchError) no match of right hand side value: {:error, :eaddrnotavail} It appears that the IP_FREEBIND option is simply not set. If I use IP_TRANSPARENT instead: iex(worker@REDACTED)35> {:ok, s} = :gen_tcp.connect('localhost', 80, [{:raw, 6, 19, <<1>>}, {:ip, {10, 138, 69, 63}}]) {:ok, #Port<0.12113>} iex(worker@REDACTED)34> :inet.getopts(s, [{:raw, 6, 19, 32}]) {:ok, [{:raw, 6, 19, <<0, 0, 0, 0>>}]} In this case, the raw option 19 (IP_TRANSPARENT) is set, but to false rather than true. I believe this is because the beam file needs root or CAP_NET_ADMIN. Any thoughts? On 2017-10-16 01:18 PM, Guilherme Andrade wrote: > I believe you can use raw socket options[1] for that, but it won't be > portable. > > [1]: http://erlang.org/doc/man/inet.html#setopts-2 - {raw, Protocol, > OptionNum, ValueBin} > > On 16 October 2017 at 15:09, Michael L Martin > wrote: > > Hi all, > > Is there a way to specify the FREEBIND option when opening a > socket? I don't see any reference to it in the documentation. > > > Thanks, > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > Guilherme -------------- next part -------------- An HTML attachment was scrubbed... URL: From aschultz@REDACTED Fri Oct 20 18:04:08 2017 From: aschultz@REDACTED (Andreas Schultz) Date: Fri, 20 Oct 2017 18:04:08 +0200 (CEST) Subject: [erlang-questions] Socket option FREEBIND In-Reply-To: References: <865fda37-c29c-89b6-a8bb-aedd2d19f344@gmail.com> Message-ID: <542205910.3060266.1508515448634.JavaMail.zimbra@tpip.net> Hi, ----- On Oct 20, 2017, at 5:10 PM, Michael L Martin wrote: > Raw socket options looks like it should be the answer, but I'm not getting good > results: > iex( [ mailto:worker@REDACTED | worker@REDACTED ] )28> {:ok, s} = > :gen_tcp.connect('localhost', 80, [{:raw, 6, 15, <<1>>}, {:ip, {10, 138, 69, > 63}}]) > {:ok, #Port<0.12110>} > iex( [ mailto:worker@REDACTED | worker@REDACTED ] )29> :inet.getopts(s, > [{:raw, 6, 15, 32}]) > {:ok, []} <== option not set? > iex( [ mailto:worker@REDACTED | worker@REDACTED ] )29> {:ok, s} = > :gen_tcp.connect('localhost', 80, [{:raw, 6, 15, <<1>>}, {:ip, {10, 138, 69, > 64}}]) > ** (MatchError) no match of right hand side value: {:error, :eaddrnotavail} IP_FREEBIND on connect doesn't make sense, it's only really useful on listen sockets. The option lets you bind to an IP address that does not yet exist on the host. That is ok for listen, but when you attempt to build an outgoing connection from that IP, then it has to be configured. > It appears that the IP_FREEBIND option is simply not set. If I use > IP_TRANSPARENT instead: > iex( [ mailto:worker@REDACTED | worker@REDACTED ] )35> {:ok, s} = > :gen_tcp.connect('localhost', 80, [{:raw, 6, 19, <<1>>}, {:ip, {10, 138, 69, > 63}}]) > {:ok, #Port<0.12113>} > iex( [ mailto:worker@REDACTED | worker@REDACTED ] )34> :inet.getopts(s, > [{:raw, 6, 19, 32}]) > {:ok, [{:raw, 6, 19, <<0, 0, 0, 0>>}]} IP_TRANSPARENT needs special iptables rules to work correctly and is for a very specific use case. I don't think it would be very useful for an Erlang application to use that. > In this case, the raw option 19 (IP_TRANSPARENT) is set, but to false rather > than true. I believe this is because the beam file needs root or CAP_NET_ADMIN. You need root and a specific iptables setup for it to work. > Any thoughts? What are you attempting to achieve anyway? Regards Andreas > On 2017-10-16 01:18 PM, Guilherme Andrade wrote: >> I believe you can use raw socket options[1] for that, but it won't be portable. >> [1]: [ http://erlang.org/doc/man/inet.html#setopts-2 | >> http://erlang.org/doc/man/inet.html#setopts-2 ] - {raw, Protocol, OptionNum, >> ValueBin} >> On 16 October 2017 at 15:09, Michael L Martin < [ mailto:mmartin4242@REDACTED | >> mmartin4242@REDACTED ] > wrote: >>> Hi all, >>> Is there a way to specify the FREEBIND option when opening a socket? I don't see >>> any reference to it in the documentation. >>> Thanks, >>> _______________________________________________ >>> erlang-questions mailing list >>> [ mailto:erlang-questions@REDACTED | erlang-questions@REDACTED ] >>> [ http://erlang.org/mailman/listinfo/erlang-questions | >>> http://erlang.org/mailman/listinfo/erlang-questions ] >> -- >> Guilherme > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmartin4242@REDACTED Fri Oct 20 18:17:44 2017 From: mmartin4242@REDACTED (Michael L Martin) Date: Fri, 20 Oct 2017 12:17:44 -0400 Subject: [erlang-questions] Socket option FREEBIND In-Reply-To: <542205910.3060266.1508515448634.JavaMail.zimbra@tpip.net> References: <865fda37-c29c-89b6-a8bb-aedd2d19f344@gmail.com> <542205910.3060266.1508515448634.JavaMail.zimbra@tpip.net> Message-ID: <6e1b6845-a8fb-771a-c5ff-bbb540928b1b@gmail.com> Thanks for the quick response. As for what I'm? attempting to achieve, well, I'm not really sure. I have a customer that is adamant that we use IP_FREEBIND, so I've been poking around in :gen_tcp and :inet trying to see how to do that. Here's what happens when I try to open a listen socket: iex(5)> {:ok, s} = :gen_tcp.listen(8000, [{:raw, 6, 15, <<1 :: size(32)>>}, {:ip, {10, 138, 69, 63}}]) {:ok, #Port<0.1305>} iex(6)> :inet.getopts(s, [{:raw, 6, 15, 4}]) {:ok, []} iex(7)> :gen_tcp.close(s) :ok iex(8)> {:ok, s} = :gen_tcp.listen(8000, [{:raw, 6, 15, <<1 :: size(32)>>}, {:ip, {10, 138, 69, 64}}]) ** (MatchError) no match of right hand side value: {:error, :eaddrnotavail} iex(8)> So it looks like it's not working for listen sockets, either. Could very well be that I simply don't know what the heck I'm doing... On 2017-10-20 12:04 PM, Andreas Schultz wrote: > Hi, > > ----- On Oct 20, 2017, at 5:10 PM, Michael L Martin > wrote: > > Raw socket options looks like it should be the answer, but I'm not > getting good results: > > iex(worker@REDACTED)28> {:ok, s} = > :gen_tcp.connect('localhost', 80, [{:raw, 6, 15, <<1>>}, {:ip, > {10, 138, 69, 63}}]) > > {:ok, #Port<0.12110>} > > iex(worker@REDACTED)29> :inet.getopts(s, [{:raw, 6, 15, 32}]) > > {:ok, []}??? <== option not set? > > iex(worker@REDACTED)29> {:ok, s} = > :gen_tcp.connect('localhost', 80, [{:raw, 6, 15, <<1>>}, {:ip, > {10, 138, 69, 64}}]) > > ** (MatchError) no match of right hand side value: {:error, > :eaddrnotavail} > > > IP_FREEBIND on connect doesn't make sense, it's only really useful on > listen sockets. > > The option lets you bind to an IP address that does not yet exist on > the host. That is ok for listen, but when you attempt to build an > outgoing connection from that IP, then it has to be configured. > > It appears that the IP_FREEBIND option is simply not set. If I use > IP_TRANSPARENT instead: > > iex(worker@REDACTED)35> {:ok, s} = > :gen_tcp.connect('localhost', 80, [{:raw, 6, 19, <<1>>}, {:ip, > {10, 138, 69, 63}}]) > > {:ok, #Port<0.12113>} > > iex(worker@REDACTED)34> :inet.getopts(s, [{:raw, 6, 19, 32}]) > > {:ok, [{:raw, 6, 19, <<0, 0, 0, 0>>}]} > > > IP_TRANSPARENT needs special iptables rules to work correctly and is > for a very specific use case. I don't think it would be very useful > for an Erlang application to use that. > > In this case, the raw option 19 (IP_TRANSPARENT) is set, but to > false rather than true. I believe this is because the beam file > needs root or CAP_NET_ADMIN. > > You need root and a specific iptables setup for it to work. > > Any thoughts? > > > What are you attempting to achieve anyway? > > Regards > Andreas > > > > > On 2017-10-16 01:18 PM, Guilherme Andrade wrote: > > I believe you can use raw socket options[1] for that, but it > won't be portable. > > [1]: http://erlang.org/doc/man/inet.html#setopts-2 - {raw, > Protocol, OptionNum, ValueBin} > > On 16 October 2017 at 15:09, Michael L Martin > > wrote: > > Hi all, > > Is there a way to specify the FREEBIND option when opening > a socket? I don't see any reference to it in the > documentation. > > > Thanks, > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Guilherme > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rj@REDACTED Fri Oct 20 18:33:27 2017 From: rj@REDACTED (Richard Jones) Date: Fri, 20 Oct 2017 17:33:27 +0100 Subject: [erlang-questions] Socket option FREEBIND In-Reply-To: <6e1b6845-a8fb-771a-c5ff-bbb540928b1b@gmail.com> References: <865fda37-c29c-89b6-a8bb-aedd2d19f344@gmail.com> <542205910.3060266.1508515448634.JavaMail.zimbra@tpip.net> <6e1b6845-a8fb-771a-c5ff-bbb540928b1b@gmail.com> Message-ID: I'm able to make outbound connections bound to an arbitrary IP from my range by specifying a socket option like: {raw, 0, 15, <<1>>} note that that protocol part of {raw, ???, _, _} is 0, not 6, since it's an IP level socket option (not tcp-level). btw there's also a system-wide option so you don't need to specify ip_freebind per socket: echo 1 > /proc/sys/net/ipv6/ip_nonlocal_bind RJ On 20 October 2017 at 17:17, Michael L Martin wrote: > Thanks for the quick response. As for what I'm attempting to achieve, well, > I'm not really sure. > > I have a customer that is adamant that we use IP_FREEBIND, so I've been > poking around in :gen_tcp and :inet trying to see how to do that. > > Here's what happens when I try to open a listen socket: > > iex(5)> {:ok, s} = :gen_tcp.listen(8000, [{:raw, 6, 15, <<1 :: size(32)>>}, > {:ip, {10, 138, 69, 63}}]) > {:ok, #Port<0.1305>} > iex(6)> :inet.getopts(s, [{:raw, 6, 15, 4}]) > {:ok, []} > iex(7)> :gen_tcp.close(s) > :ok > iex(8)> {:ok, s} = :gen_tcp.listen(8000, [{:raw, 6, 15, <<1 :: size(32)>>}, > {:ip, {10, 138, 69, 64}}]) > ** (MatchError) no match of right hand side value: {:error, :eaddrnotavail} > > iex(8)> > > So it looks like it's not working for listen sockets, either. Could very > well be that I simply don't know what the heck I'm doing... > > > > On 2017-10-20 12:04 PM, Andreas Schultz wrote: > > Hi, > > ----- On Oct 20, 2017, at 5:10 PM, Michael L Martin > wrote: > > Raw socket options looks like it should be the answer, but I'm not getting > good results: > > iex(worker@REDACTED)28> {:ok, s} = :gen_tcp.connect('localhost', 80, > [{:raw, 6, 15, <<1>>}, {:ip, {10, 138, 69, 63}}]) > > {:ok, #Port<0.12110>} > > iex(worker@REDACTED)29> :inet.getopts(s, [{:raw, 6, 15, 32}]) > > {:ok, []} <== option not set? > > iex(worker@REDACTED)29> {:ok, s} = :gen_tcp.connect('localhost', 80, > [{:raw, 6, 15, <<1>>}, {:ip, {10, 138, 69, 64}}]) > > ** (MatchError) no match of right hand side value: {:error, :eaddrnotavail} > > > IP_FREEBIND on connect doesn't make sense, it's only really useful on listen > sockets. > > The option lets you bind to an IP address that does not yet exist on the > host. That is ok for listen, but when you attempt to build an outgoing > connection from that IP, then it has to be configured. > > It appears that the IP_FREEBIND option is simply not set. If I use > IP_TRANSPARENT instead: > > iex(worker@REDACTED)35> {:ok, s} = :gen_tcp.connect('localhost', 80, > [{:raw, 6, 19, <<1>>}, {:ip, {10, 138, 69, 63}}]) > > {:ok, #Port<0.12113>} > > iex(worker@REDACTED)34> :inet.getopts(s, [{:raw, 6, 19, 32}]) > > {:ok, [{:raw, 6, 19, <<0, 0, 0, 0>>}]} > > > IP_TRANSPARENT needs special iptables rules to work correctly and is for a > very specific use case. I don't think it would be very useful for an Erlang > application to use that. > > In this case, the raw option 19 (IP_TRANSPARENT) is set, but to false rather > than true. I believe this is because the beam file needs root or > CAP_NET_ADMIN. > > You need root and a specific iptables setup for it to work. > > Any thoughts? > > > What are you attempting to achieve anyway? > > Regards > Andreas > > > > > On 2017-10-16 01:18 PM, Guilherme Andrade wrote: > > I believe you can use raw socket options[1] for that, but it won't be > portable. > > [1]: http://erlang.org/doc/man/inet.html#setopts-2 - {raw, Protocol, > OptionNum, ValueBin} > > On 16 October 2017 at 15:09, Michael L Martin wrote: >> >> Hi all, >> >> Is there a way to specify the FREEBIND option when opening a socket? I >> don't see any reference to it in the documentation. >> >> >> Thanks, >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Guilherme > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From rj@REDACTED Fri Oct 20 18:47:25 2017 From: rj@REDACTED (Richard Jones) Date: Fri, 20 Oct 2017 17:47:25 +0100 Subject: [erlang-questions] Socket option FREEBIND In-Reply-To: References: <865fda37-c29c-89b6-a8bb-aedd2d19f344@gmail.com> <542205910.3060266.1508515448634.JavaMail.zimbra@tpip.net> <6e1b6845-a8fb-771a-c5ff-bbb540928b1b@gmail.com> Message-ID: I should add, I do this with the AnyIP routing rule like so: ip route add local 2604:8300:100:200b:6667:8::/96 dev lo (i actually use dummy0 instead of "lo", ymmv) per: https://blog.widodh.nl/2016/04/anyip-bind-a-whole-subnet-to-your-linux-machine/ also testable using socat like this: socat -d -d TCP6:[2600:3c02::f03c:91ff:fe59:7d2e]:6667,bind=[2604:8300:100:200b:6667:8::1],setsockopt-int=0:15:1 - this example connects to v6 freenode irc (2600:3c02::f03c:91ff:fe59:7d2e) from an ipv6 address in the range i set up anyip routing for. So i can connect from that source ip (2604:8300:100:200b:6667:8::1) but "ip addr" or "ifconfig" don't list that exact IP as being configured. Regards, RJ On 20 October 2017 at 17:33, Richard Jones wrote: > I'm able to make outbound connections bound to an arbitrary IP from my > range by specifying a socket option like: {raw, 0, 15, <<1>>} > > note that that protocol part of {raw, ???, _, _} is 0, not 6, since > it's an IP level socket option (not tcp-level). > > btw there's also a system-wide option so you don't need to specify > ip_freebind per socket: > > echo 1 > /proc/sys/net/ipv6/ip_nonlocal_bind > > > RJ > > On 20 October 2017 at 17:17, Michael L Martin wrote: >> Thanks for the quick response. As for what I'm attempting to achieve, well, >> I'm not really sure. >> >> I have a customer that is adamant that we use IP_FREEBIND, so I've been >> poking around in :gen_tcp and :inet trying to see how to do that. >> >> Here's what happens when I try to open a listen socket: >> >> iex(5)> {:ok, s} = :gen_tcp.listen(8000, [{:raw, 6, 15, <<1 :: size(32)>>}, >> {:ip, {10, 138, 69, 63}}]) >> {:ok, #Port<0.1305>} >> iex(6)> :inet.getopts(s, [{:raw, 6, 15, 4}]) >> {:ok, []} >> iex(7)> :gen_tcp.close(s) >> :ok >> iex(8)> {:ok, s} = :gen_tcp.listen(8000, [{:raw, 6, 15, <<1 :: size(32)>>}, >> {:ip, {10, 138, 69, 64}}]) >> ** (MatchError) no match of right hand side value: {:error, :eaddrnotavail} >> >> iex(8)> >> >> So it looks like it's not working for listen sockets, either. Could very >> well be that I simply don't know what the heck I'm doing... >> >> >> >> On 2017-10-20 12:04 PM, Andreas Schultz wrote: >> >> Hi, >> >> ----- On Oct 20, 2017, at 5:10 PM, Michael L Martin >> wrote: >> >> Raw socket options looks like it should be the answer, but I'm not getting >> good results: >> >> iex(worker@REDACTED)28> {:ok, s} = :gen_tcp.connect('localhost', 80, >> [{:raw, 6, 15, <<1>>}, {:ip, {10, 138, 69, 63}}]) >> >> {:ok, #Port<0.12110>} >> >> iex(worker@REDACTED)29> :inet.getopts(s, [{:raw, 6, 15, 32}]) >> >> {:ok, []} <== option not set? >> >> iex(worker@REDACTED)29> {:ok, s} = :gen_tcp.connect('localhost', 80, >> [{:raw, 6, 15, <<1>>}, {:ip, {10, 138, 69, 64}}]) >> >> ** (MatchError) no match of right hand side value: {:error, :eaddrnotavail} >> >> >> IP_FREEBIND on connect doesn't make sense, it's only really useful on listen >> sockets. >> >> The option lets you bind to an IP address that does not yet exist on the >> host. That is ok for listen, but when you attempt to build an outgoing >> connection from that IP, then it has to be configured. >> >> It appears that the IP_FREEBIND option is simply not set. If I use >> IP_TRANSPARENT instead: >> >> iex(worker@REDACTED)35> {:ok, s} = :gen_tcp.connect('localhost', 80, >> [{:raw, 6, 19, <<1>>}, {:ip, {10, 138, 69, 63}}]) >> >> {:ok, #Port<0.12113>} >> >> iex(worker@REDACTED)34> :inet.getopts(s, [{:raw, 6, 19, 32}]) >> >> {:ok, [{:raw, 6, 19, <<0, 0, 0, 0>>}]} >> >> >> IP_TRANSPARENT needs special iptables rules to work correctly and is for a >> very specific use case. I don't think it would be very useful for an Erlang >> application to use that. >> >> In this case, the raw option 19 (IP_TRANSPARENT) is set, but to false rather >> than true. I believe this is because the beam file needs root or >> CAP_NET_ADMIN. >> >> You need root and a specific iptables setup for it to work. >> >> Any thoughts? >> >> >> What are you attempting to achieve anyway? >> >> Regards >> Andreas >> >> >> >> >> On 2017-10-16 01:18 PM, Guilherme Andrade wrote: >> >> I believe you can use raw socket options[1] for that, but it won't be >> portable. >> >> [1]: http://erlang.org/doc/man/inet.html#setopts-2 - {raw, Protocol, >> OptionNum, ValueBin} >> >> On 16 October 2017 at 15:09, Michael L Martin wrote: >>> >>> Hi all, >>> >>> Is there a way to specify the FREEBIND option when opening a socket? I >>> don't see any reference to it in the documentation. >>> >>> >>> Thanks, >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> -- >> Guilherme >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From mmartin4242@REDACTED Fri Oct 20 20:22:31 2017 From: mmartin4242@REDACTED (Michael L Martin) Date: Fri, 20 Oct 2017 14:22:31 -0400 Subject: [erlang-questions] Socket option FREEBIND In-Reply-To: References: <865fda37-c29c-89b6-a8bb-aedd2d19f344@gmail.com> <542205910.3060266.1508515448634.JavaMail.zimbra@tpip.net> <6e1b6845-a8fb-771a-c5ff-bbb540928b1b@gmail.com> Message-ID: Posting again to all... iex(4)> {:ok, s} = :gen_tcp.connect('localhost', 80, [{:raw, 0, 15, <<1>>}, {:ip, {10, 138, 69, 63}}]) {:ok, #Port<0.1310>} iex(5)> :inet.getopts(s, [{:raw, 0, 15, 4}]) {:ok, [{:raw, 0, 15, <<1, 0, 0, 0>>}]} iex(6)> :gen_tcp.close(s) :ok iex(7)> {:ok, s} = :gen_tcp.connect('localhost', 80, [{:raw, 0, 15, <<1>>}, {:ip, {10, 138, 69, 64}}]) ** (exit) :badarg (kernel) gen_tcp.erl:149: :gen_tcp.connect/4 iex(7)> I see badarg when trying to connect using a bogus IP address. But,? the option is being set now. Using IP_TRANSPARENT instead of IP_FREEBIND: iex(8)> {:ok, s} = :gen_tcp.connect('localhost', 80, [{:raw, 0, 19, <<1>>}, {:ip, {10, 138, 69, 64}}]) At this point, :gen_tcp.connect/3 blocks. The ip address does not exist, so now I add it: sudo ip addr add 10.138.69.64 dev eno1 ip addr list dev eno12: eno1: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 ??? link/ether 8c:89:a5:c6:e1:7f brd ff:ff:ff:ff:ff:ff ??? inet 10.138.69.63/16 brd 10.138.255.255 scope global eno1 ?????? valid_lft forever preferred_lft forever ??? inet 10.138.69.64/32 scope global eno1 ?????? valid_lft forever preferred_lft forever ??? inet6 fe80::8e89:a5ff:fec6:e17f/64 scope link ?????? valid_lft forever preferred_lft forever I would think that :gen_tcp.connect would then connect, but it's still blocked, and eventually times out. After the timeout: iex(8)> {:ok, s} = :gen_tcp.connect('localhost', 80, [{:raw, 0, 19, <<1>>}, {:ip, {10, 138, 69, 64}}]) {:ok, #Port<0.1300>} iex(9)> :inet.getopts(s, [{:raw, 0, 19, 4}])??????????????????????????????????? {:ok, [{:raw, 0, 19, <<1, 0, 0, 0>>}]} iex(10)> On 2017-10-20 11:10 AM, Michael L Martin wrote: 6, 15, <<1>>}, {:ip, {10, 138, 69, 64}}]) ** (MatchError) no match of right hand side value: {:error, :eaddrnotavail} On 2017-10-20 12:33 PM, Richard Jones wrote: > I'm able to make outbound connections bound to an arbitrary IP from my > range by specifying a socket option like: {raw, 0, 15, <<1>>} > > note that that protocol part of {raw, ???, _, _} is 0, not 6, since > it's an IP level socket option (not tcp-level). > > btw there's also a system-wide option so you don't need to specify > ip_freebind per socket: > > echo 1 > /proc/sys/net/ipv6/ip_nonlocal_bind > > > RJ > > On 20 October 2017 at 17:17, Michael L Martin wrote: >> Thanks for the quick response. As for what I'm attempting to achieve, well, >> I'm not really sure. >> >> I have a customer that is adamant that we use IP_FREEBIND, so I've been >> poking around in :gen_tcp and :inet trying to see how to do that. >> >> Here's what happens when I try to open a listen socket: >> >> iex(5)> {:ok, s} = :gen_tcp.listen(8000, [{:raw, 6, 15, <<1 :: size(32)>>}, >> {:ip, {10, 138, 69, 63}}]) >> {:ok, #Port<0.1305>} >> iex(6)> :inet.getopts(s, [{:raw, 6, 15, 4}]) >> {:ok, []} >> iex(7)> :gen_tcp.close(s) >> :ok >> iex(8)> {:ok, s} = :gen_tcp.listen(8000, [{:raw, 6, 15, <<1 :: size(32)>>}, >> {:ip, {10, 138, 69, 64}}]) >> ** (MatchError) no match of right hand side value: {:error, :eaddrnotavail} >> >> iex(8)> >> >> So it looks like it's not working for listen sockets, either. Could very >> well be that I simply don't know what the heck I'm doing... >> >> >> >> On 2017-10-20 12:04 PM, Andreas Schultz wrote: >> >> Hi, >> >> ----- On Oct 20, 2017, at 5:10 PM, Michael L Martin >> wrote: >> >> Raw socket options looks like it should be the answer, but I'm not getting >> good results: >> >> iex(worker@REDACTED)28> {:ok, s} = :gen_tcp.connect('localhost', 80, >> [{:raw, 6, 15, <<1>>}, {:ip, {10, 138, 69, 63}}]) >> >> {:ok, #Port<0.12110>} >> >> iex(worker@REDACTED)29> :inet.getopts(s, [{:raw, 6, 15, 32}]) >> >> {:ok, []} <== option not set? >> >> iex(worker@REDACTED)29> {:ok, s} = :gen_tcp.connect('localhost', 80, >> [{:raw, 6, 15, <<1>>}, {:ip, {10, 138, 69, 64}}]) >> >> ** (MatchError) no match of right hand side value: {:error, :eaddrnotavail} >> >> >> IP_FREEBIND on connect doesn't make sense, it's only really useful on listen >> sockets. >> >> The option lets you bind to an IP address that does not yet exist on the >> host. That is ok for listen, but when you attempt to build an outgoing >> connection from that IP, then it has to be configured. >> >> It appears that the IP_FREEBIND option is simply not set. If I use >> IP_TRANSPARENT instead: >> >> iex(worker@REDACTED)35> {:ok, s} = :gen_tcp.connect('localhost', 80, >> [{:raw, 6, 19, <<1>>}, {:ip, {10, 138, 69, 63}}]) >> >> {:ok, #Port<0.12113>} >> >> iex(worker@REDACTED)34> :inet.getopts(s, [{:raw, 6, 19, 32}]) >> >> {:ok, [{:raw, 6, 19, <<0, 0, 0, 0>>}]} >> >> >> IP_TRANSPARENT needs special iptables rules to work correctly and is for a >> very specific use case. I don't think it would be very useful for an Erlang >> application to use that. >> >> In this case, the raw option 19 (IP_TRANSPARENT) is set, but to false rather >> than true. I believe this is because the beam file needs root or >> CAP_NET_ADMIN. >> >> You need root and a specific iptables setup for it to work. >> >> Any thoughts? >> >> >> What are you attempting to achieve anyway? >> >> Regards >> Andreas >> >> >> >> >> On 2017-10-16 01:18 PM, Guilherme Andrade wrote: >> >> I believe you can use raw socket options[1] for that, but it won't be >> portable. >> >> [1]: http://erlang.org/doc/man/inet.html#setopts-2 - {raw, Protocol, >> OptionNum, ValueBin} >> >> On 16 October 2017 at 15:09, Michael L Martin wrote: >>> Hi all, >>> >>> Is there a way to specify the FREEBIND option when opening a socket? I >>> don't see any reference to it in the documentation. >>> >>> >>> Thanks, >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> -- >> Guilherme >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sat Oct 21 14:33:53 2017 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 21 Oct 2017 14:33:53 +0200 Subject: [erlang-questions] What does "soft" real-time mean? In-Reply-To: References: <92b20510-42e4-09ad-4a03-26362553a941@comarch.pl> Message-ID: Hard real time: All requests MUST be satisfied within a specified time. Note this does not mean fast. If the deadline is five years and you respond with a year it's hard real time :-) Soft real time: Some requests might miss the deadlines. Fast hard real time is really difficult. Here you'll need special purpose hardware, a custom OS and a special programming language. The programming language must be constructed so that you can prove assertions about the worse case execution time of code. The OS must not introduce unexpected delays, and the hardware must have predictable delays. Hard real time is mostly associated with hardware control - examples: Braking systems in cars, control system in unstable aircraft, pulling out fuel rods in a nuclear reactor if things go wrong. In the software world, hard real time usually means "fast with predictable delays" (but peoples idea of what is fast varies) Erlang was designed for programming telephone exchanges - we wanted response times of milliseconds for simple requests. We also didn't want long lived computations to block computations that could be performed immediately - so each process gets an equal time slice of the CPU. Telecoms protocols are designed with failure in mind - so if the odd deadline is missed the system just tries again. My mental model of a telecomms system is that we have tens to hundreds of thousands of long-lived connections. Each of these sleeps for a very long time (from the computer's point of view) then wakes up and requests a computation which takes (from the computer's point of view) a very small time. This is what Erlang was designed for - hundreds of thousands of processes that are largely inactive - then they wake up and request a small amount of computation. The odd process might request a large amount of processing but this should not block the other processes. Also some processes will fail, due to software errors, the errors should be logged and not damage the unfailing parts of the system. Responses times should be of the order of milliseconds. This model works pretty well for large multiplexing web-servers (like WhatsApp) - the VM has be designed for fast context switching, fast process creation and fast message passing and have process isolation and be non-blocking. /Joe On Fri, Oct 20, 2017 at 3:58 PM, Avinash Dhumane wrote: > Hello! > > The application runs directly on latest Intel Xeon. > > The source code of specific module in question is emailed to you separately. > > There is a process per instance of "algorithm trading strategy", but no two > processes share anything except the ordering socket to the market, which is > accessed through synchronization (ETS). But, during the market test, we run > only one or two strategies so there is no synchronization overhead. > > The major computation is market depth update and price matching. > > On Fri, Oct 20, 2017 at 6:55 PM, Aleksander Nycz > wrote: >> >> Hello, >> >> >> My private definition: >> >> Hard real time -> 100% requests/jobs/processes must be completed before >> deadline >> >> Soft real time -> N% requests/jobs/processes must be completed before >> deadline, where N < 100%, >> >> so we can/must accept that some jobs will not be finished on time. >> >> >> And few question: >> >> 1. Erlang VM is running on Virtual Machine on bare metal? >> >> 2. What you erlang process really do? Can you show us some code? >> >> >> Regards >> >> Aleksander Nycz >> >> >> W dniu 2017-10-20 o 14:00, Avinash Dhumane pisze: >> >> I have programmed high-frequency trading application in Erlang and >> comparing its reaction performance in the live market with other competing >> applications (mostly, in C). >> >> It has been a consistent observation that my Erlang application lags >> substantially behind the competing applications. In fact, it never matched >> the reaction time of the so-called "slowest" C application in the market. >> >> The benchmark reaction time is about 5 microseconds. It is the time >> elapsed between tick to order. That is, the difference between the >> timestamps when I receive the tick (market event) from the mailbox of my >> process and when I submit the TCP send call on my order. >> >> My process maintains the market depths of the securities and when my >> desired price (as a function of best buyers and/or sellers on multiple legs >> of the order) is observed, I compute the order terms and submit. >> >> The entire computation and communication (with the market) takes place >> within the single process and there are no collaborating processes (hence, >> no inter-process messaging; not even ETS). Still, the reaction time is in >> the order of 20 to 60 microseconds - that is, nowhere closer to the >> benchmark reaction time (5 microseconds). >> >> What I wish to hear from the experienced people on this forum here is >> whether I am missing on something in my Erlang code, or should I just go >> ahead and program in C. >> >> Thanks. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> -- >> Aleksander Nycz >> Chief Designer >> Telco_021 BSS R&D >> Comarch SA >> Phone: +48 17 785 5909 >> Mobile: +48 691 464 275 >> website: www.comarch.pl >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mfidelman@REDACTED Sat Oct 21 16:08:06 2017 From: mfidelman@REDACTED (Miles Fidelman) Date: Sat, 21 Oct 2017 10:08:06 -0400 Subject: [erlang-questions] What does "soft" real-time mean? In-Reply-To: References: Message-ID: "Hard real time" generally refers to things where exact timing matters - things like machine control, sensor-to-weapon linkages, autopilots, software-defined radio, and such.? As some definitions put it, missing a deadline is considered a system failure. Generally, hard real time code involves direct hardware access, referenced to real-time clocks, and you can't miss any events. "Soft real time" generally refers to things that are still "real-time" but where things are "looser."? For example, VoIP is time sensitive, but can tolerate the occasional lost packet. Other kinds of soft real-time can tolerate a little jitter in the timing. Another term you'll sometimes come across is "near real time" - which tends to imply that a little delay is tolerable. More generally, "hard real time" is exactly that.? Everything else involves loosened constraints. When it comes to languages & run-times, anything that relies on asynchronous operating system services is not going to support hard real-time.? Hard real-time generally requires that software run very close to bare iron, with complete control of the hardware (i.e, no interrupts, no pre-emption), and access to very accurate timing data.? Erlang is known for high performance, but is very far from able to support hard real-time. Miles Fidelman On 10/20/17 8:00 AM, Avinash Dhumane wrote: > I have programmed high-frequency trading application in Erlang and > comparing its reaction performance in the live market with other > competing applications (mostly, in C). > > It has been a consistent observation that my Erlang application lags > substantially behind the competing applications. In fact, it never > matched the reaction time of the so-called "slowest" C application in > the market. > > The benchmark reaction time is about 5 microseconds. It is the time > elapsed between tick to order. That is, the difference between the > timestamps when I receive the tick (market event) from the mailbox of > my process and when I submit the TCP send call on my order. > > My process maintains the market depths of the securities and when my > desired price (as a function of best buyers and/or sellers on multiple > legs of the order) is observed, I compute the order terms and submit. > > The entire computation and communication (with the market) takes place > within the single process and there are no collaborating processes > (hence, no inter-process messaging; not even ETS). Still, the reaction > time is in the order of 20 to 60 microseconds - that is, nowhere > closer to the benchmark reaction time (5 microseconds). > > What I wish to hear from the experienced people on this forum here is > whether I am missing on something in my Erlang code, or should I just > go ahead and program in C. > > Thanks. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidnwelton@REDACTED Sat Oct 21 16:36:59 2017 From: davidnwelton@REDACTED (David Welton) Date: Sat, 21 Oct 2017 07:36:59 -0700 Subject: [erlang-questions] Updating common resources in a release? Message-ID: Hi, >From my understanding, Erlang's release machinery does not overwrite files, so that if I had, say, , {copy, "certs/my.crt", "certs/my.crt"} Or an index.html or something similar, it's not going to update it. I have inherited some code like this, and I'm wondering how people generally handle things like that? I can think of a few ideas: * Keep everything in priv/ directories within applications, and avoid 'common' files, so that updating the application updates whatever it contains. * Use the appup of some application to copy something in its priv/ directory over to the main 'common' area in the application? How do you handle this situation? Thanks! -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From kjnilsson@REDACTED Sun Oct 22 09:39:21 2017 From: kjnilsson@REDACTED (Karl Nilsson) Date: Sun, 22 Oct 2017 07:39:21 +0000 Subject: [erlang-questions] ANN: fez - an fsharp to core erlang compiler Message-ID: Hi, For a few months I've been hacking on fez - a compiler that takes fsharp code and turns it into core erlang and yesterday I released the first alpha release and thought perhaps it would be of some interest to this list. The idea was to take a mature statically typed functional (first) language with decent tooling and use it to generate code to run on the beam. The README on the github page outlines the current status and how to get started. FSharp is a multi-paradigmatic language so there are limits to what degree certain features (e.g. imperative programming) can be covered. The 0.1.0 release is mostly focussed on covering the functional paradigm and the standard fsharp libraries. There is also a simple FFI and some support for object programming. https://github.com/kjnilsson/fez https://github.com/kjnilsson/fez/releases/tag/v0.1.0-alpha.1 Feedback and questions are very welcome. There will be holes and obscure errors and I would like to hear about them all. Cheers Karl -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sun Oct 22 17:42:23 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 22 Oct 2017 15:42:23 +0000 Subject: [erlang-questions] What does "soft" real-time mean? In-Reply-To: References: Message-ID: On Fri, Oct 20, 2017 at 3:12 PM Avinash Dhumane wrote: > What I wish to hear from the experienced people on this forum here is > whether I am missing on something in my Erlang code, or should I just go > ahead and program in C. > > Your problem is one where you want to have a language which can execute computations quickly, and preferably well under 1ms. In that case, you often want explicit concurrency modeling as well as fast computation. Languages such a C, C++, Rust and OCaml are good candidates. Soft realtime isn't that well-defined in the literature. But one can define it from hard realtime. Other people have already written about what hard realtime is, so I'm not going to bother with that. But I do note one thing about hard-realtime systems in the context of concurrency which I want to make: If you have an existing schedule of processes, and you want to add a new one, you must say how often you want to be called, and for how long you will run. Now, the system can either accept your process, in which case it must fulfill the periodic schedulings you wanted. If you use too much time, it is valid for the system to kill your process. Importantly, if the scheduler cannot fit you into its schedule, it must reject starting your process. So if you ask that 10000 processes be started at the same point in time, the scheduler must reject you, unless it has access to 10000 processor cores. Now, soft realtime is like hard realtime, but there is an SLA and certain objectives which must be met. For instance that 99th percentile of all requests are handled within 2ms. The last percent is ignored and we make no rules for those. Also note that "soft" realtime is a practical thing: you measure the system and fail it if it doesn't meet the SLA. Erlang, as a system is good for handling such soft realtime requirements, provided that they tend to fall in the millisecond range and that you don't excessively load the system with work. Then you can stay within the SLA in practice, easily. However, if your task is that you want to answer a query in less than a millisecond, you need something closer to the steel. C is one option, but note that if you want a nanosecond computation, you probably want to move from C to FPGAs. Often, you also want explicitly control over what you are doing. If you hog the core with something else, then even switching away from the current work to the newly arrived work can be enough to mess with your timing schedule. Likewise, if the operating system takes an interrupt at the wrong point in time, your timing will be off. Hence, a good solution starts by defining the acceptable window of lag before answering what language to use. At, say, 5us, you have around 20 memory reads from DRAM into cache lines. Tight schedule which requires you have some control you wont have in a typical Erlang system, even if you factor out the computation to a NIF. Even in C, this will be hard to get close to. If your process is switched out, you are looking at TLB misses as well and it can easily prod you into a direction where you can only answer in 5us for synthetic benchmarks. The trick, of course, is to "cheat" by doing more than a single piece of work in say 50us and then divide down to claim that it is 5us. Or by keeping the data set small enough it fits into L1 or L2. Erlang could be a nice tool for the orchestration on top of the C programs however, if you happen to need some massive concurrency there. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Sun Oct 22 21:15:22 2017 From: lloyd@REDACTED (lloyd@REDACTED) Date: Sun, 22 Oct 2017 15:15:22 -0400 (EDT) Subject: [erlang-questions] =?utf-8?q?Can_one_have_more_than_one_secondary?= =?utf-8?q?_index_on_a_mnesia_table=3F?= Message-ID: <1508699722.233326556@apps.rackspace.com> Hello, Initialized table: init_table() -> mnesia:create_table(?TABLE, [ {disc_copies, [node()] }, {attributes, record_info(fields, ?TABLE)}, {index, [member_id, penname]} ]). Checked info: $ wg_schema(wg_table). .... {index,3} -> 376866 {index,5} -> 380964 This query works: get_author_by_member_id(MemberID) -> Query = fun() -> mnesia:index_read(?TABLE, MemberID, 3) end, {atomic, Result} = mnesia:transaction(Query), Result. But this doesn't: get_author_by_penname(Penname) -> Query = fun() -> mnesia:index_read({?TABLE, Penname, 5}) end, {atomic, Result} = mnesia:transaction(Query), {length(Result), Result}. Am I doing something wrong? Or is there something I should be doing altogether? Thanks to all, LRP ********************************************* My books: THE GOSPEL OF ASHES http://thegospelofashes.com Strength is not enough. Do they have the courage and the cunning? Can they survive long enough to save the lives of millions? FREEIN' PANCHO http://freeinpancho.com A community of misfits help a troubled boy find his way AYA TAKEO http://ayatakeo.com Star-crossed love, war and power in an alternative universe Available through Amazon or by request from your favorite bookstore ********************************************** From aschultz@REDACTED Mon Oct 23 09:44:32 2017 From: aschultz@REDACTED (Andreas Schultz) Date: Mon, 23 Oct 2017 09:44:32 +0200 (CEST) Subject: [erlang-questions] Socket option FREEBIND In-Reply-To: References: <865fda37-c29c-89b6-a8bb-aedd2d19f344@gmail.com> <542205910.3060266.1508515448634.JavaMail.zimbra@tpip.net> <6e1b6845-a8fb-771a-c5ff-bbb540928b1b@gmail.com> Message-ID: <1103484921.3060910.1508744672083.JavaMail.zimbra@tpip.net> ----- On Oct 20, 2017, at 8:22 PM, Michael L Martin wrote: > Posting again to all... > iex(4)> {:ok, s} = :gen_tcp.connect('localhost', 80, [{:raw, 0, 15, <<1>>}, > {:ip, {10, 138, 69, 63}}]) Very small nitpick (does not impact the function), setsockopt boolean and integer options usually use a 32bit integer. Debugging tools like strace are much easier to read then. So I would recommend to write the options as {:raw, 0, 15, <<1:32/native>>} > {:ok, #Port<0.1310>} > iex(5)> :inet.getopts(s, [{:raw, 0, 15, 4}]) > {:ok, [{:raw, 0, 15, <<1, 0, 0, 0>>}]} > iex(6)> :gen_tcp.close(s) :ok > iex(7)> {:ok, s} = :gen_tcp.connect('localhost', 80, [{:raw, 0, 15, <<1>>}, > {:ip, {10, 138, 69, 64}}]) 10.138.69.64 does not exist on you system, so attempting to use it will fail (as you just discovered). > ** (exit) :badarg > (kernel) gen_tcp.erl:149: :gen_tcp.connect/4 > iex(7)> > I see badarg when trying to connect using a bogus IP address. But, the option is > being set now. > Using IP_TRANSPARENT instead of IP_FREEBIND: > iex(8)> {:ok, s} = :gen_tcp.connect('localhost', 80, [{:raw, 0, 19, <<1>>}, > {:ip, {10, 138, 69, 64}}]) IP_TRANSPARENT is very different from IP_FREEBIND. You need iptables rules to make it work (and a lot more about what it is supposed to to before playing with it). > At this point, :gen_tcp.connect/3 blocks. The ip address does not exist, so now > I add it: > sudo ip addr add 10.138.69.64 dev eno1 > ip addr list dev eno12: eno1: mtu 1500 qdisc > pfifo_fast state UP group default qlen 1000 > link/ether 8c:89:a5:c6:e1:7f brd ff:ff:ff:ff:ff:ff > inet 10.138.69.63/16 brd 10.138.255.255 scope global eno1 > valid_lft forever preferred_lft forever > inet 10.138.69.64/32 scope global eno1 > valid_lft forever preferred_lft forever > inet6 fe80::8e89:a5ff:fec6:e17f/64 scope link > valid_lft forever preferred_lft forever > I would think that :gen_tcp.connect would then connect, but it's still blocked, > and eventually times out. > After the timeout: > iex(8)> {:ok, s} = :gen_tcp.connect('localhost', 80, [{:raw, 0, 19, <<1>>}, > {:ip, {10, 138, 69, 64}}]) > {:ok, #Port<0.1300>} > iex(9)> :inet.getopts(s, [{:raw, 0, 19, 4}]) {:ok, [{:raw, 0, 19, <<1, 0, 0, > 0>>}]} > iex(10)> As I have said before, using FREEBIND for connect makes no sense. What you probably want it to bind a listening socket to an IP address that will later be added by systemd. For that include the raw option in the listening call. Regards Andreas >> On 2017-10-20 11:10 AM, Michael L Martin wrote: >>> 6, 15, <<1>>}, {:ip, {10, 138, 69, 64}}]) >>> ** (MatchError) no match of right hand side value: {:error, :eaddrnotavail} > On 2017-10-20 12:33 PM, Richard Jones wrote: >> I'm able to make outbound connections bound to an arbitrary IP from my >> range by specifying a socket option like: {raw, 0, 15, <<1>>} >> note that that protocol part of {raw, ???, _, _} is 0, not 6, since >> it's an IP level socket option (not tcp-level). >> btw there's also a system-wide option so you don't need to specify >> ip_freebind per socket: >> echo 1 > /proc/sys/net/ipv6/ip_nonlocal_bind >> RJ >> On 20 October 2017 at 17:17, Michael L Martin [ mailto:mmartin4242@REDACTED | >> ] wrote: >>> Thanks for the quick response. As for what I'm attempting to achieve, well, >>> I'm not really sure. >>> I have a customer that is adamant that we use IP_FREEBIND, so I've been >>> poking around in :gen_tcp and :inet trying to see how to do that. >>> Here's what happens when I try to open a listen socket: >>> iex(5)> {:ok, s} = :gen_tcp.listen(8000, [{:raw, 6, 15, <<1 :: size(32)>>}, >>> {:ip, {10, 138, 69, 63}}]) >>> {:ok, #Port<0.1305>} >>> iex(6)> :inet.getopts(s, [{:raw, 6, 15, 4}]) >>> {:ok, []} >>> iex(7)> :gen_tcp.close(s) >>> :ok >>> iex(8)> {:ok, s} = :gen_tcp.listen(8000, [{:raw, 6, 15, <<1 :: size(32)>>}, >>> {:ip, {10, 138, 69, 64}}]) >>> ** (MatchError) no match of right hand side value: {:error, :eaddrnotavail} >>> iex(8)> >>> So it looks like it's not working for listen sockets, either. Could very >>> well be that I simply don't know what the heck I'm doing... >>> On 2017-10-20 12:04 PM, Andreas Schultz wrote: >>> Hi, >>> ----- On Oct 20, 2017, at 5:10 PM, Michael L Martin [ >>> mailto:mmartin4242@REDACTED | ] wrote: >>> Raw socket options looks like it should be the answer, but I'm not getting >>> good results: >>> iex( [ mailto:worker@REDACTED | worker@REDACTED ] )28> {:ok, s} = >>> :gen_tcp.connect('localhost', 80, >>> [{:raw, 6, 15, <<1>>}, {:ip, {10, 138, 69, 63}}]) >>> {:ok, #Port<0.12110>} >>> iex( [ mailto:worker@REDACTED | worker@REDACTED ] )29> :inet.getopts(s, >>> [{:raw, 6, 15, 32}]) >>> {:ok, []} <== option not set? >>> iex( [ mailto:worker@REDACTED | worker@REDACTED ] )29> {:ok, s} = >>> :gen_tcp.connect('localhost', 80, >>> [{:raw, 6, 15, <<1>>}, {:ip, {10, 138, 69, 64}}]) >>> ** (MatchError) no match of right hand side value: {:error, :eaddrnotavail} >>> IP_FREEBIND on connect doesn't make sense, it's only really useful on listen >>> sockets. >>> The option lets you bind to an IP address that does not yet exist on the >>> host. That is ok for listen, but when you attempt to build an outgoing >>> connection from that IP, then it has to be configured. >>> It appears that the IP_FREEBIND option is simply not set. If I use >>> IP_TRANSPARENT instead: >>> iex( [ mailto:worker@REDACTED | worker@REDACTED ] )35> {:ok, s} = >>> :gen_tcp.connect('localhost', 80, >>> [{:raw, 6, 19, <<1>>}, {:ip, {10, 138, 69, 63}}]) >>> {:ok, #Port<0.12113>} >>> iex( [ mailto:worker@REDACTED | worker@REDACTED ] )34> :inet.getopts(s, >>> [{:raw, 6, 19, 32}]) >>> {:ok, [{:raw, 6, 19, <<0, 0, 0, 0>>}]} >>> IP_TRANSPARENT needs special iptables rules to work correctly and is for a >>> very specific use case. I don't think it would be very useful for an Erlang >>> application to use that. >>> In this case, the raw option 19 (IP_TRANSPARENT) is set, but to false rather >>> than true. I believe this is because the beam file needs root or >>> CAP_NET_ADMIN. >>> You need root and a specific iptables setup for it to work. >>> Any thoughts? >>> What are you attempting to achieve anyway? >>> Regards >>> Andreas >>> On 2017-10-16 01:18 PM, Guilherme Andrade wrote: >>> I believe you can use raw socket options[1] for that, but it won't be >>> portable. >>> [1]: [ http://erlang.org/doc/man/inet.html#setopts-2 | >>> http://erlang.org/doc/man/inet.html#setopts-2 ] - {raw, Protocol, >>> OptionNum, ValueBin} >>> On 16 October 2017 at 15:09, Michael L Martin [ mailto:mmartin4242@REDACTED | >>> ] wrote: >>>> Hi all, >>>> Is there a way to specify the FREEBIND option when opening a socket? I >>>> don't see any reference to it in the documentation. >>>> Thanks, >>>> _______________________________________________ >>>> erlang-questions mailing list [ mailto:erlang-questions@REDACTED | >>>> erlang-questions@REDACTED ] [ >>>> http://erlang.org/mailman/listinfo/erlang-questions | >>>> http://erlang.org/mailman/listinfo/erlang-questions ] >>> -- >>> Guilherme >>> _______________________________________________ >>> erlang-questions mailing list [ mailto:erlang-questions@REDACTED | >>> erlang-questions@REDACTED ] [ >>> http://erlang.org/mailman/listinfo/erlang-questions | >>> http://erlang.org/mailman/listinfo/erlang-questions ] >>> _______________________________________________ >>> erlang-questions mailing list [ mailto:erlang-questions@REDACTED | >>> erlang-questions@REDACTED ] [ >>> http://erlang.org/mailman/listinfo/erlang-questions | >>> http://erlang.org/mailman/listinfo/erlang-questions ] -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Mon Oct 23 10:23:26 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 23 Oct 2017 08:23:26 +0000 Subject: [erlang-questions] Pretty print list of strings in tabular way Message-ID: Hey guys, Is there any library which helps pretty print list of ?list of strings? into a nice table? I?ve found this module prettyptr shipped with the VM. But I couldn?t find any example on how to use it. Any idea is very welcome. Thanks /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From dgud@REDACTED Mon Oct 23 10:35:58 2017 From: dgud@REDACTED (Dan Gudmundsson) Date: Mon, 23 Oct 2017 08:35:58 +0000 Subject: [erlang-questions] Can one have more than one secondary index on a mnesia table? In-Reply-To: <1508699722.233326556@apps.rackspace.com> References: <1508699722.233326556@apps.rackspace.com> Message-ID: Remove the tuple in the second call: mnesia:index_read(?TABLE, MemberID, 3) vs mnesia:index_read({?TABLE, Penname, 5}) /Dan On Sun, Oct 22, 2017 at 9:15 PM wrote: > Hello, > > Initialized table: > > init_table() -> > mnesia:create_table(?TABLE, > [ {disc_copies, [node()] }, > {attributes, record_info(fields, ?TABLE)}, > {index, [member_id, penname]} > ]). > > Checked info: > > $ wg_schema(wg_table). > .... > {index,3} -> 376866 > {index,5} -> 380964 > > This query works: > > get_author_by_member_id(MemberID) -> > Query = > fun() -> > mnesia:index_read(?TABLE, MemberID, 3) > end, > {atomic, Result} = mnesia:transaction(Query), > Result. > > But this doesn't: > > get_author_by_penname(Penname) -> > Query = > fun() -> > mnesia:index_read({?TABLE, Penname, 5}) > end, > {atomic, Result} = mnesia:transaction(Query), > {length(Result), Result}. > > Am I doing something wrong? Or is there something I should be doing > altogether? > > Thanks to all, > > LRP > > > > > ********************************************* > My books: > > THE GOSPEL OF ASHES > http://thegospelofashes.com > > Strength is not enough. Do they have the courage > and the cunning? Can they survive long enough to > save the lives of millions? > > FREEIN' PANCHO > http://freeinpancho.com > > A community of misfits help a troubled boy find his way > > AYA TAKEO > http://ayatakeo.com > > Star-crossed love, war and power in an alternative > universe > > Available through Amazon or by request from your > favorite bookstore > > > ********************************************** > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cean.ebengt@REDACTED Mon Oct 23 10:36:42 2017 From: cean.ebengt@REDACTED (bengt e) Date: Mon, 23 Oct 2017 10:36:42 +0200 Subject: [erlang-questions] Pretty print list of strings in tabular way In-Reply-To: References: Message-ID: Greetings, Where did you find a module called prettyptr? Searching for it I found nothing. I have used the module erl_prettypr for code, but I do not think that is what you want. Would you mind explaining slightly more about what you are trying to achieve? bengt On Mon, Oct 23, 2017 at 10:23 AM, Frank Muller wrote: > Hey guys, > > Is there any library which helps pretty print list of ?list of strings? > into a nice table? > > I?ve found this module prettyptr shipped with the VM. But I couldn?t find > any example on how to use it. > > Any idea is very welcome. > > Thanks > /Frank > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Mon Oct 23 10:39:57 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 23 Oct 2017 08:39:57 +0000 Subject: [erlang-questions] Pretty print list of strings in tabular way In-Reply-To: References: Message-ID: erlang/19.1/lib/syntax_tools-2.1/src/prettypr.erl http://erlang.org/doc/man/prettypr.html Greetings, > > Where did you find a module called prettyptr? Searching for it I found > nothing. > I have used the module erl_prettypr for code, but I do not think that is > what you want. > Would you mind explaining slightly more about what you are trying to > achieve? > > bengt > > On Mon, Oct 23, 2017 at 10:23 AM, Frank Muller > wrote: > >> Hey guys, >> >> Is there any library which helps pretty print list of ?list of strings? >> into a nice table? >> >> I?ve found this module prettyptr shipped with the VM. But I couldn?t find >> any example on how to use it. >> >> Any idea is very welcome. >> >> Thanks >> /Frank >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Mon Oct 23 10:42:06 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 23 Oct 2017 08:42:06 +0000 Subject: [erlang-questions] Pretty print list of strings in tabular way In-Reply-To: References: Message-ID: What I would like to achieve is something similar to Scribe but in pure Erlang: https://github.com/codedge-llc/scribe /Frank erlang/19.1/lib/syntax_tools-2.1/src/prettypr.erl > > http://erlang.org/doc/man/prettypr.html > > > Greetings, >> >> Where did you find a module called prettyptr? Searching for it I found >> nothing. >> I have used the module erl_prettypr for code, but I do not think that is >> what you want. >> Would you mind explaining slightly more about what you are trying to >> achieve? >> >> bengt >> >> On Mon, Oct 23, 2017 at 10:23 AM, Frank Muller < >> frank.muller.erl@REDACTED> wrote: >> >>> Hey guys, >>> >>> Is there any library which helps pretty print list of ?list of strings? >>> into a nice table? >>> >>> I?ve found this module prettyptr shipped with the VM. But I couldn?t >>> find any example on how to use it. >>> >>> Any idea is very welcome. >>> >>> Thanks >>> /Frank >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Mon Oct 23 11:09:03 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 23 Oct 2017 11:09:03 +0200 Subject: [erlang-questions] Pretty print list of strings in tabular way In-Reply-To: References: Message-ID: The erl_prettypr.erl module uses the prettypr.erl module to do the formatting, so you can look at that as an example. Generating nice-looking tables is however not something that fits well with how prettypr.erl works. It wants to be able to make local choices between vertical or horizontal alignment, and a table requires that the lines separating cells are aligned both horizontally and vertically across the whole table. Maybe it would be straightforward to add such grid alignment as an extra capability to prettypr.erl though. Otherwise, you could use it to format individual pieces of text to fit into the cells of the table once you have chosen the cell sizes. /Richard 2017-10-23 10:42 GMT+02:00 Frank Muller : > What I would like to achieve is something similar to Scribe but in pure > Erlang: > > https://github.com/codedge-llc/scribe > > /Frank > > > erlang/19.1/lib/syntax_tools-2.1/src/prettypr.erl >> >> http://erlang.org/doc/man/prettypr.html >> >> >> Greetings, >>> >>> Where did you find a module called prettyptr? Searching for it I found >>> nothing. >>> I have used the module erl_prettypr for code, but I do not think that is >>> what you want. >>> Would you mind explaining slightly more about what you are trying to >>> achieve? >>> >>> bengt >>> >>> On Mon, Oct 23, 2017 at 10:23 AM, Frank Muller < >>> frank.muller.erl@REDACTED> wrote: >>> >>>> Hey guys, >>>> >>>> Is there any library which helps pretty print list of ?list of strings? >>>> into a nice table? >>>> >>>> I?ve found this module prettyptr shipped with the VM. But I couldn?t >>>> find any example on how to use it. >>>> >>>> Any idea is very welcome. >>>> >>>> Thanks >>>> /Frank >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Mon Oct 23 12:11:50 2017 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Mon, 23 Oct 2017 10:11:50 +0000 Subject: [erlang-questions] Pretty print list of strings in tabular way In-Reply-To: References: Message-ID: How about using the elixir lib you linked? @benoitc has a rebar3 provider that can compile elixir libs for you. On Mon 23 Oct 2017 at 11:09, Richard Carlsson wrote: > The erl_prettypr.erl module uses the prettypr.erl module to do the > formatting, so you can look at that as an example. > > Generating nice-looking tables is however not something that fits well > with how prettypr.erl works. It wants to be able to make local choices > between vertical or horizontal alignment, and a table requires that the > lines separating cells are aligned both horizontally and vertically across > the whole table. Maybe it would be straightforward to add such grid > alignment as an extra capability to prettypr.erl though. Otherwise, you > could use it to format individual pieces of text to fit into the cells of > the table once you have chosen the cell sizes. > > /Richard > > 2017-10-23 10:42 GMT+02:00 Frank Muller : > >> What I would like to achieve is something similar to Scribe but in pure >> Erlang: >> >> https://github.com/codedge-llc/scribe >> >> /Frank >> >> >> erlang/19.1/lib/syntax_tools-2.1/src/prettypr.erl >>> >>> http://erlang.org/doc/man/prettypr.html >>> >>> >>> Greetings, >>>> >>>> Where did you find a module called prettyptr? Searching for it I found >>>> nothing. >>>> I have used the module erl_prettypr for code, but I do not think that >>>> is what you want. >>>> Would you mind explaining slightly more about what you are trying to >>>> achieve? >>>> >>>> bengt >>>> >>>> On Mon, Oct 23, 2017 at 10:23 AM, Frank Muller < >>>> frank.muller.erl@REDACTED> wrote: >>>> >>>>> Hey guys, >>>>> >>>>> Is there any library which helps pretty print list of ?list of >>>>> strings? into a nice table? >>>>> >>>>> I?ve found this module prettyptr shipped with the VM. But I couldn?t >>>>> find any example on how to use it. >>>>> >>>>> Any idea is very welcome. >>>>> >>>>> Thanks >>>>> /Frank >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Cheers, -- Pierre Fenoll -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Mon Oct 23 13:05:26 2017 From: roger@REDACTED (Roger Lipscombe) Date: Mon, 23 Oct 2017 12:05:26 +0100 Subject: [erlang-questions] Getting system start time in UTC? Message-ID: I can get the node's uptime with erlang:statistics(wall_clock), which is in milliseconds, and can easily be converted into days, hours, etc. representing how long this node has been up. Is there an easy way to get the start time of the node, represented as a UTC date/time? I'd like to be able to print "Up since dd/MM/yyyy (D days ...", etc. Obviously, I *could* subtract the uptime from the current date/time, but that seems more complicated than necessary, assuming the node has (somewhere) the actual start time. Thanks, Roger. From hq@REDACTED Mon Oct 23 13:15:35 2017 From: hq@REDACTED (Adam Rutkowski) Date: Mon, 23 Oct 2017 13:15:35 +0200 Subject: [erlang-questions] Pretty print list of strings in tabular way In-Reply-To: References: Message-ID: <1508757335.2728429.1147794232.03F2A03A@webmail.messagingengine.com> On Mon, Oct 23, 2017, at 12:11, Pierre Fenoll wrote: > How about using the elixir lib you linked? > @benoitc has a rebar3 provider that can compile elixir libs for you. That lib looks pretty comprehensive! If the rebar3 plugin option is not entirely satisfying, I'd recommend looking at this [1] implementation in Clojure, for a very basic and elegant approach that should be easily transferrable. I was looking at it when writing yet another Elixir version here [2] -- it's not as advanced as the library you linked though, but perhaps a good point to get started. [1] https://github.com/clojure/clojure/blob/93d13d0c0671130b329863570080c72799563ac7/src/clj/clojure/pprint/print_table.clj#L11-L35 [2] https://github.com/aerosol/Tabula /A. -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Mon Oct 23 13:29:18 2017 From: erlangsiri@REDACTED (Siri Hansen) Date: Mon, 23 Oct 2017 13:29:18 +0200 Subject: [erlang-questions] Netconf Support In-Reply-To: References: Message-ID: Hi Greg, ct_netconfc is a Netconf client used for testing. It is part for the common_test application and is meant to be used within this framework (meaning that it has dependencies towards other parts of common_test). Regards /siri 2017-10-18 20:34 GMT+02:00 Greg : > Does Erlang/OTP has Netconf client and server implementation? > > Google brings me to http://erlang.org/doc/man/ct_netconfc.html; what is > it? > > The other project that I found seems to be not maintained > https://github.com/FlowForwarding/enetconf > > Is there anyone who is using Netconf with Erlang and can help provide some > starting point. > > Greg > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From z@REDACTED Mon Oct 23 14:15:34 2017 From: z@REDACTED (Danil Zagoskin) Date: Mon, 23 Oct 2017 15:15:34 +0300 Subject: [erlang-questions] Getting system start time in UTC? In-Reply-To: References: Message-ID: Hi! Maybe erlang:system_info(start_time) is what are you looking for. Get Unix time at system start: erlang:convert_time_unit(erlang:system_info(start_time) + erlang:time_offset(), native, seconds). On Mon, Oct 23, 2017 at 2:05 PM, Roger Lipscombe wrote: > I can get the node's uptime with erlang:statistics(wall_clock), which > is in milliseconds, and can easily be converted into days, hours, etc. > representing how long this node has been up. > > Is there an easy way to get the start time of the node, represented as > a UTC date/time? I'd like to be able to print "Up since dd/MM/yyyy (D > days ...", etc. > > Obviously, I *could* subtract the uptime from the current date/time, > but that seems more complicated than necessary, assuming the node has > (somewhere) the actual start time. > > Thanks, > Roger. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From Dinislam.Salikhov@REDACTED Mon Oct 23 14:35:40 2017 From: Dinislam.Salikhov@REDACTED (Salikhov Dinislam) Date: Mon, 23 Oct 2017 15:35:40 +0300 Subject: [erlang-questions] Synchronous communication with prim_file port Message-ID: <673e3cac-ac77-27ff-df96-a44dcf1e8ba3@kaspersky.com> Hello. I have an erlang process that receives logs from other processes and writes them into the file. The logs are written to the file via file:write() which in turn boils down to erlang:port_command/2 call followed by getting response from driver via "receive {Port, {data, Data}} -> ... end" (see ./erts/preloaded/src/prim_file.erl, write/2 and follow the calls). And here comes a trouble: if log rate is too high then while driver does its job, a lot of messages are waiting in the process's message queue and the receive has to walk through all of them to find the driver's answer. Is there any rationale for write operation being done this way? I'd like to propose an optimization here: if we know in advance that standard drv_get_response/1 will be used to get driver's response, then we can use synchronous erlang:port_control/3 instead to avoid the whole message queue traverse. Salikhov Dinislam From carlsson.richard@REDACTED Mon Oct 23 15:44:57 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 23 Oct 2017 15:44:57 +0200 Subject: [erlang-questions] Netconf Support In-Reply-To: References: Message-ID: See also my answer here: https://stackoverflow.com/a/46837612/240949 /Richard 2017-10-23 13:29 GMT+02:00 Siri Hansen : > Hi Greg, > > ct_netconfc is a Netconf client used for testing. It is part for the > common_test application and is meant to be used within this framework > (meaning that it has dependencies towards other parts of common_test). > > Regards > /siri > > > 2017-10-18 20:34 GMT+02:00 Greg : > >> Does Erlang/OTP has Netconf client and server implementation? >> >> Google brings me to http://erlang.org/doc/man/ct_netconfc.html; what is >> it? >> >> The other project that I found seems to be not maintained >> https://github.com/FlowForwarding/enetconf >> >> Is there anyone who is using Netconf with Erlang and can help provide >> some starting point. >> >> Greg >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Oct 23 16:15:53 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 23 Oct 2017 17:15:53 +0300 Subject: [erlang-questions] Synchronous communication with prim_file port In-Reply-To: <673e3cac-ac77-27ff-df96-a44dcf1e8ba3@kaspersky.com> References: <673e3cac-ac77-27ff-df96-a44dcf1e8ba3@kaspersky.com> Message-ID: This is a common problem. Perhaps you can buffer these log records in another process that will accumulate it in memory. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth@REDACTED Mon Oct 23 21:50:09 2017 From: kenneth@REDACTED (Kenneth Lundin) Date: Mon, 23 Oct 2017 21:50:09 +0200 Subject: [erlang-questions] Netconf Support In-Reply-To: References: Message-ID: You can also check this: https://github.com/FlowForwarding/enetconf A Netconf server in Erlang. I have never used it and can't say anything about quality or capability. /Regards Kenneth On Mon, Oct 23, 2017 at 3:44 PM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > See also my answer here: https://stackoverflow.com/a/46837612/240949 > > > /Richard > > 2017-10-23 13:29 GMT+02:00 Siri Hansen : > >> Hi Greg, >> >> ct_netconfc is a Netconf client used for testing. It is part for the >> common_test application and is meant to be used within this framework >> (meaning that it has dependencies towards other parts of common_test). >> >> Regards >> /siri >> >> >> 2017-10-18 20:34 GMT+02:00 Greg : >> >>> Does Erlang/OTP has Netconf client and server implementation? >>> >>> Google brings me to http://erlang.org/doc/man/ct_netconfc.html; what is >>> it? >>> >>> The other project that I found seems to be not maintained >>> https://github.com/FlowForwarding/enetconf >>> >>> Is there anyone who is using Netconf with Erlang and can help provide >>> some starting point. >>> >>> Greg >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin@REDACTED Tue Oct 24 00:56:31 2017 From: martin@REDACTED (Martin Karlsson) Date: Tue, 24 Oct 2017 11:56:31 +1300 Subject: [erlang-questions] Can one have more than one secondary index on a mnesia table? In-Reply-To: <1508699722.233326556@apps.rackspace.com> References: <1508699722.233326556@apps.rackspace.com> Message-ID: <20171023225631.tcqjphxtvoskzx4g@littledevil.rpk> >Am I doing something wrong? Or is there something I should be doing altogether? Your second index_read is not correct: >mnesia:index_read({?TABLE, Penname, 5}) Should be: mnesia:index_read/3 i.e mnesia:index_read(?TABLE, Penname, 5). I also tend to use the names rather than index position for the queries. mnesia:index_read(?TABLE, Penname, #yourrecord.penname). Cheers, Martin From ok@REDACTED Tue Oct 24 04:34:00 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 24 Oct 2017 15:34:00 +1300 Subject: [erlang-questions] Pretty print list of strings in tabular way In-Reply-To: References: Message-ID: <0bc9b130-b532-6a7d-0f62-beff00dd472b@cs.otago.ac.nz> I would normally handle tabular output by delegating to another program. Two simple ones (in Python and AWK) are described in http://www.cs.otago.ac.nz/staffpriv/ok/tale.pdf. These days I'd probably write a to a temporary file and tell a web browser to open it. Or possibly get RStudio to read it as a data frame and view it in the data viewer. Much depends on - the number of rows in the table - the number of columns in the table - the size of the fields (are they tolerably short, or are they paragraphs) - whether the fields contain tabs, line break separators, bidirectional text, or other odd stuff - whether the table is to be read by people or programs - whether the table is large enough that a human reader might need machine help to search it - or do things like highlighting the current row/column - or sort on selected columns ... From ok@REDACTED Tue Oct 24 06:21:38 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 24 Oct 2017 17:21:38 +1300 Subject: [erlang-questions] ANN: fez - an fsharp to core erlang compiler In-Reply-To: References: Message-ID: On 22/10/17 8:39 PM, Karl Nilsson wrote: > Hi, > > For a few months I've been hacking on fez - a compiler that takes fsharp > code and turns it into core erlang and yesterday I released the first > alpha release and thought perhaps it would be of some interest to this list. You're such a hero Marvel are talking to your lawyer about basing a comic series on you. That's pretty amazing. From lukas@REDACTED Tue Oct 24 09:39:40 2017 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 24 Oct 2017 09:39:40 +0200 Subject: [erlang-questions] Synchronous communication with prim_file port In-Reply-To: <673e3cac-ac77-27ff-df96-a44dcf1e8ba3@kaspersky.com> References: <673e3cac-ac77-27ff-df96-a44dcf1e8ba3@kaspersky.com> Message-ID: Hello, On Mon, Oct 23, 2017 at 2:35 PM, Salikhov Dinislam < Dinislam.Salikhov@REDACTED> wrote: > Is there any rationale for write operation being done this way? > I'd like to propose an optimization here: if we know in advance that > standard drv_get_response/1 will be used to get driver's response, then we > can use synchronous erlang:port_control/3 instead to avoid the whole > message queue traverse. Using erlang:port_control/3 will not help as the file I/O job is scheduled on the async thread poll which means that it has to ack by sending a message. We are in the process of re-writing the efile driver for OTP 21 using a nif, this re-write will solve this problem as no message passing is needed when ack:ing writes done on dirty schedulers. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Oct 24 10:31:02 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 24 Oct 2017 11:31:02 +0300 Subject: [erlang-questions] Synchronous communication with prim_file port In-Reply-To: References: <673e3cac-ac77-27ff-df96-a44dcf1e8ba3@kaspersky.com> Message-ID: But it will not save from overflowing message queue. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Tue Oct 24 10:36:00 2017 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 24 Oct 2017 10:36:00 +0200 Subject: [erlang-questions] Where are the binaries in message queue but not part of process_info(Pid, binary) ? In-Reply-To: References: Message-ID: Hello, On Tue, Oct 17, 2017 at 11:38 PM, Bekes, Andras G < Andras.Bekes@REDACTED> wrote: > - It seems the reliable way to measure the amount of data in the message > queue is setting message_queue_data process flag to off_heap beforehand > (ideally at spawn time with spawn_opt), and retrieve the full message queue > content with process_info(Pid,messages) ? which must be fully scanned. Is > there a better solution? (I originally aimed at refc binaries only because > I just wanted a quick estimation, and in my case anything else was > negligible.) > The off_heap flag should not effect anything when you are using process_info(Pid, messages). If you want the size of the data in the message queue then there is no better option than process_info(Pid, messages). Do keep in mind though, that any message queue inspection tools only show the messages that is part of what we call the internal message queue. Messages that are part of the external queue are considered to be in-transit to the process and thus not part of the message queue. In an overloaded process the majority of the messages will most likely be in the external message queue. > > > - May I suggest to improve the documentation of erlang:process_info(Pid, > binary) by the below facts, also referring to message_queue_data process > flag? > Yes, good idea. > > > - Is it possible to implement a new process_info_item() of > erlang:process_info/2 that gives information about binaries currently in > the message queue but off-heap? > > It should be possible to do for messages with off_heap data, however for a on_heap message queue it is only easy to get the messages that ended up off-heap, the binaries of messages that are on-heap would not be part of the result. This makes the API hard to explain which is not good. Also this inspection would again only be for the internal message queue. > > > - The documentation suggests using off_heap message_queue_data ?if the > process potentially can get many messages?, but this is rather vague. Does > this refer to processes expected to have many **queued** messages? I > guess a process receiving but not queuing many messages performs better > with on_heap (?Performance of the actual message passing is however > generally better when not using flag off_heap.?). > yes it does. I'll try to clarify that in the documentation. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From saintderanged.web@REDACTED Tue Oct 24 11:38:54 2017 From: saintderanged.web@REDACTED (Rob A'Court) Date: Tue, 24 Oct 2017 10:38:54 +0100 Subject: [erlang-questions] How to downgrade a TLS connection to TCP without an TLS Alert exchange Message-ID: Hi Is there a way to downgrade a TLS connection to TCP without an TLS Alert exchange in Erlang/Elixir? To give a bit of context: We've implemented the MS-TDS protocol (used by Microsoft SQL Server) using Elixir and the ssl and gen_tcp libraries. A TDS connection typically is unencrypted apart from the login. It starts unencrypted with a prelogin handshake, after that comes the TLS handshake, and encrypted login packet is sent and then all further packets are unencrypted. Using :ssl.close to downgrade the socket to TCP expects a TLS alert exchange to end the SSL connection however TDS does not expect or support TLS alerts, it simply goes back to being unencrypted after the login. I appreciate this is not standard TLS but I'm not expecting to change Microsoft's protocol to SQL Server any time soon! Currently we stop encryption by sending data directly to the TCP socket and changing the controlling process of the TCP socket to our receiving process. This works however it leaves the ssl_connection process hanging around. It monitors our receiving process and if that process dies a TLS alert is sent which TDS is not expecting. The alert is only sent in some teardown scenarios but it just seems rather messy. We're wondering if there's a better way to approach our problem? Many thanks Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Tue Oct 24 16:05:07 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Tue, 24 Oct 2017 14:05:07 +0000 Subject: [erlang-questions] Pretty print list of strings in tabular way In-Reply-To: References: Message-ID: Awesome, thanks guys for feedbacks/code /Frank > I was bored, and thought it was nice problem, below is some code if you > still need a solution. > > /Dan > > On Mon, Oct 23, 2017 at 10:23 AM Frank Muller > wrote: > >> Hey guys, >> >> Is there any library which helps pretty print list of ?list of strings? >> into a nice table? >> >> I?ve found this module prettyptr shipped with the VM. But I couldn?t find >> any example on how to use it. >> >> Any idea is very welcome. >> >> Thanks >> /Frank >> > _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -compile([export_all, nowarn_export_all]). > > def_opts() -> > #{pad=>$\s, dir=>trailing, c_sep=>" | ", r_sep => "-"}. > > table([Map|_] = LofMaps0) when is_map(Map) -> > table(lists:sort(maps:keys(Map)), LofMaps0, def_opts()). > > table(Keys, Rows) -> > table(Keys, Rows, def_opts()). > > table(Keys0, Rows0, Opts) -> > KeyStrs = [element(2, to_string(Key)) || Key <- Keys0], > Rows = [to_strings(Keys0, V) || V <- Rows0], > Ws = ws(Rows, [{undefined, string:length(Key)} || Key <- KeyStrs]), > Col = fun({_,Str}, {Type, Width}) -> > Dir = case Type of > number -> leading; > _ -> maps:get(dir, Opts, trailing) > end, > string:pad(Str, Width, Dir) > end, > Row = fun(Row) -> > R0 = [Col(Str, W) || {Str,W} <- lists:zip(Row, Ws)], > lists:join(maps:get(c_sep, Opts, " | "), R0) > end, > Header0 = [string:pad(Str, W, both) || > {Str,{_,W}} <- lists:zip(KeyStrs, Ws)], > Header = lists:join(maps:get(c_sep, Opts, " | "), Header0), > Delim = lists:duplicate(string:length(Header), maps:get(r_sep, Opts, > "-")), > [Header, $\n, Delim, $\n, > lists:join($\n, [Row(R) || R <- Rows]), $\n, > Delim, $\n,$\n]. > > ws([Row|Rs], Ws) -> > ws(Rs, ws_1(Row, Ws)); > ws([], Ws) -> Ws. > > ws_1([{T0,V}|Vs], [{Type,Max}|Ms]) -> > [{type(T0,Type),max(string:length(V),Max)}|ws_1(Vs,Ms)]; > ws_1([], []) -> []. > > type(T, undefined) -> T; > type(T, T) -> T; > type(_, _) -> string. > > to_strings(Keys, Map) when is_map(Map) -> > [to_string(maps:get(Key, Map, undefined)) || Key <- Keys]; > to_strings(_Keys, List) when is_list(List) -> > [to_string(Entry) || Entry <- List]; > to_strings(_Keys, Tuple) when is_tuple(Tuple) -> > [to_string(Entry) || Entry <- tuple_to_list(Tuple)]. > > to_string(Int) when is_integer(Int) -> {number, integer_to_list(Int)}; > to_string(Float) when is_float(Float) -> {number, > io_lib:format("~.4f",[Float])}; > to_string(Str) when is_list(Str) -> {string, Str}; > to_string(Bin) when is_binary(Bin) -> {string, Bin}; > to_string(T) -> {string, io_lib:format("~tp", [T])}. > > test() -> > Maps = [#{k=>K, v=> "string " ++ integer_to_list(K), > pi=>math:pi()*K, xfoo=>{term, "another"}} || > K <- lists:seq(8,12) ++ lists:seq(95, 110)], > MapTab = a:table(Maps), > io:put_chars(MapTab), > > Tuples = [{K, "string:" ++ integer_to_list(K), math:pi()*K, "another"} > || > K <- lists:seq(9,13) ++ lists:seq(90, 100)], > TupleTab = a:table([key,val,pi, str], Tuples), > io:put_chars(TupleTab), > > Lists = [{K, "string " ++ integer_to_list(K), math:pi()*K, {"another", > K*K}} || > K <- lists:seq(9,13) ++ lists:seq(995, 1004)], > Opts = #{pad=>$\s, dir=>both, c_sep=>" ", r_sep => " "}, > ListsTab = a:table([key,val,"pi multiplied", term], Lists, Opts), > io:put_chars(ListsTab), > ok. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hellkvist@REDACTED Wed Oct 25 08:56:51 2017 From: hellkvist@REDACTED (Stefan Hellkvist) Date: Wed, 25 Oct 2017 08:56:51 +0200 Subject: [erlang-questions] "prefix" ++ Rest = Something Message-ID: Hi, Erlang has this syntactic sugar for matching string prefixes ( http://erlang.org/doc/reference_manual/expressions.html#id80508) where you can do: "prefix" ++ Rest = "prefixsomething" , which would bind Rest to "something" in this case. I'm curious why however it is ok to do: 1> "prefix" ++ Rest = "prefixsomething". "prefixsomething" 2> Rest. "something" but it is not ok to do: 1> Prefix = "prefix". "prefix" 2> Prefix ++ Rest = "prefixsomething". * 1: illegal pattern Is it because this syntactic sugar is transformed more or less as a preprocessing step where the value of Prefix needs to be known, or why else is "Prefix ++ Rest = Something" not allowed even when Prefix is bound? /Stefan -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.hogberg@REDACTED Wed Oct 25 09:11:56 2017 From: john.hogberg@REDACTED (=?utf-8?B?Sm9obiBIw7ZnYmVyZw==?=) Date: Wed, 25 Oct 2017 07:11:56 +0000 Subject: [erlang-questions] Patch Package OTP 20.1.3 Released Message-ID: <1508915516.2025.5.camel@ericsson.com> Patch Package:???????????OTP 20.1.3 Git Tag:?????????????????OTP-20.1.3 Date:????????????????????2017-10-25 Trouble Report Id:???????OTP-13372, OTP-14672, OTP-14684, OTP-14694 Seq num:?????????????????ERIERL-73, ERIERL-85, ERL-494 System:??????????????????OTP Release:?????????????????20 Application:?????????????diameter-2.1.2, erts-9.1.3, snmp-5.2.8 Predecessor:?????????????OTP 20.1.2 ?Check out the git tag OTP-20.1.3, and build a full OTP system ?including documentation. Apply one or more applications from this ?build as patches to your installation using the 'otp_patch_apply' ?tool. For information on install requirements, see descriptions for ?each application version below. ?--------------------------------------------------------------------- ?--- diameter-2.1.2 -------------------------------------------------- ?--------------------------------------------------------------------- ?The diameter-2.1.2 application can be applied independently of other ?applications on a full OTP 20 installation. ?--- Fixed Bugs and Malfunctions --- ? OTP-14684????Application(s): diameter ???????????????Related Id(s): ERIERL-85 ???????????????A fault introduced in diameter 2.1 could cause decode ???????????????errors to be ignored in AVPs following the header of a ???????????????Grouped AVP. ?Full runtime dependencies of diameter-2.1.2: erts-6.4, kernel-3.2, ?ssl-6.0, stdlib-2.4 ?--------------------------------------------------------------------- ?--- erts-9.1.3 ------------------------------------------------------ ?--------------------------------------------------------------------- ?The erts-9.1.3 application can be applied independently of other ?applications on a full OTP 20 installation. ?--- Fixed Bugs and Malfunctions --- ? OTP-14672????Application(s): erts ???????????????Related Id(s): ERL-494 ???????????????Added zlib:set_controlling_process/2 to move a ???????????????zstream() between processes. ? OTP-14694????Application(s): erts ???????????????Fix so that schedulers are bound correctly when the ???????????????first available cpu is not the first detected cpu. e.g. ???????????????when using "taskset -c X..Y" when X is not equal to 0. ?Full runtime dependencies of erts-9.1.3: kernel-5.0, sasl-3.0.1, ?stdlib-3.0 ?--------------------------------------------------------------------- ?--- snmp-5.2.8 ------------------------------------------------------ ?--------------------------------------------------------------------- ?The snmp-5.2.8 application can be applied independently of other ?applications on a full OTP 20 installation. ?--- Fixed Bugs and Malfunctions --- ? OTP-13372????Application(s): snmp ???????????????Related Id(s): ERIERL-73 ???????????????The recbuf configuration option was not propagated ???????????????correctly to the socket for the SNMP Manager. ?Full runtime dependencies of snmp-5.2.8: crypto-3.3, erts-6.0, ?kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, stdlib-2.5 ?--------------------------------------------------------------------- ?--------------------------------------------------------------------- ?--------------------------------------------------------------------- From kostis@REDACTED Wed Oct 25 09:27:21 2017 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 25 Oct 2017 09:27:21 +0200 Subject: [erlang-questions] "prefix" ++ Rest = Something In-Reply-To: References: Message-ID: <7f610c2e-72b2-4f32-afc5-d270bb774326@cs.ntua.gr> On 10/25/2017 08:56 AM, Stefan Hellkvist wrote: > Is it because this syntactic sugar is transformed more or less as a > preprocessing step where the value of Prefix needs to be known, Yes, that's the reason. It's a purely _syntactic_ thing. Kostis From pierrefenoll@REDACTED Wed Oct 25 12:54:59 2017 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Wed, 25 Oct 2017 12:54:59 +0200 Subject: [erlang-questions] "prefix" ++ Rest = Something In-Reply-To: <7f610c2e-72b2-4f32-afc5-d270bb774326@cs.ntua.gr> References: <7f610c2e-72b2-4f32-afc5-d270bb774326@cs.ntua.gr> Message-ID: It's the same with binaries: <<"a", Rest/binary>> = <<"abc">> %%=> Rest = <<"bc">> a() -> <<"a">> Prefix = <<(a())/binary>> PrefixSize = byte_size(Prefix) %% Compilation fails if size is not provided <> = <<"abc">> %%=> Rest = <<"bc">> Not sure what the differences may be due to compilation v. in the REPL. I never understood however why this was "just sugar" or why PrefixSize is explicitly needed: in f(Prefix) when is_list(Prefix) -> Prefix ++ Rest = "my_list_thing", Rest. why isn't the compiler able to generate a pattern match? Is it missing some concept, some structures? What are the missing pieces and what can be done to add them? Same question with binaries: since we have a "prefix-match of binaries only providing PrefixSize at runtime" instruction, * Why don't we have one for lists? * And oh God why do we have to provide that PrefixSize "manually"? (binding the variable ourselves, when the compiler could do that itself) * Why isn't suffix-matching of binaries implemented yet? (how different to prefix-matching can it be?) I could never find the answer to all of these questions. WRT binaries my thinking is that they are actually a mix of "bytes" and references to binaries, making some crucial operations O(log n) instead of O(1)... but prefix match exists... I really want to be able to write things like: <<"{", Name/binary, "}">> = PathToken %%=> Name = <<"id">> given PathToken = <<"{id}">> Cheers, -- Pierre Fenoll On 25 October 2017 at 09:27, Kostis Sagonas wrote: > On 10/25/2017 08:56 AM, Stefan Hellkvist wrote: > >> Is it because this syntactic sugar is transformed more or less as a >> preprocessing step where the value of Prefix needs to be known, >> > > Yes, that's the reason. It's a purely _syntactic_ thing. > > Kostis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Wed Oct 25 13:53:09 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 25 Oct 2017 13:53:09 +0200 Subject: [erlang-questions] "prefix" ++ Rest = Something In-Reply-To: References: <7f610c2e-72b2-4f32-afc5-d270bb774326@cs.ntua.gr> Message-ID: <20171025115309.GA50371@erix.ericsson.se> On Wed, Oct 25, 2017 at 12:54:59PM +0200, Pierre Fenoll wrote: > It's the same with binaries: > > <<"a", Rest/binary>> = <<"abc">> > %%=> Rest = <<"bc">> > > a() -> <<"a">> > Prefix = <<(a())/binary>> > PrefixSize = byte_size(Prefix) %% Compilation fails if size is not provided > <> = <<"abc">> > %%=> Rest = <<"bc">> The only time an unspecified size is allowed in the bit syntax is for the last field in a bit expression. > > Not sure what the differences may be due to compilation v. in the REPL. > > I never understood however why this was "just sugar" or why PrefixSize is > explicitly needed: > in > f(Prefix) when is_list(Prefix) -> > Prefix ++ Rest = "my_list_thing", > Rest. > why isn't the compiler able to generate a pattern match? Is it missing > some concept, some structures? > What are the missing pieces and what can be done to add them? "abc" ++ Rest is compiled to [$a, $b, $c | Rest] The variable Prefix is a complete list hence there is only a pointer to the head in runtime, so the only way to append to it is via lists:append/2 i.e the actual operator erlang:'++'/2, which creates a new list. So this can not be described as a pattern. The visual appearence of "abc"++Rest vs Prefix++Rest is misleading: [$a, $b, $c | Rest] vs lists:append(Prefix, Rest) > > Same question with binaries: > since we have a "prefix-match of binaries only providing PrefixSize at > runtime" instruction, > * Why don't we have one for lists? > * And oh God why do we have to provide that PrefixSize "manually"? > (binding the variable ourselves, when the compiler could do that itself) > * Why isn't suffix-matching of binaries implemented yet? (how > different to prefix-matching can it be?) > > I could never find the answer to all of these questions. > WRT binaries my thinking is that they are actually a mix of "bytes" and > references to binaries, > making some crucial operations O(log n) instead of O(1)... but prefix match > exists... > > I really want to be able to write things like: > <<"{", Name/binary, "}">> = PathToken > > %%=> Name = <<"id">> given PathToken = <<"{id}">> > > > > Cheers, > -- > Pierre Fenoll > > > On 25 October 2017 at 09:27, Kostis Sagonas wrote: > > > On 10/25/2017 08:56 AM, Stefan Hellkvist wrote: > > > >> Is it because this syntactic sugar is transformed more or less as a > >> preprocessing step where the value of Prefix needs to be known, > >> > > > > Yes, that's the reason. It's a purely _syntactic_ thing. > > > > Kostis > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From bchesneau@REDACTED Wed Oct 25 14:29:11 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 25 Oct 2017 14:29:11 +0200 Subject: [erlang-questions] sharing config via a dynamically compiled module? Message-ID: I'm temped to compile dynamically a module for some highly demanded resources to share their config instead of using ETS for it. But I'm wondering if's not to hackish. (I'm also tempted by just using something more pure like sharing it via message passing). Are other people do such thing? I can see the following pro/cons: pro: * fast and constant access, it is just a call to a module function and some literals. cons: * maybe too much hackish? * What happen if the resource is deleted when someone is using it? Will the module be really deleted or will it leaks? What do you think about it? Any feedback is welcome :) - beno?t From roger@REDACTED Wed Oct 25 15:04:46 2017 From: roger@REDACTED (Roger Lipscombe) Date: Wed, 25 Oct 2017 14:04:46 +0100 Subject: [erlang-questions] sharing config via a dynamically compiled module? In-Reply-To: References: Message-ID: On 25 October 2017 at 13:29, Benoit Chesneau wrote: > I'm temped to compile dynamically a module for some highly demanded resources > to share their config instead of using ETS for it. But I'm wondering if's not > to hackish. (I'm also tempted by just using something more pure like sharing it > via message passing). > > Are other people do such thing? Yes: https://github.com/mochi/mochiweb/blob/master/src/mochiglobal.erl From bchesneau@REDACTED Wed Oct 25 15:27:17 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 25 Oct 2017 15:27:17 +0200 Subject: [erlang-questions] sharing config via a dynamically compiled module? In-Reply-To: References: Message-ID: <92C15A74-A693-4189-9FB0-76FFB3F7B8F1@gmail.com> > On 25 Oct 2017, at 15:04, Roger Lipscombe wrote: > > On 25 October 2017 at 13:29, Benoit Chesneau wrote: >> I'm temped to compile dynamically a module for some highly demanded resources >> to share their config instead of using ETS for it. But I'm wondering if's not >> to hackish. (I'm also tempted by just using something more pure like sharing it >> via message passing). >> >> Are other people do such thing? > > Yes: https://github.com/mochi/mochiweb/blob/master/src/mochiglobal.erl Indeed :) I?ve also somewhat simpler code: -include_lib("syntax_tools/include/merl.hrl"). %% @doc Utility that converts a given property list into a module that provides %% constant time access to the various key/value pairs. %% %% Example: %% %% load_config(store_config, [{backends, [{rocksdb_ram, barrel_rocksdb}, %% {rocksdb_disk, barrel_rocksdb}]}, %% {data_dir, "/path/to_datadir"}]). %% %% creates the module store_config: %% store_config:backends(). => [{rocksdb_ram,barrel_rocksdb},{rocksdb_disk,barrel_rocksdb}] %% store_config:data_dir => "/path/to_datadir" %% -spec load_config(atom(), [{atom(), any()}]) -> ok. load_config(Resource, Config) when is_atom(Resource), is_list(Config) -> Module = ?Q("-module(" ++ atom_to_list(Resource) ++ ")."), Functions = lists:foldl(fun({K, V}, Acc) -> [make_function(K,V) | Acc] end, [], Config), Exported = [?Q("-export([" ++ atom_to_list(K) ++ "/0]).") || {K, _V} <- Config], Forms = lists:flatten([Module, Exported, Functions]), merl:compile_and_load(Forms, [verbose]), ok. make_function(K, V) -> Cs = [?Q("() -> _@REDACTED@")], F = erl_syntax:function(merl:term(K), Cs), ?Q("'@_F'() -> []."). %% @doc unload a config module loaded with the `load_config/2' function. -spec unload_config(atom()) -> true | false. unload_config(Resource) -> _ = code:purge(Resource), code:delete(Resource). which is doing something similar but create a function for each keys in the proplists. But I?m wondering if there are some cons to it apart the fact it is designed for case where little changes happen to the conf . It seems lot of people are using ets to share a config generally. Maybe becauseit feels a little hackish ? - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Wed Oct 25 15:55:51 2017 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Wed, 25 Oct 2017 16:55:51 +0300 Subject: [erlang-questions] sharing config via a dynamically compiled module? In-Reply-To: <92C15A74-A693-4189-9FB0-76FFB3F7B8F1@gmail.com> References: <92C15A74-A693-4189-9FB0-76FFB3F7B8F1@gmail.com> Message-ID: <4991037F-1F6D-4702-B023-72E62D7E33DB@gmail.com> Hello, > On 25 Oct 2017, at 16.27, Benoit Chesneau wrote: > > It seems lot of people are using ets to share a config generally. Maybe becauseit feels a little hackish ? The usage of code-as-a-config is ?valid? approach. I?ve used that for some project. However, the reloading of config is something to consider. You need to use code:purge(?) and take all pain associated with it. The code-as-a-config works as build time config but runtime config is easy with build-in application:get_env? - Dmitry -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Wed Oct 25 16:11:18 2017 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Wed, 25 Oct 2017 16:11:18 +0200 Subject: [erlang-questions] "prefix" ++ Rest = Something In-Reply-To: <20171025115309.GA50371@erix.ericsson.se> References: <7f610c2e-72b2-4f32-afc5-d270bb774326@cs.ntua.gr> <20171025115309.GA50371@erix.ericsson.se> Message-ID: > So this can not be described as a pattern. I'm saying let's fix that. Prefix ++ Rest as a pattern does not need to be calling ++/2. Instead, I am suggesting to allow "partial matches": adding support for these patterns. What makes this hard? Why is this only sugar? Why is a similar pattern allowed in binaries but not with lists? (the <> pattern) Cheers, -- Pierre Fenoll On 25 October 2017 at 13:53, Raimo Niskanen < raimo+erlang-questions@REDACTED> wrote: > On Wed, Oct 25, 2017 at 12:54:59PM +0200, Pierre Fenoll wrote: > > It's the same with binaries: > > > > <<"a", Rest/binary>> = <<"abc">> > > %%=> Rest = <<"bc">> > > > > a() -> <<"a">> > > Prefix = <<(a())/binary>> > > PrefixSize = byte_size(Prefix) %% Compilation fails if size is not > provided > > <> = <<"abc">> > > %%=> Rest = <<"bc">> > > The only time an unspecified size is allowed in the bit syntax is for the > last field in a bit expression. > > > > > Not sure what the differences may be due to compilation v. in the REPL. > > > > I never understood however why this was "just sugar" or why PrefixSize is > > explicitly needed: > > in > > f(Prefix) when is_list(Prefix) -> > > Prefix ++ Rest = "my_list_thing", > > Rest. > > why isn't the compiler able to generate a pattern match? Is it missing > > some concept, some structures? > > What are the missing pieces and what can be done to add them? > > "abc" ++ Rest > > is compiled to > > [$a, $b, $c | Rest] > > The variable Prefix is a complete list hence there is only a pointer to the > head in runtime, so the only way to append to it is via lists:append/2 i.e > the actual operator erlang:'++'/2, which creates a new list. So this can > not be described as a pattern. > > The visual appearence of "abc"++Rest vs Prefix++Rest is misleading: > > [$a, $b, $c | Rest] vs lists:append(Prefix, Rest) > > > > > > Same question with binaries: > > since we have a "prefix-match of binaries only providing PrefixSize at > > runtime" instruction, > > * Why don't we have one for lists? > > * And oh God why do we have to provide that PrefixSize "manually"? > > (binding the variable ourselves, when the compiler could do that itself) > > * Why isn't suffix-matching of binaries implemented yet? (how > > different to prefix-matching can it be?) > > > > I could never find the answer to all of these questions. > > WRT binaries my thinking is that they are actually a mix of "bytes" and > > references to binaries, > > making some crucial operations O(log n) instead of O(1)... but prefix > match > > exists... > > > > I really want to be able to write things like: > > <<"{", Name/binary, "}">> = PathToken > > > > %%=> Name = <<"id">> given PathToken = <<"{id}">> > > > > > > > > Cheers, > > -- > > Pierre Fenoll > > > > > > On 25 October 2017 at 09:27, Kostis Sagonas wrote: > > > > > On 10/25/2017 08:56 AM, Stefan Hellkvist wrote: > > > > > >> Is it because this syntactic sugar is transformed more or less as a > > >> preprocessing step where the value of Prefix needs to be known, > > >> > > > > > > Yes, that's the reason. It's a purely _syntactic_ thing. > > > > > > Kostis > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erdemaksu@REDACTED Wed Oct 25 17:04:49 2017 From: erdemaksu@REDACTED (Erdem Aksu) Date: Wed, 25 Oct 2017 17:04:49 +0200 Subject: [erlang-questions] sharing config via a dynamically compiled module? In-Reply-To: <4991037F-1F6D-4702-B023-72E62D7E33DB@gmail.com> References: <92C15A74-A693-4189-9FB0-76FFB3F7B8F1@gmail.com> <4991037F-1F6D-4702-B023-72E62D7E33DB@gmail.com> Message-ID: Hello Benoit, I am using following application: https://github.com/pundunlabs/gb_reg It keeps object code for the module in files for later recovery. It loads the latest object code from files on application start. One can share the object code(binary) in-between nodes and each node can load the module. (code:load_binary/3) There is no documentation, but usage is like following if you are interested: application:start(gb_reg), {ok, Module} = gb_reg:new(Name), %% See new/2 that gets an Init list of Key/Values. gb_reg:insert(Module, Key, Value), gb_reg:insert(Module, [{K1, V1},{K2, V2}]), %% File dumped once for all entries. Module:lookup(Key), gb_reg:delete(Module, Key), Module:entries(), %% Same as gb_reg:all(Module) gb_reg:add_keys(Module, [K]), %% Two entries K -> Ref and Ref -> K are created where Ref is auto incremented integer that is encoded as unsigned integer. gb_reg:purge(Module). Cons: Slow insertion. Keeping configurations is a good use case for this if they are not updated frequently. Pros: gen_server serialise addition/deletion of new entries thus the file access and code reload will be sequential. I use this in a wide column database implementation, where I lookup Column Id <-> Column Name at writes and reads to pack/unpack the data. And I use a similar approach for keeping key (hash) ring for fast lookup on db shards. Regards, Erdem On Wed, Oct 25, 2017 at 3:55 PM, Dmitry Kolesnikov wrote: > Hello, > > On 25 Oct 2017, at 16.27, Benoit Chesneau wrote: > > It seems lot of people are using ets to share a config generally. Maybe > becauseit feels a little hackish ? > > > The usage of code-as-a-config is ?valid? approach. I?ve used that for some > project. However, the reloading of config is something to consider. You > need to use code:purge(?) and take all pain associated with it. The > code-as-a-config works as build time config but runtime config is easy with > build-in application:get_env? > > > - Dmitry > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Wed Oct 25 18:06:25 2017 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 25 Oct 2017 18:06:25 +0200 Subject: [erlang-questions] How to downgrade a TLS connection to TCP without an TLS Alert exchange In-Reply-To: References: Message-ID: Hi! 2017-10-24 11:38 GMT+02:00 Rob A'Court : > Hi > > Is there a way to downgrade a TLS connection to TCP without an TLS Alert > exchange in Erlang/Elixir? > No! That would break the TLS protocol: FROM the RFC. "If the application protocol using TLS provides that any data may be carried over the underlying transport after the TLS connection is closed, the TLS implementation must receive the responding close_notify alert before indicating to the application layer that the TLS connection has ended. " To give a bit of context: We've implemented the MS-TDS protocol (used by > Microsoft SQL Server) using Elixir and the ssl and gen_tcp libraries. A TDS > connection typically is unencrypted apart from the login. It starts > unencrypted with a prelogin handshake, after that comes the TLS handshake, > and encrypted login packet is sent and then all further packets are > unencrypted. > > Using :ssl.close to downgrade the socket to TCP expects a TLS alert > exchange to end the SSL connection however TDS does not expect or support > TLS alerts, it simply goes back to being unencrypted after the login. I > appreciate this is not standard TLS but I'm not expecting to change > Microsoft's protocol to SQL Server any time soon! > > Currently we stop encryption by sending data directly to the TCP socket > and changing the controlling process of the TCP socket to our receiving > process. This works however it leaves the ssl_connection process hanging > around. It monitors our receiving process and if that process dies a TLS > alert is sent which TDS is not expecting. The alert is only sent in some > teardown scenarios but it just seems rather messy. We're wondering if > there's a better way to approach our problem? > Not any clean way that I can think of, but you might be a able to make an unclean one if you are willing to do what you describe above, which is not very clean. Regards Ingela Erlang/OTP Team - Ericsson AB > Many thanks > > Rob > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From z@REDACTED Wed Oct 25 18:12:29 2017 From: z@REDACTED (Danil Zagoskin) Date: Wed, 25 Oct 2017 19:12:29 +0300 Subject: [erlang-questions] "prefix" ++ Rest = Something In-Reply-To: References: Message-ID: One can implement different ways of matching string against prefix-as-binding: 7> Prefix = "hello ". "hello " 8> {Prefix, Rest} = lists:split(length(Prefix), "hello world"). {"hello ","world"} 9> Rest. "world" 10> Rest = lists:foldl(fun(C, [C|RestAcc]) -> RestAcc; (_, _) -> error(badmatch) end, "hello world", Prefix). "world" 11> lists:foldl(fun(C, [C|RestAcc]) -> RestAcc; (_, _) -> error(badmatch) end, "help world", Prefix). ** exception error: badmatch in function shell:apply_fun/3 (shell.erl, line 900) in call from lists:foldl/3 (lists.erl, line 1263) 12> MatchPref = fun MatchPref([], SRest) -> SRest; MatchPref([C|Pref], [C|SRest]) -> MatchPref(Pref, SRest); MatchPref(_, _) -> error(badmatch) end. #Fun 13> Rest = MatchPref(Prefix, "hello world"). "world" 14> Rest = MatchPref(Prefix, "help world"). ** exception error: badmatch So it should be possible to implement a syntactic sugar for Prefix ++ Rest = String. I suppose one can do that even for function/case clauses (but that would be more tricky). Seems like parse_transform would be enough for a proof-of-concept. In this case even no OTP patch is required. On Wed, Oct 25, 2017 at 9:56 AM, Stefan Hellkvist wrote: > Hi, > > Erlang has this syntactic sugar for matching string prefixes ( > http://erlang.org/doc/reference_manual/expressions.html#id80508) where > you can do: > > "prefix" ++ Rest = "prefixsomething" > > , which would bind Rest to "something" in this case. > > > I'm curious why however it is ok to do: > > 1> "prefix" ++ Rest = "prefixsomething". > "prefixsomething" > 2> Rest. > "something" > > > but it is not ok to do: > > 1> Prefix = "prefix". > "prefix" > 2> Prefix ++ Rest = "prefixsomething". > * 1: illegal pattern > > > Is it because this syntactic sugar is transformed more or less as a > preprocessing step where the value of Prefix needs to be known, or why else > is "Prefix ++ Rest = Something" not allowed even when Prefix is bound? > > /Stefan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.geib.44@REDACTED Wed Oct 25 21:12:45 2017 From: mark.geib.44@REDACTED (Mark Geib) Date: Wed, 25 Oct 2017 13:12:45 -0600 Subject: [erlang-questions] how to run rebar3 release with "-config /path/my_sys.config" Message-ID: I just recently migrated an erlang project to use rebar3 from rebar. With the generated rebar release I was able to start the application with ?/some_path/bin/my_app start -config ?/some_other_path/my_app.config? which allowed for dynamic configuration. With rebar3 I don?t seem to have the same behavior, it always uses /some_path/releases//sys.config included during the release build. Is there a way to something equivalent in rebar3.? Thanks, Mark. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 528 bytes Desc: Message signed with OpenPGP URL: From essen@REDACTED Wed Oct 25 21:23:43 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Wed, 25 Oct 2017 20:23:43 +0100 Subject: [erlang-questions] Patch Package OTP 20.1.3 Released In-Reply-To: <1508915516.2025.5.camel@ericsson.com> References: <1508915516.2025.5.camel@ericsson.com> Message-ID: <01fd0506-ec82-7ffe-0e72-12613b359ea2@ninenines.eu> On 10/25/2017 08:11 AM, John H?gberg wrote:> ?--------------------------------------------------------------------- > ?--- erts-9.1.3 ------------------------------------------------------ > ?--------------------------------------------------------------------- > > ?The erts-9.1.3 application can be applied independently of other > ?applications on a full OTP 20 installation. > > ?--- Fixed Bugs and Malfunctions --- > > ? OTP-14672????Application(s): erts > ???????????????Related Id(s): ERL-494 > > ???????????????Added zlib:set_controlling_process/2 to move a > ???????????????zstream() between processes. Thanks for that! I will try it as soon as possible. -- Lo?c Hoguin https://ninenines.eu From mark.geib.44@REDACTED Wed Oct 25 21:51:38 2017 From: mark.geib.44@REDACTED (Mark Geib) Date: Wed, 25 Oct 2017 13:51:38 -0600 Subject: [erlang-questions] how to run rebar3 release with "-config /path/my_sys.config" In-Reply-To: References: Message-ID: <7DED9706-F659-464E-B09B-3AF90BA4FF0D@gmail.com> After some more digging it looks like this works fine for starting the application in ?console? or ?foreground? mode, but not when I do a ?start?. Mark. > On Oct 25, 2017, at 1:12 PM, Mark Geib wrote: > > I just recently migrated an erlang project to use rebar3 from rebar. With the generated rebar release I was able to start the application with ?/some_path/bin/my_app start -config ?/some_other_path/my_app.config? which allowed for dynamic configuration. With rebar3 I don?t seem to have the same behavior, it always uses /some_path/releases//sys.config included during the release build. > > Is there a way to something equivalent in rebar3.? > > Thanks, > > Mark. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 528 bytes Desc: Message signed with OpenPGP URL: From ok@REDACTED Thu Oct 26 04:47:28 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 26 Oct 2017 15:47:28 +1300 Subject: [erlang-questions] "prefix" ++ Rest = Something In-Reply-To: References: <7f610c2e-72b2-4f32-afc5-d270bb774326@cs.ntua.gr> <20171025115309.GA50371@erix.ericsson.se> Message-ID: On 26/10/17 3:11 AM, Pierre Fenoll wrote: >> So this can not be described as a pattern. > > I'm saying let's fix that. > > Prefix ++ Rest as a pattern does not need to be calling ++/2. > Instead, I am suggesting to allow "partial matches": adding support for > these patterns. > What makes this hard? Why is this only sugar? (1) Consider Prefix ++ Rest = List where List has N elements. If Prefix and List are both unbound, there are N+1 solutions. If List is bound, there is at most one solution. If Prefix is bound, there is at most one solution. If Prefix is a list of patterns some of which are unbound, but the length of Prefix is fixed, there is at most one solution. > Why is a similar pattern allowed in binaries but not with lists? (the > <> pattern) Did you notice the :PrefixSize part? That has to be known. Otherwise it would have the same multiple-solutions problem. (2) "prefix" ++ Rest as a pattern _doesn't_ call ++/2. It is translated to [$p|[$r|[$e|[$f|[$i|[$x|Rest]]]]]] by the parser. (As noted above, the _elements_ of the list could perfectly well be any pattern, but the *spine* of the list must be visibly present.) It's rather like the way Haskell used to allow f 0 = 1 f (n+1) = f n * (n+1) but did not allow n+m as a pattern. Just how much of a problem _is_ this anyway? From bchesneau@REDACTED Thu Oct 26 13:17:54 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 26 Oct 2017 13:17:54 +0200 Subject: [erlang-questions] question: "Erlang literals are no longer copied when sending messages." Message-ID: With Erlang 20, "Erlang literals are no longer copied when sending messages" according to the changelog. Does it means something like #{ somelist => [], map => #{ } } will be returned from a gen_server call without beeing copied? (ie the content of somelist and somemap will only be referenced ? Benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Thu Oct 26 15:21:46 2017 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Thu, 26 Oct 2017 15:21:46 +0200 Subject: [erlang-questions] "prefix" ++ Rest = Something In-Reply-To: References: <7f610c2e-72b2-4f32-afc5-d270bb774326@cs.ntua.gr> <20171025115309.GA50371@erix.ericsson.se> Message-ID: I am not sure I understand your (1): we have been talking about the case where Prefix is bound, just not with a value known at compile time. Of course I noticed the :PrefixSize part, I have been writing about it for 3 emails now. So the :PrefixSize is needed, okay. Why isn't the compiler binding this variable automatically then? You know, write: <> = Bla (with Prefix bound) and the compiler should generate: PrefixSize = byte_size(Prefix), <> = Bla Obviously the compiler can do this. Now why doesn't it do that already? Then, let's implement the same matching for lists. And I believe the fact that a list's length is O(n) will not be an issue: you can already pattern match lists today. FYI your Haskell example is also a valid Erlang pattern match. See http://erlang.org/doc/reference_manual/expressions.html#id80508 section "Expressions in Patterns". > Just how much of a problem _is_ this anyway? I'm repeating the example I gave before: <<"{", Name/binary, "}">> = PathToken With latest Erlang/OTP (20.1) one has to write: <<${, Rest/binary>> = PathToken, <<$}, Eman/binary>> = binary:reverse(Rest), Name = binary:reverse(Eman) which wastes copying, creates garbage, is of probably worse complexity and uses a function that doesn't even exist. Yes, it is maybe time to add binary:reverse/1, from https://stackoverflow.com/a/43310493/1418165 probably. Further improvement to pattern matching: How about allowing matching non-local functions? case F of fun io:format/2 -> blip(); fun erlang:display/1 -> blop() end Of course this would only really be matching {M,F,Arity}. Cheers, -- Pierre Fenoll On 26 October 2017 at 04:47, Richard A. O'Keefe wrote: > > > On 26/10/17 3:11 AM, Pierre Fenoll wrote: > >> So this can not be described as a pattern. >>> >> >> I'm saying let's fix that. >> >> Prefix ++ Rest as a pattern does not need to be calling ++/2. >> Instead, I am suggesting to allow "partial matches": adding support for >> these patterns. >> What makes this hard? Why is this only sugar? >> > > (1) Consider Prefix ++ Rest = List where List has N elements. > If Prefix and List are both unbound, there are N+1 solutions. > If List is bound, there is at most one solution. > If Prefix is bound, there is at most one solution. > If Prefix is a list of patterns some of which are unbound, > but the length of Prefix is fixed, there is at most one solution. > > Why is a similar pattern allowed in binaries but not with lists? (the >> <> pattern) >> > > Did you notice the :PrefixSize part? That has to be known. > Otherwise it would have the same multiple-solutions problem. > > (2) "prefix" ++ Rest as a pattern _doesn't_ call ++/2. > It is translated to [$p|[$r|[$e|[$f|[$i|[$x|Rest]]]]]] > by the parser. (As noted above, the _elements_ of the list > could perfectly well be any pattern, but the *spine* of the > list must be visibly present.) > > It's rather like the way Haskell used to allow > f 0 = 1 > f (n+1) = f n * (n+1) > but did not allow n+m as a pattern. > > > Just how much of a problem _is_ this anyway? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Oct 26 16:39:07 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 26 Oct 2017 14:39:07 +0000 Subject: [erlang-questions] question: "Erlang literals are no longer copied when sending messages." In-Reply-To: References: Message-ID: The compiler detects literal constants. A function such as f() -> ["I", "am", "a", "literal", "constant", "valiue"]. will return a constant list. These lists go into a "constant heap" which means that other heaps can just reference those values directly from there. Prior to Erlang version 20.0, if you sent such a constant (locally) from one process to another, the send would copy the constant literal into the mailbox of the receiving process. Not anymore. The value is referenced from the constant heap. There is a subtlety when you load new code in which the old constants are correctly retained by the VM, but otherwise, things works seamlessly for programmers. Your example is somewhat degenerate because the empty list and empty map already has special representation in the VM. One way to detect if a value is a constant is to look at the output from the compiler. The module will have a special section for constant literals, and you can verify that the compiler indeed did detect a value as being a constant. In general, constants are congruences in the sense that compounding structures are constants if all their children are in the term tree. On Thu, Oct 26, 2017 at 1:18 PM Benoit Chesneau wrote: > With Erlang 20, "Erlang literals are no longer copied when sending > messages" according to the changelog. > Does it means something like #{ somelist => [], map => #{ } } will be > returned from a gen_server call without beeing copied? (ie the content of > somelist and somemap will only be referenced ? > > Benoit > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Thu Oct 26 17:30:23 2017 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 26 Oct 2017 17:30:23 +0200 Subject: [erlang-questions] map elements in defined order Message-ID: Looking at the maps API, I see no function that can present the contents of a map in the defined order. Because there IS a defined order, described in the Erlang Reference Manual: ?Maps are ordered by size, two maps with the same size are compared by keys in ascending term order and then by values in key order. In maps key order integers types are considered less than floats types.? Note that the key ordering described above doesn't follow normal Erlang term ordering, so lists:sort(maps:to_list(Map)) will produce something will a well-defined order, but comparing the sorted lists of two maps is not guaranteed to have the same outcome as comparing the maps themselves. I can solve this by using lists:sort/2, but this seems contrived. To clarify, e.g. msgpack:pack(Map) will perform a maps:to_list(Map). This is not guaranteed to produce the same result across Erlang versions, since maps:to_list/1 says pairs "are returned in arbitrary order". Yet there is no alternative API function that will return pairs in the defined sort order, although presumably there is an efficient internal implementation for producing it. Wouldn't it be reasonable to have such a function? BR, Ulf W -------------- next part -------------- An HTML attachment was scrubbed... URL: From luke@REDACTED Thu Oct 26 16:10:09 2017 From: luke@REDACTED (Luke Bakken) Date: Thu, 26 Oct 2017 07:10:09 -0700 Subject: [erlang-questions] how to run rebar3 release with "-config /path/my_sys.config" Message-ID: I did some investigation into the runner script that ships with relx and compared it to rebar's. The end result is this PR: https://github.com/erlware/relx/pull/617 Thanks, Luke > After some more digging it looks like this works fine for starting the application in ?console? or ?foreground? mode, but not when I do a ?start?. > > Mark. > > > On Oct 25, 2017, at 1:12 PM, Mark Geib <> wrote: > > > > I just recently migrated an erlang project to use rebar3 from rebar. With the generated rebar release I was able to start the application with ?/some_path/bin/my_app start -config ?/some_other_path/my_app.config? which allowed for dynamic configuration. With rebar3 I don?t seem to have the same behavior, it always uses /some_path/releases//sys.config included during the release build. > > > > Is there a way to something equivalent in rebar3.? > > > > Thanks, > > > > Mark. From sunboshan@REDACTED Thu Oct 26 18:03:30 2017 From: sunboshan@REDACTED (Boshan Sun) Date: Thu, 26 Oct 2017 09:03:30 -0700 Subject: [erlang-questions] question: "Erlang literals are no longer copied when sending messages." In-Reply-To: References: Message-ID: Does those literals lives in the constant heap has a ref counter, as the refc binary? If not, how do they get garbage collected? If so, will those constant suffers same "memory leak" issue has refc binary?(a process touched lots of refc binary but never get garbaged collected, so those refc binaries cannot be garbage collected for a long time) Thanks! 2017-10-26 7:39 GMT-07:00 Jesper Louis Andersen < jesper.louis.andersen@REDACTED>: > The compiler detects literal constants. A function such as > > f() -> ["I", "am", "a", "literal", "constant", "valiue"]. > > will return a constant list. These lists go into a "constant heap" which > means that other heaps can just reference those values directly from there. > Prior to Erlang version 20.0, if you sent such a constant (locally) from > one process to another, the send would copy the constant literal into the > mailbox of the receiving process. Not anymore. The value is referenced from > the constant heap. There is a subtlety when you load new code in which the > old constants are correctly retained by the VM, but otherwise, things works > seamlessly for programmers. > > Your example is somewhat degenerate because the empty list and empty map > already has special representation in the VM. One way to detect if a value > is a constant is to look at the output from the compiler. The module will > have a special section for constant literals, and you can verify that the > compiler indeed did detect a value as being a constant. In general, > constants are congruences in the sense that compounding structures are > constants if all their children are in the term tree. > > > On Thu, Oct 26, 2017 at 1:18 PM Benoit Chesneau > wrote: > >> With Erlang 20, "Erlang literals are no longer copied when sending >> messages" according to the changelog. >> Does it means something like #{ somelist => [], map => #{ } } will be >> returned from a gen_server call without beeing copied? (ie the content of >> somelist and somemap will only be referenced ? >> >> Benoit >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Thu Oct 26 18:20:19 2017 From: roger@REDACTED (Roger Lipscombe) Date: Thu, 26 Oct 2017 17:20:19 +0100 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: On 26 October 2017 at 16:30, Ulf Wiger wrote: > Wouldn't it be reasonable to have such a function? First thought: No, because people would start abusing it. Maps *aren't* ordered. From roger@REDACTED Thu Oct 26 18:22:09 2017 From: roger@REDACTED (Roger Lipscombe) Date: Thu, 26 Oct 2017 17:22:09 +0100 Subject: [erlang-questions] how to run rebar3 release with "-config /path/my_sys.config" In-Reply-To: References: Message-ID: There's also the RELX_CONFIG_PATH environment variable: https://github.com/erlware/relx/commit/60dda02625d96c0473dcd03ecf1cea2c1600af18 On 26 October 2017 at 15:10, Luke Bakken wrote: > I did some investigation into the runner script that ships with relx > and compared it to rebar's. The end result is this PR: > > https://github.com/erlware/relx/pull/617 > > Thanks, > Luke > >> After some more digging it looks like this works fine for starting the application in ?console? or ?foreground? mode, but not when I do a ?start?. >> >> Mark. >> >> > On Oct 25, 2017, at 1:12 PM, Mark Geib <> wrote: >> > >> > I just recently migrated an erlang project to use rebar3 from rebar. With the generated rebar release I was able to start the application with ?/some_path/bin/my_app start -config ?/some_other_path/my_app.config? which allowed for dynamic configuration. With rebar3 I don?t seem to have the same behavior, it always uses /some_path/releases//sys.config included during the release build. >> > >> > Is there a way to something equivalent in rebar3.? >> > >> > Thanks, >> > >> > Mark. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From sverker.eriksson@REDACTED Thu Oct 26 19:10:34 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Thu, 26 Oct 2017 19:10:34 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: On 10/26/2017 05:30 PM, Ulf Wiger wrote: > Looking at the maps API, I see no function that can present the > contents of a map in the defined order. > > Because there IS a defined order, described in the Erlang Reference > Manual: > > ?Maps are ordered by size, two maps with the same size are compared by > keys in ascending term order and then by values in key order. In maps > key order integers types are considered less than floats types.? > > Note that the key ordering described above doesn't follow normal > Erlang term ordering, so lists:sort(maps:to_list(Map)) will produce > something will a well-defined order, but comparing the sorted lists of > two maps is not guaranteed to have the same outcome as comparing the > maps themselves. > > I can solve this by using lists:sort/2, but this seems contrived. > > To clarify, e.g. msgpack:pack(Map) will perform a maps:to_list(Map). > This is not guaranteed to produce the same result across Erlang > versions, since maps:to_list/1 says pairs "are returned in arbitrary > order". > > Yet there is no alternative API function that will return pairs in the > defined sort order, although presumably there is an efficient internal > implementation for producing it. > There is an efficient internal implementation to compare two terms in map-key order (undocumented erts_internal:cmp_term/2). But there is no efficient implementation to produce a map-key-value ordered list. That would require N*log(N) sorting. '==' for hashmaps (> 32 keys) does a one pass iteration in key hash order over both trees (HAMTs) and keeps track of the minimal key or value seen so far. /Sverker From ulf@REDACTED Thu Oct 26 20:07:08 2017 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 26 Oct 2017 20:07:08 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: But they *are* ordered. Otherwise, comparison of two maps would be undefined. BR, Ulf W Den 26 okt. 2017 18:20 skrev "Roger Lipscombe" : On 26 October 2017 at 16:30, Ulf Wiger wrote: > Wouldn't it be reasonable to have such a function? First thought: No, because people would start abusing it. Maps *aren't* ordered. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Thu Oct 26 20:13:39 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Thu, 26 Oct 2017 20:13:39 +0200 Subject: [erlang-questions] question: "Erlang literals are no longer copied when sending messages." In-Reply-To: References: Message-ID: <278452a5-215b-bebf-0f7b-c2f6d9054159@ericsson.com> On 10/26/2017 06:03 PM, Boshan Sun wrote: > Does those literals lives in the constant heap has a ref counter, as > the refc binary? If not, how do they get garbage collected? If so, > will those constant suffers same "memory leak" issue has refc > binary?(a process touched lots of refc binary but never get garbaged > collected, so those refc binaries cannot be garbage collected for a > long time) > No, literals are not ref counted. All literals belong to a loaded module instance. When that (old) module instance is purged, all process heaps are scanned and heaps containing those literals will do a special garbage collection where the literals are copied. /Sverker -------------- next part -------------- An HTML attachment was scrubbed... URL: From kjnilsson@REDACTED Thu Oct 26 20:28:41 2017 From: kjnilsson@REDACTED (Karl Nilsson) Date: Thu, 26 Oct 2017 18:28:41 +0000 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: Doesn?t it just mean that for any two maps with the same keyvalues the internal order is the same for a given erlang version? That should be sufficient for comparison but unlikely be useful for anything else. On Thu, 26 Oct 2017 at 19:07, Ulf Wiger wrote: > But they *are* ordered. Otherwise, comparison of two maps would be > undefined. > > BR, > Ulf W > > > Den 26 okt. 2017 18:20 skrev "Roger Lipscombe" : > > On 26 October 2017 at 16:30, Ulf Wiger wrote: > > Wouldn't it be reasonable to have such a function? > > First thought: No, because people would start abusing it. Maps *aren't* > ordered. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Oct 26 20:31:43 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 26 Oct 2017 18:31:43 +0000 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: It is generally a bad idea to rely on order in anything which uses a hash function. The hash function is often subject to change---rather quickly I might add if it proves to be a security bug. Picking a family of hashes and seeding it randomly is usually a good trick. Our "sister language" Go *randomizes* iteration order on its maps. This is to force programmers into not relying on the map order at all, even if it happens to be ordered right now. This opens up implementations in the future. If you wanted order in a map, it would be *far* better if you could create a map based on RB-trees or the like. Those are naturally ordered by structure. OCaml, for instance, defines Hash Tables as well as Maps. The latter is the ordered variant. On Thu, Oct 26, 2017 at 8:07 PM Ulf Wiger wrote: > But they *are* ordered. Otherwise, comparison of two maps would be > undefined. > > BR, > Ulf W > > > Den 26 okt. 2017 18:20 skrev "Roger Lipscombe" : > > On 26 October 2017 at 16:30, Ulf Wiger wrote: > > Wouldn't it be reasonable to have such a function? > > First thought: No, because people would start abusing it. Maps *aren't* > ordered. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Thu Oct 26 21:13:07 2017 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 26 Oct 2017 21:13:07 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: But again, Jesper, just about everyone relies on the fact that maps follow the general principle that there is a well-defined term comparison order. Otherwise, maps would be highly unsuitable to use in keys, and generally treacherous to use as a replacement for records. Following the Principle of Least Surprise, it's a darned good thing that Erlang doesn't randomize the key order in its maps. I doubt that anyone would abuse an extra function that produces the map element pairs in the internally defined sort order, given that the documentation would clearly state that it's more expensive than maps:to_list/1 (though likely faster than lists:sort(maps:to_list(M), fun custom_sort_fun/2) - not to mention less error-prone.) But it's not a feature I'm willing to go to war over. If no one else sees a use for it, I'm willing to concede that it has low priority. :) BR, Ulf W 2017-10-26 20:31 GMT+02:00 Jesper Louis Andersen < jesper.louis.andersen@REDACTED>: > It is generally a bad idea to rely on order in anything which uses a hash > function. The hash function is often subject to change---rather quickly I > might add if it proves to be a security bug. Picking a family of hashes and > seeding it randomly is usually a good trick. > > Our "sister language" Go *randomizes* iteration order on its maps. This is > to force programmers into not relying on the map order at all, even if it > happens to be ordered right now. This opens up implementations in the > future. > > If you wanted order in a map, it would be *far* better if you could create > a map based on RB-trees or the like. Those are naturally ordered by > structure. OCaml, for instance, defines Hash Tables as well as Maps. The > latter is the ordered variant. > > > On Thu, Oct 26, 2017 at 8:07 PM Ulf Wiger wrote: > >> But they *are* ordered. Otherwise, comparison of two maps would be >> undefined. >> >> BR, >> Ulf W >> >> >> Den 26 okt. 2017 18:20 skrev "Roger Lipscombe" : >> >> On 26 October 2017 at 16:30, Ulf Wiger wrote: >> > Wouldn't it be reasonable to have such a function? >> >> First thought: No, because people would start abusing it. Maps *aren't* >> ordered. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Thu Oct 26 21:42:40 2017 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Thu, 26 Oct 2017 21:42:40 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: > Our "sister language" Go *randomizes* iteration order on its maps. This is > to force programmers into not relying on the map order at all, even if it > happens to be ordered right now. This opens up implementations in the > future. > Randomizing how elements are stored/hashed is also useful to avoid hash collision attacks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Thu Oct 26 21:43:39 2017 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 26 Oct 2017 12:43:39 -0700 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: <59F23AEB.9040405@gmail.com> On 10/26/2017 12:13 PM, Ulf Wiger wrote: > But again, Jesper, just about everyone relies on the fact that maps follow the general principle that there is a well-defined term comparison order. Otherwise, maps would be highly unsuitable to use in keys, and generally treacherous to use as a replacement for records. Following the Principle of Least Surprise, it's a darned good thing that Erlang doesn't randomize the key order in its maps. > > I doubt that anyone would abuse an extra function that produces the map element pairs in the internally defined sort order, given that the documentation would clearly state that it's more expensive than maps:to_list/1 (though likely faster than lists:sort(maps:to_list(M), fun custom_sort_fun/2) - not to mention less error-prone.) > > But it's not a feature I'm willing to go to war over. If no one else sees a use for it, I'm willing to concede that it has low priority. :) > At a low-level, a hash array mapped trie is unordered because it relies on a hash function, so using the Erlang map in an ordered way might be possible but would be inefficient. If you need an ordered mapping, I think my Erlang trie is a better option at https://github.com/okeuday/trie , though it requires Erlang string keys (binary keys are supported by the btrie module, but that is slow). That doesn't necessarily help msgpack source code though, unless it would be a separate output option that was added. Best Regards, Michael From ulf@REDACTED Thu Oct 26 22:33:22 2017 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 26 Oct 2017 22:33:22 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: <59F23AEB.9040405@gmail.com> References: <59F23AEB.9040405@gmail.com> Message-ID: There are some things in Erlang that people have come to rely on, but for which there's a price. Having a globally defined sort order is such a thing. And maps must be implemented in such a way that the following is not only supported, but efficient: comp(#{} = A, #{} = B) when A > B -> greater; ... That is, maps must behave, efficiently, as an ordered mapping on request. I've had reason to ponder the Erlang term comparison semantics while maintaining the sext library. The ordering semantics of maps threw a wrench into the works, with its non-standard ordering of keys (I should have left some room between the type tags for such unexpected developments.) The jury (i.e. QuickCheck) is still out on whether I'll be able to support it with the existing tag scheme. Now, for sext, maps are at least better than refs, pids and ports, since there is no documentation on how they are sorted internally. As it turns out (for now), using term_to_binary() and some bit syntax seems to work in practice for those. For the external term format for maps, actually, the documentation says: ?Key and value pairs (Ki => Vi) are encoded in section Pairs in the following order: K1, V1, K2, V2,..., Kn, Vn. ? ... which does seem to imply that the ordering in the external term format is defined. I assume this is not the intended meaning, but rather that whichever key K1 is, the corresponding value V1 is the item that follows it, and so on. Anyway, I've come across code that uses maps as a serialization format for a cryptographic hash. Based on what is known about maps, this would seem like a Bad Idea, since in that case, presumably, the result of the hash operation might change between implementations. BR, Ulf W Den 26 okt. 2017 21:43 skrev "Michael Truog" : On 10/26/2017 12:13 PM, Ulf Wiger wrote: > But again, Jesper, just about everyone relies on the fact that maps follow > the general principle that there is a well-defined term comparison order. > Otherwise, maps would be highly unsuitable to use in keys, and generally > treacherous to use as a replacement for records. Following the Principle of > Least Surprise, it's a darned good thing that Erlang doesn't randomize the > key order in its maps. > > I doubt that anyone would abuse an extra function that produces the map > element pairs in the internally defined sort order, given that the > documentation would clearly state that it's more expensive than > maps:to_list/1 (though likely faster than lists:sort(maps:to_list(M), fun > custom_sort_fun/2) - not to mention less error-prone.) > > But it's not a feature I'm willing to go to war over. If no one else sees > a use for it, I'm willing to concede that it has low priority. :) > > At a low-level, a hash array mapped trie is unordered because it relies on a hash function, so using the Erlang map in an ordered way might be possible but would be inefficient. If you need an ordered mapping, I think my Erlang trie is a better option at https://github.com/okeuday/trie , though it requires Erlang string keys (binary keys are supported by the btrie module, but that is slow). That doesn't necessarily help msgpack source code though, unless it would be a separate output option that was added. Best Regards, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Thu Oct 26 22:49:23 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 26 Oct 2017 22:49:23 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: <1B98F60F-074C-4E58-9522-412EEE9FD1B3@gmail.com> > On 26 Oct 2017, at 20:31, Jesper Louis Andersen wrote: > > It is generally a bad idea to rely on order in anything which uses a hash function. The hash function is often subject to change---rather quickly I might add if it proves to be a security bug. Picking a family of hashes and seeding it randomly is usually a good trick. > > Our "sister language" Go *randomizes* iteration order on its maps. This is to force programmers into not relying on the map order at all, even if it happens to be ordered right now. This opens up implementations in the future. > > If you wanted order in a map, it would be *far* better if you could create a map based on RB-trees or the like. Those are naturally ordered by structure. OCaml, for instance, defines Hash Tables as well as Maps. The latter is the ordered variant. > > > On Thu, Oct 26, 2017 at 8:07 PM Ulf Wiger > wrote: > But they *are* ordered. Otherwise, comparison of two maps would be undefined. > > BR, > Ulf W > > > Den 26 okt. 2017 18:20 skrev "Roger Lipscombe" >: > On 26 October 2017 at 16:30, Ulf Wiger > wrote: > > Wouldn't it be reasonable to have such a function? > > First thought: No, because people would start abusing it. Maps *aren't* ordered. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions I guess if at least `maps:to_list/1` would return the keys in order it would be already a benefit. Another usage of such order is when you want to sign the object to compare with others across the network. I have such usage when I?m using maps as a representation for JSON. - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From silviu.cpp@REDACTED Thu Oct 26 23:37:23 2017 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Fri, 27 Oct 2017 00:37:23 +0300 Subject: [erlang-questions] eheap_alloc Message-ID: Hello, I see suddenly one of my erlang apps memory usage increased from 5 GB to 15 GB without no load or useful logs. After some debugging I see that the 10 GB leaked are allocated by eheap_alloc, but I cannot stop what process allocated that. My question is : NIF library leaks can show the memory under eheap_alloc or I can safely assume that there is no NIF leak here Any other idea to stop what I can do to identify the issue is welcome Silviu -------------- next part -------------- An HTML attachment was scrubbed... URL: From dch@REDACTED Fri Oct 27 00:01:50 2017 From: dch@REDACTED (Dave Cottlehuber) Date: Fri, 27 Oct 2017 00:01:50 +0200 Subject: [erlang-questions] how to run rebar3 release with "-config /path/my_sys.config" In-Reply-To: References: Message-ID: <1509055310.927794.1152338832.092B023C@webmail.messagingengine.com> > > > I just recently migrated an erlang project to use rebar3 from rebar. With the generated rebar release I was able to start the application with ?/some_path/bin/my_app start -config ?/some_other_path/my_app.config? which allowed for dynamic configuration. With rebar3 I don?t seem to have the same behavior, it always uses /some_path/releases//sys.config included during the release build. > > > > > > Is there a way to something equivalent in rebar3.? try putting it in ERL_AFLAGS instead? env ERL_AFLAGS="-config /usr/local/etc/myapp/sys.config" ./bin/myapps console let me know if that works. A similar variant is in use here. A+ Dave From icfp.publicity@REDACTED Fri Oct 27 03:37:19 2017 From: icfp.publicity@REDACTED (Lindsey Kuper) Date: Thu, 26 Oct 2017 18:37:19 -0700 Subject: [erlang-questions] Call for Participation: ICFP 2017 Message-ID: <59f28dcfd15bb_59703fcb06059be47473b@landin.local.mail> CALL FOR WORKSHOP AND CO-LOCATED EVENT PROPOSALS ICFP 2018 23rd ACM SIGPLAN International Conference on Functional Programming September 23-29, 2018 St. Louis, Missouri, United States http://conf.researchr.org/home/icfp-2018 The 23rd ACM SIGPLAN International Conference on Functional Programming will be held in St. Louis, Missouri, United States on September 23-29, 2018. ICFP provides a forum for researchers and developers to hear about the latest work on the design, implementations, principles, and uses of functional programming. Proposals are invited for workshops (and other co-located events, such as tutorials) to be affiliated with ICFP 2018 and sponsored by SIGPLAN. These events should be less formal and more focused than ICFP itself, include sessions that enable interaction among the attendees, and foster the exchange of new ideas. The preference is for one-day events, but other schedules can also be considered. The workshops are scheduled to occur on September 23 (the day before ICFP) and September 27-29 (the three days after ICFP). ---------------------------------------------------------------------- Submission details Deadline for submission: November 20, 2017 Notification of acceptance: December 18, 2017 Prospective organizers of workshops or other co-located events are invited to submit a completed workshop proposal form in plain text format to the ICFP 2017 workshop co-chairs (Christophe Scholliers and David Christiansen), via email to icfp-workshops-2018@REDACTED by November 20, 2017. (For proposals of co-located events other than workshops, please fill in the workshop proposal form and just leave blank any sections that do not apply.) Please note that this is a firm deadline. Organizers will be notified if their event proposal is accepted by December 18, 2017, and if successful, depending on the event, they will be asked to produce a final report after the event has taken place that is suitable for publication in SIGPLAN Notices. The proposal form is available at: http://www.icfpconference.org/icfp2018-files/icfp18-workshops-form.txt Further information about SIGPLAN sponsorship is available at: http://www.sigplan.org/Resources/Proposals/Sponsored/ ---------------------------------------------------------------------- Selection committee The proposals will be evaluated by a committee comprising the following members of the ICFP 2018 organizing committee, together with the members of the SIGPLAN executive committee. Workshop Co-Chair: Christophe Scholliers (University of Ghent) Workshop Co-Chair: David Christiansen (Indiana University) General Chair: Robby Findler (Northwestern University) Program Chair: Matthew Flatt (University of Utah) ---------------------------------------------------------------------- Further information Any queries should be addressed to the workshop co-chairs (Christophe Scholliers and David Christiansen), via email to icfp-workshops-2018@REDACTED From icfp.publicity@REDACTED Fri Oct 27 03:40:21 2017 From: icfp.publicity@REDACTED (Lindsey Kuper) Date: Thu, 26 Oct 2017 18:40:21 -0700 Subject: [erlang-questions] Call for Workshop Proposals: ICFP 2018 Message-ID: <59f28e8514101_59753fc3c5457be4514ea@landin.local.mail> [ Please disregard previous version sent with the wrong subject line. ] CALL FOR WORKSHOP AND CO-LOCATED EVENT PROPOSALS ICFP 2018 23rd ACM SIGPLAN International Conference on Functional Programming September 23-29, 2018 St. Louis, Missouri, United States http://conf.researchr.org/home/icfp-2018 The 23rd ACM SIGPLAN International Conference on Functional Programming will be held in St. Louis, Missouri, United States on September 23-29, 2018. ICFP provides a forum for researchers and developers to hear about the latest work on the design, implementations, principles, and uses of functional programming. Proposals are invited for workshops (and other co-located events, such as tutorials) to be affiliated with ICFP 2018 and sponsored by SIGPLAN. These events should be less formal and more focused than ICFP itself, include sessions that enable interaction among the attendees, and foster the exchange of new ideas. The preference is for one-day events, but other schedules can also be considered. The workshops are scheduled to occur on September 23 (the day before ICFP) and September 27-29 (the three days after ICFP). ---------------------------------------------------------------------- Submission details Deadline for submission: November 20, 2017 Notification of acceptance: December 18, 2017 Prospective organizers of workshops or other co-located events are invited to submit a completed workshop proposal form in plain text format to the ICFP 2017 workshop co-chairs (Christophe Scholliers and David Christiansen), via email to icfp-workshops-2018@REDACTED by November 20, 2017. (For proposals of co-located events other than workshops, please fill in the workshop proposal form and just leave blank any sections that do not apply.) Please note that this is a firm deadline. Organizers will be notified if their event proposal is accepted by December 18, 2017, and if successful, depending on the event, they will be asked to produce a final report after the event has taken place that is suitable for publication in SIGPLAN Notices. The proposal form is available at: http://www.icfpconference.org/icfp2018-files/icfp18-workshops-form.txt Further information about SIGPLAN sponsorship is available at: http://www.sigplan.org/Resources/Proposals/Sponsored/ ---------------------------------------------------------------------- Selection committee The proposals will be evaluated by a committee comprising the following members of the ICFP 2018 organizing committee, together with the members of the SIGPLAN executive committee. Workshop Co-Chair: Christophe Scholliers (University of Ghent) Workshop Co-Chair: David Christiansen (Indiana University) General Chair: Robby Findler (Northwestern University) Program Chair: Matthew Flatt (University of Utah) ---------------------------------------------------------------------- Further information Any queries should be addressed to the workshop co-chairs (Christophe Scholliers and David Christiansen), via email to icfp-workshops-2018@REDACTED From ok@REDACTED Fri Oct 27 04:04:53 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 27 Oct 2017 15:04:53 +1300 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: <11a6bcc6-8fc7-0863-98de-ac071fdcc8b0@cs.otago.ac.nz> The current thread has left me somewhat confused. Suppose E1 and E2 are different computations, delivering maps M1 and M2 respectively, such that M1 =:= M2 is true. Are we guaranteed that term_to_binary(M1) and term_to_binary(M2) are equal binaries? From zxq9@REDACTED Fri Oct 27 06:01:31 2017 From: zxq9@REDACTED (zxq9) Date: Fri, 27 Oct 2017 13:01:31 +0900 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: <2046414.yBd3MWPVTy@changa> On 2017?10?26? ??? 21:13:07 Ulf Wiger wrote: > ... to use as a replacement for records. But they are IN NO WAY a replacement for records. -Craig From antoine.koener@REDACTED Fri Oct 27 08:58:06 2017 From: antoine.koener@REDACTED (Antoine Koener) Date: Fri, 27 Oct 2017 08:58:06 +0200 Subject: [erlang-questions] how to run rebar3 release with "-config /path/my_sys.config" In-Reply-To: <1509055310.927794.1152338832.092B023C@webmail.messagingengine.com> References: <1509055310.927794.1152338832.092B023C@webmail.messagingengine.com> Message-ID: >From my own tests, and if I remind correctly, the extended script provided in the bin/ directory is, at first, look in the current directory for a sys.config file, then only in the release directory. Have a look in the bin/ script to see. Hope that helps :-) On Fri, Oct 27, 2017 at 12:01 AM, Dave Cottlehuber wrote: > > > > I just recently migrated an erlang project to use rebar3 from rebar. > With the generated rebar release I was able to start the application with > ?/some_path/bin/my_app start -config ?/some_other_path/my_app.config? > which allowed for dynamic configuration. With rebar3 I don?t seem to have > the same behavior, it always uses /some_path/releases//sys.config > included during the release build. > > > > > > > > Is there a way to something equivalent in rebar3.? > > try putting it in ERL_AFLAGS instead? > > env ERL_AFLAGS="-config /usr/local/etc/myapp/sys.config" ./bin/myapps > console > > let me know if that works. A similar variant is in use here. > > A+ > Dave > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri Oct 27 09:25:31 2017 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 27 Oct 2017 09:25:31 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: <2046414.yBd3MWPVTy@changa> References: <2046414.yBd3MWPVTy@changa> Message-ID: Actually, they ARE, in some ways. I should have been clearer. I was, of course, referring to the uses of records where they were never optimal in the first place, but were tolerated e.g. because of the great convenience of referencing elements by name in pattern-matching. Quoted from EEP-0043: ?The idea was a data-type, a syntax-aware mapping of key-value associations with pattern matching. A syntax similar to records but without the hassle of compile-time dependency and with arbitrary terms as keys. Order was not important and it could be implemented with a Hash-Array-Mapped-Trie with good performance and memory trade-offs. This was a different approach than to replace records. It was meant to replace records where suitable and in other regards not be a replacement but its own _thing_.? Further, relevant to this discussion: ?A restriction set on the implementation by the Erlang specification is that order is total, i.e. satisfies _antisymmetry, transitivity and totality_. ... - Ordered maps impose restrictions on the underlying implementation and a hashing approach will be nearly impossible. - The underlying structure does not need to be sorted, an order could be produced when needed,? The thing I tried to highlight was that "an order could be produced when needed" is actually a bit more involved than you'd think, given that no API function does it for you. At least if you want to reflect the internal sort order - given that the actual implementation is NOT as described in the EEP: "If a key or value differs, the order of the respective terms, in Erlang term order". You'd have to write your own sort function, in which you visit all elements of complex structures, finding all instances of floats and ensuring that they get the right priority. Or... you simply do this: lists:sort(fun(A, B) -> #{A => 0} =< #{B => 0} end, maps:to_list(Map)) As evidenced e.g. by Benoit's comment above, I believe lots of people already rely on the order of map elements being IN SOME WAY stable. This is likely a brittle assumption, but one that may - in time - trap the OTP team into having to preserve the current APPARENT order. And the apparent order does appear to be sorted, as long as only maps of size =< 32 ("flatmaps") are inspected. For larger maps ("hashmaps"), the order of elements returned by maps:to_list/1 indeed appears arbitrary. BR, Ulf W 2017-10-27 6:01 GMT+02:00 zxq9 : > On 2017?10?26? ??? 21:13:07 Ulf Wiger wrote: > > ... to use as a replacement for records. > > But they are IN NO WAY a replacement for records. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Fri Oct 27 10:38:46 2017 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Fri, 27 Oct 2017 10:38:46 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: <2046414.yBd3MWPVTy@changa> Message-ID: 2017-10-27 9:25 GMT+02:00 Ulf Wiger : [...] > As evidenced e.g. by Benoit's comment above, I believe lots of people > already rely on the order of map elements being IN SOME WAY stable. This is > likely a brittle assumption, but one that may - in time - trap the OTP team > into having to preserve the current APPARENT order. > > And the apparent order does appear to be sorted, as long as only maps of > size =< 32 ("flatmaps") are inspected. For larger maps ("hashmaps"), the > order of elements returned by maps:to_list/1 indeed appears arbitrary. I already run into a bug where the code expected the map to be sorted. When the 33rd element was added, it started to show pathological behavior. From carlsson.richard@REDACTED Fri Oct 27 10:49:43 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 27 Oct 2017 10:49:43 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: <2046414.yBd3MWPVTy@changa> Message-ID: My interpretation of the reference manual is that maps compare to each other like the tuples created by this transformation: t(Map) -> {Ks,Vs} = lists:unzip(lists:keysort(1, [{{if is_integer(K) -> 0; is_float(K) -> 1; true -> 2 end, K}, V} || {K,V} <- maps:to_list(M)])), list_to_tuple(Ks ++ Vs). For example: t(#{65 => "A", 2.71 => e, 1.0e308 => alot, pi => 3.14, [] => 0}) yields {{0,65}, {1,2.71}, {1,1.0e308}, {2,pi}, {2,[]}, "A", e, alot, 3.14, 0} Which: 1) should be independent of the underlying implementation and choice of hash function, and thus stable across future versions of Erlang; 2) is cumbersome to express using the normal term order over the basic data types, since float keys always sort higher than integers (not really following the rule of least surprise). /Richard 2017-10-27 9:25 GMT+02:00 Ulf Wiger : > Actually, they ARE, in some ways. > > I should have been clearer. I was, of course, referring to the uses of > records where they were never optimal in the first place, but were > tolerated e.g. because of the great convenience of referencing elements by > name in pattern-matching. > > Quoted from EEP-0043: > > ?The idea was a data-type, a syntax-aware mapping of key-value > associations with pattern matching. A syntax similar to records but without > the hassle of compile-time dependency and with arbitrary terms as keys. > Order was not important and it could be implemented with a > Hash-Array-Mapped-Trie with good performance and memory trade-offs. This > was a different approach than to replace records. It was meant to replace > records where suitable and in other regards not be a replacement but its > own _thing_.? > > Further, relevant to this discussion: > > ?A restriction set on the implementation by the Erlang specification is > that order is total, i.e. satisfies _antisymmetry, transitivity and > totality_. > ... > - Ordered maps impose restrictions on the underlying implementation and a > hashing approach will be nearly impossible. > - The underlying structure does not need to be sorted, an order could be > produced when needed,? > > The thing I tried to highlight was that "an order could be produced when > needed" is actually a bit more involved than you'd think, given that no API > function does it for you. At least if you want to reflect the internal sort > order - given that the actual implementation is NOT as described in the > EEP: "If a key or value differs, the order of the respective terms, in > Erlang term order". > > You'd have to write your own sort function, in which you visit all > elements of complex structures, finding all instances of floats and > ensuring that they get the right priority. > > Or... you simply do this: > > lists:sort(fun(A, B) -> #{A => 0} =< #{B => 0} end, maps:to_list(Map)) > > > As evidenced e.g. by Benoit's comment above, I believe lots of people > already rely on the order of map elements being IN SOME WAY stable. This is > likely a brittle assumption, but one that may - in time - trap the OTP team > into having to preserve the current APPARENT order. > > And the apparent order does appear to be sorted, as long as only maps of > size =< 32 ("flatmaps") are inspected. For larger maps ("hashmaps"), the > order of elements returned by maps:to_list/1 indeed appears arbitrary. > > BR, > Ulf W > > 2017-10-27 6:01 GMT+02:00 zxq9 : > >> On 2017?10?26? ??? 21:13:07 Ulf Wiger wrote: >> > ... to use as a replacement for records. >> >> But they are IN NO WAY a replacement for records. >> >> -Craig >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Oct 27 12:56:36 2017 From: zxq9@REDACTED (zxq9) Date: Fri, 27 Oct 2017 19:56:36 +0900 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: <4467466.eSNuECovTi@changa> On 2017?10?27? ??? 10:38:46 Attila Rajmund Nohl wrote: > 2017-10-27 9:25 GMT+02:00 Ulf Wiger : > [...] > > As evidenced e.g. by Benoit's comment above, I believe lots of people > > already rely on the order of map elements being IN SOME WAY stable. This is > > likely a brittle assumption, but one that may - in time - trap the OTP team > > into having to preserve the current APPARENT order. > > > > And the apparent order does appear to be sorted, as long as only maps of > > size =< 32 ("flatmaps") are inspected. For larger maps ("hashmaps"), the > > order of elements returned by maps:to_list/1 indeed appears arbitrary. > > I already run into a bug where the code expected the map to be sorted. > When the 33rd element was added, it started to show pathological > behavior. ...and this means there is a bug in the calling code, NOT in maps (which I'm pretty sure is the point Attila is pointing out). I can't believe we are having this discussion. Again. Having a discussion about internal ordering in the context of efficient matches and comparisons *in implementation*: totally logical Relying on that implementation detail to leak out: ridiculous If we really want ordered maps we should make ordered maps. And highlight in big bold letters all the baggage that brings with it. Alternately, we already have other data structures that work this way, perhaps they could be utilized. I think people just really really wish they could add wazoo syntax to various tree structures... ugh. This discussion comes up a little more frequently than once a year and every time it reminds me of watching distributed systems engineers (try desperately to) explain CAP theorem tradeoffs to a VP of marketing. -Craig From silviu.cpp@REDACTED Fri Oct 27 14:25:47 2017 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Fri, 27 Oct 2017 15:25:47 +0300 Subject: [erlang-questions] eheap_alloc In-Reply-To: References: Message-ID: Found myself the problem, I dumped all the memory for the beam.smp process and looked with an editor inside that file. Seems a bug in the http client we are using as time 10 GB of file is full with strings for the HTTP requests. On Fri, Oct 27, 2017 at 12:37 AM, Caragea Silviu wrote: > Hello, > > I see suddenly one of my erlang apps memory usage increased from 5 GB to > 15 GB without no load or useful logs. > > After some debugging I see that the 10 GB leaked are allocated by > eheap_alloc, but I cannot stop what process allocated that. > > My question is : NIF library leaks can show the memory under eheap_alloc > or I can safely assume that there is no NIF leak here > > Any other idea to stop what I can do to identify the issue is welcome > > Silviu > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Oct 27 14:27:25 2017 From: zxq9@REDACTED (zxq9) Date: Fri, 27 Oct 2017 21:27:25 +0900 Subject: [erlang-questions] eheap_alloc In-Reply-To: References: Message-ID: <2163454.uMEqVOThsT@changa> On 2017?10?27? ??? 15:25:47 Caragea Silviu wrote: > Found myself the problem, I dumped all the memory for the beam.smp process > and looked with an editor inside that file. Seems a bug in the http client > we are using as time 10 GB of file is full with strings for the HTTP > requests. References to sub-binaries perhaps? -Craig From ulf@REDACTED Fri Oct 27 15:13:34 2017 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 27 Oct 2017 15:13:34 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: <2046414.yBd3MWPVTy@changa> Message-ID: Actually, that's roughly what I started out assuming too, but it's not sufficient. Consider: 1> lists:sort([#{[-1.0] => 0}, #{[1] => 0}]). [#{[1] => 0},#{[-1.0] => 0}] You must dig into each structure and find occurrences of floats to determine whether they affect the ordering. But yes, the documented sort order should be stable across versions. This means, in practice, that it's safe to rely on the sorting properties of maps. BR, Ulf 2017-10-27 10:49 GMT+02:00 Richard Carlsson : > My interpretation of the reference manual is that maps compare to each > other like the tuples created by this transformation: > > t(Map) -> > {Ks,Vs} = lists:unzip(lists:keysort(1, [{{if is_integer(K) -> 0; > is_float(K) -> 1; true -> 2 end, K}, V} || {K,V} <- maps:to_list(M)])), > list_to_tuple(Ks ++ Vs). > > For example: > > t(#{65 => "A", 2.71 => e, 1.0e308 => alot, pi => 3.14, [] => 0}) > > yields > > {{0,65}, {1,2.71}, {1,1.0e308}, {2,pi}, {2,[]}, > "A", e, alot, 3.14, 0} > > Which: > 1) should be independent of the underlying implementation and choice of > hash function, and thus stable across future versions of Erlang; > 2) is cumbersome to express using the normal term order over the basic > data types, since float keys always sort higher than integers (not really > following the rule of least surprise). > > > > > > /Richard > > 2017-10-27 9:25 GMT+02:00 Ulf Wiger : > >> Actually, they ARE, in some ways. >> >> I should have been clearer. I was, of course, referring to the uses of >> records where they were never optimal in the first place, but were >> tolerated e.g. because of the great convenience of referencing elements by >> name in pattern-matching. >> >> Quoted from EEP-0043: >> >> ?The idea was a data-type, a syntax-aware mapping of key-value >> associations with pattern matching. A syntax similar to records but without >> the hassle of compile-time dependency and with arbitrary terms as keys. >> Order was not important and it could be implemented with a >> Hash-Array-Mapped-Trie with good performance and memory trade-offs. This >> was a different approach than to replace records. It was meant to replace >> records where suitable and in other regards not be a replacement but its >> own _thing_.? >> >> Further, relevant to this discussion: >> >> ?A restriction set on the implementation by the Erlang specification is >> that order is total, i.e. satisfies _antisymmetry, transitivity and >> totality_. >> ... >> - Ordered maps impose restrictions on the underlying implementation and a >> hashing approach will be nearly impossible. >> - The underlying structure does not need to be sorted, an order could be >> produced when needed,? >> >> The thing I tried to highlight was that "an order could be produced when >> needed" is actually a bit more involved than you'd think, given that no API >> function does it for you. At least if you want to reflect the internal sort >> order - given that the actual implementation is NOT as described in the >> EEP: "If a key or value differs, the order of the respective terms, in >> Erlang term order". >> >> You'd have to write your own sort function, in which you visit all >> elements of complex structures, finding all instances of floats and >> ensuring that they get the right priority. >> >> Or... you simply do this: >> >> lists:sort(fun(A, B) -> #{A => 0} =< #{B => 0} end, maps:to_list(Map)) >> >> >> As evidenced e.g. by Benoit's comment above, I believe lots of people >> already rely on the order of map elements being IN SOME WAY stable. This is >> likely a brittle assumption, but one that may - in time - trap the OTP team >> into having to preserve the current APPARENT order. >> >> And the apparent order does appear to be sorted, as long as only maps of >> size =< 32 ("flatmaps") are inspected. For larger maps ("hashmaps"), the >> order of elements returned by maps:to_list/1 indeed appears arbitrary. >> >> BR, >> Ulf W >> >> 2017-10-27 6:01 GMT+02:00 zxq9 : >> >>> On 2017?10?26? ??? 21:13:07 Ulf Wiger wrote: >>> > ... to use as a replacement for records. >>> >>> But they are IN NO WAY a replacement for records. >>> >>> -Craig >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Fri Oct 27 15:26:22 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 27 Oct 2017 15:26:22 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: <2046414.yBd3MWPVTy@changa> Message-ID: <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> Yes. All Erlang terms, including maps, have a globally defined, implementation independent, total order. The reason, for this "surprising" order of maps, is the alternative of using the normal arithmetic order for keys is much worse. If we want 1 and 1.0 to be different keys (which we do as we use matching (=:=) to lookup keys) then we need to order 1 and 1.0, and normal arithmetic term ordering does not (1 == 1.0). It has been discussed (more and less serious) to expose this "map-key-order" with operators like :<, :>, =:<, >:=. /Sverker On 10/27/2017 03:13 PM, Ulf Wiger wrote: > Actually, that's roughly what I started out assuming too, but it's not > sufficient. Consider: > > 1> lists:sort([#{[-1.0] => 0}, #{[1] => 0}]). > [#{[1] => 0},#{[-1.0] => 0}] > > You must dig into each structure and find occurrences of floats to > determine whether they affect the ordering. > > But yes, the documented sort order should be stable across versions. > This means, in practice, that it's safe to rely?on the sorting > properties of maps. > > BR, > Ulf > > 2017-10-27 10:49 GMT+02:00 Richard Carlsson > >: > > My interpretation of the reference manual is that maps compare to > each other like the tuples created by this transformation: > > t(Map) -> > ? ? {Ks,Vs} = lists:unzip(lists:keysort(1, [{{if is_integer(K) -> > 0; is_float(K) -> 1; true -> 2 end, K}, V} || {K,V} <- > maps:to_list(M)])), > ? ? list_to_tuple(Ks ++ Vs). > > For example: > > ? ? t(#{65 => "A", 2.71 => e, 1.0e308 => alot, pi => 3.14, [] => 0}) > > yields > > ? ? {{0,65}, {1,2.71}, {1,1.0e308}, {2,pi}, {2,[]}, > ? ? ? ? "A", e,? ? ? ? alot,? ? ? ? 3.14,? ?0} > > Which: > 1) should be independent of the underlying implementation and > choice of hash function, and thus stable across future versions of > Erlang; > 2) is cumbersome to express using the normal term order over the > basic data types, since float keys always sort higher than > integers (not really following the rule of least surprise). > > > > > > ? ? ? ? /Richard > > 2017-10-27 9:25 GMT+02:00 Ulf Wiger >: > > Actually, they ARE, in some ways. > > I should have been clearer. I was, of course, referring?to the > uses of records where they were never optimal in the first > place, but were tolerated e.g. because of the great > convenience of referencing elements by name in pattern-matching. > > Quoted from EEP-0043: > > ?The idea was a data-type, a syntax-aware mapping of key-value > associations with pattern matching. A syntax similar to > records but without the hassle of compile-time dependency and > with arbitrary terms as keys. Order was not important and it > could be implemented with a Hash-Array-Mapped-Trie with good > performance and memory trade-offs. This was a different > approach than to replace records. It was meant to replace > records where suitable and in other regards not be a > replacement but its own _thing_.? > > Further, relevant to this discussion: > > ?A restriction set on the implementation by the Erlang > specification is that order is total, i.e. satisfies > _antisymmetry, transitivity and totality_. > ... > - Ordered maps impose restrictions on the underlying > implementation and a hashing approach will be nearly impossible. > - The underlying structure does not need to be sorted, an > order could be produced when needed,? > > The thing I tried to highlight was that "an order could be > produced when needed" is actually a bit more involved than > you'd think, given that no API function does it for you. At > least if you want to reflect the internal sort order - given > that the actual implementation is NOT as described in the EEP: > "If a key or value differs, the order of the respective terms, > in Erlang term order". > > You'd have to write your own sort function, in which you visit > all elements of complex structures, finding all instances of > floats and ensuring that they get the right priority. > > Or... you simply do this: > > lists:sort(fun(A, B) -> #{A => 0} =< #{B => 0} end, > maps:to_list(Map)) > > > As evidenced e.g. by Benoit's comment above, I believe lots of > people already rely?on the order of map elements being IN SOME > WAY stable. This is likely a brittle assumption, but one that > may - in time - trap the OTP team into having to preserve the > current APPARENT order. > > And the apparent order does appear to be sorted, as long as > only maps of size =< 32 ("flatmaps") are inspected. For larger > maps ("hashmaps"), the order of elements returned by > maps:to_list/1 indeed appears arbitrary. > > BR, > Ulf W > > 2017-10-27 6:01 GMT+02:00 zxq9 >: > > On 2017?10?26? ??? 21:13:07 Ulf Wiger wrote: > > ... to use as a replacement for records. > > But they are IN NO WAY a replacement for records. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri Oct 27 15:28:16 2017 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 27 Oct 2017 15:28:16 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: <4467466.eSNuECovTi@changa> References: <4467466.eSNuECovTi@changa> Message-ID: 2017-10-27 12:56 GMT+02:00 zxq9 : > > I can't believe we are having this discussion. Again. > I will admit that I haven't been following the list that closely recently, but I wasn't aware that this discussion has been had before. > Having a discussion about internal ordering in the context of efficient > matches and comparisons *in implementation*: totally logical > > Relying on that implementation detail to leak out: ridiculous > Again, quoting from the EEP: ?The underlying structure does not need to be sorted, an order could be produced when needed? I think that among those who fully accepted the maps API returning elements in arbitrary order, most would have assumed that lists:sort(maps:to_list(M)) would do the trick (and, according to the EEP, it would), and were perfectly content with that. What I brought up was the (admittedly subtle) point that even if you WANTED to settle for that, it actually doesn't produce the sort order that applies to maps internally, and there is no function, anywhere, that will give you that result - even one where the docs are riddled with warnings about its inefficiency. > This discussion comes up a little more frequently than once a year and > every time it reminds me of watching distributed systems engineers (try > desperately to) explain CAP theorem tradeoffs to a VP of marketing. > Am I the VP of marketing in this story? BR, Ulf -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Fri Oct 27 15:34:59 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 27 Oct 2017 15:34:59 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: <11a6bcc6-8fc7-0863-98de-ac071fdcc8b0@cs.otago.ac.nz> References: <11a6bcc6-8fc7-0863-98de-ac071fdcc8b0@cs.otago.ac.nz> Message-ID: <018e93d8-a53d-c5e9-c920-57e7c7086e68@ericsson.com> On 10/27/2017 04:04 AM, Richard A. O'Keefe wrote: > The current thread has left me somewhat confused. > > Suppose E1 and E2 are different computations, > delivering maps M1 and M2 respectively, such > that M1 =:= M2 is true. > > Are we guaranteed that term_to_binary(M1) and > term_to_binary(M2) are equal binaries? > > No. term_to_binary does not give any such guarantees. And in the case where M1 and M2 were created by different VM instances, you can with current implementation get different binaries. /Sverker From ulf@REDACTED Fri Oct 27 15:42:30 2017 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 27 Oct 2017 15:42:30 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> References: <2046414.yBd3MWPVTy@changa> <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> Message-ID: Sverker, I understand and agree with the reasons behind the implementation decision. I can expand a little on discussions I've been involved in: We have a scenario where we need to serialize objects partly for network comms, partly for storage and partly for signing and cryptographic hasing. We also prefer the serialization to work easily across programming languages. The trouble is of course the signing/crypto-hashing. For this to be stable, the order must be fixed. Obviously, adding a function that makes Erlang maps appear ordered will not suffice, since other language environments would either not expect maps to be ordered at all, or are likely to have a different take on how keys are ordered*. Better then to stick with the assumption that map key ordering is undefined. Picking e.g. msgpack encoding as an example, a possible alternative would be to use arrays of 1-element maps: [#{K1 => V1, #{K2 => V2}, ...] This is easy enough to decode and handle in e.g. JavaScript, and should be stable enough to sign (using the msgpack-encoded representation as input to the signing/hash function). The encoding overhead compared to using a single map seems to be about one byte per key. * For example, the Go implementation of msgpack has a function to make maps appear ordered, but with its own idea of what key types can be sorted. BR, Ulf W 2017-10-27 15:26 GMT+02:00 Sverker Eriksson : > Yes. All Erlang terms, including maps, have a globally defined, > implementation independent, total order. > The reason, for this "surprising" order of maps, is the alternative of > using the normal arithmetic order for keys is much worse. > > If we want 1 and 1.0 to be different keys (which we do as we use matching > (=:=) to lookup keys) > then we need to order 1 and 1.0, and normal arithmetic term ordering does > not (1 == 1.0). > > > It has been discussed (more and less serious) to expose this > "map-key-order" with operators like :<, :>, =:<, >:=. > > > /Sverker > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Oct 27 15:47:28 2017 From: zxq9@REDACTED (zxq9) Date: Fri, 27 Oct 2017 22:47:28 +0900 Subject: [erlang-questions] map elements in defined order In-Reply-To: <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> References: <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> Message-ID: <1641231.Z61AEZxsX8@changa> On 2017?10?27? ??? 15:26:22 Sverker Eriksson wrote: > Yes. All Erlang terms, including maps, have a globally defined, > implementation independent, total order. > > The reason, for this "surprising" order of maps, is the alternative of > using the normal arithmetic order for keys is much worse. I am curious, though, why a compound data type was added to the language as a primitive data type. This was the only thing that really bothered me about maps. Second-class data type, sure. Ordering unknown. Whatever. Same with all the other compound data types that we make up for ourselves. Having it as a data primitive introduces inconsistency in the language itself, as we either have to have a tradeoff to amortize enforcement of an arbitrary ordering to make comparisons faster, or force an ordering at the time of comparison (and/or serialization, maybe) at the cost of some computation time in order for things to work. That adds one more odd point of weirdness to the language we *never* once worried about before but now need to remember in edge cases where performance matters. In other words, this creates a new edge case for performance, if I understand thing correctly. Who before was ever seriously considering using dicts as keys to dicts? "But lists are compound data types!" Sort of. The ordering of a list very often IS its meaning -- thus strings. Not so for maps by their very nature. -Craig From zxq9@REDACTED Fri Oct 27 16:04:28 2017 From: zxq9@REDACTED (zxq9) Date: Fri, 27 Oct 2017 23:04:28 +0900 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> Message-ID: <1812753.kKhOPjOCaU@changa> On 2017?10?27? ??? 15:42:30 Ulf Wiger wrote: > We have a scenario where we need to serialize objects partly for network > comms, partly for storage and partly for signing and cryptographic hasing. > We also prefer the serialization to work easily across programming > languages. > > The trouble is of course the signing/crypto-hashing. For this to be stable, > the order must be fixed. Obviously, adding a function that makes Erlang > maps appear ordered will not suffice, since other language environments > would either not expect maps to be ordered at all, or are likely to have a > different take on how keys are ordered*. Interesting. I have been dealing with this exact case myself for quite some time (before the advent of maps, actually). The tradeoff we had to make was exactly as you describe: Universal ordered representation of pretty much any K/V sort of data type (especially across languages) only works as an ordered list, so the external, canonical representation of the data must be defined as an ordered list (sorted some specific way). Occasionally this also means internal lists must be themselves sorted. That is one critical part of the definition of any serialization procedure for any data that needs to be verifiable via signature. Either that or build an ASN.1 DER representation for everything; which isn't actually so bad, but would be way better if there were tutorials on just the DDL part of ASN.1 for the cool kids to brush up on... Having maps inside an Erlang program (and dicts in Python and blats in Frozz and so on) is quite nice, but it can never be relied on for things like canonical serialization. Imagine if every SQL query return had to suddenly be ordered! Those internal representations can only ever be immediate conveniences, never canonical data representations. I think this is easy to forget because most of the time we never even have to worry about what a "canonical representation" even might be for 99% of the data most of us ever deal with. -Craig From carlsson.richard@REDACTED Fri Oct 27 16:39:08 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 27 Oct 2017 16:39:08 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> References: <2046414.yBd3MWPVTy@changa> <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> Message-ID: 2017-10-27 15:26 GMT+02:00 Sverker Eriksson : > Yes. All Erlang terms, including maps, have a globally defined, > implementation independent, total order. > The reason, for this "surprising" order of maps, is the alternative of > using the normal arithmetic order for keys is much worse. > > If we want 1 and 1.0 to be different keys (which we do as we use matching > (=:=) to lookup keys) > then we need to order 1 and 1.0, and normal arithmetic term ordering does > not (1 == 1.0). > Yes... but I'm not sure that the term order when comparing one map to another needs to have anything to do with how lookup works within maps (much like it doesn't in an orddict, gb_trees, or similar). The global term order only needs to be fixed and preferably straightforward. I could, for example, implement maps using the sort of tuple I showed: {K1, ..., Kn, V1, ..., Vn}, and they would have one order given by '>', but using #{K1=>V1, ... Kn=>Vn}, they would be ordered differently if keys were floats. I think that key order as used internally by the maps could and should be kept separate from the global term order. > It has been discussed (more and less serious) to expose this > "map-key-order" with operators like :<, :>, =:<, >:=. > And as I recall, that was suggested even before maps were a thing, since it would occasionally be useful also for things like lists of key/value tuples where the keys may be floats. /Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From unix1@REDACTED Fri Oct 27 16:41:50 2017 From: unix1@REDACTED (Unix One) Date: Fri, 27 Oct 2017 14:41:50 +0000 Subject: [erlang-questions] [ANN] PHP on Cowboy Message-ID: Hello list, I have taken up a couple of no longer maintained projects to make PHP work on modern Cowboy web server again. Check it out here - https://github.com/unix1/cowboy_fcgi cowboy_fcgi is a FastCGI handler for Cowboy that uses the ex_fcgi [1] client to communicate with php-fpm [2] - PHP's FastCGI process manager. The stack now works with Cowboy 2 and PHP 7, but support for Cowboy 1 and PHP 5 is also available. So if you are interested in running PHP along side your Erlang applications on the same web server, hop over to cowboy_fcgi project readme, try the quick start example and let me know what you think. Thank you! [1] https://github.com/unix1/ex_fcgi [2] https://secure.php.net/manual/en/install.fpm.php From sverker.eriksson@REDACTED Fri Oct 27 17:45:10 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 27 Oct 2017 17:45:10 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: <2046414.yBd3MWPVTy@changa> <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> Message-ID: <60697bca-33a6-bf15-c154-dc45ea7119f7@ericsson.com> On 10/27/2017 04:39 PM, Richard Carlsson wrote: > 2017-10-27 15:26 GMT+02:00 Sverker Eriksson > >: > > Yes. All Erlang terms, including maps, have a globally defined, > implementation independent, total order. > > The reason, for this "surprising" order of maps, is the > alternative of using the normal arithmetic order for keys is much > worse. > > If we want 1 and 1.0 to be different keys (which we do as we use > matching (=:=) to lookup keys) > then we need to order 1 and 1.0, and normal arithmetic term > ordering does not (1 == 1.0). > > > Yes... but I'm not sure that the term order when comparing one map to > another needs to have anything to do with how lookup works within maps > (much like it doesn't in an orddict, gb_trees, or similar). The global > term order only needs to be fixed and preferably straightforward. I > could, for example, implement maps using the sort of tuple I showed: > {K1, ..., Kn, V1, ..., Vn}, and they would have one order given by > '>', but using #{K1=>V1, ... Kn=>Vn}, they would be ordered > differently if keys were floats. I think that key order as used > internally by the maps could and should be kept separate from the > global term order. > orddict and gb_trees both use '==' to distinguish keys, which makes it possible to order them with '>'. How would you order? #{1 => x, 1.0 => y} and #{1 => y, 1.0 => x} if you can't order 1 and 1.0? /Sverker -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Fri Oct 27 21:36:58 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 27 Oct 2017 21:36:58 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: <60697bca-33a6-bf15-c154-dc45ea7119f7@ericsson.com> References: <2046414.yBd3MWPVTy@changa> <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> <60697bca-33a6-bf15-c154-dc45ea7119f7@ericsson.com> Message-ID: 2017-10-27 17:45 GMT+02:00 Sverker Eriksson > > > How would you order #{1 => x, 1.0 => y} and #{1 => y, 1.0 => x} > if you can't order 1 and 1.0? > As long as we're talking about the arithmetic term order (<, >, ==), I don't see why they would need to be. Look at tuples: {1, 1.0} < {1.0, 1}. false {1, 1.0} > {1.0, 1}. false {1, 1.0} == {1.0, 1}. true {1, 1.0} =:= {1.0, 1}. false Maps ought to behave analogously, in the arithmetic ordering. The weirdness comes from enforcing strict ordering in the middle of the arithmetic one. The current ordering rule for maps should only be used in the strict ordering (the suggested :<), where it would also apply to tuples: {1, 1.0} :< {1.0, 1}. -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang.org@REDACTED Fri Oct 27 22:25:51 2017 From: erlang.org@REDACTED (Stanislaw Klekot) Date: Fri, 27 Oct 2017 22:25:51 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: <2046414.yBd3MWPVTy@changa> <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> <60697bca-33a6-bf15-c154-dc45ea7119f7@ericsson.com> Message-ID: <20171027202551.GA27157@jarowit.net> On Fri, Oct 27, 2017 at 09:36:58PM +0200, Richard Carlsson wrote: > 2017-10-27 17:45 GMT+02:00 Sverker Eriksson > > > > > > How would you order #{1 => x, 1.0 => y} and #{1 => y, 1.0 => x} > > if you can't order 1 and 1.0? > > > > As long as we're talking about the arithmetic term order (<, >, ==), I > don't see why they would need to be. Look at tuples: > > {1, 1.0} < {1.0, 1}. > false > {1, 1.0} > {1.0, 1}. > false > {1, 1.0} == {1.0, 1}. > true > {1, 1.0} =:= {1.0, 1}. > false > > Maps ought to behave analogously, in the arithmetic ordering. The problem is, the tuples you provide and comparison operators (<, >, =<, >=, and == (not =:= one)) form a well-defined partial order (total order, actually); mainly, if A =< B and B =< A, then A == B. Being a partial order is a very important property of Erlang's type system, one that was quite explicitly baked in the VM and is used in many different places. -- Stanislaw Klekot From carlsson.richard@REDACTED Fri Oct 27 23:10:22 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 27 Oct 2017 23:10:22 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: <20171027202551.GA27157@jarowit.net> References: <2046414.yBd3MWPVTy@changa> <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> <60697bca-33a6-bf15-c154-dc45ea7119f7@ericsson.com> <20171027202551.GA27157@jarowit.net> Message-ID: 2017-10-27 22:25 GMT+02:00 Stanislaw Klekot : > On Fri, Oct 27, 2017 at 09:36:58PM +0200, Richard Carlsson wrote: > > 2017-10-27 17:45 GMT+02:00 Sverker Eriksson < > sverker.eriksson@REDACTED> > > > > > > How would you order #{1 => x, 1.0 => y} and #{1 => y, 1.0 => x} > > > if you can't order 1 and 1.0? > > > > As long as we're talking about the arithmetic term order (<, >, ==), I > > don't see why they would need to be. Look at tuples: > > > > {1, 1.0} < {1.0, 1}. > > false > > {1, 1.0} > {1.0, 1}. > > false > > {1, 1.0} == {1.0, 1}. > > true > > {1, 1.0} =:= {1.0, 1}. > > false > > > > Maps ought to behave analogously, in the arithmetic ordering. > > The problem is, the tuples you provide and comparison operators (<, >, > =<, >=, and == (not =:= one)) form a well-defined partial order (total > order, actually); mainly, if A =< B and B =< A, then A == B. Being > a partial order is a very important property of Erlang's type system, > one that was quite explicitly baked in the VM and is used in many > different places. No, the real problem, that I begin to see now, is that if arithmetic ordering is used, two keys could appear to be the same, and if their corresponding values differ, it is not clear in which order to compare them. I guess that's what Sverker was trying to convey, but I missed it at first. So, using tuples {K1, ... Kn, V1, ... Vn} again for comparison, Sverker's example could become either: {1, 1.0, x, y} < {1, 1.0, y, x} or {1.0, 1, y, x} == {1, 1.0, y, x} (switching the order of the keys in the left tuple), both of which would be equally "legal" since 1 == 1.0. But this switches the order of x and y, so that the comparisons of the maps would give different results. One way to avoid this situation would be to say that you can't have two keys that compare equal with == in the same map, just as for an orddict. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Fri Oct 27 23:58:16 2017 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 27 Oct 2017 17:58:16 -0400 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: <2046414.yBd3MWPVTy@changa> <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> <60697bca-33a6-bf15-c154-dc45ea7119f7@ericsson.com> <20171027202551.GA27157@jarowit.net> Message-ID: <20171027215815.GA19562@ferdmbp.local> On 10/27, Richard Carlsson wrote: >(switching the order of the keys in the left tuple), both of which >would be >equally "legal" since 1 == 1.0. But this switches the order of x and y, so >that the comparisons of the maps would give different results. One way to >avoid this situation would be to say that you can't have two keys that >compare equal with == in the same map, just as for an orddict. This would, unfortunately, break pattern matching: M = #{1 => 1}, case M of #{1.0 := _} -> % it is safe to add 1.0 if we match M#{1.0 := 2}; % does this crush a value and change the key? _ -> other end. The fourth line here is problematic. Either (1) maps have a special magical pattern matching case where 1.0 and 1 compare equal (which happens nowhere else) and decide whether to replace the key or keep it the same, or (2) you keep current pattern matching semantics and you can't use existing pattern matching to enforce the constraints above. (1) is particularly nasty for cases such as: X = 1.0, case {#{1 => 1}, 1} of {#{X := _}, X} -> true = X =:= X; % works {#{X := _}, Y} -> false = X =/= Y % crashes end Which one should match? In the first clause, the map would match fine, even though X =/= X! We just broke a lot of language here. To preserve safe pattern matching, you should probably not be able to make 1 be equal to 1.0 in a map through pattern insertions (M#{K := NewVal}). From ilya.khaprov@REDACTED Sat Oct 28 08:06:51 2017 From: ilya.khaprov@REDACTED (Ilya Khaprov) Date: Sat, 28 Oct 2017 06:06:51 +0000 Subject: [erlang-questions] sharing config via a dynamically compiled module? In-Reply-To: References: Message-ID: Hi, I did this, feels good. If this is for config, I would say use config_change and regen. Also code:delete/1 and code:purge/1. Ilya From: Benoit Chesneau Sent: Wednesday, October 25, 2017 03:29 PM To: Erlang Questions Subject: [erlang-questions] sharing config via a dynamically compiled module? I'm temped to compile dynamically a module for some highly demanded resources to share their config instead of using ETS for it. But I'm wondering if's not to hackish. (I'm also tempted by just using something more pure like sharing it via message passing). Are other people do such thing? I can see the following pro/cons: pro: * fast and constant access, it is just a call to a module function and some literals. cons: * maybe too much hackish? * What happen if the resource is deleted when someone is using it? Will the module be really deleted or will it leaks? What do you think about it? Any feedback is welcome :) - beno?t _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Oct 28 08:38:35 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 28 Oct 2017 09:38:35 +0300 Subject: [erlang-questions] sharing config via a dynamically compiled module? In-Reply-To: References: Message-ID: Take a look at thread nearby. > All literals belong to a loaded module instance. When that (old) module instance is purged, all process heaps are scanned and heaps containing those literals will do a special garbage collection where the literals are copied. Each config change may lead to rescanning heaps of all processes. Frankly speaking global ets is much easier =) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ilya.khaprov@REDACTED Sat Oct 28 09:07:38 2017 From: ilya.khaprov@REDACTED (Ilya Khaprov) Date: Sat, 28 Oct 2017 07:07:38 +0000 Subject: [erlang-questions] sharing config via a dynamically compiled module? In-Reply-To: References: , Message-ID: Sure, then the question is how often config change happens. I doubt that often. But having careful measurements always helps of course. From: Max Lapshin Sent: Saturday, October 28, 2017 09:38 AM To: Ilya Khaprov Cc: Benoit Chesneau; Erlang Questions Subject: Re: [erlang-questions] sharing config via a dynamically compiled module? Take a look at thread nearby. > All literals belong to a loaded module instance. When that (old) module instance is purged, all process heaps are scanned and heaps containing those literals will do a special garbage collection where the literals are copied. Each config change may lead to rescanning heaps of all processes. Frankly speaking global ets is much easier =) -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Sat Oct 28 09:39:07 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 28 Oct 2017 09:39:07 +0200 Subject: [erlang-questions] sharing config via a dynamically compiled module? In-Reply-To: References: Message-ID: <92698F02-4ABC-457E-923C-42DBDD59B476@gmail.com> > On 28 Oct 2017, at 08:38, Max Lapshin wrote: > > Take a look at thread nearby. > > > All literals belong to a loaded module instance. When that (old) module instance is purged, all process heaps are scanned and heaps containing those literals will do a special garbage collection where the literals are copied. > > Each config change may lead to rescanning heaps of all processes. > > > Frankly speaking global ets is much easier =) > Right. One another advantage to ETS i that you can store any terms even refs where it's not possible . At the cost of increasing the number of ETS tables used though... - beno?t From jesper.louis.andersen@REDACTED Sat Oct 28 17:41:56 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 28 Oct 2017 15:41:56 +0000 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: On Thu, Oct 26, 2017 at 9:13 PM Ulf Wiger wrote: > But again, Jesper, just about everyone relies on the fact that maps follow > the general principle that there is a well-defined term comparison order. > Otherwise, maps would be highly unsuitable to use in keys, and generally > treacherous to use as a replacement for records. Following the Principle of > Least Surprise, it's a darned good thing that Erlang doesn't randomize the > key order in its maps. > > I largely regard a total term order as a language mistake. The right solution, obviously, is to have several equalities, where the programmer can define what they mean by equality in a certain part of the program. Equality is way too important in programming for you to be left with a single one! That said, there is a well-defined order of maps currently, but it is not the logical one you might expect (which is by ordering the keys of the map). Rather the order is defined on * What the key hashes to * What the internal HAMT structure looks like right now It is a total order even! So you can use maps as keys in a balanced search tree for instance. However, your point does seem to touch on a couple of important things that should be considered for future inclusion: * We may want to have an "ordered map" in the language. These are self-balancing binary trees. They are more costly in lookup time and they take up more memory space, but they have the "natural ordering" of keys which means they are well-defined in their traversal. * Your "sext" library exists to plug yet another hole in the language, namely that binary_to_term have certain freedoms with certain data structures and this leads to situations where you cannot rely on the binary output for, e.g., cryptographic applications. In short, one has to weigh different implementation details when building data structures. If you want to have it all, your efficiency eventually has to give. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Sat Oct 28 22:46:10 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Sat, 28 Oct 2017 22:46:10 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: <20171027215815.GA19562@ferdmbp.local> References: <2046414.yBd3MWPVTy@changa> <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> <60697bca-33a6-bf15-c154-dc45ea7119f7@ericsson.com> <20171027202551.GA27157@jarowit.net> <20171027215815.GA19562@ferdmbp.local> Message-ID: 2017-10-27 23:58 GMT+02:00 Fred Hebert : > On 10/27, Richard Carlsson wrote: > >> One way to >> avoid this situation would be to say that you can't have two keys that >> compare equal with == in the same map, just as for an orddict. >> > > This would, unfortunately, break pattern matching: > > M = #{1 => 1}, > case M of > #{1.0 := _} -> % it is safe to add 1.0 if we match > M#{1.0 := 2}; % does this crush a value and change the key? > _ -> > other > end. > > The fourth line here is problematic. Either (1) maps have a special > magical pattern matching case where 1.0 and 1 compare equal (which happens > nowhere else) and decide whether to replace the key or keep it the same, or > (2) you keep current pattern matching semantics and you can't use existing > pattern matching to enforce the constraints above. > Yes, you can't change it now that it's in existing code; it would have had to be done when maps were new. But, having slept on the issue, I now think that the current state of things is fine. The keys must be seen as existing on a different level than the values they map to, more part of the structure of the thing, much like the arity of a tuple. As in your example, they are used in matching, where exact equality is the expected behaviour. It is still bothersome for people like Ulf who want to easily generate a list of the key-value pairs in the order that '<' would consider them - but that is a fairly unusual use case and can be fixed by making separate ordering functions accessible (either as a new :< operator or as a regular BIF). One thing can be noted: there is no strict need as far as I can see for the key order to be related to any other particular order like < or :< (apart from convenience), as long as there is always a canonical key order independent of underlying implementation. One could hypothetically use lexicographical order on the stringified keys, and it would work just as well, since the ordering is really only used for deciding which maps are at all comparable. It does decide the ordering of maps of the same size and with different keys (in which case the values are never examined), but if that by necessity must be different from the arithmetic order anyway, it doesn't matter much which order it is. The main thing is that it's cheap to compute, and the :< order is as cheap as any. But until :< is actually available as an operator or function, poor Ulf will need to emulate it using 0/1/2-tagging like I showed (recursively, as he pointed out). Or write a NIF for performance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Sat Oct 28 23:09:13 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Sat, 28 Oct 2017 23:09:13 +0200 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: 2017-10-28 17:41 GMT+02:00 Jesper Louis Andersen < jesper.louis.andersen@REDACTED>: > On Thu, Oct 26, 2017 at 9:13 PM Ulf Wiger wrote: > >> But again, Jesper, just about everyone relies on the fact that maps >> follow the general principle that there is a well-defined term comparison >> order. Otherwise, maps would be highly unsuitable to use in keys, and >> generally treacherous to use as a replacement for records. Following the >> Principle of Least Surprise, it's a darned good thing that Erlang doesn't >> randomize the key order in its maps. >> > > I largely regard a total term order as a language mistake. The right > solution, obviously, is to have several equalities, where the programmer > can define what they mean by equality in a certain part of the program. > Equality is way too important in programming for you to be left with a > single one! > That may well be right, but doesn't change the fact that Erlang is pretty much permeated with the idea of the total term order, and in many cases it does make things very straightforward. So it would be a very bad thing if some language change would break this property. > That said, there is a well-defined order of maps currently, but it is not > the logical one you might expect (which is by ordering the keys of the > map). Rather the order is defined on > > * What the key hashes to > * What the internal HAMT structure looks like right now > As we quoted from the reference manual, the < ordering on maps is actually implementation independent and future proof (while still being a total order). The sacrifice is that to compare two maps with <, their keys must be mapped into canonical order (with integers before floats as discussed). This is clearly more costly than just taking whatever order the current underlying hash+HAMT produces, but worth it since it preserves those nice properties. On the other hand, the result from maps:to_list(M) is returned in the current internal order, which is efficient but subject to change between versions (albeit also well defined and total as you said), and there is also no current comparison function available that you could give to lists:sort(CompFunc, List) which would produce the key order used by < for comparing maps - which is what Ulf needs for sext. * Your "sext" library exists to plug yet another hole in the language, > namely that binary_to_term have certain freedoms with certain data > structures and this leads to situations where you cannot rely on the binary > output for, e.g., cryptographic applications. > Not only that it wants to provide a stable mapping from terms to their serialized forms, but also ensure that these binary representations sort in the same order as the corresponding terms - a trickier problem. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Oct 30 01:09:18 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 30 Oct 2017 13:09:18 +1300 Subject: [erlang-questions] map elements in defined order In-Reply-To: <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> References: <2046414.yBd3MWPVTy@changa> <0bfa6fae-7a59-4eb5-c350-7cda28d645f5@ericsson.com> Message-ID: On 28/10/17 2:26 AM, Sverker Eriksson wrote: > Yes. All Erlang terms, including maps, have a globally defined, > implementation independent, total order. > > The reason, for this "surprising" order of maps, is the alternative of > using the normal arithmetic order for keys is much worse. > > If we want 1 and 1.0 to be different keys (which we do as we use > matching (=:=) to lookup keys) > then we need to order 1 and 1.0, and normal arithmetic term ordering > does not (1 == 1.0). The fundamental mistake here is confusing *arithmetic* ordering with *term* ordering. In *term* ordering, if X and Y are behaviourally distinguishable then either X < Y or X > Y should be true. Since integers and floats are behaviourally distinguishable, 1> X = 1 bsl 53. 9007199254740992 2> Y = float(X). 9007199254740992.0 3> X == Y. true 4> X+1 == Y+1. false it follows that either X < Y or Y > X should be true in *term* (but not *arithmetic*) order. Arithmetic order in Erlang has some seriously weird issues, like you can find numbers X Y such that X - Y is 0.0 but X == Y is false. > > > It has been discussed (more and less serious) to expose this > "map-key-order" with operators like :<, :>, =:<, >:=. Surely we deserve *some* sane ordering in Erlang? Prolog had two sets of comparison operators: term ordering and arithmetic ordering. There were *reasons* for that. From ok@REDACTED Mon Oct 30 01:16:42 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 30 Oct 2017 13:16:42 +1300 Subject: [erlang-questions] map elements in defined order In-Reply-To: <018e93d8-a53d-c5e9-c920-57e7c7086e68@ericsson.com> References: <11a6bcc6-8fc7-0863-98de-ac071fdcc8b0@cs.otago.ac.nz> <018e93d8-a53d-c5e9-c920-57e7c7086e68@ericsson.com> Message-ID: <098c5d52-5435-8c77-9fad-8c8247447034@cs.otago.ac.nz> On 28/10/17 2:34 AM, Sverker Eriksson wrote: [term_to_binary/1 is not a pure function; it depends on the representation of its argument, not just its value]. OUCH. For term_to_binary/2, of course the result depends on the Options argument, but I take it now that even being explicit about the Options is not enough. Can we have a 'canonical' option? > > And in the case where M1 and M2 were created by different VM instances, > you can with current implementation get different binaries. I can live with different *versions* of the VM using different versions of the binary term format, but two instances of the *same* VM turning mathematically identical terms into different binaries is, well, did Nyarlathotep, the Crawling Chaos, have a hand in the design? In all seriousness, *OUCH*. Please mention this in LARGE red letters in the documentation for term_to_binary; I don't see it at the moment, but it puts limits on what you can reasonably do with terms-as-binaries. From prakashkumar_parmar@REDACTED Mon Oct 30 13:38:53 2017 From: prakashkumar_parmar@REDACTED (Prakash Parmar) Date: Mon, 30 Oct 2017 18:08:53 +0530 (IST) Subject: [erlang-questions] Decompiling Erlang source code Message-ID: <1509367133.852130323@webmail.thbs.com> I have Erlang application's beam files which I want to decompile. Application is Compiled and build with 'debug_info' options. here is snippet of emake file to which is used to build application : {"apps/my_app/src/*", [debug_info, nowarn_export_all, {outdir, "apps/my_app/ebin"},{pa, "deps"},{parse_transform, lager_transform}, {parse_transform, events_transform}]}. I have tried function from beam_lib module but getting following Error : 1>{ok,{_,[{abstract_code,{_,AC}}]}} = beam_lib:chunks(abc,[abstract_code]). ** exception error: no match of right hand side value {error,beam_lib,{not_a_beam_file,'abc.beam'}} I have also tried solution mentioned in[ following post ]( https://reverseengineering..stackexchange.com/questions/8895/decompile-erlang-beam-files-compiled-without-debug-info ) but Its not working for me. I'm getting following error : 1> file:write_file("/tmp/my_module_disasm", io_lib:fwrite("~p.\n", [beam_disasm:file(abc)])). ok /tmp/my_module_disasm file contains : {error,beam_lib,{not_a_beam_file,'abc.beam'}}. Anyone have Idea ? ******* DISCLAIMER: This email and any files transmitted with it are privileged and confidential information and intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Torry Harris Business Solutions has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. The recipient should check this email and any attachments for the presence of viruses. THBS reserves the right to monitor and review the content of all messages sent to or from this e-mail address******** ******* DISCLAIMER: This email and any files transmitted with it are privileged and confidential information and intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Torry Harris Business Solutions has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. The recipient should check this email and any attachments for the presence of viruses. THBS reserves the right to monitor and review the content of all messages sent to or from this e-mail address.******** -------------- next part -------------- An HTML attachment was scrubbed... URL: From elbrujohalcon@REDACTED Mon Oct 30 13:47:36 2017 From: elbrujohalcon@REDACTED (Brujo Benavides) Date: Mon, 30 Oct 2017 09:47:36 -0300 Subject: [erlang-questions] Decompiling Erlang source code In-Reply-To: <1509367133.852130323@webmail.thbs.com> References: <1509367133.852130323@webmail.thbs.com> Message-ID: <21B6C295-F740-4AF7-8CE7-7211307E63E0@gmail.com> Have you tried using recon:source/1 ? Brujo Benavides > On Oct 30, 2017, at 09:38, Prakash Parmar wrote: > > I have Erlang application's beam files which I want to decompile. > > Application is Compiled and build with 'debug_info' options. here is snippet of emake file to which is used to build application : > > {"apps/my_app/src/*", [debug_info, nowarn_export_all, {outdir, "apps/my_app/ebin"},{pa, "deps"},{parse_transform, lager_transform}, {parse_transform, events_transform}]}. > > I have tried function from beam_lib module but getting following Error : > > 1>{ok,{_,[{abstract_code,{_,AC}}]}} = beam_lib:chunks(abc,[abstract_code]). > ** exception error: no match of right hand side value {error,beam_lib,{not_a_beam_file,'abc.beam'}} > > I have also tried solution mentioned in following post but Its not working for me. I'm getting following error : > > 1> file:write_file("/tmp/my_module_disasm", io_lib:fwrite("~p.\n", [beam_disasm:file(abc)])). > ok > > /tmp/my_module_disasm file contains : > > {error,beam_lib,{not_a_beam_file,'abc.beam'}}. > > Anyone have Idea ? > > > > > ******* DISCLAIMER: This email and any files transmitted with it are privileged and confidential information and intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Torry Harris Business Solutions has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. The recipient should check this email and any attachments for the presence of viruses. THBS reserves the right to monitor and review the content of all messages sent to or from this e-mail address******** > ******* DISCLAIMER: This email and any files transmitted with it are privileged and confidential information and intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Torry Harris Business Solutions has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. The recipient should check this email and any attachments for the presence of viruses. THBS reserves the right to monitor and review the content of all messages sent to or from this e-mail address.******** > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Mon Oct 30 14:06:55 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 30 Oct 2017 14:06:55 +0100 Subject: [erlang-questions] Decompiling Erlang source code In-Reply-To: <1509367133.852130323@webmail.thbs.com> References: <1509367133.852130323@webmail.thbs.com> Message-ID: The first argument to beam_lib:chunks/2 should be a path to the beam file, not just the module name. /Richard 2017-10-30 13:38 GMT+01:00 Prakash Parmar : > I have Erlang application's beam files which I want to decompile. > > Application is Compiled and build with 'debug_info' options. here is > snippet of emake file to which is used to build application : > > {"apps/my_app/src/*", [debug_info, nowarn_export_all, {outdir, > "apps/my_app/ebin"},{pa, "deps"},{parse_transform, lager_transform}, > {parse_transform, events_transform}]}. > > I have tried function from beam_lib module but getting following Error : > > 1>{ok,{_,[{abstract_code,{_,AC}}]}} = beam_lib:chunks(abc,[abstract_ > code]). > ** exception error: no match of right hand side value > {error,beam_lib,{not_a_beam_file,'abc.beam'}} > > I have also tried solution mentioned in following post > > but Its not working for me. I'm getting following error : > > 1> file:write_file("/tmp/my_module_disasm", io_lib:fwrite("~p.\n", > [beam_disasm:file(abc)])). > ok > > /tmp/my_module_disasm file contains : > > {error,beam_lib,{not_a_beam_file,'abc.beam'}}. > > Anyone have Idea ? > > > > > ******* DISCLAIMER: This email and any files transmitted with it are > privileged and confidential information and intended solely for the use of > the individual or entity to which they are addressed. If you are not the > intended recipient, please notify the sender by e-mail and delete the > original message. Further, you are not to copy, disclose, or distribute > this e-mail or its contents to any other person and any such actions are > unlawful. This e-mail may contain viruses. Torry Harris Business Solutions > has taken every reasonable precaution to minimize this risk, but is not > liable for any damage you may sustain as a result of any virus in this > e-mail. The recipient should check this email and any attachments for the > presence of viruses. THBS reserves the right to monitor and review the > content of all messages sent to or from this e-mail address******** > > ******* DISCLAIMER: This email and any files transmitted with it are privileged and confidential information and intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Torry Harris Business Solutions has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. The recipient should check this email and any attachments for the presence of viruses. THBS reserves the right to monitor and review the content of all messages sent to or from this e-mail address.******** > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Mon Oct 30 14:49:38 2017 From: erlangsiri@REDACTED (Siri Hansen) Date: Mon, 30 Oct 2017 14:49:38 +0100 Subject: [erlang-questions] PR#1602, Refactor supervisor, and store children in a map instead of a list Message-ID: Hi all! Inspired by https://github.com/erlang/otp/pull/1467, I have done https://github.com/erlang/otp/pull/1602. Due to quite a bit of refactoring, the diff on the supervisor module is rather big. There should be no functional changes, except the speed of starting many children under a non-simple_one_for_one supervisor. The goals of the change are * speed up starting of many children (as in the original PR#1467) * improve readability * improve maintainability and prepare for further changes We have done internal reviews, and tests have quite good coverage, so we are getting close to merge (on master). Still - if anyone out there has got the time and interest, some further reviews and/or tests in existing systems would be much appreciated. Thanks /siri -------------- next part -------------- An HTML attachment was scrubbed... URL: From jaroslaw.p.75@REDACTED Mon Oct 30 16:36:44 2017 From: jaroslaw.p.75@REDACTED (=?UTF-8?B?SmFyb3PFgmF3IFByZcWb?=) Date: Mon, 30 Oct 2017 16:36:44 +0100 Subject: [erlang-questions] sorting of type definitions using edoc Message-ID: Hi, Is it possible to turn off sorting of type definitions for generation of htmls with use of edoc:files? BR/Jarek -------------- next part -------------- An HTML attachment was scrubbed... URL: From jaroslaw.p.75@REDACTED Mon Oct 30 16:41:00 2017 From: jaroslaw.p.75@REDACTED (=?UTF-8?B?SmFyb3PFgmF3IFByZcWb?=) Date: Mon, 30 Oct 2017 16:41:00 +0100 Subject: [erlang-questions] one-time evaluated functions Message-ID: Hi, Is it possible to make an erlang function be evaluated only once at first call and then return this value each time? BR/Jarek -------------- next part -------------- An HTML attachment was scrubbed... URL: From tthread@REDACTED Mon Oct 30 16:45:00 2017 From: tthread@REDACTED (Ilya Shcherbak) Date: Mon, 30 Oct 2017 22:45:00 +0700 Subject: [erlang-questions] Decompiling Erlang source code In-Reply-To: <1509367133.852130323@webmail.thbs.com> References: <1509367133.852130323@webmail.thbs.com> Message-ID: hey Prakash, can you attach the file which you want to decompile? . 2017-10-30 19:38 GMT+07:00 Prakash Parmar : > I have Erlang application's beam files which I want to decompile. > > Application is Compiled and build with 'debug_info' options. here is > snippet of emake file to which is used to build application : > > {"apps/my_app/src/*", [debug_info, nowarn_export_all, {outdir, > "apps/my_app/ebin"},{pa, "deps"},{parse_transform, lager_transform}, > {parse_transform, events_transform}]}. > > I have tried function from beam_lib module but getting following Error : > > 1>{ok,{_,[{abstract_code,{_,AC}}]}} = beam_lib:chunks(abc,[abstract_ > code]). > ** exception error: no match of right hand side value > {error,beam_lib,{not_a_beam_file,'abc.beam'}} > > I have also tried solution mentioned in following post > > but Its not working for me. I'm getting following error : > > 1> file:write_file("/tmp/my_module_disasm", io_lib:fwrite("~p.\n", > [beam_disasm:file(abc)])). > ok > > /tmp/my_module_disasm file contains : > > {error,beam_lib,{not_a_beam_file,'abc.beam'}}. > > Anyone have Idea ? > > > > > ******* DISCLAIMER: This email and any files transmitted with it are > privileged and confidential information and intended solely for the use of > the individual or entity to which they are addressed. If you are not the > intended recipient, please notify the sender by e-mail and delete the > original message. Further, you are not to copy, disclose, or distribute > this e-mail or its contents to any other person and any such actions are > unlawful. This e-mail may contain viruses. Torry Harris Business Solutions > has taken every reasonable precaution to minimize this risk, but is not > liable for any damage you may sustain as a result of any virus in this > e-mail. The recipient should check this email and any attachments for the > presence of viruses. THBS reserves the right to monitor and review the > content of all messages sent to or from this e-mail address******** > > ******* DISCLAIMER: This email and any files transmitted with it are privileged and confidential information and intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Torry Harris Business Solutions has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. The recipient should check this email and any attachments for the presence of viruses. THBS reserves the right to monitor and review the content of all messages sent to or from this e-mail address.******** > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Mon Oct 30 18:05:59 2017 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Mon, 30 Oct 2017 18:05:59 +0100 Subject: [erlang-questions] one-time evaluated functions In-Reply-To: References: Message-ID: Yes. 1. (Boring) Have a persistent store somewhere, and have the function check if the value is stored there or not. If not, compute it and store. An ETS table would work fine, but you need to ensure it's lifetime matches your requirements. A plain file containing term_to_binary/1 of the value should also work. Or use a DB. 2. (More fun) Place the function in a module M. Initial version computes the value and generates a new version of the module where the function now just returns that constant value. There will probably be some restrictions on what kinds of values you can return this way, i.e. I wouldn't expect a function value with closed over free variables to work, but anything you could include in a pattern match should work. On Mon, Oct 30, 2017 at 4:41 PM, Jaros?aw Pre? wrote: > Hi, > > Is it possible to make an erlang function be evaluated only once at first > call and then return this value each time? > > BR/Jarek > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Mon Oct 30 18:20:27 2017 From: chandrashekhar.mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Mon, 30 Oct 2017 17:20:27 +0000 Subject: [erlang-questions] one-time evaluated functions In-Reply-To: References: Message-ID: <42452B05-60CC-4EEF-9D42-1E0FE64C4157@gmail.com> Here is the code for the boring solution. case ets:insert_new(EtsTab, {singleton, undefined}) of false -> ets:lookup_element(EtsTab, singleton, 2); true -> Val = evaluate_one_time_function(), ets:insert(EtsTab, {singleton, Val}), Val end. You have to make sure that the ETS table is owned by some long lived process and is of type public if you are going to invoke this function from multiple processes. Cheers Chandru > On 30 Oct 2017, at 17:05, Mikael Pettersson wrote: > > Yes. > > 1. (Boring) Have a persistent store somewhere, and have the function check if the value is stored there or not. If not, compute it and store. An ETS table would work fine, but you need to ensure it's lifetime matches your requirements. A plain file containing term_to_binary/1 of the value should also work. Or use a DB. > > 2. (More fun) Place the function in a module M. Initial version computes the value and generates a new version of the module where the function now just returns that constant value. There will probably be some restrictions on what kinds of values you can return this way, i.e. I wouldn't expect a function value with closed over free variables to work, but anything you could include in a pattern match should work. > >> On Mon, Oct 30, 2017 at 4:41 PM, Jaros?aw Pre? wrote: >> Hi, >> >> Is it possible to make an erlang function be evaluated only once at first call and then return this value each time? >> >> BR/Jarek >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Mon Oct 30 18:01:01 2017 From: anthonym@REDACTED (Anthony Molinaro) Date: Mon, 30 Oct 2017 10:01:01 -0700 Subject: [erlang-questions] one-time evaluated functions In-Reply-To: References: Message-ID: <196B5972-4C58-4FC8-8968-9DBC47CEFA29@alumni.caltech.edu> You can use mochiglobal or a similar trick to do what you want. https://github.com/mochi/mochiweb/blob/master/src/mochiglobal.erl -Anthony > On Oct 30, 2017, at 8:41 AM, Jaros?aw Pre? wrote: > > Hi, > > Is it possible to make an erlang function be evaluated only once at first call and then return this value each time? > > BR/Jarek > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Mon Oct 30 18:29:21 2017 From: chandrashekhar.mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Mon, 30 Oct 2017 17:29:21 +0000 Subject: [erlang-questions] one-time evaluated functions In-Reply-To: <42452B05-60CC-4EEF-9D42-1E0FE64C4157@gmail.com> References: <42452B05-60CC-4EEF-9D42-1E0FE64C4157@gmail.com> Message-ID: Actually this won?t work properly if you have a bunch of requests come in concurrently the first time around. Some of them will see undefined as the result. So you?ll need extra code to sleep and retry if the ets lookup returns undefined (unless undefined is a valid value!) > On 30 Oct 2017, at 17:20, Chandrashekhar Mullaparthi wrote: > > Here is the code for the boring solution. > > case ets:insert_new(EtsTab, {singleton, undefined}) of > false -> > ets:lookup_element(EtsTab, singleton, 2); > true -> > Val = evaluate_one_time_function(), > ets:insert(EtsTab, {singleton, Val}), > Val > end. > > You have to make sure that the ETS table is owned by some long lived process and is of type public if you are going to invoke this function from multiple processes. > > Cheers > Chandru > > >> On 30 Oct 2017, at 17:05, Mikael Pettersson wrote: >> >> Yes. >> >> 1. (Boring) Have a persistent store somewhere, and have the function check if the value is stored there or not. If not, compute it and store. An ETS table would work fine, but you need to ensure it's lifetime matches your requirements. A plain file containing term_to_binary/1 of the value should also work. Or use a DB. >> >> 2. (More fun) Place the function in a module M. Initial version computes the value and generates a new version of the module where the function now just returns that constant value. There will probably be some restrictions on what kinds of values you can return this way, i.e. I wouldn't expect a function value with closed over free variables to work, but anything you could include in a pattern match should work. >> >>> On Mon, Oct 30, 2017 at 4:41 PM, Jaros?aw Pre? wrote: >>> Hi, >>> >>> Is it possible to make an erlang function be evaluated only once at first call and then return this value each time? >>> >>> BR/Jarek >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Mon Oct 30 18:43:13 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 30 Oct 2017 18:43:13 +0100 Subject: [erlang-questions] sorting of type definitions using edoc In-Reply-To: References: Message-ID: No, there is currently no support for that, /Richard 2017-10-30 16:36 GMT+01:00 Jaros?aw Pre? : > Hi, > > Is it possible to turn off sorting of type definitions for generation of > htmls with use of edoc:files? > > BR/Jarek > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Mon Oct 30 19:05:18 2017 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Mon, 30 Oct 2017 18:05:18 +0000 Subject: [erlang-questions] one-time evaluated functions In-Reply-To: References: <42452B05-60CC-4EEF-9D42-1E0FE64C4157@gmail.com> Message-ID: When I don?t really know the best way to store such a value and it doesn?t change I just create a dedicated and locally named gen_server. If the state I want to store is ?large? I put it in the gen_server?s pdict which may or may not be a good idea. The process dictionary gets dumped when the server crashes. Fred Herbert has an interesting article about the pdict: https://ferd.ca/on-the-use-of-the-process-dictionary-in-erlang.html On Mon 30 Oct 2017 at 18:29, Chandrashekhar Mullaparthi < chandrashekhar.mullaparthi@REDACTED> wrote: > Actually this won?t work properly if you have a bunch of requests come in > concurrently the first time around. Some of them will see undefined as the > result. So you?ll need extra code to sleep and retry if the ets lookup > returns undefined (unless undefined is a valid value!) > > > On 30 Oct 2017, at 17:20, Chandrashekhar Mullaparthi < > chandrashekhar.mullaparthi@REDACTED> wrote: > > Here is the code for the boring solution. > > case ets:insert_new(EtsTab, {singleton, undefined}) of > false -> > ets:lookup_element(EtsTab, singleton, 2); > true -> > Val = evaluate_one_time_function(), > ets:insert(EtsTab, {singleton, Val}), > Val > end. > > You have to make sure that the ETS table is owned by some long lived > process and is of type public if you are going to invoke this function from > multiple processes. > > Cheers > Chandru > > > On 30 Oct 2017, at 17:05, Mikael Pettersson wrote: > > Yes. > > 1. (Boring) Have a persistent store somewhere, and have the function check > if the value is stored there or not. If not, compute it and store. An ETS > table would work fine, but you need to ensure it's lifetime matches your > requirements. A plain file containing term_to_binary/1 of the value should > also work. Or use a DB. > > 2. (More fun) Place the function in a module M. Initial version computes > the value and generates a new version of the module where the function now > just returns that constant value. There will probably be some restrictions > on what kinds of values you can return this way, i.e. I wouldn't expect a > function value with closed over free variables to work, but anything you > could include in a pattern match should work. > > On Mon, Oct 30, 2017 at 4:41 PM, Jaros?aw Pre? > wrote: > >> Hi, >> >> Is it possible to make an erlang function be evaluated only once at first >> call and then return this value each time? >> >> BR/Jarek >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Cheers, -- Pierre Fenoll -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Mon Oct 30 22:06:26 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Mon, 30 Oct 2017 22:06:26 +0100 Subject: [erlang-questions] map elements in defined order In-Reply-To: <098c5d52-5435-8c77-9fad-8c8247447034@cs.otago.ac.nz> References: <11a6bcc6-8fc7-0863-98de-ac071fdcc8b0@cs.otago.ac.nz> <018e93d8-a53d-c5e9-c920-57e7c7086e68@ericsson.com> <098c5d52-5435-8c77-9fad-8c8247447034@cs.otago.ac.nz> Message-ID: <306b99f2-7ffd-5d8a-7cc7-83ba06ad696b@ericsson.com> A clear statement of lack of guarantee makes sense. A 'canonical' option could make sense. That would require N*log(N) sorting to get map keys in canonical order (deja vu). About VM instance and version; I don't see the big divide there. Would it not be treacherous to base an application design on term_to_binary being a pure function among VM instances as long as you don't upgrade them. /Sverker On 10/30/2017 01:16 AM, Richard A. O'Keefe wrote: > > > On 28/10/17 2:34 AM, Sverker Eriksson wrote: > [term_to_binary/1 is not a pure function; it > depends on the representation of its argument, > not just its value]. > > OUCH. > > For term_to_binary/2, of course the result > depends on the Options argument, but I take > it now that even being explicit about the > Options is not enough.? Can we have a > 'canonical' option? > >> >> And in the case where M1 and M2 were created by different VM instances, >> you can with current implementation get different binaries. > > I can live with different *versions* of the VM using different > versions of the binary term format, but two instances of the *same* > VM turning mathematically identical terms into different binaries > is, well, did Nyarlathotep, the Crawling Chaos, have a hand in the > design?? In all seriousness, *OUCH*.? Please mention this in LARGE > red letters in the documentation for term_to_binary; I don't see it > at the moment, but it puts limits on what you can reasonably do > with terms-as-binaries. > > > From kostis@REDACTED Mon Oct 30 22:15:55 2017 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 30 Oct 2017 22:15:55 +0100 Subject: [erlang-questions] one-time evaluated functions In-Reply-To: References: Message-ID: On 10/30/2017 04:41 PM, Jaros?aw Pre? wrote: > Hi, > > Is it possible to make an erlang function be evaluated only once at > first call and then return this value each time? Many answers have already been given for this question. I only want to point out that the above only works for functions that are pure, in the sense that they are not only side-effect free (e.g. do not print, send messages, etc.), but also do not have dependencies on the state in any way. Kostis From sverker.eriksson@REDACTED Mon Oct 30 22:19:47 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Mon, 30 Oct 2017 22:19:47 +0100 Subject: [erlang-questions] one-time evaluated functions In-Reply-To: References: <42452B05-60CC-4EEF-9D42-1E0FE64C4157@gmail.com> Message-ID: And you should do ets:lookup_element *first* as that is the normal case and it only requires a read lock. /Sverker On 10/30/2017 06:29 PM, Chandrashekhar Mullaparthi wrote: > Actually this won?t work properly if you have a bunch of requests come > in concurrently the first time around. Some of them will see undefined > as the result. So you?ll need extra code to sleep and retry if the ets > lookup returns undefined (unless undefined is a valid value!) > > On 30 Oct 2017, at 17:20, Chandrashekhar Mullaparthi > > wrote: > >> Here is the code for the boring solution. >> >> case ets:insert_new(EtsTab, {singleton, undefined}) of >> ? ? false -> >> ? ? ? ? ets:lookup_element(EtsTab, singleton, 2); >> ? ? true -> >> ? ? ? ? Val = evaluate_one_time_function(), >> ? ? ? ? ?ets:insert(EtsTab, {singleton, Val}), >> ? ? ? ? Val >> ? ? end. >> >> You have to make sure that the ETS table is owned by some long lived >> process and is of type public if you are going to invoke this >> function from multiple processes. >> >> Cheers >> Chandru >> >> >> On 30 Oct 2017, at 17:05, Mikael Pettersson > > wrote: >> >>> Yes. >>> >>> 1. (Boring) Have a persistent store somewhere, and have the function >>> check if the value is stored there or not.? If not, compute it and >>> store.? An ETS table would work fine, but you need to ensure it's >>> lifetime matches your requirements.? A plain file containing >>> term_to_binary/1 of the value should also work.? Or use a DB. >>> >>> 2. (More fun) Place the function in a module M. Initial version >>> computes the value and generates a new version of the module where >>> the function now just returns that constant value.? There will >>> probably be some restrictions on what kinds of values you can return >>> this way, i.e. I wouldn't expect a function value with closed over >>> free variables to work, but anything you could include in a pattern >>> match should work. >>> >>> On Mon, Oct 30, 2017 at 4:41 PM, Jaros?aw Pre? >>> > wrote: >>> >>> Hi, >>> >>> Is it possible to make an erlang function be evaluated only once >>> at first call and then return this value each time? >>> >>> BR/Jarek >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From cchalasani@REDACTED Mon Oct 30 23:35:21 2017 From: cchalasani@REDACTED (Chaitanya Chalasani) Date: Tue, 31 Oct 2017 04:05:21 +0530 Subject: [erlang-questions] Timer implementation models Message-ID: <447C9AEF-2516-43CC-914C-AB56CC1C6888@me.com> Hi, I have hundred?s of processes each running its own timers. I can think of the following models. Wanted to know the pros and cons of each model. ** Model 1 ** loop1(State) -> receive after 1000 -> do_someting() end. ** Model 2 ** loop2(State) -> erlang:send_after(1000, self(), timeout), receive timeout -> do_someting(), erlang:send_after(1000, self(), timeout) end. ** Model 3 ** loop3(State) -> timer:send_interval(1000, self(), timeout), loop3a(State). loop3a(State) -> receive timeout -> do_someting() end. The above three models using the gen_server - ** Model 1 ** init(_Args) -> {ok, State, 1000}. handle_info(timeout, State) -> do_someting(), {noreply, State, 1000} ** Model 2 ** init(_Args) -> erlang:send_after(1000, self(), timeout), {ok, State}. handle_info(timeout, State) -> do_someting(), erlang:send_after(1000, self(), timeout), {noreply, State} ** Model 3 ** init(_Args) -> timer:send_interval(1000, self(), timeout), {ok, State}. handle_info(timeout, State) -> do_someting(), {noreply, State} /Chaitanya From nathaniel@REDACTED Tue Oct 31 00:16:07 2017 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Mon, 30 Oct 2017 19:16:07 -0400 Subject: [erlang-questions] Timer implementation models In-Reply-To: <447C9AEF-2516-43CC-914C-AB56CC1C6888@me.com> References: <447C9AEF-2516-43CC-914C-AB56CC1C6888@me.com> Message-ID: Well for one, send_after/3 and send_interval/3 will produce slightly different timing. Suppose your do_something/0 takes 50ms to run. Then in "Model 2", ignoring any complexities of scheduling, your function will be invoked at t+1000, t+2050, t+3100, t+4150, t+5200, t+6250, ... While "Model 3", under the same simplifying assumption would be invoked at t+1000, t+2000, t+3000, t+4000, t+5000, t+6000, ... To give you a few more choices: - When using gen_server, I prefer to call timer:apply_after/4 and timer:apply_interval/4. I either have them call gen_server:cast/2 or a local function that calls cast. - When using gen_server, you can give a timeout in your callback response. This works quite differently from the timer module, but might be useful for some things - When using gen_statem, you can give a timeouts like gen_server, but a wider variety of behavior is available > On Oct 30, 2017, at 6:35 PM, Chaitanya Chalasani wrote: > > Hi, > > I have hundred?s of processes each running its own timers. > I can think of the following models. Wanted to know the pros and cons of each model. > > ** Model 1 ** > loop1(State) -> > receive > after 1000 -> > do_someting() > end. > > ** Model 2 ** > loop2(State) -> > erlang:send_after(1000, self(), timeout), > receive > timeout -> > do_someting(), > erlang:send_after(1000, self(), timeout) > end. > > ** Model 3 ** > loop3(State) -> > timer:send_interval(1000, self(), timeout), > loop3a(State). > > loop3a(State) -> > receive > timeout -> > do_someting() > end. > > The above three models using the gen_server - > > ** Model 1 ** > init(_Args) -> > {ok, State, 1000}. > > handle_info(timeout, State) -> > do_someting(), > {noreply, State, 1000} > > ** Model 2 ** > init(_Args) -> > erlang:send_after(1000, self(), timeout), > {ok, State}. > > handle_info(timeout, State) -> > do_someting(), > erlang:send_after(1000, self(), timeout), > {noreply, State} > > ** Model 3 ** > init(_Args) -> > timer:send_interval(1000, self(), timeout), > {ok, State}. > > handle_info(timeout, State) -> > do_someting(), > {noreply, State} > > /Chaitanya > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Tue Oct 31 03:26:31 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 31 Oct 2017 15:26:31 +1300 Subject: [erlang-questions] one-time evaluated functions In-Reply-To: References: <42452B05-60CC-4EEF-9D42-1E0FE64C4157@gmail.com> Message-ID: <142b0519-64df-c7ae-f69b-c9ab5e1728c1@cs.otago.ac.nz> Some obvious questions: - when does this make sense? As others have noted, the function must be pure. The cost of re-evaluating the function should also be significantly higher than the cost of checking, in the context of the whole program. - how much does it actually save? This depends of course on the actual function. It also depends on how much the compiler can optimise the function, so the way the function is written may be significant. - is there prior art for this? Eiffel has "once" functions. In fact it has two kinds: once per class and once per instance. In Erlang, the equivalent might be that it makes as much sense, if not more, to cache f(X) as f(). It also reminds us that there are at least two versions that might make sense for Erlang: once per module, and once per process. Smalltalk historically made much use of "lazy initialisation" where you do cachedValue ifNil: [cachedValue := ...]. I used that a lot in my Smalltalk system, up to the point where I implemented threads. And then, mindful of the "double-checked locking is broken" issue, changed it all to load-time initialisation. The one exception is encoding and decoding tables, which are loaded on demand, and protected by a lock. In this context, the cost of locking was negligible. What do we learn from this? (1) It would be possible to have -once(f/0, module). f() -> ... implemented by having the loader load the module as usual, call f() getting V, and then patch the module to f() -> V. Others have explained using ETS tables, which has higher overhead, but can be done right now. (2) It is possible to simulate -once(f/0, process). by having f() -> case get({?MODULE,f}) of undefined -> V = real_f(), put({?MODULE,f}, [V]) ; [V] -> ok end, V. real_f() -> whatever f()'s body would have been. This could be done using a parse transform. (3) Concrete details matter. From icfp.publicity@REDACTED Tue Oct 31 04:15:34 2017 From: icfp.publicity@REDACTED (Lindsey Kuper) Date: Mon, 30 Oct 2017 20:15:34 -0700 Subject: [erlang-questions] Call for Sponsorships: ICFP 2018 Message-ID: <59f7ead6e8b0d_6d3c3fc7de057be4431e3@landin.local.mail> ICFP 2018 The 23rd ACM SIGPLAN International Conference on Functional Programming https://icfp18.sigplan.org Call for Sponsorships Web version of this call for sponsorships: https://icfp18.sigplan.org/attending/supporting-icfp ## Why Sponsor ICFP 2018? ICFP is the premier conference on functional programming languages, covering all aspects of theory, implementation, and application. Every year, we bring together over 500 world-leading researchers, practitioners, and students to discuss the latest findings, collaborate on new ideas, and meet new people. By sponsoring ICFP, your organization can demonstrate its commitment to supporting high quality research and to developing the next generation of functional programming experts. Most of our sponsorship funds are used to help students from around the world afford to attend the conference and get the most out of their experience. We believe that this commitment will pay dividends for our students, our sponsors, and the public for years to come. If you're interested in becoming a sponsor, we'd love to hear from you: get in touch with our sponsorship team at sponsorship-2018@REDACTED ## Sponsorship Opportunities and Benefits ### Bronze - $750 * Your logo on the ICFP 2018 website * Your name listed in the proceedings ### Silver - $3,000 * All of the benefits of Bronze sponsorship * One complimentary 3-day ICFP registration * A table at the industrial reception * Your logo in the proceedings * Your logo on publicity materials such as banners and posters ### Gold - $6,000 * All of the benefits of Silver sponsorship * One additional complimentary 3-day ICFP registration (2 in total) * A named supporter of the industrial reception * An opportunity to include branded merchandise in participants' swag bag ### Platinum - $10,000 * All the benefits of Gold sponsorship * One additional complimentary 3-day ICFP registration (3 in total) * A named supporter of ICFP 2018 * An opportunity to speak to the audience at the industrial reception * A table/booth-like space in the coffee break areas ## Additional Sponsorship Opportunities We offer some additional sponsorship options to sponsors at the silver level or above. ### Lanyard Sponsor - $4,000 You provide the lanyards that every attendee will wear around their neck. ### Video Sponsor - $4,000 ICFP makes videos available for free to non-attendees following the conference. As a video sponsor, you support the recording and release of these videos. In exchange, your logo will be displayed as part of every ICFP video. ### Banner - $1,000 Post a free-standing banner (up to 2m high and 1m wide, provided by you) on the ICFP main stage throughout the conference. From Jouni.Ryno@REDACTED Tue Oct 31 07:57:53 2017 From: Jouni.Ryno@REDACTED (=?utf-8?B?Unluw7YgSm91bmkgKEZNSSk=?=) Date: Tue, 31 Oct 2017 06:57:53 +0000 Subject: [erlang-questions] run_erl and systemd Message-ID: <1509433072.2295.97.camel@fmi.fi> Hi list I have an odd systemd and run_erl issue. This service definition more or less "works" on a "physical" machine, but not on a KVM-virtual machine. [Unit] Description=My node After=network.target [Service] Environment=RUNNER_LOG_DIR=/var/log/myapp Type=forking ExecStartPre=/bin/sleep 60 ExecStart=/proj/myapp/bin/myapp start ExecStop=/proj/myapp/bin/myapp stop RestartSec=20 Restart=on-failure User=me Group=me [Install] WantedBy=multi-user.target epmd is running and I just try to start it via "systemctl start my.service" from a command line Plus I do get some odd inet_gethost and epmd died errors even in the physical machine, but at least the run_erl starts at some point ... Is there something, that I should do differently or add in the environment? Jouni PS.somehow the init.d shell scripts were easier to debug ... -- Jouni Ryn? mailto://Jouni.Ryno@REDACTED/ http://space.fmi.fi/~ryno/ Finnish Meteorological Institute http://www.fmi.fi/ Observation services P.O.BOX 503 Tel (+358)-29-5394656 FI-00101 Helsinki FAX (+358)-29-5395703 Finland priv-GSM (+358)-50-5302903 md5sum(the Ultimate Question of Life, The Universe, and Everything) = 0000000000000000000000000000002A From ali.sabil@REDACTED Tue Oct 31 10:52:00 2017 From: ali.sabil@REDACTED (Ali Sabil) Date: Tue, 31 Oct 2017 15:22:00 +0530 Subject: [erlang-questions] run_erl and systemd In-Reply-To: <1509433072.2295.97.camel@fmi.fi> References: <1509433072.2295.97.camel@fmi.fi> Message-ID: I tend to use `/proj/myapp/bin/myapp foreground` and avoid run_erl altogether. On Tue, Oct 31, 2017 at 12:27 PM, Ryn? Jouni (FMI) wrote: > Hi list > > I have an odd systemd and run_erl issue. This service definition more > or less "works" on a "physical" machine, but not on a KVM-virtual > machine. > > [Unit] > Description=My node > After=network.target > > [Service] > Environment=RUNNER_LOG_DIR=/var/log/myapp > Type=forking > ExecStartPre=/bin/sleep 60 > ExecStart=/proj/myapp/bin/myapp start > ExecStop=/proj/myapp/bin/myapp stop > RestartSec=20 > Restart=on-failure > User=me > Group=me > > [Install] > WantedBy=multi-user.target > > > epmd is running and I just try to start it via "systemctl start > my.service" from a command line > > Plus I do get some odd inet_gethost and epmd died errors even in the > physical machine, but at least the run_erl starts at some point ... > > > Is there something, that I should do differently or add in the > environment? > Jouni > > PS.somehow the init.d shell scripts were easier to debug ... > > -- > Jouni Ryn? mailto://Jouni.Ryno@REDACTED/ > http://space.fmi.fi/~ryno/ > Finnish Meteorological Institute http://www.fmi.fi/ > Observation services > P.O.BOX 503 Tel (+358)-29-5394656 > FI-00101 Helsinki FAX (+358)-29-5395703 > Finland priv-GSM (+358)-50-5302903 > > md5sum(the Ultimate Question of Life, The Universe, and Everything) = > 0000000000000000000000000000002A > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Oct 31 11:51:46 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 31 Oct 2017 10:51:46 +0000 Subject: [erlang-questions] map elements in defined order In-Reply-To: References: Message-ID: On Sat, Oct 28, 2017 at 11:09 PM Richard Carlsson < carlsson.richard@REDACTED> wrote: > As we quoted from the reference manual, the < ordering on maps is actually > implementation independent and future proof (while still being a total > order). The sacrifice is that to compare two maps with <, their keys must > be mapped into canonical order (with integers before floats as discussed). > This is clearly more costly than just taking whatever order the current > underlying hash+HAMT produces, but worth it since it preserves those nice > properties. > > Oh, then I was wrong! Thanks for the correction. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cchalasani@REDACTED Tue Oct 31 11:55:32 2017 From: cchalasani@REDACTED (Chaitanya Chalasani) Date: Tue, 31 Oct 2017 16:25:32 +0530 Subject: [erlang-questions] Timer implementation models In-Reply-To: References: <447C9AEF-2516-43CC-914C-AB56CC1C6888@me.com> Message-ID: <99534059-99E5-441B-975C-63367353C4BE@me.com> Thank you Nathaniel, understood about the precision. If precision isn?t my criteria and considering 100s to 1000s of timers running concurrently, is there any advantage for the model 2 over model 3. > On 31-Oct-2017, at 4:46 AM, Nathaniel Waisbrot wrote: > > Well for one, send_after/3 and send_interval/3 will produce slightly different timing. > > Suppose your do_something/0 takes 50ms to run. Then in "Model 2", ignoring any complexities of scheduling, your function will be invoked at > > t+1000, t+2050, t+3100, t+4150, t+5200, t+6250, ... > > While "Model 3", under the same simplifying assumption would be invoked at > > t+1000, t+2000, t+3000, t+4000, t+5000, t+6000, ... > > > To give you a few more choices: > > - When using gen_server, I prefer to call timer:apply_after/4 and timer:apply_interval/4. I either have them call gen_server:cast/2 or a local function that calls cast. > - When using gen_server, you can give a timeout in your callback response. This works quite differently from the timer module, but might be useful for some things > - When using gen_statem, you can give a timeouts like gen_server, but a wider variety of behavior is available > > > >> On Oct 30, 2017, at 6:35 PM, Chaitanya Chalasani wrote: >> >> Hi, >> >> I have hundred?s of processes each running its own timers. >> I can think of the following models. Wanted to know the pros and cons of each model. >> >> ** Model 1 ** >> loop1(State) -> >> receive >> after 1000 -> >> do_someting() >> end. >> >> ** Model 2 ** >> loop2(State) -> >> erlang:send_after(1000, self(), timeout), >> receive >> timeout -> >> do_someting(), >> erlang:send_after(1000, self(), timeout) >> end. >> >> ** Model 3 ** >> loop3(State) -> >> timer:send_interval(1000, self(), timeout), >> loop3a(State). >> >> loop3a(State) -> >> receive >> timeout -> >> do_someting() >> end. >> >> The above three models using the gen_server - >> >> ** Model 1 ** >> init(_Args) -> >> {ok, State, 1000}. >> >> handle_info(timeout, State) -> >> do_someting(), >> {noreply, State, 1000} >> >> ** Model 2 ** >> init(_Args) -> >> erlang:send_after(1000, self(), timeout), >> {ok, State}. >> >> handle_info(timeout, State) -> >> do_someting(), >> erlang:send_after(1000, self(), timeout), >> {noreply, State} >> >> ** Model 3 ** >> init(_Args) -> >> timer:send_interval(1000, self(), timeout), >> {ok, State}. >> >> handle_info(timeout, State) -> >> do_someting(), >> {noreply, State} >> >> /Chaitanya >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From essen@REDACTED Tue Oct 31 15:33:17 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Tue, 31 Oct 2017 14:33:17 +0000 Subject: [erlang-questions] The export_all warning is now useless Message-ID: Hello, The export_all warning used to be very useful before. It was only enabled when you used an option (warn_export_all). The warning has now been enabled by default. As a result, an additional option has been added to disable it when necessary (nowarn_export_all). This made the warn_export_all warning useless. Why? Because you can disable this warning at the module level. For example: -module(abc). -compile(export_all). -compile(nowarn_export_all). A module with both of these options will not trigger a warning anymore and still export everything. This is great if you're writing a test suite with a lot of functions because you don't have to worry about the warning. It's not so great when you want to actually detect export_all uses because developers can now just use both options in a module and completely silence this warning in every module where it should trigger! The only way I can see to detect it now is to grep through source files... Flawed design. Cheers, -- Lo?c Hoguin https://ninenines.eu From codewiget95@REDACTED Tue Oct 31 15:41:26 2017 From: codewiget95@REDACTED (code wiget) Date: Tue, 31 Oct 2017 10:41:26 -0400 Subject: [erlang-questions] How does Erlang TCP determine the end of a TCP stream? Message-ID: <3A3B337C-E459-4039-BD4A-8805EFD7DC07@gmail.com> Hello, From my understanding, tcp is a stream based protocol, and you can?t really tell one packet from another. You can make sure the packets arrive in order, but whether or not they should be ?grouped? is not seen by the application layer. But somehow Erlang ssl and tcp abstract this away. In my code, I have some sort of the following: Handle_info({tcp/ssl, Sock, Data}, State) -> ? and every time that I receive data into the socket, it comes as the ?full package?, meaning that when another server sends a message to it over the socket, It is received as the full length of the packet every time with no extra bytes here or there. On the other hand, a co-worker?s server has to implement socket reads and can?t determine one ?group? from another. They have to read in the length or set delimiters. What is special about OTP that delimits these packets? From vimal7370@REDACTED Tue Oct 31 16:25:33 2017 From: vimal7370@REDACTED (Vimal Kumar) Date: Tue, 31 Oct 2017 20:55:33 +0530 Subject: [erlang-questions] How does Erlang TCP determine the end of a TCP stream? In-Reply-To: <3A3B337C-E459-4039-BD4A-8805EFD7DC07@gmail.com> References: <3A3B337C-E459-4039-BD4A-8805EFD7DC07@gmail.com> Message-ID: Hi, I believe you are using with the inet:setopts/2 'active' option set to true? Set it to false or once and then use gen_tcp:recv/2,3 to read from tcp stream upto the length you want. http://erlang.org/doc/man/inet.html#setopts-2 On Tue, Oct 31, 2017 at 8:11 PM, code wiget wrote: > Hello, > > From my understanding, tcp is a stream based protocol, and you can?t > really tell one packet from another. You can make sure the packets arrive > in order, but whether or not they should be ?grouped? is not seen by the > application layer. > > But somehow Erlang ssl and tcp abstract this away. > > In my code, I have some sort of the following: > > Handle_info({tcp/ssl, Sock, Data}, State) -> ? and every time that I > receive data into the socket, it comes as the ?full package?, meaning that > when another server sends a message to it over the socket, It is received > as the full length of the packet every time with no extra bytes here or > there. On the other hand, a co-worker?s server has to implement socket > reads and can?t determine one ?group? from another. They have to read in > the length or set delimiters. > > > What is special about OTP that delimits these packets? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From scott_ribe@REDACTED Tue Oct 31 16:42:48 2017 From: scott_ribe@REDACTED (scott ribe) Date: Tue, 31 Oct 2017 09:42:48 -0600 Subject: [erlang-questions] How does Erlang TCP determine the end of a TCP stream? In-Reply-To: <3A3B337C-E459-4039-BD4A-8805EFD7DC07@gmail.com> References: <3A3B337C-E459-4039-BD4A-8805EFD7DC07@gmail.com> Message-ID: On Oct 31, 2017, at 8:41 AM, code wiget wrote: > > Handle_info({tcp/ssl, Sock, Data}, State) -> ? and every time that I receive data into the socket, it comes as the ?full package?, meaning that when another server sends a message to it over the socket, It is received as the full length of the packet every time with no extra bytes here or there. Pure coincidence. Yes, it is highly likely that a single send on one end comes through a single receive on the other end, but there is no guarantee so you must figure out framing yourself. -- Scott Ribe https://www.linkedin.com/in/scottribe/ (303) 722-0567 From codewiget95@REDACTED Tue Oct 31 17:10:49 2017 From: codewiget95@REDACTED (code wiget) Date: Tue, 31 Oct 2017 12:10:49 -0400 Subject: [erlang-questions] How does Erlang TCP determine the end of a TCP stream? In-Reply-To: References: <3A3B337C-E459-4039-BD4A-8805EFD7DC07@gmail.com> Message-ID: Vimal and Scott, I would like to clarify quickly that this is not a coincidence, and that I am processing ~500 messages/min over 1 socket connected to 1 client server and all of the messages are ?framed? properly. There is something in the OTP that is framing the packets. Vimal - I am using {active, once}. And I like when it comes in as a full packet, so I do not want to change anything that could eliminate that behavior. When I get a handle info and ensure that the data is the right length in the guard, I can then do validation on the packet (we use a start code and stop code of 4 bytes) to ensure the validity of the packet from there. Having a stream and using gen_tcp:recv would complicate the gen_server in my opinion and it is clean now. I just went to the Erlang man pages on the set_opts, and it seems like {active, once} works for my purposes: "If the value is once ({active, once}), onedata message from the socket is sent to the process?.. Use active mode only if your high-level protocol provides its own flow control (for example, acknowledging received messages) or the amount of data exchanged is small.? It does not define how long ?one data message? is, but from reading on, it seems like the small size of my packets is the reason it is working. All of the packets I am expecting are < 500 bytes. This settles this a bit for me, I can sleep at night, but we are working on an assumption about erlang?s ?one data message?. If anyone can elaborate on this, or has some more in depth knowledge, I would greatly appreciate your expertise. -------------- next part -------------- An HTML attachment was scrubbed... URL: From scott_ribe@REDACTED Tue Oct 31 17:20:36 2017 From: scott_ribe@REDACTED (scott ribe) Date: Tue, 31 Oct 2017 10:20:36 -0600 Subject: [erlang-questions] How does Erlang TCP determine the end of a TCP stream? In-Reply-To: References: <3A3B337C-E459-4039-BD4A-8805EFD7DC07@gmail.com> Message-ID: <070C5402-1C5C-4F5F-A319-D83B1C1D1258@elevated-dev.com> On Oct 31, 2017, at 10:10 AM, code wiget wrote: > > I would like to clarify quickly that this is not a coincidence, and that I am processing ~500 messages/min over 1 socket connected to 1 client server and all of the messages are ?framed? properly. There is something in the OTP that is framing the packets. It is a coincidence and there is nothing framing the *MESSAGES*. > It does not define how long ?one data message? is, but from reading on, it seems like the small size of my packets is the reason it is working. All of the packets I am expecting are < 500 bytes. That does make it extremely likely that each message will be delivered in a single packet, but there is still no guarantee. There is also no guarantee 2 or more messages won't be delivered in a single packet. -- Scott Ribe https://www.linkedin.com/in/scottribe/ (303) 722-0567 From erlang.org@REDACTED Tue Oct 31 17:21:49 2017 From: erlang.org@REDACTED (Stanislaw Klekot) Date: Tue, 31 Oct 2017 17:21:49 +0100 Subject: [erlang-questions] How does Erlang TCP determine the end of a TCP stream? In-Reply-To: References: <3A3B337C-E459-4039-BD4A-8805EFD7DC07@gmail.com> Message-ID: <20171031162149.GA9596@jarowit.net> On Tue, Oct 31, 2017 at 12:10:49PM -0400, code wiget wrote: > Vimal - I am using {active, once}. And I like when it comes in as > a full packet, so I do not want to change anything that could > eliminate that behavior. This behaviour is a coincidence that bases on your Erlang code being able to keep up with the sender, and as such, it's brittle. Do a test: wait some time (1s) in your code before setting {active,once} again and see what you'll get. My bet is several packets clumped together. This is how read() and recv() work on stream BSD sockets. If the socket's buffer is non-empty, they return whatever is in the buffer (up to read length). There's nothing magical that Erlang would do on the sockets for {packet,raw}. -- Stanislaw Klekot From kjnilsson@REDACTED Tue Oct 31 17:38:11 2017 From: kjnilsson@REDACTED (Karl Nilsson) Date: Tue, 31 Oct 2017 16:38:11 +0000 Subject: [erlang-questions] The export_all warning is now useless In-Reply-To: References: Message-ID: I think you should have words with these pesky developers exporting all things. I wasn't aware of the new option and it will probably work nicely for me as I can now re-enable warn_export_all in my vim syntax checker and explicitly ignore it in test modules and such. I don't think there was a nice way to achieve this before. Swings and roundabouts I suppose. On Tue, 31 Oct 2017 at 14:33 Lo?c Hoguin wrote: > Hello, > > The export_all warning used to be very useful before. It was only > enabled when you used an option (warn_export_all). > > The warning has now been enabled by default. As a result, an additional > option has been added to disable it when necessary (nowarn_export_all). > > This made the warn_export_all warning useless. Why? Because you can > disable this warning at the module level. For example: > > -module(abc). > -compile(export_all). > -compile(nowarn_export_all). > > A module with both of these options will not trigger a warning anymore > and still export everything. This is great if you're writing a test > suite with a lot of functions because you don't have to worry about the > warning. > > It's not so great when you want to actually detect export_all uses > because developers can now just use both options in a module and > completely silence this warning in every module where it should trigger! > The only way I can see to detect it now is to grep through source files... > > Flawed design. > > Cheers, > > -- > Lo?c Hoguin > https://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Oct 31 17:39:59 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Tue, 31 Oct 2017 16:39:59 +0000 Subject: [erlang-questions] The export_all warning is now useless In-Reply-To: References: Message-ID: <2c665eaa-87f0-e4e6-4fdc-ae79d1f270f7@ninenines.eu> Before I could have ERLC_OPTS with warn_export_all and TEST_ERLC_OPTS without, effectively disabling it only for normal code and allowing it for tests. Now? I better read those patches carefully. On 10/31/2017 04:38 PM, Karl Nilsson wrote: > I think you should have words with these pesky developers exporting all > things. > > I wasn't aware of the new option and it will probably work nicely for me > as I can now re-enable warn_export_all in my vim syntax checker and > explicitly ignore it in test modules and such. I don't think there was a > nice way to achieve this before. Swings and roundabouts I suppose. > > On Tue, 31 Oct 2017 at 14:33 Lo?c Hoguin > wrote: > > Hello, > > The export_all warning used to be very useful before. It was only > enabled when you used an option (warn_export_all). > > The warning has now been enabled by default. As a result, an additional > option has been added to disable it when necessary (nowarn_export_all). > > This made the warn_export_all warning useless. Why? Because you can > disable this warning at the module level. For example: > > -module(abc). > -compile(export_all). > -compile(nowarn_export_all). > > A module with both of these options will not trigger a warning anymore > and still export everything. This is great if you're writing a test > suite with a lot of functions because you don't have to worry about the > warning. > > It's not so great when you want to actually detect export_all uses > because developers can now just use both options in a module and > completely silence this warning in every module where it should trigger! > The only way I can see to detect it now is to grep through source > files... > > Flawed design. > > Cheers, > > -- > Lo?c Hoguin > https://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From kuna.prime@REDACTED Tue Oct 31 19:24:18 2017 From: kuna.prime@REDACTED (Karlo Kuna) Date: Tue, 31 Oct 2017 19:24:18 +0100 Subject: [erlang-questions] mnesia bound keys Message-ID: i was wondering what exactly "key is bound" means in context of mnesia -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Tue Oct 31 20:15:31 2017 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Tue, 31 Oct 2017 20:15:31 +0100 Subject: [erlang-questions] mnesia bound keys In-Reply-To: References: Message-ID: Presumably you're referring to mnesia:match_object/1 or /3. It means that the key position of the pattern is a ground value and not a variable. On Tue, Oct 31, 2017 at 7:24 PM, Karlo Kuna wrote: > i was wondering what exactly "key is bound" means in context of mnesia > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From codewiget95@REDACTED Tue Oct 31 20:23:50 2017 From: codewiget95@REDACTED (asdf asdf) Date: Tue, 31 Oct 2017 15:23:50 -0400 Subject: [erlang-questions] How does Erlang TCP determine the end of a TCP stream? In-Reply-To: <20171031162149.GA9596@jarowit.net> References: <3A3B337C-E459-4039-BD4A-8805EFD7DC07@gmail.com> <20171031162149.GA9596@jarowit.net> Message-ID: <08bb6384-d7ca-402c-b4c6-e0f4e65e96a6@Spark> Stanislaw, Is there a common solution to this problem? Erlang was built to be used in servers/switches, i would assume that there was a solution built into OTP? If not, do you handle it just at the application later with gen_tcp:recv() and read X bytes at a time from the socket based on delimiters and such? How would you reconcile this with using a gen_server implementation that receives messages from the ssl socket in handle info ? I appreciate the help, I?m happy I didn?t roll this further On Oct 31, 2017, 12:21 PM -0400, Stanislaw Klekot , wrote: > On Tue, Oct 31, 2017 at 12:10:49PM -0400, code wiget wrote: > > Vimal - I am using {active, once}. And I like when it comes in as > > a full packet, so I do not want to change anything that could > > eliminate that behavior. > > This behaviour is a coincidence that bases on your Erlang code being > able to keep up with the sender, and as such, it's brittle. > > Do a test: wait some time (1s) in your code before setting {active,once} > again and see what you'll get. My bet is several packets clumped > together. > > This is how read() and recv() work on stream BSD sockets. If the > socket's buffer is non-empty, they return whatever is in the buffer (up > to read length). There's nothing magical that Erlang would do on the > sockets for {packet,raw}. > > -- > Stanislaw Klekot -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Oct 31 20:26:19 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 31 Oct 2017 19:26:19 +0000 Subject: [erlang-questions] Timer implementation models In-Reply-To: <99534059-99E5-441B-975C-63367353C4BE@me.com> References: <447C9AEF-2516-43CC-914C-AB56CC1C6888@me.com> <99534059-99E5-441B-975C-63367353C4BE@me.com> Message-ID: The timer module is often a performance problem, so avoiding that is a good idea (It's in the efficiency guide as well). Gen server variants: Model 1 will reset the timer if an event arrives. This means the timer will never trigger if events arrive before 1000ms each time (it'll starve the timeout message forever). Model 2, seems a bit better to me. Raw variants: All models look fine to me. On Tue, Oct 31, 2017 at 11:55 AM Chaitanya Chalasani wrote: > Thank you Nathaniel, understood about the precision. > > If precision isn?t my criteria and considering 100s to 1000s of timers > running concurrently, is there any advantage for the model 2 over model 3. > > > On 31-Oct-2017, at 4:46 AM, Nathaniel Waisbrot > wrote: > > > > Well for one, send_after/3 and send_interval/3 will produce slightly > different timing. > > > > Suppose your do_something/0 takes 50ms to run. Then in "Model 2", > ignoring any complexities of scheduling, your function will be invoked at > > > > t+1000, t+2050, t+3100, t+4150, t+5200, t+6250, ... > > > > While "Model 3", under the same simplifying assumption would be invoked > at > > > > t+1000, t+2000, t+3000, t+4000, t+5000, t+6000, ... > > > > > > To give you a few more choices: > > > > - When using gen_server, I prefer to call timer:apply_after/4 and > timer:apply_interval/4. I either have them call gen_server:cast/2 or a > local function that calls cast. > > - When using gen_server, you can give a timeout in your callback > response. This works quite differently from the timer module, but might be > useful for some things > > - When using gen_statem, you can give a timeouts like gen_server, but a > wider variety of behavior is available > > > > > > > >> On Oct 30, 2017, at 6:35 PM, Chaitanya Chalasani > wrote: > >> > >> Hi, > >> > >> I have hundred?s of processes each running its own timers. > >> I can think of the following models. Wanted to know the pros and cons > of each model. > >> > >> ** Model 1 ** > >> loop1(State) -> > >> receive > >> after 1000 -> > >> do_someting() > >> end. > >> > >> ** Model 2 ** > >> loop2(State) -> > >> erlang:send_after(1000, self(), timeout), > >> receive > >> timeout -> > >> do_someting(), > >> erlang:send_after(1000, self(), timeout) > >> end. > >> > >> ** Model 3 ** > >> loop3(State) -> > >> timer:send_interval(1000, self(), timeout), > >> loop3a(State). > >> > >> loop3a(State) -> > >> receive > >> timeout -> > >> do_someting() > >> end. > >> > >> The above three models using the gen_server - > >> > >> ** Model 1 ** > >> init(_Args) -> > >> {ok, State, 1000}. > >> > >> handle_info(timeout, State) -> > >> do_someting(), > >> {noreply, State, 1000} > >> > >> ** Model 2 ** > >> init(_Args) -> > >> erlang:send_after(1000, self(), timeout), > >> {ok, State}. > >> > >> handle_info(timeout, State) -> > >> do_someting(), > >> erlang:send_after(1000, self(), timeout), > >> {noreply, State} > >> > >> ** Model 3 ** > >> init(_Args) -> > >> timer:send_interval(1000, self(), timeout), > >> {ok, State}. > >> > >> handle_info(timeout, State) -> > >> do_someting(), > >> {noreply, State} > >> > >> /Chaitanya > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Oct 31 20:38:36 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 31 Oct 2017 19:38:36 +0000 Subject: [erlang-questions] How does Erlang TCP determine the end of a TCP stream? In-Reply-To: <3A3B337C-E459-4039-BD4A-8805EFD7DC07@gmail.com> References: <3A3B337C-E459-4039-BD4A-8805EFD7DC07@gmail.com> Message-ID: On Tue, Oct 31, 2017 at 3:41 PM code wiget wrote: > Handle_info({tcp/ssl, Sock, Data}, State) -> ? and every time that I > receive data into the socket, it comes as the ?full package?, meaning that > when another server sends a message to it over the socket, It is received > as the full length of the packet every time with no extra bytes here or > there. On the other hand, a co-worker?s server has to implement socket > reads and can?t determine one ?group? from another. They have to read in > the length or set delimiters. > If you set an option such as {packet, 4} on the socket, then the VM will expect a 4 byte big endian length header followed by that many bytes in payload. Running with {active, once} or {active, N} will have the socket send you payloads one message at a time, stripped of said header. If you don't set an option such as {packet, 4} then the VM can send you anything from 1 byte to buffering everything for a while and deliver a megabyte to you. In this case, you are receiving chunks of the stream at a time. It is common to receive around the MTU of the underlying network if your rate is fairly low (something like 1440-1460 bytes on ethernet is typical). You will have to do some work on your end in order to handle the case where one chunk doesn't have all the data necessary for a correct decode. The typical solution is to buffer the partial chunk in the process and then append it when the next message arrives, trying to decode then. It is a coincidence if you are in the latter of the above cases and happen to receive things in "full package" form. It'll break in any real network setting. Beware localhost as an interface, which often has a 16K or 64K MTU. Packets are then not broken here, but will be in any real network. If you can, run with something like {packet, 4}. It is simple and works. Delimiters are worse because you have to scan for them and you often need to escape them in your payload. A Length-Value-Type encoding provides framing in which you don't have to scan and you don't need to escape. Putting the type last in the packet is debatable, but one advantage is that other languages have to allocate a full buffer which can often eliminate certain security-concerning bad implementations. -------------- next part -------------- An HTML attachment was scrubbed... URL: From scott_ribe@REDACTED Tue Oct 31 21:04:31 2017 From: scott_ribe@REDACTED (scott ribe) Date: Tue, 31 Oct 2017 14:04:31 -0600 Subject: [erlang-questions] How does Erlang TCP determine the end of a TCP stream? In-Reply-To: References: <3A3B337C-E459-4039-BD4A-8805EFD7DC07@gmail.com> Message-ID: On Oct 31, 2017, at 1:38 PM, Jesper Louis Andersen wrote: > > If you can, run with something like {packet, 4}. It is simple and works. Delimiters are worse because you have to scan for them and you often need to escape them in your payload. A Length-Value-Type encoding provides framing in which you don't have to scan and you don't need to escape. Putting the type last in the packet is debatable, but one advantage is that other languages have to allocate a full buffer which can often eliminate certain security-concerning bad implementations. I like length-prefixed messages, and I like that Erlang has such direct & easy support built-in. I'm just going to suggest one more thing: add your own checksum at the end. In case there's any bug anywhere in decoding packets, this prevents the kind of error where you get out of sync and are then taking what should be message data as your 4 bytes of length. Admittedly, that is far less likely with OTP's built-in implementation than if you're writing your buffering and decoding. But it's also possible for a sender to send a bogus length, or maybe the length is right but it fails to send the last byte before sending the length of the next message... -- Scott Ribe https://www.linkedin.com/in/scottribe/ (303) 722-0567 From kuna.prime@REDACTED Tue Oct 31 22:48:37 2017 From: kuna.prime@REDACTED (Karlo Kuna) Date: Tue, 31 Oct 2017 22:48:37 +0100 Subject: [erlang-questions] mnesia bound keys In-Reply-To: References: Message-ID: i still don't fully grasp the concept. short example would be greatly appreciated. On Tue, Oct 31, 2017 at 8:15 PM, Mikael Pettersson wrote: > Presumably you're referring to mnesia:match_object/1 or /3. It means that > the key position of the pattern is a ground value and not a variable. > > On Tue, Oct 31, 2017 at 7:24 PM, Karlo Kuna wrote: > >> i was wondering what exactly "key is bound" means in context of mnesia >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ilya.khlopotov@REDACTED Tue Oct 31 23:19:48 2017 From: ilya.khlopotov@REDACTED (ILYA Khlopotov) Date: Tue, 31 Oct 2017 15:19:48 -0700 Subject: [erlang-questions] sharing config via a dynamically compiled module? In-Reply-To: <92698F02-4ABC-457E-923C-42DBDD59B476@gmail.com> References: <92698F02-4ABC-457E-923C-42DBDD59B476@gmail.com> Message-ID: config as module approach is also used in CouchDB plugin system here https://github.com/apache/couchdb/blob/master/src/couch_epi/src/couch_epi_data_gen.erl On October 28, 2017 12:39:07 AM PDT, Benoit Chesneau wrote: > > >> On 28 Oct 2017, at 08:38, Max Lapshin wrote: >> >> Take a look at thread nearby. >> >> > All literals belong to a loaded module instance. When that (old) >module instance is purged, all process heaps are scanned and heaps >containing those literals will do a special garbage collection where >the literals are copied. >> >> Each config change may lead to rescanning heaps of all processes. >> >> >> Frankly speaking global ets is much easier =) >> > > >Right. One another advantage to ETS i that you can store any terms even >refs where it's not possible . At the cost of increasing the number of >ETS tables used though... > >- beno?t >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: