From fritchie@REDACTED Thu Apr 1 00:56:33 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 31 Mar 2010 17:56:33 -0500 Subject: Using UBF outside of UBF, was: Using -spec for callbacks when defining behaviours In-Reply-To: Message of "Fri, 26 Mar 2010 04:14:41 PDT." <8800f3ba-898a-4445-94e4-36189dd1627e@n34g2000yqb.googlegroups.com> Message-ID: <19355.1270076193@snookles.snookles.com> Mikael Karlsson wrote: mk> Yes, I think that contract checkers that are "pluggable" at mk> validation time is a good way to go, and if you find a standardized mk> way or "formalized" pattern to put it in the process of Erlang code mk> making the better. Joe Armstrongs UBF contracts is a good example. I mk> agree that you should not rely on static checks, but on the other mk> hand they are a good first check and especially for Erlang I think mk> you can catch errors that you would later at run-time (Dialyzer do mk> find errors). For those readers who don't know what UBF is or where to find its source code, please see the following projects at GitHub: http://github.com/norton/ubf http://github.com/norton/ubf-abnf http://github.com/norton/ubf-eep8 http://github.com/norton/ubf-jsonrpc I think it'd be cool to see a paper at this year's ACM ICFP Erlang Workshop that tried to bridge the gap between dynamic and static typing by, for example, putting the UBF contract checker in between calls to & from Erlang functions (without TCP or whatever in the middle). During my talk last week at Erlang Factory San Francisco, Joe Armstrong asked me if we (my employer, Gemini Mobile) bothered to turn off the contract checker in production or if we'd measured its overhead. Gemini hasn't bothered because it has been fast enough not to worry about(*). If someone wants to really measure the cost and write up the results, that'd be cool too. -Scott (*) Besides, the security blanket of the checker's extreme fussiness is nice. :-) From hakan@REDACTED Thu Apr 1 02:58:17 2010 From: hakan@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Thu, 1 Apr 2010 02:58:17 +0200 Subject: [erlang-questions] About mnesia in multi-nodes In-Reply-To: References: Message-ID: On Tue, Mar 30, 2010 at 5:12 PM, ??? wrote: > Hi, I create mnesia schema in 2 nodes successfully, and create tables with > disc_copies option. After writing one record into one table, I close all > erlang nodes. Then I restart one node, call mnesia:start(), use tv:start() > to view records in that table, but I see none. If I start 2 nodes, I can see > records. Is there any way I can make sure records wrote all nodes, so I can > start my system when only one node runing? The records has safely been written to disk on both nodes. Even if it seems a little bit confusing, Mnesia works as expected. The table load mechanism is a part of the transaction recovery mechanism. When Mnesia on one node looses contact with Mnesia on another node, this is noted in the transaction log. At that time point Mnesia knows that the node that still is up and running has a consistent replica of its tables. The replica on he node that was shutdown first may be outdated since the other nodes may update their replicas of the table. At startup Mnesia tries to connect with the other nodes and if that suceeds it loads its tables from them. If the other nodes are down, it looks for mnesia_down marks in its local transaction log in order to determine if it has a consistent replica or not of its tables. The node that was shutdown last has mnesia_down's from all the other nodes. This means that it safely can load its tables. If some of the other nodes where started first (as in your case) Mnesia will wait indefinitely for another node to connect in order to load its tables However, if the other node cannot be started for some reason, it is possible to bypass the transaction recovery mechanism by explicitly calling mnesia:force_load_table/1 to load the local replica of the table (regardless of it is up-to-date or not). But as it can lead to database inconsistency it must be used with care. The same goes for the configuration parameter max_wait_for_decision (see the reference manual) which also may cause Mnesia to bypass the transaction recovery mechansism when some nodes refuses to start. You trade availability for consistency. /H?kan Erlang/OTP, Ericsson AB From hd2010@REDACTED Thu Apr 1 04:03:12 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 01 Apr 2010 04:03:12 +0200 Subject: [erlang-questions] The erlang baby and the green JVM In-Reply-To: <4BB22AAD.7020104@erlang-solutions.com> References: <466fba8d1003271202h74ae19b5gbeb49cd1c5cdf696@mail.gmail.com> <4bae685a.091abc0a.1f68.6a4dSMTPIN_ADDED@mx.google.com> <3dbc6d1c1003271345r3d87e054y5221452315121107@mail.gmail.com> <5287075483C24C4D87E0BA93CFADE093@SSI.CORP> <4BB22AAD.7020104@erlang-solutions.com> Message-ID: <4BB3FEE0.6000902@eonblast.com> Judging from the comments, this is going to be around forever, like Cedric's and Damien's rants. So before somebody does it wrong, I added the info to Wikipedia's Erlang and JVM entries. At least it's official then, it was a serious omission. And tomorrow, maybe somebody can add a "newbs read here" tag to the blog post, at the top, blinking and cyan. Henning Ulf Wiger wrote: > David Mercer wrote: >> My favorite line: >> >> "A lot of the fault tolerance aspects of Erlang come from people >> trying to >> work around Joe's buggy VM." > > This took on a whole new meaning at the SF Erlang Factory, as > Kresten Krab Thorup ("Kresten Krarup"?) demonstrated how Erlang > produced nice error reports when calling not-yet-implemented > BIFs in the Erjang VM. Erlang code is in fact tolerant of > bugs in the VM. :) > > BR, > Ulf W From g9414002.pccu.edu.tw@REDACTED Thu Apr 1 04:28:26 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Thu, 1 Apr 2010 10:28:26 +0800 Subject: [erlang-questions] List processing In-Reply-To: References: Message-ID: You can make it an array. 1> A = array:from_list(tuple_to_list({1,45,55,67,89})). {array,5,10,undefined, {1,45,55,67,89,undefined,undefined,undefined,undefined, undefined}} 2> array:get(5, A). undefined 2> array:get(4, A). 89 On Thu, Apr 1, 2010 at 2:43 AM, Saravana wrote: > Hi friends, > > Good morning,I have doubts > > Here I use the code to select the result for id from mysql db > > ConnString = "DSN=mysqldb", > {ok, Conn} = odbc:connect(ConnString, []), > {selected, _Cols, Results} = odbc:sql_query(Conn, "SELECT * FROM > marks where id=1"), > io:format("~p",Results), > i get the output in terminal {1,45,55,67,89} > > Here I wanna to process each field seperately,How to seperate each > field in the above tuple? > > I wanna print the result 1,45,55,67,89 like wise individually > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mononcqc@REDACTED Thu Apr 1 04:47:12 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 31 Mar 2010 23:47:12 -0300 Subject: [erlang-questions] The erlang baby and the green JVM In-Reply-To: <4BB3FEE0.6000902@eonblast.com> References: <466fba8d1003271202h74ae19b5gbeb49cd1c5cdf696@mail.gmail.com> <4bae685a.091abc0a.1f68.6a4dSMTPIN_ADDED@mx.google.com> <3dbc6d1c1003271345r3d87e054y5221452315121107@mail.gmail.com> <5287075483C24C4D87E0BA93CFADE093@SSI.CORP> <4BB22AAD.7020104@erlang-solutions.com> <4BB3FEE0.6000902@eonblast.com> Message-ID: It sadly got reverted pretty fast. On Wed, Mar 31, 2010 at 11:03 PM, Henning Diedrich wrote: > Judging from the comments, this is going to be around forever, like > Cedric's and Damien's rants. > > So before somebody does it wrong, I added the info to Wikipedia's Erlang > and JVM entries. At least it's official then, it was a serious omission. > > And tomorrow, maybe somebody can add a "newbs read here" tag to the blog > post, at the top, blinking and cyan. > > Henning > > > Ulf Wiger wrote: > >> David Mercer wrote: >> >>> My favorite line: >>> >>> "A lot of the fault tolerance aspects of Erlang come from people trying >>> to >>> work around Joe's buggy VM." >>> >> >> This took on a whole new meaning at the SF Erlang Factory, as >> Kresten Krab Thorup ("Kresten Krarup"?) demonstrated how Erlang >> produced nice error reports when calling not-yet-implemented >> BIFs in the Erjang VM. Erlang code is in fact tolerant of >> bugs in the VM. :) >> >> BR, >> Ulf W >> > > From hd2010@REDACTED Thu Apr 1 05:48:53 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 01 Apr 2010 05:48:53 +0200 Subject: [erlang-questions] The erlang baby and the green JVM In-Reply-To: References: <466fba8d1003271202h74ae19b5gbeb49cd1c5cdf696@mail.gmail.com> <4bae685a.091abc0a.1f68.6a4dSMTPIN_ADDED@mx.google.com> <3dbc6d1c1003271345r3d87e054y5221452315121107@mail.gmail.com> <5287075483C24C4D87E0BA93CFADE093@SSI.CORP> <4BB22AAD.7020104@erlang-solutions.com> <4BB3FEE0.6000902@eonblast.com> Message-ID: <4BB417A5.5050903@eonblast.com> Clearly, I should have kept my mouth shut. Thinking about it, I guess I don't want to be a Wikipedia staffer today. Too bad. There must be a Wiki out there that still appreciates some humor. I'll try it on C&C. Fred Hebert wrote: > It sadly got reverted pretty fast. > > On Wed, Mar 31, 2010 at 11:03 PM, Henning Diedrich > > wrote: > > Judging from the comments, this is going to be around forever, > like Cedric's and Damien's rants. > > So before somebody does it wrong, I added the info to Wikipedia's > Erlang and JVM entries. At least it's official then, it was a > serious omission. > > And tomorrow, maybe somebody can add a "newbs read here" tag to > the blog post, at the top, blinking and cyan. > > Henning > > > Ulf Wiger wrote: > > David Mercer wrote: > > My favorite line: > > "A lot of the fault tolerance aspects of Erlang come from > people trying to > work around Joe's buggy VM." > > > This took on a whole new meaning at the SF Erlang Factory, as > Kresten Krab Thorup ("Kresten Krarup"?) demonstrated how Erlang > produced nice error reports when calling not-yet-implemented > BIFs in the Erjang VM. Erlang code is in fact tolerant of > bugs in the VM. :) > > BR, > Ulf W > > > From carnildo@REDACTED Thu Apr 1 05:58:36 2010 From: carnildo@REDACTED (Mark Wagner) Date: Wed, 31 Mar 2010 20:58:36 -0700 Subject: [erlang-questions] The erlang baby and the green JVM In-Reply-To: <4BB417A5.5050903@eonblast.com> References: <466fba8d1003271202h74ae19b5gbeb49cd1c5cdf696@mail.gmail.com> <4bae685a.091abc0a.1f68.6a4dSMTPIN_ADDED@mx.google.com> <3dbc6d1c1003271345r3d87e054y5221452315121107@mail.gmail.com> <5287075483C24C4D87E0BA93CFADE093@SSI.CORP> <4BB22AAD.7020104@erlang-solutions.com> <4BB3FEE0.6000902@eonblast.com> <4BB417A5.5050903@eonblast.com> Message-ID: Wikipedia appreciates humor, just not in the articles. If you look at Wikipedia's front page, or at the community or internal-process pages, there are plenty of April Fools' jokes going around. -- Mark On Wed, Mar 31, 2010 at 20:48, Henning Diedrich wrote: > Clearly, I should have kept my mouth shut. > > Thinking about it, I guess I don't want to be a Wikipedia staffer today. > > Too bad. There must be a Wiki out there that still appreciates some humor. > I'll try it on C&C. > > Fred Hebert wrote: >> >> It sadly got reverted pretty fast. >> >> On Wed, Mar 31, 2010 at 11:03 PM, Henning Diedrich > > wrote: >> >> Judging from the comments, this is going to be around forever, >> like Cedric's and Damien's rants. >> >> So before somebody does it wrong, I added the info to Wikipedia's >> Erlang and JVM entries. At least it's official then, it was a >> serious omission. >> >> And tomorrow, maybe somebody can add a "newbs read here" tag to >> the blog post, at the top, blinking and cyan. >> >> Henning >> >> >> Ulf Wiger wrote: >> >> David Mercer wrote: >> >> My favorite line: >> >> "A lot of the fault tolerance aspects of Erlang come from >> people trying to >> work around Joe's buggy VM." >> >> >> This took on a whole new meaning at the SF Erlang Factory, as >> Kresten Krab Thorup ("Kresten Krarup"?) demonstrated how Erlang >> produced nice error reports when calling not-yet-implemented >> BIFs in the Erjang VM. Erlang code is in fact tolerant of >> bugs in the VM. :) >> >> BR, >> Ulf W >> >> >> > From ingela.andin@REDACTED Thu Apr 1 08:56:29 2010 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 1 Apr 2010 08:56:29 +0200 Subject: [erlang-questions] httpc sending empty te header In-Reply-To: <201003311410.40664.vincent.dephily@mobile-devices.fr> References: <201003311410.40664.vincent.dephily@mobile-devices.fr> Message-ID: Hi! 2010/3/31 Vincent de Phily : > Hi, > > the inets http client seems to add an empty "te:" http headers in all > requests. At first I thought it was a bug, but looking at the code of > httpc:validate_headers/3 it seems like it was done on purpose. > > So a few questions about this : > * Is it indeed the intended behaviour ? What's the purpose ? The inets http client has been quite extensively rewritten since the it was first contributed and is still evolving. However this particular feature has been there since the start I think, and the behavior is currently documented in the inets users guide. I guess it was viewed as a good default as the only currently supported transfer-encoding is chunked. >From HTTP rfc: "If the TE field-value is empty or if no TE field is present, the only transfer-coding is "chunked"" > * Shouldn't httpc provide a default value (whatever transfer encodings httpc > ?does support) instead ? The empty te-header does work as a default as it will only be added if the user has not specified a value for the te-header. Of course since other transfer-encodings currently are not supported anything but default is not so useful. > * What's the usefull difference between the "accept-encoding" and the "te" > ?headers ? The first one concerns content encoding the second one transfer encoding. Regards Ingela Erlang/OTP-team Ericsson From ingela.andin@REDACTED Thu Apr 1 09:13:18 2010 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 1 Apr 2010 09:13:18 +0200 Subject: [erlang-questions] error msg in http:request In-Reply-To: References: <4BB2F8E5.3010305@lightpole.net> Message-ID: Hi! 2010/3/31 caox : > The request URL was indeed invalid, as I referred. But I am just wondering > whether a common request failure should cause a server process to shutdown. I was a gen_server-process that was shutdown though it was not really a "http client server process" it was only a "request-handler process" spawned by the "http client server process" and it is perfectly ok that it was shutdown, maybe it should have been shutdown quitely as done in some other cases! Regards Ingela Erlang/OTP Ericssson > ? 2010-3-31???5:19? Gleb Peregud ??? > >> I don't think it is a bug. Looks like it is trying to connect to >> 127.0.0.1:3002 but TCP connection is being refused by a server. Try running >> `nmap -p 3002 127.0.0.1` and check if the port is indeed open. >> >> On Wed, Mar 31, 2010 at 09:25, caox wrote: >> After updated erlang version to R13B04, I found unexpected error messages >> prompted when using http:request to call an invalid URL. Is this reasonable? >> >> (ewp@REDACTED)1> Url="http://127.0.0.1:3002/mobilebank/srvStatus.jsp". >> "http://127.0.0.1:3002/mobilebank/srvStatus.jsp" >> (ewp@REDACTED)2> http:request(Url). >> {error,{error,{connect_failed,{#Ref<0.0.0.271>, >> {error,econnrefused}}}}} >> (ewp@REDACTED)3> >> =ERROR REPORT==== 31-Mar-2010::14:39:49 === >> ** Generic server <0.103.0> terminating >> ** Last message in was {connect_and_send, >> {request,#Ref<0.0.0.271>,<0.44.0>,0,http, >> {"127.0.0.1",3002}, >> "/mobilebank/srvStatus.jsp",[],get, >> {http_request_h,undefined,"keep-alive", >> undefined,undefined,undefined,undefined, >> undefined,undefined,undefined,undefined, >> undefined,undefined,undefined,undefined, >> undefined,undefined,"127.0.0.1:3002", >> undefined,undefined,undefined,undefined, >> undefined,undefined,undefined,undefined, >> >> undefined,[],undefined,undefined,undefined, >> undefined,"0",undefined,undefined, >> >> undefined,undefined,undefined,undefined,[]}, >> {[],[]}, >> {http_options,"HTTP/1.1",infinity,true,[], >> undefined,false,infinity}, >> "http://127.0.0.1:3002/mobilebank/srvStatus.jsp", >> >> [],none,[],1270017589276,undefined,undefined}} >> ** When Server state == {state, >> {request,#Ref<0.0.0.271>,<0.44.0>,0,http, >> {"127.0.0.1",3002}, >> "/mobilebank/srvStatus.jsp",[],get, >> {http_request_h,undefined,"keep-alive", >> undefined,undefined,undefined,undefined, >> undefined,undefined,undefined,undefined, >> undefined,undefined,undefined,undefined, >> undefined,undefined,"127.0.0.1:3002", >> undefined,undefined,undefined,undefined, >> undefined,undefined,undefined,undefined, >> undefined,[],undefined,undefined, >> undefined,undefined,"0",undefined, >> undefined,undefined,undefined,undefined, >> undefined,[]}, >> {[],[]}, >> {http_options,"HTTP/1.1",infinity,true,[], >> undefined,false,infinity}, >> "http://127.0.0.1:3002/mobilebank/srvStatus.jsp", >> >> [],none,[],1270017589276,undefined,undefined}, >> >> undefined,undefined,undefined,undefined,undefined, >> {[],[]}, >> {[],[]}, >> undefined,[],nolimit,nolimit, >> {options, >> {undefined,[]}, >> 0,2,5,120000,2,disabled,false,inet,default, >> default,[]}, >> {timers,[],undefined}, >> httpc_manager,undefined} >> ** Reason for termination == >> ** {error,{connect_failed,{#Ref<0.0.0.271>,{error,econnrefused}}}} >> >> =ERROR REPORT==== 31-Mar-2010::14:39:49 === >> HTTPC-MANAGER handler (<0.103.0>, started) failed to >> connect and/or send request #Ref<0.0.0.271> >> Result: {error,{connect_failed,{#Ref<0.0.0.271>,{error,econnrefused}}}} >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > From vincent.dephily@REDACTED Thu Apr 1 10:16:23 2010 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Thu, 1 Apr 2010 10:16:23 +0200 Subject: [erlang-questions] httpc sending empty te header In-Reply-To: References: <201003311410.40664.vincent.dephily@mobile-devices.fr> Message-ID: <201004011016.23394.vincent.dephily@mobile-devices.fr> On Thursday 01 April 2010 08:56:29 Ingela Andin wrote: > From HTTP rfc: "If the TE field-value is empty or if no TE field is > present, the only transfer-coding is "chunked"" Fair enough, that explains why the behaviour is not wrong. But in that case, why add the header at all ? You'd save yourself two lines of code, a tiny bit of CPU, 5 bytes on the wire, and some confused users. > > * What's the usefull difference between the "accept-encoding" and the > > "te" headers ? > > The first one concerns content encoding the second one transfer encoding. Thanks for these explanations, and sorry for not taking the time to check the RFC myself. -- Vincent de Phily Mobile Devices +33 (0) 142 119 325 +353 (0) 85 710 6320 Warning This message (and any associated files) is intended only for the use of its intended recipient and may contain information that is confidential, subject to copyright or constitutes a trade secret. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this message, or files associated with this message, is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. Any views or opinions presented are solely those of the author vincent.dephily@REDACTED and do not necessarily represent those of the company. Although the company has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. From B.Candler@REDACTED Thu Apr 1 11:45:20 2010 From: B.Candler@REDACTED (Brian Candler) Date: Thu, 1 Apr 2010 10:45:20 +0100 Subject: List processing In-Reply-To: References: Message-ID: <20100401094520.GA10812@uk.tiscali.com> On Wed, Mar 31, 2010 at 11:43:50AM -0700, Saravana wrote: > Here I wanna to process each field seperately,How to seperate each > field in the above tuple? > > I wanna print the result 1,45,55,67,89 like wise individually Here's another way, output taken from 'man erlang': element(N, Tuple) -> term() Types N = 1..tuple_size(Tuple) Tuple = tuple() Returns the Nth element (numbering from 1) of Tuple. > element(2, {a, b, c}). b Allowed in guard tests. From mononcqc@REDACTED Thu Apr 1 16:02:27 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 1 Apr 2010 10:02:27 -0400 Subject: [erlang-questions] The erlang baby and the green JVM In-Reply-To: <4BB417A5.5050903@eonblast.com> References: <466fba8d1003271202h74ae19b5gbeb49cd1c5cdf696@mail.gmail.com> <4bae685a.091abc0a.1f68.6a4dSMTPIN_ADDED@mx.google.com> <3dbc6d1c1003271345r3d87e054y5221452315121107@mail.gmail.com> <5287075483C24C4D87E0BA93CFADE093@SSI.CORP> <4BB22AAD.7020104@erlang-solutions.com> <4BB3FEE0.6000902@eonblast.com> <4BB417A5.5050903@eonblast.com> Message-ID: I'm glad to announce that following these revelations, Learn You Some Erlang (http://learnyousomeerlang.com) will be renamed Learn You Some Scala. See the main page and the FAQ for details. This is also covered there: http://functional-orbitz.blogspot.com/2010/03/learn-you-some-erlang-renamed-learn-you.html From hd2010@REDACTED Thu Apr 1 17:38:42 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 01 Apr 2010 17:38:42 +0200 Subject: [erlang-questions] The erlang baby and the green JVM In-Reply-To: References: <466fba8d1003271202h74ae19b5gbeb49cd1c5cdf696@mail.gmail.com> <4bae685a.091abc0a.1f68.6a4dSMTPIN_ADDED@mx.google.com> <3dbc6d1c1003271345r3d87e054y5221452315121107@mail.gmail.com> <5287075483C24C4D87E0BA93CFADE093@SSI.CORP> <4BB22AAD.7020104@erlang-solutions.com> <4BB3FEE0.6000902@eonblast.com> <4BB417A5.5050903@eonblast.com> Message-ID: <4BB4BE02.6090307@eonblast.com> Somehow postings in this particular tone go down so much nicer when it's complete BS. I think I actually prefer it that way. Can we have more of that? I could put up a Scarlang wiki. Can somebody enlighten us what triggered the noble impulse to hand the mantle of worthiness over to Scala? Fred Hebert wrote: > I'm glad to announce that following these revelations, Learn You Some Erlang > (http://learnyousomeerlang.com) will be renamed Learn You Some Scala. > > See the main page and the FAQ for details. > > This is also covered there: > http://functional-orbitz.blogspot.com/2010/03/learn-you-some-erlang-renamed-learn-you.html > > From mononcqc@REDACTED Thu Apr 1 18:39:12 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 1 Apr 2010 12:39:12 -0400 Subject: [erlang-questions] The erlang baby and the green JVM In-Reply-To: <4BB4BE02.6090307@eonblast.com> References: <466fba8d1003271202h74ae19b5gbeb49cd1c5cdf696@mail.gmail.com> <4bae685a.091abc0a.1f68.6a4dSMTPIN_ADDED@mx.google.com> <3dbc6d1c1003271345r3d87e054y5221452315121107@mail.gmail.com> <5287075483C24C4D87E0BA93CFADE093@SSI.CORP> <4BB22AAD.7020104@erlang-solutions.com> <4BB3FEE0.6000902@eonblast.com> <4BB417A5.5050903@eonblast.com> <4BB4BE02.6090307@eonblast.com> Message-ID: It all started when Francesco Cesarini reposted a question he overhead at QCon earlier in March (http://twitter.com/FrancescoC/status/10271901570). I mentioned that in #erlang on freenode and we all had a good laugh. Sarcastic comments about "oh of course Erlang was first on the JVM" started there. Orbitz (an #erlang user who likes the anonymity given by his nickname) and I thought it would be funny to actually write a sarcastic blog post for an alternative Erlang history to see how people would react. We were always a bit irked by people claiming Scala actors are a good replacement for Erlang and we decided to push it. So we went writing and finding all kinds of little things that could annoy people who love to overreact to small details ("Richard Virding") and then just huge lies for the rest of people. The reaction was pretty good, so we decided we'd make a come back with the idea for April's 1st. Orbitz and I took our respective sites (LYSE and his blog) and pushed a bit further. I'd say that so far, it's totally been worth it. Good reactions around the place. It's not that we dislike scala at all, but the QCon question was just asked at the right time in the right context. Erjang being developed also let confusion get a little more place too -- you could google for Erlang on the JVM and get results or something. It's a fun meme so far. No idea if the scala guys like it though. On Thu, Apr 1, 2010 at 11:38 AM, Henning Diedrich wrote: > Somehow postings in this particular tone go down so much nicer when it's > complete BS. I think I actually prefer it that way. > > Can we have more of that? I could put up a Scarlang wiki. > > Can somebody enlighten us what triggered the noble impulse to hand the > mantle of worthiness over to Scala? > > > Fred Hebert wrote: > >> I'm glad to announce that following these revelations, Learn You Some >> Erlang >> (http://learnyousomeerlang.com) will be renamed Learn You Some Scala. >> >> See the main page and the FAQ for details. >> >> This is also covered there: >> >> http://functional-orbitz.blogspot.com/2010/03/learn-you-some-erlang-renamed-learn-you.html >> >> >> > > From g@REDACTED Thu Apr 1 19:31:51 2010 From: g@REDACTED (Garrett Smith) Date: Thu, 1 Apr 2010 11:31:51 -0600 Subject: [erlang-questions] The erlang baby and the green JVM In-Reply-To: References: <466fba8d1003271202h74ae19b5gbeb49cd1c5cdf696@mail.gmail.com> <3dbc6d1c1003271345r3d87e054y5221452315121107@mail.gmail.com> <5287075483C24C4D87E0BA93CFADE093@SSI.CORP> <4BB22AAD.7020104@erlang-solutions.com> <4BB3FEE0.6000902@eonblast.com> <4BB417A5.5050903@eonblast.com> <4BB4BE02.6090307@eonblast.com> Message-ID: I'm tempted to think this resolves the thread. But it is April 1. I mean, as far as sense of humor, there's "dry", and then there's "straight gin martini, mouth puckering bone dry!" (But now I'm comfortable forwarding the Orbitz link to my Java/Scala chums.) Evil, evil. On Thu, Apr 1, 2010 at 10:39 AM, Fred Hebert wrote: > It all started when Francesco Cesarini reposted a question he overhead at > QCon earlier in March (http://twitter.com/FrancescoC/status/10271901570). > > I mentioned that in #erlang on freenode and we all had a good laugh. > Sarcastic comments about "oh of course Erlang was first on the JVM" started > there. Orbitz (an #erlang user who likes the anonymity given by his > nickname) and I thought it would be funny to actually write a sarcastic blog > post for an alternative Erlang history to see how people would react. We > were always a bit irked by people claiming Scala actors are a good > replacement for Erlang and we decided to push it. > > ?So we went writing and finding all kinds of little things that could annoy > people who love to overreact to small details ("Richard Virding") and then > just huge lies for the rest of people. > > The reaction was pretty good, so we decided we'd make a come back with the > idea for April's 1st. > > Orbitz and I took our respective sites (LYSE and his blog) and pushed a bit > further. I'd say that so far, it's totally been worth it. Good reactions > around the place. > > It's not that we dislike scala at all, but the QCon question was just asked > at the right time in the right context. Erjang being developed also let > confusion get a little more place too -- you could google for Erlang on the > JVM and get results or something. > > It's a fun meme so far. No idea if the scala guys like it though. > > On Thu, Apr 1, 2010 at 11:38 AM, Henning Diedrich wrote: > >> Somehow postings in this particular tone ?go down so much nicer when it's >> complete BS. I think I actually prefer it that way. >> >> Can we have more of that? I could put up a Scarlang wiki. >> >> Can somebody enlighten us what triggered the noble impulse to hand the >> mantle of worthiness over to Scala? >> >> >> Fred Hebert wrote: >> >>> I'm glad to announce that following these revelations, Learn You Some >>> Erlang >>> (http://learnyousomeerlang.com) will be renamed Learn You Some Scala. >>> >>> See the main page and the FAQ for details. >>> >>> This is also covered there: >>> >>> http://functional-orbitz.blogspot.com/2010/03/learn-you-some-erlang-renamed-learn-you.html >>> >>> >>> >> >> > From max.lapshin@REDACTED Thu Apr 1 20:46:59 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 1 Apr 2010 22:46:59 +0400 Subject: Looking for help with crypto:dh_generate_key/1 Message-ID: I can't understand, what do I need to pass to dh_generate_key for generating public and private keys. Documentation tells: dh_generate_key(DHParams) DHParameters = [P, G] P, G = Mpint Where P is the shared prime number and G is the shared generator. I'm trying crypto:dh_generate_key([2, 137]). ** exception error: bad argument in function port_control/3 called as port_control(crypto_drv02,45,[<<0,0,0,0>>,[2,137]]) in call from crypto:control/2 in call from crypto:dh_generate_key/2 So, what should I pass? Just an example. Google doesn't tell anything interesting. From steven.charles.davis@REDACTED Thu Apr 1 22:35:48 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 1 Apr 2010 13:35:48 -0700 (PDT) Subject: Implementation of RFC-5841 for gen_tcp? Message-ID: <91c66845-71eb-4e58-8200-5e9b277866ef@b33g2000yqc.googlegroups.com> I'm wondering when the modifications to TCP, proposed by Google, will be made in the gen_tcp module? See: http://tools.ietf.org/html/rfc5841 /s ps happy 1st From kyleqian@REDACTED Fri Apr 2 00:29:22 2010 From: kyleqian@REDACTED (=?GB2312?B?x67P/sP3?=) Date: Fri, 2 Apr 2010 06:29:22 +0800 Subject: How to write a handle_call for gen_server callback with checking user? Message-ID: Just a simple question! My handle_call just like this: handle_call({start, User, Passwd, Algorithm, ScheduledTime, XmlParams}, _From, State) -> Reply = case check_user(User, Passwd) of false -> {error, rejected}; _ -> ?INFOLOG("Received request from user ~p for ~p.~n", [User,Algorithm]) % other code for processing user request... % other code for processing user request... end, {reply, Reply, State}. Can I return {reply, {error, rejected}, State} immediately when I find that check_user return false? How can I write such code? I was a C++ programmer before, so expecting something like "return" keyword... Thanks! All of you give me great favor! From acton@REDACTED Fri Apr 2 01:11:05 2010 From: acton@REDACTED (Brian Acton) Date: Thu, 1 Apr 2010 16:11:05 -0700 Subject: Mnesia deadlock with large volume of dirty operations? Message-ID: Hi guys, I am running R13B04 SMP on FreeBSD 7.3. I have a cluster of 7 nodes running mnesia. I have a table of 1196143 records using about 1.504GB of storage. It's a reasonably hot table doing a fair number of insert operations at any given time. I decided that since there was a 2GB limit in mnesia that I should do some cleanup on the system and specifically this table. Trying to avoid major problems with Mnesia, transaction load, and deadlock, I decided to do dirty_select and dirty_delete_object individually on the records. I started slow, deleting first 10, then 100, then 1000, then 10000, then 100,000 records. My goal was to delete 192593 records total. The first five deletions went through nicely and caused minimal to no impact. Unfortunately, the very last delete blew up the system. My delete command completed successfully but on the other nodes, it caused mnesia to get stuck on pending transactions, caused my message queues to fill up and basically brought down the whole system. We saw some mnesia is overloaded messages in our logs on these nodes but did not see a ton of them. Does anyone have any clues on what went wrong? I am attaching my code below for your review. --b Mnesia configuration tunables: -mnesia no_table_loaders 20 -mnesia dc_dump_limit 40 -mnesia dump_log_write_threshold 10000 Example error message: ** WARNING ** Mnesia is overloaded: {mnesia_tm, message_queue_len, [387,842]} Sample code: Select = fun(Days) -> {MegaSecs, Secs, _MicroSecs} = now(), T = MegaSecs * 1000000 + Secs - 86400 * Days, TimeStamp = {T div 1000000, T rem 1000000, 0}, mnesia:dirty_select(offline_msg, [{'$1', [{'<', {element, 3, '$1'}, {TimeStamp} }], ['$1']}]) end. Count = fun(Days) -> length(Select(Days)) end. Delete = fun(Days, Total) -> C = Select(Days), D = lists:sublist(C, Total), lists:foreach(fun(Rec) -> ok = mnesia:dirty_delete_object(Rec) end, D), length(D) end. From acton@REDACTED Fri Apr 2 02:12:32 2010 From: acton@REDACTED (Brian Acton) Date: Thu, 1 Apr 2010 17:12:32 -0700 Subject: how to clear a hot mnesia table Message-ID: Hi guys, I am running R13B04 SMP on FreeBSD 7.3 I have a very large table (i.e. 5.98M records covering 1.66GB) in Mnesia that I need to clear out. Unfortunately, the transaction volume is reasonably high and the servers are 24/7 servers. I'd like to clear out the table without experiencing any downtime. I know about mnesia:clear_table. However, I am concerned that since this requires transactional semantics that mnesia will succeed on one node but basically bring down the rest of the nodes as transactions pile up waiting on some form of table lock. Is there a clean and simple way to empty out the table in a less risky fashion ? --b From hd2010@REDACTED Fri Apr 2 02:20:21 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Fri, 02 Apr 2010 02:20:21 +0200 Subject: [erlang-questions] The erlang baby and the green JVM In-Reply-To: References: <466fba8d1003271202h74ae19b5gbeb49cd1c5cdf696@mail.gmail.com> <3dbc6d1c1003271345r3d87e054y5221452315121107@mail.gmail.com> <5287075483C24C4D87E0BA93CFADE093@SSI.CORP> <4BB22AAD.7020104@erlang-solutions.com> <4BB3FEE0.6000902@eonblast.com> <4BB417A5.5050903@eonblast.com> <4BB4BE02.6090307@eonblast.com> Message-ID: <4BB53845.3010209@eonblast.com> Do you think we can we continue that tommorrow anyway? From rvirding@REDACTED Fri Apr 2 02:44:19 2010 From: rvirding@REDACTED (Robert Virding) Date: Fri, 2 Apr 2010 02:44:19 +0200 Subject: [erlang-questions] The erlang baby and the green JVM In-Reply-To: References: <466fba8d1003271202h74ae19b5gbeb49cd1c5cdf696@mail.gmail.com> <5287075483C24C4D87E0BA93CFADE093@SSI.CORP> <4BB22AAD.7020104@erlang-solutions.com> <4BB3FEE0.6000902@eonblast.com> <4BB417A5.5050903@eonblast.com> <4BB4BE02.6090307@eonblast.com> Message-ID: Once I got past "what the hell have they been smoking stage", which went quickly when I saw who had taken part in it, I enjoyed it. My only problem with it is that the original question was probably serious and there are certain to be some, shall we say, less knowledgeable people who believe it. "Against stupidity the gods themselves contend in vain". But stick them, more of the same. "Richard" On 1 April 2010 19:31, Garrett Smith wrote: > I'm tempted to think this resolves the thread. But it is April 1. > > I mean, as far as sense of humor, there's "dry", and then there's > "straight gin martini, mouth puckering bone dry!" > > (But now I'm comfortable forwarding the Orbitz link to my Java/Scala chums.) > > Evil, evil. > > On Thu, Apr 1, 2010 at 10:39 AM, Fred Hebert wrote: >> It all started when Francesco Cesarini reposted a question he overhead at >> QCon earlier in March (http://twitter.com/FrancescoC/status/10271901570). >> >> I mentioned that in #erlang on freenode and we all had a good laugh. >> Sarcastic comments about "oh of course Erlang was first on the JVM" started >> there. Orbitz (an #erlang user who likes the anonymity given by his >> nickname) and I thought it would be funny to actually write a sarcastic blog >> post for an alternative Erlang history to see how people would react. We >> were always a bit irked by people claiming Scala actors are a good >> replacement for Erlang and we decided to push it. >> >> ?So we went writing and finding all kinds of little things that could annoy >> people who love to overreact to small details ("Richard Virding") and then >> just huge lies for the rest of people. >> >> The reaction was pretty good, so we decided we'd make a come back with the >> idea for April's 1st. >> >> Orbitz and I took our respective sites (LYSE and his blog) and pushed a bit >> further. I'd say that so far, it's totally been worth it. Good reactions >> around the place. >> >> It's not that we dislike scala at all, but the QCon question was just asked >> at the right time in the right context. Erjang being developed also let >> confusion get a little more place too -- you could google for Erlang on the >> JVM and get results or something. >> >> It's a fun meme so far. No idea if the scala guys like it though. >> >> On Thu, Apr 1, 2010 at 11:38 AM, Henning Diedrich wrote: >> >>> Somehow postings in this particular tone ?go down so much nicer when it's >>> complete BS. I think I actually prefer it that way. >>> >>> Can we have more of that? I could put up a Scarlang wiki. >>> >>> Can somebody enlighten us what triggered the noble impulse to hand the >>> mantle of worthiness over to Scala? >>> >>> >>> Fred Hebert wrote: >>> >>>> I'm glad to announce that following these revelations, Learn You Some >>>> Erlang >>>> (http://learnyousomeerlang.com) will be renamed Learn You Some Scala. >>>> >>>> See the main page and the FAQ for details. >>>> >>>> This is also covered there: >>>> >>>> http://functional-orbitz.blogspot.com/2010/03/learn-you-some-erlang-renamed-learn-you.html >>>> >>>> >>>> >>> >>> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From dgud@REDACTED Fri Apr 2 09:05:31 2010 From: dgud@REDACTED (Dan Gudmundsson) Date: Fri, 2 Apr 2010 09:05:31 +0200 Subject: [erlang-questions] Looking for help with crypto:dh_generate_key/1 In-Reply-To: References: Message-ID: Use crypto:dh_generate_parameters(512,2) /Dan On Thu, Apr 1, 2010 at 8:46 PM, Max Lapshin wrote: > I can't understand, what do I need to pass to dh_generate_key for > generating public and private keys. > > Documentation tells: dh_generate_key(DHParams) > > DHParameters = [P, G] > P, G = Mpint > Where P is the shared prime number and G is the shared generator. > > I'm trying > > crypto:dh_generate_key([2, 137]). > > ** exception error: bad argument > ? ? in function ?port_control/3 > ? ? ? ?called as port_control(crypto_drv02,45,[<<0,0,0,0>>,[2,137]]) > ? ? in call from crypto:control/2 > ? ? in call from crypto:dh_generate_key/2 > > So, what should I pass? Just an example. Google doesn't tell anything > interesting. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From max.lapshin@REDACTED Fri Apr 2 09:05:52 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 2 Apr 2010 11:05:52 +0400 Subject: [erlang-questions] Looking for help with crypto:dh_generate_key/1 In-Reply-To: References: Message-ID: On Fri, Apr 2, 2010 at 11:05 AM, Dan Gudmundsson wrote: > Use crypto:dh_generate_parameters(512,2) > Oh, thank you a lot! From koops.j@REDACTED Fri Apr 2 09:08:39 2010 From: koops.j@REDACTED (Jeroen Koops) Date: Fri, 2 Apr 2010 09:08:39 +0200 Subject: [erlang-questions] How to write a handle_call for gen_server callback with checking user? In-Reply-To: References: Message-ID: You could change it into: handle_call({start, User, Passwd, Algorithm, ScheduledTime, XmlParams}, _From, State) -> case check_user(User, Passwd) of false -> { reply, {error, rejected}, State }; _ -> ?INFOLOG("Received request from user ~p for ~p.~n",[User,Algorithm]) % other code for processing user request... % other code for processing user request... { reply, { ok, SomeResponse }, State } end end. ... if that's what you mean. Regards, Jeroen On Fri, Apr 2, 2010 at 12:29 AM, ??? wrote: > Just a simple question! My handle_call just like this: > > handle_call({start, User, Passwd, Algorithm, ScheduledTime, XmlParams}, > _From, > State) -> > Reply = case check_user(User, Passwd) of > false -> > {error, rejected}; > _ -> > ?INFOLOG("Received request from user ~p for ~p.~n", > [User,Algorithm]) > % other code for processing user request... > % other code for processing user request... > end, > {reply, Reply, State}. > > Can I return {reply, {error, rejected}, State} immediately when I find that > check_user return false? How can I write such code? > I was a C++ programmer before, so expecting something like "return" > keyword... > Thanks! All of you give me great favor! > From dgud@REDACTED Fri Apr 2 09:19:10 2010 From: dgud@REDACTED (Dan Gudmundsson) Date: Fri, 2 Apr 2010 09:19:10 +0200 Subject: [erlang-questions] Mnesia deadlock with large volume of dirty operations? In-Reply-To: References: Message-ID: When you are using dirty, every operation is sent separately to all nodes, i.e. 192593*6 messages, actually a transaction could have been faster in this case. With one message (large) containing all ops to each node. What you get is an overloaded mnesia_tm (very long msg queues), which do the actual writing of the data on the other (participating mnesia nodes). So transactions will be blocked waiting on mnesia_tm to process those 200000 messages on the other nodes. /Dan On Fri, Apr 2, 2010 at 1:11 AM, Brian Acton wrote: > Hi guys, > > I am running R13B04 SMP on FreeBSD 7.3. I have a cluster of 7 nodes running > mnesia. > > I have a table of 1196143 records using about 1.504GB of storage. It's a > reasonably hot table doing a fair number of insert operations at any given > time. > > I decided that since there was a 2GB limit in mnesia that I should do some > cleanup on the system and specifically this table. > > Trying to avoid major problems with Mnesia, transaction load, and deadlock, > I decided to do dirty_select and dirty_delete_object individually on the > records. > > I started slow, deleting first 10, then 100, then 1000, then 10000, then > 100,000 records. My goal was to delete 192593 records total. > > The first five deletions went through nicely and caused minimal to no > impact. > > Unfortunately, the very last delete blew up the system. My delete command > completed successfully but on the other nodes, it caused mnesia to get stuck > on pending transactions, caused my message queues to fill up and basically > brought down the whole system. We saw some mnesia is overloaded messages in > our logs on these nodes but did not see a ton of them. > > Does anyone have any clues on what went wrong? I am attaching my code below > for your review. > > --b > > Mnesia configuration tunables: > > ? ? ?-mnesia no_table_loaders 20 > ? ? ?-mnesia dc_dump_limit 40 > ? ? ?-mnesia dump_log_write_threshold 10000 > > Example error message: > > ** WARNING ** Mnesia is overloaded: {mnesia_tm, message_queue_len, > [387,842]} > > Sample code: > > Select = fun(Days) -> > ? ? ? ? {MegaSecs, Secs, _MicroSecs} = now(), > ? ? ? ? T = MegaSecs * 1000000 + Secs - 86400 * Days, > ? ? ? ? TimeStamp = {T div 1000000, T rem 1000000, 0}, > ? ? ? ? mnesia:dirty_select(offline_msg, > ? ? ? ? ? ? ? ? ? ? [{'$1', > ? ? ? ? ? ? ? ? ? ? ? [{'<', {element, 3, '$1'}, > ? ? ? ? ? ? ? ? ? ? {TimeStamp} }], > ? ? ? ? ? ? ? ? ? ? ? ['$1']}]) > ? ? end. > > Count = fun(Days) -> length(Select(Days)) end. > > Delete = fun(Days, Total) -> > ? ? ? ? C = Select(Days), > ? ? ? ? D = lists:sublist(C, Total), > ? ? ? ? lists:foreach(fun(Rec) -> > ? ? ? ? ? ? ? ? ? ? ? ok = mnesia:dirty_delete_object(Rec) > ? ? ? ? ? ? ? ? ? end, > ? ? ? ? ? ? ? ? ? D), > ? ? ? ? length(D) > ? ? end. > From dgud@REDACTED Fri Apr 2 09:22:04 2010 From: dgud@REDACTED (Dan Gudmundsson) Date: Fri, 2 Apr 2010 09:22:04 +0200 Subject: [erlang-questions] Looking for help with crypto:dh_generate_key/1 In-Reply-To: References: Message-ID: You can take a peek at the test code at github for example usage, which is what I did :-) /Dan On Fri, Apr 2, 2010 at 9:05 AM, Max Lapshin wrote: > On Fri, Apr 2, 2010 at 11:05 AM, Dan Gudmundsson wrote: >> Use crypto:dh_generate_parameters(512,2) >> > > Oh, thank you a lot! > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From max.lapshin@REDACTED Fri Apr 2 09:23:36 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 2 Apr 2010 11:23:36 +0400 Subject: [erlang-questions] Looking for help with crypto:dh_generate_key/1 In-Reply-To: References: Message-ID: On Fri, Apr 2, 2010 at 11:22 AM, Dan Gudmundsson wrote: > You can take a peek at the test code at github for example usage, > which is what I did :-) > I've looked in OTP sources, and seems that misses test code. http://github.com/search?q=dh_generate_key&type=Everything&repo=&langOverride=&start_value=1 this search also haven't help me. Where have you seen test code? From dgud@REDACTED Fri Apr 2 09:29:06 2010 From: dgud@REDACTED (Dan Gudmundsson) Date: Fri, 2 Apr 2010 09:29:06 +0200 Subject: [erlang-questions] Looking for help with crypto:dh_generate_key/1 In-Reply-To: References: Message-ID: Line 910 - 940 in http://github.com/erlang/otp/blob/dev/lib/crypto/test/crypto_SUITE.erl But I knew where to look. /Dan On Fri, Apr 2, 2010 at 9:23 AM, Max Lapshin wrote: > On Fri, Apr 2, 2010 at 11:22 AM, Dan Gudmundsson wrote: >> You can take a peek at the test code at github for example usage, >> which is what I did :-) >> > > I've looked in OTP sources, and seems that misses test code. > http://github.com/search?q=dh_generate_key&type=Everything&repo=&langOverride=&start_value=1 > this search also haven't help me. Where have you seen test code? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From max.lapshin@REDACTED Fri Apr 2 09:32:43 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 2 Apr 2010 11:32:43 +0400 Subject: [erlang-questions] Looking for help with crypto:dh_generate_key/1 In-Reply-To: References: Message-ID: On Fri, Apr 2, 2010 at 11:29 AM, Dan Gudmundsson wrote: > Line 910 - 940 in > http://github.com/erlang/otp/blob/dev/lib/crypto/test/crypto_SUITE.erl > > But I knew where to look. Great, now I also know =) From ovidiudeac@REDACTED Fri Apr 2 14:47:04 2010 From: ovidiudeac@REDACTED (Ovidiu Deac) Date: Fri, 2 Apr 2010 15:47:04 +0300 Subject: [erlang-questions] Mnesia deadlock with large volume of dirty operations? In-Reply-To: References: Message-ID: To me it sounds like another example of premature optimization which went wrong? :) On Fri, Apr 2, 2010 at 10:19 AM, Dan Gudmundsson wrote: > When you are using dirty, every operation is sent separately to all nodes, > i.e. 192593*6 messages, actually a transaction could have been faster > in this case. > With one message (large) containing all ops to each node. > > What you get is an overloaded mnesia_tm (very long msg queues), > which do the actual writing of the data on the other (participating > mnesia nodes). > > So transactions will be blocked waiting on mnesia_tm to process those 200000 > messages on the other nodes. > > /Dan > > On Fri, Apr 2, 2010 at 1:11 AM, Brian Acton wrote: >> Hi guys, >> >> I am running R13B04 SMP on FreeBSD 7.3. I have a cluster of 7 nodes running >> mnesia. >> >> I have a table of 1196143 records using about 1.504GB of storage. It's a >> reasonably hot table doing a fair number of insert operations at any given >> time. >> >> I decided that since there was a 2GB limit in mnesia that I should do some >> cleanup on the system and specifically this table. >> >> Trying to avoid major problems with Mnesia, transaction load, and deadlock, >> I decided to do dirty_select and dirty_delete_object individually on the >> records. >> >> I started slow, deleting first 10, then 100, then 1000, then 10000, then >> 100,000 records. My goal was to delete 192593 records total. >> >> The first five deletions went through nicely and caused minimal to no >> impact. >> >> Unfortunately, the very last delete blew up the system. My delete command >> completed successfully but on the other nodes, it caused mnesia to get stuck >> on pending transactions, caused my message queues to fill up and basically >> brought down the whole system. We saw some mnesia is overloaded messages in >> our logs on these nodes but did not see a ton of them. >> >> Does anyone have any clues on what went wrong? I am attaching my code below >> for your review. >> >> --b >> >> Mnesia configuration tunables: >> >> ? ? ?-mnesia no_table_loaders 20 >> ? ? ?-mnesia dc_dump_limit 40 >> ? ? ?-mnesia dump_log_write_threshold 10000 >> >> Example error message: >> >> ** WARNING ** Mnesia is overloaded: {mnesia_tm, message_queue_len, >> [387,842]} >> >> Sample code: >> >> Select = fun(Days) -> >> ? ? ? ? {MegaSecs, Secs, _MicroSecs} = now(), >> ? ? ? ? T = MegaSecs * 1000000 + Secs - 86400 * Days, >> ? ? ? ? TimeStamp = {T div 1000000, T rem 1000000, 0}, >> ? ? ? ? mnesia:dirty_select(offline_msg, >> ? ? ? ? ? ? ? ? ? ? [{'$1', >> ? ? ? ? ? ? ? ? ? ? ? [{'<', {element, 3, '$1'}, >> ? ? ? ? ? ? ? ? ? ? {TimeStamp} }], >> ? ? ? ? ? ? ? ? ? ? ? ['$1']}]) >> ? ? end. >> >> Count = fun(Days) -> length(Select(Days)) end. >> >> Delete = fun(Days, Total) -> >> ? ? ? ? C = Select(Days), >> ? ? ? ? D = lists:sublist(C, Total), >> ? ? ? ? lists:foreach(fun(Rec) -> >> ? ? ? ? ? ? ? ? ? ? ? ok = mnesia:dirty_delete_object(Rec) >> ? ? ? ? ? ? ? ? ? end, >> ? ? ? ? ? ? ? ? ? D), >> ? ? ? ? length(D) >> ? ? end. >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From golubovsky@REDACTED Fri Apr 2 15:15:17 2010 From: golubovsky@REDACTED (Dimitry Golubovsky) Date: Fri, 2 Apr 2010 09:15:17 -0400 Subject: ANN: wxi, a library to build composable GUI on top of wxErlang Message-ID: Hi, I have been working on this library for some time; now that it is getting into shape I'd like to announce it to the Erlang community. The wxi library [1] is based on the same ideas as found in the Haskell GUI library Fudgets [2]: each element of graphics user interface (a widget) also acts as a "stream processor" for streams of events (messages) generated from user input and I/O operations. Processed messages are passed around from widget to widget. Thus, desired user interface layout and dynamics can be achieved via functional composition of such GUI elements. Each widget is defined as a function of certain structure, widgets can be nested, combined in parallel and in sequence (in terms of message handling). There is no central event loop: each widget is responsible for capturing and handling its own events. There is no central GUI state: widgets that encapsulate state are implemented as processes; stateless widgets are just functions tail-calling other widget functions to pass messages around. The wxi library is in early development state: while the core functionality (composition of widgets) is more or less working, the library of general purpose widgets is not yet completed, but this is only a matter of time and users contributions. For now, the library contains only one module [3] containing widgets composition functions and few generic widgets (panel, button, text display). there is a brief tutorial [4], and an "obligatory" example of calculator [5] with detailed comments in code. To give the library a try, download a repo snapshot [6], or clone the repo: (see http://code.google.com/p/wxi/source/checkout). I will be happy to hear from the developers of Erlang-based projects that involve wx-based GUI whether using such a library would be more convenient and productive in comparison with "traditional" way (e. g. the wx-based GUI to dialyzer which is based on a central event loop). Thanks in advance for any ideas and suggestions. ===================================== 1. http://code.google.com/p/wxi/ 2. http://www.md.chalmers.se/Cs/Research/Functional/Fudgets/ 3. http://wxi.googlecode.com/hg/src/wxi.erl 4. http://code.google.com/p/wxi/wiki/AnatomyOfWidget 5. http://wxi.googlecode.com/hg/examples/wxicalc.erl 6. http://wxi.googlecode.com/files/wxicalc-2010-04-01.tar.gz -- Dimitry Golubovsky Anywhere on the Web From williamvdoorn@REDACTED Fri Apr 2 17:48:22 2010 From: williamvdoorn@REDACTED (William v Doorn) Date: Fri, 2 Apr 2010 17:48:22 +0200 Subject: [erlang-questions] ANN: wxi, a library to build composable GUI on top of wxErlang In-Reply-To: References: Message-ID: Hello, This looks like a very interesting approach after I gave it a quick look, since I've never used this I might start using this for my first GUI's in Erlang. Good job William van Doorn 2010/4/2 Dimitry Golubovsky > Hi, > > I have been working on this library for some time; now that it is > getting into shape I'd like to announce it to the Erlang community. > > The wxi library [1] is based on the same ideas as found in the Haskell > GUI library Fudgets [2]: each element of graphics user interface (a > widget) also acts as a "stream processor" for streams of events > (messages) generated from user input and I/O operations. Processed > messages are passed around from widget to widget. Thus, desired user > interface layout and dynamics can be achieved via functional > composition of such GUI elements. > > Each widget is defined as a function of certain structure, widgets can > be nested, combined in parallel and in sequence (in terms of message > handling). There is no central event loop: each widget is responsible > for capturing and handling its own events. There is no central GUI > state: widgets that encapsulate state are implemented as processes; > stateless widgets are just functions tail-calling other widget > functions to pass messages around. > > The wxi library is in early development state: while the core > functionality (composition of widgets) is more or less working, the > library of general purpose widgets is not yet completed, but this is > only a matter of time and users contributions. > > For now, the library contains only one module [3] containing widgets > composition functions and few generic widgets (panel, button, text > display). there is a brief tutorial [4], and an "obligatory" example > of calculator [5] with detailed comments in code. > > To give the library a try, download a repo snapshot [6], or clone the > repo: (see http://code.google.com/p/wxi/source/checkout). > > I will be happy to hear from the developers of Erlang-based projects > that involve wx-based GUI whether using such a library would be more > convenient and productive in comparison with "traditional" way (e. g. > the wx-based GUI to dialyzer which is based on a central event loop). > > Thanks in advance for any ideas and suggestions. > > ===================================== > 1. http://code.google.com/p/wxi/ > 2. http://www.md.chalmers.se/Cs/Research/Functional/Fudgets/ > 3. http://wxi.googlecode.com/hg/src/wxi.erl > 4. http://code.google.com/p/wxi/wiki/AnatomyOfWidget > 5. http://wxi.googlecode.com/hg/examples/wxicalc.erl > 6. http://wxi.googlecode.com/files/wxicalc-2010-04-01.tar.gz > > -- > Dimitry Golubovsky > > Anywhere on the Web > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ivan@REDACTED Fri Apr 2 18:08:11 2010 From: ivan@REDACTED (Ivan Uemlianin) Date: Fri, 02 Apr 2010 17:08:11 +0100 Subject: [erlang-questions] Generating sine waves and/or audio stream output In-Reply-To: <407d9ef81003251204u36a716b7tab2e66a686f19ea8@mail.gmail.com> References: <4BA9E946.5000009@llaisdy.com> <407d9ef81003241134v6d119a7cl4538c38ef90f24ec@mail.gmail.com> <4BAB2403.10306@llaisdy.com> <407d9ef81003251204u36a716b7tab2e66a686f19ea8@mail.gmail.com> Message-ID: <4BB6166B.20009@llaisdy.com> Dear Willem This hacky little script (my first in erlang!) will write a sound and pipe it to audio. play(Sound) -> Fn = sinus, file:write_file(io_lib:format("~p.raw", [Fn]), Sound), os:cmd(io_lib:format("sox -r ~B -c 1 -w -s ~p.raw ~p.wav", [?SamplingRate, Fn, Fn])), os:cmd(io_lib:format("afplay ~p.wav", [Fn])). sox is a unix tool, here it converts the raw audio file to wav format file. On a proper unix platform sox can output to audio. On MacOSX you have to write to a file and use afplay to play the file. This is not a very nice function. I shall write a function to convert a raw sound into a wav format binary. It would be v nice to habe something like an erlang binding for PortAudio. In the meantime, the function above makes it possible to listen to generated sounds fmor the erlang shell. With thanks and apologies Ivan On 25/03/2010 19:04, Willem de Jong wrote: > Hmm, I had forgotten about some of the complexities involved. > > I do not have a function that writes wav files. I put the pcm data > that is generated by this into a flash file. For wav files you have to > do add a header and framing, I believe. > > I added a function that puts together the parts of the sound. You can > write it to a file like this: > > file:write_file("sinus.raw", sound:sound(440, 3, 0.5, sinus)). > > (440 hertz, 3 seconds, volume 0.5 (50%), sinus wave form). > > This file will be in "raw" format (pcm without headers). With the > current settings the sample rate is 22050, mono, sample size 16 bits > big endian. > > Regards, > Willem -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From pablo.platt@REDACTED Fri Apr 2 17:14:08 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Fri, 2 Apr 2010 08:14:08 -0700 (PDT) Subject: Complex gproc queries Message-ID: <566312.80019.qm@web112609.mail.gq1.yahoo.com> Hi, I'm building a simple chat server and using gproc to keep track of online users and the process for each user session. Basically, each online user has a process that register a unique session id and non unique user id (same user can have several clients): gproc:add_local_name(SessionId) gproc:add_local_property(UserId, undefined) When a user change his presence(online/busy/offline) I need to inform all the users on his roster. After I retrieved the roster from the db, how can I select all the processes with UserId that is a part of the roster? Rusty Klupaus's blog about gproc note that gproc:select support QLC but I don't understand how to do it. http://rklophaus.com/blog/2009/9/16/gproc-erlang-global-process-registry.html Thanks From diginux@REDACTED Fri Apr 2 18:20:07 2010 From: diginux@REDACTED (Jordan Wilberding) Date: Fri, 2 Apr 2010 10:20:07 -0600 Subject: [erlang-questions] ANN: wxi, a library to build composable GUI on top of wxErlang In-Reply-To: References: Message-ID: I have been wanting to do something like this for a long time, glad someone finally did :) JW On Fri, Apr 2, 2010 at 7:15 AM, Dimitry Golubovsky wrote: > Hi, > > I have been working on this library for some time; now that it is > getting into shape I'd like to announce it to the Erlang community. > > The wxi library [1] is based on the same ideas as found in the Haskell > GUI library Fudgets [2]: each element of graphics user interface (a > widget) also acts as a "stream processor" for streams of events > (messages) generated from user input and I/O operations. Processed > messages are passed around from widget to widget. Thus, desired user > interface layout and dynamics can be achieved via functional > composition of such GUI elements. > > Each widget is defined as a function of certain structure, widgets can > be nested, combined in parallel and in sequence (in terms of message > handling). There is no central event loop: each widget is responsible > for capturing and handling its own events. There is no central GUI > state: widgets that encapsulate state are implemented as processes; > stateless widgets are just functions tail-calling other widget > functions to pass messages around. > > The wxi library is in early development state: while the core > functionality (composition of widgets) is more or less working, the > library of general purpose widgets is not yet completed, but this is > only a matter of time and users contributions. > > For now, the library contains only one module [3] containing widgets > composition functions and few generic widgets (panel, button, text > display). there is a brief tutorial [4], and an "obligatory" example > of calculator [5] with detailed comments in code. > > To give the library a try, download a repo snapshot [6], or clone the > repo: (see http://code.google.com/p/wxi/source/checkout). > > I will be happy to hear from the developers of Erlang-based projects > that involve wx-based GUI whether using such a library would be more > convenient and productive in comparison with "traditional" way (e. g. > the wx-based GUI to dialyzer which is based on a central event loop). > > Thanks in advance for any ideas and suggestions. > > ===================================== > 1. http://code.google.com/p/wxi/ > 2. http://www.md.chalmers.se/Cs/Research/Functional/Fudgets/ > 3. http://wxi.googlecode.com/hg/src/wxi.erl > 4. http://code.google.com/p/wxi/wiki/AnatomyOfWidget > 5. http://wxi.googlecode.com/hg/examples/wxicalc.erl > 6. http://wxi.googlecode.com/files/wxicalc-2010-04-01.tar.gz > > -- > Dimitry Golubovsky > > Anywhere on the Web > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ulf.wiger@REDACTED Fri Apr 2 18:37:25 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 02 Apr 2010 18:37:25 +0200 Subject: [erlang-questions] Complex gproc queries In-Reply-To: <566312.80019.qm@web112609.mail.gq1.yahoo.com> References: <566312.80019.qm@web112609.mail.gq1.yahoo.com> Message-ID: <4BB61D45.9090601@erlang-solutions.com> Pablo Platt wrote: > Hi, > > I'm building a simple chat server and using gproc to keep track of > online users and the process for each user session. > > Basically, each online user has a process that register a unique session id and non unique user id (same user can have several clients): > gproc:add_local_name(SessionId) > gproc:add_local_property(UserId, undefined) > > When a user change his presence(online/busy/offline) I need to inform all the users on his roster. > After I retrieved the roster from the db, how can I select all the processes with UserId that is a part of the roster? > Rusty Klupaus's blog about gproc note that gproc:select support QLC but I don't understand how to do it. > http://rklophaus.com/blog/2009/9/16/gproc-erlang-global-process-registry.html > > Thanks Not sure if you need any complex queries for that. If all members of a roster have a property {p,l,Roster}, then all it should take is: gproc:send({p,l,Roster}, Msg) If you want to e.g. exclude the sender, you could use: [P ! Msg || P <- gproc:lookup_pids({p,l,Roster}) -- [self()]] Disclaimer: When verifying this, I found that gproc:send/2 was broken for properties. I fixed it in the last commit, 1 minute ago. The commit also changes anther bug: aggregated counters had to have an integer value when created. This doesn't make sense, since the value is derived; therefore the only sensible value is 'undefined', which is what you get if you call gproc:reg({a,l,AggrCounter}). Unfortunately, the commit broke the QuickCheck suite... I had forgot that I started writing a test for gproc:await/1 at the SF Erlang factory, didn't finish it. Oh well, if anyone is really hurt by this condition, let me know. (Hans, I haven't merged your changes yet). BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From acton@REDACTED Fri Apr 2 20:22:52 2010 From: acton@REDACTED (Brian Acton) Date: Fri, 2 Apr 2010 11:22:52 -0700 Subject: [erlang-questions] Mnesia deadlock with large volume of dirty operations? In-Reply-To: References: Message-ID: I'm sorry. I neglected to tell you what I had done on the previous day. On the previous day, I had attempted to delete some old records using this methodology: mnesia:write_lock_table(offline_msg), mnesia:foldl( fun(Rec, _Acc) -> case Rec#offline_msg.expire of never -> ok; TS -> if TS < TimeStamp -> mnesia:delete_object(Rec); true -> ok end end end, ok, offline_msg) This delete finished on the 1st node but subsequently locked up all the other nodes on a table lock. The cluster blew up and my 24/7 service went into 1 hr of recovery of downtime. So to recap, on day 1 - transaction start, table lock, delete objects - finished in about 2 minutes on day 2 - dirty select, dirty delete objects - finished in about 2 minutes In both cases, the cluster blew up and became unusable for at least 20-30 minutes. After 20-30 minutes, we initiated recovery protocols. Should I try day 3 - transaction start, no table lock, delete objects ? is the table lock too coarse grained ? considering that the cluster has blown up twice, i'm obviously a little scared to try another variation.... --b On Fri, Apr 2, 2010 at 5:47 AM, Ovidiu Deac wrote: > To me it sounds like another example of premature optimization which > went wrong? :) > > On Fri, Apr 2, 2010 at 10:19 AM, Dan Gudmundsson wrote: > > When you are using dirty, every operation is sent separately to all > nodes, > > i.e. 192593*6 messages, actually a transaction could have been faster > > in this case. > > With one message (large) containing all ops to each node. > > > > What you get is an overloaded mnesia_tm (very long msg queues), > > which do the actual writing of the data on the other (participating > > mnesia nodes). > > > > So transactions will be blocked waiting on mnesia_tm to process those > 200000 > > messages on the other nodes. > > > > /Dan > > > > On Fri, Apr 2, 2010 at 1:11 AM, Brian Acton wrote: > >> Hi guys, > >> > >> I am running R13B04 SMP on FreeBSD 7.3. I have a cluster of 7 nodes > running > >> mnesia. > >> > >> I have a table of 1196143 records using about 1.504GB of storage. It's a > >> reasonably hot table doing a fair number of insert operations at any > given > >> time. > >> > >> I decided that since there was a 2GB limit in mnesia that I should do > some > >> cleanup on the system and specifically this table. > >> > >> Trying to avoid major problems with Mnesia, transaction load, and > deadlock, > >> I decided to do dirty_select and dirty_delete_object individually on the > >> records. > >> > >> I started slow, deleting first 10, then 100, then 1000, then 10000, then > >> 100,000 records. My goal was to delete 192593 records total. > >> > >> The first five deletions went through nicely and caused minimal to no > >> impact. > >> > >> Unfortunately, the very last delete blew up the system. My delete > command > >> completed successfully but on the other nodes, it caused mnesia to get > stuck > >> on pending transactions, caused my message queues to fill up and > basically > >> brought down the whole system. We saw some mnesia is overloaded messages > in > >> our logs on these nodes but did not see a ton of them. > >> > >> Does anyone have any clues on what went wrong? I am attaching my code > below > >> for your review. > >> > >> --b > >> > >> Mnesia configuration tunables: > >> > >> -mnesia no_table_loaders 20 > >> -mnesia dc_dump_limit 40 > >> -mnesia dump_log_write_threshold 10000 > >> > >> Example error message: > >> > >> ** WARNING ** Mnesia is overloaded: {mnesia_tm, message_queue_len, > >> [387,842]} > >> > >> Sample code: > >> > >> Select = fun(Days) -> > >> {MegaSecs, Secs, _MicroSecs} = now(), > >> T = MegaSecs * 1000000 + Secs - 86400 * Days, > >> TimeStamp = {T div 1000000, T rem 1000000, 0}, > >> mnesia:dirty_select(offline_msg, > >> [{'$1', > >> [{'<', {element, 3, '$1'}, > >> {TimeStamp} }], > >> ['$1']}]) > >> end. > >> > >> Count = fun(Days) -> length(Select(Days)) end. > >> > >> Delete = fun(Days, Total) -> > >> C = Select(Days), > >> D = lists:sublist(C, Total), > >> lists:foreach(fun(Rec) -> > >> ok = mnesia:dirty_delete_object(Rec) > >> end, > >> D), > >> length(D) > >> end. > >> > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > From dgud@REDACTED Fri Apr 2 22:05:21 2010 From: dgud@REDACTED (Dan Gudmundsson) Date: Fri, 2 Apr 2010 22:05:21 +0200 Subject: [erlang-questions] Mnesia deadlock with large volume of dirty operations? In-Reply-To: References: Message-ID: clear_table is the fastest way you can delete it, but it will take a while when there is a lot of data. /Dan On Fri, Apr 2, 2010 at 8:22 PM, Brian Acton wrote: > I'm sorry. I neglected to tell you what I had done on the previous day. > > On the previous day, I had attempted to delete some old records using this > methodology: > > ? ? ? ? ? ? ? ?mnesia:write_lock_table(offline_msg), > ? ? ? ? ? ? ? ?mnesia:foldl( > ? ? ? ? ? ? ? ? ?fun(Rec, _Acc) -> > ? ? ? ? ? ? ? ? ? ? ? ? ?case Rec#offline_msg.expire of > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?never -> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ok; > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?TS -> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?if > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?TS < TimeStamp -> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mnesia:delete_object(Rec); > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?true -> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ok > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?end > ? ? ? ? ? ? ? ? ? ? ? ? ?end > ? ? ? ? ? ? ? ? ?end, ok, offline_msg) > > > This delete finished on the 1st node but subsequently locked up all the > other nodes on a table lock. The cluster blew up and my 24/7 service went > into 1 hr of recovery of downtime. > > So to recap, > > on day 1 - transaction start, table lock, delete objects - finished in about > 2 minutes > on day 2 - dirty select, dirty delete objects - finished in about 2 minutes > > In both cases, the cluster blew up and became unusable for at least 20-30 > minutes. After 20-30 minutes, we initiated recovery protocols. > > Should I try > > day 3 - transaction start, no table lock, delete objects > > ? is the table lock too coarse grained ? considering that the cluster has > blown up twice, i'm obviously a little scared to try another variation.... > > --b > > > On Fri, Apr 2, 2010 at 5:47 AM, Ovidiu Deac wrote: > >> To me it sounds like another example of premature optimization which >> went wrong? :) >> >> On Fri, Apr 2, 2010 at 10:19 AM, Dan Gudmundsson wrote: >> > When you are using dirty, every operation is sent separately to all >> nodes, >> > i.e. 192593*6 messages, actually a transaction could have been faster >> > in this case. >> > With one message (large) containing all ops to each node. >> > >> > What you get is an overloaded mnesia_tm (very long msg queues), >> > which do the actual writing of the data on the other (participating >> > mnesia nodes). >> > >> > So transactions will be blocked waiting on mnesia_tm to process those >> 200000 >> > messages on the other nodes. >> > >> > /Dan >> > >> > On Fri, Apr 2, 2010 at 1:11 AM, Brian Acton wrote: >> >> Hi guys, >> >> >> >> I am running R13B04 SMP on FreeBSD 7.3. I have a cluster of 7 nodes >> running >> >> mnesia. >> >> >> >> I have a table of 1196143 records using about 1.504GB of storage. It's a >> >> reasonably hot table doing a fair number of insert operations at any >> given >> >> time. >> >> >> >> I decided that since there was a 2GB limit in mnesia that I should do >> some >> >> cleanup on the system and specifically this table. >> >> >> >> Trying to avoid major problems with Mnesia, transaction load, and >> deadlock, >> >> I decided to do dirty_select and dirty_delete_object individually on the >> >> records. >> >> >> >> I started slow, deleting first 10, then 100, then 1000, then 10000, then >> >> 100,000 records. My goal was to delete 192593 records total. >> >> >> >> The first five deletions went through nicely and caused minimal to no >> >> impact. >> >> >> >> Unfortunately, the very last delete blew up the system. My delete >> command >> >> completed successfully but on the other nodes, it caused mnesia to get >> stuck >> >> on pending transactions, caused my message queues to fill up and >> basically >> >> brought down the whole system. We saw some mnesia is overloaded messages >> in >> >> our logs on these nodes but did not see a ton of them. >> >> >> >> Does anyone have any clues on what went wrong? I am attaching my code >> below >> >> for your review. >> >> >> >> --b >> >> >> >> Mnesia configuration tunables: >> >> >> >> ? ? ?-mnesia no_table_loaders 20 >> >> ? ? ?-mnesia dc_dump_limit 40 >> >> ? ? ?-mnesia dump_log_write_threshold 10000 >> >> >> >> Example error message: >> >> >> >> ** WARNING ** Mnesia is overloaded: {mnesia_tm, message_queue_len, >> >> [387,842]} >> >> >> >> Sample code: >> >> >> >> Select = fun(Days) -> >> >> ? ? ? ? {MegaSecs, Secs, _MicroSecs} = now(), >> >> ? ? ? ? T = MegaSecs * 1000000 + Secs - 86400 * Days, >> >> ? ? ? ? TimeStamp = {T div 1000000, T rem 1000000, 0}, >> >> ? ? ? ? mnesia:dirty_select(offline_msg, >> >> ? ? ? ? ? ? ? ? ? ? [{'$1', >> >> ? ? ? ? ? ? ? ? ? ? ? [{'<', {element, 3, '$1'}, >> >> ? ? ? ? ? ? ? ? ? ? {TimeStamp} }], >> >> ? ? ? ? ? ? ? ? ? ? ? ['$1']}]) >> >> ? ? end. >> >> >> >> Count = fun(Days) -> length(Select(Days)) end. >> >> >> >> Delete = fun(Days, Total) -> >> >> ? ? ? ? C = Select(Days), >> >> ? ? ? ? D = lists:sublist(C, Total), >> >> ? ? ? ? lists:foreach(fun(Rec) -> >> >> ? ? ? ? ? ? ? ? ? ? ? ok = mnesia:dirty_delete_object(Rec) >> >> ? ? ? ? ? ? ? ? ? end, >> >> ? ? ? ? ? ? ? ? ? D), >> >> ? ? ? ? length(D) >> >> ? ? end. >> >> >> > >> > ________________________________________________________________ >> > erlang-questions (at) erlang.org mailing list. >> > See http://www.erlang.org/faq.html >> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > >> > >> > From acton@REDACTED Fri Apr 2 22:19:09 2010 From: acton@REDACTED (Brian Acton) Date: Fri, 2 Apr 2010 13:19:09 -0700 Subject: [erlang-questions] Mnesia deadlock with large volume of dirty operations? In-Reply-To: References: Message-ID: On this particular table, I do not want to delete all entries. This is why I posted a separate post to the mailing list. Combining the two threads back, I want: One table, I want to delete entries > n days. Another table, I want to delete all entries. Both tables are reasonably hot (~1-2 ops per second) and reasonably large (> 1.5GB). I'm hitting the 2GB limit and I need to clean up these tables. So far, any attempts at maintenance (as outlined in previous emails) have resulted in Mnesia seizing up and bringing down the cluster. It sounds like I have to do this in very small increments with wait time between increments. However, I do not have a method and mechanism for determining the size of an increment or a wait time between increments. I'm fine doing ten deletes per 1 second if that's what it takes. However, I'd like to be able to figure out the maximum number of deletes that I can do in the minimum amount of time. I'm definitely open to suggestion on this. --b On Fri, Apr 2, 2010 at 1:05 PM, Dan Gudmundsson wrote: > clear_table is the fastest way you can delete it, but it will take a > while when there is a lot of data. > > /Dan > > On Fri, Apr 2, 2010 at 8:22 PM, Brian Acton wrote: > > I'm sorry. I neglected to tell you what I had done on the previous day. > > > > On the previous day, I had attempted to delete some old records using > this > > methodology: > > > > mnesia:write_lock_table(offline_msg), > > mnesia:foldl( > > fun(Rec, _Acc) -> > > case Rec#offline_msg.expire of > > never -> > > ok; > > TS -> > > if > > TS < TimeStamp -> > > mnesia:delete_object(Rec); > > true -> > > ok > > end > > end > > end, ok, offline_msg) > > > > > > This delete finished on the 1st node but subsequently locked up all the > > other nodes on a table lock. The cluster blew up and my 24/7 service went > > into 1 hr of recovery of downtime. > > > > So to recap, > > > > on day 1 - transaction start, table lock, delete objects - finished in > about > > 2 minutes > > on day 2 - dirty select, dirty delete objects - finished in about 2 > minutes > > > > In both cases, the cluster blew up and became unusable for at least 20-30 > > minutes. After 20-30 minutes, we initiated recovery protocols. > > > > Should I try > > > > day 3 - transaction start, no table lock, delete objects > > > > ? is the table lock too coarse grained ? considering that the cluster has > > blown up twice, i'm obviously a little scared to try another > variation.... > > > > --b > > > > > > On Fri, Apr 2, 2010 at 5:47 AM, Ovidiu Deac > wrote: > > > >> To me it sounds like another example of premature optimization which > >> went wrong? :) > >> > >> On Fri, Apr 2, 2010 at 10:19 AM, Dan Gudmundsson > wrote: > >> > When you are using dirty, every operation is sent separately to all > >> nodes, > >> > i.e. 192593*6 messages, actually a transaction could have been faster > >> > in this case. > >> > With one message (large) containing all ops to each node. > >> > > >> > What you get is an overloaded mnesia_tm (very long msg queues), > >> > which do the actual writing of the data on the other (participating > >> > mnesia nodes). > >> > > >> > So transactions will be blocked waiting on mnesia_tm to process those > >> 200000 > >> > messages on the other nodes. > >> > > >> > /Dan > >> > > >> > On Fri, Apr 2, 2010 at 1:11 AM, Brian Acton > wrote: > >> >> Hi guys, > >> >> > >> >> I am running R13B04 SMP on FreeBSD 7.3. I have a cluster of 7 nodes > >> running > >> >> mnesia. > >> >> > >> >> I have a table of 1196143 records using about 1.504GB of storage. > It's a > >> >> reasonably hot table doing a fair number of insert operations at any > >> given > >> >> time. > >> >> > >> >> I decided that since there was a 2GB limit in mnesia that I should do > >> some > >> >> cleanup on the system and specifically this table. > >> >> > >> >> Trying to avoid major problems with Mnesia, transaction load, and > >> deadlock, > >> >> I decided to do dirty_select and dirty_delete_object individually on > the > >> >> records. > >> >> > >> >> I started slow, deleting first 10, then 100, then 1000, then 10000, > then > >> >> 100,000 records. My goal was to delete 192593 records total. > >> >> > >> >> The first five deletions went through nicely and caused minimal to no > >> >> impact. > >> >> > >> >> Unfortunately, the very last delete blew up the system. My delete > >> command > >> >> completed successfully but on the other nodes, it caused mnesia to > get > >> stuck > >> >> on pending transactions, caused my message queues to fill up and > >> basically > >> >> brought down the whole system. We saw some mnesia is overloaded > messages > >> in > >> >> our logs on these nodes but did not see a ton of them. > >> >> > >> >> Does anyone have any clues on what went wrong? I am attaching my code > >> below > >> >> for your review. > >> >> > >> >> --b > >> >> > >> >> Mnesia configuration tunables: > >> >> > >> >> -mnesia no_table_loaders 20 > >> >> -mnesia dc_dump_limit 40 > >> >> -mnesia dump_log_write_threshold 10000 > >> >> > >> >> Example error message: > >> >> > >> >> ** WARNING ** Mnesia is overloaded: {mnesia_tm, message_queue_len, > >> >> [387,842]} > >> >> > >> >> Sample code: > >> >> > >> >> Select = fun(Days) -> > >> >> {MegaSecs, Secs, _MicroSecs} = now(), > >> >> T = MegaSecs * 1000000 + Secs - 86400 * Days, > >> >> TimeStamp = {T div 1000000, T rem 1000000, 0}, > >> >> mnesia:dirty_select(offline_msg, > >> >> [{'$1', > >> >> [{'<', {element, 3, '$1'}, > >> >> {TimeStamp} }], > >> >> ['$1']}]) > >> >> end. > >> >> > >> >> Count = fun(Days) -> length(Select(Days)) end. > >> >> > >> >> Delete = fun(Days, Total) -> > >> >> C = Select(Days), > >> >> D = lists:sublist(C, Total), > >> >> lists:foreach(fun(Rec) -> > >> >> ok = mnesia:dirty_delete_object(Rec) > >> >> end, > >> >> D), > >> >> length(D) > >> >> end. > >> >> > >> > > >> > ________________________________________________________________ > >> > erlang-questions (at) erlang.org mailing list. > >> > See http://www.erlang.org/faq.html > >> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > >> > > >> > > > From dgud@REDACTED Fri Apr 2 22:55:11 2010 From: dgud@REDACTED (Dan Gudmundsson) Date: Fri, 2 Apr 2010 22:55:11 +0200 Subject: [erlang-questions] Mnesia deadlock with large volume of dirty operations? In-Reply-To: References: Message-ID: Well, I can't much advice, but I would definitely test this on a non live system first. mnesia:fold is not the best tool when you are changing a lot of records, it will have to keep every change in memory until you have traversed the whole table. And it is slow with lot of the changes, since have to compensate for the things you have done earlier in the transaction. I assume your are using dets (disc_only) since you are afraid of the 2G limit, or is it memory limit on windows? dets is slow, mnesia is primarly a ram database. The only way I see it is to chunk though the tables a couple 100~1000 records per transaction or something. And have code that can deal with both the new and old format during the changing of the database. Good luck /Dan On Fri, Apr 2, 2010 at 10:19 PM, Brian Acton wrote: > On this particular table, I do not want to delete all entries. This is why I > posted a separate post to the mailing list. Combining the two threads back, > I want: > > One table, I want to delete entries > n days. > Another table, I want to delete all entries. > > Both tables are reasonably hot (~1-2 ops per second) and reasonably large (> > 1.5GB). I'm hitting the 2GB limit and I need to clean up these tables. > > So far, any attempts at maintenance (as outlined in previous emails) have > resulted in Mnesia seizing up and bringing down the cluster. > > It sounds like I have to do this in very small increments with wait time > between increments. However, I do not have a method and mechanism for > determining the size of an increment or a wait time between increments. I'm > fine doing ten deletes per 1 second if that's what it takes. However, I'd > like to be able to figure out the maximum number of deletes that I can do in > the minimum amount of time. > > I'm definitely open to suggestion on this. > > --b > > On Fri, Apr 2, 2010 at 1:05 PM, Dan Gudmundsson wrote: > >> clear_table is the fastest way you can delete it, but it will take a >> while when there is a lot of data. >> >> /Dan >> >> On Fri, Apr 2, 2010 at 8:22 PM, Brian Acton wrote: >> > I'm sorry. I neglected to tell you what I had done on the previous day. >> > >> > On the previous day, I had attempted to delete some old records using >> this >> > methodology: >> > >> > ? ? ? ? ? ? ? ?mnesia:write_lock_table(offline_msg), >> > ? ? ? ? ? ? ? ?mnesia:foldl( >> > ? ? ? ? ? ? ? ? ?fun(Rec, _Acc) -> >> > ? ? ? ? ? ? ? ? ? ? ? ? ?case Rec#offline_msg.expire of >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?never -> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ok; >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?TS -> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?if >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?TS < TimeStamp -> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mnesia:delete_object(Rec); >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?true -> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ok >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?end >> > ? ? ? ? ? ? ? ? ? ? ? ? ?end >> > ? ? ? ? ? ? ? ? ?end, ok, offline_msg) >> > >> > >> > This delete finished on the 1st node but subsequently locked up all the >> > other nodes on a table lock. The cluster blew up and my 24/7 service went >> > into 1 hr of recovery of downtime. >> > >> > So to recap, >> > >> > on day 1 - transaction start, table lock, delete objects - finished in >> about >> > 2 minutes >> > on day 2 - dirty select, dirty delete objects - finished in about 2 >> minutes >> > >> > In both cases, the cluster blew up and became unusable for at least 20-30 >> > minutes. After 20-30 minutes, we initiated recovery protocols. >> > >> > Should I try >> > >> > day 3 - transaction start, no table lock, delete objects >> > >> > ? is the table lock too coarse grained ? considering that the cluster has >> > blown up twice, i'm obviously a little scared to try another >> variation.... >> > >> > --b >> > >> > >> > On Fri, Apr 2, 2010 at 5:47 AM, Ovidiu Deac >> wrote: >> > >> >> To me it sounds like another example of premature optimization which >> >> went wrong? :) >> >> >> >> On Fri, Apr 2, 2010 at 10:19 AM, Dan Gudmundsson >> wrote: >> >> > When you are using dirty, every operation is sent separately to all >> >> nodes, >> >> > i.e. 192593*6 messages, actually a transaction could have been faster >> >> > in this case. >> >> > With one message (large) containing all ops to each node. >> >> > >> >> > What you get is an overloaded mnesia_tm (very long msg queues), >> >> > which do the actual writing of the data on the other (participating >> >> > mnesia nodes). >> >> > >> >> > So transactions will be blocked waiting on mnesia_tm to process those >> >> 200000 >> >> > messages on the other nodes. >> >> > >> >> > /Dan >> >> > >> >> > On Fri, Apr 2, 2010 at 1:11 AM, Brian Acton >> wrote: >> >> >> Hi guys, >> >> >> >> >> >> I am running R13B04 SMP on FreeBSD 7.3. I have a cluster of 7 nodes >> >> running >> >> >> mnesia. >> >> >> >> >> >> I have a table of 1196143 records using about 1.504GB of storage. >> It's a >> >> >> reasonably hot table doing a fair number of insert operations at any >> >> given >> >> >> time. >> >> >> >> >> >> I decided that since there was a 2GB limit in mnesia that I should do >> >> some >> >> >> cleanup on the system and specifically this table. >> >> >> >> >> >> Trying to avoid major problems with Mnesia, transaction load, and >> >> deadlock, >> >> >> I decided to do dirty_select and dirty_delete_object individually on >> the >> >> >> records. >> >> >> >> >> >> I started slow, deleting first 10, then 100, then 1000, then 10000, >> then >> >> >> 100,000 records. My goal was to delete 192593 records total. >> >> >> >> >> >> The first five deletions went through nicely and caused minimal to no >> >> >> impact. >> >> >> >> >> >> Unfortunately, the very last delete blew up the system. My delete >> >> command >> >> >> completed successfully but on the other nodes, it caused mnesia to >> get >> >> stuck >> >> >> on pending transactions, caused my message queues to fill up and >> >> basically >> >> >> brought down the whole system. We saw some mnesia is overloaded >> messages >> >> in >> >> >> our logs on these nodes but did not see a ton of them. >> >> >> >> >> >> Does anyone have any clues on what went wrong? I am attaching my code >> >> below >> >> >> for your review. >> >> >> >> >> >> --b >> >> >> >> >> >> Mnesia configuration tunables: >> >> >> >> >> >> ? ? ?-mnesia no_table_loaders 20 >> >> >> ? ? ?-mnesia dc_dump_limit 40 >> >> >> ? ? ?-mnesia dump_log_write_threshold 10000 >> >> >> >> >> >> Example error message: >> >> >> >> >> >> ** WARNING ** Mnesia is overloaded: {mnesia_tm, message_queue_len, >> >> >> [387,842]} >> >> >> >> >> >> Sample code: >> >> >> >> >> >> Select = fun(Days) -> >> >> >> ? ? ? ? {MegaSecs, Secs, _MicroSecs} = now(), >> >> >> ? ? ? ? T = MegaSecs * 1000000 + Secs - 86400 * Days, >> >> >> ? ? ? ? TimeStamp = {T div 1000000, T rem 1000000, 0}, >> >> >> ? ? ? ? mnesia:dirty_select(offline_msg, >> >> >> ? ? ? ? ? ? ? ? ? ? [{'$1', >> >> >> ? ? ? ? ? ? ? ? ? ? ? [{'<', {element, 3, '$1'}, >> >> >> ? ? ? ? ? ? ? ? ? ? {TimeStamp} }], >> >> >> ? ? ? ? ? ? ? ? ? ? ? ['$1']}]) >> >> >> ? ? end. >> >> >> >> >> >> Count = fun(Days) -> length(Select(Days)) end. >> >> >> >> >> >> Delete = fun(Days, Total) -> >> >> >> ? ? ? ? C = Select(Days), >> >> >> ? ? ? ? D = lists:sublist(C, Total), >> >> >> ? ? ? ? lists:foreach(fun(Rec) -> >> >> >> ? ? ? ? ? ? ? ? ? ? ? ok = mnesia:dirty_delete_object(Rec) >> >> >> ? ? ? ? ? ? ? ? ? end, >> >> >> ? ? ? ? ? ? ? ? ? D), >> >> >> ? ? ? ? length(D) >> >> >> ? ? end. >> >> >> >> >> > >> >> > ________________________________________________________________ >> >> > erlang-questions (at) erlang.org mailing list. >> >> > See http://www.erlang.org/faq.html >> >> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > >> >> > >> >> >> > >> > From acton@REDACTED Fri Apr 2 23:08:32 2010 From: acton@REDACTED (Brian Acton) Date: Fri, 2 Apr 2010 14:08:32 -0700 Subject: [erlang-questions] Mnesia deadlock with large volume of dirty operations? In-Reply-To: References: Message-ID: Yes. I am using dets (disc_only) tables in mnesia. Since I was able to delete 10k records previously. I think I am going to start with a baseline of 10k record with a 60 second sleep interval. Hopefully this will work successfully. I wish I knew what a more appropriate sleep period would be as the maintenance is now going to take a very long time. Thanks for your help, --b On Fri, Apr 2, 2010 at 1:55 PM, Dan Gudmundsson wrote: > Well, I can't much advice, but I would definitely test this on a non > live system first. > > mnesia:fold is not the best tool when you are changing a lot of > records, it will have to keep > every change in memory until you have traversed the whole table. And > it is slow with lot > of the changes, since have to compensate for the things you have done > earlier in the transaction. > > I assume your are using dets (disc_only) since you are afraid of the > 2G limit, or is it > memory limit on windows? > > dets is slow, mnesia is primarly a ram database. > > The only way I see it is to chunk though the tables a couple 100~1000 > records > per transaction or something. > And have code that can deal with both the new and old format during the > changing > of the database. > > Good luck > /Dan > > On Fri, Apr 2, 2010 at 10:19 PM, Brian Acton wrote: > > On this particular table, I do not want to delete all entries. This is > why I > > posted a separate post to the mailing list. Combining the two threads > back, > > I want: > > > > One table, I want to delete entries > n days. > > Another table, I want to delete all entries. > > > > Both tables are reasonably hot (~1-2 ops per second) and reasonably large > (> > > 1.5GB). I'm hitting the 2GB limit and I need to clean up these tables. > > > > So far, any attempts at maintenance (as outlined in previous emails) have > > resulted in Mnesia seizing up and bringing down the cluster. > > > > It sounds like I have to do this in very small increments with wait time > > between increments. However, I do not have a method and mechanism for > > determining the size of an increment or a wait time between increments. > I'm > > fine doing ten deletes per 1 second if that's what it takes. However, I'd > > like to be able to figure out the maximum number of deletes that I can do > in > > the minimum amount of time. > > > > I'm definitely open to suggestion on this. > > > > --b > > > > On Fri, Apr 2, 2010 at 1:05 PM, Dan Gudmundsson wrote: > > > >> clear_table is the fastest way you can delete it, but it will take a > >> while when there is a lot of data. > >> > >> /Dan > >> > >> On Fri, Apr 2, 2010 at 8:22 PM, Brian Acton wrote: > >> > I'm sorry. I neglected to tell you what I had done on the previous > day. > >> > > >> > On the previous day, I had attempted to delete some old records using > >> this > >> > methodology: > >> > > >> > mnesia:write_lock_table(offline_msg), > >> > mnesia:foldl( > >> > fun(Rec, _Acc) -> > >> > case Rec#offline_msg.expire of > >> > never -> > >> > ok; > >> > TS -> > >> > if > >> > TS < TimeStamp -> > >> > mnesia:delete_object(Rec); > >> > true -> > >> > ok > >> > end > >> > end > >> > end, ok, offline_msg) > >> > > >> > > >> > This delete finished on the 1st node but subsequently locked up all > the > >> > other nodes on a table lock. The cluster blew up and my 24/7 service > went > >> > into 1 hr of recovery of downtime. > >> > > >> > So to recap, > >> > > >> > on day 1 - transaction start, table lock, delete objects - finished in > >> about > >> > 2 minutes > >> > on day 2 - dirty select, dirty delete objects - finished in about 2 > >> minutes > >> > > >> > In both cases, the cluster blew up and became unusable for at least > 20-30 > >> > minutes. After 20-30 minutes, we initiated recovery protocols. > >> > > >> > Should I try > >> > > >> > day 3 - transaction start, no table lock, delete objects > >> > > >> > ? is the table lock too coarse grained ? considering that the cluster > has > >> > blown up twice, i'm obviously a little scared to try another > >> variation.... > >> > > >> > --b > >> > > >> > > >> > On Fri, Apr 2, 2010 at 5:47 AM, Ovidiu Deac > >> wrote: > >> > > >> >> To me it sounds like another example of premature optimization which > >> >> went wrong? :) > >> >> > >> >> On Fri, Apr 2, 2010 at 10:19 AM, Dan Gudmundsson > >> wrote: > >> >> > When you are using dirty, every operation is sent separately to all > >> >> nodes, > >> >> > i.e. 192593*6 messages, actually a transaction could have been > faster > >> >> > in this case. > >> >> > With one message (large) containing all ops to each node. > >> >> > > >> >> > What you get is an overloaded mnesia_tm (very long msg queues), > >> >> > which do the actual writing of the data on the other (participating > >> >> > mnesia nodes). > >> >> > > >> >> > So transactions will be blocked waiting on mnesia_tm to process > those > >> >> 200000 > >> >> > messages on the other nodes. > >> >> > > >> >> > /Dan > >> >> > > >> >> > On Fri, Apr 2, 2010 at 1:11 AM, Brian Acton > >> wrote: > >> >> >> Hi guys, > >> >> >> > >> >> >> I am running R13B04 SMP on FreeBSD 7.3. I have a cluster of 7 > nodes > >> >> running > >> >> >> mnesia. > >> >> >> > >> >> >> I have a table of 1196143 records using about 1.504GB of storage. > >> It's a > >> >> >> reasonably hot table doing a fair number of insert operations at > any > >> >> given > >> >> >> time. > >> >> >> > >> >> >> I decided that since there was a 2GB limit in mnesia that I should > do > >> >> some > >> >> >> cleanup on the system and specifically this table. > >> >> >> > >> >> >> Trying to avoid major problems with Mnesia, transaction load, and > >> >> deadlock, > >> >> >> I decided to do dirty_select and dirty_delete_object individually > on > >> the > >> >> >> records. > >> >> >> > >> >> >> I started slow, deleting first 10, then 100, then 1000, then > 10000, > >> then > >> >> >> 100,000 records. My goal was to delete 192593 records total. > >> >> >> > >> >> >> The first five deletions went through nicely and caused minimal to > no > >> >> >> impact. > >> >> >> > >> >> >> Unfortunately, the very last delete blew up the system. My delete > >> >> command > >> >> >> completed successfully but on the other nodes, it caused mnesia to > >> get > >> >> stuck > >> >> >> on pending transactions, caused my message queues to fill up and > >> >> basically > >> >> >> brought down the whole system. We saw some mnesia is overloaded > >> messages > >> >> in > >> >> >> our logs on these nodes but did not see a ton of them. > >> >> >> > >> >> >> Does anyone have any clues on what went wrong? I am attaching my > code > >> >> below > >> >> >> for your review. > >> >> >> > >> >> >> --b > >> >> >> > >> >> >> Mnesia configuration tunables: > >> >> >> > >> >> >> -mnesia no_table_loaders 20 > >> >> >> -mnesia dc_dump_limit 40 > >> >> >> -mnesia dump_log_write_threshold 10000 > >> >> >> > >> >> >> Example error message: > >> >> >> > >> >> >> ** WARNING ** Mnesia is overloaded: {mnesia_tm, message_queue_len, > >> >> >> [387,842]} > >> >> >> > >> >> >> Sample code: > >> >> >> > >> >> >> Select = fun(Days) -> > >> >> >> {MegaSecs, Secs, _MicroSecs} = now(), > >> >> >> T = MegaSecs * 1000000 + Secs - 86400 * Days, > >> >> >> TimeStamp = {T div 1000000, T rem 1000000, 0}, > >> >> >> mnesia:dirty_select(offline_msg, > >> >> >> [{'$1', > >> >> >> [{'<', {element, 3, '$1'}, > >> >> >> {TimeStamp} }], > >> >> >> ['$1']}]) > >> >> >> end. > >> >> >> > >> >> >> Count = fun(Days) -> length(Select(Days)) end. > >> >> >> > >> >> >> Delete = fun(Days, Total) -> > >> >> >> C = Select(Days), > >> >> >> D = lists:sublist(C, Total), > >> >> >> lists:foreach(fun(Rec) -> > >> >> >> ok = mnesia:dirty_delete_object(Rec) > >> >> >> end, > >> >> >> D), > >> >> >> length(D) > >> >> >> end. > >> >> >> > >> >> > > >> >> > ________________________________________________________________ > >> >> > erlang-questions (at) erlang.org mailing list. > >> >> > See http://www.erlang.org/faq.html > >> >> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> >> > > >> >> > > >> >> > >> > > >> > > > From bob@REDACTED Fri Apr 2 23:14:11 2010 From: bob@REDACTED (Bob Ippolito) Date: Fri, 2 Apr 2010 14:14:11 -0700 Subject: [erlang-questions] Mnesia deadlock with large volume of dirty operations? In-Reply-To: References: Message-ID: You might want to measure the message_queue_len of the mnesia_tm processes (on each node) to see if it's getting behind, and tune your waits based upon if the message queues are small/empty or not. On Fri, Apr 2, 2010 at 2:08 PM, Brian Acton wrote: > Yes. I am using dets (disc_only) tables in mnesia. > > Since I was able to delete 10k records previously. I think I am going to > start with a baseline of 10k record with a 60 second sleep interval. > Hopefully this will work successfully. I wish I knew what a more appropriate > sleep period would be as the maintenance is now going to take a very long > time. > > Thanks for your help, > > --b > > On Fri, Apr 2, 2010 at 1:55 PM, Dan Gudmundsson wrote: > >> Well, I can't much advice, but I would definitely test this on a non >> live system first. >> >> mnesia:fold is not the best tool when you are changing a lot of >> records, it will have to keep >> every change in memory until you have traversed the whole table. And >> it is slow with lot >> of the changes, since have to compensate for the things you have done >> earlier in the transaction. >> >> I assume your are using dets (disc_only) since you are afraid of the >> 2G limit, or is it >> memory limit on windows? >> >> dets is slow, mnesia is primarly a ram database. >> >> The only way I see it is to chunk though the tables a couple 100~1000 >> records >> per transaction or something. >> And have code that can deal with both the new and old format during the >> changing >> of the database. >> >> Good luck >> /Dan >> >> On Fri, Apr 2, 2010 at 10:19 PM, Brian Acton wrote: >> > On this particular table, I do not want to delete all entries. This is >> why I >> > posted a separate post to the mailing list. Combining the two threads >> back, >> > I want: >> > >> > One table, I want to delete entries > n days. >> > Another table, I want to delete all entries. >> > >> > Both tables are reasonably hot (~1-2 ops per second) and reasonably large >> (> >> > 1.5GB). I'm hitting the 2GB limit and I need to clean up these tables. >> > >> > So far, any attempts at maintenance (as outlined in previous emails) have >> > resulted in Mnesia seizing up and bringing down the cluster. >> > >> > It sounds like I have to do this in very small increments with wait time >> > between increments. However, I do not have a method and mechanism for >> > determining the size of an increment or a wait time between increments. >> I'm >> > fine doing ten deletes per 1 second if that's what it takes. However, I'd >> > like to be able to figure out the maximum number of deletes that I can do >> in >> > the minimum amount of time. >> > >> > I'm definitely open to suggestion on this. >> > >> > --b >> > >> > On Fri, Apr 2, 2010 at 1:05 PM, Dan Gudmundsson wrote: >> > >> >> clear_table is the fastest way you can delete it, but it will take a >> >> while when there is a lot of data. >> >> >> >> /Dan >> >> >> >> On Fri, Apr 2, 2010 at 8:22 PM, Brian Acton wrote: >> >> > I'm sorry. I neglected to tell you what I had done on the previous >> day. >> >> > >> >> > On the previous day, I had attempted to delete some old records using >> >> this >> >> > methodology: >> >> > >> >> > ? ? ? ? ? ? ? ?mnesia:write_lock_table(offline_msg), >> >> > ? ? ? ? ? ? ? ?mnesia:foldl( >> >> > ? ? ? ? ? ? ? ? ?fun(Rec, _Acc) -> >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ?case Rec#offline_msg.expire of >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?never -> >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ok; >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?TS -> >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?if >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?TS < TimeStamp -> >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mnesia:delete_object(Rec); >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?true -> >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ok >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?end >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ?end >> >> > ? ? ? ? ? ? ? ? ?end, ok, offline_msg) >> >> > >> >> > >> >> > This delete finished on the 1st node but subsequently locked up all >> the >> >> > other nodes on a table lock. The cluster blew up and my 24/7 service >> went >> >> > into 1 hr of recovery of downtime. >> >> > >> >> > So to recap, >> >> > >> >> > on day 1 - transaction start, table lock, delete objects - finished in >> >> about >> >> > 2 minutes >> >> > on day 2 - dirty select, dirty delete objects - finished in about 2 >> >> minutes >> >> > >> >> > In both cases, the cluster blew up and became unusable for at least >> 20-30 >> >> > minutes. After 20-30 minutes, we initiated recovery protocols. >> >> > >> >> > Should I try >> >> > >> >> > day 3 - transaction start, no table lock, delete objects >> >> > >> >> > ? is the table lock too coarse grained ? considering that the cluster >> has >> >> > blown up twice, i'm obviously a little scared to try another >> >> variation.... >> >> > >> >> > --b >> >> > >> >> > >> >> > On Fri, Apr 2, 2010 at 5:47 AM, Ovidiu Deac >> >> wrote: >> >> > >> >> >> To me it sounds like another example of premature optimization which >> >> >> went wrong? :) >> >> >> >> >> >> On Fri, Apr 2, 2010 at 10:19 AM, Dan Gudmundsson >> >> wrote: >> >> >> > When you are using dirty, every operation is sent separately to all >> >> >> nodes, >> >> >> > i.e. 192593*6 messages, actually a transaction could have been >> faster >> >> >> > in this case. >> >> >> > With one message (large) containing all ops to each node. >> >> >> > >> >> >> > What you get is an overloaded mnesia_tm (very long msg queues), >> >> >> > which do the actual writing of the data on the other (participating >> >> >> > mnesia nodes). >> >> >> > >> >> >> > So transactions will be blocked waiting on mnesia_tm to process >> those >> >> >> 200000 >> >> >> > messages on the other nodes. >> >> >> > >> >> >> > /Dan >> >> >> > >> >> >> > On Fri, Apr 2, 2010 at 1:11 AM, Brian Acton >> >> wrote: >> >> >> >> Hi guys, >> >> >> >> >> >> >> >> I am running R13B04 SMP on FreeBSD 7.3. I have a cluster of 7 >> nodes >> >> >> running >> >> >> >> mnesia. >> >> >> >> >> >> >> >> I have a table of 1196143 records using about 1.504GB of storage. >> >> It's a >> >> >> >> reasonably hot table doing a fair number of insert operations at >> any >> >> >> given >> >> >> >> time. >> >> >> >> >> >> >> >> I decided that since there was a 2GB limit in mnesia that I should >> do >> >> >> some >> >> >> >> cleanup on the system and specifically this table. >> >> >> >> >> >> >> >> Trying to avoid major problems with Mnesia, transaction load, and >> >> >> deadlock, >> >> >> >> I decided to do dirty_select and dirty_delete_object individually >> on >> >> the >> >> >> >> records. >> >> >> >> >> >> >> >> I started slow, deleting first 10, then 100, then 1000, then >> 10000, >> >> then >> >> >> >> 100,000 records. My goal was to delete 192593 records total. >> >> >> >> >> >> >> >> The first five deletions went through nicely and caused minimal to >> no >> >> >> >> impact. >> >> >> >> >> >> >> >> Unfortunately, the very last delete blew up the system. My delete >> >> >> command >> >> >> >> completed successfully but on the other nodes, it caused mnesia to >> >> get >> >> >> stuck >> >> >> >> on pending transactions, caused my message queues to fill up and >> >> >> basically >> >> >> >> brought down the whole system. We saw some mnesia is overloaded >> >> messages >> >> >> in >> >> >> >> our logs on these nodes but did not see a ton of them. >> >> >> >> >> >> >> >> Does anyone have any clues on what went wrong? I am attaching my >> code >> >> >> below >> >> >> >> for your review. >> >> >> >> >> >> >> >> --b >> >> >> >> >> >> >> >> Mnesia configuration tunables: >> >> >> >> >> >> >> >> ? ? ?-mnesia no_table_loaders 20 >> >> >> >> ? ? ?-mnesia dc_dump_limit 40 >> >> >> >> ? ? ?-mnesia dump_log_write_threshold 10000 >> >> >> >> >> >> >> >> Example error message: >> >> >> >> >> >> >> >> ** WARNING ** Mnesia is overloaded: {mnesia_tm, message_queue_len, >> >> >> >> [387,842]} >> >> >> >> >> >> >> >> Sample code: >> >> >> >> >> >> >> >> Select = fun(Days) -> >> >> >> >> ? ? ? ? {MegaSecs, Secs, _MicroSecs} = now(), >> >> >> >> ? ? ? ? T = MegaSecs * 1000000 + Secs - 86400 * Days, >> >> >> >> ? ? ? ? TimeStamp = {T div 1000000, T rem 1000000, 0}, >> >> >> >> ? ? ? ? mnesia:dirty_select(offline_msg, >> >> >> >> ? ? ? ? ? ? ? ? ? ? [{'$1', >> >> >> >> ? ? ? ? ? ? ? ? ? ? ? [{'<', {element, 3, '$1'}, >> >> >> >> ? ? ? ? ? ? ? ? ? ? {TimeStamp} }], >> >> >> >> ? ? ? ? ? ? ? ? ? ? ? ['$1']}]) >> >> >> >> ? ? end. >> >> >> >> >> >> >> >> Count = fun(Days) -> length(Select(Days)) end. >> >> >> >> >> >> >> >> Delete = fun(Days, Total) -> >> >> >> >> ? ? ? ? C = Select(Days), >> >> >> >> ? ? ? ? D = lists:sublist(C, Total), >> >> >> >> ? ? ? ? lists:foreach(fun(Rec) -> >> >> >> >> ? ? ? ? ? ? ? ? ? ? ? ok = mnesia:dirty_delete_object(Rec) >> >> >> >> ? ? ? ? ? ? ? ? ? end, >> >> >> >> ? ? ? ? ? ? ? ? ? D), >> >> >> >> ? ? ? ? length(D) >> >> >> >> ? ? end. >> >> >> >> >> >> >> > >> >> >> > ________________________________________________________________ >> >> >> > erlang-questions (at) erlang.org mailing list. >> >> >> > See http://www.erlang.org/faq.html >> >> >> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> > >> >> >> > >> >> >> >> >> > >> >> >> > >> > From pablo.platt@REDACTED Fri Apr 2 22:37:14 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Fri, 2 Apr 2010 13:37:14 -0700 (PDT) Subject: [erlang-questions] Complex gproc queries In-Reply-To: <4BB61D45.9090601@erlang-solutions.com> References: <566312.80019.qm@web112609.mail.gq1.yahoo.com> <4BB61D45.9090601@erlang-solutions.com> Message-ID: <974379.86275.qm@web112601.mail.gq1.yahoo.com> I should have said that Roster is a list with the contact list IDs Roster = [id1, id2, id3...] each user have a different list. I need to send a message to all the the users in the list that are online. Thanks ________________________________ From: Ulf Wiger To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Fri, April 2, 2010 7:37:25 PM Subject: Re: [erlang-questions] Complex gproc queries Pablo Platt wrote: > Hi, > > I'm building a simple chat server and using gproc to keep track of online users and the process for each user session. > > Basically, each online user has a process that register a unique session id and non unique user id (same user can have several clients): > gproc:add_local_name(SessionId) > gproc:add_local_property(UserId, undefined) > > When a user change his presence(online/busy/offline) I need to inform all the users on his roster. > After I retrieved the roster from the db, how can I select all the processes with UserId that is a part of the roster? > Rusty Klupaus's blog about gproc note that gproc:select support QLC but I don't understand how to do it. > http://rklophaus.com/blog/2009/9/16/gproc-erlang-global-process-registry.html > > Thanks Not sure if you need any complex queries for that. If all members of a roster have a property {p,l,Roster}, then all it should take is: gproc:send({p,l,Roster}, Msg) If you want to e.g. exclude the sender, you could use: [P ! Msg || P <- gproc:lookup_pids({p,l,Roster}) -- [self()]] Disclaimer: When verifying this, I found that gproc:send/2 was broken for properties. I fixed it in the last commit, 1 minute ago. The commit also changes anther bug: aggregated counters had to have an integer value when created. This doesn't make sense, since the value is derived; therefore the only sensible value is 'undefined', which is what you get if you call gproc:reg({a,l,AggrCounter}). Unfortunately, the commit broke the QuickCheck suite... I had forgot that I started writing a test for gproc:await/1 at the SF Erlang factory, didn't finish it. Oh well, if anyone is really hurt by this condition, let me know. (Hans, I haven't merged your changes yet). BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From acton@REDACTED Sat Apr 3 01:00:06 2010 From: acton@REDACTED (Brian Acton) Date: Fri, 2 Apr 2010 16:00:06 -0700 Subject: [erlang-questions] Mnesia deadlock with large volume of dirty operations? In-Reply-To: References: Message-ID: Well, I went ahead and deleted about 82k messages in 10k batches. I did this over about a 15 minute period. The good news is that the system has not crashed. The bad news is that some of the reported sizes of the tables have grown dangerously close to the 2GB limit and further that the tables appear to be wholly inconsistent: Here is a dump of my 7 nodes node 1: offline_msg : with 995638 records occupying 2039455556 bytes on disc node 2: offline_msg : with 1015600 records occupying 2097112225 bytes on disc node 3: offline_msg : with 995641 records occupying 1797758788 bytes on disc node 4: offline_msg : with 1015204 records occupying 2096658267 bytes on disc node 5: offline_msg : with 995615 records occupying 1776787268 bytes on disc node 6: offline_msg : with 995618 records occupying 1388054291 bytes on disc node 7: offline_msg : with 995611 records occupying 1388054291 bytes on disc before I started, the nodes were about 1.36GB on disc. Some of them are now close to the 2GB limit. the delete operation was initiated on node 5. the message queue's on all of the nodes are zero across the board. my logs are clean and give the indication that everything proceeded normally. i think at this point, my only recourse is to restart node 1-5 in the hopes that they clone from 6 and 7 providing the best space reclamation.... anyone else have thoughts on the matter ? --b On Fri, Apr 2, 2010 at 2:14 PM, Bob Ippolito wrote: > You might want to measure the message_queue_len of the mnesia_tm > processes (on each node) to see if it's getting behind, and tune your > waits based upon if the message queues are small/empty or not. > > On Fri, Apr 2, 2010 at 2:08 PM, Brian Acton wrote: > > Yes. I am using dets (disc_only) tables in mnesia. > > > > Since I was able to delete 10k records previously. I think I am going to > > start with a baseline of 10k record with a 60 second sleep interval. > > Hopefully this will work successfully. I wish I knew what a more > appropriate > > sleep period would be as the maintenance is now going to take a very long > > time. > > > > Thanks for your help, > > > > --b > > > > On Fri, Apr 2, 2010 at 1:55 PM, Dan Gudmundsson wrote: > > > >> Well, I can't much advice, but I would definitely test this on a non > >> live system first. > >> > >> mnesia:fold is not the best tool when you are changing a lot of > >> records, it will have to keep > >> every change in memory until you have traversed the whole table. And > >> it is slow with lot > >> of the changes, since have to compensate for the things you have done > >> earlier in the transaction. > >> > >> I assume your are using dets (disc_only) since you are afraid of the > >> 2G limit, or is it > >> memory limit on windows? > >> > >> dets is slow, mnesia is primarly a ram database. > >> > >> The only way I see it is to chunk though the tables a couple 100~1000 > >> records > >> per transaction or something. > >> And have code that can deal with both the new and old format during the > >> changing > >> of the database. > >> > >> Good luck > >> /Dan > >> > >> On Fri, Apr 2, 2010 at 10:19 PM, Brian Acton > wrote: > >> > On this particular table, I do not want to delete all entries. This is > >> why I > >> > posted a separate post to the mailing list. Combining the two threads > >> back, > >> > I want: > >> > > >> > One table, I want to delete entries > n days. > >> > Another table, I want to delete all entries. > >> > > >> > Both tables are reasonably hot (~1-2 ops per second) and reasonably > large > >> (> > >> > 1.5GB). I'm hitting the 2GB limit and I need to clean up these tables. > >> > > >> > So far, any attempts at maintenance (as outlined in previous emails) > have > >> > resulted in Mnesia seizing up and bringing down the cluster. > >> > > >> > It sounds like I have to do this in very small increments with wait > time > >> > between increments. However, I do not have a method and mechanism for > >> > determining the size of an increment or a wait time between > increments. > >> I'm > >> > fine doing ten deletes per 1 second if that's what it takes. However, > I'd > >> > like to be able to figure out the maximum number of deletes that I can > do > >> in > >> > the minimum amount of time. > >> > > >> > I'm definitely open to suggestion on this. > >> > > >> > --b > >> > > >> > On Fri, Apr 2, 2010 at 1:05 PM, Dan Gudmundsson > wrote: > >> > > >> >> clear_table is the fastest way you can delete it, but it will take a > >> >> while when there is a lot of data. > >> >> > >> >> /Dan > >> >> > >> >> On Fri, Apr 2, 2010 at 8:22 PM, Brian Acton > wrote: > >> >> > I'm sorry. I neglected to tell you what I had done on the previous > >> day. > >> >> > > >> >> > On the previous day, I had attempted to delete some old records > using > >> >> this > >> >> > methodology: > >> >> > > >> >> > mnesia:write_lock_table(offline_msg), > >> >> > mnesia:foldl( > >> >> > fun(Rec, _Acc) -> > >> >> > case Rec#offline_msg.expire of > >> >> > never -> > >> >> > ok; > >> >> > TS -> > >> >> > if > >> >> > TS < TimeStamp -> > >> >> > mnesia:delete_object(Rec); > >> >> > true -> > >> >> > ok > >> >> > end > >> >> > end > >> >> > end, ok, offline_msg) > >> >> > > >> >> > > >> >> > This delete finished on the 1st node but subsequently locked up all > >> the > >> >> > other nodes on a table lock. The cluster blew up and my 24/7 > service > >> went > >> >> > into 1 hr of recovery of downtime. > >> >> > > >> >> > So to recap, > >> >> > > >> >> > on day 1 - transaction start, table lock, delete objects - finished > in > >> >> about > >> >> > 2 minutes > >> >> > on day 2 - dirty select, dirty delete objects - finished in about 2 > >> >> minutes > >> >> > > >> >> > In both cases, the cluster blew up and became unusable for at least > >> 20-30 > >> >> > minutes. After 20-30 minutes, we initiated recovery protocols. > >> >> > > >> >> > Should I try > >> >> > > >> >> > day 3 - transaction start, no table lock, delete objects > >> >> > > >> >> > ? is the table lock too coarse grained ? considering that the > cluster > >> has > >> >> > blown up twice, i'm obviously a little scared to try another > >> >> variation.... > >> >> > > >> >> > --b > >> >> > > >> >> > > >> >> > On Fri, Apr 2, 2010 at 5:47 AM, Ovidiu Deac > >> >> wrote: > >> >> > > >> >> >> To me it sounds like another example of premature optimization > which > >> >> >> went wrong? :) > >> >> >> > >> >> >> On Fri, Apr 2, 2010 at 10:19 AM, Dan Gudmundsson > > >> >> wrote: > >> >> >> > When you are using dirty, every operation is sent separately to > all > >> >> >> nodes, > >> >> >> > i.e. 192593*6 messages, actually a transaction could have been > >> faster > >> >> >> > in this case. > >> >> >> > With one message (large) containing all ops to each node. > >> >> >> > > >> >> >> > What you get is an overloaded mnesia_tm (very long msg queues), > >> >> >> > which do the actual writing of the data on the other > (participating > >> >> >> > mnesia nodes). > >> >> >> > > >> >> >> > So transactions will be blocked waiting on mnesia_tm to process > >> those > >> >> >> 200000 > >> >> >> > messages on the other nodes. > >> >> >> > > >> >> >> > /Dan > >> >> >> > > >> >> >> > On Fri, Apr 2, 2010 at 1:11 AM, Brian Acton > > >> >> wrote: > >> >> >> >> Hi guys, > >> >> >> >> > >> >> >> >> I am running R13B04 SMP on FreeBSD 7.3. I have a cluster of 7 > >> nodes > >> >> >> running > >> >> >> >> mnesia. > >> >> >> >> > >> >> >> >> I have a table of 1196143 records using about 1.504GB of > storage. > >> >> It's a > >> >> >> >> reasonably hot table doing a fair number of insert operations > at > >> any > >> >> >> given > >> >> >> >> time. > >> >> >> >> > >> >> >> >> I decided that since there was a 2GB limit in mnesia that I > should > >> do > >> >> >> some > >> >> >> >> cleanup on the system and specifically this table. > >> >> >> >> > >> >> >> >> Trying to avoid major problems with Mnesia, transaction load, > and > >> >> >> deadlock, > >> >> >> >> I decided to do dirty_select and dirty_delete_object > individually > >> on > >> >> the > >> >> >> >> records. > >> >> >> >> > >> >> >> >> I started slow, deleting first 10, then 100, then 1000, then > >> 10000, > >> >> then > >> >> >> >> 100,000 records. My goal was to delete 192593 records total. > >> >> >> >> > >> >> >> >> The first five deletions went through nicely and caused minimal > to > >> no > >> >> >> >> impact. > >> >> >> >> > >> >> >> >> Unfortunately, the very last delete blew up the system. My > delete > >> >> >> command > >> >> >> >> completed successfully but on the other nodes, it caused mnesia > to > >> >> get > >> >> >> stuck > >> >> >> >> on pending transactions, caused my message queues to fill up > and > >> >> >> basically > >> >> >> >> brought down the whole system. We saw some mnesia is overloaded > >> >> messages > >> >> >> in > >> >> >> >> our logs on these nodes but did not see a ton of them. > >> >> >> >> > >> >> >> >> Does anyone have any clues on what went wrong? I am attaching > my > >> code > >> >> >> below > >> >> >> >> for your review. > >> >> >> >> > >> >> >> >> --b > >> >> >> >> > >> >> >> >> Mnesia configuration tunables: > >> >> >> >> > >> >> >> >> -mnesia no_table_loaders 20 > >> >> >> >> -mnesia dc_dump_limit 40 > >> >> >> >> -mnesia dump_log_write_threshold 10000 > >> >> >> >> > >> >> >> >> Example error message: > >> >> >> >> > >> >> >> >> ** WARNING ** Mnesia is overloaded: {mnesia_tm, > message_queue_len, > >> >> >> >> [387,842]} > >> >> >> >> > >> >> >> >> Sample code: > >> >> >> >> > >> >> >> >> Select = fun(Days) -> > >> >> >> >> {MegaSecs, Secs, _MicroSecs} = now(), > >> >> >> >> T = MegaSecs * 1000000 + Secs - 86400 * Days, > >> >> >> >> TimeStamp = {T div 1000000, T rem 1000000, 0}, > >> >> >> >> mnesia:dirty_select(offline_msg, > >> >> >> >> [{'$1', > >> >> >> >> [{'<', {element, 3, '$1'}, > >> >> >> >> {TimeStamp} }], > >> >> >> >> ['$1']}]) > >> >> >> >> end. > >> >> >> >> > >> >> >> >> Count = fun(Days) -> length(Select(Days)) end. > >> >> >> >> > >> >> >> >> Delete = fun(Days, Total) -> > >> >> >> >> C = Select(Days), > >> >> >> >> D = lists:sublist(C, Total), > >> >> >> >> lists:foreach(fun(Rec) -> > >> >> >> >> ok = mnesia:dirty_delete_object(Rec) > >> >> >> >> end, > >> >> >> >> D), > >> >> >> >> length(D) > >> >> >> >> end. > >> >> >> >> > >> >> >> > > >> >> >> > ________________________________________________________________ > >> >> >> > erlang-questions (at) erlang.org mailing list. > >> >> >> > See http://www.erlang.org/faq.html > >> >> >> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> >> >> > > >> >> >> > > >> >> >> > >> >> > > >> >> > >> > > >> > > > From max.lapshin@REDACTED Sat Apr 3 08:57:49 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 3 Apr 2010 10:57:49 +0400 Subject: mmap file to binary Message-ID: I want to add mmap reading of files, because it have shown better results on server, written in C, than plain reading. There was discussion in 2007 http://www.erlang.org/pipermail/erlang-questions/2007-September/029560.html and some blog posts: http://steve.vinoski.net/blog/2007/09/29/more-file-processing-with-erlang/ In the latter, author say: > because of mmap?s alignment restrictions combined with the way the Erlang runtime allocate binaries, I was forced to copy the data into the binary, Have anything changed? It seems to be possible to create binary from existing area in memory, which is controlled by C code, not by erlang. I'm ready to test with linked-in driver, but maybe, there are some other ideas or implementations? From mikpe@REDACTED Sat Apr 3 14:23:28 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Sat, 3 Apr 2010 14:23:28 +0200 Subject: [erlang-questions] mmap file to binary In-Reply-To: References: Message-ID: <19383.13120.860173.157003@pilspetsen.it.uu.se> On Sat, 3 Apr 2010 10:57:49 +0400, Max Lapshin wrote: > I want to add mmap reading of files, because it have shown better > results on server, written in C, than plain reading. > > There was discussion in 2007 > http://www.erlang.org/pipermail/erlang-questions/2007-September/029560.html > and some blog posts: > http://steve.vinoski.net/blog/2007/09/29/more-file-processing-with-erlang/ > > In the latter, author say: > > because of mmap's alignment restrictions combined with the way the > > Erlang runtime allocate binaries, I was forced to copy the data into the > > binary, > > Have anything changed? > > It seems to be possible to create binary from existing area in memory, > which is controlled by C code, not by erlang. > I'm ready to test with linked-in driver, but maybe, there are some > other ideas or implementations? It looks to me like an off-heap ProcBin should work. They separate the object header (required by Erlang) from the actual data bytes, so the data bytes should be able to reside in an mmap:ed blob. The runtime system may need to be tweaked to munmap() the data rather than free() it when the binary dies, but that's a minor issue. From max.lapshin@REDACTED Sat Apr 3 14:26:55 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 3 Apr 2010 16:26:55 +0400 Subject: [erlang-questions] mmap file to binary In-Reply-To: <19383.13120.860173.157003@pilspetsen.it.uu.se> References: <19383.13120.860173.157003@pilspetsen.it.uu.se> Message-ID: On Sat, Apr 3, 2010 at 4:23 PM, Mikael Pettersson wrote: > On Sat, 3 Apr 2010 10:57:49 +0400, Max Lapshin wrote: > > It looks to me like an off-heap ProcBin should work. They separate > the object header (required by Erlang) from the actual data bytes, > so the data bytes should be able to reside in an mmap:ed blob. > > The runtime system may need to be tweaked to munmap() the data rather > than free() it when the binary dies, but that's a minor issue. > So, I need to make patch to erlang vm? {ok, File} = file:open() {ok, Bin} = file:mmap(File, ...) From ulf.wiger@REDACTED Sat Apr 3 16:34:19 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 03 Apr 2010 16:34:19 +0200 Subject: [erlang-questions] Complex gproc queries In-Reply-To: <546145.41042.qm@web112611.mail.gq1.yahoo.com> References: <566312.80019.qm@web112609.mail.gq1.yahoo.com> <4BB61D45.9090601@erlang-solutions.com> <974379.86275.qm@web112601.mail.gq1.yahoo.com> <4BB67534.5040005@erlang-solutions.com> <107850.13694.qm@web112601.mail.gq1.yahoo.com> <4BB73361.3050801@erlang-solutions.com> <546145.41042.qm@web112611.mail.gq1.yahoo.com> Message-ID: <4BB751EB.7080601@erlang-solutions.com> Pablo Platt wrote: > Thank you very much for the detailed answer. > gproc is super powerful with QLC. Thanks. Yes, I think so too. Now that you seem satisfied with the answer, I copy the list again. I didn't want to litter it with my attempts at understanding what you were really after. BR, Ulf W > > ------------------------------------------------------------------------ > *From:* Ulf Wiger > *To:* Pablo Platt > *Sent:* Sat, April 3, 2010 3:24:01 PM > *Subject:* Re: [erlang-questions] Complex gproc queries > > Pablo Platt wrote: > > I think that this assumes that users in the roster are online. > > > Wouldn't this just be [gproc:send({p,l,Id}, Msg) || Id <- Roster] then > > If some users are not online and the roster is big I'll need to make > a lot of queries every time. > > Ok, so let's look at QLC. In this first example, I still assume that > you have a list of Ids to check in, so it's pretty much doing the > same thing as the snippet above: > > 2> spawn(fun() -> gproc:reg({p,l,foo}), receive die -> die end end). > <0.41.0> > 3> spawn(fun() -> gproc:reg({p,l,foo}), receive die -> die end end). > <0.43.0> > 4> spawn(fun() -> gproc:reg({p,l,foo}), receive die -> die end end). > <0.45.0> > 5> spawn(fun() -> gproc:reg({p,l,foo}), receive die -> die end end). > <0.47.0> > 6> spawn(fun() -> gproc:reg({p,l,bar}), receive die -> die end end). > <0.49.0> > 7> spawn(fun() -> gproc:reg({p,l,bar}), receive die -> die end end). > <0.51.0> > 8> spawn(fun() -> gproc:reg({p,l,bar}), receive die -> die end end). > <0.53.0> > 9> Q1 = qlc:q([P || {{p,l,X},P,_} <- gproc:table(props), > lists:member(X, [foo])])). > {qlc_handle,{qlc_lc,#Fun, > {qlc_opt,false,false,-1,any,[],any,524288,allowed}}} > 10> qlc:e(Q1). > [<0.41.0>,<0.43.0>,<0.45.0>,<0.47.0>] > > Now, let's assume you have a mnesia table that makes up the roster. > I don't know the structure of this table, but one possible way to > represent it would be to have it as an ordered_set disc_copy, > and then use the key {Roster, UserId}. > > 11> mnesia:create_schema([node()]). > ok > 12> mnesia:start(). > ok > 13> mnesia:create_table(roster,[{ram_copies,[node()]},{type,ordered_set}]). > {atomic,ok} > 14> mnesia:dirty_write({roster,{r1,foo},[]}). > ok > 15> catch qlc:e(qlc:q([P || {{p,l,X},P,_} <- gproc:table(props), > ets:member(roster,{r1,X})])). > [<0.41.0>,<0.43.0>,<0.45.0>,<0.47.0>] > > I used ets:member/2 here, which would only work if executed on a > node that actually has a copy in RAM of the roster table, and > it doesn't respect any mnesia-level locks. You could write your > own function, e.g. is_member(R, UId) that does a mnesia lookup, > and then run the query inside a transaction... > > You can make this into a more complex QLC query, but I'm not > certain what the performance would be: > > 16> rd(roster,{key,value}). % define the record in the shell > roster > 17> Q2 = qlc:q([P || > {{p,l,X},P,_} <- gproc:table(props), > #roster{key={r1,X1}} <- mnesia:table(roster), > X==X1]). > {qlc_handle,{qlc_lc,#Fun, > {qlc_opt,false,false,-1,any,[],any,524288,allowed}}} > 18> mnesia:activity(transaction, fun() -> qlc:e(Q2) end). > [<0.41.0>,<0.43.0>,<0.45.0>,<0.47.0>] > > The above is rather trivially generalized to work with rosters other > than 'r1', of course. > > Hope this was more helpful. > > BR, > Ulf W > > > What I thought doing is to first query for a list of users that are > both in the roster and online > > and only then send them a message. > > > > ------------------------------------------------------------------------ > > *From:* Ulf Wiger > > > *To:* Pablo Platt > > > *Sent:* Sat, April 3, 2010 1:52:36 AM > > *Subject:* Re: [erlang-questions] Complex gproc queries > > > > > > Ah, I thought you used gproc to keep up with the roster > > as well. :) > > > > Wouldn't this just be [gproc:send({p,l,Id}, Msg) || Id <- Roster] then? > > > > Or am I missing something again? If so, you may respond to me directly. > > > > BTW, I have merged minor changes from Hans Svensson and fixed > > the EQC suite. New commit, in other words. > > > > BR, > > Ulf W > > > > Pablo Platt wrote: > > > I should have said that Roster is a list with the contact list IDs > > > Roster = [id1, id2, id3...] > > > each user have a different list. > > > I need to send a message to all the the users in the list that are > online. > > > > > > Thanks > > > > > > > > > > ------------------------------------------------------------------------ > > > *From:* Ulf Wiger > >> > > > *To:* Pablo Platt >> > > > *Cc:* erlang-questions@REDACTED > > > > > *Sent:* Fri, April 2, 2010 7:37:25 PM > > > *Subject:* Re: [erlang-questions] Complex gproc queries > > > > > > Pablo Platt wrote: > > > > Hi, > > > > > > > > I'm building a simple chat server and using gproc to keep track > of online users and the process for each user session. > > > > > > > > Basically, each online user has a process that register a > unique session id and non unique user id (same user can have several > clients): > > > > gproc:add_local_name(SessionId) > > > > gproc:add_local_property(UserId, undefined) > > > > > > > > When a user change his presence(online/busy/offline) I need to > inform all the users on his roster. > > > > After I retrieved the roster from the db, how can I select all > the processes with UserId that is a part of the roster? > > > > Rusty Klupaus's blog about gproc note that gproc:select support > QLC but I don't understand how to do it. > > >> > >> >> > http://rklophaus.com/blog/2009/9/16/gproc-erlang-global-process-registry.html > > > > > > > > Thanks > > > > > > Not sure if you need any complex queries for that. > > > If all members of a roster have a property {p,l,Roster}, > > > then all it should take is: > > > > > > gproc:send({p,l,Roster}, Msg) > > > > > > If you want to e.g. exclude the sender, you could use: > > > > > > [P ! Msg || P <- gproc:lookup_pids({p,l,Roster}) -- [self()]] > > > > > > Disclaimer: When verifying this, I found that gproc:send/2 was > > > broken for properties. I fixed it in the last commit, 1 minute ago. > > > > > > The commit also changes anther bug: aggregated counters had to > > > have an integer value when created. This doesn't make sense, since > > > the value is derived; therefore the only sensible value is > 'undefined', > > > which is what you get if you call gproc:reg({a,l,AggrCounter}). > > > > > > Unfortunately, the commit broke the QuickCheck suite... I had forgot > > > that I started writing a test for gproc:await/1 at the SF Erlang > > > factory, didn't finish it. Oh well, if anyone is really hurt by > > > this condition, let me know. (Hans, I haven't merged your changes > > > yet). > > > > > > BR, > > > Ulf W > > > -- Ulf Wiger > > > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd >> > http://www.erlang-solutions.com > > > --------------------------------------------------- > > > > > > --------------------------------------------------- > > > > > > WE'VE CHANGED NAMES! > > > > > > Since January 1st 2010 Erlang Training and Consulting Ltd. has > become ERLANG SOLUTIONS LTD. > > > > > > www.erlang-solutions.com > > > > > > > > > > > > > -- Ulf Wiger > > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > > http://www.erlang-solutions.com > > > > > -- Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > http://www.erlang-solutions.com > -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From karol.skocik@REDACTED Sat Apr 3 20:17:20 2010 From: karol.skocik@REDACTED (karol skocik) Date: Sat, 3 Apr 2010 20:17:20 +0200 Subject: initializing library applications without processes Message-ID: Hi, sometimes, I just need a library "application" without processes, which works with some state in ETS. To function properly, they have to make the ETS table and populate it with values from config files. When that step is done, there is nothing left to do for processes, they would just serve as a useless proxy for querying the ETS table, and that's not needed when you can (in some contexts) access the table directly. I like the mechanism of applications and their dependency solving, and I would ideally do application:start(my_app_with_ets_initialised) but that won't do, since {ok, SupPid} is expected as a result. But I don't want to start any processes, I just want to ensure that application_master:init/4 calls my "start" function, and declare application as "running" when 'ok' is returned or fail otherwise (unless it's {ok, SupPid} - as it is now). Currently, when we have library application, application:start(...) does not call anything since 'mod' is not specified in .app file. Could it be changed, and application behaviour would just call start/2 and stop/1 as usual, but start the SupPid optionally when it's supplied? Or - alternatively - is there any way how to ensure that some init function is done fitting nicely with application dependencies as declared in .rel file? Thanks, Karol From bob@REDACTED Sat Apr 3 20:34:59 2010 From: bob@REDACTED (Bob Ippolito) Date: Sat, 3 Apr 2010 11:34:59 -0700 Subject: [erlang-questions] initializing library applications without processes In-Reply-To: References: Message-ID: You do need some process to own the ets tables, just don't send messages to it after it's up. An alternative is something like mochiglobal (in the mochiweb project) which can be faster than ets for this kind of use case (only important if you're reading from it often and/or the data is large). It's not pretty because it's an abuse of the module system and code server... but until something like it is officially in the VM it's worthy of consideration ;) On Sat, Apr 3, 2010 at 11:17 AM, karol skocik wrote: > Hi, > ?sometimes, I just need a library "application" without processes, > which works with some state in ETS. To function properly, > they have to make the ETS table and populate it with values > from config files. When that step is done, there is nothing left to do > for processes, they would just serve as a useless proxy for > querying the ETS table, and that's not needed when you can (in some contexts) > access the table directly. > > I like the mechanism of applications and their dependency solving, > and I would ideally do application:start(my_app_with_ets_initialised) > but that won't do, since {ok, SupPid} is expected as a result. > > But I don't want to start any processes, I just want to ensure that > application_master:init/4 calls my "start" function, > and declare application as "running" when 'ok' is returned > or fail otherwise (unless it's {ok, SupPid} - as it is now). > > Currently, when we have library application, application:start(...) > does not call anything since 'mod' is not specified in .app file. > Could it be changed, and application behaviour would just call > start/2 and stop/1 as usual, but start the SupPid optionally when it's supplied? > Or - alternatively - is there any way how to ensure that some init > function is done > fitting nicely with application dependencies as declared in .rel file? > > Thanks, > ?Karol > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ulf.wiger@REDACTED Sat Apr 3 20:41:37 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 03 Apr 2010 20:41:37 +0200 Subject: [erlang-questions] initializing library applications without processes In-Reply-To: References: Message-ID: <4BB78BE1.5050905@erlang-solutions.com> I received exactly this question while at the SF Erlang Factory. My suggestion is: start a supervisor with no children. A reasonable "cheat" is to put all of this in the app file -module(myapp). -behaviour(application). -export([start/2, stop/1]). -behaviour(supervisor). -export([init/1]). start(_, _) -> supervisor:start_link({local,?MODULE},?MODULE,[]). stop(_) -> ok. init([]) -> %% create tables here... {ok, {{one_for_one,3,10},[]}}. Baking the supervisor into the application callback module is in a sense against the OTP design rules, but the most notable example of this is surely OTP's 'kernel' application, so you won't be tarred and feathered for doing so. :) BR, Ulf W karol skocik wrote: > Hi, > sometimes, I just need a library "application" without processes, > which works with some state in ETS. To function properly, > they have to make the ETS table and populate it with values > from config files. When that step is done, there is nothing left to do > for processes, they would just serve as a useless proxy for > querying the ETS table, and that's not needed when you can (in some contexts) > access the table directly. > > I like the mechanism of applications and their dependency solving, > and I would ideally do application:start(my_app_with_ets_initialised) > but that won't do, since {ok, SupPid} is expected as a result. > > But I don't want to start any processes, I just want to ensure that > application_master:init/4 calls my "start" function, > and declare application as "running" when 'ok' is returned > or fail otherwise (unless it's {ok, SupPid} - as it is now). > > Currently, when we have library application, application:start(...) > does not call anything since 'mod' is not specified in .app file. > Could it be changed, and application behaviour would just call > start/2 and stop/1 as usual, but start the SupPid optionally when it's supplied? > Or - alternatively - is there any way how to ensure that some init > function is done > fitting nicely with application dependencies as declared in .rel file? > > Thanks, > Karol > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From karol.skocik@REDACTED Sat Apr 3 20:53:33 2010 From: karol.skocik@REDACTED (karol skocik) Date: Sat, 3 Apr 2010 20:53:33 +0200 Subject: [erlang-questions] initializing library applications without processes In-Reply-To: <4BB78BE1.5050905@erlang-solutions.com> References: <4BB78BE1.5050905@erlang-solutions.com> Message-ID: "Hijacking" supervisor this way is pretty nice idea :) Will do for my case. Thank you guys for suggestions. Karol On Sat, Apr 3, 2010 at 8:41 PM, Ulf Wiger wrote: > > I received exactly this question while at the SF Erlang Factory. > > My suggestion is: start a supervisor with no children. A reasonable > "cheat" is to put all of this in the app file > > -module(myapp). > -behaviour(application). > -export([start/2, stop/1]). > -behaviour(supervisor). > -export([init/1]). > > start(_, _) -> > ? supervisor:start_link({local,?MODULE},?MODULE,[]). > > stop(_) -> ok. > > init([]) -> > ? %% create tables here... > ? {ok, {{one_for_one,3,10},[]}}. > > Baking the supervisor into the application callback module is > in a sense against the OTP design rules, but the most notable > example of this is surely OTP's 'kernel' application, so you > won't be tarred and feathered for doing so. :) > > BR, > Ulf W > > karol skocik wrote: >> >> Hi, >> ?sometimes, I just need a library "application" without processes, >> which works with some state in ETS. To function properly, >> they have to make the ETS table and populate it with values >> from config files. When that step is done, there is nothing left to do >> for processes, they would just serve as a useless proxy for >> querying the ETS table, and that's not needed when you can (in some >> contexts) >> access the table directly. >> >> I like the mechanism of applications and their dependency solving, >> and I would ideally do application:start(my_app_with_ets_initialised) >> but that won't do, since {ok, SupPid} is expected as a result. >> >> But I don't want to start any processes, I just want to ensure that >> application_master:init/4 calls my "start" function, >> and declare application as "running" when 'ok' is returned >> or fail otherwise (unless it's {ok, SupPid} - as it is now). >> >> Currently, when we have library application, application:start(...) >> does not call anything since 'mod' is not specified in .app file. >> Could it be changed, and application behaviour would just call >> start/2 and stop/1 as usual, but start the SupPid optionally when it's >> supplied? >> Or - alternatively - is there any way how to ensure that some init >> function is done >> fitting nicely with application dependencies as declared in .rel file? >> >> Thanks, >> ?Karol >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > -- > Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > http://www.erlang-solutions.com > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG > SOLUTIONS LTD. > > www.erlang-solutions.com > > From ulf.wiger@REDACTED Sat Apr 3 22:16:28 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 03 Apr 2010 22:16:28 +0200 Subject: [erlang-questions] initializing library applications without processes In-Reply-To: References: <4BB78BE1.5050905@erlang-solutions.com> Message-ID: <4BB7A21C.5070808@erlang-solutions.com> Just beware that my example is mailing-list-quality code. I've not tried to compile or run it. It is only "obviously correct". BR, Ulf W karol skocik wrote: > "Hijacking" supervisor this way is pretty nice idea :) > Will do for my case. > > Thank you guys for suggestions. > Karol > > On Sat, Apr 3, 2010 at 8:41 PM, Ulf Wiger > wrote: >> I received exactly this question while at the SF Erlang Factory. >> >> My suggestion is: start a supervisor with no children. A reasonable >> "cheat" is to put all of this in the app file >> >> -module(myapp). >> -behaviour(application). >> -export([start/2, stop/1]). >> -behaviour(supervisor). >> -export([init/1]). >> >> start(_, _) -> >> supervisor:start_link({local,?MODULE},?MODULE,[]). >> >> stop(_) -> ok. >> >> init([]) -> >> %% create tables here... >> {ok, {{one_for_one,3,10},[]}}. >> >> Baking the supervisor into the application callback module is >> in a sense against the OTP design rules, but the most notable >> example of this is surely OTP's 'kernel' application, so you >> won't be tarred and feathered for doing so. :) >> >> BR, >> Ulf W >> >> karol skocik wrote: >>> Hi, >>> sometimes, I just need a library "application" without processes, >>> which works with some state in ETS. To function properly, >>> they have to make the ETS table and populate it with values >>> from config files. When that step is done, there is nothing left to do >>> for processes, they would just serve as a useless proxy for >>> querying the ETS table, and that's not needed when you can (in some >>> contexts) >>> access the table directly. >>> >>> I like the mechanism of applications and their dependency solving, >>> and I would ideally do application:start(my_app_with_ets_initialised) >>> but that won't do, since {ok, SupPid} is expected as a result. >>> >>> But I don't want to start any processes, I just want to ensure that >>> application_master:init/4 calls my "start" function, >>> and declare application as "running" when 'ok' is returned >>> or fail otherwise (unless it's {ok, SupPid} - as it is now). >>> >>> Currently, when we have library application, application:start(...) >>> does not call anything since 'mod' is not specified in .app file. >>> Could it be changed, and application behaviour would just call >>> start/2 and stop/1 as usual, but start the SupPid optionally when it's >>> supplied? >>> Or - alternatively - is there any way how to ensure that some init >>> function is done >>> fitting nicely with application dependencies as declared in .rel file? >>> >>> Thanks, >>> Karol >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> -- >> Ulf Wiger >> CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd >> http://www.erlang-solutions.com >> --------------------------------------------------- >> >> --------------------------------------------------- >> >> WE'VE CHANGED NAMES! >> >> Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG >> SOLUTIONS LTD. >> >> www.erlang-solutions.com >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From karol.skocik@REDACTED Sat Apr 3 22:23:47 2010 From: karol.skocik@REDACTED (karol skocik) Date: Sat, 3 Apr 2010 22:23:47 +0200 Subject: [erlang-questions] initializing library applications without processes In-Reply-To: <4BB7A21C.5070808@erlang-solutions.com> References: <4BB78BE1.5050905@erlang-solutions.com> <4BB7A21C.5070808@erlang-solutions.com> Message-ID: No worries, my lib/app already worked that way 5 minutes after your suggestion :) Thanks, Karol On Sat, Apr 3, 2010 at 10:16 PM, Ulf Wiger wrote: > > Just beware that my example is mailing-list-quality code. > I've not tried to compile or run it. It is only "obviously > correct". > > BR, > Ulf W > > karol skocik wrote: >> >> "Hijacking" supervisor this way is pretty nice idea :) >> Will do for my case. >> >> Thank you guys for suggestions. >> Karol >> >> On Sat, Apr 3, 2010 at 8:41 PM, Ulf Wiger >> wrote: >>> >>> I received exactly this question while at the SF Erlang Factory. >>> >>> My suggestion is: start a supervisor with no children. A reasonable >>> "cheat" is to put all of this in the app file >>> >>> -module(myapp). >>> -behaviour(application). >>> -export([start/2, stop/1]). >>> -behaviour(supervisor). >>> -export([init/1]). >>> >>> start(_, _) -> >>> ?supervisor:start_link({local,?MODULE},?MODULE,[]). >>> >>> stop(_) -> ok. >>> >>> init([]) -> >>> ?%% create tables here... >>> ?{ok, {{one_for_one,3,10},[]}}. >>> >>> Baking the supervisor into the application callback module is >>> in a sense against the OTP design rules, but the most notable >>> example of this is surely OTP's 'kernel' application, so you >>> won't be tarred and feathered for doing so. :) >>> >>> BR, >>> Ulf W >>> >>> karol skocik wrote: >>>> >>>> Hi, >>>> ?sometimes, I just need a library "application" without processes, >>>> which works with some state in ETS. To function properly, >>>> they have to make the ETS table and populate it with values >>>> from config files. When that step is done, there is nothing left to do >>>> for processes, they would just serve as a useless proxy for >>>> querying the ETS table, and that's not needed when you can (in some >>>> contexts) >>>> access the table directly. >>>> >>>> I like the mechanism of applications and their dependency solving, >>>> and I would ideally do application:start(my_app_with_ets_initialised) >>>> but that won't do, since {ok, SupPid} is expected as a result. >>>> >>>> But I don't want to start any processes, I just want to ensure that >>>> application_master:init/4 calls my "start" function, >>>> and declare application as "running" when 'ok' is returned >>>> or fail otherwise (unless it's {ok, SupPid} - as it is now). >>>> >>>> Currently, when we have library application, application:start(...) >>>> does not call anything since 'mod' is not specified in .app file. >>>> Could it be changed, and application behaviour would just call >>>> start/2 and stop/1 as usual, but start the SupPid optionally when it's >>>> supplied? >>>> Or - alternatively - is there any way how to ensure that some init >>>> function is done >>>> fitting nicely with application dependencies as declared in .rel file? >>>> >>>> Thanks, >>>> ?Karol >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>> >>> -- >>> Ulf Wiger >>> CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd >>> http://www.erlang-solutions.com >>> --------------------------------------------------- >>> >>> --------------------------------------------------- >>> >>> WE'VE CHANGED NAMES! >>> >>> Since January 1st 2010 Erlang Training and Consulting Ltd. has become >>> ERLANG >>> SOLUTIONS LTD. >>> >>> www.erlang-solutions.com >>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > -- > Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > http://www.erlang-solutions.com > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG > SOLUTIONS LTD. > > www.erlang-solutions.com > > From g@REDACTED Sun Apr 4 01:58:53 2010 From: g@REDACTED (Garrett Smith) Date: Sat, 3 Apr 2010 17:58:53 -0600 Subject: [erlang-questions] initializing library applications without processes In-Reply-To: <4BB78BE1.5050905@erlang-solutions.com> References: <4BB78BE1.5050905@erlang-solutions.com> Message-ID: On Sat, Apr 3, 2010 at 12:41 PM, Ulf Wiger wrote: > > I received exactly this question while at the SF Erlang Factory. > > My suggestion is: start a supervisor with no children. A reasonable > "cheat" is to put all of this in the app file > > -module(myapp). > -behaviour(application). > -export([start/2, stop/1]). > -behaviour(supervisor). > -export([init/1]). > > start(_, _) -> > ? supervisor:start_link({local,?MODULE},?MODULE,[]). > > stop(_) -> ok. > > init([]) -> > ? %% create tables here... > ? {ok, {{one_for_one,3,10},[]}}. The docs say you can return 'ignore' here as well. I've never tried that, but it might give this hack even more panache, assuming it doesn't implicitly terminate the process :) Garrett From sgardell@REDACTED Sun Apr 4 02:41:09 2010 From: sgardell@REDACTED (Gardell, Steven) Date: Sat, 3 Apr 2010 20:41:09 -0400 Subject: Multiple versions of Megaco via the Megaco lib? Message-ID: <6B3E29721F78364AA2C43697AFB15D91021526CC@sonusmail07.sonusnet.com> I am trying to build an application that works at V3. Like, I imagine, many others I am starting with simple_mgc. My first confusion is how to properly deal with the mix of V1 (service change) and V3 (everything else) messages. The Megaco library defines several sets of hrl files for different versions. What is the recommended way to structure things so I am using the V1 record definitions when I need to and the V3 otherwise? The following works (when referencing megaco_message_v3.hrl), but it seems evil: case ProtocolVersion of 1 -> Address=element(3,SCP), Version=element(4,SCP), Profile=element(5,SCP); 3 -> #'ServiceChangeParm'{serviceChangeAddress = Address, serviceChangeVersion = Version, serviceChangeProfile = Profile} = SCP end, (I could simply just use the items in the "1" case since they are the same from V1 to V3) I tried various approaches to directly referencing the V1 record definition to no avail. Secondly, I am seeing complaints from megaco_filter which sorta suggest that it is not accounting for the megaco version. How do I configure megaco_filter for the version of interest? Thanks Steven Gardell Engineering Manager sgardell@REDACTED t +1 978 614 8831 f +1 978 614 8101 7 Technology Park Drive Westford, MA 01886 USA www.sonusnet.com Deliver the Future First with Sonus Networks. Disclaimer: Content provided is for information purposes only and is subject to change without notice. Sonus has no obligation or commitment to develop or deliver any future release, upgrade, feature, enhancement or function described in this email or any attachment or presentation except as specifically set forth in a written agreement. From navaneethanit@REDACTED Sun Apr 4 06:39:11 2010 From: navaneethanit@REDACTED (Saravana) Date: Sat, 3 Apr 2010 21:39:11 -0700 (PDT) Subject: tsung + erlang Message-ID: <960b4b8d-f003-4290-a79d-aed98c98f29b@r1g2000yqb.googlegroups.com> Hello friends, I need to know that how to use tsung??Actually my erlang web application running on http://localhost:8000,then i should measure the performance of my application,here i installed tsung,then i entered the command "tsung-recorder start",it is started,my objective is i have to measure the performance of my web application i don't know where i have to enter the url of my web application?How will give the input? How to measure it?,How ll i get the text file?Is any easy tool available to test the loading of server?? please help me From kheinz57@REDACTED Sun Apr 4 09:32:56 2010 From: kheinz57@REDACTED (heinz, haeberle) Date: Sun, 04 Apr 2010 09:32:56 +0200 Subject: [erlang-questions] Implementation of RFC-5841 for gen_tcp? In-Reply-To: <91c66845-71eb-4e58-8200-5e9b277866ef@b33g2000yqc.googlegroups.com> References: <91c66845-71eb-4e58-8200-5e9b277866ef@b33g2000yqc.googlegroups.com> Message-ID: <4BB840A8.3090107@gmx.de> Hey, great! I always wanted to build my own Marvin (the Robot in the famous 42 Book). would only use one mood though. Heinz Am 01.04.2010 22:35, schrieb Steve Davis: > I'm wondering when the modifications to TCP, proposed by Google, will > be made in the gen_tcp module? > See: http://tools.ietf.org/html/rfc5841 > /s > ps happy 1st > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ulf.wiger@REDACTED Sun Apr 4 10:45:27 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 04 Apr 2010 10:45:27 +0200 Subject: [erlang-questions] initializing library applications without processes In-Reply-To: References: <4BB78BE1.5050905@erlang-solutions.com> Message-ID: <4BB851A7.8080804@erlang-solutions.com> Garrett Smith wrote: > > The docs say you can return 'ignore' here as well. I've never tried > that, but it might give this hack even more panache, assuming it > doesn't implicitly terminate the process :) > > Garrett Actually, and unfortunately(?), that's exactly what happens. Here's the relevant code from gen_server:init_it/6: init_it(Starter, Parent, Name0, Mod, Args, Options) -> Name = name(Name0), Debug = debug_options(Name, Options), case catch Mod:init(Args) of {ok, State} -> proc_lib:init_ack(Starter, {ok, self()}), loop(Parent, Name, State, Mod, infinity, Debug); ...; ignore -> unregister_name(Name0), proc_lib:init_ack(Starter, ignore), exit(normal); (supervisor.erl is implemented as a gen_server, and simply passes on the 'ignore' return value from the Mod:init/1 function.) Furthermore, in the specific case of application start, the Mod:start/2 function is required to return a pid: (from kernel/src/application_master.erl) start_it_old(Tag, From, Type, ApplData) -> {M,A} = ApplData#appl_data.mod, case catch M:start(Type, A) of {ok, Pid} -> link(Pid), From ! {Tag, {ok, self()}}, loop_it(From, Pid, M, []); {ok, Pid, AppState} -> link(Pid), From ! {Tag, {ok, self()}}, loop_it(From, Pid, M, AppState); {'EXIT', normal} -> From ! {Tag, {error, {{'EXIT',normal},{M,start,[Type,A]}}}}; {error, Reason} -> From ! {Tag, {error, {Reason, {M,start,[Type,A]}}}}; Other -> From ! {Tag, {error, {bad_return,{{M,start,[Type,A]},Other}}}} end. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From essiene@REDACTED Sun Apr 4 12:28:23 2010 From: essiene@REDACTED (Essien Essien) Date: Sun, 4 Apr 2010 11:28:23 +0100 Subject: simple_one_for_one being unjustly supervised? Message-ID: Hi all, Maybe I'm missing something pretty obvious, but I'm noticing that the children of a simple_one_for_one supervisor are being killed off without honoring their "shutdown" interval, when they're taken down by a top-level application. Everything work fine if I sent an exit(SupPid, shutdown) to the top-level supervisor PID though. I have attached a contrived example application called "boss"... The application callback and top-level supervisor behaviours are implemented in the same module. The boss supervisor is a one_for_one supervisor which supervises a "nanny", "manage" and "delay" modules. "nanny" is a simple_one_for_one supervisor which supervises "children" (a gen_server). "manage" is a gen_server which spins up the children by calling nanny:start_child when it starts up. The number of children is read from the boss.app file. "delay" is a gen_server which just delays for a configurable amount of time before shutting down. "children" is a simple gen_server which delays for 5 seconds before shutting down. >From my tests in an erl session, if i do: 1 > {ok, Pid} = boss:start_link([8, 5000]). %these arguments mean, 8 children, 5 seconds delay for the delay gen_server And then: 2 > exit(Pid, shutdown). Everything happens nicely. I see _all_ my logging messages which suggests that things cleaned up nice and sweet. But if I do: 3 > application:start(boss). 4 > application:stop(boss). ONLY the "delay" module cleans up nicely... I never get to see the logs from the "children" instances... which means the tree just kills them off very fast. Is this a known behaviour? Am I missing something critical? Any wisdoms will be much appreciated. Also, I tried peeking at the code for supervisor.erl (went all the way to do_terminate/2 and shutdown/2) and nothing looks amiss, the code for application_controller.erl would take a bit more than a cursory glance to trace through all that is happening, so I wanted to ask here first, before going any further. cheers, Essien -------------- next part -------------- A non-text attachment was scrubbed... Name: boss.tar.gz Type: application/x-gzip Size: 1459 bytes Desc: not available URL: From ulf.wiger@REDACTED Sun Apr 4 12:48:06 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 04 Apr 2010 12:48:06 +0200 Subject: [erlang-questions] simple_one_for_one being unjustly supervised? In-Reply-To: References: Message-ID: <4BB86E66.4010105@erlang-solutions.com> The relevant code in the application controller (since I happened to have it open, as I snipped from it earlier today) is in application_master.erl: loop_it(Parent, Child, Mod, AppState) -> receive {Parent, get_child} -> Parent ! {self(), Child, Mod}, loop_it(Parent, Child, Mod, AppState); {Parent, terminate} -> NewAppState = prep_stop(Mod, AppState), exit(Child, shutdown), receive {'EXIT', Child, _} -> ok end, catch Mod:stop(NewAppState), exit(normal); As you can see, the supervisor is terminated using exit(Child,shutdown). (Child is the pid returned from Mod:start/2). I have not looked at your logs, nor do I have a good explanation for the discrepancy you're observing. BR, Ulf W Essien Essien wrote: > Hi all, > > Maybe I'm missing something pretty obvious, but I'm noticing that the > children of a simple_one_for_one supervisor are being killed off > without honoring their "shutdown" interval, when they're taken down by > a top-level application. Everything work fine if I sent an > exit(SupPid, shutdown) to the top-level supervisor PID though. > > I have attached a contrived example application called "boss"... > > The application callback and top-level supervisor behaviours are > implemented in the same module. > > The boss supervisor is a one_for_one supervisor which supervises a > "nanny", "manage" and "delay" modules. > > "nanny" is a simple_one_for_one supervisor which supervises "children" > (a gen_server). > > "manage" is a gen_server which spins up the children by calling > nanny:start_child when it starts up. The number of children is read > from the boss.app file. > > "delay" is a gen_server which just delays for a configurable amount of > time before shutting down. > > "children" is a simple gen_server which delays for 5 seconds before > shutting down. > > From my tests in an erl session, if i do: > > 1 > {ok, Pid} = boss:start_link([8, 5000]). %these arguments mean, 8 > children, 5 seconds delay for the delay gen_server > > And then: > > 2 > exit(Pid, shutdown). > > Everything happens nicely. I see _all_ my logging messages which > suggests that things cleaned up nice and sweet. > > But if I do: > > 3 > application:start(boss). > 4 > application:stop(boss). > > ONLY the "delay" module cleans up nicely... I never get to see the > logs from the "children" instances... which means the tree just kills > them off very fast. > > > Is this a known behaviour? Am I missing something critical? > > Any wisdoms will be much appreciated. > > Also, I tried peeking at the code for supervisor.erl (went all the way > to do_terminate/2 and shutdown/2) and nothing looks amiss, the code > for application_controller.erl would take a bit more than a cursory > glance to trace through all that is happening, so I wanted to ask here > first, before going any further. > > cheers, > Essien > > > ------------------------------------------------------------------------ > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From steven.charles.davis@REDACTED Sun Apr 4 13:32:26 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 4 Apr 2010 04:32:26 -0700 (PDT) Subject: Complex gproc queries In-Reply-To: <4BB751EB.7080601@erlang-solutions.com> References: <566312.80019.qm@web112609.mail.gq1.yahoo.com> <4BB61D45.9090601@erlang-solutions.com> <974379.86275.qm@web112601.mail.gq1.yahoo.com> <4BB67534.5040005@erlang-solutions.com> <107850.13694.qm@web112601.mail.gq1.yahoo.com> <4BB73361.3050801@erlang-solutions.com> <546145.41042.qm@web112611.mail.gq1.yahoo.com> <4BB751EB.7080601@erlang-solutions.com> Message-ID: <43f484eb-efa9-4d4f-8864-c1d581793103@x12g2000yqx.googlegroups.com> Interesting and useful discussion -- thanks to you both -- Now I'm "on board" with the ideas, the functionality that gproc provides appears stunningly practical and useful... and, because of this, I'm left somewhat bemused as to why it hasn't already been absorbed into the otp platform. Does anyone know/have a comment on this? Is it something to do with a back-compat issue with the gen_* patches it requires? /s On Apr 3, 9:34?am, Ulf Wiger wrote: > Pablo Platt wrote: > > Thank you very much for the detailed answer. > > gproc is super powerful with QLC. > > Thanks. Yes, I think so too. > > Now that you seem satisfied with the answer, I copy the list > again. I didn't want to litter it with my attempts at > understanding what you were really after. > > BR, > Ulf W > > > > > > > > > ------------------------------------------------------------------------ > > *From:* Ulf Wiger > > *To:* Pablo Platt > > *Sent:* Sat, April 3, 2010 3:24:01 PM > > *Subject:* Re: [erlang-questions] Complex gproc queries > > > Pablo Platt wrote: > > ?> I think that this assumes that users in the roster are online. > > ?> ?> Wouldn't this just be [gproc:send({p,l,Id}, Msg) || Id <- Roster] then > > ?> If some users are not online and the roster is big I'll need to make > > a lot of queries every time. > > > Ok, so let's look at QLC. In this first example, I still assume that > > you have a list of Ids to check in, so it's pretty much doing the > > same thing as the snippet above: > > > 2> spawn(fun() -> gproc:reg({p,l,foo}), receive die -> die end end). > > <0.41.0> > > 3> spawn(fun() -> gproc:reg({p,l,foo}), receive die -> die end end). > > <0.43.0> > > 4> spawn(fun() -> gproc:reg({p,l,foo}), receive die -> die end end). > > <0.45.0> > > 5> spawn(fun() -> gproc:reg({p,l,foo}), receive die -> die end end). > > <0.47.0> > > 6> spawn(fun() -> gproc:reg({p,l,bar}), receive die -> die end end). > > <0.49.0> > > 7> spawn(fun() -> gproc:reg({p,l,bar}), receive die -> die end end). > > <0.51.0> > > 8> spawn(fun() -> gproc:reg({p,l,bar}), receive die -> die end end). > > <0.53.0> > > 9> Q1 = qlc:q([P || {{p,l,X},P,_} <- gproc:table(props), > > ? ? ? ? ? ? ? lists:member(X, [foo])])). > > {qlc_handle,{qlc_lc,#Fun, > > ? ? ? ? ? ? ? ? ? ? {qlc_opt,false,false,-1,any,[],any,524288,allowed}}} > > 10> qlc:e(Q1). > > [<0.41.0>,<0.43.0>,<0.45.0>,<0.47.0>] > > > Now, let's assume you have a mnesia table that makes up the roster. > > I don't know the structure of this table, but one possible way to > > represent it would be to have it as an ordered_set disc_copy, > > and then use the key {Roster, UserId}. > > > 11> mnesia:create_schema([node()]). > > ok > > 12> mnesia:start(). > > ok > > 13> mnesia:create_table(roster,[{ram_copies,[node()]},{type,ordered_set}]). > > {atomic,ok} > > 14> mnesia:dirty_write({roster,{r1,foo},[]}). > > ok > > 15> catch qlc:e(qlc:q([P || {{p,l,X},P,_} <- gproc:table(props), > > ets:member(roster,{r1,X})])). > > [<0.41.0>,<0.43.0>,<0.45.0>,<0.47.0>] > > > I used ets:member/2 here, which would only work if executed on a > > node that actually has a copy in RAM of the roster table, and > > it doesn't respect any mnesia-level locks. You could write your > > own function, e.g. is_member(R, UId) that does a mnesia lookup, > > and then run the query inside a transaction... > > > You can make this into a more complex QLC query, but I'm not > > certain what the performance would be: > > > 16> rd(roster,{key,value}). ?% define the record in the shell ? ? ? ? ? ? > > ? ? ? ? ? ? ? ? ? ? ? roster > > 17> Q2 = qlc:q([P || > > ? ? ? ? ? ? {{p,l,X},P,_} <- gproc:table(props), > > ? ? ? ? ? ? #roster{key={r1,X1}} <- mnesia:table(roster), > > ? ? ? ? ? ? X==X1]). > > {qlc_handle,{qlc_lc,#Fun, > > ? ? ? ? ? ? ? ? ? ? {qlc_opt,false,false,-1,any,[],any,524288,allowed}}} > > 18> mnesia:activity(transaction, fun() -> qlc:e(Q2) end). > > [<0.41.0>,<0.43.0>,<0.45.0>,<0.47.0>] > > > The above is rather trivially generalized to work with rosters other > > than 'r1', of course. > > > Hope this was more helpful. > > > BR, > > Ulf W > > > ?> What I thought doing is to first query for a list of users that are > > both in the roster and online > > ?> and only then send them a message. > > > ?> ------------------------------------------------------------------------ > > ?> *From:* Ulf Wiger > > > > ?> *To:* Pablo Platt > > > ?> *Sent:* Sat, April 3, 2010 1:52:36 AM > > ?> *Subject:* Re: [erlang-questions] Complex gproc queries > > > ?> Ah, I thought you used gproc to keep up with the roster > > ?> as well. :) > > > ?> Wouldn't this just be [gproc:send({p,l,Id}, Msg) || Id <- Roster] then? > > > ?> Or am I missing something again? If so, you may respond to me directly. > > > ?> BTW, I have merged minor changes from Hans Svensson and fixed > > ?> the EQC suite. New commit, in other words. > > > ?> BR, > > ?> Ulf W > > > ?> Pablo Platt wrote: > > ?> ?> ?I should have said that Roster is a list with the contact list IDs > > ?> ?> Roster = [id1, id2, id3...] > > ?> ?> each user have a different list. > > ?> ?> I need to send a message to all the the users in the list that are > > online. > > > ?> ?> Thanks > > > ------------------------------------------------------------------------ > > ?> ?> *From:* Ulf Wiger > > > > >> > > ?> ?> *To:* Pablo Platt > > >> > > ?> ?> *Cc:* erlang-questi...@REDACTED > > > > > > ?> ?> *Sent:* Fri, April 2, 2010 7:37:25 PM > > ?> ?> *Subject:* Re: [erlang-questions] Complex gproc queries > > > ?> ?> Pablo Platt wrote: > > ?> ?> ?> Hi, > > > ?> ?> ?> I'm building a simple chat server and using gproc to keep track > > of online users and the process for each user session. > > > ?> ?> ?> Basically, each online user has a process that register a > > unique session id and non unique user id (same user can have several > > clients): > > ?> ?> ?> gproc:add_local_name(SessionId) > > ?> ?> ?> gproc:add_local_property(UserId, undefined) > > > ?> ?> ?> When a user change his presence(online/busy/offline) I need to > > inform all the users on his roster. > > ?> ?> ?> After I retrieved the roster from the db, how can I select all > > the processes with UserId that is a part of the roster? > > ?> ?> ?> Rusty Klupaus's blog about gproc note that gproc:select support > > QLC but I don't understand how to do it. > > >http://rklophaus.com/blog/2009/9/16/gproc-erlang-global-process-regis... > > > ?> ?> ?> Thanks > > > ?> ?> Not sure if you need any complex queries for that. > > ?> ?> If all members of a roster have a property {p,l,Roster}, > > ?> ?> then all it should take is: > > > ?> ?> gproc:send({p,l,Roster}, Msg) > > > ?> ?> If you want to e.g. exclude the sender, you could use: > > > ?> ?> [P ! Msg || P <- gproc:lookup_pids({p,l,Roster}) -- [self()]] > > > ?> ?> Disclaimer: When verifying this, I found that gproc:send/2 was > > ?> ?> broken for properties. I fixed it in the last commit, 1 minute ago. > > > ?> ?> The commit also changes anther bug: aggregated counters had to > > ?> ?> have an integer value when created. This doesn't make sense, since > > ?> ?> the value is derived; therefore the only sensible value is > > 'undefined', > > ?> ?> which is what you get if you call gproc:reg({a,l,AggrCounter}). > > > ?> ?> Unfortunately, the commit broke the QuickCheck suite... I had forgot > > ?> ?> that I started writing a test for gproc:await/1 at the SF Erlang > > ?> ?> factory, didn't finish it. Oh well, if anyone is really hurt by > > ?> ?> this condition, let me know. (Hans, I haven't merged your changes > > ?> ?> yet). > > > ?> ?> BR, > > ?> ?> Ulf W > > ?> ?> -- Ulf Wiger > > ?> ?> CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > >> > ?http://www.erlang-solutions.com > > ?> ?> --------------------------------------------------- > > > ?> ?> --------------------------------------------------- > > > ?> ?> WE'VE CHANGED NAMES! > > > ?> ?> Since January 1st 2010 Erlang Training and Consulting Ltd. has > > become ERLANG SOLUTIONS LTD. > > > ?> ?>www.erlang-solutions.com > > > > > ?> -- Ulf Wiger > > ?> CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > > ?>http://www.erlang-solutions.com > > > -- Ulf Wiger > > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > >http://www.erlang-solutions.com > > -- > Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltdhttp://www.erlang-solutions.com > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED From radu@REDACTED Sun Apr 4 13:33:54 2010 From: radu@REDACTED (Radu Voicilas) Date: Sun, 04 Apr 2010 14:33:54 +0300 Subject: Erlang SSH examples Message-ID: <4BB87922.3010608@picohost.net> Hi guys, Are there any [very] basic examples of how to use the ssh module from Erlang? I've only found this one (http://easyerl.blogspot.com/2007/11/experimenting-with-erlang-ssh-support.html) and the documentation. Thanks, Radu. From ulf.wiger@REDACTED Sun Apr 4 14:05:50 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 04 Apr 2010 14:05:50 +0200 Subject: [erlang-questions] Re: Complex gproc queries In-Reply-To: <43f484eb-efa9-4d4f-8864-c1d581793103@x12g2000yqx.googlegroups.com> References: <566312.80019.qm@web112609.mail.gq1.yahoo.com> <4BB61D45.9090601@erlang-solutions.com> <974379.86275.qm@web112601.mail.gq1.yahoo.com> <4BB67534.5040005@erlang-solutions.com> <107850.13694.qm@web112601.mail.gq1.yahoo.com> <4BB73361.3050801@erlang-solutions.com> <546145.41042.qm@web112611.mail.gq1.yahoo.com> <4BB751EB.7080601@erlang-solutions.com> <43f484eb-efa9-4d4f-8864-c1d581793103@x12g2000yqx.googlegroups.com> Message-ID: <4BB8809E.7040105@erlang-solutions.com> Steve Davis wrote: > Interesting and useful discussion -- thanks to you both -- > > Now I'm "on board" with the ideas, the functionality that gproc > provides appears stunningly practical and useful... and, because of > this, I'm left somewhat bemused as to why it hasn't already been > absorbed into the otp platform. > > Does anyone know/have a comment on this? Is it something to do with a > back-compat issue with the gen_* patches it requires? I guess much of that falls on my shoulders. First of all, though, gproc requires no patches to OTP. In my workshop paper, I wanted to illustrate how things could look if gproc were to be included early in the boot process, and used by the OTP behaviours. Doing so also forced me to mess with gen_leader, since Distributed Erlang was not yet started. In retrospect, while it made for a good demo, it seems to have confused matters rather than clarify them. I also ended up patching the wrong version of gen_leader, so for a long time, the global part of gproc was simply broken. From what I can tell, this has now been satisfactorily resolved. I had also intended to write an EEP proposing inclusion of gproc into OTP. I haven't done so, largely because I haven't had time. For the OTP team, there is also the question whether to incorporate gproc as-is, or whether to extend the existing process registry to include some of the key features in gproc. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From g9414002.pccu.edu.tw@REDACTED Sun Apr 4 15:25:19 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Sun, 4 Apr 2010 21:25:19 +0800 Subject: [erlang-questions] Erlang SSH examples In-Reply-To: <4BB87922.3010608@picohost.net> References: <4BB87922.3010608@picohost.net> Message-ID: Jungle of Erlang codes ( http://jungerl.sourceforge.net/ ) have an implementation of SSH, too. On Sun, Apr 4, 2010 at 7:33 PM, Radu Voicilas wrote: > Hi guys, > > Are there any [very] basic examples of how to use the ssh module from > Erlang? > I've only found this one ( > http://easyerl.blogspot.com/2007/11/experimenting-with-erlang-ssh-support.html) > and the documentation. > > Thanks, > Radu. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From essiene@REDACTED Sun Apr 4 17:14:56 2010 From: essiene@REDACTED (Essien Essien) Date: Sun, 4 Apr 2010 16:14:56 +0100 Subject: [erlang-questions] simple_one_for_one being unjustly supervised? In-Reply-To: <4BB86E66.4010105@erlang-solutions.com> References: <4BB86E66.4010105@erlang-solutions.com> Message-ID: Hi Ulf, Appreciate your time!! On Sun, Apr 4, 2010 at 11:48 AM, Ulf Wiger wrote: > > The relevant code in the application controller (since I > happened to have it open, as I snipped from it earlier today) > is in application_master.erl: > > loop_it(Parent, Child, Mod, AppState) -> > ? ?receive > ? ? ? ?{Parent, get_child} -> > ? ? ? ? ? ?Parent ! {self(), Child, Mod}, > ? ? ? ? ? ?loop_it(Parent, Child, Mod, AppState); > ? ? ? ?{Parent, terminate} -> > ? ? ? ? ? ?NewAppState = prep_stop(Mod, AppState), > ? ? ? ? ? ?exit(Child, shutdown), > ? ? ? ? ? ?receive > ? ? ? ? ? ? ? ?{'EXIT', Child, _} -> ok > ? ? ? ? ? ?end, > ? ? ? ? ? ?catch Mod:stop(NewAppState), > ? ? ? ? ? ?exit(normal); > > As you can see, the supervisor is terminated using exit(Child,shutdown). > (Child is the pid returned from Mod:start/2). Yup... looks fine to me too. > > I have not looked at your logs, nor do I have a good explanation > for the discrepancy you're observing. It has suprised me as well, though I think I've never really been interested in how children of a simple_one_for_one cleanup in previous use cases. Right now though, I'm using them for load balancing, so its important how they cleanup, which is why I've taken an interest and noticed them now. I'll try to dig deeper and see if I can at least understand why it is happening. cheers, Essien > > BR, > Ulf W > > > Essien Essien wrote: >> >> Hi all, >> >> Maybe I'm missing something pretty obvious, but I'm noticing that the >> children of a simple_one_for_one supervisor are being killed off >> without honoring their "shutdown" interval, when they're taken down by >> a top-level application. Everything work fine if I sent an >> exit(SupPid, shutdown) to the top-level supervisor PID though. >> >> I have attached a contrived example application called "boss"... >> >> The application callback and top-level supervisor behaviours are >> implemented in the same module. >> >> The boss supervisor is a one_for_one supervisor which supervises a >> "nanny", "manage" and "delay" modules. >> >> "nanny" is a simple_one_for_one supervisor which supervises "children" >> (a gen_server). >> >> "manage" is a gen_server which spins up the children by calling >> nanny:start_child when it starts up. The number of children is read >> from the boss.app file. >> >> "delay" is a gen_server which just delays for a configurable amount of >> time before shutting down. >> >> "children" is a simple gen_server which delays for 5 seconds before >> shutting down. >> >> From my tests in an erl session, if i do: >> >> 1 > {ok, Pid} = boss:start_link([8, 5000]). %these arguments mean, 8 >> children, 5 seconds delay for the delay gen_server >> >> And then: >> >> 2 > exit(Pid, shutdown). >> >> Everything happens nicely. I see _all_ my logging messages which >> suggests that things cleaned up nice and sweet. >> >> But if I do: >> >> 3 > application:start(boss). >> 4 > application:stop(boss). >> >> ONLY the "delay" module cleans up nicely... I never get to see the >> logs from the "children" instances... which means the tree just kills >> them off very fast. >> >> >> Is this a known behaviour? Am I missing something critical? >> >> Any wisdoms will be much appreciated. >> >> Also, I tried peeking at the code for supervisor.erl (went all the way >> to do_terminate/2 and shutdown/2) and nothing looks amiss, the code >> for application_controller.erl would take a bit more than a cursory >> glance to trace through all that is happening, so I wanted to ask here >> first, before going any further. >> >> cheers, >> Essien >> >> >> ------------------------------------------------------------------------ >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > -- > Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > http://www.erlang-solutions.com > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG > SOLUTIONS LTD. > > www.erlang-solutions.com > > From max.lapshin@REDACTED Sun Apr 4 17:24:38 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 4 Apr 2010 19:24:38 +0400 Subject: Redirect all output (io:format) to file or remote shell Message-ID: Hi. I've decided that using runit for starting erlyvideo is not usable because of problems with restarting. Now I'm interested if it is possible to save somehow io:format output into file? I haven't found how to tell standard_io to save all io requests to file or to some server (it is even better). From g@REDACTED Sun Apr 4 17:26:05 2010 From: g@REDACTED (Garrett Smith) Date: Sun, 4 Apr 2010 10:26:05 -0500 Subject: Variable naming conventions Message-ID: I routinely run into two cases of variable naming that frustrate me. I'll try to give them each a name: - Variable iterations - Variable leakage A really common case of iteration is when process state is updated. Something like this: handle_call(_, #state{name=Name}=State) -> UpdatedName = strip_white_space(Name), {noreply, State#state{name=UpdatedName)}}. This isn't an awful example, but tacking "Updated" onto a variable that changes just feels too arbitrary and it's too long. I've tried this approach: StrippedName = strip_white_space(Name) This is probably most in the spirit of Erlang's immutability naming scheme -- pick good, descriptive names. But it also gets to be painful. The names are long and you end up using weird abbreviations, and the names become an unnecessary point of maintenance. I saw the use of a "0" suffix to denote the "naught" version of a variable. This seems like a nice convention: it's short, fairly obvious and doesn't require a bit of hand wringing. There's also something like Name1, Name2, Name3, ... I've avoided this in principle because it feels like a cop-out: it's probably a sign that the function needs refactoring. The other frustrating case -- and far more so -- is something like this: Name = case get_name() of undefined -> ""; {ok, NameValueReturnedByGetName} -> NameValueReturnedByGetName end (I'm calling this "leakage" because I'd probably just call it Name if it didn't leak. While that'd be clunky, it at least says what I think it should.) I've tried names like this: N, Val, Str, S. All yuck! I know I can refactor this into a function and solve the problem, but of course there are cases where I don't want to. And in general, I don't know if "having problems picking a variable name" is a good reason for creating a function. (Maybe it is but what about cases when you just don't?) I'm curious what others use for these cases. Is there one convention that tends to be used more frequently? Garrett From lemenkov@REDACTED Sun Apr 4 17:30:46 2010 From: lemenkov@REDACTED (Peter Lemenkov) Date: Sun, 4 Apr 2010 19:30:46 +0400 Subject: [erlang-questions] Redirect all output (io:format) to file or remote shell In-Reply-To: References: Message-ID: Hello! 2010/4/4 Max Lapshin : > Now I'm interested if it is possible to save somehow io:format output > into file? I advise you to write your own error_logger module for that purpose (event journaling). I, personally, would rather send all messages through syslog (and adjust syslog daemon according to your needs). -- With best regards, Peter Lemenkov. From max.lapshin@REDACTED Sun Apr 4 17:34:23 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 4 Apr 2010 19:34:23 +0400 Subject: [erlang-questions] Redirect all output (io:format) to file or remote shell In-Reply-To: References: Message-ID: On Sun, Apr 4, 2010 at 7:30 PM, Peter Lemenkov wrote: > Hello! > > 2010/4/4 Max Lapshin : > >> Now I'm interested if it is possible to save somehow io:format output >> into file? > > I advise you to write your own error_logger module for that purpose > (event journaling). I, personally, would rather send all messages > through syslog (and adjust syslog daemon according to your needs). > I want not to use error_logger, but to redirect all debug output (?D(..)) to somewhere. From dale@REDACTED Sun Apr 4 18:27:45 2010 From: dale@REDACTED (Dale Harvey) Date: Sun, 4 Apr 2010 17:27:45 +0100 Subject: [erlang-questions] Redirect all output (io:format) to file or remote shell In-Reply-To: References: Message-ID: I would checkout run_erl for redirecting stdio to named pipes http://stackoverflow.com/questions/2531150/how-to-receive-stdio-and-error-logger-messages-on-a-remote-shell On 4 April 2010 16:34, Max Lapshin wrote: > On Sun, Apr 4, 2010 at 7:30 PM, Peter Lemenkov wrote: > > Hello! > > > > 2010/4/4 Max Lapshin : > > > >> Now I'm interested if it is possible to save somehow io:format output > >> into file? > > > > I advise you to write your own error_logger module for that purpose > > (event journaling). I, personally, would rather send all messages > > through syslog (and adjust syslog daemon according to your needs). > > > > I want not to use error_logger, but to redirect all debug output > (?D(..)) to somewhere. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From max.lapshin@REDACTED Sun Apr 4 18:50:50 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 4 Apr 2010 20:50:50 +0400 Subject: [erlang-questions] Redirect all output (io:format) to file or remote shell In-Reply-To: References: Message-ID: It's nice, I will try it, thank you. From karol.skocik@REDACTED Sun Apr 4 19:21:54 2010 From: karol.skocik@REDACTED (karol skocik) Date: Sun, 4 Apr 2010 19:21:54 +0200 Subject: [erlang-questions] initializing library applications without processes In-Reply-To: References: Message-ID: Hi, well, I am refactoring some of my older code dealing with paypal payments and amazon (ec2, s3) stuff. They are basically libraries - they don't execute any activity, and there is nothing I want them to do except being functioning when some other module uses their API. To be functioning means executing in the context of the caller of the function (block caller until the library function finishes), but at the same time have access to some state related to what they do - typically configuration (authorization keys, shopping list, ...). Starting the application should ensure that the state is loaded/initialized ok - I want to fail at application:start()'s time, not later finding out initialization wasn't correct, or rely on manual app:start() invocation. It should play nicely with what's written in .rel file. Ulf's solution was exactly what I needed, since I need the supervisor process anyway (to be a owner of ETS table). Karol On Sun, Apr 4, 2010 at 6:49 PM, Eric Merritt wrote: > Karol, > > When I hear a question like this I want to step back and ask about the > bigger picture. There are a bunch of ways to do this, bob and ulf have > gone into some detail on those. Those address the question your asking > but not the higher level issue that this is a symptom of. If you can > would you mind stepping back and describing what it is you are trying > to do at a higher level? I suspect that the best answer will be a > solution to that, not an answer to the specific question you are > asking here. > > Eric > > On Sat, Apr 3, 2010 at 12:17 PM, karol skocik wrote: >> Hi, >> ?sometimes, I just need a library "application" without processes, >> which works with some state in ETS. To function properly, >> they have to make the ETS table and populate it with values >> from config files. When that step is done, there is nothing left to do >> for processes, they would just serve as a useless proxy for >> querying the ETS table, and that's not needed when you can (in some contexts) >> access the table directly. >> >> I like the mechanism of applications and their dependency solving, >> and I would ideally do application:start(my_app_with_ets_initialised) >> but that won't do, since {ok, SupPid} is expected as a result. >> >> But I don't want to start any processes, I just want to ensure that >> application_master:init/4 calls my "start" function, >> and declare application as "running" when 'ok' is returned >> or fail otherwise (unless it's {ok, SupPid} - as it is now). >> >> Currently, when we have library application, application:start(...) >> does not call anything since 'mod' is not specified in .app file. >> Could it be changed, and application behaviour would just call >> start/2 and stop/1 as usual, but start the SupPid optionally when it's supplied? >> Or - alternatively - is there any way how to ensure that some init >> function is done >> fitting nicely with application dependencies as declared in .rel file? >> >> Thanks, >> ?Karol >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From radu@REDACTED Sun Apr 4 20:04:36 2010 From: radu@REDACTED (Radu Voicilas) Date: Sun, 04 Apr 2010 21:04:36 +0300 Subject: [erlang-questions] Erlang SSH examples In-Reply-To: References: <4BB87922.3010608@picohost.net> Message-ID: <4BB8D4B4.5070903@picohost.net> Thanks for the pointer to jungerl - I was a little bit confused about ssh_cm, but it became clear after I checked the source for that module. I am trying to come up with an automatic deployment system for a legacy code base (which is written in Python) and the only way that I can do that is by ssh-ing as my user on the given host, sudo as root and run the deploy command. The only problem I encounter here is how I could do something like this (sorry for the imperative pseudocode): if ssh_connection:send(Connection, ChannelId, "sudo su -").startswith("[sudo] password"): ssh_connection:send(Connection, ChannelId, "password) ssh_connection:send(Connection, ChannelId, "deploy project") ... Thank you, Radu. ??? (Yau-Hsien Huang) wrote: > Jungle of Erlang codes ( > http://jungerl.sourceforge.net/ > ) > have > an implementation of SSH, too. > > > On Sun, Apr 4, 2010 at 7:33 PM, Radu Voicilas wrote: > >> Hi guys, >> >> Are there any [very] basic examples of how to use the ssh module from >> Erlang? >> I've only found this one ( >> http://easyerl.blogspot.com/2007/11/experimenting-with-erlang-ssh-support.html) >> and the documentation. >> >> Thanks, >> Radu. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From cowboymathu@REDACTED Sun Apr 4 20:11:24 2010 From: cowboymathu@REDACTED (=?UTF-8?B?TUF0aHV2YXRoYW5hbiBNT3UgfHwgIOCuruCupOCvgeCuteCupOCuqeCuqeCvjSDgrq7gr4w=?=) Date: Sun, 4 Apr 2010 23:41:24 +0530 Subject: How to change number of erlang logs on the fly? Message-ID: Hi all, I want to know how can I change number of erlang logs on the fly. Is there any way to set *RUN_ERL_LOG_GENERATIONS* variable dynamically? thanks-- Mathuvathanan Mou. From erlang@REDACTED Sun Apr 4 20:36:38 2010 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 4 Apr 2010 20:36:38 +0200 Subject: [erlang-questions] Variable naming conventions In-Reply-To: References: Message-ID: Could this be a sign of a deeper problem? I don't excited about variable names - but I do worry about the accuracy of function names. It seems to me that variable Names only become a problem when the functions (or clauses) are large. A very small function will have few variables and very little to get confused about. I tend to use H, T (heads and tails of lists) L (lists) I,J,K,N,M integers (The habits of Fortran dig deep) My rule is - short functions with accurate names. If the function is more than about 15 lines refactor it. Use short variable names. If this is not abundantly clear add a comment and a type signature and a test case. I actually very much prefer very short variable names - if I say file:open(F, [...]) I *know* F is a file since it's strongly signaled by the "file:..." function call - no need to say file:open(InFile) or whatever. /Joe On Sun, Apr 4, 2010 at 5:26 PM, Garrett Smith wrote: > I routinely run into two cases of variable naming that frustrate me. > I'll try to give them each a name: > > - Variable iterations > - Variable leakage > > A really common case of iteration is when process state is updated. > Something like this: > > ?handle_call(_, #state{name=Name}=State) -> > ? ?UpdatedName = strip_white_space(Name), > ? ?{noreply, State#state{name=UpdatedName)}}. > > This isn't an awful example, but tacking "Updated" onto a variable > that changes just feels too arbitrary and it's too long. > > I've tried this approach: > > ?StrippedName = strip_white_space(Name) > > This is probably most in the spirit of Erlang's immutability naming > scheme -- pick good, descriptive names. But it also gets to be > painful. The names are long and you end up using weird abbreviations, > and the names become an unnecessary point of maintenance. > > I saw the use of a "0" suffix to denote the "naught" version of a > variable. This seems like a nice convention: it's short, fairly > obvious and doesn't require a bit of hand wringing. > > There's also something like Name1, Name2, Name3, ... > > I've avoided this in principle because it feels like a cop-out: it's > probably a sign that the function needs refactoring. > > The other frustrating case -- and far more so -- is something like this: > > ?Name = case get_name() of > ? ?undefined -> ""; > ? ?{ok, NameValueReturnedByGetName} -> NameValueReturnedByGetName > ?end > > (I'm calling this "leakage" because I'd probably just call it Name if > it didn't leak. While that'd be clunky, it at least says what I think > it should.) > > I've tried names like this: N, Val, Str, S. > > All yuck! > > I know I can refactor this into a function and solve the problem, but > of course there are cases where I don't want to. And in general, I > don't know if "having problems picking a variable name" is a good > reason for creating a function. (Maybe it is but what about cases when > you just don't?) > > I'm curious what others use for these cases. Is there one convention > that tends to be used more frequently? > > Garrett > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From max.lapshin@REDACTED Sun Apr 4 20:38:40 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 4 Apr 2010 22:38:40 +0400 Subject: [erlang-questions] Variable naming conventions In-Reply-To: References: Message-ID: On Sun, Apr 4, 2010 at 10:36 PM, Joe Armstrong wrote: > > My rule is - short functions with accurate names. If the function > is more than about 15 lines refactor it. Use short variable > names. If this is not abundantly clear add a comment and > a type signature and a test case. > Erlang makes you to write short functions, it is the only way to determine where exactly error was raised =) From rvirding@REDACTED Sun Apr 4 22:20:46 2010 From: rvirding@REDACTED (Robert Virding) Date: Sun, 4 Apr 2010 22:20:46 +0200 Subject: [erlang-questions] Variable naming conventions In-Reply-To: References: Message-ID: I agree Joe, though I probably have short function names than you do. I also find that having short variable tends to make the code more readable. Using indexes on names is a very good way of indicating that this references some data which flows through the function and may have been updated, for example State0, State1, ..., or as I would probably write it: St0, St1, ... . If you use it when the variables are not connected then you are asking for trouble. This is sort of my take on overlong and over descriptive variable, and function, names: http://www.geekherocomic.com/2008/11/12/real-programmers-dont-write-documentation/ Robert On 4 April 2010 20:36, Joe Armstrong wrote: > Could this be a sign of a deeper problem? > > I don't excited about variable names - but I do worry about > the accuracy of function names. It seems to me that variable > Names only become a problem when the functions (or clauses) are large. > A very small function will have few variables and very little to get > confused about. > > I tend to use H, T (heads and tails of lists) L (lists) > I,J,K,N,M integers (The habits of Fortran dig deep) > > My rule is - short functions with accurate names. If the function > is more than about 15 lines refactor it. Use short variable > names. If this is not abundantly clear add a comment and > a type signature and a test case. > > I actually very much prefer very short variable names - > if I say file:open(F, [...]) I *know* F is a file since it's strongly > signaled by the "file:..." function call - no need to say > file:open(InFile) or whatever. > > /Joe > > > > > On Sun, Apr 4, 2010 at 5:26 PM, Garrett Smith wrote: >> I routinely run into two cases of variable naming that frustrate me. >> I'll try to give them each a name: >> >> - Variable iterations >> - Variable leakage >> >> A really common case of iteration is when process state is updated. >> Something like this: >> >> ?handle_call(_, #state{name=Name}=State) -> >> ? ?UpdatedName = strip_white_space(Name), >> ? ?{noreply, State#state{name=UpdatedName)}}. >> >> This isn't an awful example, but tacking "Updated" onto a variable >> that changes just feels too arbitrary and it's too long. >> >> I've tried this approach: >> >> ?StrippedName = strip_white_space(Name) >> >> This is probably most in the spirit of Erlang's immutability naming >> scheme -- pick good, descriptive names. But it also gets to be >> painful. The names are long and you end up using weird abbreviations, >> and the names become an unnecessary point of maintenance. >> >> I saw the use of a "0" suffix to denote the "naught" version of a >> variable. This seems like a nice convention: it's short, fairly >> obvious and doesn't require a bit of hand wringing. >> >> There's also something like Name1, Name2, Name3, ... >> >> I've avoided this in principle because it feels like a cop-out: it's >> probably a sign that the function needs refactoring. >> >> The other frustrating case -- and far more so -- is something like this: >> >> ?Name = case get_name() of >> ? ?undefined -> ""; >> ? ?{ok, NameValueReturnedByGetName} -> NameValueReturnedByGetName >> ?end >> >> (I'm calling this "leakage" because I'd probably just call it Name if >> it didn't leak. While that'd be clunky, it at least says what I think >> it should.) >> >> I've tried names like this: N, Val, Str, S. >> >> All yuck! >> >> I know I can refactor this into a function and solve the problem, but >> of course there are cases where I don't want to. And in general, I >> don't know if "having problems picking a variable name" is a good >> reason for creating a function. (Maybe it is but what about cases when >> you just don't?) >> >> I'm curious what others use for these cases. Is there one convention >> that tends to be used more frequently? >> >> Garrett >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From g@REDACTED Sun Apr 4 22:44:36 2010 From: g@REDACTED (Garrett Smith) Date: Sun, 4 Apr 2010 15:44:36 -0500 Subject: [erlang-questions] Variable naming conventions In-Reply-To: References: Message-ID: On Sun, Apr 4, 2010 at 12:05 PM, Eric Merritt wrote: > Garrett, > > ?You have already mention most of the approaches to solving this problem. > >> A really common case of iteration is when process state is updated. >> Something like this: >> >> ?handle_call(_, #state{name=Name}=State) -> >> ? ?UpdatedName = strip_white_space(Name), >> ? ?{noreply, State#state{name=UpdatedName)}}. >> >> This isn't an awful example, but tacking "Updated" onto a variable >> that changes just feels too arbitrary and it's too long. >> >> I've tried this approach: >> >> ?StrippedName = strip_white_space(Name) > > I usually wouldn't have an intermediate variable here. ?I usually end > up with something like. > > handle_call(_, #state{name = Name} = State) -> > ?? ?{noreply, State#state{name = strip_white_space(Name))}}. Yeah, I generally would as well, but for the example. Though, as you illustrate below, breaking calls out into separate assignments can help readability, IMO. > > basically passing the returned value of a 'thing' directly into the > next 'thing' you are going to call. This is the most common approach > in the code I write, I also think its pretty common in Erlang code in > general. > > >> I saw the use of a "0" suffix to denote the "naught" version of a >> variable. This seems like a nice convention: it's short, fairly >> obvious and doesn't require a bit of hand wringing. >> >> There's also something like Name1, Name2, Name3, ... > > This is also very common. You could say the names aren't tremendously > descriptive, but the function calls themselves tend to be. For an > arbitrary example. > > do_some_mangling(Somewhere) -> > ?Name = get_some_initial_values_from(Somewhere), > ?Name1 = mangle_those_values_as_a_first_pass(Name), > ?Name2 = mangle_them_in_some_different_way(Name1), > ?return_them_as_a_final_mangle(Name2). > > Of course, the function names aren't usually quite this verbose, they > are usually fairly descriptive. So while you could argue that Name, > Name1, Name2 aren't really providing that much information. The > function as a whole is still very readable and its still very > understandable whats going on. Good. I'm going to starting using this where it makes sense. >> I've avoided this in principle because it feels like a cop-out: it's >> probably a sign that the function needs refactoring. > > Often, but not always. take the above example. We could place each > function call inside the parameter list of the next call. However, > that would probably be less readable then the example above. In > general though, when you see this. You should probably take a good > hard look at your code and make sure your factoring is good. > >> >> The other frustrating case -- and far more so -- is something like this: >> >> ?Name = case get_name() of >> ? ?undefined -> ""; >> ? ?{ok, NameValueReturnedByGetName} -> NameValueReturnedByGetName >> ?end >> >> (I'm calling this "leakage" because I'd probably just call it Name if >> it didn't leak. While that'd be clunky, it at least says what I think >> it should.) > > I don't see this as a tremendous problem. Functions tend to be pretty > short and specific in Erlang, that dramatically reduces the kind of > collision you that this is intended to solve. I can't say that I run > into this problem very frequently at all. On the rare occasion that I > do a slightly longer name suffices without too much pain. Yes, after posting the original email, I tackled the block of code that was annoying me, refactoring into a couple of meaningful, cogent functions and am much happier. > >> >> I've tried names like this: N, Val, Str, S. > > Ugh, I would stay away from those if at all possible. Much better a > verbose name then something that offers little or no information. > >> >> All yuck! >> >> I know I can refactor this into a function and solve the problem, but >> of course there are cases where I don't want to. And in general, I >> don't know if "having problems picking a variable name" is a good >> reason for creating a function. (Maybe it is but what about cases when >> you just don't?) > > I don't think so. As you alluded to before, this kind of problem is a > bad 'smell' it usually means you need to think about refactoring your > function. I wouldn't, however, refactor them with variable names in > mind. I would look at the problem you are trying to solve and the > 'actions' you are taking in your code and refactor along those lines. > I suspect very strongly that if you do that your naming problems will > go away or be vastly reduced as well. > Thanks! From g@REDACTED Sun Apr 4 23:01:44 2010 From: g@REDACTED (Garrett Smith) Date: Sun, 4 Apr 2010 16:01:44 -0500 Subject: [erlang-questions] Variable naming conventions In-Reply-To: References: Message-ID: On Sun, Apr 4, 2010 at 1:36 PM, Joe Armstrong wrote: > Could this be a sign of a deeper problem? I think so. I'm new to functional programming and am carrying over some habits from imperative languages that are contributing to this problem. > I don't excited about variable names - but I do worry about > the accuracy of function names. It seems to me that variable > Names only become a problem when the functions (or clauses) are large. > A very small function will have few variables and very little to get > confused about. > > I tend to use H, T (heads and tails of lists) L (lists) > I,J,K,N,M integers (The habits of Fortran dig deep) > > My rule is - short functions with accurate names. If the function > is more than about 15 lines refactor it. Use short variable > names. If this is not abundantly clear add a comment and > a type signature and a test case. > > I actually very much prefer very short variable names - > if I say file:open(F, [...]) I *know* F is a file since it's strongly > signaled by the "file:..." function call - no need to say > file:open(InFile) or whatever. This makes sense in light of the short function requirement. I need to get better at spotting the right factoring sooner so I'm not spending as much time rewriting code :) > /Joe > > > > > On Sun, Apr 4, 2010 at 5:26 PM, Garrett Smith wrote: >> I routinely run into two cases of variable naming that frustrate me. >> I'll try to give them each a name: >> >> - Variable iterations >> - Variable leakage >> >> A really common case of iteration is when process state is updated. >> Something like this: >> >> ?handle_call(_, #state{name=Name}=State) -> >> ? ?UpdatedName = strip_white_space(Name), >> ? ?{noreply, State#state{name=UpdatedName)}}. >> >> This isn't an awful example, but tacking "Updated" onto a variable >> that changes just feels too arbitrary and it's too long. >> >> I've tried this approach: >> >> ?StrippedName = strip_white_space(Name) >> >> This is probably most in the spirit of Erlang's immutability naming >> scheme -- pick good, descriptive names. But it also gets to be >> painful. The names are long and you end up using weird abbreviations, >> and the names become an unnecessary point of maintenance. >> >> I saw the use of a "0" suffix to denote the "naught" version of a >> variable. This seems like a nice convention: it's short, fairly >> obvious and doesn't require a bit of hand wringing. >> >> There's also something like Name1, Name2, Name3, ... >> >> I've avoided this in principle because it feels like a cop-out: it's >> probably a sign that the function needs refactoring. >> >> The other frustrating case -- and far more so -- is something like this: >> >> ?Name = case get_name() of >> ? ?undefined -> ""; >> ? ?{ok, NameValueReturnedByGetName} -> NameValueReturnedByGetName >> ?end >> >> (I'm calling this "leakage" because I'd probably just call it Name if >> it didn't leak. While that'd be clunky, it at least says what I think >> it should.) >> >> I've tried names like this: N, Val, Str, S. >> >> All yuck! >> >> I know I can refactor this into a function and solve the problem, but >> of course there are cases where I don't want to. And in general, I >> don't know if "having problems picking a variable name" is a good >> reason for creating a function. (Maybe it is but what about cases when >> you just don't?) >> >> I'm curious what others use for these cases. Is there one convention >> that tends to be used more frequently? >> >> Garrett >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From kagato@REDACTED Sun Apr 4 23:24:52 2010 From: kagato@REDACTED (Jayson Vantuyl) Date: Sun, 4 Apr 2010 14:24:52 -0700 Subject: [erlang-questions] Variable naming conventions In-Reply-To: References: Message-ID: > I also find that having short variable tends to make the code more > readable. Ditto here. > Using indexes on names is a very good way of indicating that this > references some data which flows through the function and may have > been updated, for example State0, State1, ..., or as I would probably > write it: St0, St1, ... . If you use it when the variables are not > connected then you are asking for trouble. I do this all the time. It's really not that bad. I can't claim that my functions are exhibit ideal style (especially in the grittier systems code), but I find that a numerical sequence has higher information per character than using more words to imbue more specific meaning. For example, in erlctl I do this a bit when processing the command-line into my options proplist. Code here: http://github.com/jvantuyl/erlctl/blob/master/lib/erlctl-0.1/src/erlctl_cmdline.erl#L21 In that function, I can't imagine a better way to name Opts. I suppose I could have broken it into different functions and traversed them somehow, but I just don't see that as being any clearer. -- Jayson Vantuyl kagato@REDACTED From tony@REDACTED Sun Apr 4 23:27:40 2010 From: tony@REDACTED (Tony Rogvall) Date: Sun, 4 Apr 2010 23:27:40 +0200 Subject: [erlang-questions] Erlang SSH examples In-Reply-To: References: <4BB87922.3010608@picohost.net> Message-ID: <6B3236F2-1C18-444E-BE01-35CBAF63F138@rogvall.se> Do not use the jungerl version! It was merged with the OTP ssh a long time ago. (And supported ;-) /Tony On 4 apr 2010, at 15.25, ??? (Yau-Hsien Huang) wrote: > Jungle of Erlang codes ( > http://jungerl.sourceforge.net/ > ) > have > an implementation of SSH, too. > > > On Sun, Apr 4, 2010 at 7:33 PM, Radu Voicilas wrote: > >> Hi guys, >> >> Are there any [very] basic examples of how to use the ssh module from >> Erlang? >> I've only found this one ( >> http://easyerl.blogspot.com/2007/11/experimenting-with-erlang-ssh-support.html) >> and the documentation. >> >> Thanks, >> Radu. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From steven.charles.davis@REDACTED Mon Apr 5 04:20:19 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 4 Apr 2010 19:20:19 -0700 (PDT) Subject: Erlang SSH examples In-Reply-To: <4BB87922.3010608@picohost.net> References: <4BB87922.3010608@picohost.net> Message-ID: <2eb3da8b-6e7f-459c-83a3-55a159637227@o30g2000yqb.googlegroups.com> Don't forget there's also a "client_server.erl" example in erts/lib/ ssl-xxx/examples/src /s On Apr 4, 6:33?am, Radu Voicilas wrote: > Hi guys, > > Are there any [very] basic examples of how to use the ssh module from Erlang? > I've only found this one (http://easyerl.blogspot.com/2007/11/experimenting-with-erlang-ssh-sup...) and the ?documentation. > > Thanks, > Radu. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED From leap@REDACTED Mon Apr 5 10:08:45 2010 From: leap@REDACTED (Michael Turner) Date: Mon, 05 Apr 2010 08:08:45 +0000 Subject: [erlang-questions] Variable naming conventions In-Reply-To: Message-ID: >A really common case of iteration is when process state is updated. >Something like this: > > handle_call(_, #state{name=Name}=State) -> > UpdatedName = strip_white_space(Name), > {noreply, State#state{name=UpdatedName)}}. > > This isn't an awful example, but tacking "Updated" onto a variable > that changes just feels too arbitrary and it's too long. Well, if you really must make every variable self-descriptive, you could pass UnstrippedName and update it to Name. But consider this: --- handle_call(_, #state{name=S}=State) -> N = strip_white_space(S), {noreply, State#state{name=N)}}. --- I almost immediately understand S to mean "some string", and N to mean "a (more proper) name for the state in question"; I immediately understood that handle_call might get an unstripped string as a name; and that names in state records should be stripped. The really important thing that's happening here is that a name is being put into more canonical form. Shorter variable names with only a hint of meaning will "stand out of the way", so that someone reading your code can more easily see what sort of canonical form is required. Maybe I'm old school, but variable names should generally be short, and functions should be so short so that you can usually see ALL of the context of use for a variable, and thereby infer a great deal about its meaning, whenever you're in doubt. When I see code with lots of variable names spelled out, I don't think, "Ah, wonderfully self-documenting code!" I think, "Uh-oh, maybe I'm looking at code so poorly structured that every variable has to explain itself at every point. Worse, I'll have a harder time figuring out the poor structure, because it's harder to see structure even in well-structured code after it's been bulked out with long variable names." Don't bother trying to come up with some near-optimal factoring the first time -- refactor *all* the time ("without mercy" as some say.) If you're afraid to do that because you're afraid of breaking things, the problem is not with the habit of refactoring, it's that you're not paying enough attention to testing. -michael turner From kheinz57@REDACTED Mon Apr 5 13:42:59 2010 From: kheinz57@REDACTED (heinz, haeberle) Date: Mon, 05 Apr 2010 13:42:59 +0200 Subject: mod_cgi available for windows? Message-ID: <4BB9CCC3.9070407@gmx.de> Is mod_cgi available for windows as well? I got the impression from the erlang web page it is not. Any comments? Heinz From james.aimonetti@REDACTED Mon Apr 5 21:27:37 2010 From: james.aimonetti@REDACTED (James Aimonetti) Date: Mon, 05 Apr 2010 15:27:37 -0400 Subject: Sets and Dict test suites Message-ID: <4BBA39A9.6030803@gmail.com> Does anyone on the list have test suites for testing sets and dict functionality using different backends? I'm going through Okasaki's book and currently implementing the splay trees, and, having looked at the rbsets and rbdict using red-black trees, I wanted to try implementing the API using the splay trees and testing the results against the official implementations. Also, is there any writeup of the backends' strengths vs weaknesses for various use cases (as concerns Erlang specifically)? If not, I'd be interested in developing one if those with more experience could offer help as I worked on it. Thanks, James -- James Aimonetti mobile: 314.809.6307 work: 540.459.2220 email: james.aimonetti@REDACTED website: http://jamesaimonetti.com From rvirding@REDACTED Tue Apr 6 01:17:02 2010 From: rvirding@REDACTED (Robert Virding) Date: Tue, 6 Apr 2010 01:17:02 +0200 Subject: [erlang-questions] Sets and Dict test suites In-Reply-To: <4BBA39A9.6030803@gmail.com> References: <4BBA39A9.6030803@gmail.com> Message-ID: Hi James, Sorry, I saw your earlier mail but I missed it. I have used a relatively simple test suite when writing my RB and AA stuff. First I use all the examples given in books and papers and check that my trees look the same as theirs. This, of course, only works for small data sets. I also just sit and add and delete elements from the shell and check that the tree looks right. When I am satisfied that it looks right I try some bigger data sets. Easy ways to generate them are functions like: random_data(N) -> [ {random:uniform(),I} || I <- lists:seq(1, N) ]. which generates a set of {Key,Value} pairs with random keys. I use such a set to build a dict. I then use the same set to 'fetch' all the keys to test that they are there and that I can reach them. Then I use the same and delete all the keys to check that I can and that the final dict is empty as it should be. It would better to both 'fetch' and delete each key to make sure that the tree is always consistent during deletes. I found deleting harder than adding. You can also do the same thing with non-random data sets to see if you get some funny trees, for example your tree degenerates to a list. Another good and simple test is to just count the depth of each node and see that you get a reasonable spread of the depth, i.e. that it really lg n like it should be. I do some simple unit testing by saving problem cases so I can retry them. I also have a test file for comparing various algorithms which is useful for testing as well. Run it through the compiler and generate the file after the macro expansions to see the resultant code, c(db_test, ['E']). There are some comments but it is really an internal file. Not very scientific I know but it seems to work. Also the trees quickly become unreadable when they grow big. I hope you do the splaydict and splaysets, I would like to see them. Real soon now (TM) I will adding rbdict and rbsets to the distribution. There are also some extensions to the API I would like to add. I hope this helps, Robert On 5 April 2010 21:27, James Aimonetti wrote: > Does anyone on the list have test suites for testing sets and dict > functionality using different backends? I'm going through Okasaki's book and > currently implementing the splay trees, and, having looked at the rbsets and > rbdict using red-black trees, I wanted to try implementing the API using the > splay trees and testing the results against the official implementations. > > Also, is there any writeup of the backends' strengths vs weaknesses for > various use cases (as concerns Erlang specifically)? If not, I'd be > interested in developing one if those with more experience could offer help > as I worked on it. > > Thanks, > > James > > -- > James Aimonetti > mobile: 314.809.6307 > work: 540.459.2220 > email: james.aimonetti@REDACTED > website: http://jamesaimonetti.com > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -------------- next part -------------- A non-text attachment was scrubbed... Name: db_test.erl Type: application/octet-stream Size: 6037 bytes Desc: not available URL: From kenji.rikitake@REDACTED Tue Apr 6 07:44:45 2010 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Tue, 6 Apr 2010 14:44:45 +0900 Subject: [erlang-questions] Erlang SSH examples In-Reply-To: <4BB87922.3010608@picohost.net> References: <4BB87922.3010608@picohost.net> Message-ID: <20100406054445.GA91150@k2r.org> http://github.com/jj1bdx/sshrpc/ Kenji Rikitake In the message <4BB87922.3010608@REDACTED> dated Sun, Apr 04, 2010 at 02:33:30PM +0300, Radu Voicilas writes: > Hi guys, > > Are there any [very] basic examples of how to use the ssh module from Erlang? > I've only found this one (http://easyerl.blogspot.com/2007/11/experimenting-with-erlang-ssh-support.html) and the documentation. From steven.charles.davis@REDACTED Tue Apr 6 08:29:01 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 5 Apr 2010 23:29:01 -0700 (PDT) Subject: Erlang SSH examples In-Reply-To: <2eb3da8b-6e7f-459c-83a3-55a159637227@o30g2000yqb.googlegroups.com> References: <4BB87922.3010608@picohost.net> <2eb3da8b-6e7f-459c-83a3-55a159637227@o30g2000yqb.googlegroups.com> Message-ID: <0899802b-8550-4f8b-8087-a185ce210cde@y14g2000yqm.googlegroups.com> My apologies for this total misdirection! /s On Apr 4, 9:20?pm, Steve Davis wrote: > Don't forget there's also a "client_server.erl" example in erts/lib/ > ssl-xxx/examples/src > > /s > > On Apr 4, 6:33?am, Radu Voicilas wrote: > > > Hi guys, > > > Are there any [very] basic examples of how to use the ssh module from Erlang? > > I've only found this one (http://easyerl.blogspot.com/2007/11/experimenting-with-erlang-ssh-sup...) and the ?documentation. > > > Thanks, > > Radu. > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > Seehttp://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED From ingela.andin@REDACTED Tue Apr 6 09:06:50 2010 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 6 Apr 2010 09:06:50 +0200 Subject: [erlang-questions] httpc sending empty te header In-Reply-To: <201004011016.23394.vincent.dephily@mobile-devices.fr> References: <201003311410.40664.vincent.dephily@mobile-devices.fr> <201004011016.23394.vincent.dephily@mobile-devices.fr> Message-ID: Hi! 2010/4/1 Vincent de Phily : > On Thursday 01 April 2010 08:56:29 Ingela Andin wrote: >> From HTTP rfc: ?"If the TE field-value is empty or if no TE field is >> present, the only transfer-coding is "chunked"" > > Fair enough, that explains why the behaviour is not wrong. But in that case, > why add the header at all ? You'd save yourself two lines of code, a tiny bit > of CPU, 5 bytes on the wire, and some confused users. Well as it is legacy code I can only guess as to the reason why, and my guess would acctualy be clarity, e.i. by explicitly sending the empty te-header stating that we only want to handle chunked transfer-encoding and it is intentional. Kind of like settig a default value explicitly for clarity. I think that clarity in the code could be achived also with a comment and the minimalistic approch. Maybe it would be better from a user perspective as it would not requier that they understand the te-header. Something for us to think about. Regards Ingela Erlang/OTP team Ericsson From vladdu55@REDACTED Tue Apr 6 09:48:19 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 6 Apr 2010 09:48:19 +0200 Subject: erlide: moving on to Eclipse 3.5 and Erlang R12 Message-ID: Hi all, -- Regarding Eclipse 3.5: Last week the last users that I know of that required Eclipse 3.4 have upgraded to 3.5. Therefore we are planning to start building against 3.5 from the next version. If you still need 3.4 support, please let us know (possibly off this list). Unless there are protests, from the end of April, the 3.4 version will remain available, but will not get updated anymore. Unfortunately we don't have resources to keep multiple development baselines alive. -- Regarding Erlang R12: In the same line of thought, we would like to be able to update the minimum supported Erlang version to R12. Actually, we'd gladly skip R12 too and only target R13, but I don't have too high hopes on that one. It's quite painful to have to support three different versions of Erlang (and in practice the result is that if you use R11 or R12, not all results from the erlide tools may be 100% correct if the tool picks the wrong release to check against (yes, unfortunately we still have some bugs :-)). Please let us know if you are still stuck with R11 and use erlide. Hopefully we may take this step at the same time as with the Eclipse version. Thank you very much! best regards, Vlad From ulf.wiger@REDACTED Tue Apr 6 16:04:05 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 06 Apr 2010 16:04:05 +0200 Subject: limit on ets:select/3 Message-ID: <4BBB3F55.6040300@erlang-solutions.com> By chance, I stumbled upon this particular limitation in ets:select/3: ** exception exit: badarg in function ets:select/3 called as ets:select(16400,[{{'$1'},[],['$1']}],628091364) It seems as if the upper bound for the select limit is 134217727, that is, no bignums accepted. It's not a practical problem in my case, and I can feel that it's justified, but is it documented anywhere? BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From waelsamaha@REDACTED Tue Apr 6 16:19:10 2010 From: waelsamaha@REDACTED (Eng. Wael Samaha) Date: Tue, 6 Apr 2010 07:19:10 -0700 (PDT) Subject: Application Start Type Message-ID: <292625.85508.qm@web62205.mail.re1.yahoo.com> Dear all, ??? Is there a way to know the start type for currently started applications?. I'm asking because I have a live system with applications started on top of it. Some of these application started as temporary and others started as permanent. I need now to know only which are started as permanent to stop and re-start again as temporary. I know I can stop all applications and start them again with the default type (temporary), but I'm seeking for a way to only stop those are needed to be stopped. Regards, Wael From hd2010@REDACTED Tue Apr 6 18:43:55 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Tue, 06 Apr 2010 18:43:55 +0200 Subject: [erlang-questions] limit on ets:select/3 In-Reply-To: <4BBB3F55.6040300@erlang-solutions.com> References: <4BBB3F55.6040300@erlang-solutions.com> Message-ID: <4BBB64CB.5070107@eonblast.com> I tried a couple of searches for a bit, looks like No to me. Just for curiosity, what is the use of the limit you tried? Henning Ulf Wiger wrote: > > By chance, I stumbled upon this particular limitation in ets:select/3: > > ** exception exit: badarg > in function ets:select/3 > called as ets:select(16400,[{{'$1'},[],['$1']}],628091364) > > It seems as if the upper bound for the select limit is 134217727, > that is, no bignums accepted. > > It's not a practical problem in my case, and I can feel that it's > justified, but is it documented anywhere? > > BR, > Ulf W From david.rowe@REDACTED Tue Apr 6 19:11:22 2010 From: david.rowe@REDACTED (David Rowe) Date: Tue, 06 Apr 2010 18:11:22 +0100 Subject: OS X Trace/BPT trap error loading linked in driver Message-ID: I am writing a linked-in driver on OS X. The driver relies on some of the OS X API "framework" libraries. The problem is that when I call erl_ddll:load_driver, there is a Trace/BPT trap error and everything dies. If I look at the crash report in Console I see this: Exception Type: EXC_BREAKPOINT (SIGTRAP) Exception Codes: 0x0000000000000002, 0x0000000000000000 Crashed Thread: 5 Thread 5 Crashed: 0 com.apple.CoreFoundation 0x00007fff867811c0 __CFInitialize + 1808 1 dyld 0x00007fff5fc0d5ce ImageLoaderMachO::doImageInit(ImageLoader::LinkContext const&) + 138 2 dyld 0x00007fff5fc0d607 ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 27 3 dyld 0x00007fff5fc0bcec ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int) + 236 4 dyld 0x00007fff5fc0bc9d ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int) + 157 5 dyld 0x00007fff5fc0bc9d ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int) + 157 6 dyld 0x00007fff5fc0bc9d ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int) + 157 7 dyld 0x00007fff5fc0bda6 ImageLoader::runInitializers(ImageLoader::LinkContext const&) + 58 8 dyld 0x00007fff5fc08fbb dlopen + 573 9 libSystem.B.dylib 0x00007fff831a7720 dlopen + 61 10 beam.smp 0x000000001011c4ef erts_sys_ddll_open_noext + 47 (erl_unix_sys_ddll.c:129) I'm pretty sure that the Exception is being caused by this line in the __CFInitialize function: if (!pthread_main_np()) HALT; // CoreFoundation must be initialized on the main thread So my question is, does anyone know how to get Erlang to load my driver on its main thread? Or is there any kind of workaround? The only options I can see at the moment are rewriting the driver as a "c node" (possibly too slow) or moving to Linux? Thanks, David Rowe From ulf.wiger@REDACTED Tue Apr 6 19:51:55 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 06 Apr 2010 19:51:55 +0200 Subject: [erlang-questions] limit on ets:select/3 In-Reply-To: <4BBB64CB.5070107@eonblast.com> References: <4BBB3F55.6040300@erlang-solutions.com> <4BBB64CB.5070107@eonblast.com> Message-ID: <4BBB74BB.8000601@erlang-solutions.com> Henning Diedrich wrote: > I tried a couple of searches for a bit, looks like No to me. > > Just for curiosity, what is the use of the limit you tried? No use, in fact. I called a function to pull records out of a table (sorted on timestamp) that would fit inside a frequency window. The first time called, LastTime was 0, and now() was a timestamp in microseconds. The assumption was that the two timestamps would otherwise be pretty close, but I figured it wasn't worth plugging the hole, since LastTime = 0 would imply that the table was empty anyway. It didn't occur to me that select(EmptyTab, Pat, BigNum) would crash. Like I said, I stumbled upon it by chance. :) BR, Ulf W > Henning > > Ulf Wiger wrote: >> >> By chance, I stumbled upon this particular limitation in ets:select/3: >> >> ** exception exit: badarg >> in function ets:select/3 >> called as ets:select(16400,[{{'$1'},[],['$1']}],628091364) >> >> It seems as if the upper bound for the select limit is 134217727, >> that is, no bignums accepted. >> >> It's not a practical problem in my case, and I can feel that it's >> justified, but is it documented anywhere? >> >> BR, >> Ulf W > > -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From acton@REDACTED Wed Apr 7 00:07:00 2010 From: acton@REDACTED (Brian Acton) Date: Tue, 6 Apr 2010 15:07:00 -0700 Subject: more try/catch confusion Message-ID: Hi guys, I recently went down the road of doing some xml parsing. Using the xmerl libraries, I tried writing the following construct: 1> try xmerl_scan:file("broken.xml") catch H -> H end. 2404- fatal: unexpected_end ** exception exit: {fatal, {unexpected_end,{file,"broken.xml"},{line,92},{col,4}}} I then tried this construct 2> try xmerl_scan:file("broken.xml") of I -> I catch J -> J end. 2404- fatal: unexpected_end ** exception exit: {fatal, {unexpected_end,{file,"broken.xml"},{line,92},{col,4}}} I finally got this to work: 3> case catch xmerl_scan:file("broken.xml") of G -> G end. 2404- fatal: unexpected_end {'EXIT',{fatal,{unexpected_end,{file,"broken.xml"}, {line,92}, {col,4}}}} Now, the question is why do #1 and #2 fail while #3 succeeds? I would have expected both #1 and #2 to have caught the exception. --b From vinoski@REDACTED Wed Apr 7 00:21:38 2010 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 6 Apr 2010 18:21:38 -0400 Subject: [erlang-questions] more try/catch confusion In-Reply-To: References: Message-ID: On Tue, Apr 6, 2010 at 6:07 PM, Brian Acton wrote: > Hi guys, > > I recently went down the road of doing some xml parsing. Using the xmerl > libraries, I tried writing the following construct: > > 1> try xmerl_scan:file("broken.xml") catch H -> H end. You're missing the exception type part of your catch. If you leave it out, it defaults to throw, but this isn't a throw, it's an exit. Use this instead: try xmerl_scan:file("broken.xml") catch exit:H -> H end. or try xmerl_scan:file("broken.xml") catch _:H -> H end. --steve From kagato@REDACTED Wed Apr 7 00:22:29 2010 From: kagato@REDACTED (Jayson Vantuyl) Date: Tue, 6 Apr 2010 15:22:29 -0700 Subject: [erlang-questions] more try/catch confusion In-Reply-To: References: Message-ID: <25137263-B7BC-4CF0-B3A4-F9C128214FBE@souja.net> There are three types of exceptions in Erlang: * exits * errors * throws The old "catch" keyword catches all three. The "try" keyword can differentiate the three. The catch pattern is usually of the form A:B, where A is one of error, exit, or throw. When you don't specify the type of exception, it defaults to "throw". You're getting an exit and the first two default to only catching throw exceptions, hence only the old-style "catch" is really catching the exception. On Apr 6, 2010, at 3:07 PM, Brian Acton wrote: > Hi guys, > > I recently went down the road of doing some xml parsing. Using the xmerl > libraries, I tried writing the following construct: > > 1> try xmerl_scan:file("broken.xml") catch H -> H end. > 2404- fatal: unexpected_end > ** exception exit: {fatal, > > {unexpected_end,{file,"broken.xml"},{line,92},{col,4}}} > > I then tried this construct > > 2> try xmerl_scan:file("broken.xml") of I -> I catch J -> J end. > 2404- fatal: unexpected_end > ** exception exit: {fatal, > > {unexpected_end,{file,"broken.xml"},{line,92},{col,4}}} > > I finally got this to work: > > 3> case catch xmerl_scan:file("broken.xml") of G -> G end. > 2404- fatal: unexpected_end > {'EXIT',{fatal,{unexpected_end,{file,"broken.xml"}, > {line,92}, > {col,4}}}} > > Now, the question is why do #1 and #2 fail while #3 succeeds? I would have > expected both #1 and #2 to have caught the exception. > > --b -- Jayson Vantuyl kagato@REDACTED From dangud@REDACTED Wed Apr 7 00:40:32 2010 From: dangud@REDACTED (Dan Gudmundsson) Date: Wed, 7 Apr 2010 00:40:32 +0200 Subject: [erlang-questions] OS X Trace/BPT trap error loading linked in driver In-Reply-To: References: Message-ID: It should work anyway, it does for wx, see how I initialize the graphics in wx. It complains but it works. /Dan On Tue, Apr 6, 2010 at 7:11 PM, David Rowe wrote: > I am writing a linked-in driver on OS X. The driver relies on some of the OS > X API "framework" libraries. The problem is that when I call > erl_ddll:load_driver, there is a Trace/BPT trap error and everything dies. > > If I look at the crash report in Console I see this: > > Exception Type: ?EXC_BREAKPOINT (SIGTRAP) > Exception Codes: 0x0000000000000002, 0x0000000000000000 > Crashed Thread: ?5 > > Thread 5 Crashed: > 0 ? com.apple.CoreFoundation ? ? ? ? ?0x00007fff867811c0 __CFInitialize + > 1808 > 1 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0d5ce > ImageLoaderMachO::doImageInit(ImageLoader::LinkContext const&) + 138 > 2 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0d607 > ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 27 > 3 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bcec > ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, > unsigned int) + 236 > 4 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d > ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, > unsigned int) + 157 > 5 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d > ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, > unsigned int) + 157 > 6 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d > ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, > unsigned int) + 157 > 7 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bda6 > ImageLoader::runInitializers(ImageLoader::LinkContext const&) + 58 > 8 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc08fbb dlopen + 573 > 9 ? libSystem.B.dylib ? ? ? ? ? ? ? ? 0x00007fff831a7720 dlopen + 61 > 10 ?beam.smp ? ? ? ? ? ? ? ? ? ? ? ? ?0x000000001011c4ef > erts_sys_ddll_open_noext + 47 (erl_unix_sys_ddll.c:129) > > > I'm pretty sure that the Exception is being caused by this line in the > __CFInitialize function: > > if (!pthread_main_np()) HALT; ? ?// CoreFoundation must be initialized on > the main thread > > > So my question is, does anyone know how to get Erlang to load my driver on > its main thread? Or is there any kind of workaround? The only options I can > see at the moment are rewriting the driver as a "c node" (possibly too slow) > or moving to Linux? > > Thanks, > > David Rowe > From ok@REDACTED Wed Apr 7 08:36:23 2010 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 7 Apr 2010 18:36:23 +1200 Subject: [erlang-questions] Variable naming conventions In-Reply-To: References: Message-ID: <4A7DD9F6-A14F-4ED3-A3C7-7FBBA5A1E346@cs.otago.ac.nz> On Apr 5, 2010, at 3:26 AM, Garrett Smith wrote: (a) Multiple "versions" of a value named in the same scope. I can tell you something worse that numeric suffixes, and that's *no* annotation, so that you have to trace through the control flow to find out which version you are looking at. > handle_call(_, #state{name=Name}=State) -> > UpdatedName = strip_white_space(Name), > {noreply, State#state{name=UpdatedName)}}. This can be written as handle_call(_, State) -> {noreply, State#state{name = strip_white_space(State#State.name)}. You could even refactor it as state_sans_white_space_in_name(State = #state{name = Name})) -> State#state{name = strip_white_space(Name)}. handle_call(_, State) -> {noreply, state_sans_white_space_in_name(State)}. The real problem of repetition here is State, state, State, State name, Name, name, So here is a general pattern: we have a variable V which is an instance of record R and we want to update field F from old value O to new value N. -define(update(V,R,F,O,N), (#R{F=O}=V, V#R{F=N})). handle_call(_, State) -> {noreply, update(State, state,name, O, strip_white_space(O))}. Or a higher order function: update@REDACTED(F, State = #state{name = Old}) when is_function(F, 1) -> State#state{name = F(Old)}. handle_call(_, State) -> {noreply, update@REDACTED(fun strip_white_space/1, State)}. > The other frustrating case -- and far more so -- is something like > this: > > Name = case get_name() of > undefined -> ""; > {ok, NameValueReturnedByGetName} -> NameValueReturnedByGetName > end > > (I'm calling this "leakage" because I'd probably just call it Name if > it didn't leak. While that'd be clunky, it at least says what I think > it should.) But you can perfectly well write case get_name() of undefined -> Name = "" ; {ok, Name} -> ok end There is some feeling in the Erlang community against exporting variable bindings from 'case'. I reckon sometimes it's a good idea, sometimes not. In cases like this where you are supplying a "default" for a function that didn't have one, again, write a function. get_name(Default_Name) -> case get_name() of undefined -> Default_Name ; {ok, Name} -> Name end. and then use Name = get_name("") Presto chango, no leakage. If you like longer names, get_name_with_default/1 might be clearer. Or if you have several functions with a common undefined | {ok, X} result type, you could use with_default(undefined, Default) -> Default; with_default({ok, Value}, _) -> Value. Name = with_default(get_name(), "") > I know I can refactor this into a function and solve the problem, but > of course there are cases where I don't want to. So of course the best thing to do is to show us one of those cases. There's an "old" Erlang style that goes back before Erlang had funs at all. (And there were good reasons to stay with a first-order language.) With the introduction of funs, it is possible to move closer to an ML or Haskell style. From sverker@REDACTED Wed Apr 7 11:12:02 2010 From: sverker@REDACTED (Sverker Eriksson) Date: Wed, 07 Apr 2010 11:12:02 +0200 Subject: [erlang-questions] limit on ets:select/3 In-Reply-To: <4BBB3F55.6040300@erlang-solutions.com> References: <4BBB3F55.6040300@erlang-solutions.com> Message-ID: <4BBC4C62.4090903@erix.ericsson.se> Ulf Wiger wrote: > > By chance, I stumbled upon this particular limitation in ets:select/3: > > ** exception exit: badarg > in function ets:select/3 > called as ets:select(16400,[{{'$1'},[],['$1']}],628091364) > > It seems as if the upper bound for the select limit is 134217727, > that is, no bignums accepted. > > It's not a practical problem in my case, and I can feel that it's > justified, but is it documented anywhere? Yes, the BIF (ets_select_3) throws badarg if Limit is a bignum. The reference documentation does not mention any bounds on Limit. Not sure what the best solution is: 1. Document the current behavior, an upper bound that depends on architecture (32 or 64 bit). 2. Enforce the same upper bound on 64-bit as the current on 32-bit and document that. 3. Remove the badarg and instead continue with the upper bound as Limit and document that. 4. Combine 2&3. 5. Any other ideas... /Sverker, Erlang/OTP From ulf.wiger@REDACTED Wed Apr 7 11:35:23 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 07 Apr 2010 11:35:23 +0200 Subject: [erlang-questions] limit on ets:select/3 In-Reply-To: <4BBC4C62.4090903@erix.ericsson.se> References: <4BBB3F55.6040300@erlang-solutions.com> <4BBC4C62.4090903@erix.ericsson.se> Message-ID: <4BBC51DB.3030102@erlang-solutions.com> Sverker Eriksson wrote: > Yes, the BIF (ets_select_3) throws badarg if Limit is a bignum. The > reference documentation does not mention any bounds on Limit. > > Not sure what the best solution is: > 1. Document the current behavior, an upper bound that depends on > architecture (32 or 64 bit). > 2. Enforce the same upper bound on 64-bit as the current on 32-bit and > document that. > 3. Remove the badarg and instead continue with the upper bound as Limit > and document that. > 4. Combine 2&3. > 5. Any other ideas... I vote for (1). I doubt that people need to use limits in the bignum range, and if they do, it had better be on a 64-bit machine anyway - as trying to create a list of hundreds of million objects on the process heap in a single operation is likely to be fatal. The trivial workaround is to make multiple calls to select/3, and the overhead of doing so will be entirely insignificant compared to the cost of copying millions of objects onto the heap anyway. A similar limit exists on receive timeouts, and there it is much more likely that one would like to exceed the limit. However, most people seem to accept the current situation as adding more checks will introduce overhead in the vast number of cases where the limit is not a problem. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From david.rowe@REDACTED Wed Apr 7 15:32:20 2010 From: david.rowe@REDACTED (David Rowe) Date: Wed, 07 Apr 2010 14:32:20 +0100 Subject: [erlang-questions] OS X Trace/BPT trap error loading linked in driver In-Reply-To: Message-ID: But does it though? I can't tell: I'm using 64-bit Snow Leopard, so I get "Can not combine 64bits erlang with wxWidgets on MacOSX" when I compile R13B04. I might well be wrong, but if I'm right I can't see anyway around that pthread_main_np() call. It will always return 0, because Erlang will always load a linked-in driver on a thread other than the main thread, won't it? David On 06/04/2010 23:40, "Dan Gudmundsson" wrote: > It should work anyway, it does for wx, see how I initialize the graphics in > wx. > It complains but it works. > > /Dan > > On Tue, Apr 6, 2010 at 7:11 PM, David Rowe wrote: >> I am writing a linked-in driver on OS X. The driver relies on some of the OS >> X API "framework" libraries. The problem is that when I call >> erl_ddll:load_driver, there is a Trace/BPT trap error and everything dies. >> >> If I look at the crash report in Console I see this: >> >> Exception Type: ?EXC_BREAKPOINT (SIGTRAP) >> Exception Codes: 0x0000000000000002, 0x0000000000000000 >> Crashed Thread: ?5 >> >> Thread 5 Crashed: >> 0 ? com.apple.CoreFoundation ? ? ? ? ?0x00007fff867811c0 __CFInitialize + >> 1808 >> 1 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0d5ce >> ImageLoaderMachO::doImageInit(ImageLoader::LinkContext const&) + 138 >> 2 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0d607 >> ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 27 >> 3 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bcec >> ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, >> unsigned int) + 236 >> 4 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d >> ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, >> unsigned int) + 157 >> 5 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d >> ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, >> unsigned int) + 157 >> 6 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d >> ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, >> unsigned int) + 157 >> 7 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bda6 >> ImageLoader::runInitializers(ImageLoader::LinkContext const&) + 58 >> 8 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc08fbb dlopen + 573 >> 9 ? libSystem.B.dylib ? ? ? ? ? ? ? ? 0x00007fff831a7720 dlopen + 61 >> 10 ?beam.smp ? ? ? ? ? ? ? ? ? ? ? ? ?0x000000001011c4ef >> erts_sys_ddll_open_noext + 47 (erl_unix_sys_ddll.c:129) >> >> >> I'm pretty sure that the Exception is being caused by this line in the >> __CFInitialize function: >> >> if (!pthread_main_np()) HALT; ? ?// CoreFoundation must be initialized on >> the main thread >> >> >> So my question is, does anyone know how to get Erlang to load my driver on >> its main thread? Or is there any kind of workaround? The only options I can >> see at the moment are rewriting the driver as a "c node" (possibly too slow) >> or moving to Linux? >> >> Thanks, >> >> David Rowe >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From dangud@REDACTED Wed Apr 7 17:14:59 2010 From: dangud@REDACTED (Dan Gudmundsson) Date: Wed, 7 Apr 2010 17:14:59 +0200 Subject: [erlang-questions] OS X Trace/BPT trap error loading linked in driver In-Reply-To: References: Message-ID: I don't know, wxWidgets-2.8 uses carbon, which sets the 32 bit requirement. I havn't tried anything else. Erlang calls the driver from a scheduler thread, if the "main" thread is the first started thread then it will not be called from there, that thread is used for other things. /Dan 2010/4/7 David Rowe : > But does it though? I can't tell: I'm using 64-bit Snow Leopard, so I get > "Can not combine 64bits erlang with wxWidgets on MacOSX" when I compile > R13B04. > > I might well be wrong, but if I'm right I can't see anyway around that > pthread_main_np() call. It will always return 0, because Erlang will always > load a linked-in driver on a thread other than the main thread, won't it? > > David > > > > > On 06/04/2010 23:40, "Dan Gudmundsson" wrote: > >> It should work anyway, it does for wx, see how I initialize the graphics in >> wx. >> It complains but it works. >> >> /Dan >> >> On Tue, Apr 6, 2010 at 7:11 PM, David Rowe wrote: >>> I am writing a linked-in driver on OS X. The driver relies on some of the OS >>> X API "framework" libraries. The problem is that when I call >>> erl_ddll:load_driver, there is a Trace/BPT trap error and everything dies. >>> >>> If I look at the crash report in Console I see this: >>> >>> Exception Type: ?EXC_BREAKPOINT (SIGTRAP) >>> Exception Codes: 0x0000000000000002, 0x0000000000000000 >>> Crashed Thread: ?5 >>> >>> Thread 5 Crashed: >>> 0 ? com.apple.CoreFoundation ? ? ? ? ?0x00007fff867811c0 __CFInitialize + >>> 1808 >>> 1 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0d5ce >>> ImageLoaderMachO::doImageInit(ImageLoader::LinkContext const&) + 138 >>> 2 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0d607 >>> ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 27 >>> 3 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bcec >>> ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, >>> unsigned int) + 236 >>> 4 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d >>> ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, >>> unsigned int) + 157 >>> 5 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d >>> ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, >>> unsigned int) + 157 >>> 6 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d >>> ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, >>> unsigned int) + 157 >>> 7 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bda6 >>> ImageLoader::runInitializers(ImageLoader::LinkContext const&) + 58 >>> 8 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc08fbb dlopen + 573 >>> 9 ? libSystem.B.dylib ? ? ? ? ? ? ? ? 0x00007fff831a7720 dlopen + 61 >>> 10 ?beam.smp ? ? ? ? ? ? ? ? ? ? ? ? ?0x000000001011c4ef >>> erts_sys_ddll_open_noext + 47 (erl_unix_sys_ddll.c:129) >>> >>> >>> I'm pretty sure that the Exception is being caused by this line in the >>> __CFInitialize function: >>> >>> if (!pthread_main_np()) HALT; ? ?// CoreFoundation must be initialized on >>> the main thread >>> >>> >>> So my question is, does anyone know how to get Erlang to load my driver on >>> its main thread? Or is there any kind of workaround? The only options I can >>> see at the moment are rewriting the driver as a "c node" (possibly too slow) >>> or moving to Linux? >>> >>> Thanks, >>> >>> David Rowe >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From paul.joseph.davis@REDACTED Wed Apr 7 17:41:18 2010 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Wed, 7 Apr 2010 11:41:18 -0400 Subject: [erlang-questions] OS X Trace/BPT trap error loading linked in driver In-Reply-To: References: Message-ID: On Tue, Apr 6, 2010 at 1:11 PM, David Rowe wrote: > I am writing a linked-in driver on OS X. The driver relies on some of the OS > X API "framework" libraries. The problem is that when I call > erl_ddll:load_driver, there is a Trace/BPT trap error and everything dies. > > If I look at the crash report in Console I see this: > > Exception Type: ?EXC_BREAKPOINT (SIGTRAP) > Exception Codes: 0x0000000000000002, 0x0000000000000000 > Crashed Thread: ?5 > > Thread 5 Crashed: > 0 ? com.apple.CoreFoundation ? ? ? ? ?0x00007fff867811c0 __CFInitialize + > 1808 > 1 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0d5ce > ImageLoaderMachO::doImageInit(ImageLoader::LinkContext const&) + 138 > 2 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0d607 > ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 27 > 3 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bcec > ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, > unsigned int) + 236 > 4 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d > ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, > unsigned int) + 157 > 5 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d > ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, > unsigned int) + 157 > 6 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d > ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, > unsigned int) + 157 > 7 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bda6 > ImageLoader::runInitializers(ImageLoader::LinkContext const&) + 58 > 8 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc08fbb dlopen + 573 > 9 ? libSystem.B.dylib ? ? ? ? ? ? ? ? 0x00007fff831a7720 dlopen + 61 > 10 ?beam.smp ? ? ? ? ? ? ? ? ? ? ? ? ?0x000000001011c4ef > erts_sys_ddll_open_noext + 47 (erl_unix_sys_ddll.c:129) > > > I'm pretty sure that the Exception is being caused by this line in the > __CFInitialize function: > > if (!pthread_main_np()) HALT; ? ?// CoreFoundation must be initialized on > the main thread > > > So my question is, does anyone know how to get Erlang to load my driver on > its main thread? Or is there any kind of workaround? The only options I can > see at the moment are rewriting the driver as a "c node" (possibly too slow) > or moving to Linux? > > Thanks, > > David Rowe > David, I ran into that same issue when attempting to link against Spidermonkey in a driver. At the time I found another report [1] that found exactly what you pointed out, loading CoreFoundation halts the process when it wasn't initialized on the main thread. In my case I could get away with just making sure that nothing touched the CoreFoundation classes, while in your case that doesn't appear to be possible. The only thing I could think of would be to patch Erlang to do whatever CoreFoundation initializations is needed. HTH, Paul Davis [1] http://openradar.appspot.com/7209349 From david.rowe@REDACTED Wed Apr 7 18:21:30 2010 From: david.rowe@REDACTED (David Rowe) Date: Wed, 07 Apr 2010 17:21:30 +0100 Subject: [erlang-questions] OS X Trace/BPT trap error loading linked in driver In-Reply-To: Message-ID: Thanks everyone. I think the Open Radar report is pretty conclusive :-( David On 07/04/2010 16:41, "Paul Davis" wrote: > On Tue, Apr 6, 2010 at 1:11 PM, David Rowe wrote: >> I am writing a linked-in driver on OS X. The driver relies on some of the OS >> X API "framework" libraries. The problem is that when I call >> erl_ddll:load_driver, there is a Trace/BPT trap error and everything dies. >> >> If I look at the crash report in Console I see this: >> >> Exception Type: ?EXC_BREAKPOINT (SIGTRAP) >> Exception Codes: 0x0000000000000002, 0x0000000000000000 >> Crashed Thread: ?5 >> >> Thread 5 Crashed: >> 0 ? com.apple.CoreFoundation ? ? ? ? ?0x00007fff867811c0 __CFInitialize + >> 1808 >> 1 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0d5ce >> ImageLoaderMachO::doImageInit(ImageLoader::LinkContext const&) + 138 >> 2 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0d607 >> ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 27 >> 3 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bcec >> ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, >> unsigned int) + 236 >> 4 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d >> ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, >> unsigned int) + 157 >> 5 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d >> ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, >> unsigned int) + 157 >> 6 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bc9d >> ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, >> unsigned int) + 157 >> 7 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc0bda6 >> ImageLoader::runInitializers(ImageLoader::LinkContext const&) + 58 >> 8 ? dyld ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x00007fff5fc08fbb dlopen + 573 >> 9 ? libSystem.B.dylib ? ? ? ? ? ? ? ? 0x00007fff831a7720 dlopen + 61 >> 10 ?beam.smp ? ? ? ? ? ? ? ? ? ? ? ? ?0x000000001011c4ef >> erts_sys_ddll_open_noext + 47 (erl_unix_sys_ddll.c:129) >> >> >> I'm pretty sure that the Exception is being caused by this line in the >> __CFInitialize function: >> >> if (!pthread_main_np()) HALT; ? ?// CoreFoundation must be initialized on >> the main thread >> >> >> So my question is, does anyone know how to get Erlang to load my driver on >> its main thread? Or is there any kind of workaround? The only options I can >> see at the moment are rewriting the driver as a "c node" (possibly too slow) >> or moving to Linux? >> >> Thanks, >> >> David Rowe >> > > David, > > I ran into that same issue when attempting to link against > Spidermonkey in a driver. At the time I found another report [1] that > found exactly what you pointed out, loading CoreFoundation halts the > process when it wasn't initialized on the main thread. In my case I > could get away with just making sure that nothing touched the > CoreFoundation classes, while in your case that doesn't appear to be > possible. The only thing I could think of would be to patch Erlang to > do whatever CoreFoundation initializations is needed. > > HTH, > Paul Davis > > [1] http://openradar.appspot.com/7209349 > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From kaiduanx@REDACTED Wed Apr 7 19:39:09 2010 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 7 Apr 2010 13:39:09 -0400 Subject: Idea on stateful server load sharing & fail over Message-ID: Hi, all, Consider the following case, a system consists of a farm of state-full call servers, and needs to support massive number of users, and zero down time or at least 5-9 availability. The number of servers are dynamic, in other words, server can come up and go away. The server is a state-full server, it stores information of the call, the state should be able to survive server crash so that in-call feature can be supported after crash. The system has two tiers architecture. At the front, a dispatcher dispatches the incoming request to the back end state-full servers, the dispatcher is stateless. So the following questions comes, 1. How to dispatch the incoming request to back end servers? Please note that the back-end server is a state-full server, for example, all requests in a call should be dispatched to the same server. The call is identified with a call id that is a random string. Also the back end server can be added or removed dynamically. 2. How to replicate the call state information among back end servers? 3. How front end server detects a back end server is down? How back end servers detects that one of their peer is down? Can we use erlang's monitor in distribution to achieve that? Thanks all for the pointers and thoughts, Kaiduan From mevans@REDACTED Wed Apr 7 20:42:32 2010 From: mevans@REDACTED (Evans, Matthew) Date: Wed, 7 Apr 2010 14:42:32 -0400 Subject: [erlang-questions] Idea on stateful server load sharing & fail over In-Reply-To: References: Message-ID: Hi, Is there any reason why you couldn't have a "back end server" process per user? That would be the Erlang way of doing things. Even if there are millions of users/processes you should be fine. The "back end server" process would save state in its state record (assuming it is a gen_server or gen_fsm), and when it is in a stable state it could checkpoint that information to ets, mnesia or a peer process. You could have an ETS table that maps the call id string to the back end server pid. And use erlang:monitor to determine if the back end server dies. If it does, restart it or map the record in the ets table elsewhere. Matt -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Kaiduan Xie Sent: Wednesday, April 07, 2010 1:39 PM To: erlang-questions Subject: [erlang-questions] Idea on stateful server load sharing & fail over Hi, all, Consider the following case, a system consists of a farm of state-full call servers, and needs to support massive number of users, and zero down time or at least 5-9 availability. The number of servers are dynamic, in other words, server can come up and go away. The server is a state-full server, it stores information of the call, the state should be able to survive server crash so that in-call feature can be supported after crash. The system has two tiers architecture. At the front, a dispatcher dispatches the incoming request to the back end state-full servers, the dispatcher is stateless. So the following questions comes, 1. How to dispatch the incoming request to back end servers? Please note that the back-end server is a state-full server, for example, all requests in a call should be dispatched to the same server. The call is identified with a call id that is a random string. Also the back end server can be added or removed dynamically. 2. How to replicate the call state information among back end servers? 3. How front end server detects a back end server is down? How back end servers detects that one of their peer is down? Can we use erlang's monitor in distribution to achieve that? Thanks all for the pointers and thoughts, Kaiduan ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From kaiduanx@REDACTED Wed Apr 7 21:09:56 2010 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 7 Apr 2010 15:09:56 -0400 Subject: [erlang-questions] Idea on stateful server load sharing & fail over In-Reply-To: References: Message-ID: Thank you Evans for the reply. We are doing process per call instead of process per user because not all users make/receive calls at the same time. We can save all call sate information to shared/distributed database, like Mnesia. There are two issues to be addressed, for example, server A crashes or be removed from the cluster, which server in group will re-construct the gen_gsm/gen_server from the crahsed server A. Saving all call states in all server to distributed database is not optimal also. Another problem as I stated, how to dispatch the request to back end servers. Basic hash is not enough because back end server can come up or goes away. kaiduan On Wed, Apr 7, 2010 at 2:42 PM, Evans, Matthew wrote: > Hi, > > Is there any reason why you couldn't have a "back end server" process per user? That would be the Erlang way of doing things. Even if there are millions of users/processes you should be fine. > > The "back end server" process would save state in its state record (assuming it is a gen_server or gen_fsm), and when it is in a stable state it could checkpoint that information to ets, mnesia or a peer process. > > You could have an ETS table that maps the call id string to the back end server pid. And use erlang:monitor to determine if the back end server dies. If it does, restart it or map the record in the ets table elsewhere. > > Matt > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Kaiduan Xie > Sent: Wednesday, April 07, 2010 1:39 PM > To: erlang-questions > Subject: [erlang-questions] Idea on stateful server load sharing & fail over > > Hi, all, > > Consider the following case, a system consists of a farm of state-full > call servers, and needs to support massive number of users, and zero > down time or at least 5-9 availability. The number of servers are > dynamic, in other words, server can come up and go away. The server is > a state-full server, it stores information of the call, the state > should be able to survive server crash so that in-call feature can be > supported after crash. The system has two tiers architecture. At the > front, a dispatcher dispatches the incoming request to the back end > state-full servers, the dispatcher is stateless. So the following > questions comes, > > 1. How to dispatch the incoming request to back end servers? Please > note that the back-end server is a state-full server, for example, all > requests in a call should be dispatched to the same server. The call > is identified with a call id that is a random string. Also the back > end server can be added or removed dynamically. > > 2. How to replicate the call state information among back end servers? > > 3. How front end server detects a back end server is down? How back > end servers detects that one of their peer is down? Can we use > erlang's monitor in distribution to achieve that? > > Thanks all for the pointers and thoughts, > > Kaiduan > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From waelsamaha@REDACTED Thu Apr 8 01:48:44 2010 From: waelsamaha@REDACTED (Eng. Wael Samaha) Date: Wed, 7 Apr 2010 16:48:44 -0700 (PDT) Subject: Application Start Type Message-ID: <973322.68519.qm@web62205.mail.re1.yahoo.com> I got a way to get the application StartType using the below interface. application_controller:info(). But this module is an internal module within the Kernel application and not documented or recommended to be used externally. From mihai@REDACTED Thu Apr 8 05:23:21 2010 From: mihai@REDACTED (Mihai Balea) Date: Wed, 7 Apr 2010 23:23:21 -0400 Subject: [erlang-questions] Idea on stateful server load sharing & fail over In-Reply-To: References: Message-ID: On Apr 7, 2010, at 3:09 PM, Kaiduan Xie wrote: > Thank you Evans for the reply. > > We are doing process per call instead of process per user because not > all users make/receive calls at the same time. > > We can save all call sate information to shared/distributed database, > like Mnesia. There are two issues to be addressed, for example, server > A crashes or be removed from the cluster, which server in group will > re-construct the gen_gsm/gen_server from the crahsed server A. Saving > all call states in all server to distributed database is not optimal > also. You can simply let your dispatcher decide which backend node to redirect calls in case of node crash. When a node receives a message belonging to a call it doesn't know about, it can simply attempt to look it up in the distributed store and recreate the gen_* process. This assumes the dispatcher will forward packets from a call to the same backend node as long as the call is ongoing and the node is up. It also assumes the state you save in the distributed store is sufficient to recreate the internal state of the gen_* process. Here's an example: let's say call id 42 gets dispatched to node A. Node A updates the distributed call state store whenever needed. At some point node A dies. The dispatcher detects it and decides that call 42 should go to node D from now on, so it starts forwarding call 42 packets to node D. Node starts receiving packets with call id 42, but it has no info about this call. It looks up id 42 in the distributed call state store, finds it, and creates a gen process to handle the call. > > Another problem as I stated, how to dispatch the request to back end > servers. Basic hash is not enough because back end server can come up > or goes away. Your dispatcher needs to be aware of what backend nodes are up and running. You can monitor nodes in a cluster using net_kernel functions but the default tick time is too long for realtime operations like voip call handling. You can increase the tick frequency, but I don't know how feasible it would be to make it fast enough to work for you. Alternatively, you can implement your own heartbeat protocol with a high enough time resolution. Once the dispatcher is made aware of backend nodes coming and going in realtime, then it is a matter of maintaining a table of call ids -> backend nodes and adjusting it on the fly. Hope this helps Mihai From kaiduanx@REDACTED Thu Apr 8 05:43:09 2010 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 7 Apr 2010 23:43:09 -0400 Subject: [erlang-questions] Idea on stateful server load sharing & fail over In-Reply-To: References: Message-ID: Mihai, Thanks for the explanation and excellent example. However, you need to store ALL call information on ALL nodes to the distributed store, this may not be an optimal solution if the number of nodes and number of calls are big. Thanks, kaiduan On Wed, Apr 7, 2010 at 11:23 PM, Mihai Balea wrote: > > On Apr 7, 2010, at 3:09 PM, Kaiduan Xie wrote: > >> Thank you Evans for the reply. >> >> We are doing process per call instead of process per user because not >> all users make/receive calls at the same time. >> >> We can save all call sate information to shared/distributed database, >> like Mnesia. There are two issues to be addressed, for example, server >> A crashes or be removed from the cluster, which server in group will >> re-construct the gen_gsm/gen_server from the crahsed server A. Saving >> all call states in all server to distributed database is not optimal >> also. > > You can simply let your dispatcher decide which backend node to redirect calls in case of node crash. When a node receives a message belonging to a call it doesn't know about, it can simply attempt to look it up in the distributed store and recreate the gen_* process. This assumes the dispatcher will forward packets from a call to the same backend node as long as the call is ongoing and the node is up. It also assumes the state you save in the distributed store is sufficient to recreate the internal state of the gen_* process. > > Here's an example: let's say call id 42 gets dispatched to node A. Node A updates the distributed call state store whenever needed. At some point node A dies. The dispatcher detects it and decides that call 42 should go to node D from now on, so it starts forwarding call 42 packets to node D. Node starts receiving packets with call id 42, but it has no info about this call. It looks up id 42 in the distributed call state store, finds it, and creates a gen process to handle the call. > >> >> Another problem as I stated, how to dispatch the request to back end >> servers. Basic hash is not enough because back end server can come up >> or goes away. > > Your dispatcher needs to be aware of what backend nodes are up and running. > You can monitor nodes in a cluster using net_kernel functions but the default tick time is too long for realtime operations like voip call handling. You can increase the tick frequency, but I don't know how feasible it would be to make it fast enough to work for you. Alternatively, you can implement your own heartbeat protocol with a high enough time resolution. > Once the dispatcher is made aware of backend nodes coming and going in realtime, then it is a matter of maintaining a table of call ids -> backend nodes and adjusting it on the fly. > > Hope this helps > > Mihai From max.lapshin@REDACTED Thu Apr 8 11:51:46 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 8 Apr 2010 13:51:46 +0400 Subject: Linear scaling on multicore Message-ID: Hi. I've made benchmarks of erlyvideo yesterday. Largest load was 1800 clients, totally consuming about 700 MBit live video stream from one server. Erlyvideo used 450% of CPU for it (5 cores). I want to thank all Erlang team, for such wonderful platform: I've got full linear scaling on multicore computer without any problems, from 150 up to 1800 clients, it took 0,25% CPU per client and not more. Here is graphic of load: http://erlyvideo.org/erlyvideo-load.png Thank you! From rapsey@REDACTED Thu Apr 8 12:16:50 2010 From: rapsey@REDACTED (Rapsey) Date: Thu, 8 Apr 2010 12:16:50 +0200 Subject: [erlang-questions] Linear scaling on multicore In-Reply-To: References: Message-ID: If you have the will and time, you can also write a linked-in driver for stream output to clients. It requires abandoning gen_tcp and writing your own socket stuff. We can saturate a gigabit connection on a 600$ mac mini with about half CPU utilization. Sergej On Thu, Apr 8, 2010 at 11:51 AM, Max Lapshin wrote: > Hi. I've made benchmarks of erlyvideo yesterday. Largest load was 1800 > clients, totally consuming about 700 MBit live > video stream from one server. Erlyvideo used 450% of CPU for it (5 cores). > > I want to thank all Erlang team, for such wonderful platform: I've got > full linear scaling on multicore computer without any problems, > from 150 up to 1800 clients, it took 0,25% CPU per client and not more. > > Here is graphic of load: http://erlyvideo.org/erlyvideo-load.png > > Thank you! > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From max.lapshin@REDACTED Thu Apr 8 12:21:02 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 8 Apr 2010 14:21:02 +0400 Subject: [erlang-questions] Linear scaling on multicore In-Reply-To: References: Message-ID: On Thu, Apr 8, 2010 at 2:16 PM, Rapsey wrote: > If you have the will and time, you can also write a linked-in driver for > stream output to clients. It requires abandoning gen_tcp and writing your > own socket stuff. > We can saturate a gigabit connection on a 600$ mac mini with about half CPU > utilization. > Understand, I will take a look. From zeno490@REDACTED Thu Apr 8 16:46:07 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Thu, 8 Apr 2010 10:46:07 -0400 Subject: [erlang-questions] Idea on stateful server load sharing & fail over In-Reply-To: References: Message-ID: IMO, with a setup like that, you are very likely to bump against scalability issues. You'll have to save a TON of information, outside the nodes that will use it thus increasing latency as well. While this might be fine for thousands or even a few millions, if you plan on having >5 million simultaneous calls, you'll probably see performance degrade quite rapidly. Another alternative would be to run small groups of nodes in mirror: Nodes A,B,C could handle call X Nodes D,E,F could handle call Y Nodes A,B,F could handle call Z etc If say node A goes down, you could resume call X on nodes B,C and call Z on nodes B,F. Because all nodes have identical state, any one of them can pick up where the others left. Simply scatter the nodes accross different OS processes, different computers and you should be fine. When you initiate a call, simply pick an algorithm that chooses at least 2 physically different computers, and optionally any number of nodes on those computers to handle it. I use a similar system at home. One of the nodes handling the request, will be the master (the one that gets to reply) but all nodes do the work. So from outside, you simply broadcast the information send to all handling nodes. If a slave goes down, it doesn't matter, nothing is affected. If the master goes down, the slaves can elect a new master and it'll resume/reply with no loss of information. With the scenario above, you still have a single point of failure where the process handling the call still has to "post" the information to the distributed data cache, if it fails before it can post, that call will have to be dropped. Not so with a scenario like mine above. Obviously, it'll highly depend on your exact scenario but yeah, your mileage may vary. 2cents, Nicholas On Wed, Apr 7, 2010 at 11:43 PM, Kaiduan Xie wrote: > Mihai, > > Thanks for the explanation and excellent example. However, you need to > store ALL call information on ALL nodes to the distributed store, this > may not be an optimal solution if the number of nodes and number of > calls are big. > > Thanks, > > kaiduan > > On Wed, Apr 7, 2010 at 11:23 PM, Mihai Balea wrote: > > > > On Apr 7, 2010, at 3:09 PM, Kaiduan Xie wrote: > > > >> Thank you Evans for the reply. > >> > >> We are doing process per call instead of process per user because not > >> all users make/receive calls at the same time. > >> > >> We can save all call sate information to shared/distributed database, > >> like Mnesia. There are two issues to be addressed, for example, server > >> A crashes or be removed from the cluster, which server in group will > >> re-construct the gen_gsm/gen_server from the crahsed server A. Saving > >> all call states in all server to distributed database is not optimal > >> also. > > > > You can simply let your dispatcher decide which backend node to redirect > calls in case of node crash. When a node receives a message belonging to a > call it doesn't know about, it can simply attempt to look it up in the > distributed store and recreate the gen_* process. This assumes the > dispatcher will forward packets from a call to the same backend node as long > as the call is ongoing and the node is up. It also assumes the state you > save in the distributed store is sufficient to recreate the internal state > of the gen_* process. > > > > Here's an example: let's say call id 42 gets dispatched to node A. Node A > updates the distributed call state store whenever needed. At some point node > A dies. The dispatcher detects it and decides that call 42 should go to node > D from now on, so it starts forwarding call 42 packets to node D. Node > starts receiving packets with call id 42, but it has no info about this > call. It looks up id 42 in the distributed call state store, finds it, and > creates a gen process to handle the call. > > > >> > >> Another problem as I stated, how to dispatch the request to back end > >> servers. Basic hash is not enough because back end server can come up > >> or goes away. > > > > Your dispatcher needs to be aware of what backend nodes are up and > running. > > You can monitor nodes in a cluster using net_kernel functions but the > default tick time is too long for realtime operations like voip call > handling. You can increase the tick frequency, but I don't know how feasible > it would be to make it fast enough to work for you. Alternatively, you can > implement your own heartbeat protocol with a high enough time resolution. > > Once the dispatcher is made aware of backend nodes coming and going in > realtime, then it is a matter of maintaining a table of call ids -> backend > nodes and adjusting it on the fly. > > > > Hope this helps > > > > Mihai > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From pablo.platt@REDACTED Thu Apr 8 15:57:49 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Thu, 8 Apr 2010 06:57:49 -0700 (PDT) Subject: connection pooling with sockets Message-ID: <293577.18679.qm@web112615.mail.gq1.yahoo.com> Hi, What is the correct way to build a TCP socket connection pool? Each process will open a TCP socket. 1. N processes each handle a single request. A gen_server will receive requests and if there is no connection free will queue it. 2. N processes each can send several requests over the socket at the same time. Requests have a unique id so the process can reply to the client, if needed when it decode the response. A gen_server will pass requests with round robin. Is it better to use simple processes linked to the gen_server so they will be light weight and faster? In case a process die the gen_server manually restart it. Another option is to use a gen_server per connection process and a supervisor to restart it. Thanks From roberto@REDACTED Thu Apr 8 18:35:08 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 8 Apr 2010 18:35:08 +0200 Subject: boot script from escript file Message-ID: dear list, i'm starting an application in a new erlang node with run_erl like this: run_erl -daemon var/pipe/icrocco@REDACTED var/logs "exec erl +P 1000000 +K true -name icrocco@REDACTED -setcookie srthoius5eoisurtoiunshrt -pa ebin -boot icrocco_rel-0.1" from bash, everything works perfectly. if however i run the same script from an escript file: #!/usr/bin/env escript %% -*- erlang -*- %%! -smp enable debug verbose -mode(compile). -vsn('0.1'). main(["start"]) -> ??? Command = "run_erl -daemon var/pipe/icrocco@REDACTED var/logs \"exec erl +P 1000000 +K true -name icrocco@REDACTED -setcookie srthoius5eoisurtoiunshrt -pa ebin -boot icrocco_rel-0.1\"", ??? os:cmd(Command). in this case the node is not started and i see in the log: ===== ===== LOGGING STARTED Thu Apr 8 18:29:05 CEST 2010 ===== {"init terminating in do_boot",{badarg,[{erlang,list_to_atom,[[<<15 bytes>>,<<1 byte>>,46,98,111,111,116]]},{init,get_boot,2},{init,do_boot,3}]}} (no error logger present) error: "Error in process <0.1.0> with exit value: {badarg,[{erlang,list_to_atom,[[<<15 bytes>>,<<1 byte>>,46,98,111,111,116]]},{init,get_boot,2},{init,do_boot,3}]}\n" anyone can kindly tell me what i am doing wrong? thank you in advance, r. From navaneethanit@REDACTED Thu Apr 8 20:32:52 2010 From: navaneethanit@REDACTED (Saravana) Date: Thu, 8 Apr 2010 11:32:52 -0700 (PDT) Subject: make tabular form Message-ID: How to create a tabular form in erlang, Message = wf:f("***RollNo ~B,\tTamil ~B,\nEnglish ~B,\nMaths ~B, \nScience ~B*****", tuple_to_list(hd(Results))), wf:flash(Message), I am printing those values in a row,just i wanna show them in a tabular form how to do it? From navaneethanit@REDACTED Thu Apr 8 20:46:12 2010 From: navaneethanit@REDACTED (Saravana) Date: Thu, 8 Apr 2010 11:46:12 -0700 (PDT) Subject: ab with erlang and PHP?which is better?? Message-ID: <60936eb4-6f0c-4044-9fff-364d5000d711@b23g2000yqn.googlegroups.com> Here i created the application that is used to show the results from the mysql database by both erlang and PHP ..to measure the performance... just here i used ab benchmark to make it as sample.... PHP application by apache benchmark results: nava@REDACTED:~$ ab -n 1000 -c 5 http://localhost/sample.php This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking localhost (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: Apache/2.2.11 Server Hostname: localhost Server Port: 80 Document Path: /sample.php Document Length: 461 bytes Concurrency Level: 5 Time taken for tests: 0.616 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 713713 bytes HTML transferred: 461461 bytes Requests per second: 1622.77 [#/sec] (mean) Time per request: 3.081 [ms] (mean) Time per request: 0.616 [ms] (mean, across all concurrent requests) Transfer rate: 1131.05 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.2 0 4 Processing: 1 3 2.5 2 44 Waiting: 1 3 2.1 2 44 Total: 1 3 2.5 2 44 Percentage of the requests served within a certain time (ms) 50% 2 66% 3 75% 4 80% 4 90% 5 95% 7 98% 8 99% 11 100% 44 (longest request) Erlang web application by apache benchmark:: nava@REDACTED:~$ ab -n 1000 -c 5 http://localhost:8000/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking localhost (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: inets/5.0.12 Server Hostname: localhost Server Port: 8000 Document Path: / Document Length: 2689 bytes Concurrency Level: 5 Time taken for tests: 6.255 seconds Complete requests: 1000 Failed requests: 962 (Connect: 0, Receive: 0, Length: 962, Exceptions: 0) Write errors: 0 Total transferred: 2982026 bytes HTML transferred: 2696682 bytes Requests per second: 159.88 [#/sec] (mean) Time per request: 31.273 [ms] (mean) Time per request: 6.255 [ms] (mean, across all concurrent requests) Transfer rate: 465.59 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.2 0 4 Processing: 20 31 7.1 31 91 Waiting: 20 31 6.8 30 72 Total: 20 31 7.1 31 91 Percentage of the requests served within a certain time (ms) 50% 31 66% 33 75% 35 80% 36 90% 38 95% 43 98% 49 99% 62 100% 91 (longest request) when we compared both ...incredibly happened..erlang has high response time how? what happened?what went wrong??just give some suggestions??? From mononcqc@REDACTED Thu Apr 8 20:56:18 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 8 Apr 2010 14:56:18 -0400 Subject: [erlang-questions] ab with erlang and PHP?which is better?? In-Reply-To: <60936eb4-6f0c-4044-9fff-364d5000d711@b23g2000yqn.googlegroups.com> References: <60936eb4-6f0c-4044-9fff-364d5000d711@b23g2000yqn.googlegroups.com> Message-ID: You should include the code (or link to it) if you want people to have an idea of what's going wrong. On Thu, Apr 8, 2010 at 2:46 PM, Saravana wrote: > Here i created the application that is used to show the results from > the mysql database by both erlang and PHP ..to measure the > performance... > just here i used ab benchmark to make it as sample.... > > > PHP application by apache benchmark results: > > nava@REDACTED:~$ ab -n 1000 -c 5 http://localhost/sample.php > This is ApacheBench, Version 2.3 <$Revision: 655654 $> > Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ > Licensed to The Apache Software Foundation, http://www.apache.org/ > > Benchmarking localhost (be patient) > Completed 100 requests > Completed 200 requests > Completed 300 requests > Completed 400 requests > Completed 500 requests > Completed 600 requests > Completed 700 requests > Completed 800 requests > Completed 900 requests > Completed 1000 requests > Finished 1000 requests > > > Server Software: Apache/2.2.11 > Server Hostname: localhost > Server Port: 80 > > Document Path: /sample.php > Document Length: 461 bytes > > Concurrency Level: 5 > Time taken for tests: 0.616 seconds > Complete requests: 1000 > Failed requests: 0 > Write errors: 0 > Total transferred: 713713 bytes > HTML transferred: 461461 bytes > Requests per second: 1622.77 [#/sec] (mean) > Time per request: 3.081 [ms] (mean) > Time per request: 0.616 [ms] (mean, across all concurrent > requests) > Transfer rate: 1131.05 [Kbytes/sec] received > > Connection Times (ms) > min mean[+/-sd] median max > Connect: 0 0 0.2 0 4 > Processing: 1 3 2.5 2 44 > Waiting: 1 3 2.1 2 44 > Total: 1 3 2.5 2 44 > > Percentage of the requests served within a certain time (ms) > 50% 2 > 66% 3 > 75% 4 > 80% 4 > 90% 5 > 95% 7 > 98% 8 > 99% 11 > 100% 44 (longest request) > > Erlang web application by apache benchmark:: > > nava@REDACTED:~$ ab -n 1000 -c 5 http://localhost:8000/ > This is ApacheBench, Version 2.3 <$Revision: 655654 $> > Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ > Licensed to The Apache Software Foundation, http://www.apache.org/ > > Benchmarking localhost (be patient) > Completed 100 requests > Completed 200 requests > Completed 300 requests > Completed 400 requests > Completed 500 requests > Completed 600 requests > Completed 700 requests > Completed 800 requests > Completed 900 requests > Completed 1000 requests > Finished 1000 requests > > > Server Software: inets/5.0.12 > Server Hostname: localhost > Server Port: 8000 > > Document Path: / > Document Length: 2689 bytes > > Concurrency Level: 5 > Time taken for tests: 6.255 seconds > Complete requests: 1000 > Failed requests: 962 > (Connect: 0, Receive: 0, Length: 962, Exceptions: 0) > Write errors: 0 > Total transferred: 2982026 bytes > HTML transferred: 2696682 bytes > Requests per second: 159.88 [#/sec] (mean) > Time per request: 31.273 [ms] (mean) > Time per request: 6.255 [ms] (mean, across all concurrent > requests) > Transfer rate: 465.59 [Kbytes/sec] received > > Connection Times (ms) > min mean[+/-sd] median max > Connect: 0 0 0.2 0 4 > Processing: 20 31 7.1 31 91 > Waiting: 20 31 6.8 30 72 > Total: 20 31 7.1 31 91 > > Percentage of the requests served within a certain time (ms) > 50% 31 > 66% 33 > 75% 35 > 80% 36 > 90% 38 > 95% 43 > 98% 49 > 99% 62 > 100% 91 (longest request) > > when we compared both ...incredibly happened..erlang has high response > time how? > what happened?what went wrong??just give some suggestions??? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jeraymond@REDACTED Thu Apr 8 21:03:04 2010 From: jeraymond@REDACTED (Jeremy Raymond) Date: Thu, 8 Apr 2010 15:03:04 -0400 Subject: Increasing maximum number of concurrent processes directly increases memory usage? Message-ID: Hello, If I startup erlang with no arguments (just run erl) it's memory usage as reported by top starts off at around 10 000 bytes. If I start it and increase the maximum number of threads (erl +P 134217727) the memory usage out of the gate is 1.0 gigabytes. Why is this the case? -- Jeremy Raymond From tony@REDACTED Thu Apr 8 21:15:35 2010 From: tony@REDACTED (Tony Arcieri) Date: Thu, 8 Apr 2010 13:15:35 -0600 Subject: [erlang-questions] ab with erlang and PHP?which is better?? In-Reply-To: <60936eb4-6f0c-4044-9fff-364d5000d711@b23g2000yqn.googlegroups.com> References: <60936eb4-6f0c-4044-9fff-364d5000d711@b23g2000yqn.googlegroups.com> Message-ID: On Thu, Apr 8, 2010 at 12:46 PM, Saravana wrote: > Here i created the application that is used to show the results from > the mysql database by both erlang and PHP ..to measure the > performance... > Ok... your PHP results have: Total transferred: 713713 bytes > HTML transferred: 461461 bytes > And the Erlang results: > Total transferred: 2982026 bytes > HTML transferred: 2696682 bytes > Clearly these two applications do not generate the same results. The Erlang application is generating over 4X the amount of output data as the PHP version. Why? -- Tony Arcieri Medioh! A Kudelski Brand From williamstw@REDACTED Thu Apr 8 21:22:06 2010 From: williamstw@REDACTED (Tim Williams) Date: Thu, 8 Apr 2010 15:22:06 -0400 Subject: [erlang-questions] Increasing maximum number of concurrent processes directly increases memory usage? In-Reply-To: References: Message-ID: On Thu, Apr 8, 2010 at 3:03 PM, Jeremy Raymond wrote: > Hello, > > If I startup erlang with no arguments (just run erl) it's memory usage as > reported by top starts off at around 10 000 bytes. If I start it and > increase the maximum number of threads (erl +P 134217727) the memory usage > out of the gate is 1.0 gigabytes. Why is this the case? I'll hazard a guess and say 8 bytes per pid[1] gives you the nearly a gig + the environment itself? --tim [1] - http://www.erlang.org/doc/efficiency_guide/advanced.html From jeraymond@REDACTED Thu Apr 8 21:26:28 2010 From: jeraymond@REDACTED (Jeremy Raymond) Date: Thu, 8 Apr 2010 15:26:28 -0400 Subject: Increasing maximum number of concurrent processes directly increases memory usage? In-Reply-To: References: Message-ID: That math seems to work out. Interesting that Erlang reserves this space at startup. On 4/8/10, Tim Williams wrote: > On Thu, Apr 8, 2010 at 3:03 PM, Jeremy Raymond wrote: >> Hello, >> >> If I startup erlang with no arguments (just run erl) it's memory usage as >> reported by top starts off at around 10 000 bytes. If I start it and >> increase the maximum number of threads (erl +P 134217727) the memory usage >> out of the gate is 1.0 gigabytes. Why is this the case? > > I'll hazard a guess and say 8 bytes per pid[1] gives you the nearly a > gig + the environment itself? > > --tim > > [1] - http://www.erlang.org/doc/efficiency_guide/advanced.html > -- Sent from my mobile device Jeremy Raymond From james.aimonetti@REDACTED Thu Apr 8 21:38:30 2010 From: james.aimonetti@REDACTED (James Aimonetti) Date: Thu, 08 Apr 2010 15:38:30 -0400 Subject: Upgrade to R13B04 Produces Clashes in Code Path Message-ID: <4BBE30B6.2070301@gmail.com> I upgraded to the latest Erlang and was greeted with 1600+ clashes in the code paths. Installed from src, vanilla configure/make/make install. $> erl Erlang R13B04 (erts-5.7.5) [source] [64-bit] [rq:1] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> code:clash(). ... ... ** /usr/local/lib/erlang/lib/appmon-2.1.11/ebin/appmon_a.beam hides /usr/local/lib/erlang/lib/appmon-2.1.11/ebin/appmon_a.beam ** /usr/local/lib/erlang/lib/appmon-2.1.11/ebin/appmon.beam hides /usr/local/lib/erlang/lib/appmon-2.1.11/ebin/appmon.beam ** Found 1669 name clashes in code paths ok 2> Did I miss something? -- James Aimonetti mobile: 314.809.6307 work: 540.459.2220 email: james.aimonetti@REDACTED website: http://jamesaimonetti.com From james.aimonetti@REDACTED Thu Apr 8 22:16:53 2010 From: james.aimonetti@REDACTED (James Aimonetti) Date: Thu, 08 Apr 2010 16:16:53 -0400 Subject: [erlang-questions] Sets and Dict test suites In-Reply-To: References: <4BBA39A9.6030803@gmail.com> Message-ID: <4BBE39B5.5090406@gmail.com> Robert, Thanks for the suggestions and the db_test.erl. I've been playing with my implementation of sets and dict using splay trees and think I've concluded that they are not useful as a backend for the sets and dict API (barring a complete mucking of the implementation: http://github.com/jamesaimonetti/splay). The main issue I ran into is that splay trees rely on blindly restructuring after queries, as well as inserts and deletions, to improve the balanced nature of the tree. I re-read the section (5.4) in Okasaki's book and he does mention this shortcoming in using splays for sets and finite maps. I've included a bare bones splay implementation with add, find_min, has, delete, and delete_min, as well as some others like size and height, for creating a heap within an application. Any feedback is appreciated :) James Robert Virding wrote: > Hi James, > > Sorry, I saw your earlier mail but I missed it. > > I have used a relatively simple test suite when writing my RB and AA > stuff. First I use all the examples given in books and papers and > check that my trees look the same as theirs. This, of course, only > works for small data sets. I also just sit and add and delete elements > from the shell and check that the tree looks right. > > When I am satisfied that it looks right I try some bigger data sets. > Easy ways to generate them are functions like: > > random_data(N) -> [ {random:uniform(),I} || I <- lists:seq(1, N) ]. > > which generates a set of {Key,Value} pairs with random keys. I use > such a set to build a dict. I then use the same set to 'fetch' all the > keys to test that they are there and that I can reach them. Then I use > the same and delete all the keys to check that I can and that the > final dict is empty as it should be. It would better to both 'fetch' > and delete each key to make sure that the tree is always consistent > during deletes. I found deleting harder than adding. > > You can also do the same thing with non-random data sets to see if you > get some funny trees, for example your tree degenerates to a list. > Another good and simple test is to just count the depth of each node > and see that you get a reasonable spread of the depth, i.e. that it > really lg n like it should be. > > I do some simple unit testing by saving problem cases so I can retry them. > > I also have a test file for comparing various algorithms which is > useful for testing as well. Run it through the compiler and generate > the file after the macro expansions to see the resultant code, > c(db_test, ['E']). There are some comments but it is really an > internal file. > > Not very scientific I know but it seems to work. Also the trees > quickly become unreadable when they grow big. I hope you do the > splaydict and splaysets, I would like to see them. Real soon now (TM) > I will adding rbdict and rbsets to the distribution. There are also > some extensions to the API I would like to add. > > I hope this helps, > > Robert > > > On 5 April 2010 21:27, James Aimonetti wrote: > >> Does anyone on the list have test suites for testing sets and dict >> functionality using different backends? I'm going through Okasaki's book and >> currently implementing the splay trees, and, having looked at the rbsets and >> rbdict using red-black trees, I wanted to try implementing the API using the >> splay trees and testing the results against the official implementations. >> >> Also, is there any writeup of the backends' strengths vs weaknesses for >> various use cases (as concerns Erlang specifically)? If not, I'd be >> interested in developing one if those with more experience could offer help >> as I worked on it. >> >> Thanks, >> >> James >> >> -- >> James Aimonetti >> mobile: 314.809.6307 >> work: 540.459.2220 >> email: james.aimonetti@REDACTED >> website: http://jamesaimonetti.com >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> -- James Aimonetti mobile: 314.809.6307 work: 540.459.2220 email: james.aimonetti@REDACTED website: http://jamesaimonetti.com From max.lapshin@REDACTED Thu Apr 8 22:36:58 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 9 Apr 2010 00:36:58 +0400 Subject: [erlang-questions] mmap file to binary In-Reply-To: References: <19383.13120.860173.157003@pilspetsen.it.uu.se> Message-ID: I've thought a lot about mmap usage in erlang and haven't found any good solution. Main idea is that when file is mmaped to address space, it is possible to take chunks from it without any copying: {ok, Bin} = file:mmap(File), <<_:LargeOffset/binary, InsideBinary:Size/binary, _/binary>> = Bin, But what to do, when file is closed and all binaries, that referred into this address space becomes unusable? From waelsamaha@REDACTED Fri Apr 9 01:04:33 2010 From: waelsamaha@REDACTED (Eng. Wael Samaha) Date: Thu, 8 Apr 2010 16:04:33 -0700 (PDT) Subject: Application Start Type Message-ID: <929442.67592.qm@web62202.mail.re1.yahoo.com> I found it also in the official "application" module, application:info() , (which is calling application_controller internally) , but it is not specified in the documentation. Is there any other documented and reliable way ?. --- On Thu, 4/8/10, Eng. Wael Samaha wrote: From: Eng. Wael Samaha Subject: Re: Application Start Type To: erlang-questions@REDACTED Date: Thursday, April 8, 2010, 1:48 AM I got a way to get the application StartType using the below interface. application_controller:info(). But this module is an internal module within the Kernel application and not documented or recommended to be used externally. --- On Tue, 4/6/10, Eng. Wael Samaha wrote: From: Eng. Wael Samaha Subject: Application Start Type To: erlang-questions@REDACTED Date: Tuesday, April 6, 2010, 4:19 PM Dear all, ??? Is there a way to know the start type for currently started applications?. I'm asking because I have a live system with applications started on top of it. Some of these application started as temporary and others started as permanent. I need now to know only which are started as permanent to stop and re-start again as temporary. I know I can stop all applications and start them again with the default type (temporary), but I'm seeking for a way to only stop those are needed to be stopped. Regards, Wael From kuenishi@REDACTED Fri Apr 9 02:32:21 2010 From: kuenishi@REDACTED (UENISHI Kota) Date: Thu, 8 Apr 2010 16:32:21 -0800 Subject: automatically net-configuring node with net_kernel:start/1 Message-ID: I'm writing a self-configuring node cluster with net_kernel:start/1. It starts without -name nor -sname option and gets its IP adress from inet:getif/0. And they configure themselves with net_kernel:start/1 as follows: net_kernel:start( [ 'appname@REDACTED', longnames ] ). Servers find each other by sending its longname by UDP broadcasting, and they connects each other. The problem is, starting erl without -name option it doesn't kick epmd and following net_kernel:start/1 fails in tcp_inet_dist:listen (maybe because of epmd not running). And another way - starting with -name option with bad ip address ( or local ip) makes net_kernel:start/1 result in {error, not_allowed}. Is it a wrong to think that if epmd would start when net_kernel:start/1 had been called, isn't it? -- UENISHI Kota from PC :D From mikpe@REDACTED Fri Apr 9 07:18:11 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Fri, 9 Apr 2010 07:18:11 +0200 Subject: [erlang-questions] mmap file to binary In-Reply-To: References: <19383.13120.860173.157003@pilspetsen.it.uu.se> Message-ID: <19390.47251.682492.569399@pilspetsen.it.uu.se> Max Lapshin writes: > I've thought a lot about mmap usage in erlang and haven't found any > good solution. > Main idea is that when file is mmaped to address space, it is possible > to take chunks from it without any copying: > > {ok, Bin} = file:mmap(File), > <<_:LargeOffset/binary, InsideBinary:Size/binary, _/binary>> = Bin, > > But what to do, when file is closed and all binaries, that referred > into this address space becomes unusable? If we're talking Unix-like OSs, then closing a file descriptor does not nuke any mappings made on it. So this is a non-problem. The only problem I see is the munmap rather than free when the off-heap wrapper binary dies, as I wrote before. But that should be fixable with a flag in the off-heap binary. From max.lapshin@REDACTED Fri Apr 9 07:35:30 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 9 Apr 2010 09:35:30 +0400 Subject: [erlang-questions] mmap file to binary In-Reply-To: <19390.47251.682492.569399@pilspetsen.it.uu.se> References: <19383.13120.860173.157003@pilspetsen.it.uu.se> <19390.47251.682492.569399@pilspetsen.it.uu.se> Message-ID: Indeed, you are right. Just need to look, how to change free() to munmap() From sapan.gcet@REDACTED Fri Apr 9 08:48:25 2010 From: sapan.gcet@REDACTED (sapan shah) Date: Fri, 9 Apr 2010 12:18:25 +0530 Subject: platform independent new line char... Message-ID: Hi, Java provides a way to get the new line character in platform independent way. String newLine = System.getProperty("line.separator"); Is there any way in erlang to do the same?? -- Regards, Sapan Shah From max.lapshin@REDACTED Fri Apr 9 09:14:30 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 9 Apr 2010 11:14:30 +0400 Subject: [erlang-questions] mmap file to binary In-Reply-To: References: <19383.13120.860173.157003@pilspetsen.it.uu.se> <19390.47251.682492.569399@pilspetsen.it.uu.se> Message-ID: There is one interesting function: erts_create_magic_binary(size, destructor), but now it is required to make the same one, that doesn't allocate binary data. From kenneth.lundin@REDACTED Fri Apr 9 09:17:34 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Fri, 9 Apr 2010 09:17:34 +0200 Subject: [erlang-questions] Upgrade to R13B04 Produces Clashes in Code Path In-Reply-To: <4BBE30B6.2070301@gmail.com> References: <4BBE30B6.2070301@gmail.com> Message-ID: Check your environment variables. If you have set the ERL_LIBS variable to point to a directory where you can find the same modules as in the default lib you can get this result from code:clash(). Another possibility is the ERL_FLAGS variable if it contains -pa or -pz flags. /Kenneth Erlang/OTP Ericsson On Thu, Apr 8, 2010 at 9:38 PM, James Aimonetti wrote: > I upgraded to the latest Erlang and was greeted with 1600+ clashes in the > code paths. Installed from src, vanilla configure/make/make install. > > $> erl > Erlang R13B04 (erts-5.7.5) [source] [64-bit] [rq:1] [async-threads:0] [hipe] > [kernel-poll:false] > > Eshell V5.7.5 (abort with ^G) > 1> code:clash(). > ... > ... > ** /usr/local/lib/erlang/lib/appmon-2.1.11/ebin/appmon_a.beam hides > /usr/local/lib/erlang/lib/appmon-2.1.11/ebin/appmon_a.beam > ** /usr/local/lib/erlang/lib/appmon-2.1.11/ebin/appmon.beam hides > /usr/local/lib/erlang/lib/appmon-2.1.11/ebin/appmon.beam > ** Found 1669 name clashes in code paths > ok > 2> > > Did I miss something? > > -- > James Aimonetti > mobile: 314.809.6307 > work: 540.459.2220 > email: james.aimonetti@REDACTED > website: http://jamesaimonetti.com > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From schramm.ingo@REDACTED Fri Apr 9 09:29:03 2010 From: schramm.ingo@REDACTED (ingo.schramm) Date: Fri, 9 Apr 2010 00:29:03 -0700 (PDT) Subject: ab with erlang and PHP?which is better?? In-Reply-To: <60936eb4-6f0c-4044-9fff-364d5000d711@b23g2000yqn.googlegroups.com> References: <60936eb4-6f0c-4044-9fff-364d5000d711@b23g2000yqn.googlegroups.com> Message-ID: <59e524e7-80f3-4052-bf37-ac7b66f1ea3b@g11g2000yqe.googlegroups.com> Another pointer to what went wrong: > Complete requests: ? ? ?1000 > Failed requests: ? ? ? ?962 > ? ?(Connect: 0, Receive: 0, Length: 962, Exceptions: 0) There is obviously something wrong with your Erlang application ;) Does it return documents - or errors?? From max.lapshin@REDACTED Fri Apr 9 09:38:20 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 9 Apr 2010 11:38:20 +0400 Subject: [erlang-questions] mmap file to binary In-Reply-To: References: <19383.13120.860173.157003@pilspetsen.it.uu.se> <19390.47251.682492.569399@pilspetsen.it.uu.se> Message-ID: Yes, all we need, is an small extension to nif interface to make not only enif_alloc_binary, but also enif_alloc_magick_binary From bengt.kleberg@REDACTED Fri Apr 9 09:56:58 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 09 Apr 2010 09:56:58 +0200 Subject: [erlang-questions] platform independent new line char... In-Reply-To: References: Message-ID: <1270799818.4982.8.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, I do not have Windows available. This is theoretical. You can get a new line string by doing: io_lib:nl() -> chars() Returns a character list which represents a new line character. bengt On Fri, 2010-04-09 at 08:48 +0200, sapan shah wrote: > Hi, > > Java provides a way to get the new line character in platform independent > way. > String newLine = System.getProperty("line.separator"); > > Is there any way in erlang to do the same?? > From julian.tibble@REDACTED Fri Apr 9 11:11:25 2010 From: julian.tibble@REDACTED (Julian Tibble) Date: Fri, 09 Apr 2010 10:11:25 +0100 Subject: [erlang-questions] mmap file to binary In-Reply-To: References: <19383.13120.860173.157003@pilspetsen.it.uu.se> <19390.47251.682492.569399@pilspetsen.it.uu.se> Message-ID: <1270804285.2277.30.camel@wazowski.semmle> > Indeed, you are right. Just need to look, how to change free() to munmap() That's also how it is done in the Sun Java VM, but it can cause a really annoying cross-platform problem [1]. The trouble is that on Windows a mapped file may not be deleted, but the point in time at which a file is unmapped with this scheme is non-deterministic because it relies on the garbage-collector. Thus, there is no reliable way to delete a file after it's been mapped. Obviously on a Unix-like OS, there is no problem with unlinking a file that is in use, so everything works fine - but this can be even worse: things work fine on the developer's machine but when they distribute the software it breaks. Julian [1]: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4715154 From max.lapshin@REDACTED Fri Apr 9 11:19:58 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 9 Apr 2010 13:19:58 +0400 Subject: [erlang-questions] mmap file to binary In-Reply-To: <1270804285.2277.30.camel@wazowski.semmle> References: <19383.13120.860173.157003@pilspetsen.it.uu.se> <19390.47251.682492.569399@pilspetsen.it.uu.se> <1270804285.2277.30.camel@wazowski.semmle> Message-ID: > The trouble is that on Windows a mapped file may not be deleted, but the > point in time at which a file is unmapped with this scheme is > non-deterministic because it relies on the garbage-collector. Thus, > there is no reliable way to delete a file after it's been mapped. > Thank you for explanation, but frankly speaking, I have no interest about how my software works under windows. I even havent' thought that windows has mmap =) It will be ok for me just to have instruments to implement functionality, that works under Linux, OS X and FreeBSD: I've never met with anything else. > Obviously on a Unix-like OS, there is no problem with unlinking a file > that is in use, so everything works fine Not exactly so: hard disk space will be still used, so problem is not with ability to delete, problem is in deterministic deletion of file. I understand this risk and it is ok, because speed matters more, than delay in deletion of file. > [1]: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4715154 > > From clist@REDACTED Fri Apr 9 12:23:47 2010 From: clist@REDACTED (Angel Alvarez) Date: Fri, 9 Apr 2010 12:23:47 +0200 Subject: [erlang-questions] mmap file to binary In-Reply-To: <1270804285.2277.30.camel@wazowski.semmle> References: <1270804285.2277.30.camel@wazowski.semmle> Message-ID: <201004091223.47956.clist@uah.es> Maybe a new alloc for mmaped files will allow manage diferences among OSes in a easy way... http://www.erlang.org/doc/man/erts_alloc.html It seems that the mmaped case is already covered using the mseg_alloc: mseg_alloc A memory segment allocator. mseg_alloc is used by other allocators for allocating memory segments and is currently only available on systems that have the mmap system call. Memory segments that are deallocated are kept for a while in a segment cache before they are destroyed. When segments are allocated, cached segments are used if possible instead of creating new segments. This in order to reduce the number of system calls made. Seem the code is in place, so cloning thw whole allocator and tweaking a bit the behaviour will suffice? /Angel El Viernes, 9 de Abril de 2010 11:11:25 Julian Tibble escribi?: > > Indeed, you are right. Just need to look, how to change free() to munmap() > > That's also how it is done in the Sun Java VM, but it can cause a really > annoying cross-platform problem [1]. > > The trouble is that on Windows a mapped file may not be deleted, but the > point in time at which a file is unmapped with this scheme is > non-deterministic because it relies on the garbage-collector. Thus, > there is no reliable way to delete a file after it's been mapped. > > Obviously on a Unix-like OS, there is no problem with unlinking a file > that is in use, so everything works fine - but this can be even worse: > things work fine on the developer's machine but when they distribute the > software it breaks. > > > Julian > > [1]: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4715154 > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Este correo no tiene dibujos. Las formas extra?as en la pantalla son letras. __________________________________________ Clist UAH a.k.a Angel __________________________________________ Vivimos en un sociedad exquisitamente dependiente de la ciencia y la tecnolog?a, cuando muy poco gente conoce algo de la ciencia ? de la tecnolog?a. From max.lapshin@REDACTED Fri Apr 9 12:32:48 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 9 Apr 2010 14:32:48 +0400 Subject: [erlang-questions] mmap file to binary In-Reply-To: <201004091223.47956.clist@uah.es> References: <1270804285.2277.30.camel@wazowski.semmle> <201004091223.47956.clist@uah.es> Message-ID: On Fri, Apr 9, 2010 at 2:23 PM, Angel Alvarez wrote: > > Maybe a new alloc for mmaped files will allow manage diferences among OSes in a easy way... > > http://www.erlang.org/doc/man/erts_alloc.html > > It seems that the mmaped case is already covered using the ?mseg_alloc: > > mseg_alloc > Maybe it will be good to dig inside this. However, there seems to be easy way: erl_binary.h has erts_create_magic_binary. It should be changed to: ERTS_GLB_INLINE Binary * erts_create_mmap_binary(size_t size, int fd) { Uint bsize = ERTS_MAGIC_BIN_SIZE(0); Binary* bptr = erts_alloc_fnf(ERTS_ALC_T_BINARY, 0); if (!bptr) erts_alloc_n_enomem(ERTS_ALC_T2N(ERTS_ALC_T_BINARY), bsize); ERTS_CHK_BIN_ALIGNMENT(bptr); bptr->flags = BIN_FLAG_MAGIC; bptr->orig_size = ERTS_MAGIC_BIN_ORIG_SIZE(size); bptr->orig_bytes = mmap(0, size, PROT_READ, MAP_FILE, fd, 0); erts_refc_init(&bptr->refc, 0); ERTS_MAGIC_BIN_DESTRUCTOR(bptr) = binary_mmap_destructor; return bptr; } static void binary_mmap_destructor(Binary *bptr) { munmap(bptr->orig_bytes, bptr->orig_size); } From tony@REDACTED Fri Apr 9 13:37:56 2010 From: tony@REDACTED (Tony Rogvall) Date: Fri, 9 Apr 2010 13:37:56 +0200 Subject: [erlang-questions] mmap file to binary In-Reply-To: References: <1270804285.2277.30.camel@wazowski.semmle> <201004091223.47956.clist@uah.es> Message-ID: <7A08415A-8730-4B61-9D6B-0394A1EC4F55@rogvall.se> Last time I checked the orig_bytes was not separated pointer. It's currently (r14) a dynamic field. typedef struct binary { ERTS_INTERNAL_BINARY_FIELDS long orig_size; char orig_bytes[1]; /* to be continued */ } Binary; So this must be changed to be able to mmap data. You are on the right track ;-) Check out the ErlNifResource stuff, this will handle the garage collection for data like this. (yes, it is aso using the magic stuff) My suggestions for drivers is one ErlDrvObject where you can store the actual object data inline and pass it back and forth from driver to runtime. It will appear as a binary. One other suggestion is to make a simple wrapper for C(++) objects ErlDrvHandle to simplify the allocation when you only want to pass a pointer back and forth. To make it safe the resource scheme used by erl_nif.c should be adopted. Just think what will happen if the driver is unloaded and the destructors are leaving with it ;-) Still the orig_bytes must be change. There are not to many places in the VM that uses the fact that orig_bytes is not a pointer, so the change should not be hard (if it's not done already ;-) /Tony On 9 apr 2010, at 12.32, Max Lapshin wrote: > On Fri, Apr 9, 2010 at 2:23 PM, Angel Alvarez wrote: >> >> Maybe a new alloc for mmaped files will allow manage diferences among OSes in a easy way... >> >> http://www.erlang.org/doc/man/erts_alloc.html >> >> It seems that the mmaped case is already covered using the mseg_alloc: >> >> mseg_alloc >> > > Maybe it will be good to dig inside this. However, there seems to be easy way: > > erl_binary.h has erts_create_magic_binary. It should be changed to: > > ERTS_GLB_INLINE Binary * > erts_create_mmap_binary(size_t size, int fd) > { > Uint bsize = ERTS_MAGIC_BIN_SIZE(0); > Binary* bptr = erts_alloc_fnf(ERTS_ALC_T_BINARY, 0); > if (!bptr) > erts_alloc_n_enomem(ERTS_ALC_T2N(ERTS_ALC_T_BINARY), bsize); > ERTS_CHK_BIN_ALIGNMENT(bptr); > bptr->flags = BIN_FLAG_MAGIC; > bptr->orig_size = ERTS_MAGIC_BIN_ORIG_SIZE(size); > bptr->orig_bytes = mmap(0, size, PROT_READ, MAP_FILE, fd, 0); > erts_refc_init(&bptr->refc, 0); > ERTS_MAGIC_BIN_DESTRUCTOR(bptr) = binary_mmap_destructor; > return bptr; > } > > static void binary_mmap_destructor(Binary *bptr) > { > munmap(bptr->orig_bytes, bptr->orig_size); > } > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From max.lapshin@REDACTED Fri Apr 9 13:46:27 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 9 Apr 2010 15:46:27 +0400 Subject: [erlang-questions] mmap file to binary In-Reply-To: <7A08415A-8730-4B61-9D6B-0394A1EC4F55@rogvall.se> References: <1270804285.2277.30.camel@wazowski.semmle> <201004091223.47956.clist@uah.es> <7A08415A-8730-4B61-9D6B-0394A1EC4F55@rogvall.se> Message-ID: On Fri, Apr 9, 2010 at 3:37 PM, Tony Rogvall wrote: > Check out the ErlNifResource stuff, this will handle the garage collection for data like this. > (yes, it is aso using the magic stuff) No, it is not enough, because ErlNifResourceDtor receives only pointer to data, but mmap also requires size in bytes. So to implement mmap we need to use other function, not nif_resource_dtor > My suggestions for drivers is one ErlDrvObject where you can store the actual object data > inline and pass it back and forth from driver to runtime. It will appear as a binary. you think, that it may be easier to implement it as a driver? > > Still the orig_bytes must be change. There are not to many places in the VM that uses the fact > that orig_bytes is not a pointer, so the change should not be hard (if it's not done already ;-) > I haven't understood: is the problem with orig_bytes now? From james.aimonetti@REDACTED Fri Apr 9 14:33:45 2010 From: james.aimonetti@REDACTED (James Aimonetti) Date: Fri, 09 Apr 2010 08:33:45 -0400 Subject: [erlang-questions] Upgrade to R13B04 Produces Clashes in Code Path In-Reply-To: References: <4BBE30B6.2070301@gmail.com> Message-ID: <4BBF1EA9.1040409@gmail.com> Thanks Kenneth. Cleared ERL_LIBS and the clashes went away. James Kenneth Lundin wrote: > Check your environment variables. > If you have set the ERL_LIBS variable to point to a directory where you can > find the same modules as in the default lib you can get this result > from code:clash(). > > Another possibility is the ERL_FLAGS variable if it contains -pa or -pz flags. > > /Kenneth Erlang/OTP Ericsson > > On Thu, Apr 8, 2010 at 9:38 PM, James Aimonetti > wrote: > >> I upgraded to the latest Erlang and was greeted with 1600+ clashes in the >> code paths. Installed from src, vanilla configure/make/make install. >> >> $> erl >> Erlang R13B04 (erts-5.7.5) [source] [64-bit] [rq:1] [async-threads:0] [hipe] >> [kernel-poll:false] >> >> Eshell V5.7.5 (abort with ^G) >> 1> code:clash(). >> ... >> ... >> ** /usr/local/lib/erlang/lib/appmon-2.1.11/ebin/appmon_a.beam hides >> /usr/local/lib/erlang/lib/appmon-2.1.11/ebin/appmon_a.beam >> ** /usr/local/lib/erlang/lib/appmon-2.1.11/ebin/appmon.beam hides >> /usr/local/lib/erlang/lib/appmon-2.1.11/ebin/appmon.beam >> ** Found 1669 name clashes in code paths >> ok >> 2> >> >> Did I miss something? >> >> -- >> James Aimonetti >> mobile: 314.809.6307 >> work: 540.459.2220 >> email: james.aimonetti@REDACTED >> website: http://jamesaimonetti.com >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> > > -- James Aimonetti mobile: 314.809.6307 work: 540.459.2220 email: james.aimonetti@REDACTED website: http://jamesaimonetti.com From kenji.rikitake@REDACTED Fri Apr 9 15:46:07 2010 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Fri, 9 Apr 2010 22:46:07 +0900 Subject: a "vulnerability" report about Erlang on Ubuntu Security Notice Message-ID: <20100409134607.GA56474@k2r.org> While this is actually a PCRE vulnerability (CVE-2008-2371), in a recent Ubuntu Security Notice this is described as an "erlang-base" issue. FYI. http://www.ubuntu.com/usn/USN-624-2 Kenji Rikitake See also http://www.erlang.org/cgi-bin/ezmlm-cgi/3/820 for my suggestion to deal with this sort of advisories. From dmercer@REDACTED Fri Apr 9 15:49:27 2010 From: dmercer@REDACTED (David Mercer) Date: Fri, 9 Apr 2010 08:49:27 -0500 Subject: [erlang-questions] platform independent new line char... In-Reply-To: <1270799818.4982.8.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> References: <1270799818.4982.8.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: Returns [10] on WinXP/R13B03. Windows newline is [13, 10]. > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Bengt Kleberg > Sent: Friday, April 09, 2010 2:57 AM > To: undisclosed-recipients: > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] platform independent new line char... > > Greetings, > > I do not have Windows available. This is theoretical. > > You can get a new line string by doing: > io_lib:nl() -> chars() > Returns a character list which represents a new line character. > > bengt > > On Fri, 2010-04-09 at 08:48 +0200, sapan shah wrote: > > Hi, > > > > Java provides a way to get the new line character in platform > independent > > way. > > String newLine = System.getProperty("line.separator"); > > > > Is there any way in erlang to do the same?? > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From daniel.goertzen@REDACTED Fri Apr 9 15:53:15 2010 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Fri, 9 Apr 2010 08:53:15 -0500 Subject: ssh password protected private keys Message-ID: Does anybody know if password protected ssh private keys are in the works? I looked at the Python module "paramiko" and at first glance key decryption doesn't look too tricky. I have some ambitions to add this support to the erlang ssh module, but obviously would like to know if someone else is working on it. Thanks, Dan. From tony@REDACTED Fri Apr 9 15:21:42 2010 From: tony@REDACTED (Tony Rogvall) Date: Fri, 9 Apr 2010 15:21:42 +0200 Subject: [erlang-questions] mmap file to binary In-Reply-To: References: <1270804285.2277.30.camel@wazowski.semmle> <201004091223.47956.clist@uah.es> <7A08415A-8730-4B61-9D6B-0394A1EC4F55@rogvall.se> Message-ID: <4D758E50-91CC-4E0C-873E-4018CF0F4AC6@rogvall.se> On 9 apr 2010, at 13.46, Max Lapshin wrote: > On Fri, Apr 9, 2010 at 3:37 PM, Tony Rogvall wrote: >> Check out the ErlNifResource stuff, this will handle the garage collection for data like this. >> (yes, it is aso using the magic stuff) > > No, it is not enough, because ErlNifResourceDtor receives only pointer > to data, but mmap also requires size in bytes. > So to implement mmap we need to use other function, not nif_resource_dtor > I know that ErlNifResource is not enough but it's a step in the right direction. What I am saying is that what you want is not really possible to implement in the way the runtime system handles binaries today. But that the amount of work to fix it is not that much. >> My suggestions for drivers is one ErlDrvObject where you can store the actual object data >> inline and pass it back and forth from driver to runtime. It will appear as a binary. > > you think, that it may be easier to implement it as a driver? > No. Not for mmap'ed data, since you do not need to poll any file descriptors. >> >> Still the orig_bytes must be change. There are not to many places in the VM that uses the fact >> that orig_bytes is not a pointer, so the change should not be hard (if it's not done already ;-) >> > > I haven't understood: is the problem with orig_bytes now? Well... OTP team? ;-) /Tony From jeraymond@REDACTED Fri Apr 9 18:32:41 2010 From: jeraymond@REDACTED (Jeremy Raymond) Date: Fri, 9 Apr 2010 09:32:41 -0700 (PDT) Subject: How to use rb:filter/1? Message-ID: Hi, I'm trying to figure out how to use rb:fitler/1. What I'm unclear on is what the {Key, Value} items are matching against in the reports. From sverker@REDACTED Fri Apr 9 18:40:56 2010 From: sverker@REDACTED (Sverker Eriksson) Date: Fri, 09 Apr 2010 18:40:56 +0200 Subject: [erlang-questions] mmap file to binary In-Reply-To: <4D758E50-91CC-4E0C-873E-4018CF0F4AC6@rogvall.se> References: <1270804285.2277.30.camel@wazowski.semmle> <201004091223.47956.clist@uah.es> <7A08415A-8730-4B61-9D6B-0394A1EC4F55@rogvall.se> <4D758E50-91CC-4E0C-873E-4018CF0F4AC6@rogvall.se> Message-ID: <4BBF5898.4020205@erix.ericsson.se> Tony Rogvall wrote: >>> Still the orig_bytes must be change. There are not to many places in the VM that uses the fact >>> that orig_bytes is not a pointer, so the change should not be hard (if it's not done already ;-) >>> >>> >> I haven't understood: is the problem with orig_bytes now? >> > > > Well... OTP team? ;-) > > No, orig_bytes is still an array, but I don't think you need to change that. You could instead use the pointer 'bytes' in ProcBin to point to your mmap'ed data. typedef struct proc_bin { Eterm thing_word; /* Subtag REFC_BINARY_SUBTAG. */ Uint size; /* Binary size in bytes. */ struct proc_bin *next; /* Pointer to next ProcBin. */ Binary *val; /* Pointer to Binary structure. */ byte *bytes; /* Pointer to the actual data bytes. */ Uint flags; /* Flag word. */ } ProcBin; /Sverker, Erlang/OTP From roberto@REDACTED Fri Apr 9 19:39:57 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 9 Apr 2010 19:39:57 +0200 Subject: ssl:setopts Message-ID: dear list, i'm currently setting {active, once} as option of an ssl socket like this: ssl:setopts(Sock, [{active, once}]). this is what i get: =ERROR REPORT==== 9-Apr-2010::19:31:25 === ** Generic server <0.49.0> terminating ** Last message in was {setopts,<0.50.0>,[{active,once}]} ** When Server state == {st,acceptor,<0.42.0>,<0.50.0>,<0.50.0>,6,false, [{mode,binary}, {packet,http}, {ip,{0,0,0,0}}, {active,false}, {nodelay,false}], {sslsocket,6,<0.49.0>}, nil,54428,nil,false,false} ** Reason for termination == ** {function_clause,[{prim_inet,setopts,[nil,[{active,once}]]}, {ssl_broker,handle_call,3}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} i have ssl 3.10.3 and i thought this was correct. could someone please tell me what i'm doing wrong? thank you in advance, r. From mikpe@REDACTED Fri Apr 9 20:53:32 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Fri, 9 Apr 2010 20:53:32 +0200 Subject: [erlang-questions] mmap file to binary In-Reply-To: <4BBF5898.4020205@erix.ericsson.se> References: <1270804285.2277.30.camel@wazowski.semmle> <201004091223.47956.clist@uah.es> <7A08415A-8730-4B61-9D6B-0394A1EC4F55@rogvall.se> <4D758E50-91CC-4E0C-873E-4018CF0F4AC6@rogvall.se> <4BBF5898.4020205@erix.ericsson.se> Message-ID: <19391.30636.410791.181135@pilspetsen.it.uu.se> Sverker Eriksson writes: > Tony Rogvall wrote: > >>> Still the orig_bytes must be change. There are not to many places in the VM that uses the fact > >>> that orig_bytes is not a pointer, so the change should not be hard (if it's not done already ;-) > >>> > >>> > >> I haven't understood: is the problem with orig_bytes now? > >> > > > > > > Well... OTP team? ;-) > > > > > No, orig_bytes is still an array, but I don't think you need to change > that. You could instead use the pointer 'bytes' in ProcBin to point to > your mmap'ed data. > > typedef struct proc_bin { > Eterm thing_word; /* Subtag REFC_BINARY_SUBTAG. */ > Uint size; /* Binary size in bytes. */ > struct proc_bin *next; /* Pointer to next ProcBin. */ > Binary *val; /* Pointer to Binary structure. */ > byte *bytes; /* Pointer to the actual data bytes. */ > Uint flags; /* Flag word. */ > } ProcBin; ... which is exactly what I wrote some number of messages ago. From tony@REDACTED Sat Apr 10 00:04:38 2010 From: tony@REDACTED (Tony Rogvall) Date: Sat, 10 Apr 2010 00:04:38 +0200 Subject: [erlang-questions] mmap file to binary In-Reply-To: <19391.30636.410791.181135@pilspetsen.it.uu.se> References: <1270804285.2277.30.camel@wazowski.semmle> <201004091223.47956.clist@uah.es> <7A08415A-8730-4B61-9D6B-0394A1EC4F55@rogvall.se> <4D758E50-91CC-4E0C-873E-4018CF0F4AC6@rogvall.se> <4BBF5898.4020205@erix.ericsson.se> <19391.30636.410791.181135@pilspetsen.it.uu.se> Message-ID: On 9 apr 2010, at 20.53, Mikael Pettersson wrote: > Sverker Eriksson writes: >> Tony Rogvall wrote: >>>>> Still the orig_bytes must be change. There are not to many places in the VM that uses the fact >>>>> that orig_bytes is not a pointer, so the change should not be hard (if it's not done already ;-) >>>>> >>>>> >>>> I haven't understood: is the problem with orig_bytes now? >>>> >>> >>> >>> Well... OTP team? ;-) >>> >>> >> No, orig_bytes is still an array, but I don't think you need to change >> that. You could instead use the pointer 'bytes' in ProcBin to point to >> your mmap'ed data. >> >> typedef struct proc_bin { >> Eterm thing_word; /* Subtag REFC_BINARY_SUBTAG. */ >> Uint size; /* Binary size in bytes. */ >> struct proc_bin *next; /* Pointer to next ProcBin. */ >> Binary *val; /* Pointer to Binary structure. */ >> byte *bytes; /* Pointer to the actual data bytes. */ >> Uint flags; /* Flag word. */ >> } ProcBin; > > ... which is exactly what I wrote some number of messages ago. Sorry I missed that. You are right, I just found one spot in the gc where there may be a potential problem. But apart from that the bytes pointer is untouched. I was also looking for the ERL_ONHEAP_BIN_LIMIT to see if the ProcBin was copied to heap somewhere, but all clear ;-) A user_data field to pass to the creation of the magic binary could work I guess, then you could store any kind of extra information like mmap'ed size etc. The user_data is then passed to the destructor. The 'Binary' in this case is just a reference counting object? /Tony From jacob.vorreuter@REDACTED Sat Apr 10 00:55:17 2010 From: jacob.vorreuter@REDACTED (Jacob Vorreuter) Date: Fri, 9 Apr 2010 15:55:17 -0700 Subject: epmd spamming syslog Message-ID: I've noticed that epmd is logging this message to syslog about twice per minute. The server is running an instance of rabbitmq and no other Erlang processes. Can anyone provide some insight as to why epmd would log this message so frequently? It appears to be logged at the end of the epmd_main function in epmd.c. Apr 9 15:42:49 asdf epmd: epmd: epmd running - daemon = 1 Apr 9 15:42:49 asdf epmd: epmd: epmd running - daemon = 1 Apr 9 15:43:43 asdf epmd: epmd: epmd running - daemon = 1 Apr 9 15:43:43 asdf epmd: epmd: epmd running - daemon = 1 static int epmd_main(int argc, char** argv, int free_argv) ... dbg_printf(g,0,"epmd running - daemon = %d",g->is_daemon); From b.nicholson@REDACTED Sat Apr 10 02:45:49 2010 From: b.nicholson@REDACTED (B. Nicholson) Date: Fri, 09 Apr 2010 19:45:49 -0500 Subject: [erlang-questions] epmd spamming syslog In-Reply-To: References: Message-ID: <4BBFCA3D.3060404@niceng.com> Each time I've seen this it was due to epmd restarting for some reason. Since there are many reasons that epmd can restart I couldn't guess in your case what's happening. Check epmd's pid over the course of several minutes and see if it is stopping and then restarting. Barry Nicholson On 04/09/2010 05:55 PM, Jacob Vorreuter wrote: > I've noticed that epmd is logging this message to syslog about twice per minute. The server is running an instance of rabbitmq and no other Erlang processes. Can anyone provide some insight as to why epmd would log this message so frequently? It appears to be logged at the end of the epmd_main function in epmd.c. > > Apr 9 15:42:49 asdf epmd: epmd: epmd running - daemon = 1 > Apr 9 15:42:49 asdf epmd: epmd: epmd running - daemon = 1 > Apr 9 15:43:43 asdf epmd: epmd: epmd running - daemon = 1 > Apr 9 15:43:43 asdf epmd: epmd: epmd running - daemon = 1 > > > static int epmd_main(int argc, char** argv, int free_argv) > ... > dbg_printf(g,0,"epmd running - daemon = %d",g->is_daemon); > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From b.nicholson@REDACTED Sat Apr 10 03:02:38 2010 From: b.nicholson@REDACTED (B. Nicholson) Date: Fri, 09 Apr 2010 20:02:38 -0500 Subject: [erlang-questions] epmd spamming syslog In-Reply-To: <4BBFCA3D.3060404@niceng.com> References: <4BBFCA3D.3060404@niceng.com> Message-ID: <4BBFCE2E.5000606@niceng.com> One other thing, any time another distributed vm is started (i.e. "erl -sname newnode") this message is logged. Are you sure another instance of a distributed node isn't being started about once or twice a minute. I haven't seen rabbitmq do this except when restarting. Barry Nicholson On 04/09/2010 07:45 PM, B. Nicholson wrote: > Each time I've seen this it was due to epmd restarting for some > reason. Since there are many reasons that epmd can restart I couldn't > guess in your case what's happening. Check epmd's pid over the > course of several minutes and see if it is stopping and then restarting. > > Barry Nicholson > > On 04/09/2010 05:55 PM, Jacob Vorreuter wrote: >> I've noticed that epmd is logging this message to syslog about twice >> per minute. The server is running an instance of rabbitmq and no >> other Erlang processes. Can anyone provide some insight as to why >> epmd would log this message so frequently? It appears to be logged >> at the end of the epmd_main function in epmd.c. >> >> Apr 9 15:42:49 asdf epmd: epmd: epmd running - daemon = 1 >> Apr 9 15:42:49 asdf epmd: epmd: epmd running - daemon = 1 >> Apr 9 15:43:43 asdf epmd: epmd: epmd running - daemon = 1 >> Apr 9 15:43:43 asdf epmd: epmd: epmd running - daemon = 1 >> >> >> static int epmd_main(int argc, char** argv, int free_argv) >> ... >> dbg_printf(g,0,"epmd running - daemon = %d",g->is_daemon); >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mjtruog@REDACTED Sat Apr 10 12:22:22 2010 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 10 Apr 2010 03:22:22 -0700 Subject: [erlang-questions] [proposal] Declarative syntax for metadata (long!) In-Reply-To: <95be1d3b1003180622h3a3a5acdrc2e0109b7918b736@mail.gmail.com> References: <95be1d3b1003180434l7e76c610if40faed29efa647c@mail.gmail.com> <9b08084c1003180606g68f1c060r10635f571b6c642c@mail.gmail.com> <95be1d3b1003180622h3a3a5acdrc2e0109b7918b736@mail.gmail.com> Message-ID: <4BC0515E.1010500@gmail.com> Hi, I was hoping to bring up this thread because it does relate to a task I am trying to do in Erlang. To avoid re-parsing a compiled module to generate this information I was hoping to use the loaded binary beam data. My use case is for automatically generating RPC function definitions from an API module, which I think would be neat. One small thing that would make this easier, is if "-compile(debug_info)." worked. The documentation says it does: "Note that all the options except the include path ({i,Dir}) can also be given in the file" (http://erlang.org/doc/man/compile.html#debug_info) For my use case, if that worked, I could just insert the debug_info compile option into the API module and then have access to the code from the binary beam format, with something like: {_, Beam, _} = code:get_object_code(Module), % example in http://erlang.org/doc/man/beam_lib.html {ok,{_,[{abstract_code,{_,AC}}]}} = beam_lib:chunks(Beam,[abstract_code]), io:fwrite("~s~n", [erl_prettypr:format(erl_syntax:form_list(AC))]). With the absence of the "-compile(debug_info)." I must use manual re-parsing of the module if I want to avoid unique command-line configuration for module compilation... and that seems silly. The abstract_code format does not provide the preprocessor/comment lines, but it doesn't seem as if it would be a problem to add that data, except for the bloat of the debug data. Applications can always be queried through: [{FirstApplication, Description, Vsn} | _] = application:loaded_applications(), {ok, ModuleList} = application:get_key(FirstApplication, modules). I agree that these tasks could be made a little easier by gathering them into a single place somehow. - Michael Vlad Dumitrescu wrote: > On Thu, Mar 18, 2010 at 14:06, Joe Armstrong wrote: > >> We should be able to introspect a module at run-time and recover a >> list of funs, we should be able to take a fun and recover its syntax >> tree or an ascii representation of the fun. The same should be true >> for types, specs, edoc strings an so on. >> > > Yep, that too! I think even applications could exist as 'things': get > a list of applications, query an application for its modules. > > There is no such thing in Erlang as a module, just a name > in the documentation and the attribute with the said name. Otherwise > one does handle and refer to beam files. There should be an API to > handle modules that hides the underlying beam_lib, as an example. > > regards, > Vlad > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From ulf.wiger@REDACTED Sat Apr 10 13:58:11 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 10 Apr 2010 13:58:11 +0200 Subject: [erlang-questions] [proposal] Declarative syntax for metadata (long!) In-Reply-To: <4BC0515E.1010500@gmail.com> References: <95be1d3b1003180434l7e76c610if40faed29efa647c@mail.gmail.com> <9b08084c1003180606g68f1c060r10635f571b6c642c@mail.gmail.com> <95be1d3b1003180622h3a3a5acdrc2e0109b7918b736@mail.gmail.com> <4BC0515E.1010500@gmail.com> Message-ID: <4BC067D3.2010808@erlang-solutions.com> Michael Truog wrote: > Hi, > > I was hoping to bring up this thread because it does relate to a task I > am trying to do in Erlang. To avoid re-parsing a compiled module to > generate this information I was hoping to use the loaded binary beam > data. My use case is for automatically generating RPC function > definitions from an API module, which I think would be neat. > > One small thing that would make this easier, is if > "-compile(debug_info)." worked. The documentation says it does: > "Note that all the options except the include path ({i,Dir}) can also be > given in the file" (http://erlang.org/doc/man/compile.html#debug_info) The following code might serve as a quick-and-dirty workaround, and also provide much faster access to the abstract code: -module(include_forms). -export([parse_transform/2]). parse_transform(Forms, _) -> [EoF|Rev] = lists:reverse(Forms), Forms1 = lists:reverse([EoF,absfun(Forms)|Rev]), {Attrs,Funs} = lists:splitwith(fun(F) -> element(1,F) == attribute end, Forms1), Attrs ++ [{attribute,1,export,[{'$abstract_code$',0}]} | Funs]. absfun(Forms) -> {function, 1, '$abstract_code$', 0, [{clause,1,[],[], [erl_parse:abstract(Forms)]}]}. Eshell V5.7.5 (abort with ^G) 1> c(include_forms). {ok,include_forms} 2> c(include_forms, [{parse_transform,include_forms}]). {ok,include_forms} 3> include_forms:'$abstract_code$'(). [{attribute,1,file,{"./include_forms.erl",1}}, {attribute,1,module,include_forms}, {attribute,2,export,[{parse_transform,2}]}, {function,4,parse_transform,2, [{clause,4, [{var,4,'Forms'},{var,4,'_'}], [], [{match,5, {cons,5,{var,5,'EoF'},{var,5,'Rev'}}, {call,5, {remote,5,{atom,5,lists},{atom,5,...}}, [{var,5,'Forms'}]}}, The parse_transform directive can certainly be included in a module, as well as buried in a .hrl file. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From lemenkov@REDACTED Sat Apr 10 14:56:50 2010 From: lemenkov@REDACTED (Peter Lemenkov) Date: Sat, 10 Apr 2010 16:56:50 +0400 Subject: Heads up! Erlang package becomes modularized in Fedora. Message-ID: Hello All! CC to devel@REDACTED CC to erlang-questions@REDACTED I would like to attract attention those erlangers, who use Fedora and especially those of them, who use Erlang from Fedora's main repository. Others are safe :) Since Erlang adoption steadily grows and more applications, written in Erlang, becomes popular, people starts complaining about the way Erlang is delivered in Fedora. One large bulk containing mixture of erlang/c/java sources, beam-files, examples, which has heavy dependencies like wxGTK and Java, is not that Average Joe wants when he tries to install his favorite document-oriented database, yet another web-server, xmpp-server or something similar. The problem is that we try to offer users a package with as much features enabled, as we could, so we can't simply disable some functionality at a build time. All we really can do here is to split one large package into pieces. Few of us (Fedora maintainers) briefly discussed this situation, and there is a consensus, that we should split off at least some parts of main Erlang package. I did some investigation, and find a way to completely modularize whole Erlang package. In a few words - main package consists of purely virtual 'erlang' package and a dozens of sub-packages (from erlang-appmon to erlang-xmerl - one sub-package for each %{_libdir}/erlang/lib/* item). The main virtual package 'erlang' depends on every subpackage, so if you prefer to install/upgrade erlang by typing "yum install erlang" you won't see any difference. What's new is a set of scripts to generate cross-subpackage dependencies. The approach for dependency generation is very straightforward and simple - with the addition to normal RPM dependency generator, each ebin/*.beam file within each sub-package is analyzed for imported and exported functions, and they are added (in form of erlang(Module:Finction/Arity)) to the list of RPM provides and requires. Other visible change is a removal of c_src, java_src and erlang src/*.erl files from subpackages. In fact, everything stored in /usr is not intended for recompiling by user, so it's safe to remove these files. If someone have any *technical* objections for doing this, it's time to dispute this change. At the first try, I did very sensitive mistake (already recovered) - unfortunately some *.hrl files, which should be stored in %{module}-%{version}/include intentionally stored in %{module}-%{version}/src . We can't do much here since many users hardcode these paths in their projects, so I just leaved all *.hrl files in src directory intact. I also remove empty and *.o files and fixed some installation errors with priv libraries, but these are minor changes. I repeat and emphasize - I'll do my best to achieve full compatibility with previous installations, and I'm pretty sure that all these changes wont break anything. Also I'm sure that we need to push this change into both F-12 and forthcoming F-13, since benefits are really great (if everything will be done properly and careful, of course - that's why I'm still testing every erlang-related package, included (and even yet not included) in Fedora for possible regressions). Fedora 11, EL-4 and EL-5 will remain unchanged. List of affected packages: https://fedoraproject.org/wiki/SIGs/Erlang#Current_packages Any objections and commentaries are highly welcome! -- With best regards, Peter Lemenkov. From mjtruog@REDACTED Sat Apr 10 20:18:44 2010 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 10 Apr 2010 11:18:44 -0700 Subject: [erlang-questions] [proposal] Declarative syntax for metadata (long!) In-Reply-To: <4BC067D3.2010808@erlang-solutions.com> References: <95be1d3b1003180434l7e76c610if40faed29efa647c@mail.gmail.com> <9b08084c1003180606g68f1c060r10635f571b6c642c@mail.gmail.com> <95be1d3b1003180622h3a3a5acdrc2e0109b7918b736@mail.gmail.com> <4BC0515E.1010500@gmail.com> <4BC067D3.2010808@erlang-solutions.com> Message-ID: <4BC0C104.5050907@gmail.com> Ulf Wiger wrote: > Michael Truog wrote: >> Hi, >> >> I was hoping to bring up this thread because it does relate to a task I >> am trying to do in Erlang. To avoid re-parsing a compiled module to >> generate this information I was hoping to use the loaded binary beam >> data. My use case is for automatically generating RPC function >> definitions from an API module, which I think would be neat. >> >> One small thing that would make this easier, is if >> "-compile(debug_info)." worked. The documentation says it does: >> "Note that all the options except the include path ({i,Dir}) can also be >> given in the file" (http://erlang.org/doc/man/compile.html#debug_info) > > The following code might serve as a quick-and-dirty workaround, > and also provide much faster access to the abstract code: > > -module(include_forms). > -export([parse_transform/2]). > > parse_transform(Forms, _) -> > [EoF|Rev] = lists:reverse(Forms), > Forms1 = lists:reverse([EoF,absfun(Forms)|Rev]), > {Attrs,Funs} = lists:splitwith(fun(F) -> > element(1,F) == attribute > end, Forms1), > Attrs ++ [{attribute,1,export,[{'$abstract_code$',0}]} | Funs]. > > absfun(Forms) -> > {function, 1, '$abstract_code$', 0, > [{clause,1,[],[], > [erl_parse:abstract(Forms)]}]}. > > > > Eshell V5.7.5 (abort with ^G) > 1> c(include_forms). > {ok,include_forms} > 2> c(include_forms, [{parse_transform,include_forms}]). > {ok,include_forms} > 3> include_forms:'$abstract_code$'(). > [{attribute,1,file,{"./include_forms.erl",1}}, > {attribute,1,module,include_forms}, > {attribute,2,export,[{parse_transform,2}]}, > {function,4,parse_transform,2, > [{clause,4, > [{var,4,'Forms'},{var,4,'_'}], > [], > [{match,5, > {cons,5,{var,5,'EoF'},{var,5,'Rev'}}, > {call,5, > {remote,5,{atom,5,lists},{atom,5,...}}, > [{var,5,'Forms'}]}}, > > > The parse_transform directive can certainly be included in > a module, as well as buried in a .hrl file. > > BR, > Ulf W This is a better solution! If the include_forms is put into an application dependency so that it is compiled before the module that needs the abstract code access (and so that it exists as low-level code), the module can just have "-compile({parse_transform, include_forms})." in the file to create the '$abstract_code$'/1 function. This doesn't seem dirty, since you have to opt-in to using the parse transform within the code, so in a way you are agreeing to the insertion of a function in the module. Thanks, Michael From anthonym@REDACTED Sat Apr 10 21:51:42 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Sat, 10 Apr 2010 12:51:42 -0700 Subject: [erlang-questions] automatically net-configuring node with net_kernel:start/1 In-Reply-To: References: Message-ID: <20100410195142.GB82910@alumni.caltech.edu> You might check out nodefinder http://code.google.com/p/nodefinder/ It provides various mechanisms for self configuration including multicast, ec2 tools or s3. These all end up using net_adm:ping to join the group, so probably would not exhibit the problems you are seeing. We used this quite extensively in ec2, to dynamically grow and shrink our erlang cluster. -Anthony On Thu, Apr 08, 2010 at 04:32:21PM -0800, UENISHI Kota wrote: > I'm writing a self-configuring node cluster with net_kernel:start/1. > It starts without -name nor -sname option and gets its IP adress from > inet:getif/0. > And they configure themselves with net_kernel:start/1 as follows: > > net_kernel:start( [ 'appname@REDACTED', longnames ] ). > > Servers find each other by sending its longname by UDP broadcasting, > and they connects each other. > > The problem is, starting erl without -name option it doesn't kick epmd > and following net_kernel:start/1 fails in tcp_inet_dist:listen (maybe > because of epmd not running). > And another way - starting with -name option with bad ip address ( or > local ip) makes net_kernel:start/1 result in {error, not_allowed}. > > Is it a wrong to think that if epmd would start when > net_kernel:start/1 had been called, isn't it? > > -- > UENISHI Kota from PC :D > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- ------------------------------------------------------------------------ Anthony Molinaro From matthew@REDACTED Sat Apr 10 22:32:46 2010 From: matthew@REDACTED (Matthew Sackman) Date: Sat, 10 Apr 2010 21:32:46 +0100 Subject: [erlang-questions] epmd spamming syslog In-Reply-To: <4BBFCE2E.5000606@niceng.com> References: <4BBFCA3D.3060404@niceng.com> <4BBFCE2E.5000606@niceng.com> Message-ID: <20100410203245.GA2875@wellquite.org> On Fri, Apr 09, 2010 at 08:02:38PM -0500, B. Nicholson wrote: > One other thing, any time another distributed vm is started (i.e. > "erl -sname newnode") this message is logged. Are you sure another > instance of a distributed node isn't being started about once or > twice a minute. I haven't seen rabbitmq do this except when > restarting. Do you have a "watch rabbitmqctl" or some such? That usually causes that for me. Matthew From luismarianoguerra@REDACTED Sat Apr 10 23:35:58 2010 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Sat, 10 Apr 2010 18:35:58 -0300 Subject: [ANN] efene 0.4 - a language for the erlang VM - released Message-ID: new release of efene. efene is a programming language that runs on the erlang virtual machine. the idea is to provide an alternative syntax to erlang that is most suitable for people coming from languages like Java, C, C++, C#, Javascript. the language is almost 100% compatible with erlang (and will be), the compiler allows to translate an efene source file into a readable erlang one or compile it directly to bytecode. It also adds some syntactic sugar in some places to make some tasks easier. changes * if statement syntax modifies to look more C/C++ like if Num == 1 { io.format("is one~n") } else if Num == 2 { io.format("is two~n") } else { io.format("not one not two~n") } * case statement syntax modifies to look more C/C++ like switch random.uniform(4) { case 1: io.format("one!~n") break case 2: io.format("two!~n") break case 3: io.format("three!~n") break else: io.format("else~n") break } * try/catch statement syntax modifies to look more C++ like try { make_error(Arg) } catch throw 1 { io.format("throw 1~n") } catch exit 2 { io.format("exit 2~n") } catch error (some fancy tuple 3) { io.format("error~n") } else { io.format("else~n") } * record syntax supported # create record instance R = person[firstname="Bob" lastname="Esponja" mail="bob@REDACTED"] # modify record R1 = person.R[firstname="Mariano" lastname="Guerra"] # access record attribute person.R.firstname # record pattern matching is_bob = fn (person[firstname="Bob"]) { true } (_) { false } for a complete reference see the LanguageReference: http://wiki.github.com/marianoguerra/efene/languagereference Participate a mailing list is available at librelist just send a mail to efene@REDACTED to subscribe. From lemenkov@REDACTED Sun Apr 11 08:06:18 2010 From: lemenkov@REDACTED (Peter Lemenkov) Date: Sun, 11 Apr 2010 10:06:18 +0400 Subject: Heads up! Erlang package becomes modularized in Fedora. In-Reply-To: <1270908426.15925.17.camel@arekh.okg> References: <1270908426.15925.17.camel@arekh.okg> Message-ID: Hello! 2010/4/10 Nicolas Mailhot : > So you are proposing a metapackage. Fedora has historically frowned at > metapackages, we prefer to create comps groups to bundle multiple > packages together. Sorry, but this looks like purely non-technical argument for me (I mean using verbs like "prefer", "frown" and so on). I, for example, prefer meta-package, Fedora Haskellers prefer metapackage (haskell-platform) too. My scheme introduces zero maintenance efforts for those who are not interested in using modularized erlang package, while your one (with existing for a while compatible package) will definitely cause issues and complaints. > The one problem a comps group won't solve is upgrade paths, but that can > be handled by a temporary metapackage In fact this raises more issues than solutions, so I still believe that my proposed scheme is better than scheme with comps groups.. -- With best regards, Peter Lemenkov. From kuenishi@REDACTED Sun Apr 11 11:12:27 2010 From: kuenishi@REDACTED (UENISHI Kota) Date: Sun, 11 Apr 2010 18:12:27 +0900 Subject: [erlang-questions] automatically net-configuring node with net_kernel:start/1 In-Reply-To: <20100410195142.GB82910@alumni.caltech.edu> References: <20100410195142.GB82910@alumni.caltech.edu> Message-ID: Thanks, Anthony. I'll check it out (It seems same idea and has better implementation than me!). Of course an implementation of Bonjour, or other standard rendez-vouz protocol would be much better. On Sun, Apr 11, 2010 at 4:51 AM, Anthony Molinaro wrote: > You might check out nodefinder > > http://code.google.com/p/nodefinder/ > > It provides various mechanisms for self configuration including multicast, > ec2 tools or s3. > > These all end up using net_adm:ping to join the group, so probably would > not exhibit the problems you are seeing. > > We used this quite extensively in ec2, to dynamically grow and shrink > our erlang cluster. > > -Anthony > > On Thu, Apr 08, 2010 at 04:32:21PM -0800, UENISHI Kota wrote: >> I'm writing a self-configuring node cluster with net_kernel:start/1. >> It starts without -name nor -sname option and gets its IP adress from >> inet:getif/0. >> And they configure themselves with net_kernel:start/1 as follows: >> >> net_kernel:start( [ 'appname@REDACTED', longnames ] ). >> >> Servers find each other by sending its longname by UDP broadcasting, >> and they connects each other. >> >> The problem is, starting erl without -name option it doesn't kick epmd >> and following net_kernel:start/1 fails in tcp_inet_dist:listen (maybe >> because of epmd not running). >> And another way - starting with -name option with bad ip address ( or >> local ip) makes net_kernel:start/1 result in {error, not_allowed}. >> >> Is it a wrong to think that if epmd would start when >> net_kernel:start/1 had been called, isn't it? >> >> -- >> UENISHI Kota from PC :D >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > -- > ------------------------------------------------------------------------ > Anthony Molinaro ? ? ? ? ? ? ? ? ? ? ? ? ? > -- UENISHI Kota from PC :D From roberto@REDACTED Sun Apr 11 13:49:28 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 11 Apr 2010 13:49:28 +0200 Subject: ssl and {active, once} - bug? Message-ID: dear all, i really hope some kind sould can take me out of this. i'm using ssl to provide an https service. i start a ssl listener with the following options: [binary, {packet,http}, {ip,{0,0,0,0}}, {reuseaddr,true}, {active,false}, {backlog,128}, {certfile, "/Users/roberto/Code/test/server.pem"}, {keyfile, "/Users/roberto/Code/test/privkey.pem"}, {password,"test"}] once i have the listening socket, i accept incoming requests like this: accept(ListenSocket) -> case ssl:transport_accept(ListenSocket) of {ok, NewSocket} -> case ssl:ssl_accept(NewSocket) of ok -> {ok, NewSocket}; {error, Reason} -> {error, Reason} end; {error, Reason} -> {error, Reason} end. once i get the NewSocket, i spawn a controlling process with Pid and i set it as controller of NewSocket: ssl:controlling_process(NewSocket, Pid) the controlling process has the following loop [abstract]: request(#c{sock = Sock, recv_timeout = RecvTimeout} = C, Req) -> ssl:setopts(Sock, [{active, once}]), ?LOG_DEBUG("waiting http request line", []), receive {ssl, Sock, {http_request, Method, Path, Version}} -> ?LOG_DEBUG("received full headers of a new HTTP packet", []), headers(C, Req#req{vsn = Version, method = Method, uri = Path, connection = default_connection(Version)}, []); {ssl, Sock, {http_error, "\r\n"}} -> request(C, Req); {ssl, Sock, {http_error, "\n"}} -> request(C, Req); {ssl, Sock, _Other} when is_list(_Other) -> ?LOG_WARNING("invalid tcp data received for ~p socked :~p", [SocketMode, _Other]), exit(normal); _Other -> ?LOG_WARNING("tcp error on incoming request: ~p", [_Other]), exit({error, unknown data}); after RecvTimeout -> ?LOG_DEBUG("normal receive timeout, exit", []), close(Sock, SocketMode), exit(normal) end. when i run my server and i point a browser to localhost, it all goes perfectly until i see the debug log "waiting http request line" printed in console, then i get: =ERROR REPORT==== 11-Apr-2010::13:21:45 === ** Generic server <0.45.0> terminating ** Last message in was {http,#Port<0.611>, {http_request,'GET',{abs_path,"/?id=2"},{1,1}}} ** When Server state == {st,acceptor,<0.42.0>,<0.47.0>,<0.47.0>,6,once, [{mode,binary}, {packet,http}, {ip,{0,0,0,0}}, {active,once}, {nodelay,false}, {verify,0}, {depth,1}], {sslsocket,6,<0.45.0>}, #Port<0.611>,50664,open,true,false} ** Reason for termination == ** {error,{badinfo,{http,#Port<0.611>, {http_request,'GET',{abs_path,"/"},{1,1}}}}} now: 1. i see that the incoming message has format {http, Sock, Data}. shouldn't it be {ssl, Sock, Data} as per http://www.erlang.org/doc/man/ssl.html: "When a socket is in active mode (the default), data from the socket is delivered to the owner of the socket in the form of messages: * {ssl, Socket, Data} * {ssl_closed, Socket} * {ssl_error, Socket, Reason}" i'm using R13B01 so maybe this has changed. 2. why am i receiving a badinfo message ["Call not recognized for current mode (active or passive) and state of socket"], and what can i do to solve this? thank you in advance, r. From roberto@REDACTED Sun Apr 11 19:39:26 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 11 Apr 2010 19:39:26 +0200 Subject: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: i managed to check that the badinfo error messages is received as soon as i call ssl:setopts(Sock, [{active, once}]). tried also with latest R13B04, exact same results. real lost here. r. > now: > > 1. i see that the incoming message has format {http, Sock, Data}. > shouldn't it be {ssl, Sock, Data} as per > http://www.erlang.org/doc/man/ssl.html: > > "When a socket is in active mode (the default), data from the socket > is delivered to the owner of the socket in the form of messages: > > ? ?* {ssl, Socket, Data} > ? ?* {ssl_closed, Socket} > ? ?* {ssl_error, Socket, Reason}" > > i'm using R13B01 so maybe this has changed. > > 2. why am i receiving a badinfo message ["Call not recognized for > current mode (active or passive) and state of socket"], and what can i > do to solve this? > > > thank you in advance, > > r. From tony@REDACTED Sun Apr 11 21:01:59 2010 From: tony@REDACTED (Tony Rogvall) Date: Sun, 11 Apr 2010 21:01:59 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: What is the value of Sock ? io:format /Tony On 11 apr 2010, at 19.39, Roberto Ostinelli wrote: > i managed to check that the badinfo error messages is received as soon as i call > > ssl:setopts(Sock, [{active, once}]). > > tried also with latest R13B04, exact same results. > > real lost here. > > r. > > >> now: >> >> 1. i see that the incoming message has format {http, Sock, Data}. >> shouldn't it be {ssl, Sock, Data} as per >> http://www.erlang.org/doc/man/ssl.html: >> >> "When a socket is in active mode (the default), data from the socket >> is delivered to the owner of the socket in the form of messages: >> >> * {ssl, Socket, Data} >> * {ssl_closed, Socket} >> * {ssl_error, Socket, Reason}" >> >> i'm using R13B01 so maybe this has changed. >> >> 2. why am i receiving a badinfo message ["Call not recognized for >> current mode (active or passive) and state of socket"], and what can i >> do to solve this? >> >> >> thank you in advance, >> >> r. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From roberto@REDACTED Sun Apr 11 21:36:40 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 11 Apr 2010 21:36:40 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: 2010/4/11 Tony Rogvall : > What is the value of Sock ? > ?io:format hi tony! socket is {sslsocket,6,<0.52.0>}, controlling_process is <0.53.0>. error is: =ERROR REPORT==== 11-Apr-2010::21:33:39 === ** Generic server <0.52.0> terminating ** Last message in was {http,#Port<0.688>, {http_request,'GET', {abs_path,"/favicon.ico"}, {1,1}}} ** When Server state == {st,acceptor,<0.42.0>,<0.53.0>,<0.53.0>,6,once, [{mode,binary}, {packet,http}, {ip,{0,0,0,0}}, {active,once}, {nodelay,false}, {verify,0}, {depth,1}], {sslsocket,6,<0.52.0>}, #Port<0.688>,56916,open,true,false} ** Reason for termination == ** {error, {badinfo, {http,#Port<0.688>, {http_request,'GET',{abs_path,"/"},{1,1}}}}} From roberto@REDACTED Sun Apr 11 23:03:42 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 11 Apr 2010 23:03:42 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: ok, here's the complete code to reproduce the error. this is the whole picture, and i hope someone can tell me what i am doing wrong. %%%%%%%%%%%%%%%%%%%%%%%%%% -module(sslbug). -compile(export_all). start(Port) -> application:start(ssl), Options = [ binary, {packet, http}, {reuseaddr, true}, {active, false}, {backlog, 128}, {certfile, "/Users/roberto/Code/misultin/server.pem"}, {keyfile, "/Users/roberto/Code/misultin/privkey.pem"}, {password, "roberto"} ], {ok, ListenSocket} = ssl:listen(Port, Options), {ok, Sock} = ssl:transport_accept(ListenSocket), ok = ssl:ssl_accept(Sock), ControllerPid = spawn(fun() -> receive set -> ok end, controller_loop(Sock) end), ok = ssl:controlling_process(Sock, ControllerPid), ControllerPid ! set. controller_loop(Sock) -> ssl:setopts(Sock, [{active, once}]), receive _Any -> ok end, ssl:close(Sock). %%%%%%%%%%%%%%%%%%%%%%%%%% compile and run this code with c(sslbug), then start the server with the command sslbug:start(8080). point your browser to https://localhost:8080 and you should receive the following error: =ERROR REPORT==== 11-Apr-2010::23:00:20 === ** Generic server <0.80.0> terminating ** Last message in was {http,#Port<0.2159>, {http_request,'GET',{abs_path,"/"},{1,1}}} ** When Server state == {st,acceptor,<0.70.0>,<0.81.0>,<0.81.0>,6,once, [{mode,binary}, {packet,http}, {active,once}, {nodelay,false}, {ip,{0,0,0,0}}, {verify,0}, {depth,1}], {sslsocket,6,<0.80.0>}, #Port<0.2159>,57732,open,true,false} ** Reason for termination == ** {error,{badinfo,{http,#Port<0.2159>, {http_request,'GET',{abs_path,"/"},{1,1}}}}} From roberto@REDACTED Sun Apr 11 23:36:15 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 11 Apr 2010 23:36:15 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: 2010/4/11 Tony Rogvall : > Send a message to the new process after the > call to controlling process, wait for that message before setting {active,once}. > this is exactly what i do, you might have read my latest mail with a full module to reproduce: http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:50624:201004:fpopocbfkpppecdembbe as you can see, i do set the controlling process before setting {active, once}, and no race can happen there. ?r From tony@REDACTED Mon Apr 12 00:22:10 2010 From: tony@REDACTED (Tony Rogvall) Date: Mon, 12 Apr 2010 00:22:10 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: Yepp. I saw your code after I wrote my mail. Tried it. The boring things is that ssl does not handle http packets (nor http header mode) in active/once mode, you must use passive mode. I do not really no why. Have a look in yaws to see how it's done there. But if you call crypto:start add the option {ssl_imp, new} the code will work :-) /Tony On 11 apr 2010, at 23.36, Roberto Ostinelli wrote: > 2010/4/11 Tony Rogvall : >> Send a message to the new process after the >> call to controlling process, wait for that message before setting {active,once}. >> > > this is exactly what i do, you might have read my latest mail with a > full module to reproduce: > http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:50624:201004:fpopocbfkpppecdembbe > > as you can see, i do set the controlling process before setting > {active, once}, and no race can happen there. > > ?r From tony@REDACTED Sun Apr 11 23:30:07 2010 From: tony@REDACTED (Tony Rogvall) Date: Sun, 11 Apr 2010 23:30:07 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: Looks like the ssl_broker is not ready for an http message ? I am not sure why this message end up there. May be a raise condition. Probably the active mode is set in the new controlling process before the before actual hand over is done. Try sync. Send a message to the new process after the call to controlling process, wait for that message before setting {active,once}. I normally write the code in a way that the accept is done in the spawned (child) process, and then sync back to the listener process. This way there is no need for contolling_process call. /Tony On 11 apr 2010, at 21.36, Roberto Ostinelli wrote: > 2010/4/11 Tony Rogvall : >> What is the value of Sock ? >> io:format > > hi tony! > > socket is {sslsocket,6,<0.52.0>}, controlling_process is <0.53.0>. error is: > > =ERROR REPORT==== 11-Apr-2010::21:33:39 === > ** Generic server <0.52.0> terminating > ** Last message in was {http,#Port<0.688>, > {http_request,'GET', > {abs_path,"/favicon.ico"}, > {1,1}}} > ** When Server state == {st,acceptor,<0.42.0>,<0.53.0>,<0.53.0>,6,once, > [{mode,binary}, > {packet,http}, > {ip,{0,0,0,0}}, > {active,once}, > {nodelay,false}, > {verify,0}, > {depth,1}], > {sslsocket,6,<0.52.0>}, > #Port<0.688>,56916,open,true,false} > ** Reason for termination == > ** {error, > {badinfo, > {http,#Port<0.688>, > {http_request,'GET',{abs_path,"/"},{1,1}}}}} From navaneethanit@REDACTED Mon Apr 12 03:16:13 2010 From: navaneethanit@REDACTED (Saravana) Date: Sun, 11 Apr 2010 18:16:13 -0700 (PDT) Subject: call a module within a mod Message-ID: i have created a web application in erlang using nitrogen,after this i wanna add a module to execute when i call that application..... web_index.erl process.erl i wanna call the process.erl inside web_index.erl,whenever the web_index.erl executes same time process.erl need to execute . From kenji.rikitake@REDACTED Mon Apr 12 03:49:56 2010 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Mon, 12 Apr 2010 10:49:56 +0900 Subject: [erlang-questions] ssh password protected private keys In-Reply-To: References: Message-ID: <20100412014956.GA37791@k2r.org> I don't see any ongoing work about password-protected ssh private keys. I'd like to have one. Kenji Rikitake In the message dated Fri, Apr 09, 2010 at 08:52:51AM -0500, Daniel Goertzen writes: > Does anybody know if password protected ssh private keys are in the works? > > I looked at the Python module "paramiko" and at first glance key decryption > doesn't look too tricky. I have some ambitions to add this support to the > erlang ssh module, but obviously would like to know if someone else is > working on it. > > Thanks, > Dan. From ingela@REDACTED Mon Apr 12 09:00:34 2010 From: ingela@REDACTED (Ingela Andin) Date: Mon, 12 Apr 2010 09:00:34 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: Hi! Actually it is the new ssl implementation that has driven the http packet support to become a supported feature in the inet driver. I think the reason it does proably not work with the old ssl is that it was never a supported feature when some one still actively cared about that implementation. We are currently working hard at making the new ssl implementation complete enough to be able to phase out the old one. Regards Ingela Erlang/OTP Team, Ericsson AB 2010/4/12 Tony Rogvall : > Yepp. I saw your code after I wrote my mail. > Tried it. > The boring things is that ssl does not handle http packets (nor http header mode) in active/once mode, > you must use passive mode. I do not really no why. Have a look in yaws to see how it's done there. > But if you call crypto:start add the option {ssl_imp, new} the code will work :-) > > /Tony > > > > > On 11 apr 2010, at 23.36, Roberto Ostinelli wrote: > >> 2010/4/11 Tony Rogvall : >>> Send a message to the new process after the >>> call to controlling process, wait for that message before setting {active,once}. >>> >> >> this is exactly what i do, you might have read my latest mail with a >> full module to reproduce: >> http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:50624:201004:fpopocbfkpppecdembbe >> >> as you can see, i do set the controlling process before setting >> {active, once}, and no race can happen there. >> >> ?r > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ingela.andin@REDACTED Mon Apr 12 09:16:58 2010 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 12 Apr 2010 09:16:58 +0200 Subject: [erlang-questions] ssh password protected private keys In-Reply-To: <20100412014956.GA37791@k2r.org> References: <20100412014956.GA37791@k2r.org> Message-ID: Hi! There is a solution for password protected private keys in the public_key application (used by new ssl). The plan is that the erlang ssh application should be rewritten to use public_key too. If you would like to speed up that implementation you are welcome to submit a patch for that rewrite as there are currently some other things higher on our priority list. Regards Ingela Erlang/OTP team, Ericsson AB 2010/4/12 Kenji Rikitake : > I don't see any ongoing work about password-protected ssh private keys. > I'd like to have one. > Kenji Rikitake > > In the message > dated Fri, Apr 09, 2010 at 08:52:51AM -0500, > Daniel Goertzen writes: >> Does anybody know if password protected ssh private keys are in the works? >> >> I looked at the Python module "paramiko" and at first glance key decryption >> doesn't look too tricky. ?I have some ambitions to add this support to the >> erlang ssh module, but obviously would like to know if someone else is >> working on it. >> >> Thanks, >> Dan. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tony@REDACTED Mon Apr 12 10:25:01 2010 From: tony@REDACTED (Tony Rogvall) Date: Mon, 12 Apr 2010 10:25:01 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: <4B591B0C-CDB5-4E9F-B0DA-1B5BD95ADA69@rogvall.se> Ouch! ;-) /Tony On 12 apr 2010, at 09.00, Ingela Andin wrote: > Hi! > > Actually it is the new ssl implementation that has driven the http > packet support to become a supported feature in the inet driver. I > think the reason it does proably not work with the old ssl is that it > was never a supported feature when some one still actively cared about > that implementation. We are currently working hard at making the new > ssl implementation complete enough to be able to phase out the old > one. > > Regards Ingela Erlang/OTP Team, Ericsson AB > > 2010/4/12 Tony Rogvall : >> Yepp. I saw your code after I wrote my mail. >> Tried it. >> The boring things is that ssl does not handle http packets (nor http header mode) in active/once mode, >> you must use passive mode. I do not really no why. Have a look in yaws to see how it's done there. >> But if you call crypto:start add the option {ssl_imp, new} the code will work :-) >> >> /Tony >> >> >> >> >> On 11 apr 2010, at 23.36, Roberto Ostinelli wrote: >> >>> 2010/4/11 Tony Rogvall : >>>> Send a message to the new process after the >>>> call to controlling process, wait for that message before setting {active,once}. >>>> >>> >>> this is exactly what i do, you might have read my latest mail with a >>> full module to reproduce: >>> http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:50624:201004:fpopocbfkpppecdembbe >>> >>> as you can see, i do set the controlling process before setting >>> {active, once}, and no race can happen there. >>> >>> ?r >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From roberto@REDACTED Mon Apr 12 12:46:21 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 12 Apr 2010 12:46:21 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: 2010/4/12 Ingela Andin : > Hi! > > Actually it is the new ssl implementation that has driven the http > packet support to become a supported feature in the inet driver. I > think the reason it does proably not work with the old ssl is that it > was never a supported feature when some one still actively cared about > that implementation. We are currently working hard at making the new > ssl implementation complete enough to be able to phase out the old > one. > > Regards Ingela Erlang/OTP Team, Ericsson AB hi ingela, thank you for your anwser. tony did give a working suggestion [code here below], can this be a viable solution meanwhile, or do you have a better way to implement this? also, do you happen to know minimum erlang version which allows the implementation here below? 2010/4/12 Tony Rogvall : > But if you call crypto:start add the option {ssl_imp, new} the code will work :-) indeed. here's the code for others' reference, now working. %%%%%%%%%%%%%%%%%%%%%%%%%% -module(sslbug). -compile(export_all). start(Port) -> application:start(ssl), application:start(crypto), Options = [ binary, {packet, http}, {reuseaddr, true}, {active, false}, {backlog, 128}, {ssl_imp, new}, {certfile, "/Users/roberto/Code/sslbug/server.pem"}, {keyfile, "/Users/roberto/Code/sslbug/privkey.pem"}, {password, "roberto"} ], {ok, ListenSocket} = ssl:listen(Port, Options), {ok, Sock} = ssl:transport_accept(ListenSocket), ok = ssl:ssl_accept(Sock), ControllerPid = spawn(fun() -> receive set -> ok end, controller_loop(Sock) end), ok = ssl:controlling_process(Sock, ControllerPid), ControllerPid ! set. controller_loop(Sock) -> ssl:setopts(Sock, [{active, once}]), receive _Any -> io:format("ok, received ~p~n", [_Any]) end, ssl:close(Sock). %%%%%%%%%%%%%%%%%%%%%%%%%% From ingela@REDACTED Mon Apr 12 14:01:00 2010 From: ingela@REDACTED (Ingela Andin) Date: Mon, 12 Apr 2010 14:01:00 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: Hi! Tonys code will accomplish that you run the new ssl-implementation instead of the old one. I would not run the new ssl in older versions than R13 even though there is a primitive version in R12. The main reason for not making the new one default yet is that we want it to be as complete as possible before that, but that does not necessarily mean it is not complete enough for your need. Regards Ingela Erlang/OTP team, Ericsson AB 2010/4/12 Roberto Ostinelli : > 2010/4/12 Ingela Andin : >> Hi! >> >> Actually it is the new ssl implementation that has driven the http >> packet support to become a supported feature in the inet driver. I >> think the reason it does proably not work with the old ssl is that it >> was never a supported feature when some one still actively cared about >> that implementation. We are currently working hard at making the new >> ssl implementation complete enough to be able to phase out the old >> one. >> >> Regards Ingela Erlang/OTP Team, Ericsson AB > > hi ingela, > > thank you for your anwser. tony did give a working suggestion [code > here below], can this be a viable solution meanwhile, or do you have a > better way to implement this? > > also, do you happen to know minimum erlang version which allows the > implementation here below? > > > 2010/4/12 Tony Rogvall : >> But if you call crypto:start add the option {ssl_imp, new} the code will work :-) > > indeed. > > here's the code for others' reference, now working. > > > %%%%%%%%%%%%%%%%%%%%%%%%%% > > -module(sslbug). > -compile(export_all). > > start(Port) -> > ? ? ? ?application:start(ssl), > ? ? ? ?application:start(crypto), > ? ? ? ?Options = [ > ? ? ? ? ? ? ? ?binary, {packet, http}, {reuseaddr, true}, {active, false}, > ? ? ? ? ? ? ? ?{backlog, 128}, {ssl_imp, new}, > ? ? ? ? ? ? ? ?{certfile, "/Users/roberto/Code/sslbug/server.pem"}, > ? ? ? ? ? ? ? ?{keyfile, "/Users/roberto/Code/sslbug/privkey.pem"}, > ? ? ? ? ? ? ? ?{password, "roberto"} > ? ? ? ?], > ? ? ? ?{ok, ListenSocket} = ssl:listen(Port, Options), > ? ? ? ?{ok, Sock} = ssl:transport_accept(ListenSocket), > ? ? ? ?ok = ssl:ssl_accept(Sock), > ? ? ? ?ControllerPid = spawn(fun() -> > ? ? ? ? ? ? ? ?receive > ? ? ? ? ? ? ? ? ? ? ? ?set -> ok > ? ? ? ? ? ? ? ?end, > ? ? ? ? ? ? ? ?controller_loop(Sock) > ? ? ? ?end), > ? ? ? ?ok = ssl:controlling_process(Sock, ControllerPid), > ? ? ? ?ControllerPid ! set. > > controller_loop(Sock) -> > ? ? ? ?ssl:setopts(Sock, [{active, once}]), > ? ? ? ?receive > ? ? ? ? ? ? ? ?_Any -> > ? ? ? ? ? ? ? ? ? ? ? ?io:format("ok, received ~p~n", [_Any]) > ? ? ? ?end, > ? ? ? ?ssl:close(Sock). > > %%%%%%%%%%%%%%%%%%%%%%%%%% > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jeraymond@REDACTED Mon Apr 12 14:29:25 2010 From: jeraymond@REDACTED (Jeremy Raymond) Date: Mon, 12 Apr 2010 05:29:25 -0700 (PDT) Subject: How to use rb:filter/1? In-Reply-To: References: Message-ID: <9904b078-93a4-4e50-b15e-b8eef4ec3389@z4g2000yqa.googlegroups.com> Does anyone have an idea? On Apr 9, 12:32?pm, Jeremy Raymond wrote: > Hi, > > I'm trying to figure out how to use rb:fitler/1. What I'm unclear on > is what the {Key, Value} items are matching against in the reports. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED From navaneethanit@REDACTED Mon Apr 12 18:33:26 2010 From: navaneethanit@REDACTED (Saravana) Date: Mon, 12 Apr 2010 09:33:26 -0700 (PDT) Subject: Display problem while calling sub mod Message-ID: <8a48972c-e777-49c5-be5a-93ada4bf7bb2@w42g2000yqm.googlegroups.com> event(continue) -> RES=wf:f("~B",processes:max(1000)), wf:flash(RES), when i execute these statements to display in my webpage, I get error =INFO REPORT==== 12-Apr-2010::21:35:51 === ERROR: error badarg [{io_lib,format,["~B",ok]}, {wf_utils,f,2}, {web_index,event,1}, {wf_handle_postback,run_module_event,2}, {wf_handle_postback,handle_normal_request,3}, {wf_inets,do,3}, {httpd_response,traverse_modules,2}, {httpd_response,generate_and_send_response,1}] processes.erl is another file which gives the output the below: Maximum allowed processes:32768 PROCESSING TIME FOR EACH REQUEST(CPU TIME AND ELAPSED TIME AVGLY=5.0 (5.5) microseconds what the problem is "I need to display this output in my webpage..How to do it??let me know,I hope you understood,...if any doubt please ask friend From attila.r.nohl@REDACTED Mon Apr 12 19:01:03 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 12 Apr 2010 19:01:03 +0200 Subject: [erlang-questions] Display problem while calling sub mod In-Reply-To: <8a48972c-e777-49c5-be5a-93ada4bf7bb2@w42g2000yqm.googlegroups.com> References: <8a48972c-e777-49c5-be5a-93ada4bf7bb2@w42g2000yqm.googlegroups.com> Message-ID: 2010/4/12, Saravana : > > event(continue) -> > RES=wf:f("~B",processes:max(1000)), > wf:flash(RES), > > when i execute these statements to display in my webpage, I get error I haven't got the faintest idea what this function does, but based on the info report I'd try something like this: RES=wf:f("~B",[processes:max(1000)]), By the way, are you sure that the processes:max returns what you expect? From navaneethanit@REDACTED Mon Apr 12 19:14:44 2010 From: navaneethanit@REDACTED (Saravana) Date: Mon, 12 Apr 2010 10:14:44 -0700 (PDT) Subject: Display problem while calling sub mod In-Reply-To: References: <8a48972c-e777-49c5-be5a-93ada4bf7bb2@w42g2000yqm.googlegroups.com> Message-ID: <4f329f65-eccc-4002-b67f-930702d629f4@i37g2000yqn.googlegroups.com> i simply use the below line processes:max(1000), to get the output in terminal Maximum allowed processes:32768 PROCESSING TIME FOR EACH REQUEST(CPU TIME AND ELAPSED TIME AVGLY=10.0 (7.0) mseconds when i used to give the below lines to display the result in my webpage , [RES]=wf:f("~B",[processes:max(1000)]), wf:flash([RES]), I get error, =INFO REPORT==== 12-Apr-2010::22:38:29 === ERROR: error badarg [{io_lib,format,["~B",[ok]]}, {wf_utils,f,2}, {web_index,event,1}, {wf_handle_postback,run_module_event,2}, {wf_handle_postback,handle_normal_request,3}, {wf_inets,do,3}, {httpd_response,traverse_modules,2}, {httpd_response,generate_and_send_response,1}] On Apr 12, 10:01?pm, Attila Rajmund Nohl wrote: > 2010/4/12, Saravana : > > > > > event(continue) -> > > ? ? ? RES=wf:f("~B",processes:max(1000)), > > ? ?wf:flash(RES), > > > when i execute these statements to display in my webpage, I get error > > I haven't got the faintest idea what this function does, but based on > the info report I'd try something like this: > > RES=wf:f("~B",[processes:max(1000)]), > > By the way, are you sure that the processes:max returns what you expect? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED From ryanhuffman@REDACTED Mon Apr 12 20:04:05 2010 From: ryanhuffman@REDACTED (Ryan Huffman) Date: Mon, 12 Apr 2010 11:04:05 -0700 Subject: [erlang-questions] Re: Display problem while calling sub mod In-Reply-To: <4f329f65-eccc-4002-b67f-930702d629f4@i37g2000yqn.googlegroups.com> References: <8a48972c-e777-49c5-be5a-93ada4bf7bb2@w42g2000yqm.googlegroups.com> <4f329f65-eccc-4002-b67f-930702d629f4@i37g2000yqn.googlegroups.com> Message-ID: wf:f("~B",[processes:max(1000)]) ~B is for displaying integers, but you are trying to display an atom (ok). Try ~w instead. wf:f("~w",[processes:max(1000)]) On Mon, Apr 12, 2010 at 10:14 AM, Saravana wrote: > i simply use the below line > > processes:max(1000), > > to get the output in terminal > > Maximum allowed processes:32768 > PROCESSING TIME FOR EACH REQUEST(CPU TIME AND ELAPSED TIME AVGLY=10.0 > (7.0) mseconds > > when i used to give the below lines to display the result in my > webpage , > [RES]=wf:f("~B",[processes:max(1000)]), > wf:flash([RES]), > I get error, > > =INFO REPORT==== 12-Apr-2010::22:38:29 === > ERROR: error > badarg > [{io_lib,format,["~B",[ok]]}, > {wf_utils,f,2}, > {web_index,event,1}, > {wf_handle_postback,run_module_event,2}, > {wf_handle_postback,handle_normal_request,3}, > {wf_inets,do,3}, > {httpd_response,traverse_modules,2}, > {httpd_response,generate_and_send_response,1}] > > > On Apr 12, 10:01 pm, Attila Rajmund Nohl > wrote: > > 2010/4/12, Saravana : > > > > > > > > > event(continue) -> > > > RES=wf:f("~B",processes:max(1000)), > > > wf:flash(RES), > > > > > when i execute these statements to display in my webpage, I get error > > > > I haven't got the faintest idea what this function does, but based on > > the info report I'd try something like this: > > > > RES=wf:f("~B",[processes:max(1000)]), > > > > By the way, are you sure that the processes:max returns what you expect? > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > Seehttp://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From lenartlad@REDACTED Mon Apr 12 19:53:22 2010 From: lenartlad@REDACTED (Ladislav Lenart) Date: Mon, 12 Apr 2010 19:53:22 +0200 Subject: [erlang-questions] Re: Display problem while calling sub mod In-Reply-To: <4f329f65-eccc-4002-b67f-930702d629f4@i37g2000yqn.googlegroups.com> References: <8a48972c-e777-49c5-be5a-93ada4bf7bb2@w42g2000yqm.googlegroups.com> <4f329f65-eccc-4002-b67f-930702d629f4@i37g2000yqn.googlegroups.com> Message-ID: <4BC35E12.2000206@volny.cz> Hello. From the error message it seems that the function processes:max(1000) prints the info to the console as a side effect. The function does not return the printed string, just an atom ok. Later you try to format that atom using the format string: "~B" which fails. If processes:max/1 is your own function, try using io_lib:format/2 which returns the iolist instead of io:format/2 which prints the string directly to the terminal console and returns atom ok. HTH, Ladislav Lenart Saravana wrote: > i simply use the below line > > processes:max(1000), > > to get the output in terminal > > Maximum allowed processes:32768 > PROCESSING TIME FOR EACH REQUEST(CPU TIME AND ELAPSED TIME AVGLY=10.0 > (7.0) mseconds > > when i used to give the below lines to display the result in my > webpage , > [RES]=wf:f("~B",[processes:max(1000)]), > wf:flash([RES]), > I get error, > > =INFO REPORT==== 12-Apr-2010::22:38:29 === > ERROR: error > badarg > [{io_lib,format,["~B",[ok]]}, > {wf_utils,f,2}, > {web_index,event,1}, > {wf_handle_postback,run_module_event,2}, > {wf_handle_postback,handle_normal_request,3}, > {wf_inets,do,3}, > {httpd_response,traverse_modules,2}, > {httpd_response,generate_and_send_response,1}] > > > On Apr 12, 10:01 pm, Attila Rajmund Nohl > wrote: >> 2010/4/12, Saravana : >> >> >> >>> event(continue) -> >>> RES=wf:f("~B",processes:max(1000)), >>> wf:flash(RES), >>> when i execute these statements to display in my webpage, I get error >> I haven't got the faintest idea what this function does, but based on >> the info report I'd try something like this: >> >> RES=wf:f("~B",[processes:max(1000)]), >> >> By the way, are you sure that the processes:max returns what you expect? >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> Seehttp://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From jderick@REDACTED Tue Apr 13 00:10:09 2010 From: jderick@REDACTED (John Erickson) Date: Mon, 12 Apr 2010 15:10:09 -0700 Subject: link Message-ID: Hello, I have created a bunch of processes and called link() on them from a root process. I thought this would mean that when one of them died, they would all die. However, I see in my log file one of the processes is 'not responding' but the others run along oblivious. Is there a way to have them all die when this happens? From francesco@REDACTED Tue Apr 13 00:31:26 2010 From: francesco@REDACTED (Francesco Cesarini) Date: Mon, 12 Apr 2010 23:31:26 +0100 Subject: Submit a talk to the London Erlang Factory.. Deadline approaching Message-ID: <4BC39F3E.7090508@erlang-solutions.com> Do you have what it takes to lead a conference session or tutorial? Have you made an interesting innovation, open source application or product with Erlang/OTP? Have you used Erlang in a real-world project and want to present a case study? Or have you developed a cool tool and want to give a hands-on tutorial? Tell us about it. We welcome submissions from developers, system designers, testers, community leaders, inventors, CTOs, evangelists, students, researchers and entrepreneurs alike. Participants at the Erlang Factory want to hear about how Erlang/OTP is used real-world scenarios, talks about tricks, tools and applications which increase productivity or enable developers to write better code. http://www.erlang-factory.com/conference/London2010/submit_talk The deadline for talk proposals is the 30th of April, but if there are too many submissions (as was the case in SF) the earlier you submit, the better the chances (We already have 20 confirmed speakers). The conference itself is in central London on the 10th and 11th of June. Some of the topics we hope to see at the Erlang Factory conference include: * Erlang and Web Development * Tools and Gadgets * Erlang and Test Driven Development * Give me a Break from Erlang (Non Erlang talks) * Enterprise Integration * Cool Applications and case studies * Erlang and Instant Messaging * Erlang databases If your talk gets accepted, you will get free access to the conference and attend the speaker's dinner. Looking forward to your submissions! Francesco, Joanna and the Programme Committee --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From igorrs@REDACTED Tue Apr 13 03:54:12 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Mon, 12 Apr 2010 22:54:12 -0300 Subject: Performance of listing a bag Message-ID: Hi. I've run the following test: - Create a Mnesia test table of type bag, with disc_only_copies. - Insert 170000 elements with the same key ({test, 1, 1} to {test, 1, 170000}). This took more than a minute, but I expected it. - Fetch all of those elements at once, by key: mnesia:transaction(fun mnesia:read/1, [{test, 1}]). %% Took less than a second. Good. - List all keys with mnesia: mnesia:transaction(fun mnesia:all_keys/1, [test]). %% Takes forever, consuming a lot of CPU. It seems that, no matter the way I do it (using qlc or only mnesia), I can always list all objects with key 1 very fast, but listing the full table is extremely slow, although the data involved are precisely the same in this example. What's going on here? It's probably due to the implementation of iteration on a dets table, but how does it happen? Thanks. Igor. From navaneethanit@REDACTED Tue Apr 13 05:33:44 2010 From: navaneethanit@REDACTED (Saravana) Date: Mon, 12 Apr 2010 20:33:44 -0700 (PDT) Subject: Display problem while calling sub mod In-Reply-To: References: <8a48972c-e777-49c5-be5a-93ada4bf7bb2@w42g2000yqm.googlegroups.com> <4f329f65-eccc-4002-b67f-930702d629f4@i37g2000yqn.googlegroups.com> Message-ID: <17d2fbf6-8400-489c-a350-a222af9ee433@11g2000yqr.googlegroups.com> again it appears error friend, =INFO REPORT==== 13-Apr-2010::08:58:52 === ERROR: error {badmatch,"ok"} [{web_index,event,1}, {wf_handle_postback,run_module_event,2}, {wf_handle_postback,handle_normal_request,3}, {wf_inets,do,3}, {httpd_response,traverse_modules,2}, {httpd_response,generate_and_send_response,1}, {httpd_request_handler,handle_response,1}, On Apr 12, 11:04?pm, Ryan Huffman wrote: > wf:f("~B",[processes:max(1000)]) > > ~B is for displaying integers, but you are trying to display an atom (ok). > Try ~w instead. > > wf:f("~w",[processes:max(1000)]) > > On Mon, Apr 12, 2010 at 10:14 AM, Saravana wrote: > > i simply use the below line > > > ? ? ? ? ? ? ? processes:max(1000), > > > to get the output in terminal > > > Maximum allowed processes:32768 > > PROCESSING TIME FOR EACH REQUEST(CPU TIME AND ELAPSED TIME AVGLY=10.0 > > (7.0) mseconds > > > when i used to give the below lines to display the result in my > > webpage , > > ? ? ? ? ? ? ? [RES]=wf:f("~B",[processes:max(1000)]), > > ? ? ? ? ? ? ? ?wf:flash([RES]), > > I get error, > > > =INFO REPORT==== 12-Apr-2010::22:38:29 === > > ERROR: error > > badarg > > [{io_lib,format,["~B",[ok]]}, > > ?{wf_utils,f,2}, > > ?{web_index,event,1}, > > ?{wf_handle_postback,run_module_event,2}, > > ?{wf_handle_postback,handle_normal_request,3}, > > ?{wf_inets,do,3}, > > ?{httpd_response,traverse_modules,2}, > > ?{httpd_response,generate_and_send_response,1}] > > > On Apr 12, 10:01 pm, Attila Rajmund Nohl > > wrote: > > > 2010/4/12, Saravana : > > > > > event(continue) -> > > > > ? ? ? RES=wf:f("~B",processes:max(1000)), > > > > ? ?wf:flash(RES), > > > > > when i execute these statements to display in my webpage, I get error > > > > I haven't got the faintest idea what this function does, but based on > > > the info report I'd try something like this: > > > > RES=wf:f("~B",[processes:max(1000)]), > > > > By the way, are you sure that the processes:max returns what you expect? > > > > ________________________________________________________________ > > > erlang-questions (at) erlang.org mailing list. > > > Seehttp://www.erlang.org/faq.html > > > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > Seehttp://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > From bernie@REDACTED Tue Apr 13 05:41:25 2010 From: bernie@REDACTED (Bernard Duggan) Date: Tue, 13 Apr 2010 13:41:25 +1000 Subject: [erlang-questions] Re: Display problem while calling sub mod In-Reply-To: <17d2fbf6-8400-489c-a350-a222af9ee433@11g2000yqr.googlegroups.com> References: <8a48972c-e777-49c5-be5a-93ada4bf7bb2@w42g2000yqm.googlegroups.com> <4f329f65-eccc-4002-b67f-930702d629f4@i37g2000yqn.googlegroups.com> <17d2fbf6-8400-489c-a350-a222af9ee433@11g2000yqr.googlegroups.com> Message-ID: <4BC3E7E5.5030503@m5net.com> That's because this line: [RES]=wf:f("~B",[processes:max(1000)]), is telling Erlang that the call will return a single element list (and that it should fail with a badmatch if it doesn't). Instead, it's returning the string "ok", which is a two element list. I'm not clear why, in the thread below, you added the list brackets around RES. B Saravana wrote: > again it appears error friend, > > =INFO REPORT==== 13-Apr-2010::08:58:52 === > ERROR: error > {badmatch,"ok"} > [{web_index,event,1}, > {wf_handle_postback,run_module_event,2}, > {wf_handle_postback,handle_normal_request,3}, > {wf_inets,do,3}, > {httpd_response,traverse_modules,2}, > {httpd_response,generate_and_send_response,1}, > {httpd_request_handler,handle_response,1}, > > > On Apr 12, 11:04 pm, Ryan Huffman wrote: > >> wf:f("~B",[processes:max(1000)]) >> >> ~B is for displaying integers, but you are trying to display an atom (ok). >> Try ~w instead. >> >> wf:f("~w",[processes:max(1000)]) >> >> On Mon, Apr 12, 2010 at 10:14 AM, Saravana wrote: >> >>> i simply use the below line >>> >>> processes:max(1000), >>> >>> to get the output in terminal >>> >>> Maximum allowed processes:32768 >>> PROCESSING TIME FOR EACH REQUEST(CPU TIME AND ELAPSED TIME AVGLY=10.0 >>> (7.0) mseconds >>> >>> when i used to give the below lines to display the result in my >>> webpage , >>> [RES]=wf:f("~B",[processes:max(1000)]), >>> wf:flash([RES]), >>> I get error, >>> >>> =INFO REPORT==== 12-Apr-2010::22:38:29 === >>> ERROR: error >>> badarg >>> [{io_lib,format,["~B",[ok]]}, >>> {wf_utils,f,2}, >>> {web_index,event,1}, >>> {wf_handle_postback,run_module_event,2}, >>> {wf_handle_postback,handle_normal_request,3}, >>> {wf_inets,do,3}, >>> {httpd_response,traverse_modules,2}, >>> {httpd_response,generate_and_send_response,1}] >>> >>> On Apr 12, 10:01 pm, Attila Rajmund Nohl >>> wrote: >>> >>>> 2010/4/12, Saravana : >>>> >>>>> event(continue) -> >>>>> RES=wf:f("~B",processes:max(1000)), >>>>> wf:flash(RES), >>>>> >>>>> when i execute these statements to display in my webpage, I get error >>>>> >>>> I haven't got the faintest idea what this function does, but based on >>>> the info report I'd try something like this: >>>> >>>> RES=wf:f("~B",[processes:max(1000)]), >>>> >>>> By the way, are you sure that the processes:max returns what you expect? >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> Seehttp://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED >>>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> Seehttp://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED >>> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From rapsey@REDACTED Tue Apr 13 07:50:09 2010 From: rapsey@REDACTED (Rapsey) Date: Tue, 13 Apr 2010 07:50:09 +0200 Subject: [erlang-questions] link In-Reply-To: References: Message-ID: If a process dies with exit signal normal (like if the process function stops recursion) the linked processes will not die. If there is any other exit signal, linked processes will die as well. You can call exit(stop) or something when you want to kill all processes as well. Sergej On Tue, Apr 13, 2010 at 12:10 AM, John Erickson wrote: > Hello, I have created a bunch of processes and called link() on them from a > root process. I thought this would mean that when one of them died, they > would all die. However, I see in my log file one of the processes is 'not > responding' but the others run along oblivious. Is there a way to have > them > all die when this happens? > From mazen.harake@REDACTED Tue Apr 13 10:05:29 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 13 Apr 2010 10:05:29 +0200 Subject: [erlang-questions] call a module within a mod In-Reply-To: References: Message-ID: <4BC425C9.7050409@erlang-solutions.com> Hi Saravana, Can you please be more concrete and specific in your question, I think most people will have a hard time understanding what it is you want to achieve. The last sentence makes everything confusing. /Mazen On 12/04/2010 03:16, Saravana wrote: > i have created a web application in erlang using nitrogen,after this i > wanna add a module to execute when i call that application..... > > web_index.erl > process.erl > > i wanna call the process.erl inside web_index.erl,whenever the > web_index.erl executes same time process.erl need to execute . > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From roberto.aloi@REDACTED Tue Apr 13 10:54:53 2010 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Tue, 13 Apr 2010 09:54:53 +0100 Subject: erlangotp.com Message-ID: <4BC4315D.2030203@erlang-solutions.com> Hi all, I've found this nice list of companies/projects using Erlang and I'm wondering who is maintaining it: http://erlangotp.com/wiki/companiesUsingErlang It's a really nice idea, but a couple of entries are still missing. Anyone? Roberto Aloi -- University of Kent - Erlang Solutions Ltd. Twitter: @prof3ta Blog: http://aloiroberto.wordpress.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From paul.a.johnston@REDACTED Tue Apr 13 12:30:45 2010 From: paul.a.johnston@REDACTED (Paul Johnston) Date: Tue, 13 Apr 2010 11:30:45 +0100 Subject: Installing mysql driver Message-ID: Hi Can anyone give any hints on installing the mysql drivers for erlang. I've got the files from svn checkout http://erlang-mysql-driver.googlecode.com/svn/trunk but there are no installation instructions (I'm using OpenSolaris). TIA Paul Paul Johnston Humanities ICT (Infrastructure) Samuel Alexander Building Room W1.19 Tel Internal 63327 e-mail Paul.Johnston@REDACTED web http://servalan.humanities.manchester.ac.uk/users/mcasspj/ Tuzoqlar granatalardan yuksak darajali portlovchi moddalardan yoki bosshqa narslardan qilingan? From ft@REDACTED Tue Apr 13 13:28:09 2010 From: ft@REDACTED (Fredrik Thulin) Date: Tue, 13 Apr 2010 13:28:09 +0200 Subject: [erlang-questions] Installing mysql driver In-Reply-To: References: Message-ID: <1271158089.3550.7.camel@ft-laptop.thulin.net> On Tue, 2010-04-13 at 11:30 +0100, Paul Johnston wrote: > Hi > > Can anyone give any hints on installing the mysql drivers for erlang. > > I've got the files from svn checkout > http://erlang-mysql-driver.googlecode.com/svn/trunk but there are no > installation instructions (I'm using OpenSolaris). > > TIA Paul It isn't really a driver, it is more of a client that you can use from a program you write. Thus you can't install it separately somewhere and expect different programs to find it. Look at YXA for example (where that code comes from originally). YXA has the mysql directory as /src/mysql here http://github.com/fredrikt/yxa/tree/master/src/mysql/ and the code is called from here http://github.com/fredrikt/yxa/blob/master/src/sipuserdb/sipuserdb_mysql.erl /Fredrik From dale@REDACTED Tue Apr 13 13:49:09 2010 From: dale@REDACTED (Dale Harvey) Date: Tue, 13 Apr 2010 12:49:09 +0100 Subject: [erlang-questions] erlangotp.com In-Reply-To: <4BC4315D.2030203@erlang-solutions.com> References: <4BC4315D.2030203@erlang-solutions.com> Message-ID: Hey I hadnt quite planned on erlangotp being public yet, I figured if I only told a few people about it itd stay quiet for a while :) The wiki is very much a work in progress, I have only spent a few hours on it a few weeks ago, and its completely open to vandalism, I dont escape html and there isnt a user system. The edit button was disabled exactly because someone was deleting pages. that said everything goes through git, so everything is generally recoverable, I have readded the edit cutton, and I will find some time to add some administration features in there. if anyone wants to add ideas, that can do @ http://erlangotp.com/ideasForSite Cheers Dale On 13 April 2010 09:54, Roberto Aloi wrote: > Hi all, > > I've found this nice list of companies/projects using Erlang and I'm > wondering who is maintaining it: > > http://erlangotp.com/wiki/companiesUsingErlang > > It's a really nice idea, but a couple of entries are still missing. > > Anyone? > > Roberto Aloi > -- > University of Kent - Erlang Solutions Ltd. > Twitter: @prof3ta > Blog: http://aloiroberto.wordpress.com > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become > ERLANG SOLUTIONS LTD. > > www.erlang-solutions.com > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From thomasl_erlang@REDACTED Tue Apr 13 15:12:50 2010 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 13 Apr 2010 06:12:50 -0700 (PDT) Subject: VMWare acquires RabbitMQ Message-ID: <669546.76386.qm@web111412.mail.gq1.yahoo.com> SpringSource/VMWare/EMC just bought Rabbit Technologies Ltd, makers of RabbitMQ. Congrats everyone! http://www.springsource.com/newsevents/springsource-acquires-rabbitmq-cloud-messaging (No word on the price yet.) Best, Thomas From alex.arnon@REDACTED Tue Apr 13 15:23:47 2010 From: alex.arnon@REDACTED (Alex Arnon) Date: Tue, 13 Apr 2010 16:23:47 +0300 Subject: [erlang-questions] VMWare acquires RabbitMQ In-Reply-To: <669546.76386.qm@web111412.mail.gq1.yahoo.com> References: <669546.76386.qm@web111412.mail.gq1.yahoo.com> Message-ID: Congrats indeed!!! This has certainly made my day :) On Tue, Apr 13, 2010 at 4:12 PM, Thomas Lindgren wrote: > SpringSource/VMWare/EMC just bought Rabbit Technologies Ltd, makers of > RabbitMQ. Congrats everyone! > > > http://www.springsource.com/newsevents/springsource-acquires-rabbitmq-cloud-messaging > > (No word on the price yet.) > > Best, > Thomas > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From paul.a.johnston@REDACTED Tue Apr 13 15:12:41 2010 From: paul.a.johnston@REDACTED (Paul Johnston) Date: Tue, 13 Apr 2010 14:12:41 +0100 Subject: [erlang-questions] Installing mysql driver In-Reply-To: <1271158089.3550.7.camel@ft-laptop.thulin.net> References: <1271158089.3550.7.camel@ft-laptop.thulin.net> Message-ID: Thanks! I sorted of worked it out by creating a directory paulj@REDACTED:_snv_134:/usr/local/lib/erlang/lib/mysql$ compiled the erl files from svn into there. then: -module(application1). -export([start/0,ending/0]). start() -> hello. ending() -> mysql:start_link(p1,"127.0.0.1", "erlang_db", "password","erlang"), Result1 = mysql:fetch(p1,"SELECT id, forename FROM users"), io:format("Result1: ~p~n", [Result1]). Gives me 2> application1:ending(). mysql_conn:620: greeting version "5.1.37" (protocol 10) salt "QkG`.b$*" caps 63487 serverchar <<8,2,0,0, 0,0,0,0, 0,0,0,0, 0,0,0,0>>salt2 "GfM/?1v!>UnJ" mysql_auth:187: mysql_auth send packet 1: <<5,162,0,0,64,66,15,0,8,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 101,114,108,97,110,103,95,100,98,0, 20,67,112,50,212,246,137,195,223, 208,18,41,186,105,119,91,178,5,121, 78,126>> mysql_conn:426: fetch <<"use erlang">> (id <0.85.0>) mysql_conn:426: fetch <<"SELECT id, forename FROM users">> (id <0.85.0>) Result1: {data,{mysql_result,[{<<"users">>,<<"id">>,11,'LONG'}, {<<"users">>,<<"forename">>,10,'STRING'}], [[1,<<"Paul">>]], 0,[]}} Ok The code is dated 12 Sept 2005 and is written by your good self :-) Just to say thanks for all the work you put into something like this so I can have a play around! Paul -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Fredrik Thulin Sent: 13 April 2010 12:28 To: Johnston, Paul Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Installing mysql driver On Tue, 2010-04-13 at 11:30 +0100, Paul Johnston wrote: > Hi > > Can anyone give any hints on installing the mysql drivers for erlang. > > I've got the files from svn checkout > http://erlang-mysql-driver.googlecode.com/svn/trunk but there are no > installation instructions (I'm using OpenSolaris). > > TIA Paul It isn't really a driver, it is more of a client that you can use from a program you write. Thus you can't install it separately somewhere and expect different programs to find it. Look at YXA for example (where that code comes from originally). YXA has the mysql directory as /src/mysql here http://github.com/fredrikt/yxa/tree/master/src/mysql/ and the code is called from here http://github.com/fredrikt/yxa/blob/master/src/sipuserdb/sipuserdb_mysql .erl /Fredrik ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From csanto@REDACTED Tue Apr 13 18:24:08 2010 From: csanto@REDACTED (Corrado Santoro) Date: Tue, 13 Apr 2010 18:24:08 +0200 (CEST) Subject: [erlang-questions] Installing mysql driver In-Reply-To: References: <1271158089.3550.7.camel@ft-laptop.thulin.net> Message-ID: <45711.151.97.19.54.1271175848.squirrel@webmail.cdc.unict.it> Paul, maybe this link could help you :-) http://amnesia.sourceforge.net/ All the best, --Corrado > Thanks! > > I sorted of worked it out by creating a directory > paulj@REDACTED:_snv_134:/usr/local/lib/erlang/lib/mysql$ > compiled the erl files from svn into there. > then: > > -module(application1). > -export([start/0,ending/0]). > > > start() -> hello. > > ending() -> mysql:start_link(p1,"127.0.0.1", "erlang_db", > "password","erlang"), > Result1 = mysql:fetch(p1,"SELECT id, forename FROM users"), > io:format("Result1: ~p~n", [Result1]). > > Gives me > > 2> application1:ending(). > mysql_conn:620: greeting version "5.1.37" (protocol 10) salt "QkG`.b$*" > caps 63487 serverchar <<8,2,0,0, > > 0,0,0,0, > > 0,0,0,0, > > 0,0,0,0>>salt2 "GfM/?1v!>UnJ" > mysql_auth:187: mysql_auth send packet 1: > <<5,162,0,0,64,66,15,0,8,0,0,0,0,0,0, > > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > > 101,114,108,97,110,103,95,100,98,0, > > 20,67,112,50,212,246,137,195,223, > > 208,18,41,186,105,119,91,178,5,121, > 78,126>> > mysql_conn:426: fetch <<"use erlang">> (id <0.85.0>) > mysql_conn:426: fetch <<"SELECT id, forename FROM users">> (id > <0.85.0>) > Result1: {data,{mysql_result,[{<<"users">>,<<"id">>,11,'LONG'}, > {<<"users">>,<<"forename">>,10,'STRING'}], > [[1,<<"Paul">>]], > 0,[]}} > Ok > > The code is dated 12 Sept 2005 and is written by your good self :-) > Just to say thanks for all the work you put into something like this so > I can have a play around! > > Paul > > > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] > On Behalf Of Fredrik Thulin > Sent: 13 April 2010 12:28 > To: Johnston, Paul > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Installing mysql driver > > On Tue, 2010-04-13 at 11:30 +0100, Paul Johnston wrote: >> Hi >> >> Can anyone give any hints on installing the mysql drivers for erlang. >> >> I've got the files from svn checkout >> http://erlang-mysql-driver.googlecode.com/svn/trunk but there are no >> installation instructions (I'm using OpenSolaris). >> >> TIA Paul > > It isn't really a driver, it is more of a client that you can use from a > program you write. Thus you can't install it separately somewhere and > expect different programs to find it. > > Look at YXA for example (where that code comes from originally). YXA has > the mysql directory as /src/mysql here > > http://github.com/fredrikt/yxa/tree/master/src/mysql/ > > and the code is called from here > > http://github.com/fredrikt/yxa/blob/master/src/sipuserdb/sipuserdb_mysql > .erl > > /Fredrik > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kyleqian@REDACTED Wed Apr 14 11:34:39 2010 From: kyleqian@REDACTED (=?GB2312?B?x67P/sP3?=) Date: Wed, 14 Apr 2010 17:34:39 +0800 Subject: How to select a record with an attribute which is maximal in mnesia table? Message-ID: Hi, I want to select a record with an attribute which is maximal from mnesia table, so should I use qlc? For example, I have to insert a record into a table every 5 minutes, and when my program start up, it should read the last record to get some information. So how can I achieve this? Thanks! From bengt.kleberg@REDACTED Wed Apr 14 15:22:08 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 14 Apr 2010 15:22:08 +0200 Subject: [erlang-questions] How to select a record with an attribute which is maximal in mnesia table? In-Reply-To: References: Message-ID: <1271251328.4780.22.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, Would it be ok for you to have the date/time as record key? If you have that and an ordered_set then the last record is just that, last. bengt On Wed, 2010-04-14 at 11:34 +0200, ??? wrote: > Hi, I want to select a record with an attribute which is maximal from mnesia > table, so should I use qlc? > For example, I have to insert a record into a table every 5 minutes, and > when my program start up, it should read the last record to get some > information. > So how can I achieve this? > > Thanks! From zeno490@REDACTED Wed Apr 14 16:15:40 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Wed, 14 Apr 2010 10:15:40 -0400 Subject: [erlang-questions] How to select a record with an attribute which is maximal in mnesia table? In-Reply-To: <1271251328.4780.22.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> References: <1271251328.4780.22.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: Just keep in mind mnesia doesn't support disk copies of ordered sets so you'll have to serialize to disk manually, either on every update, periodically or on app exit (tricky in cases where your app core dumps). Alternatively, you can simply write to a flat file on the side of your mnesia table in memory (or just go with the flat file) and then on app start, open the file, seek to the end - size of your record (hopefully fixed size...) and voila. 2cents. Nicholas On Wed, Apr 14, 2010 at 9:22 AM, Bengt Kleberg wrote: > Greetings, > > Would it be ok for you to have the date/time as record key? If you have > that and an ordered_set then the last record is just that, last. > > > bengt > > On Wed, 2010-04-14 at 11:34 +0200, ??? wrote: > > Hi, I want to select a record with an attribute which is maximal from > mnesia > > table, so should I use qlc? > > For example, I have to insert a record into a table every 5 minutes, and > > when my program start up, it should read the last record to get some > > information. > > So how can I achieve this? > > > > Thanks! > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ulf.wiger@REDACTED Wed Apr 14 16:24:58 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 14 Apr 2010 16:24:58 +0200 Subject: [erlang-questions] How to select a record with an attribute which is maximal in mnesia table? In-Reply-To: References: <1271251328.4780.22.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: <4BC5D03A.6070309@erlang-solutions.com> Mnesia disc_copies tables can be ordered_set. It's only disc_only_copies that can't. Disc_copy tables are essentially just spooled to disk, so the organization in RAM (hash or btree) is just a function of how the ets table holding the data in RAM is configured. BR, Ulf W Nicholas Frechette wrote: > Just keep in mind mnesia doesn't support disk copies of ordered sets so > you'll have to serialize to disk manually, either on every update, > periodically or on app exit (tricky in cases where your app core dumps). > > Alternatively, you can simply write to a flat file on the side of your > mnesia table in memory (or just go with the flat file) and then on app > start, open the file, seek to the end - size of your record (hopefully fixed > size...) and voila. > > 2cents. > Nicholas > > On Wed, Apr 14, 2010 at 9:22 AM, Bengt Kleberg > wrote: > >> Greetings, >> >> Would it be ok for you to have the date/time as record key? If you have >> that and an ordered_set then the last record is just that, last. >> >> >> bengt >> >> On Wed, 2010-04-14 at 11:34 +0200, ??? wrote: >>> Hi, I want to select a record with an attribute which is maximal from >> mnesia >>> table, so should I use qlc? >>> For example, I have to insert a record into a table every 5 minutes, and >>> when my program start up, it should read the last record to get some >>> information. >>> So how can I achieve this? >>> >>> Thanks! >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From jderick@REDACTED Wed Apr 14 19:17:48 2010 From: jderick@REDACTED (John Erickson) Date: Wed, 14 Apr 2010 10:17:48 -0700 Subject: [erlang-questions] link In-Reply-To: References: Message-ID: Hello, in this case I don't think there is any normal termination at all, just a network delay that causes the other nodes to think this node is down. Is there any way for the other nodes to die in this case? On Mon, Apr 12, 2010 at 10:50 PM, Rapsey wrote: > If a process dies with exit signal normal (like if the process function > stops recursion) the linked processes will not die. If there is any other > exit signal, linked processes will die as well. You can call exit(stop) or > something when you want to kill all processes as well. > > > Sergej > > On Tue, Apr 13, 2010 at 12:10 AM, John Erickson wrote: > > > Hello, I have created a bunch of processes and called link() on them from > a > > root process. I thought this would mean that when one of them died, they > > would all die. However, I see in my log file one of the processes is > 'not > > responding' but the others run along oblivious. Is there a way to have > > them > > all die when this happens? > > > From comptekki@REDACTED Wed Apr 14 19:44:10 2010 From: comptekki@REDACTED (Wes James) Date: Wed, 14 Apr 2010 11:44:10 -0600 Subject: why syntax error Message-ID: I am trying to debug some yaws data. yaws uses out(A)-> ..... to send data back to the browser. I did an io:format of "A" to use it in just an .erl file. The "A" data looks like this: {arg,#Port<0.2808>,{{127,0,0,1},51598},......,<0.1112.0>,[],undefined,[47],undefined} So I have start() -> Data={arg,#Port<0.2808>,{{127,0,0,1},51598},......,<0.1112.0>,[],undefined,[47],undefined}, out(Data). but when I do c(module) I get: Syntax error before Port. If I change #Port<0.2808> to undefined then do c(module) again then I get syntax error before: '<'. So I change <0.1112.0> to undefined and now it compiles fine. Why doesn't #Port<0.2808> and <0.1112.0> pass the compile stage? thx, -wes From rvirding@REDACTED Wed Apr 14 20:27:52 2010 From: rvirding@REDACTED (Robert Virding) Date: Wed, 14 Apr 2010 20:27:52 +0200 Subject: [erlang-questions] why syntax error In-Reply-To: References: Message-ID: There is no no printed representation of pids (process ids) or ports which can be read in to return the pid or port. So #Port<0.2808> and <0.1112.0> is not valid syntax. This is not accidental but deliberate as creating ports/pids in this fashion is very unsafe and can really screw things up. Robert On 14 April 2010 19:44, Wes James wrote: > I am trying to debug some yaws data. ?yaws uses out(A)-> ..... to send > data back to the browser. ?I did an io:format of "A" to use it in just > an .erl file. ?The "A" data looks like this: > > ? ? ? ?{arg,#Port<0.2808>,{{127,0,0,1},51598},......,<0.1112.0>,[],undefined,[47],undefined} > > So I have > > start() -> > ? ? ? ?Data={arg,#Port<0.2808>,{{127,0,0,1},51598},......,<0.1112.0>,[],undefined,[47],undefined}, > ? ? ? ?out(Data). > > > but when I do c(module) I get: Syntax error before Port. ?If I change > #Port<0.2808> to undefined then do c(module) again then I get syntax > error before: '<'. ?So I change <0.1112.0> to undefined and now it > compiles fine. ?Why doesn't #Port<0.2808> and <0.1112.0> pass the > compile stage? > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From nem@REDACTED Thu Apr 15 00:48:36 2010 From: nem@REDACTED (Geoff Cant) Date: Wed, 14 Apr 2010 15:48:36 -0700 Subject: [erlang-questions] why syntax error In-Reply-To: (Robert Virding's message of "Wed, 14 Apr 2010 20:27:52 +0200") References: Message-ID: Robert Virding writes: > There is no no printed representation of pids (process ids) or ports > which can be read in to return the pid or port. So #Port<0.2808> and > <0.1112.0> is not valid syntax. > > This is not accidental but deliberate as creating ports/pids in this > fashion is very unsafe and can really screw things up. > > Robert I'd agree it's not something you should want often. That said, I wouldn't mind a '[{debugging, production}, {please, help_help_help}]' option for the shell that would add reader syntax (only in the shell) for #Port and so that I could copy and paste pids/ports from log messages and debugging output right back into new commands. Would save me typing pid(X,Y,Z) and/or list_to_pid("") quite so often. The standard OTP disclaimer[1] would of course would apply. -- Geoff Cant [1] I, undersigned, do hereby acknowledge that I know that what I'm about to do is generally a bad idea. The OTP team documented that it was a bad idea, explained why it was a bad idea, generally tried to encourage me not to do it and yet I am going to do it anyway. When my hubris leads me to trash the production cluster, not only do I disclaim any right to get mad, complain on the mailing list or twitter, I will in fact praise the OTP team for their foresight and volunteer to write an even sterner warning in the documentation for future reference. Should I break this agreement by uttering so much as a groan when I use this undocumented/deprecated/debugging-only function/feature shoot myself in the foot, I promise to buy the OTP team a round at the next Erlounge. From stondage123@REDACTED Thu Apr 15 01:19:41 2010 From: stondage123@REDACTED (Andrew Stone) Date: Wed, 14 Apr 2010 16:19:41 -0700 (PDT) Subject: [erlang-questions] why syntax error In-Reply-To: References: Message-ID: <544158.59575.qm@web35902.mail.mud.yahoo.com> I would also appreciate having some type of access to writeable pids this in the shell , so I don't have to do the list_to_pid/1 dance so often. However, I fully understand the rationale behind the original decision to make those values read-only. BTW, Geoff, your disclaimer reminds me of this. http://github.com/SFEley/candy/blob/master/LICENSE.markdown -Andrew ----- Original Message ---- From: Geoff Cant To: Robert Virding Cc: erlang-questions@REDACTED Sent: Wed, April 14, 2010 6:48:36 PM Subject: Re: [erlang-questions] why syntax error Robert Virding writes: > There is no no printed representation of pids (process ids) or ports > which can be read in to return the pid or port. So #Port<0.2808> and > <0.1112.0> is not valid syntax. > > This is not accidental but deliberate as creating ports/pids in this > fashion is very unsafe and can really screw things up. > > Robert I'd agree it's not something you should want often. That said, I wouldn't mind a '[{debugging, production}, {please, help_help_help}]' option for the shell that would add reader syntax (only in the shell) for #Port and so that I could copy and paste pids/ports from log messages and debugging output right back into new commands. Would save me typing pid(X,Y,Z) and/or list_to_pid("") quite so often. The standard OTP disclaimer[1] would of course would apply. -- Geoff Cant [1] I, undersigned, do hereby acknowledge that I know that what I'm about to do is generally a bad idea. The OTP team documented that it was a bad idea, explained why it was a bad idea, generally tried to encourage me not to do it and yet I am going to do it anyway. When my hubris leads me to trash the production cluster, not only do I disclaim any right to get mad, complain on the mailing list or twitter, I will in fact praise the OTP team for their foresight and volunteer to write an even sterner warning in the documentation for future reference. Should I break this agreement by uttering so much as a groan when I use this undocumented/deprecated/debugging-only function/feature shoot myself in the foot, I promise to buy the OTP team a round at the next Erlounge. ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From koops.j@REDACTED Thu Apr 15 09:05:55 2010 From: koops.j@REDACTED (Jeroen Koops) Date: Thu, 15 Apr 2010 09:05:55 +0200 Subject: [erlang-questions] why syntax error In-Reply-To: <544158.59575.qm@web35902.mail.mud.yahoo.com> References: <544158.59575.qm@web35902.mail.mud.yahoo.com> Message-ID: As an alternative to list_to_pid/1, you can use pid/3: 1> pid(0,12,0). <0.12.0> I think it's part of the c module. On Thu, Apr 15, 2010 at 1:19 AM, Andrew Stone wrote: > I would also appreciate having some type of access to writeable pids this > in the shell , so I don't have to do the list_to_pid/1 dance so often. > However, I fully understand the rationale behind the original decision to > make those values read-only. > > BTW, Geoff, your disclaimer reminds me of this. > > http://github.com/SFEley/candy/blob/master/LICENSE.markdown > > -Andrew > > > > ----- Original Message ---- > From: Geoff Cant > To: Robert Virding > Cc: erlang-questions@REDACTED > Sent: Wed, April 14, 2010 6:48:36 PM > Subject: Re: [erlang-questions] why syntax error > > > > Robert Virding writes: > > > There is no no printed representation of pids (process ids) or ports > > which can be read in to return the pid or port. So #Port<0.2808> and > > <0.1112.0> is not valid syntax. > > > > This is not accidental but deliberate as creating ports/pids in this > > fashion is very unsafe and can really screw things up. > > > > Robert > > I'd agree it's not something you should want often. > > That said, I wouldn't mind a '[{debugging, production}, {please, > help_help_help}]' option for the shell that would add reader syntax > (only in the shell) for #Port and so that I could copy > and paste pids/ports from log messages and debugging output right back > into new commands. Would save me typing pid(X,Y,Z) and/or > list_to_pid("") quite so often. > > The standard OTP disclaimer[1] would of course would apply. > > -- > Geoff Cant > > [1] I, undersigned, do hereby acknowledge that I > know that what I'm about to do is generally a bad idea. The OTP team > documented that it was a bad idea, explained why it was a bad idea, > generally tried to encourage me not to do it and yet I am going to > do it anyway. > > When my hubris leads me to trash the production cluster, not only do > I disclaim any right to get mad, complain on the mailing list or > twitter, I will in fact praise the OTP team for their foresight and > volunteer to write an even sterner warning in the documentation for > future reference. > > Should I break this agreement by uttering so much as a groan when I > use this undocumented/deprecated/debugging-only function/feature > shoot myself in the foot, I promise to buy the OTP team a round at > the next Erlounge. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From dmercer@REDACTED Thu Apr 15 15:12:08 2010 From: dmercer@REDACTED (David Mercer) Date: Thu, 15 Apr 2010 08:12:08 -0500 Subject: [erlang-questions] why syntax error In-Reply-To: References: Message-ID: <585EC25D5A4247499BB7D887ED375B2F@SSI.CORP> On Wednesday, April 14, 2010, Geoff Cant wrote: > That said, I wouldn't mind a '[{debugging, production}, {please, > help_help_help}]' option for the shell that would add reader syntax > (only in the shell) for #Port and so that I could copy > and paste pids/ports from log messages and debugging output right back > into new commands. Would save me typing pid(X,Y,Z) and/or > list_to_pid("") quite so often. Is it possible to just change the output format of PID's from "" to "pid(X,Y,Z)"? From als@REDACTED Thu Apr 15 16:21:01 2010 From: als@REDACTED (Anthony Shipman) Date: Fri, 16 Apr 2010 01:21:01 +1100 Subject: floating point syntax Message-ID: <201004160021.01795.als@iinet.net.au> The reference manual says "There are two types of numeric literals, integers and floats. Besides the conventional notation,..." But what is the "conventional notation" exactly? I find that this fails: 19> io_lib:fread("~f", "1e+2"). {error,{fread,float}} Also 1.e+2 fails 1 fails 1. fails 1.0 succeeds .1 fails 0.1 succeeds 1.0e+2 succeeds Is it supposed to be like that or is there a bug? -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From daniel.goertzen@REDACTED Thu Apr 15 17:21:50 2010 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Thu, 15 Apr 2010 10:21:50 -0500 Subject: windows network scalability? Message-ID: I am designing a Windows network application and am having a hard time getting a sense of erlang network performance on Windows. My app will establish up to 1000 outgoing SSH connections carrying low volume interactive traffic. It is a GUI desktop app so when launched, it should establish all the connections as quickly as possible. My questions are: - Does Erlang on Windows use select()? I see dire warning about how this is not scalable, but is it "good enough" for 1000 sockets on a modern PC? - Does Erlang (or Windows) serialize DNS requests? (I was burned on this years ago with python/cygwin) - Any other performance hazards I should worry about? Thanks, Dan. From g9414002.pccu.edu.tw@REDACTED Thu Apr 15 18:07:25 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Fri, 16 Apr 2010 00:07:25 +0800 Subject: [erlang-questions] floating point syntax In-Reply-To: <201004160021.01795.als@iinet.net.au> References: <201004160021.01795.als@iinet.net.au> Message-ID: On Thu, Apr 15, 2010 at 10:21 PM, Anthony Shipman wrote: > The reference manual says "There are two types of numeric literals, > integers > and floats. Besides the conventional notation,..." > > But what is the "conventional notation" exactly? I find that this fails: > 19> io_lib:fread("~f", "1e+2"). > {error,{fread,float}} > > Also > 1.e+2 fails > 1 fails > 1. fails > 1.0 succeeds > .1 fails > 0.1 succeeds > 1.0e+2 succeeds > > Is it supposed to be like that or is there a bug? In human sense, do you think 1, (1.), and (.1) floating point numbers? Keep things from C or Java out, and you will be satisfied. From rvirding@REDACTED Thu Apr 15 18:30:45 2010 From: rvirding@REDACTED (Robert Virding) Date: Thu, 15 Apr 2010 18:30:45 +0200 Subject: [erlang-questions] floating point syntax In-Reply-To: <201004160021.01795.als@iinet.net.au> References: <201004160021.01795.als@iinet.net.au> Message-ID: Yes, it is supposed to be like that, no, it is not a bug. Floats consist of both parts before and after the decimal point and an optional exponent. Robert On 15 April 2010 16:21, Anthony Shipman wrote: > The reference manual says "There are two types of numeric literals, integers > and floats. Besides the conventional notation,..." > > But what is the "conventional notation" exactly? I find that this fails: > 19> io_lib:fread("~f", "1e+2"). > {error,{fread,float}} > > Also > 1.e+2 fails > 1 ? ? fails > 1. ? ?fails > 1.0 ? succeeds > .1 ? ?fails > 0.1 ? succeeds > 1.0e+2 succeeds > > Is it supposed to be like that or is there a bug? > > -- > Anthony Shipman ? ? ? ? ? ? ? ? ? ?Mamas don't let your babies > als@REDACTED ? ? ? ? ? ? ? ? ? grow up to be outsourced. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From comptekki@REDACTED Thu Apr 15 23:10:18 2010 From: comptekki@REDACTED (Wes James) Date: Thu, 15 Apr 2010 15:10:18 -0600 Subject: command-line args in erl Message-ID: I am running through the mandelbrot example from: http://projects.trapexit.org/web/#/web/project/github::ghulette::mandelbrot-erlang and it has: main([WArg,HArg]) ->..... so you can provide command-line args like main 200 200, but how do you do provide command line args in erl? render:main([200,200]). spits out: ** exception error: bad argument in function atom_to_list/1 called as atom_to_list(200) in call from render:main/1 what should main(??) look like? thx, -wes From comptekki@REDACTED Thu Apr 15 23:25:05 2010 From: comptekki@REDACTED (Wes James) Date: Thu, 15 Apr 2010 15:25:05 -0600 Subject: command-line args in erl In-Reply-To: References: Message-ID: On Thu, Apr 15, 2010 at 3:10 PM, Wes James wrote: > I am running through the mandelbrot example from: > > http://projects.trapexit.org/web/#/web/project/github::ghulette::mandelbrot-erlang > > and it has: > > main([WArg,HArg]) ->..... > > so you can provide command-line args like main 200 200, but how do you > do provide command line args in erl? > I changed main to main(WArg,HArg) -> and this code: main(WArg,HArg) -> W = WArg, %list_to_integer(atom_to_list(WArg)), H = HArg, %list_to_integer(atom_to_list(HArg)), MaxI = 255, render(W,H,MaxI), init:stop(). to just have single value and it works for testing. I'm still wondering how to pass args in erl though - or is the [var1, var2] always interpreted as command line args are expected? thx, -wes From dmercer@REDACTED Thu Apr 15 23:28:32 2010 From: dmercer@REDACTED (David Mercer) Date: Thu, 15 Apr 2010 16:28:32 -0500 Subject: [erlang-questions] command-line args in erl In-Reply-To: References: Message-ID: <2A741BFB9B7A4A7E822E9747313CE679@SSI.CORP> On Thursday, April 15, 2010, Wes James wrote: > render:main([200,200]). > > spits out: > > ** exception error: bad argument > in function atom_to_list/1 > called as atom_to_list(200) > in call from render:main/1 > > what should main(??) look like? Looking at the Makefile, it looks like it is designed to be run from the command line, where the arguments are passed in as atoms. Try: render:main(['200', '200']). Cheers, David From diginux@REDACTED Thu Apr 15 23:28:57 2010 From: diginux@REDACTED (Jordan Wilberding) Date: Thu, 15 Apr 2010 16:28:57 -0500 Subject: [erlang-questions] Re: command-line args in erl In-Reply-To: References: Message-ID: erl -noshell -s render main 200 200 > Mandelbrot.ppm (using the original code) Thanks! JW On Thu, Apr 15, 2010 at 4:25 PM, Wes James wrote: > On Thu, Apr 15, 2010 at 3:10 PM, Wes James wrote: > > I am running through the mandelbrot example from: > > > > > http://projects.trapexit.org/web/#/web/project/github::ghulette::mandelbrot-erlang > > > > and it has: > > > > main([WArg,HArg]) ->..... > > > > so you can provide command-line args like main 200 200, but how do you > > do provide command line args in erl? > > > > > > I changed main to main(WArg,HArg) -> > > and this code: > > main(WArg,HArg) -> > W = WArg, %list_to_integer(atom_to_list(WArg)), > H = HArg, %list_to_integer(atom_to_list(HArg)), > MaxI = 255, > render(W,H,MaxI), > init:stop(). > > > to just have single value and it works for testing. I'm still > wondering how to pass args in erl though - or is the [var1, var2] > always interpreted as command line args are expected? > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From diginux@REDACTED Thu Apr 15 23:31:24 2010 From: diginux@REDACTED (Jordan Wilberding) Date: Thu, 15 Apr 2010 16:31:24 -0500 Subject: [erlang-questions] command-line args in erl In-Reply-To: References: Message-ID: render:main(['200','200']). On Thu, Apr 15, 2010 at 4:10 PM, Wes James wrote: > I am running through the mandelbrot example from: > > > http://projects.trapexit.org/web/#/web/project/github::ghulette::mandelbrot-erlang > > and it has: > > main([WArg,HArg]) ->..... > > so you can provide command-line args like main 200 200, but how do you > do provide command line args in erl? > > render:main([200,200]). > > spits out: > > ** exception error: bad argument > in function atom_to_list/1 > called as atom_to_list(200) > in call from render:main/1 > > what should main(??) look like? > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From comptekki@REDACTED Thu Apr 15 23:34:53 2010 From: comptekki@REDACTED (Wes James) Date: Thu, 15 Apr 2010 15:34:53 -0600 Subject: [erlang-questions] command-line args in erl In-Reply-To: References: Message-ID: On Thu, Apr 15, 2010 at 3:31 PM, Jordan Wilberding wrote: > render:main(['200','200']). > > On Thu, Apr 15, 2010 at 4:10 PM, Wes James wrote: >> >> I am running through the mandelbrot example from: >> >> >> http://projects.trapexit.org/web/#/web/project/github::ghulette::mandelbrot-erlang >> >> and it has: >> >> main([WArg,HArg]) ->..... thx - that did it. I guess an atom is '200' but not 200 in this case. -wes From comptekki@REDACTED Thu Apr 15 23:36:26 2010 From: comptekki@REDACTED (Wes James) Date: Thu, 15 Apr 2010 15:36:26 -0600 Subject: [erlang-questions] Re: command-line args in erl In-Reply-To: References: Message-ID: On Thu, Apr 15, 2010 at 3:28 PM, Jordan Wilberding wrote: > erl -noshell -s render main 200 200 > Mandelbrot.ppm > > (using the original code) > > Thanks! > JW > I am running the code through a debugger to see what is happening so I'm trying to run this inside erl. thx, -wes From comptekki@REDACTED Thu Apr 15 23:41:36 2010 From: comptekki@REDACTED (Wes James) Date: Thu, 15 Apr 2010 15:41:36 -0600 Subject: command-line args in erl In-Reply-To: References: Message-ID: On Thu, Apr 15, 2010 at 3:25 PM, Wes James wrote: > > and this code: > > main(WArg,HArg) -> > ?W = WArg, %list_to_integer(atom_to_list(WArg)), > ?H = HArg, %list_to_integer(atom_to_list(HArg)), hmm, interesting. erl.... A='200' '200' B=atom_to_list(A). "200" C=list_to_integer(B). 100 atom is in single quote and list is in double quote?? -wes From diginux@REDACTED Fri Apr 16 02:34:59 2010 From: diginux@REDACTED (Jordan Wilberding) Date: Thu, 15 Apr 2010 19:34:59 -0500 Subject: [erlang-questions] Re: command-line args in erl In-Reply-To: References: Message-ID: Yep! 'anything in single quotes even with spaces is an atom' "in double quotes it is a string" For more info: http://www.erlang.org/doc/reference_manual/data_types.html JW On Thu, Apr 15, 2010 at 4:41 PM, Wes James wrote: > On Thu, Apr 15, 2010 at 3:25 PM, Wes James wrote: > > > > > > > and this code: > > > > main(WArg,HArg) -> > > W = WArg, %list_to_integer(atom_to_list(WArg)), > > H = HArg, %list_to_integer(atom_to_list(HArg)), > > hmm, interesting. > > erl.... > > A='200' > '200' > B=atom_to_list(A). > "200" > C=list_to_integer(B). > 100 > > atom is in single quote and list is in double quote?? > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jacob.vorreuter@REDACTED Fri Apr 16 02:56:06 2010 From: jacob.vorreuter@REDACTED (Jacob Vorreuter) Date: Thu, 15 Apr 2010 17:56:06 -0700 Subject: gen_tcp:connect taking 3 seconds under load Message-ID: Hi, I'm load testing an Erlang proxy server that accepts incoming client connections and proxies data to backend server sockets. Normally, the call to gen_tcp:connect to open the backend socket takes under 3 milliseconds. However, under heavy load this call sometimes takes 3 seconds. The strange thing is that it rarely takes any amount of time in the middle of that range. It's either 3 milliseconds or 3000. This is the call I'm making: gen_tcp:connect(Ip, Port, [binary, {packet, 0}]). Has anyone seen this behavior before? I'm load testing with ab, making anywhere from 64 to 1000 concurrent requests. Thanks, Jake From bob@REDACTED Fri Apr 16 03:01:09 2010 From: bob@REDACTED (Bob Ippolito) Date: Thu, 15 Apr 2010 18:01:09 -0700 Subject: [erlang-questions] gen_tcp:connect taking 3 seconds under load In-Reply-To: References: Message-ID: Are you testing on Mac OS X? I think this is a bug in the Mac OS X kernel. I've seen behavior like this before only on Mac OS X and it's reproducible with other environments, not just Erlang. On Thu, Apr 15, 2010 at 5:56 PM, Jacob Vorreuter wrote: > Hi, > > I'm load testing an Erlang proxy server that accepts incoming client connections and proxies data to backend server sockets. ?Normally, the call to gen_tcp:connect to open the backend socket takes under 3 milliseconds. ?However, under heavy load this call sometimes takes 3 seconds. ?The strange thing is that it rarely takes any amount of time in the middle of that range. ?It's either 3 milliseconds or 3000. ?This is the call I'm making: > > gen_tcp:connect(Ip, Port, [binary, {packet, 0}]). > > Has anyone seen this behavior before? ?I'm load testing with ab, making anywhere from 64 to 1000 concurrent requests. > > Thanks, > > Jake > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From adam.kocoloski@REDACTED Fri Apr 16 03:09:13 2010 From: adam.kocoloski@REDACTED (Adam Kocoloski) Date: Thu, 15 Apr 2010 21:09:13 -0400 Subject: [erlang-questions] gen_tcp:connect taking 3 seconds under load In-Reply-To: References: Message-ID: On Apr 15, 2010, at 8:56 PM, Jacob Vorreuter wrote: > Hi, > > I'm load testing an Erlang proxy server that accepts incoming client connections and proxies data to backend server sockets. Normally, the call to gen_tcp:connect to open the backend socket takes under 3 milliseconds. However, under heavy load this call sometimes takes 3 seconds. The strange thing is that it rarely takes any amount of time in the middle of that range. It's either 3 milliseconds or 3000. This is the call I'm making: > > gen_tcp:connect(Ip, Port, [binary, {packet, 0}]). > > Has anyone seen this behavior before? I'm load testing with ab, making anywhere from 64 to 1000 concurrent requests. > > Thanks, > > Jake Hi Jake, 3 seconds is a magic number in TCP -- it's the TCP retransmission timeout. You might double check for an abnormally high number of lost packets, but the last time I had this happen I was able to solve the problem by increasing the listen backlog on the backend. If you're using gen_tcp:listen on the backend you can supply the {backlog, integer()} option, but be sure to check that your backend's SOMAXCONN setting will allow the value you supply. Best, Adam From navaneethanit@REDACTED Fri Apr 16 09:28:19 2010 From: navaneethanit@REDACTED (Saravana) Date: Fri, 16 Apr 2010 00:28:19 -0700 (PDT) Subject: nitrogen unable to start the server Message-ID: I created a web application in erlang using nitrogen,It was wored fine yesterday just i opened the application but the server doesn't work i don't know why??? what is the problem??please help me In terminal displayed nava@REDACTED:~$ cd navapro/navatest/ nava@REDACTED:~/navapro/navatest$ ./start.sh Starting Nitrogen. {error_logger,{{2010,4,16},{12,27,12}},"Cookie file /home/ nava/.erlang.cookie must be accessible by owner only",[]} {error_logger,{{2010,4,16},{12,27,12}},crash_report,[[{pid,<0.20.0>}, {registered_name,auth},{error_info,{exit,{"Cookie file /home/ nava/.erlang.cookie must be accessible by owner only", [{auth,init_cookie,0},{auth,init,1},{gen_server,init_it,6}, {proc_lib,init_p_do_apply,3}]},[{gen_server,init_it,6}, {proc_lib,init_p_do_apply,3}]}},{initial_call,{auth,init, ['Argument__1']}},{ancestors,[net_sup,kernel_sup,<0.8.0>]},{messages, []},{links,[<0.18.0>]},{dictionary,[]},{trap_exit,true}, {status,running},{heap_size,987},{stack_size,23},{reductions,442}], []]} {error_logger,{{2010,4,16},{12,27,12}},supervisor_report,[{supervisor, {local,net_sup}},{errorContext,start_error},{reason,{"Cookie file / home/nava/.erlang.cookie must be accessible by owner only", [{auth,init_cookie,0},{auth,init,1},{gen_server,init_it,6}, {proc_lib,init_p_do_apply,3}]}},{offender,[{pid,undefined},{name,auth}, {mfa,{auth,start_link,[]}},{restart_type,permanent},{shutdown,2000}, {child_type,worker}]}]} {error_logger,{{2010,4,16},{12,27,12}},supervisor_report,[{supervisor, {local,kernel_sup}},{errorContext,start_error},{reason,shutdown}, {offender,[{pid,undefined},{name,net_sup},{mfa, {erl_distribution,start_link,[]}},{restart_type,permanent}, {shutdown,infinity},{child_type,supervisor}]}]} {error_logger,{{2010,4,16},{12,27,12}},crash_report,[[{pid,<0.7.0>}, {registered_name,[]},{error_info,{exit,{shutdown,{kernel,start,[normal, []]}},[{application_master,init,4},{proc_lib,init_p_do_apply,3}]}}, {initial_call,{application_master,init, ['Argument__1','Argument__2','Argument__3','Argument__4']}},{ancestors, [<0.6.0>]},{messages,[{'EXIT',<0.8.0>,normal}]},{links, [<0.6.0>,<0.5.0>]},{dictionary,[]},{trap_exit,true},{status,running}, {heap_size,233},{stack_size,23},{reductions,123}],[]]} {error_logger,{{2010,4,16},{12,27,12}},std_info,[{application,kernel}, {exited,{shutdown,{kernel,start,[normal,[]]}}},{type,permanent}]} {"Kernel pid terminated",application_controller,"{application_start_failure,kernel, {shutdown,{kernel,start,[normal,[]]}}}"} Crash dump was written to: erl_crash.dump Kernel pid terminated (application_controller) ({application_start_failure,kernel,{shutdown,{kernel,start,[normal, []]}}}) ./start.sh: 9: -eval: not found ./start.sh: 10: -eval: not found ./start.sh: 11: -eval: not found nava@REDACTED:~/navapro/navatest$ From rumata-estor@REDACTED Fri Apr 16 09:37:40 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Fri, 16 Apr 2010 11:37:40 +0400 Subject: [erlang-questions] nitrogen unable to start the server In-Reply-To: References: Message-ID: <4BC813C4.3030103@nm.ru> Just read the error message you've got: "Cookie file /home/nava/.erlang.cookie must be accessible by owner only". use "chmod go-rwx /home/nava/.erlang.cookie" Saravana wrote: > I created a web application in erlang using nitrogen,It was wored fine > > yesterday just i opened the application but the server doesn't work > > i don't know why??? > > what is the problem??please help me > > In terminal displayed > > nava@REDACTED:~$ cd navapro/navatest/ > nava@REDACTED:~/navapro/navatest$ ./start.sh > Starting Nitrogen. > {error_logger,{{2010,4,16},{12,27,12}},"Cookie file /home/ > nava/.erlang.cookie must be accessible by owner only",[]} > {error_logger,{{2010,4,16},{12,27,12}},crash_report,[[{pid,<0.20.0>}, > {registered_name,auth},{error_info,{exit,{"Cookie file /home/ > nava/.erlang.cookie must be accessible by owner only", > [{auth,init_cookie,0},{auth,init,1},{gen_server,init_it,6}, > {proc_lib,init_p_do_apply,3}]},[{gen_server,init_it,6}, > {proc_lib,init_p_do_apply,3}]}},{initial_call,{auth,init, > ['Argument__1']}},{ancestors,[net_sup,kernel_sup,<0.8.0>]},{messages, > []},{links,[<0.18.0>]},{dictionary,[]},{trap_exit,true}, > {status,running},{heap_size,987},{stack_size,23},{reductions,442}], > []]} > {error_logger,{{2010,4,16},{12,27,12}},supervisor_report,[{supervisor, > {local,net_sup}},{errorContext,start_error},{reason,{"Cookie file / > home/nava/.erlang.cookie must be accessible by owner only", > [{auth,init_cookie,0},{auth,init,1},{gen_server,init_it,6}, > {proc_lib,init_p_do_apply,3}]}},{offender,[{pid,undefined},{name,auth}, > {mfa,{auth,start_link,[]}},{restart_type,permanent},{shutdown,2000}, > {child_type,worker}]}]} > {error_logger,{{2010,4,16},{12,27,12}},supervisor_report,[{supervisor, > {local,kernel_sup}},{errorContext,start_error},{reason,shutdown}, > {offender,[{pid,undefined},{name,net_sup},{mfa, > {erl_distribution,start_link,[]}},{restart_type,permanent}, > {shutdown,infinity},{child_type,supervisor}]}]} > {error_logger,{{2010,4,16},{12,27,12}},crash_report,[[{pid,<0.7.0>}, > {registered_name,[]},{error_info,{exit,{shutdown,{kernel,start,[normal, > []]}},[{application_master,init,4},{proc_lib,init_p_do_apply,3}]}}, > {initial_call,{application_master,init, > ['Argument__1','Argument__2','Argument__3','Argument__4']}},{ancestors, > [<0.6.0>]},{messages,[{'EXIT',<0.8.0>,normal}]},{links, > [<0.6.0>,<0.5.0>]},{dictionary,[]},{trap_exit,true},{status,running}, > {heap_size,233},{stack_size,23},{reductions,123}],[]]} > {error_logger,{{2010,4,16},{12,27,12}},std_info,[{application,kernel}, > {exited,{shutdown,{kernel,start,[normal,[]]}}},{type,permanent}]} > {"Kernel pid > terminated",application_controller,"{application_start_failure,kernel, > {shutdown,{kernel,start,[normal,[]]}}}"} > > Crash dump was written to: erl_crash.dump > Kernel pid terminated (application_controller) > ({application_start_failure,kernel,{shutdown,{kernel,start,[normal, > []]}}}) > ./start.sh: 9: -eval: not found > ./start.sh: 10: -eval: not found > ./start.sh: 11: -eval: not found > nava@REDACTED:~/navapro/navatest$ > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From navaneethanit@REDACTED Fri Apr 16 12:23:40 2010 From: navaneethanit@REDACTED (Saravana) Date: Fri, 16 Apr 2010 03:23:40 -0700 (PDT) Subject: nitrogen unable to start the server In-Reply-To: <4BC813C4.3030103@nm.ru> References: <4BC813C4.3030103@nm.ru> Message-ID: <90c1d494-2956-4ac0-9c3c-486fd3e23f90@v14g2000yqb.googlegroups.com> ya i gave chmod 600 /home/nava/.erlang.cookie,, then it worked,thanks a lot friend On Apr 16, 12:37?pm, Dmitry Belyaev wrote: > Just read the error message you've got: "Cookie file > /home/nava/.erlang.cookie must be accessible by owner only". > use "chmod go-rwx /home/nava/.erlang.cookie" > > > > Saravana wrote: > > I created a web application in erlang using nitrogen,It was wored fine > > > yesterday just i opened the application but the server doesn't work > > > i don't know why??? > > > what is the problem??please help me > > > In terminal displayed > > > nava@REDACTED:~$ cd navapro/navatest/ > > nava@REDACTED:~/navapro/navatest$ ./start.sh > > Starting Nitrogen. > > {error_logger,{{2010,4,16},{12,27,12}},"Cookie file /home/ > > nava/.erlang.cookie must be accessible by owner only",[]} > > {error_logger,{{2010,4,16},{12,27,12}},crash_report,[[{pid,<0.20.0>}, > > {registered_name,auth},{error_info,{exit,{"Cookie file /home/ > > nava/.erlang.cookie must be accessible by owner only", > > [{auth,init_cookie,0},{auth,init,1},{gen_server,init_it,6}, > > {proc_lib,init_p_do_apply,3}]},[{gen_server,init_it,6}, > > {proc_lib,init_p_do_apply,3}]}},{initial_call,{auth,init, > > ['Argument__1']}},{ancestors,[net_sup,kernel_sup,<0.8.0>]},{messages, > > []},{links,[<0.18.0>]},{dictionary,[]},{trap_exit,true}, > > {status,running},{heap_size,987},{stack_size,23},{reductions,442}], > > []]} > > {error_logger,{{2010,4,16},{12,27,12}},supervisor_report,[{supervisor, > > {local,net_sup}},{errorContext,start_error},{reason,{"Cookie file / > > home/nava/.erlang.cookie must be accessible by owner only", > > [{auth,init_cookie,0},{auth,init,1},{gen_server,init_it,6}, > > {proc_lib,init_p_do_apply,3}]}},{offender,[{pid,undefined},{name,auth}, > > {mfa,{auth,start_link,[]}},{restart_type,permanent},{shutdown,2000}, > > {child_type,worker}]}]} > > {error_logger,{{2010,4,16},{12,27,12}},supervisor_report,[{supervisor, > > {local,kernel_sup}},{errorContext,start_error},{reason,shutdown}, > > {offender,[{pid,undefined},{name,net_sup},{mfa, > > {erl_distribution,start_link,[]}},{restart_type,permanent}, > > {shutdown,infinity},{child_type,supervisor}]}]} > > {error_logger,{{2010,4,16},{12,27,12}},crash_report,[[{pid,<0.7.0>}, > > {registered_name,[]},{error_info,{exit,{shutdown,{kernel,start,[normal, > > []]}},[{application_master,init,4},{proc_lib,init_p_do_apply,3}]}}, > > {initial_call,{application_master,init, > > ['Argument__1','Argument__2','Argument__3','Argument__4']}},{ancestors, > > [<0.6.0>]},{messages,[{'EXIT',<0.8.0>,normal}]},{links, > > [<0.6.0>,<0.5.0>]},{dictionary,[]},{trap_exit,true},{status,running}, > > {heap_size,233},{stack_size,23},{reductions,123}],[]]} > > {error_logger,{{2010,4,16},{12,27,12}},std_info,[{application,kernel}, > > {exited,{shutdown,{kernel,start,[normal,[]]}}},{type,permanent}]} > > {"Kernel pid > > terminated",application_controller,"{application_start_failure,kernel, > > {shutdown,{kernel,start,[normal,[]]}}}"} > > > Crash dump was written to: erl_crash.dump > > Kernel pid terminated (application_controller) > > ({application_start_failure,kernel,{shutdown,{kernel,start,[normal, > > []]}}}) > > ./start.sh: 9: -eval: not found > > ./start.sh: 10: -eval: not found > > ./start.sh: 11: -eval: not found > > nava@REDACTED:~/navapro/navatest$ > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > Seehttp://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > -- > You received this message because you are subscribed to the Google Groups "Erlang Programming" group. > To post to this group, send email to erlang-programming@REDACTED > To unsubscribe from this group, send email to erlang-programming+unsubscribe@REDACTED > For more options, visit this group athttp://groups.google.com/group/erlang-programming?hl=en. From pablo.platt@REDACTED Fri Apr 16 14:06:47 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Fri, 16 Apr 2010 05:06:47 -0700 (PDT) Subject: atomic read+delete operation on ETS record Message-ID: <892067.26404.qm@web112603.mail.gq1.yahoo.com> ETS has atomic operations like update_element and update_counter. Is there a way to read a record and delete it in the same operation without using a gen_server that manage the table? Is there a way to update a list inside a record in an atomic way? For example, mongodb and redis let you add and remove items from the beginning and end of lists. ETS records are Erlang terms so it might be possible and will be very useful. Thanks From jeraymond@REDACTED Fri Apr 16 15:31:55 2010 From: jeraymond@REDACTED (Jeremy Raymond) Date: Fri, 16 Apr 2010 06:31:55 -0700 (PDT) Subject: code:clash/0 borking when running eunit tests with rebar Message-ID: <2f3fe3c3-b64b-4e45-a9d3-8fec31c01075@x12g2000yqx.googlegroups.com> Hi, I have a project with some eunit tests which run fine if I run them through the erl console. I'm now attempting to run them with rebar. One of my dependency applications (couchbeam) does a call to code:clash/0 which is dying with the following error message when I run $ rebar eunit =INFO REPORT==== 16-Apr-2010::09:24:16 === application: couchbeam exited: {bad_return, {{couchbeam,start,[normal,[]]}, {'EXIT', {function_clause, [{code,filter,[".beam",".eunit",error]}, {code,build,1}, {code,clash,0}, {couchbeam_deps,ensure,1}, {couchbeam,start,2}, {application_master,start_it_old,4}]}}}} I can reproduce this without couchbeam as well by just calling code:clash() directly in my eunit test: *** context setup failed *** ::error:function_clause in function code:filter/3 called as filter(".beam",".eunit",error) in call from code:build/1 in call from code:clash/0 in call from cdb_data_tests:setup/0 Any ideas as to what may be going on here (or how I might debug this further)? Thanks, Jeremy From damu.devnull@REDACTED Fri Apr 16 15:32:26 2010 From: damu.devnull@REDACTED (Damu R) Date: Fri, 16 Apr 2010 19:02:26 +0530 Subject: Read from input without echo Message-ID: Hi, I am trying to read password from the terminal input.I am unable to find a way to read input from the terminal without echoing. I am able to do it from the erlang shell, but it is not happening in a standalone script (escript). Looks like the erlang shell uses a different group leader process for i/o. Is there a way to read password from a escript? Thank you Damu From vladdu55@REDACTED Fri Apr 16 15:42:35 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 16 Apr 2010 15:42:35 +0200 Subject: [erlang-questions] code:clash/0 borking when running eunit tests with rebar In-Reply-To: <2f3fe3c3-b64b-4e45-a9d3-8fec31c01075@x12g2000yqx.googlegroups.com> References: <2f3fe3c3-b64b-4e45-a9d3-8fec31c01075@x12g2000yqx.googlegroups.com> Message-ID: On Fri, Apr 16, 2010 at 15:31, Jeremy Raymond wrote: > I can reproduce this without couchbeam as well by just calling > code:clash() directly in my eunit test: > > *** context setup failed *** > ::error:function_clause > ?in function code:filter/3 > ? ?called as filter(".beam",".eunit",error) > ?in call from code:build/1 > ?in call from code:clash/0 > ?in call from cdb_data_tests:setup/0 Hi, in code.erl: build([]) -> []; build([Dir|Tail]) -> Files = filter(objfile_extension(), Dir, erl_prim_loader:list_dir(Dir)), [decorate(Files, Dir) | build(Tail)]. filter(_Ext, Dir, {error,_}) -> io:format("** Bad path can't read ~s~n", [Dir]), []; filter(Ext, _, {ok,Files}) -> filter2(Ext, length(Ext), Files). and in erl_prim_loader.erl -spec list_dir(string()) -> {'ok', [string()]} | 'error'. list_dir(Dir) -> check_file_result(list_dir, Dir, request({list_dir,Dir})). Where you can see that erl_prim_loader returns 'error' and filter expects {ok, _} or {error, _}. It feels like Dialyzer should have caught that? best regards, Vlad From igorrs@REDACTED Fri Apr 16 16:45:34 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Fri, 16 Apr 2010 11:45:34 -0300 Subject: [erlang-questions] atomic read+delete operation on ETS record In-Reply-To: <892067.26404.qm@web112603.mail.gq1.yahoo.com> References: <892067.26404.qm@web112603.mail.gq1.yahoo.com> Message-ID: With Mnesia on top of ets, you can use transactions and record-locks (fun mnesia:read/3). But I realize Mnesia may not be good enough for what you want (not as simple, not as fast, doesn't support duplicate_bag etc.). Igor. On Fri, Apr 16, 2010 at 9:06 AM, Pablo Platt wrote: > ETS has atomic operations like update_element and update_counter. > Is there a way to read a record and delete it in the same operation without using a gen_server that manage the table? > > Is there a way to update a list inside a record in an atomic way? > For example, mongodb and redis let you add and remove items from the beginning and end of lists. > ETS records are Erlang terms so it might be possible and will be very useful. > > Thanks From tuncer.ayaz@REDACTED Fri Apr 16 16:46:26 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 16 Apr 2010 16:46:26 +0200 Subject: [erlang-questions] code:clash/0 borking when running eunit tests with rebar In-Reply-To: References: <2f3fe3c3-b64b-4e45-a9d3-8fec31c01075@x12g2000yqx.googlegroups.com> Message-ID: On Fri, Apr 16, 2010 at 3:42 PM, Vlad Dumitrescu wrote: > On Fri, Apr 16, 2010 at 15:31, Jeremy Raymond wrote: >> I can reproduce this without couchbeam as well by just calling >> code:clash() directly in my eunit test: >> >> *** context setup failed *** >> ::error:function_clause >> ?in function code:filter/3 >> ? ?called as filter(".beam",".eunit",error) >> ?in call from code:build/1 >> ?in call from code:clash/0 >> ?in call from cdb_data_tests:setup/0 > > Hi, > > in code.erl: > > build([]) -> []; > build([Dir|Tail]) -> > ? ?Files = filter(objfile_extension(), Dir, > ? ? ? ? ? ? ? ? ? erl_prim_loader:list_dir(Dir)), > ? ?[decorate(Files, Dir) | build(Tail)]. > > filter(_Ext, Dir, {error,_}) -> > ? ?io:format("** Bad path can't read ~s~n", [Dir]), []; > filter(Ext, _, {ok,Files}) -> > ? ?filter2(Ext, length(Ext), Files). > > and in erl_prim_loader.erl > > -spec list_dir(string()) -> {'ok', [string()]} | 'error'. > list_dir(Dir) -> > ? ?check_file_result(list_dir, Dir, request({list_dir,Dir})). > > Where you can see that erl_prim_loader returns 'error' and filter > expects {ok, _} or {error, _}. It feels like Dialyzer should have > caught that? Thanks, nice catch! I've introduced that in 49da83 while fixing another code:clash/0 issue. Working on a patch with extended tests atop 79194d. From ulf.wiger@REDACTED Fri Apr 16 17:39:39 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 16 Apr 2010 17:39:39 +0200 Subject: [erlang-questions] atomic read+delete operation on ETS record In-Reply-To: <892067.26404.qm@web112603.mail.gq1.yahoo.com> References: <892067.26404.qm@web112603.mail.gq1.yahoo.com> Message-ID: <4BC884BB.8020808@erlang-solutions.com> Pablo Platt wrote: > ETS has atomic operations like update_element and update_counter. > Is there a way to read a record and delete it in the same operation without using a gen_server that manage the table? > > Is there a way to update a list inside a record in an atomic way? > For example, mongodb and redis let you add and remove items from the beginning and end of lists. > ETS records are Erlang terms so it might be possible and will be very useful. I did a dirty hack 3 years ago trying to increase the number of things that could be done atomically with ets: http://www.erlang.org/pipermail/erlang-questions/2007-June/026998.html BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From mevans@REDACTED Fri Apr 16 18:06:59 2010 From: mevans@REDACTED (Evans, Matthew) Date: Fri, 16 Apr 2010 12:06:59 -0400 Subject: [erlang-questions] atomic read+delete operation on ETS record In-Reply-To: <4BC884BB.8020808@erlang-solutions.com> References: <892067.26404.qm@web112603.mail.gq1.yahoo.com> <4BC884BB.8020808@erlang-solutions.com> Message-ID: I wonder if it would be possible to use global:trans and have the ResourceId set to the name or identifier of the ETS table (and LockRequesterId as the calling processes pid)? Performance might be a dog however.... -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Ulf Wiger Sent: Friday, April 16, 2010 11:40 AM To: Pablo Platt Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] atomic read+delete operation on ETS record Pablo Platt wrote: > ETS has atomic operations like update_element and update_counter. > Is there a way to read a record and delete it in the same operation without using a gen_server that manage the table? > > Is there a way to update a list inside a record in an atomic way? > For example, mongodb and redis let you add and remove items from the beginning and end of lists. > ETS records are Erlang terms so it might be possible and will be very useful. I did a dirty hack 3 years ago trying to increase the number of things that could be done atomically with ets: http://www.erlang.org/pipermail/erlang-questions/2007-June/026998.html BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From leap@REDACTED Fri Apr 16 18:26:04 2010 From: leap@REDACTED (Michael Turner) Date: Fri, 16 Apr 2010 16:26:04 +0000 Subject: floating point syntax In-Reply-To: Message-ID: I guess this has something to do with the fact that a period is also a terminator in Erlang syntax. If you allowed "1." as a legal float, for example, you're permitting this: fp1() -> 1.. And with f() -> 1.e5; you need a bit more token lookahead (since "e5" could be seen as the start of a new function). My guess, as I say. -michael turner On 4/15/2010, "Robert Virding" wrote: >Yes, it is supposed to be like that, no, it is not a bug. Floats >consist of both parts before and after the decimal point and an >optional exponent. > >Robert > >On 15 April 2010 16:21, Anthony Shipman wrote: >> The reference manual says "There are two types of numeric literals, integers >> and floats. Besides the conventional notation,..." >> >> But what is the "conventional notation" exactly? I find that this fails: >> 19> io_lib:fread("~f", "1e+2"). >> {error,{fread,float}} >> >> Also >> 1.e+2 fails >> 1 ? ? fails >> 1. ? ?fails >> 1.0 ? succeeds >> .1 ? ?fails >> 0.1 ? succeeds >> 1.0e+2 succeeds >> >> Is it supposed to be like that or is there a bug? >> >> -- >> Anthony Shipman ? ? ? ? ? ? ? ? ? ?Mamas don't let your babies >> als@REDACTED ? ? ? ? ? ? ? ? ? grow up to be outsourced. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > >________________________________________________________________ >erlang-questions (at) erlang.org mailing list. >See http://www.erlang.org/faq.html >To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tuncer.ayaz@REDACTED Fri Apr 16 19:33:11 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 16 Apr 2010 19:33:11 +0200 Subject: [erlang-questions] code:clash/0 borking when running eunit tests with rebar In-Reply-To: References: <2f3fe3c3-b64b-4e45-a9d3-8fec31c01075@x12g2000yqx.googlegroups.com> Message-ID: On Fri, Apr 16, 2010 at 4:46 PM, Tuncer Ayaz wrote: > On Fri, Apr 16, 2010 at 3:42 PM, Vlad Dumitrescu wrote: >> On Fri, Apr 16, 2010 at 15:31, Jeremy Raymond wrote: >>> I can reproduce this without couchbeam as well by just calling >>> code:clash() directly in my eunit test: >>> >>> *** context setup failed *** >>> ::error:function_clause >>> ?in function code:filter/3 >>> ? ?called as filter(".beam",".eunit",error) >>> ?in call from code:build/1 >>> ?in call from code:clash/0 >>> ?in call from cdb_data_tests:setup/0 >> >> Hi, >> >> in code.erl: >> >> build([]) -> []; >> build([Dir|Tail]) -> >> ? ?Files = filter(objfile_extension(), Dir, >> ? ? ? ? ? ? ? ? ? erl_prim_loader:list_dir(Dir)), >> ? ?[decorate(Files, Dir) | build(Tail)]. >> >> filter(_Ext, Dir, {error,_}) -> >> ? ?io:format("** Bad path can't read ~s~n", [Dir]), []; >> filter(Ext, _, {ok,Files}) -> >> ? ?filter2(Ext, length(Ext), Files). >> >> and in erl_prim_loader.erl >> >> -spec list_dir(string()) -> {'ok', [string()]} | 'error'. >> list_dir(Dir) -> >> ? ?check_file_result(list_dir, Dir, request({list_dir,Dir})). >> >> Where you can see that erl_prim_loader returns 'error' and filter >> expects {ok, _} or {error, _}. It feels like Dialyzer should have >> caught that? > > Thanks, nice catch! > > I've introduced that in 49da83 while fixing another > code:clash/0 issue. > Working on a patch with extended tests atop 79194d. Fix posted to erlang-patches@REDACTED I think this hasn't been caught earlier as usually invalid paths are caught in add_path(). A possible failure case is when a file/directory which is part of the code path disappears. I've extended the tests to catch that. From mark.fine@REDACTED Fri Apr 16 20:09:34 2010 From: mark.fine@REDACTED (Mark Fine) Date: Fri, 16 Apr 2010 11:09:34 -0700 Subject: Parameterized module idioms Message-ID: Is there a cleaner, more idiomatic way to use object-like parameterized modules: -module(echo, [PID]). -behaviour(gen_server). -export([start_link/0, echo/1]). -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). start_link() -> case gen_server:start_link(?MODULE:new(undefined), [], []) of {ok, Pid} -> {ok, ?MODULE:new(Pid)}; Else -> Else end. echo(What) -> gen_server:call(PID, {echo, What}). handle_call({echo, What}, _From, State) -> {reply, What, State}. init([]) -> {ok, []}. handle_cast(_Msg, _State) -> undefined. handle_info(_Info, _State) -> undefined. terminate(_Reason, _State) -> undefined. code_change(_OldVsn, _State, _Extra) -> undefined. 17> {ok, E} = (echo:new(undefined)):start_link(). {ok,{echo,<0.82.0>}} 18> E:echo("hello, world"). "hello, world" 19> From jeraymond@REDACTED Fri Apr 16 22:00:37 2010 From: jeraymond@REDACTED (Jeremy Raymond) Date: Fri, 16 Apr 2010 16:00:37 -0400 Subject: [erlang-questions] code:clash/0 borking when running eunit tests with rebar In-Reply-To: References: <2f3fe3c3-b64b-4e45-a9d3-8fec31c01075@x12g2000yqx.googlegroups.com> Message-ID: I've tested the patch in my environment and it fixes the problem, code:clash/0 no longer crashes on the invalid file path. Also note that a patch to rebar_eunit is to be pushed out soon as well to use full paths rather than relative ones as erl_prim_loader doesn't seem to like them. On Fri, Apr 16, 2010 at 1:33 PM, Tuncer Ayaz wrote: > On Fri, Apr 16, 2010 at 4:46 PM, Tuncer Ayaz > wrote: > > On Fri, Apr 16, 2010 at 3:42 PM, Vlad Dumitrescu > wrote: > >> On Fri, Apr 16, 2010 at 15:31, Jeremy Raymond > wrote: > >>> I can reproduce this without couchbeam as well by just calling > >>> code:clash() directly in my eunit test: > >>> > >>> *** context setup failed *** > >>> ::error:function_clause > >>> in function code:filter/3 > >>> called as filter(".beam",".eunit",error) > >>> in call from code:build/1 > >>> in call from code:clash/0 > >>> in call from cdb_data_tests:setup/0 > >> > >> Hi, > >> > >> in code.erl: > >> > >> build([]) -> []; > >> build([Dir|Tail]) -> > >> Files = filter(objfile_extension(), Dir, > >> erl_prim_loader:list_dir(Dir)), > >> [decorate(Files, Dir) | build(Tail)]. > >> > >> filter(_Ext, Dir, {error,_}) -> > >> io:format("** Bad path can't read ~s~n", [Dir]), []; > >> filter(Ext, _, {ok,Files}) -> > >> filter2(Ext, length(Ext), Files). > >> > >> and in erl_prim_loader.erl > >> > >> -spec list_dir(string()) -> {'ok', [string()]} | 'error'. > >> list_dir(Dir) -> > >> check_file_result(list_dir, Dir, request({list_dir,Dir})). > >> > >> Where you can see that erl_prim_loader returns 'error' and filter > >> expects {ok, _} or {error, _}. It feels like Dialyzer should have > >> caught that? > > > > Thanks, nice catch! > > > > I've introduced that in 49da83 while fixing another > > code:clash/0 issue. > > Working on a patch with extended tests atop 79194d. > > Fix posted to erlang-patches@REDACTED > > I think this hasn't been caught earlier as usually invalid > paths are caught in add_path(). A possible failure case is when > a file/directory which is part of the code path disappears. > > I've extended the tests to catch that. > -- Jeremy Raymond From g@REDACTED Sat Apr 17 01:56:54 2010 From: g@REDACTED (Garrett Smith) Date: Fri, 16 Apr 2010 18:56:54 -0500 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: Message-ID: On Fri, Apr 16, 2010 at 1:09 PM, Mark Fine wrote: > Is there a cleaner, more idiomatic way to use object-like parameterized > modules: > > -module(echo, [PID]). > -behaviour(gen_server). > -export([start_link/0, echo/1]). > -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, > code_change/3]). > > start_link() -> > ? ?case gen_server:start_link(?MODULE:new(undefined), [], []) of > ? ? ? ?{ok, Pid} -> > ? ? ? ? ? ?{ok, ?MODULE:new(Pid)}; > ? ? ? ?Else -> > ? ? ? ? ? ?Else > ? ?end. > > echo(What) -> > ? ?gen_server:call(PID, {echo, What}). > > handle_call({echo, What}, _From, State) -> > ? ?{reply, What, State}. > > init([]) -> {ok, []}. > handle_cast(_Msg, _State) -> undefined. > handle_info(_Info, _State) -> undefined. > terminate(_Reason, _State) -> undefined. > code_change(_OldVsn, _State, _Extra) -> undefined. > > 17> {ok, E} = (echo:new(undefined)):start_link(). > {ok,{echo,<0.82.0>}} > 18> E:echo("hello, world"). > "hello, world" > 19> > I realize there are some fans of parameterized modules. I'm wondering though if this: E:echo("hello") provides enough payoff over this: echo(E, "hello") to justify the effort. I come from an OO background myself, but for whatever reason, the second form doesn't bother me. It may be that it's so endemic in Erlang that I've gotten used to it. Garrett From bob@REDACTED Sat Apr 17 02:06:32 2010 From: bob@REDACTED (Bob Ippolito) Date: Fri, 16 Apr 2010 17:06:32 -0700 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: Message-ID: On Fri, Apr 16, 2010 at 4:56 PM, Garrett Smith wrote: > On Fri, Apr 16, 2010 at 1:09 PM, Mark Fine wrote: >> Is there a cleaner, more idiomatic way to use object-like parameterized >> modules: >> >> -module(echo, [PID]). >> -behaviour(gen_server). >> -export([start_link/0, echo/1]). >> -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, >> code_change/3]). >> >> start_link() -> >> ? ?case gen_server:start_link(?MODULE:new(undefined), [], []) of >> ? ? ? ?{ok, Pid} -> >> ? ? ? ? ? ?{ok, ?MODULE:new(Pid)}; >> ? ? ? ?Else -> >> ? ? ? ? ? ?Else >> ? ?end. >> >> echo(What) -> >> ? ?gen_server:call(PID, {echo, What}). >> >> handle_call({echo, What}, _From, State) -> >> ? ?{reply, What, State}. >> >> init([]) -> {ok, []}. >> handle_cast(_Msg, _State) -> undefined. >> handle_info(_Info, _State) -> undefined. >> terminate(_Reason, _State) -> undefined. >> code_change(_OldVsn, _State, _Extra) -> undefined. >> >> 17> {ok, E} = (echo:new(undefined)):start_link(). >> {ok,{echo,<0.82.0>}} >> 18> E:echo("hello, world"). >> "hello, world" >> 19> >> > > I realize there are some fans of parameterized modules. I'm wondering > though if this: > > ?E:echo("hello") > > provides enough payoff over this: > > ?echo(E, "hello") > > to justify the effort. > > I come from an OO background myself, but for whatever reason, the > second form doesn't bother me. It may be that it's so endemic in > Erlang that I've gotten used to it. Technically speaking the difference is: E:echo("hello") vs. some_module:echo(E, "hello") The biggest reason I've seen to use parameterized modules is that you could have a different module that implements the same interface, e.g. sets and gb_sets and you could change which one you're using without changing any of the code that consumes it. I would definitely change the original code to use two modules, one module to do the start_link stuff, and another module to actually implement the interface. -bob From mononcqc@REDACTED Sat Apr 17 02:13:39 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 16 Apr 2010 20:13:39 -0400 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: Message-ID: On Fri, Apr 16, 2010 at 8:06 PM, Bob Ippolito wrote: > > Technically speaking the difference is: > > E:echo("hello") vs. some_module:echo(E, "hello") > > The biggest reason I've seen to use parameterized modules is that you > could have a different module that implements the same interface, e.g. > sets and gb_sets and you could change which one you're using without > changing any of the code that consumes it. > > I would definitely change the original code to use two modules, one > module to do the start_link stuff, and another module to actually > implement the interface. > > -bob > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > M = sets, D = M:new(). M:add_element(10, D). This works as well without a problem. Switch the value of M and if the interface is the same, nothing changes. From bob@REDACTED Sat Apr 17 03:04:58 2010 From: bob@REDACTED (Bob Ippolito) Date: Fri, 16 Apr 2010 18:04:58 -0700 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: Message-ID: On Fri, Apr 16, 2010 at 5:13 PM, Fred Hebert wrote: > On Fri, Apr 16, 2010 at 8:06 PM, Bob Ippolito wrote: >> >> Technically speaking the difference is: >> >> E:echo("hello") vs. some_module:echo(E, "hello") >> >> The biggest reason I've seen to use parameterized modules is that you >> could have a different module that implements the same interface, e.g. >> sets and gb_sets and you could change which one you're using without >> changing any of the code that consumes it. >> >> I would definitely change the original code to use two modules, one >> module to do the start_link stuff, and another module to actually >> implement the interface. >> > M = sets, > D = M:new(). > M:add_element(10, D). > > This works as well without a problem. Switch the value of M and if the > interface is the same, nothing changes. Yes, but now you're passing around two variables instead of one. -bob From g@REDACTED Sat Apr 17 03:06:47 2010 From: g@REDACTED (Garrett Smith) Date: Fri, 16 Apr 2010 20:06:47 -0500 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: Message-ID: On Fri, Apr 16, 2010 at 7:06 PM, Bob Ippolito wrote: > On Fri, Apr 16, 2010 at 4:56 PM, Garrett Smith wrote: >> On Fri, Apr 16, 2010 at 1:09 PM, Mark Fine wrote: >>> Is there a cleaner, more idiomatic way to use object-like parameterized >>> modules: >>> >>> -module(echo, [PID]). >>> -behaviour(gen_server). >>> -export([start_link/0, echo/1]). >>> -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, >>> code_change/3]). >>> >>> start_link() -> >>> ? ?case gen_server:start_link(?MODULE:new(undefined), [], []) of >>> ? ? ? ?{ok, Pid} -> >>> ? ? ? ? ? ?{ok, ?MODULE:new(Pid)}; >>> ? ? ? ?Else -> >>> ? ? ? ? ? ?Else >>> ? ?end. >>> >>> echo(What) -> >>> ? ?gen_server:call(PID, {echo, What}). >>> >>> handle_call({echo, What}, _From, State) -> >>> ? ?{reply, What, State}. >>> >>> init([]) -> {ok, []}. >>> handle_cast(_Msg, _State) -> undefined. >>> handle_info(_Info, _State) -> undefined. >>> terminate(_Reason, _State) -> undefined. >>> code_change(_OldVsn, _State, _Extra) -> undefined. >>> >>> 17> {ok, E} = (echo:new(undefined)):start_link(). >>> {ok,{echo,<0.82.0>}} >>> 18> E:echo("hello, world"). >>> "hello, world" >>> 19> >>> >> >> I realize there are some fans of parameterized modules. I'm wondering >> though if this: >> >> ?E:echo("hello") >> >> provides enough payoff over this: >> >> ?echo(E, "hello") >> >> to justify the effort. >> >> I come from an OO background myself, but for whatever reason, the >> second form doesn't bother me. It may be that it's so endemic in >> Erlang that I've gotten used to it. > > Technically speaking the difference is: > > E:echo("hello") vs. some_module:echo(E, "hello") Yup, sorry, I did deceptively simplify my example :) > The biggest reason I've seen to use parameterized modules is that you > could have a different module that implements the same interface, e.g. > sets and gb_sets and you could change which one you're using without > changing any of the code that consumes it. I've found that custom behaviors also work well for this, at least for interfaces to processes. > I would definitely change the original code to use two modules, one > module to do the start_link stuff, and another module to actually > implement the interface. > > -bob > From bob@REDACTED Sat Apr 17 03:09:37 2010 From: bob@REDACTED (Bob Ippolito) Date: Fri, 16 Apr 2010 18:09:37 -0700 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: Message-ID: On Fri, Apr 16, 2010 at 6:06 PM, Garrett Smith wrote: > On Fri, Apr 16, 2010 at 7:06 PM, Bob Ippolito wrote: >> On Fri, Apr 16, 2010 at 4:56 PM, Garrett Smith wrote: >>> On Fri, Apr 16, 2010 at 1:09 PM, Mark Fine wrote: >>>> Is there a cleaner, more idiomatic way to use object-like parameterized >>>> modules: >>>> >>>> -module(echo, [PID]). >>>> -behaviour(gen_server). >>>> -export([start_link/0, echo/1]). >>>> -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, >>>> code_change/3]). >>>> >>>> start_link() -> >>>> ? ?case gen_server:start_link(?MODULE:new(undefined), [], []) of >>>> ? ? ? ?{ok, Pid} -> >>>> ? ? ? ? ? ?{ok, ?MODULE:new(Pid)}; >>>> ? ? ? ?Else -> >>>> ? ? ? ? ? ?Else >>>> ? ?end. >>>> >>>> echo(What) -> >>>> ? ?gen_server:call(PID, {echo, What}). >>>> >>>> handle_call({echo, What}, _From, State) -> >>>> ? ?{reply, What, State}. >>>> >>>> init([]) -> {ok, []}. >>>> handle_cast(_Msg, _State) -> undefined. >>>> handle_info(_Info, _State) -> undefined. >>>> terminate(_Reason, _State) -> undefined. >>>> code_change(_OldVsn, _State, _Extra) -> undefined. >>>> >>>> 17> {ok, E} = (echo:new(undefined)):start_link(). >>>> {ok,{echo,<0.82.0>}} >>>> 18> E:echo("hello, world"). >>>> "hello, world" >>>> 19> >>>> >>> >>> I realize there are some fans of parameterized modules. I'm wondering >>> though if this: >>> >>> ?E:echo("hello") >>> >>> provides enough payoff over this: >>> >>> ?echo(E, "hello") >>> >>> to justify the effort. >>> >>> I come from an OO background myself, but for whatever reason, the >>> second form doesn't bother me. It may be that it's so endemic in >>> Erlang that I've gotten used to it. >> >> Technically speaking the difference is: >> >> E:echo("hello") vs. some_module:echo(E, "hello") > > Yup, sorry, I did deceptively simplify my example :) > >> The biggest reason I've seen to use parameterized modules is that you >> could have a different module that implements the same interface, e.g. >> sets and gb_sets and you could change which one you're using without >> changing any of the code that consumes it. > > I've found that custom behaviors also work well for this, at least for > interfaces to processes. Yeah well, processes can do everything that parameterized modules can do, except perform well if you're trying to do a lot of concurrency ;) -bob From g@REDACTED Sat Apr 17 03:34:58 2010 From: g@REDACTED (Garrett Smith) Date: Fri, 16 Apr 2010 20:34:58 -0500 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: Message-ID: On Fri, Apr 16, 2010 at 8:04 PM, Bob Ippolito wrote: > On Fri, Apr 16, 2010 at 5:13 PM, Fred Hebert wrote: >> On Fri, Apr 16, 2010 at 8:06 PM, Bob Ippolito wrote: >>> >>> Technically speaking the difference is: >>> >>> E:echo("hello") vs. some_module:echo(E, "hello") >>> >>> The biggest reason I've seen to use parameterized modules is that you >>> could have a different module that implements the same interface, e.g. >>> sets and gb_sets and you could change which one you're using without >>> changing any of the code that consumes it. >>> >>> I would definitely change the original code to use two modules, one >>> module to do the start_link stuff, and another module to actually >>> implement the interface. >>> >> M = sets, >> D = M:new(). >> M:add_element(10, D). >> >> This works as well without a problem. Switch the value of M and if the >> interface is the same, nothing changes. > > Yes, but now you're passing around two variables instead of one. I'd be tempted to create a wrapper that took the impl module in its new function and maintained it as internal state. -module(gen_dict). -define(state, {impl, impl_state}). new(Impl) -> #state{impl=Impl, impl_state=Impl:new()}. store(Key, Val, #state{impl=Impl, impl_state=ImplState}=S) -> S#state{impl_state=Impl:store(Key, Val, ImplState)}. ... and so on... And to use (obviously): D = gen_dict:new(dict), D2 = gen_dict:store(foo, bar, D) Granted, there's a fair amount of horsing around, but not terrible. And your generic API is explicit. Behaviors should work for this as well as they just enforce an exported API. Garrett From mononcqc@REDACTED Sat Apr 17 03:55:06 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 16 Apr 2010 21:55:06 -0400 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: Message-ID: The biggest 'advantage' of parametrized modules that I see is that you shift the burden of carrying state from a variable to a module name. In this module: -module(y, [Name,Age]). -compile(export_all). action1() -> io:format("Hello, ~p!~n",[Name]). get_age() -> Age. However, my biggest problem with them show up were I to add the following function: ret_fn() -> fun(Age) -> Age + 5 end. and then tried to compile it, all of a sudden I get the error "./y.erl:8: Warning: variable 'Age' shadowed in 'fun'". Of course, this is not problematic if you can hold all the info about the module parameters and their names in your head, but if you were to look only at the function where the error takes place, you would get no sign whatsoever of where the error comes from. Now if I were to use the variable 'Name' for any other purpose; I'd get a run time error telling "** error: no match of right hand side value < ... >." Again, there I'm having no obvious sign of where the error might come from -- I have to look for the module definition to know the cause of it. The way I see it is that parametrized modules reduce the amount of typing you need to do at the expense of logical simplicity. I'm not sure I'm too enthusiastic at the idea of making programs harder to reason about for the sake of typing less. I guess the problem is going to be less and less apparent with programmers getting used to it, but yeah. That's my point of view. From mail@REDACTED Sat Apr 17 11:39:04 2010 From: mail@REDACTED (Tim Fletcher) Date: Sat, 17 Apr 2010 02:39:04 -0700 (PDT) Subject: Parameterized module idioms In-Reply-To: References: Message-ID: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> > The biggest 'advantage' of parametrized modules that I see is that you shift > the burden of carrying state from a variable to a module name. IMO this is also a disadvantage, but not specific to parametrized modules: using variables for module references makes static analysis harder (both in your head and in code). For example, [unless you're doing some crazy rewriting tricks] it's obvious which function this is calling: some_module:echo(E, "hello") This isn't so obvious: E:echo("hello") Here you have to trace back to the definition of E to find out what module it refers to. Tim From hynek@REDACTED Sat Apr 17 12:57:33 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Sat, 17 Apr 2010 12:57:33 +0200 Subject: [erlang-questions] Re: Parameterized module idioms In-Reply-To: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> Message-ID: > For example, [unless you're doing some crazy rewriting tricks] it's > obvious which function this is calling: > > ? ?some_module:echo(E, "hello") > > This isn't so obvious: > > ? ?E:echo("hello") > > Here you have to trace back to the definition of E to find out what > module it refers to. I agree with you. It is issue especially in projects with lack of documentation as mochiweb for example. I thought parametrized modules is great idea until I met mochiweb. Hynek (Pichi) Vychodil From romanshestakov@REDACTED Sat Apr 17 11:58:50 2010 From: romanshestakov@REDACTED (Roman Shestakov) Date: Sat, 17 Apr 2010 09:58:50 +0000 (GMT) Subject: how to define multiple include dirs with i option in Emakefile? Message-ID: <218412.49896.qm@web25403.mail.ukl.yahoo.com> hello, what is the correct way to define multiple include dirs in Emakefile? e.g. I would like to be able to do this: {['src/*'], [debug_info, {outdir, "ebin"}, {i,"include;other/include"}]}. Regards, Roman From pablo.platt@REDACTED Sat Apr 17 12:16:05 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Sat, 17 Apr 2010 03:16:05 -0700 (PDT) Subject: [erlang-questions] atomic read+delete operation on ETS record In-Reply-To: <4BC884BB.8020808@erlang-solutions.com> References: <892067.26404.qm@web112603.mail.gq1.yahoo.com> <4BC884BB.8020808@erlang-solutions.com> Message-ID: <962099.7117.qm@web112613.mail.gq1.yahoo.com> >From you old post I understand that this need to be done as a BIF. Is there a c file I can see for reference so I can play with it and contribute? ________________________________ From: Ulf Wiger To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Fri, April 16, 2010 6:39:39 PM Subject: Re: [erlang-questions] atomic read+delete operation on ETS record Pablo Platt wrote: > ETS has atomic operations like update_element and update_counter. > Is there a way to read a record and delete it in the same operation without using a gen_server that manage the table? > > Is there a way to update a list inside a record in an atomic way? > For example, mongodb and redis let you add and remove items from the beginning and end of lists. > ETS records are Erlang terms so it might be possible and will be very useful. I did a dirty hack 3 years ago trying to increase the number of things that could be done atomically with ets: http://www.erlang.org/pipermail/erlang-questions/2007-June/026998.html BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From nc@REDACTED Sat Apr 17 14:05:30 2010 From: nc@REDACTED (Nicolas Charpentier) Date: Sat, 17 Apr 2010 14:05:30 +0200 Subject: [erlang-questions] how to define multiple include dirs with i option in Emakefile? In-Reply-To: <218412.49896.qm@web25403.mail.ukl.yahoo.com> References: <218412.49896.qm@web25403.mail.ukl.yahoo.com> Message-ID: <4BC9A40A.1040509@charpi.net> Hi, Roman Shestakov wrote: > hello, > > what is the correct way to define multiple include dirs in Emakefile? > Just add a tuple {i,Dir} for each include directory: {['src/*'], [{outdir, "ebin"},{i,"include"},{i,"other/include"}]}. Regards, Nicolas From bob@REDACTED Sat Apr 17 18:36:00 2010 From: bob@REDACTED (Bob Ippolito) Date: Sat, 17 Apr 2010 09:36:00 -0700 Subject: [erlang-questions] Re: Parameterized module idioms In-Reply-To: References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> Message-ID: On Sat, Apr 17, 2010 at 3:57 AM, Hynek Vychodil wrote: >> For example, [unless you're doing some crazy rewriting tricks] it's >> obvious which function this is calling: >> >> ? ?some_module:echo(E, "hello") >> >> This isn't so obvious: >> >> ? ?E:echo("hello") >> >> Here you have to trace back to the definition of E to find out what >> module it refers to. > > I agree with you. It is issue especially in projects with lack of > documentation as mochiweb for example. I thought parametrized modules > is great idea until I met mochiweb. To be fair there's edocs for all of the public functions and there's really only one parameterized module you have to use (mochiweb_request). It's really not that hard to figure it out, you only look in one place for the docs. Yes, it should have some expository documentation but that's not something I ever got around to doing, and it's why I never did the other stuff like make a webpage for it or release tarballs. -bob From kagato@REDACTED Sat Apr 17 23:28:34 2010 From: kagato@REDACTED (Jayson Vantuyl) Date: Sat, 17 Apr 2010 14:28:34 -0700 Subject: [erlang-questions] Re: Parameterized module idioms In-Reply-To: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> Message-ID: <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> > For example, [unless you're doing some crazy rewriting tricks] it's > obvious which function this is calling: > > some_module:echo(E, "hello") > > This isn't so obvious: > > E:echo("hello") > > Here you have to trace back to the definition of E to find out what > module it refers to. Actually, that's the point. There are two good reasons to do this. 1. Hide massive amounts of information. While this runs counter to your point, sometimes there's so much state data that it makes code impossible to maintain if you explicitly write it all. This neatly hides it. More to the point, Erlang already does this with records. They allow you to pick and assign bits of the state without seeing it all. You still have to trace the definition back to the record. I don't really see this as different. 2. Polymorphism This is the big win. E might refer to any number of classes which are instantiated in any number of actual combinations. But another way, imagine that you separated this out into different types. Then the call would be: > T:echo(E,"hello") This isn't nearly as nice as: > E:echo("hello") In this case, E holds all of the state, including the module. This is no different than gen_server and friends. It just considers the callback module to be part of the state, bundles them together out of side, and gives a reasonably short syntax to show them. In practice, I don't think this is that hard to debug--especially in the cases where it yields shorter, simpler code. -- Jayson Vantuyl kagato@REDACTED From hynek@REDACTED Sun Apr 18 01:48:01 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Sun, 18 Apr 2010 01:48:01 +0200 Subject: [erlang-questions] Re: Parameterized module idioms In-Reply-To: References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> Message-ID: > > To be fair there's edocs for all of the public functions and there's > really only one parameterized module you have to use > (mochiweb_request). It's really not that hard to figure it out, you > only look in one place for the docs. Yes, it should have some > expository documentation but that's not something I ever got around to > doing, and it's why I never did the other stuff like make a webpage > for it or release tarballs. > > -bob > To be fair there is edocs for all the public functions but in those days It was mostly just only name of function, one sentence for some one and may be there was some one with more but I was not lucky enough to find them. Even information that Req (parameter of almost each function) is implemented in mochiweb_request module was not pointed from most of documentation if ever. Jayson Vantuyl is pointing in another mail that same polymorphism can be done without parametrized module and all this is just bad experience with one application. Yes, it seems true. Well, parametrized modules doesn't bring more issues than present in rest of Erlang. -- --Hynek (Pichi) Vychodil From kiszl@REDACTED Sun Apr 18 10:12:43 2010 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sun, 18 Apr 2010 10:12:43 +0200 Subject: [erlang-questions] Re: Parameterized module idioms In-Reply-To: References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> Message-ID: <4BCABEFB.7020101@tmit.bme.hu> By the way, are there any news on whether and if so how will parametrized modules be supported in R14? Regards, Zoltan. From tuncer.ayaz@REDACTED Sun Apr 18 13:16:16 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 18 Apr 2010 13:16:16 +0200 Subject: [erlang-questions] Re: Parameterized module idioms In-Reply-To: <4BCABEFB.7020101@tmit.bme.hu> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <4BCABEFB.7020101@tmit.bme.hu> Message-ID: On Sun, Apr 18, 2010 at 10:12 AM, Zoltan Lajos Kis wrote: > By the way, > are there any news on whether and if so how will parametrized modules be > supported in R14? Kenneth said the following at Erlang Factory SFBay 2010: "Tentative: Parameterized modules officially supported and with more efficient implementation." From kagato@REDACTED Sat Apr 17 23:28:34 2010 From: kagato@REDACTED (Jayson Vantuyl) Date: Sat, 17 Apr 2010 14:28:34 -0700 Subject: [erlang-questions] Re: Parameterized module idioms In-Reply-To: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> Message-ID: <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> > For example, [unless you're doing some crazy rewriting tricks] it's > obvious which function this is calling: > > some_module:echo(E, "hello") > > This isn't so obvious: > > E:echo("hello") > > Here you have to trace back to the definition of E to find out what > module it refers to. Actually, that's the point. There are two good reasons to do this. 1. Hide massive amounts of information. While this runs counter to your point, sometimes there's so much state data that it makes code impossible to maintain if you explicitly write it all. This neatly hides it. More to the point, Erlang already does this with records. They allow you to pick and assign bits of the state without seeing it all. You still have to trace the definition back to the record. I don't really see this as different. 2. Polymorphism This is the big win. E might refer to any number of classes which are instantiated in any number of actual combinations. But another way, imagine that you separated this out into different types. Then the call would be: > T:echo(E,"hello") This isn't nearly as nice as: > E:echo("hello") In this case, E holds all of the state, including the module. This is no different than gen_server and friends. It just considers the callback module to be part of the state, bundles them together out of side, and gives a reasonably short syntax to show them. In practice, I don't think this is that hard to debug--especially in the cases where it yields shorter, simpler code. -- Jayson Vantuyl kagato@REDACTED From mononcqc@REDACTED Sun Apr 18 22:50:16 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Sun, 18 Apr 2010 16:50:16 -0400 Subject: A discussion in application design and event propagation Message-ID: I'm starting this thread to spark a discussion about Erlang software design. I've been writing a few applications here and there and I've noticed a few common patterns occurring over time, but with not clear solution to them. That seems to be due to the general "there is more than one way to do it" attitude that exists when organizing processes, and not because of a lack of activism. Anyway, the design question comes from designs having to do with multi-user activities, might it be chat rooms, games of cards over the web, voting system, a pub-sub mechanism, etc. The general idea is that you have a group of users and a sequence of events that has to be forwarded to every user in real time. To make this clearer, I'll take the example of a blackjack table where every player has a GUI showing the state of the game. You'd have (I imagine) one process representing the dealer and the table (D). Then you have 3 players, P1, P2 and P3, each represented as a process. A hand distribution goes like this: 1. Player 1 registers to the table 2. Player 2 registers. Player 1 is notified of this. 3. Player 3 registers, P1 and P2 are notified. 4. The game starts, All players are notified of this. 5. Each player (and the dealer) is handed a card 6. Players bet. All players are notified of this; 7. ... 8. etc. Then you could also have some messages private (imagine having a private hand as in a poker game or whatever). There are two obvious ways I can think of to model this. The first one is as a gen_event process and handlers, where all players register an event handler. When an event occurs, either the dealer or the players notify the event manager which then forwards the event to subscribers through their respective event handlers. The second way I can think of are process groups, where a message sent to the leader is automatically forwarded to all processes who are members of the group. It seems to me these are equivalent solutions, as they allow for the same propagation of events in correct order. However, the former runs the filtering and forwarding functions in the event manager's process and the latter does it in each of the group's processes. This makes it look like the process groups would make it easier to scale (you can forward the message at once and then the evaluation takes place wherever it needs to be), although it would have a higher amount of messages being sent around because no filtering could take place beforehand. So here's to launch the discussion. What would you think would be the best choice in this case? and otherwise, what other options do you see? What are the pros and the cons of each? Both options seem valid, but I don't remember seeing them being discussed too much. From rvirding@REDACTED Mon Apr 19 01:35:04 2010 From: rvirding@REDACTED (Robert Virding) Date: Mon, 19 Apr 2010 01:35:04 +0200 Subject: [erlang-questions] link In-Reply-To: References: Message-ID: >From the way you have described it, it seems like all the processes have been explicitly linked to the root process. This does not automatically mean that they are linked to each other, they are only linked to the root process. If one of them crashes then the exit signal goes to the root process. If the root process is trapping exits and does not itself crash then the exist signal will not be transmitted to the other processes which will then continue. Is this what you meant? Robert On 14 April 2010 19:17, John Erickson wrote: > Hello, in this case I don't think there is any normal termination at all, > just a network delay that causes the other nodes to think this node is down. > ?Is there any way for the other nodes to die in this case? > > On Mon, Apr 12, 2010 at 10:50 PM, Rapsey wrote: > >> If a process dies with exit signal normal (like if the process function >> stops recursion) the linked processes will not die. If there is any other >> exit signal, linked processes will die as well. You can call exit(stop) or >> something when you want to kill all processes as well. >> >> >> Sergej >> >> On Tue, Apr 13, 2010 at 12:10 AM, John Erickson wrote: >> >> > Hello, I have created a bunch of processes and called link() on them from >> a >> > root process. ?I thought this would mean that when one of them died, they >> > would all die. ?However, I see in my log file one of the processes is >> 'not >> > responding' but the others run along oblivious. ?Is there a way to have >> > them >> > all die when this happens? >> > >> > From ok@REDACTED Mon Apr 19 05:20:37 2010 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 19 Apr 2010 15:20:37 +1200 Subject: [erlang-questions] Re: Parameterized module idioms In-Reply-To: <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> Message-ID: <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> On Apr 18, 2010, at 9:28 AM, Jayson Vantuyl wrote: >> For example, [unless you're doing some crazy rewriting tricks] it's >> obvious which function this is calling: >> >> some_module:echo(E, "hello") >> >> This isn't so obvious: >> >> E:echo("hello") >> >> Here you have to trace back to the definition of E to find out what >> module it refers to. > Actually, that's the point. There are two good reasons to do this. > > 1. Hide massive amounts of information. We can already do this with plain old closures. Instead of E:echo("hello") we can do E(echo, "hello") To a first approximation, any module with parameters can be written as a function that returns a closure (the standard OO-in-FP trick). > > 2. Polymorphism Also trivially doable with plain old closures. From bob@REDACTED Mon Apr 19 05:43:53 2010 From: bob@REDACTED (Bob Ippolito) Date: Sun, 18 Apr 2010 20:43:53 -0700 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> Message-ID: On Sunday, April 18, 2010, Richard O'Keefe wrote: > > On Apr 18, 2010, at 9:28 AM, Jayson Vantuyl wrote: > > > For example, [unless you're doing some crazy rewriting tricks] it's > obvious which function this is calling: > > ?some_module:echo(E, "hello") > > This isn't so obvious: > > ?E:echo("hello") > > Here you have to trace back to the definition of E to find out what > module it refers to. > > Actually, that's the point. ?There are two good reasons to do this. > > 1. ?Hide massive amounts of information. > > > We can already do this with plain old closures. > Instead of > ? ? ? ?E:echo("hello") > we can do > ? ? ? ?E(echo, "hello") > > To a first approximation, any module with parameters can be written > as a function that returns a closure (the standard OO-in-FP trick). > > > 2. ?Polymorphism > > > Also trivially doable with plain old closures. Trivial, but requires a lot of boilerplate code and certainly isn't any easier to understand or debug than parameterized modules. It also becomes impossible to write a useful type spec if you use closures like that. -bob From jesper.pettersson@REDACTED Mon Apr 19 08:42:07 2010 From: jesper.pettersson@REDACTED (Jesper Pettersson) Date: Mon, 19 Apr 2010 08:42:07 +0200 Subject: shell-questions Message-ID: I have two questions about the erlang shell: 1) Is there any way to make the history persistent between sessions? 2) Is there any way to enable emacs-style backwards search in the command history using CTRL-R? Thanks in advance, Jesper Pettersson Klarna AB From fabian.bergstrom@REDACTED Mon Apr 19 10:38:39 2010 From: fabian.bergstrom@REDACTED (=?ISO-8859-1?Q?Fabian_Bergstr=F6m?=) Date: Mon, 19 Apr 2010 01:38:39 -0700 (PDT) Subject: Cross compilation and testing Message-ID: Hello I'm building Erlang for ARMv7, and otp_build makes it really easy, but I have a few issues. I want to compress all beam files, so I have a script that removes the prebuilt beamfiles between "otp_build boot" and "otp_build release" and edits the target otp.mk like this: sed -i 's/\+debug_info/\+compressed/' < make/arm-angstrom-linux- gnueabi/otp.mk This is horribly non-futureproof, but I was hoping to find a better way than having to patch otp.mk. Is there a less intrusive way for configuring the erlc options when building OTP? I would also like to test that the emulator works correctly when I apply -Os during compilation of it. I ran otp_build tests and copied the tests over to the host I compiled Elrang for but ts:install() doesn't seem to like it, it cannot find a suitable compiler and a quick look at the code seems to indicate it just looks for win32 or unix as the target name to decide how to install. Is there a way I can do a staged install of the test server? -- Fabian Bergstr?m From mazen.harake@REDACTED Mon Apr 19 11:41:34 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 19 Apr 2010 11:41:34 +0200 Subject: [erlang-questions] A discussion in application design and event propagation In-Reply-To: References: Message-ID: <4BCC254E.4000607@erlang-solutions.com> I try to think of even_handlers as something you dynamically hook in to and which is a design choice by the hook-ie (so to speak :), I didn't want to end that word with -er), not part of a static design. E.g. if you have a black jack table then the processes that are created (read: the players that have joined the table) will always receive events to the table and they will never ignore the events of the table (read: unregister/re-register) then there is no point in the dynamic nature of event_managers/handlers (and thus the overhead, complexity etc). Using this reasoning I would vote for "The second way". But as you said they are essentially the same thing and either way I reckon that the data transform and manipulation would be done at the Table level either way (before it is sent to the clients). My 2 cents :) /Mazen On 18/04/2010 22:50, Fred Hebert wrote: > I'm starting this thread to spark a discussion about Erlang software design. > I've been writing a few applications here and there and I've noticed a few > common patterns occurring over time, but with not clear solution to them. > That seems to be due to the general "there is more than one way to do it" > attitude that exists when organizing processes, and not because of a lack > of activism. > > Anyway, the design question comes from designs having to do with multi-user > activities, might it be chat rooms, games of cards over the web, voting > system, a pub-sub mechanism, etc. The general idea is that you have a group > of users and a sequence of events that has to be forwarded to every user in > real time. To make this clearer, I'll take the example of a blackjack table > where every player has a GUI showing the state of the game. > > You'd have (I imagine) one process representing the dealer and the table > (D). Then you have 3 players, P1, P2 and P3, each represented as a process. > A hand distribution goes like this: > > > 1. Player 1 registers to the table > 2. Player 2 registers. Player 1 is notified of this. > 3. Player 3 registers, P1 and P2 are notified. > 4. The game starts, All players are notified of this. > 5. Each player (and the dealer) is handed a card > 6. Players bet. All players are notified of this; > 7. ... > 8. etc. > > Then you could also have some messages private (imagine having a private > hand as in a poker game or whatever). > > There are two obvious ways I can think of to model this. The first one is as > a gen_event process and handlers, where all players register an event > handler. When an event occurs, either the dealer or the players notify the > event manager which then forwards the event to subscribers through their > respective event handlers. The second way I can think of are process groups, > where a message sent to the leader is automatically forwarded to all > processes who are members of the group. > > It seems to me these are equivalent solutions, as they allow for the same > propagation of events in correct order. However, the former runs the > filtering and forwarding functions in the event manager's process and the > latter does it in each of the group's processes. This makes it look like the > process groups would make it easier to scale (you can forward the message at > once and then the evaluation takes place wherever it needs to be), although > it would have a higher amount of messages being sent around because no > filtering could take place beforehand. > > So here's to launch the discussion. What would you think would be the best > choice in this case? and otherwise, what other options do you see? What are > the pros and the cons of each? Both options seem valid, but I don't remember > seeing them being discussed too much. > > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From erlang@REDACTED Mon Apr 19 11:48:08 2010 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 19 Apr 2010 11:48:08 +0200 Subject: erlang on vista Message-ID: I'm trying to run Erlang on Windows Vista. Having installed Erlang, when I try to run it I get the following error message: C:\Program Files\erl5.7.5\erts-5.7.5\bin\werl.exe The application has failed to start because its side-by-side configuration is incorrect. Please see the application event log for more deatils. My knowledge of computer science does not extend to "side-by-side" configurations - has anybody got a magic spell that I can use to make this error message go away. /Joe From mazen.harake@REDACTED Mon Apr 19 11:51:49 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 19 Apr 2010 11:51:49 +0200 Subject: [erlang-questions] erlang on vista In-Reply-To: References: Message-ID: <4BCC27B5.6060808@erlang-solutions.com> Did you try to turn it off and on again? /Mazen On 19/04/2010 11:48, Joe Armstrong wrote: > I'm trying to run Erlang on Windows Vista. > > Having installed Erlang, when I try to run it I get the following error message: > > C:\Program Files\erl5.7.5\erts-5.7.5\bin\werl.exe > > The application has failed to start because its side-by-side > configuration is incorrect. Please see the > application event log for more deatils. > > My knowledge of computer science does not extend to "side-by-side" > configurations - has anybody > got a magic spell that I can use to make this error message go away. > > /Joe > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From antoine.koener@REDACTED Mon Apr 19 12:30:30 2010 From: antoine.koener@REDACTED (Antoine Koener) Date: Mon, 19 Apr 2010 12:30:30 +0200 Subject: [erlang-questions] erlang on vista In-Reply-To: <4BCC27B5.6060808@erlang-solutions.com> References: <4BCC27B5.6060808@erlang-solutions.com> Message-ID: On Mon, Apr 19, 2010 at 11:51 AM, Mazen Harake < mazen.harake@REDACTED> wrote: > Did you try to turn it off and on again? > > /Mazen > > > On 19/04/2010 11:48, Joe Armstrong wrote: > >> I'm trying to run Erlang on Windows Vista. >> >> Having installed Erlang, when I try to run it I get the following error >> message: >> >> C:\Program Files\erl5.7.5\erts-5.7.5\bin\werl.exe >> >> The application has failed to start because its side-by-side >> configuration is incorrect. Please see the >> application event log for more deatils. >> > This is related to the 'Manifest'. Windows uses Manifest (XML Files) that contains various informations about which version of DLLs is needed to run the program. This Manifest is at the end of the werl.exe binary, you may open werl.exe with some text editor and extract this part. Then you'll need to find the good dll version. I think that your vista installation doesn't have the valid VC++Runtime Redistributable. "vcredist" So You may download vcredist_x86.exe from the MS website, search for 'vcredist 2008' in any search engine. Note that there are multiples version of this file, so try the normal version (published in 2007) then try the SP1 version... (published in 2008) Good Luck. This problem is a major headache when you need to redistribute Erlang on windows... > >> My knowledge of computer science does not extend to "side-by-side" >> configurations - has anybody >> got a magic spell that I can use to make this error message go away. >> >> /Joe >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> > > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become > ERLANG SOLUTIONS LTD. > > www.erlang-solutions.com > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From martindemello@REDACTED Mon Apr 19 12:45:12 2010 From: martindemello@REDACTED (Martin DeMello) Date: Mon, 19 Apr 2010 16:15:12 +0530 Subject: testing asynchronous code Message-ID: Anyone have general strategies or advice for testing code that makes heavy use of gen_server:cast? The best I could come up with was to write _sync versions of all my functions, with gen_server:cast replaced by gen_server:call, and then test those, but that seems both clunky and fragile. martin From roberto@REDACTED Mon Apr 19 12:46:51 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 19 Apr 2010 12:46:51 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: 2010/4/12 Ingela Andin : > Hi! > > Tonys code will accomplish that you run the new ssl-implementation > instead of the old one. I would not run the new ssl in older > versions than R13 even though there is a primitive version in R12. > The main reason for not making the new one default yet is that we > want it to be as complete as possible before that, but that does not > necessarily mean it is not complete enough for your need. > > Regards Ingela Erlang/OTP team, Ericsson AB > hello ingela, still having issues i can't understand, even with new_ssl. see the code here below: %%%%%%%%%%%%%%%%%%% -module(sslbug). -compile(export_all). start(Port) -> application:start(ssl), application:start(crypto), Options = [ binary, {packet, http}, {reuseaddr, true}, {active, false}, {backlog, 128}, {ssl_imp, new}, {certfile, "/Users/roberto/Code/sslbug/server.pem"}, {keyfile, "/Users/roberto/Code/sslbug/privkey.pem"}, {password, "roberto"} ], {ok, ListenSocket} = ssl:listen(Port, Options), {ok, Sock} = ssl:transport_accept(ListenSocket), ok = ssl:ssl_accept(Sock), ControllerPid = spawn(fun() -> receive set -> ok end, controller_loop(Sock, 0) end), ok = ssl:controlling_process(Sock, ControllerPid), ControllerPid ! set. controller_loop(_Sock, 2) -> ssl:close(Sock); controller_loop(Sock, Num) -> ssl:setopts(Sock, [{active, once}]), receive Any -> io:format("ok, received ~p~n", [Any]), controller_loop(Sock, Num + 1) end. %%%%%%%%%%%%%%%%%%% when i run this code with sslbug:start(8080), open a browser and point it to https://localhost:8080, this is what gets written: ok, received {ssl,{sslsocket,new_ssl,<0.46.0>}, {http_request,'GET',{abs_path,"/"},{1,1}}} ok, received {ssl,{sslsocket,new_ssl,<0.46.0>}, {http_error,"Host: localhost:8080\r\n"}} as second message i would expect a {ssl,{sslsocket,new_ssl,<0.46.0>}, {http_header,"Host: localhost:8080\r\n"}} message, i.e. a 'header', not a 'error' message. something i'm missing here? thank you, r. From paul.a.johnston@REDACTED Mon Apr 19 12:49:40 2010 From: paul.a.johnston@REDACTED (Paul Johnston) Date: Mon, 19 Apr 2010 11:49:40 +0100 Subject: [erlang-questions] erlang on vista In-Reply-To: References: Message-ID: Not terribly useful I know but V5.7.4 works on Windows 7. One way to get rid of Vista :-) Paul -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Joe Armstrong Sent: 19 April 2010 10:48 To: Erlang Subject: [erlang-questions] erlang on vista I'm trying to run Erlang on Windows Vista. Having installed Erlang, when I try to run it I get the following error message: C:\Program Files\erl5.7.5\erts-5.7.5\bin\werl.exe The application has failed to start because its side-by-side configuration is incorrect. Please see the application event log for more deatils. My knowledge of computer science does not extend to "side-by-side" configurations - has anybody got a magic spell that I can use to make this error message go away. /Joe ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From erlang@REDACTED Mon Apr 19 12:53:38 2010 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 19 Apr 2010 12:53:38 +0200 Subject: [erlang-questions] erlang on vista In-Reply-To: <4BCC27B5.6060808@erlang-solutions.com> References: <4BCC27B5.6060808@erlang-solutions.com> Message-ID: This half worked - after a reboot (which takes a long time, minutes would you believe) Erlang half starts - but running werl.exe gives this error message: (no error logger present) error: "Error in process <0.2.0> with exit value: {badarg,[{erl_prim_loader,check_file_result,3},{erl_prim_loader,check_file_result,3},{init,get_boot,1},{init,get_boot,2},{init,do_boot,3}]}\n" {"init terminating in do_boot",{badarg,[{erl_prim_loader,check_file_result,3},{erl_prim_loader,check_file_result,3},{init,get_boot,1},{init,get_boot,2},{init,do_boot,3}]}} Crash dump was written to: erl_crash.dump init terminating in do_boot () Abnormal termination /Joe On Mon, Apr 19, 2010 at 11:51 AM, Mazen Harake wrote: > Did you try to turn it off and on again? > > /Mazen > > On 19/04/2010 11:48, Joe Armstrong wrote: >> >> I'm trying to run Erlang on Windows Vista. >> >> Having installed Erlang, when I try to run it I get the following error >> message: >> >> ? ? C:\Program Files\erl5.7.5\erts-5.7.5\bin\werl.exe >> >> ? ? The application has failed to start because its side-by-side >> configuration is incorrect. Please see the >> ? ? application event log for more deatils. >> >> ?My knowledge of computer science does not extend to "side-by-side" >> configurations - has anybody >> got a magic spell that I can use to make this error message go away. >> >> /Joe >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG > SOLUTIONS LTD. > > www.erlang-solutions.com > > From kenneth.lundin@REDACTED Mon Apr 19 12:57:22 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Mon, 19 Apr 2010 12:57:22 +0200 Subject: [erlang-questions] erlang on vista In-Reply-To: References: Message-ID: Erlang works very well on Vista, but perhaps there is some issue with the VC++ runtime as pointed out in a previous mail. We are however provided a version of the VC++ runtime in the installer for Windows. /Kenneth Erlang/OTP Ericsson On Mon, Apr 19, 2010 at 12:49 PM, Paul Johnston wrote: > Not terribly useful I know but V5.7.4 works on Windows 7. > One way to get rid of Vista :-) > > Paul > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] > On Behalf Of Joe Armstrong > Sent: 19 April 2010 10:48 > To: Erlang > Subject: [erlang-questions] erlang on vista > > I'm trying to run Erlang on Windows Vista. > > Having installed Erlang, when I try to run it I get the following error > message: > > ? ?C:\Program Files\erl5.7.5\erts-5.7.5\bin\werl.exe > > ? ?The application has failed to start because its side-by-side > configuration is incorrect. Please see the > ? ?application event log for more deatils. > > ?My knowledge of computer science does not extend to "side-by-side" > configurations - has anybody > got a magic spell that I can use to make this error message go away. > > /Joe > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From roberto@REDACTED Mon Apr 19 13:22:17 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 19 Apr 2010 13:22:17 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: this is the sequence of the first messages received by the controlling process in a SSL socket [new implementation] when {active, once} is used, and a browser performs a request: {ssl,{sslsocket,new_ssl,<0.51.0>}, {http_request,'GET',{abs_path,"/"},{1,1}}} {ssl,{sslsocket,new_ssl,<0.51.0>}, {http_error,"Host: localhost:8080\r\n"}} {ssl,{sslsocket,new_ssl,<0.51.0>}, {http_error,"User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3\r\n"}} {ssl,{sslsocket,new_ssl,<0.51.0>}, {http_error,"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n"}} ... therefore it seems that all incoming headers are actually treated as 'http_error' instead of 'http_header'. is this intentional, something i am unaware of re the ssl modality, a bug, a mis-configuration, ...? thank you in advance for any help. r. From rickard.cardell@REDACTED Mon Apr 19 13:36:52 2010 From: rickard.cardell@REDACTED (Rickard Cardell) Date: Mon, 19 Apr 2010 13:36:52 +0200 Subject: [erlang-questions] Read from input without echo In-Reply-To: References: Message-ID: Hello I have limited knowledge of escript but have you tried io:get_password/0 ? //Rickard 2010/4/16 Damu R > Hi, > I am trying to read password from the terminal input.I am unable to find a > way to read input from the terminal without echoing. I am able to do it > from > the erlang shell, but it is not happening in a standalone script (escript). > Looks like the erlang shell uses a different group leader process for i/o. > Is there a way to read password from a escript? > > Thank you > Damu > From mail@REDACTED Mon Apr 19 13:56:35 2010 From: mail@REDACTED (Tim Fletcher) Date: Mon, 19 Apr 2010 04:56:35 -0700 (PDT) Subject: testing asynchronous code In-Reply-To: References: Message-ID: <07971735-2e27-4935-a546-4919c2150749@u21g2000yqc.googlegroups.com> > Anyone have general strategies or advice for testing code that makes > heavy use of gen_server:cast? The best I could come up with was to > write _sync versions of all my functions, with gen_server:cast > replaced by gen_server:call, and then test those, but that seems both > clunky and fragile. How about using a "test spy" process to collect a trace of the cast messages and then make assertions against the resulting trace? Tim From gordon@REDACTED Mon Apr 19 14:03:13 2010 From: gordon@REDACTED (Gordon Guthrie) Date: Mon, 19 Apr 2010 13:03:13 +0100 Subject: Regular Expressions Problems Message-ID: Folks I think I may have identified a regular expression bug in re. The following code never terminates in R13B-04: -module(fail). -export([fail/0]). fail() -> Str = "http:/www.flickr.com/slideShow/index.gne?group_id=&user_id=69845378@REDACTED", EMail_regex = "[a-z0-9!#$%&'*+/=?^_`{|}~-]+" ++ "(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*" ++ "@(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+" ++ "(?:[a-zA-Z]{2}|com|org|net|gov|mil" ++ "|biz|info|mobi|name|aero|jobs|museum)", io:format("about to run...~n"), Ret = re:run(Str, EMail_regex), io:format("Ret is ~p~n", [Ret]). Eliminating the @ in either the string or the regex and it will terminate - but if you don't it wont... There is a comment about the behaviour of '@' in Perl regular expressions in the docos: > If you want to remove the special meaning from a sequence of characters, you can do so by putting them between \Q and \E. > This is different from Perl in that $ and @ are handled as literals in \Q...\E sequences in PCRE, whereas in Perl, $ and @ cause variable interpolation.' Jeremy Zawinski's famous comment springs to mind: > Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems. Gordon From roberto@REDACTED Mon Apr 19 15:03:52 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 19 Apr 2010 15:03:52 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: <19BD41F6-E388-4D73-9D5C-40350A4CE02F@rogvall.se> References: <19BD41F6-E388-4D73-9D5C-40350A4CE02F@rogvall.se> Message-ID: 2010/4/19 Tony Rogvall : > You must switch to http header packet processing ! > Use {packet, httph} or {packet,httph_bin} after the initial request. > End of headers is signaled by http_eoh. > After http_eoh you must switch to content processing {packet, 0} and then read possible content. > The size of the content is givent by the header Content-Length or if chunked encoding is used > the size given before each chunk. > > Hope this helps > > /Tony > hi tony, well i am starting the server with the options: Options = [ binary, {packet, http}, {reuseaddr, true}, {active, false}, {backlog, 128}, {ssl_imp, new}, {certfile, "/Users/roberto/Code/sslbug/server.pem"}, {keyfile, "/Users/roberto/Code/sslbug/privkey.pem"}, {password, "roberto"} ], which does include {packet, http}. is this not enough? i normally switch to {packet, 0} only when i go passive to read body of requests, after eoh. r. From tony@REDACTED Mon Apr 19 15:23:24 2010 From: tony@REDACTED (Tony Rogvall) Date: Mon, 19 Apr 2010 15:23:24 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: <19BD41F6-E388-4D73-9D5C-40350A4CE02F@rogvall.se> Message-ID: <8D18002A-6ADA-4122-983B-C1C636C700A6@rogvall.se> The packet type for http headers is 'httph' (not the 'h' appended) So you need to switch packet type after you received the request in 'http' mode /Tony On 19 apr 2010, at 15.03, Roberto Ostinelli wrote: > 2010/4/19 Tony Rogvall : >> You must switch to http header packet processing ! >> Use {packet, httph} or {packet,httph_bin} after the initial request. >> End of headers is signaled by http_eoh. >> After http_eoh you must switch to content processing {packet, 0} and then read possible content. >> The size of the content is givent by the header Content-Length or if chunked encoding is used >> the size given before each chunk. >> >> Hope this helps >> >> /Tony >> > > hi tony, > > well i am starting the server with the options: > > Options = [ > binary, {packet, http}, {reuseaddr, true}, {active, false}, > {backlog, 128}, {ssl_imp, new}, > {certfile, "/Users/roberto/Code/sslbug/server.pem"}, > {keyfile, "/Users/roberto/Code/sslbug/privkey.pem"}, > {password, "roberto"} > ], > > which does include {packet, http}. is this not enough? i normally > switch to {packet, 0} only when i go passive to read body of requests, > after eoh. > > r. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ingela@REDACTED Mon Apr 19 15:34:47 2010 From: ingela@REDACTED (Ingela Andin) Date: Mon, 19 Apr 2010 15:34:47 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: Hi! We recently accepted a patch for http packet mode e.i: http://github.com/erlang/otp/commit/12da3796463c7d02999985ae15e98dcaa4d2bdb4 Do you have that patch? The offical ssl-release ssl-3.11 where this patch has been included is not yet released other then to git-repository. Regards Ingela Erlang/OTP team, Ericsson AB 2010/4/19 Roberto Ostinelli : > this is the sequence of the first messages received by the controlling > process in a SSL socket [new implementation] when {active, once} is > used, and a browser performs a request: > > {ssl,{sslsocket,new_ssl,<0.51.0>}, > ? ? ? ?{http_request,'GET',{abs_path,"/"},{1,1}}} > {ssl,{sslsocket,new_ssl,<0.51.0>}, > ? ? ? ?{http_error,"Host: localhost:8080\r\n"}} > {ssl,{sslsocket,new_ssl,<0.51.0>}, > ? ? ? ?{http_error,"User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X > 10.6; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3\r\n"}} > {ssl,{sslsocket,new_ssl,<0.51.0>}, > ? ? ? ?{http_error,"Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n"}} > ... > > therefore it seems that all incoming headers are actually treated as > 'http_error' instead of 'http_header'. > > is this intentional, something i am unaware of re the ssl modality, a > bug, a mis-configuration, ...? > > thank you in advance for any help. > > r. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tony@REDACTED Mon Apr 19 14:52:53 2010 From: tony@REDACTED (Tony Rogvall) Date: Mon, 19 Apr 2010 14:52:53 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: <19BD41F6-E388-4D73-9D5C-40350A4CE02F@rogvall.se> You must switch to http header packet processing ! Use {packet, httph} or {packet,httph_bin} after the initial request. End of headers is signaled by http_eoh. After http_eoh you must switch to content processing {packet, 0} and then read possible content. The size of the content is givent by the header Content-Length or if chunked encoding is used the size given before each chunk. Hope this helps /Tony On 19 apr 2010, at 13.22, Roberto Ostinelli wrote: > this is the sequence of the first messages received by the controlling > process in a SSL socket [new implementation] when {active, once} is > used, and a browser performs a request: > > {ssl,{sslsocket,new_ssl,<0.51.0>}, > {http_request,'GET',{abs_path,"/"},{1,1}}} > {ssl,{sslsocket,new_ssl,<0.51.0>}, > {http_error,"Host: localhost:8080\r\n"}} > {ssl,{sslsocket,new_ssl,<0.51.0>}, > {http_error,"User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X > 10.6; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3\r\n"}} > {ssl,{sslsocket,new_ssl,<0.51.0>}, > {http_error,"Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n"}} > ... > > therefore it seems that all incoming headers are actually treated as > 'http_error' instead of 'http_header'. > > is this intentional, something i am unaware of re the ssl modality, a > bug, a mis-configuration, ...? > > thank you in advance for any help. > > r. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From zabrane3@REDACTED Mon Apr 19 16:02:41 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Mon, 19 Apr 2010 16:02:41 +0200 Subject: Convert an OtpErlangBinary object to Java InputStream Message-ID: Hi List, Does someone knows how to build a Java InputStream object from an OtpErlangBinary object? I tried this, but it doesn't even compile: ... OtpErlangBinary bin = (OtpErlangBinary)(msg.elementAt(0)); InputStream is = (InputStream) bin; Regards Zabrane From vladdu55@REDACTED Mon Apr 19 16:13:34 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 19 Apr 2010 16:13:34 +0200 Subject: [erlang-questions] Convert an OtpErlangBinary object to Java InputStream In-Reply-To: References: Message-ID: Hi! On Mon, Apr 19, 2010 at 16:02, zabrane Mikael wrote: > Does someone knows how to build a Java InputStream object > from an OtpErlangBinary object? Try something like OtpErlangBinary bin ? ?= (OtpErlangBinary)(msg.elementAt(0)); InputStream is = new ByteArrayInputStream(bin.binaryValue()); regards, Vlad From dmercer@REDACTED Mon Apr 19 16:51:08 2010 From: dmercer@REDACTED (David Mercer) Date: Mon, 19 Apr 2010 09:51:08 -0500 Subject: [erlang-questions] testing asynchronous code In-Reply-To: References: Message-ID: <8A5415EEB6C64512BCBB2A6A922AD9EE@SSI.CORP> On Monday, April 19, 2010, Martin DeMello wrote: > Anyone have general strategies or advice for testing code that makes > heavy use of gen_server:cast? The best I could come up with was to > write _sync versions of all my functions, with gen_server:cast > replaced by gen_server:call, and then test those, but that seems both > clunky and fragile. I assume your issue is that you cannot test system state right after a cast because its processing is asynchronous? Now you could, say, wait a "reasonable" amount of time to allow the cast to be processed and then perform your assertions. Alternatively, you might could[1] add a "sync" call into the server, which does nothing, but you know when it returns that all prior casts have been processed, and then you can make your asserts on the server or system state, etc. (If the casts are passing processing off to other processes, the sync call would have to "sync" with them before returning.) Cheers, DBM [1] I know that's not correct English, but it should be. From mevans@REDACTED Mon Apr 19 16:52:01 2010 From: mevans@REDACTED (Evans, Matthew) Date: Mon, 19 Apr 2010 10:52:01 -0400 Subject: [erlang-questions] shell-questions In-Reply-To: References: Message-ID: Someone posted this a few weeks ago: > For anyone who hasn't found it, there's a great little utility (on > Linux, at least) called 'rlwrap' which gives you the full readline > feature set on any command line system (like, say, the erlang shell). > I simply have 'erl' aliased to 'rlwrap -a erl'. It gives you > everything provided by this patch as well as as a searchable history > (just like you get in bash with Ctrl-r). -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Jesper Pettersson Sent: Monday, April 19, 2010 2:42 AM To: erlang-questions@REDACTED Subject: [erlang-questions] shell-questions I have two questions about the erlang shell: 1) Is there any way to make the history persistent between sessions? 2) Is there any way to enable emacs-style backwards search in the command history using CTRL-R? Thanks in advance, Jesper Pettersson Klarna AB From zabrane3@REDACTED Mon Apr 19 17:14:56 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Mon, 19 Apr 2010 17:14:56 +0200 Subject: [erlang-questions] Convert an OtpErlangBinary object to Java InputStream In-Reply-To: References: Message-ID: Thanks Vlad. It works !!! Now, how about to convert an UTF8 String to OtpErlangBinary() ? 2010/4/19 Vlad Dumitrescu > Hi! > > On Mon, Apr 19, 2010 at 16:02, zabrane Mikael wrote: > > Does someone knows how to build a Java InputStream object > > from an OtpErlangBinary object? > > Try something like > > OtpErlangBinary bin = (OtpErlangBinary)(msg.elementAt(0)); > InputStream is = new ByteArrayInputStream(bin.binaryValue()); > > regards, > Vlad > From jay@REDACTED Mon Apr 19 17:58:59 2010 From: jay@REDACTED (Jay Nelson) Date: Mon, 19 Apr 2010 08:58:59 -0700 Subject: A discussion in application design and event propagation Message-ID: <9204A7C1-FCEA-4C0C-B390-62E3C9C3A818@duomark.com> Fred Hebert wrote: > I'm starting this thread to spark a discussion about Erlang software design. Just yesterday I wanted to start a similar thread on "concurrency patterns." I will do that as a separate thread. I've found that with other languages I think of "software design." With erlang, I find it more fruitful to think of "architectural design" because the language allows you to make system level choices and enforce them in the software structure. Limiting your thinking to "software patterns" and "software design" may not get you thinking about the right elements of your problem. It's just that other languages have no other elements to the problem than the software itself, so our tendency is to revert to what's available. > You'd have (I imagine) one process representing the dealer and the table > (D). Then you have 3 players, P1, P2 and P3, each represented as a process. That's a good start using software as the principle. Instead let's look at the architectural issues: 1) Where are users coming from? - I'll assume Internet connected. 2) What to do if a connection is lost? - Most game tables don't allow you to bail when you are losing. - A 'bot' finishes the game for you since money is on the line. 3) How many connections are you expecting? - One game table is easy inside erlang distribution. - Many tables is too much inter-node chatter. Using distributed erlang you get node-to-node pings to detect node failures. This can lead to too much data traffic. If many users are expected, you probably want TCP connected users rather than erlang node connected users. Distributed erlang (using the OTP distribution techniques) tends to require predetermined node names and process configurations, as opposed to Distributed TCP which leads to dynamic connectivity without enumerating participants in configuration files. pg also uses global names to manage the groups. This introduces potential contention for the global name (no global resource goes unpunished in concurrent architectures -- Patrick Nyblom SF 2010). If you consider the architecture, you are missing some elements in your sketch. You should include them before you evaluate the design choices: 1) User initiates a connection from a browser (assumption on my part) 2) TCP connects the user to some erlang server process 3) User needs to be represented by a proxy in case of connection loss 4) Current game should be able to finish if a user disconnects 5) New game may or may not be allowed to start if user missing New requirements: 6) Can non-players observe the game? 7) Can non-players interact with game players (chat with them)? I can see an approach that there is a gallery and the players at the table. The gallery can chat amongst themselves, and the players can interact amongst themselves. The gallery observes all, the players only see the table (for example). Architecturally, I would determine the principles that define a good solution before choosing a software pattern: 1) How many users can be supported? 2) What happens in the common failure scenarios? 3) How do you enforce the game rules (audience vs players, etc)? 4) What kind of responsiveness is needed? 5) How much hardware are you willing to configure? 6) How dynamic is your configuration and user participation? For this problem gen_event probably has the most opportunities for you as the central design element. It allows multiple groups to observe events with differing functionality for the groups. You could use pg for the players and for the gallery, and gen_event to distribute the messages. But you are missing the connecting server and the proxying user process that automatically represents (and governs) the users actions. With erlang, don't just look at software patterns, step back and look at architectural patterns. You will find yourself architecting loosely coupled systems, with layers of connectivity, routing, application constraints and application logic. Each of these elements can use different software patterns, but the way they interact should be flexible, dynamic and auto-healing. Also remember, there is no "good" or "bad" solution. Each solution has certain characteristics it exhibits such as speed, resilience, capacity, etc. One solution may be better for one situation and worse for another, based on the measurement that matters in each case. Don't be biased against an approach that was rejected in a previous problem application -- it may turn out useful in a different situation. jay From arjan@REDACTED Mon Apr 19 18:10:10 2010 From: arjan@REDACTED (Arjan Scherpenisse) Date: Mon, 19 Apr 2010 18:10:10 +0200 Subject: Announcing Zotonic 0.4.0 Message-ID: <4BCC8062.7040509@scherpenisse.net> Dear Erlang and Zotonic enthusiasts, We're pleased to announced the next version of Zotonic, release 0.4.0. After 3 months of new features and fixing bugs, the time was there to create a new "stable" release. As usual, download the source code from google code: http://code.google.com/p/zotonic/downloads/list More information about Zotonic on its website: http://zotonic.com Have fun! The Zotonic team. Release 0.4.0, released on 2010-04-19 ------------------------------------- * New modules: ** mod_pubsub Enables resource sharing over XMPP's PubSub; share content between sites and get realtime updates when content changes. See: http://scherpenisse.net/id/644 ** mod_search_solr Added a module which plugs into Zotonic's search system to support Solr (http://lucene.apache.org/solr/). Using Solr enables quick fulltext searching and facetting. * New features: ** Default site improvements The default site of a vanilla Zotonic install has been improved with nicer graphics, cleaner typography, a "contact form" example and styles for the Twitter module. ** "More results" scomp A twitter/facebook style ajaxified "read more" pager, which is a button which will fetch more results for the current search question inline on the same page. ** Windows support Initial support for building and running Zotonic on the Windows platform. ** Database schema support Multiple sites running inside one Postgres database is now possible thanks to Postgres' support for multiple table namespaces (schema's) ** Template expressions It is now possible to use full boolean and arithmetic expressions in the ErlyDTL templates. ** Webserver IPv6 support ** Yandex.Video support in mod_video_embed module (#52) ** PID-file for zotonic (#74) ** Support for HTML5 audio/video tags in TinyMCE editor (#75) ** Newer TinyMCE 3.3.2 release from upstream (#69) ** Newer Mochiweb r153 release from upstream * Bugfixes: ** page_path controller should not redirect to the default_page_url (#6) ** Get the name of the current dispatch rule (#21) ** zotonic fails after postgresql restart (#49) ** Unreliable pivot? (#50) ** Module manager should feedback when module cannot be started. (#51) ** Do not depend on the 'default' site (#59) ** i18n of scomp_pager (#62) ** Buttons and "Logoff" link problems in Chrome (#63) ** Comment form breaks on new default site (#64) ** Getting an unknown_rsc error on startup (#66) ** Zotonic fails to (re)start if an existing admin panel is open with browser supporting WebSockets (#70) ** Can't save location without e-mail (#71) ** Improve the default styles to include list bullets/numbers (#72) ** Twitter module cannot be enabled (#76) From g@REDACTED Mon Apr 19 18:14:23 2010 From: g@REDACTED (Garrett Smith) Date: Mon, 19 Apr 2010 11:14:23 -0500 Subject: [erlang-questions] Re: A discussion in application design and event propagation In-Reply-To: <9204A7C1-FCEA-4C0C-B390-62E3C9C3A818@duomark.com> References: <9204A7C1-FCEA-4C0C-B390-62E3C9C3A818@duomark.com> Message-ID: On Mon, Apr 19, 2010 at 10:58 AM, Jay Nelson wrote: > Fred Hebert wrote: > >> I'm starting this thread to spark a discussion about Erlang software >> design. > > Just yesterday I wanted to start a similar thread on "concurrency patterns." > ?I will do that as a separate thread. > > I've found that with other languages I think of "software design." ?With > erlang, I find it more fruitful to think of "architectural design" because > the language allows you to make system level choices and enforce them in the > software structure. ?Limiting your thinking to "software patterns" and > "software design" may not get you thinking about the right elements of your > problem. ?It's just that other languages have no other elements to the > problem than the software itself, so our tendency is to revert to what's > available. > > >> You'd have (I imagine) one process representing the dealer and the table >> (D). Then you have 3 players, P1, P2 and P3, each represented as a >> process. > > That's a good start using software as the principle. ?Instead let's look at > the architectural issues: > > 1) Where are users coming from? > ? ? ? ?- I'll assume Internet connected. > 2) What to do if a connection is lost? > ? ? ? ?- Most game tables don't allow you to bail when you are losing. > ? ? ? ?- A 'bot' finishes the game for you since money is on the line. > 3) How many connections are you expecting? > ? ? ? ?- One game table is easy inside erlang distribution. > ? ? ? ?- Many tables is too much inter-node chatter. > > > Using distributed erlang you get node-to-node pings to detect node failures. > ?This can lead to too much data traffic. ?If many users are expected, you > probably want TCP connected users rather than erlang node connected users. > Distributed erlang (using the OTP distribution techniques) tends to require > predetermined node names and process configurations, as opposed to > Distributed TCP which leads to dynamic connectivity without enumerating > participants in configuration files. > > pg also uses global names to manage the groups. ?This introduces potential > contention for the global name (no global resource goes unpunished in > concurrent architectures -- Patrick Nyblom SF 2010). > > If you consider the architecture, you are missing some elements in your > sketch. You should include them before you evaluate the design choices: > > 1) User initiates a connection from a browser (assumption on my part) > 2) TCP connects the user to some erlang server process > 3) User needs to be represented by a proxy in case of connection loss > 4) Current game should be able to finish if a user disconnects > 5) New game may or may not be allowed to start if user missing > > New requirements: > > 6) Can non-players observe the game? > 7) Can non-players interact with game players (chat with them)? > > I can see an approach that there is a gallery and the players at the table. > ?The gallery can chat amongst themselves, and the players can interact > amongst themselves. ?The gallery observes all, the players only see the > table (for example). > > > Architecturally, I would determine the principles that define a good > solution before choosing a software pattern: > > 1) How many users can be supported? > 2) What happens in the common failure scenarios? > 3) How do you enforce the game rules (audience vs players, etc)? > 4) What kind of responsiveness is needed? > 5) How much hardware are you willing to configure? > 6) How dynamic is your configuration and user participation? > > > For this problem gen_event probably has the most opportunities for you as > the central design element. ?It allows multiple groups to observe events > with differing functionality for the groups. ?You could use pg for the > players and for the gallery, and gen_event to distribute the messages. ?But > you are missing the connecting server and the proxying user process that > automatically represents (and governs) the users actions. > > With erlang, don't just look at software patterns, step back and look at > architectural patterns. ?You will find yourself architecting loosely coupled > systems, with layers of connectivity, routing, application constraints and > application logic. ?Each of these elements can use different software > patterns, but the way they interact should be flexible, dynamic and > auto-healing. > > Also remember, there is no "good" or "bad" solution. ?Each solution has > certain characteristics it exhibits such as speed, resilience, capacity, > etc. ?One solution may be better for one situation and worse for another, > based on the measurement that matters in each case. ?Don't be biased against > an approach that was rejected in a previous problem application -- it may > turn out useful in a different situation. Thinking in architectural terms, this strikes me as a pub/sub problem that a message broker would handle well. RabbitMQ (AMQP), e.g., would let you publish events to an exchange, which can then be routed to various queues via bindings. RabbitMQ in particular is a nice option for Erlang developers. Garrett From jderick@REDACTED Mon Apr 19 18:53:18 2010 From: jderick@REDACTED (John Erickson) Date: Mon, 19 Apr 2010 09:53:18 -0700 Subject: [erlang-questions] link In-Reply-To: References: Message-ID: Hello, I have not set up any traps on any process. Could there be another explanation? On Sun, Apr 18, 2010 at 4:35 PM, Robert Virding wrote: > From the way you have described it, it seems like all the processes > have been explicitly linked to the root process. This does not > automatically mean that they are linked to each other, they are only > linked to the root process. If one of them crashes then the exit > signal goes to the root process. If the root process is trapping exits > and does not itself crash then the exist signal will not be > transmitted to the other processes which will then continue. > > Is this what you meant? > > Robert > > On 14 April 2010 19:17, John Erickson wrote: > > Hello, in this case I don't think there is any normal termination at all, > > just a network delay that causes the other nodes to think this node is > down. > > Is there any way for the other nodes to die in this case? > > > > On Mon, Apr 12, 2010 at 10:50 PM, Rapsey wrote: > > > >> If a process dies with exit signal normal (like if the process function > >> stops recursion) the linked processes will not die. If there is any > other > >> exit signal, linked processes will die as well. You can call exit(stop) > or > >> something when you want to kill all processes as well. > >> > >> > >> Sergej > >> > >> On Tue, Apr 13, 2010 at 12:10 AM, John Erickson > wrote: > >> > >> > Hello, I have created a bunch of processes and called link() on them > from > >> a > >> > root process. I thought this would mean that when one of them died, > they > >> > would all die. However, I see in my log file one of the processes is > >> 'not > >> > responding' but the others run along oblivious. Is there a way to > have > >> > them > >> > all die when this happens? > >> > > >> > > > From mononcqc@REDACTED Mon Apr 19 19:21:07 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 19 Apr 2010 13:21:07 -0400 Subject: [erlang-questions] Re: A discussion in application design and event propagation In-Reply-To: <9204A7C1-FCEA-4C0C-B390-62E3C9C3A818@duomark.com> References: <9204A7C1-FCEA-4C0C-B390-62E3C9C3A818@duomark.com> Message-ID: On Mon, Apr 19, 2010 at 11:58 AM, Jay Nelson wrote: > Fred Hebert wrote: > > > I'm starting this thread to spark a discussion about Erlang software > design. > > Just yesterday I wanted to start a similar thread on "concurrency > patterns." I will do that as a separate thread. > > I've found that with other languages I think of "software design." With > erlang, I find it more fruitful to think of "architectural design" because > the language allows you to make system level choices and enforce them in the > software structure. Limiting your thinking to "software patterns" and > "software design" may not get you thinking about the right elements of your > problem. It's just that other languages have no other elements to the > problem than the software itself, so our tendency is to revert to what's > available. > > > > > You'd have (I imagine) one process representing the dealer and the table > > (D). Then you have 3 players, P1, P2 and P3, each represented as a > process. > > That's a good start using software as the principle. Instead let's look at > the architectural issues: > > 1) Where are users coming from? > - I'll assume Internet connected. > 2) What to do if a connection is lost? > - Most game tables don't allow you to bail when you are losing. > - A 'bot' finishes the game for you since money is on the line. > 3) How many connections are you expecting? > - One game table is easy inside erlang distribution. > - Many tables is too much inter-node chatter. > > Without wanting to get too much into a design war (well, this is a design discussion, after all), I usually find it useful to avoid making too many assumptions about the system I'm designing and where it's going to be used, whether it is from a browser, a direct socket connection, or even through XMPP or emails being sent back and forth. The point of view I usually take is "make the core of it work standalone, and then plug it in whatever system I need". I find it gives some good separation of concerns and helps modularity on its own. What happens if you want to support players from more than one platform? This break also lets you draw a line on where the user becomes a trusted entity or not. If you decide to do all your validating on the outer layers (connection) of the game server, you can write your application as if it were completely trusted and you no longer need to deal with propagation of errors and whatnot. I like to see connectivity and client calls as an implementation detail around the core application. Of course this doesn't always remain true if one app is for your friends and you and the other for 5 million people at once -- some planning has to be kept in mind. One approach seems to be a bit bottom-up while the other sounds like the opposite. > Using distributed erlang you get node-to-node pings to detect node > failures. This can lead to too much data traffic. If many users are > expected, you probably want TCP connected users rather than erlang node > connected users. Distributed erlang (using the OTP distribution techniques) > tends to require predetermined node names and process configurations, as > opposed to Distributed TCP which leads to dynamic connectivity without > enumerating participants in configuration files. > > pg also uses global names to manage the groups. This introduces potential > contention for the global name (no global resource goes unpunished in > concurrent architectures -- Patrick Nyblom SF 2010). > > If you consider the architecture, you are missing some elements in your > sketch. You should include them before you evaluate the design choices: > > 1) User initiates a connection from a browser (assumption on my part) > 2) TCP connects the user to some erlang server process > 3) User needs to be represented by a proxy in case of connection loss > 4) Current game should be able to finish if a user disconnects > 5) New game may or may not be allowed to start if user missing The proxy for a user is a pattern I've found extremely useful when handling connections from a browser -- I've designed chut (still a prototype, http://bitbucket.org/ferd/chut/) with this idea in mind, and I found it to be quite useful when I wanted to do things like handle more than N client connections from a single user. I wouldn't go without it were I to expect my real-time application to be used in a browser. > New requirements: > > 6) Can non-players observe the game? > 7) Can non-players interact with game players (chat with them)? The observation of the game is interesting, but I'm wondering whether it has any importance when considering if you'd want to use process groups or a system like gen_event. In any case, the end result is equivalent, isn't it? Now about the chat system, I'm thinking this should be part of a different application still exposed to the user. Conflating chat and whatever game/project/system you're building means bundling it with every part of it. If you suddenly want to add Poker to your online casino, you've got to re-write a new chat system for that one. A standalone chat system seems more appropriate from my point of view, but I'm getting a bit off-topic here. > > I can see an approach that there is a gallery and the players at the table. > The gallery can chat amongst themselves, and the players can interact > amongst themselves. The gallery observes all, the players only see the > table (for example). > > > Architecturally, I would determine the principles that define a good > solution before choosing a software pattern: > > 1) How many users can be supported? > 2) What happens in the common failure scenarios? > 3) How do you enforce the game rules (audience vs players, etc)? > 4) What kind of responsiveness is needed? > 5) How much hardware are you willing to configure? > 6) How dynamic is your configuration and user participation? > > > For this problem gen_event probably has the most opportunities for you as > the central design element. It allows multiple groups to observe events > with differing functionality for the groups. You could use pg for the > players and for the gallery, and gen_event to distribute the messages. But > you are missing the connecting server and the proxying user process that > automatically represents (and governs) the users actions. > > With erlang, don't just look at software patterns, step back and look at > architectural patterns. You will find yourself architecting loosely coupled > systems, with layers of connectivity, routing, application constraints and > application logic. Each of these elements can use different software > patterns, but the way they interact should be flexible, dynamic and > auto-healing. > > Agreed. Erlang and its lightweight processes/concurrency makes it much easier to get an architectural viewpoint and to design software with modularity in mind. > Also remember, there is no "good" or "bad" solution. Each solution has > certain characteristics it exhibits such as speed, resilience, capacity, > etc. One solution may be better for one situation and worse for another, > based on the measurement that matters in each case. Don't be biased against > an approach that was rejected in a previous problem application -- it may > turn out useful in a different situation. > > Hence this discussion :) > jay > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mononcqc@REDACTED Mon Apr 19 19:24:13 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 19 Apr 2010 13:24:13 -0400 Subject: [erlang-questions] Re: A discussion in application design and event propagation In-Reply-To: References: <9204A7C1-FCEA-4C0C-B390-62E3C9C3A818@duomark.com> Message-ID: On Mon, Apr 19, 2010 at 12:14 PM, Garrett Smith wrote: > > > Thinking in architectural terms, this strikes me as a pub/sub problem > that a message broker would handle well. RabbitMQ (AMQP), e.g., would > let you publish events to an exchange, which can then be routed to > various queues via bindings. RabbitMQ in particular is a nice option > for Erlang developers. > > Garrett > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > I'm not too familiar with RabbitMQ and queue usage past using them to send the load from an application's frontend to a backend (and also surviving huge peaks in users). How well does it lend itself to real-time applications? How would you make it work with N tables and 4N users? From my little experience with queuing systems like this, it sounds like the number of queues or topics required would be huge, otherwise the amount of filtering even higher. What's your take on that? From jderick@REDACTED Mon Apr 19 19:31:58 2010 From: jderick@REDACTED (John Erickson) Date: Mon, 19 Apr 2010 10:31:58 -0700 Subject: performance vs msg queue length Message-ID: Hello, I have noticed that the amount of work an erlang process gets done seems to decrease as its message queue grows. In extreme cases, this is understandable, as when the message queue fills up all available memory. However, even for more moderate cases such as ~100 messages each around 10kB there seems to be significant slowdown. These are in processes that are not scanning the msg queue; they have a single receive block which is expected to match every message. Is this expected? From vladdu55@REDACTED Mon Apr 19 19:37:07 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 19 Apr 2010 19:37:07 +0200 Subject: [erlang-questions] Convert an OtpErlangBinary object to Java InputStream In-Reply-To: References: Message-ID: On Mon, Apr 19, 2010 at 17:14, zabrane Mikael wrote: > Now, how about to convert an UTF8 String to OtpErlangBinary() ? byte[] buf = string.getBytes(); OtpErlangBinary b = new OtpErlangBinary(buf); regards, Vlad From jay@REDACTED Mon Apr 19 19:57:48 2010 From: jay@REDACTED (Jay Nelson) Date: Mon, 19 Apr 2010 10:57:48 -0700 Subject: [erlang-questions] Re: A discussion in application design and event propagation In-Reply-To: References: <9204A7C1-FCEA-4C0C-B390-62E3C9C3A818@duomark.com> Message-ID: <330E578B-A0C0-4D3E-850B-6B5076C3D2AC@duomark.com> On Apr 19, 2010, at 10:21 AM, Fred Hebert wrote: > Without wanting to get too much into a design war (well, this is a > design discussion, after all), I usually find it useful to avoid > making too many assumptions about the system I'm designing and > where it's going to be used, whether it is from a browser, a direct > socket connection, or even through XMPP or emails being sent back > and forth. Sorry, posting shorthand. I meant to differentiate between connecting from within erlang (distributed OTP erlang) or from outside erlang (browser, email, etc). The former requires pre- configuration with node names, the latter doesn't. Applications may have different choices, but I agree with your goal of loosely coupling and "separation of concerns." > The observation of the game is interesting, but I'm wondering > whether it has any importance when considering if you'd want to use > process groups or a system like gen_event. In any case, the end > result is equivalent, isn't it? It means 2 pg instances instead of one. If so, how are they related? One gets game messages (and private deal, etc messages), and the other gets some or all messages. Different rules for the 2 groups. gen_event is a good way to forward select messages to both, thus my suggestion of using gen_event as a dispatcher and pg for the group recipients. But you may find using a pub/sub solution like RabbitMQ a better routing concentrator / filter. They key feature in distributing the messages is latency depending on how real-time the game is. > > Now about the chat system, I'm thinking this should be part of a > different application still exposed to the user. Conflating chat > and whatever game/project/system you're building means bundling it > with every part of it. If you suddenly want to add Poker to your > online casino, you've got to re-write a new chat system for that > one. A standalone chat system seems more appropriate from my point > of view, but I'm getting a bit off-topic here. Makes sense. Now there are three classes of messages (private, game, and chat) and two groups (players, watchers) and two communication routes (game <-> player and person <-> person). It seems clear that a single pattern (process group or gen_event) doesn't solve your problem: you need independently addressable recipients and a routing / comms mechanism. pg treats all recipients equally and relies on them to filter the message stream. pg2 allows you to put the filtering logic in the center and group members only receive relevant messages. But the logic in the middle is becoming more complicated, and you see the need to let it flexible expand, so I believe there is a relationship that you need to capture which decouples addressability / routing from selection / filtering of messages. (Given that you are going down the messaging path.) jay From jesper.pettersson@REDACTED Mon Apr 19 20:06:19 2010 From: jesper.pettersson@REDACTED (Jesper Pettersson) Date: Mon, 19 Apr 2010 20:06:19 +0200 Subject: [erlang-questions] shell-questions In-Reply-To: References: Message-ID: Thank you, this was exactly what I was looking for! I got a "how could i have lived without this" vibe just from testing it a little bit. /Jesper Pettersson On Mon, Apr 19, 2010 at 4:52 PM, Evans, Matthew wrote: > Someone posted this a few weeks ago: > > > For anyone who hasn't found it, there's a great little utility (on > > Linux, at least) called 'rlwrap' which gives you the full readline > > feature set on any command line system (like, say, the erlang shell). > > I simply have 'erl' aliased to 'rlwrap -a erl'. It gives you > > everything provided by this patch as well as as a searchable history > > (just like you get in bash with Ctrl-r). > > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Jesper Pettersson > Sent: Monday, April 19, 2010 2:42 AM > To: erlang-questions@REDACTED > Subject: [erlang-questions] shell-questions > > I have two questions about the erlang shell: > > 1) Is there any way to make the history persistent between sessions? > 2) Is there any way to enable emacs-style backwards search in the command > history using CTRL-R? > > Thanks in advance, > > Jesper Pettersson > Klarna AB > -- Jesper Pettersson Klarna AB Norra Stationsgatan 61 113 43 Stockholm, Sweden Tel: +46 8 - 120 120 00 Mob: +46 70 - 001 27 25 Fax: +46 8 - 120 120 99 E-mail: jesper.pettersson@REDACTED Web: www.klarna.com From ulf.wiger@REDACTED Mon Apr 19 20:38:12 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 19 Apr 2010 20:38:12 +0200 Subject: [erlang-questions] performance vs msg queue length In-Reply-To: References: Message-ID: <4BCCA314.8020004@erlang-solutions.com> John Erickson wrote: > Hello, I have noticed that the amount of work an erlang process gets done > seems to decrease as its message queue grows. In extreme cases, this is > understandable, as when the message queue fills up all available memory. > However, even for more moderate cases such as ~100 messages each around > 10kB there seems to be significant slowdown. These are in processes that > are not scanning the msg queue; they have a single receive block which is > expected to match every message. Is this expected? Are you sure that the work generated by a message doesn't lead to e.g. gen_server calls to other processes. Any such call would lead to a selective receive operation, which would have to scan the message queue. We happened to be discussing this at the SF Erlang Factory (Patrik, here's your reminder): in a case such as call(P, Req) -> MRef = erlang:monitor(process, P), % Mark P ! {'$call', {self(),MRef}, Req}, receive {MRef, Reply} -> Reply; {'DOWN', MRef, _, _, Reason} -> erlang:error(Reason) after 5000 -> erlang:error(timeout) end. the compiler could figure out that it could skip to the end of the message queue (the end as of % Mark above), since the following receive clause cannot possibly match any messages that came in before this point (they all match on MRef which is a new and unique object). BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From hynek@REDACTED Mon Apr 19 21:42:02 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Mon, 19 Apr 2010 21:42:02 +0200 Subject: [erlang-questions] shell-questions In-Reply-To: References: Message-ID: Unfortunately rlwrap filters out ^G which is show stopper for me. Is there any workaround? On Mon, Apr 19, 2010 at 4:52 PM, Evans, Matthew wrote: > Someone posted this a few weeks ago: > >> For anyone who hasn't found it, there's a great little utility (on >> Linux, at least) called 'rlwrap' which gives you the full readline >> feature set on any command line system (like, say, the erlang shell). >> I simply have 'erl' aliased to 'rlwrap -a erl'. ?It gives you >> everything provided by this patch as well as as a searchable history >> (just like you get in bash with Ctrl-r). > > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Jesper Pettersson > Sent: Monday, April 19, 2010 2:42 AM > To: erlang-questions@REDACTED > Subject: [erlang-questions] shell-questions > > I have two questions about the erlang shell: > > 1) Is there any way to make the history persistent between sessions? > 2) Is there any way to enable emacs-style backwards search in the command > history using CTRL-R? > > Thanks in advance, > > Jesper Pettersson > Klarna AB > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss. Be a data hero! Try GoodData now for free: www.gooddata.com From dale@REDACTED Mon Apr 19 21:51:56 2010 From: dale@REDACTED (Dale Harvey) Date: Mon, 19 Apr 2010 20:51:56 +0100 Subject: [erlang-questions] shell-questions In-Reply-To: References: Message-ID: you can use to_erl / run_erl to use erlang through a named pipe http://stackoverflow.com/questions/2531150/how-to-receive-stdio-and-error-logger-messages-on-a-remote-shell I have found the ssh part redundant, but it seems to work well. On 19 April 2010 20:42, Hynek Vychodil wrote: > Unfortunately rlwrap filters out ^G which is show stopper for me. Is > there any workaround? > > On Mon, Apr 19, 2010 at 4:52 PM, Evans, Matthew > wrote: > > Someone posted this a few weeks ago: > > > >> For anyone who hasn't found it, there's a great little utility (on > >> Linux, at least) called 'rlwrap' which gives you the full readline > >> feature set on any command line system (like, say, the erlang shell). > >> I simply have 'erl' aliased to 'rlwrap -a erl'. It gives you > >> everything provided by this patch as well as as a searchable history > >> (just like you get in bash with Ctrl-r). > > > > > > -----Original Message----- > > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] > On Behalf Of Jesper Pettersson > > Sent: Monday, April 19, 2010 2:42 AM > > To: erlang-questions@REDACTED > > Subject: [erlang-questions] shell-questions > > > > I have two questions about the erlang shell: > > > > 1) Is there any way to make the history persistent between sessions? > > 2) Is there any way to enable emacs-style backwards search in the command > > history using CTRL-R? > > > > Thanks in advance, > > > > Jesper Pettersson > > Klarna AB > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try GoodData now for free: www.gooddata.com > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From fritchie@REDACTED Mon Apr 19 22:17:29 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Mon, 19 Apr 2010 15:17:29 -0500 Subject: ACM Erlang Workshop 2010: call for papers Message-ID: <95631.1271708249@snookles.snookles.com> Greetings, everyone. Of the many Erlang'ers that I've met at conferences in the last ten years, and of the many more that I've corresponded with via email, it's clear that there's a lot of cool stuff happening in the Erlang world. There's new features in Erlang the language itself and in its virtual machine. There are Erlang'ish features creeping into other languages. And Erlang has long ago broken away from its telecom heritage into messaging, databases, Web services, video streaming, software testing, and so many more areas. On behalf of the program committee for the ACM Erlang Workshop 2010, I'd like you to consider writing a paper for this year's workshop. This year, Kostis Sangonas is introducing two different tracks of papers; see the CFP below for details. If you can't spare the time for a full 10-12 page paper, please consider the second track at 5-6 pages. Erlang has deep roots in pragmatism and practicality. Industrialists tell the academics what really works. Academics help shape the new tools industry needs. The Erlang Workshop brings all together to knit a stronger community. Please support that community with a paper. Your resume/CV will thank you, also. :-) Please email me or Kostis if you have any questions. I look forwarding to seeing you in Baltimore! -Scott --- Scott Lystig Fritchie Gemini Mobile Technologies Snookles Music Consulting --- snip --- snip --- snip --- snip --- snip --- snip --- snip --- CALL FOR PAPERS Ninth ACM SIGPLAN Erlang Workshop Baltimore, Maryland, USA, Thursday, September 30, 2010 http://www.erlang.org/workshop/2010/ A satellite event of the 15th ACM SIGPLAN International Conference on Functional Programming (ICFP) Erlang is a concurrent, distributed functional programming language aimed at systems with requirements on massive concurrency, soft real time response, fault tolerance, and high availability. It has been available as open source for over 10 years, creating a community that actively contributes to its already existing rich set of libraries and applications. Originally created for telecom applications, its usage has spread to other domains including e-commerce, banking, databases, and computer telephony and messaging. Erlang programs are today among the largest applications written in any functional programming language. These applications offer new opportunities to evaluate functional programming and functional programming methods on a very large scale and suggest new problems for the research community to solve. This workshop will bring together the open source, academic, and industrial programming communities of Erlang. It will enable participants to familiarize themselves with recent developments on new techniques and tools tailored to Erlang, novel applications, draw lessons from users' experiences and identify research problems and common areas relevant to the practice of Erlang and functional programming. We invite two sorts of submissions: 1) technical papers describing language extensions, critical discussions of the status quo, formal semantics of language constructs, program analysis and transformation, virtual machine extensions and compilation techniques, implementations and interfaces of Erlang in/with other languages, and new tools (profilers, tracers, debuggers, testing frameworks, etc.) 2) practice and application papers describing uses of Erlang in the "real-world", Erlang libraries for specific tasks, experiences from using Erlang in specific application domains, reusable programming idioms and elegant new ways of using Erlang to approach or solve a particular problem. Workshop Chair * Scott Lystig Fritchie, Gemini Mobile Technologies, Inc. Program Chair * Konstantinos Sagonas, National Technical University of Athens, Greece Program Committee (the Workshop and Program Chairs are also committee members) * Danny Dub?? Universit?? Laval, Canada * Garry Hodgson AT&T Chief Security Office, U.S.A. * Zoltan Horvath E??tv??s Lor??nd University, Hungary * Micka??l R??mond Process One, France * Erik Stenman Klarna AB * Hans Svensson Chalmers University of Technology, Sweden * Simon Thompson University of Kent, U.K. * Ulf Wiger Erlang Solutions Ltd, U.K. Important Dates (Tentative) * Submission deadline: Friday, June 11, 2010 * Author notification: Monday, June 28, 2010 * Final submission: TBA * Workshop date: September 30, 2010 Instructions to authors Papers must be submitted online via EasyChair (via the "Erlang2010" event). Submitted papers should be in portable document format (PDF), formatted using the ACM SIGPLAN style guidelines. The length for technical papers is restricted to 12 pages. For "Practice and Application" papers, the length is restricted to 6 pages; papers in this cateogory may be allocated less time for their talk and instead be given the opportunity for a demo and/or poster session during the workshop. Each submission must adhere to SIGPLAN's republication policy. Violation risks summary rejection of the offending submission. Accepted papers will be published by the ACM and will appear in the ACM Digital Library. Venue & Registration Details * For registration, please see the ICFP web site. Related Links * ICFP 2010 web site: http://www.icfpconference.org/icfp2010/ * Past ACM SIGPLAN Erlang workshops: http://www.erlang.se/workshop * Open Source Erlang: http://www.erlang.org/ * EasyChair submission site: https://www.easychair.org/account/signin.cgi?conf=erlang2010 * Author Information for SIGPLAN Conferences: http://www.sigplan.org/authorInformation.htm From g@REDACTED Mon Apr 19 22:20:48 2010 From: g@REDACTED (Garrett Smith) Date: Mon, 19 Apr 2010 15:20:48 -0500 Subject: [erlang-questions] Re: A discussion in application design and event propagation In-Reply-To: References: <9204A7C1-FCEA-4C0C-B390-62E3C9C3A818@duomark.com> Message-ID: On Mon, Apr 19, 2010 at 12:24 PM, Fred Hebert wrote: > > > On Mon, Apr 19, 2010 at 12:14 PM, Garrett Smith wrote: >> >> >> Thinking in architectural terms, this strikes me as a pub/sub problem >> that a message broker would handle well. RabbitMQ (AMQP), e.g., would >> let you publish events to an exchange, which can then be routed to >> various queues via bindings. RabbitMQ in particular is a nice option >> for Erlang developers. >> >> Garrett >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > I'm not too familiar with RabbitMQ and queue usage past using them to send > the load from an application's frontend to a backend (and also surviving > huge peaks in users). > > How well does it lend itself to real-time applications? How would you make > it work with N tables and 4N users? From my little experience with queuing > systems like this, it sounds like the number of queues or topics required > would be huge, otherwise the amount of filtering even higher. What's your > take on that? The overhead of relaying messages is very low. In a distributed system though you'll have network latencies as well as costs in decoding and encoding messages. I'm pretty sure this doesn't qualify as "real time", but for your use case it's probably fine. If you have specific requirements, I'd post to the rabbit list. I haven't paid close attention to your card game design, but at first stab, I'd use what AMQP calls a "topic" exchange. This delivers messages to queues based on routing key patterns. Some of the possible message keys might be: blackjack.table.setup blackjack.table.deal blackjack.table.teardown blackjack.player.join blackjack.player.leave Message producers, e.g. a "table" process, would send messages to the exchange with a particular routing key designating an event. So if you spawned a new table process, you might send a message keyed with "blackjack.table.setup". The message would contain information about the table (number, capacity, etc.) Message consumers would each create a queue for receiving messages and "bind" that queue using routing key patterns -- think of this as subscribing to a message stream based on their keys. E.g. a player process is interested when a table deal event occurs -- it would bind using "blackjack.table.deal". You could "sniff" the messages for those applying to your table(s) or push the routing keys further by including table IDs (in which case you'd receive table events only for the tables you bind to -- e.g. "blackjack.table.4.deal"). As for the proliferation of queues, you'd need at least one queue per participating process. I think it's fair to say that rabbitmq will handily support tens of thousands of queues on a single machine (throughput aside), though this is also something you'd want to pass by the rabbit list. As always there are lots of variable when scale is an issue. But in general, message brokers specialize in routing messages involving lots and lots of producers and consumers. RabbitMQ happens to be one excellent option (there are certainly others) and has the benefit in your case of being very Erlang friendly. Garrett From mononcqc@REDACTED Mon Apr 19 22:27:30 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 19 Apr 2010 16:27:30 -0400 Subject: [erlang-questions] Re: A discussion in application design and event propagation In-Reply-To: References: <9204A7C1-FCEA-4C0C-B390-62E3C9C3A818@duomark.com> Message-ID: On Mon, Apr 19, 2010 at 4:20 PM, Garrett Smith wrote: > On Mon, Apr 19, 2010 at 12:24 PM, Fred Hebert wrote: > > > > > > On Mon, Apr 19, 2010 at 12:14 PM, Garrett Smith wrote: > >> > >> > >> Thinking in architectural terms, this strikes me as a pub/sub problem > >> that a message broker would handle well. RabbitMQ (AMQP), e.g., would > >> let you publish events to an exchange, which can then be routed to > >> various queues via bindings. RabbitMQ in particular is a nice option > >> for Erlang developers. > >> > >> Garrett > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > I'm not too familiar with RabbitMQ and queue usage past using them to > send > > the load from an application's frontend to a backend (and also surviving > > huge peaks in users). > > > > How well does it lend itself to real-time applications? How would you > make > > it work with N tables and 4N users? From my little experience with > queuing > > systems like this, it sounds like the number of queues or topics required > > would be huge, otherwise the amount of filtering even higher. What's your > > take on that? > > The overhead of relaying messages is very low. In a distributed system > though you'll have network latencies as well as costs in decoding and > encoding messages. I'm pretty sure this doesn't qualify as "real > time", but for your use case it's probably fine. If you have specific > requirements, I'd post to the rabbit list. > No specific requirement. I only picked the card game as a conversation point. I only thought that this kind of pattern is something I'd encounter a lot and wanted to discuss it and bounce ideas. > > I haven't paid close attention to your card game design, but at first > stab, I'd use what AMQP calls a "topic" exchange. This delivers > messages to queues based on routing key patterns. Some of the possible > message keys might be: > > blackjack.table.setup > blackjack.table.deal > blackjack.table.teardown > blackjack.player.join > blackjack.player.leave > > Message producers, e.g. a "table" process, would send messages to the > exchange with a particular routing key designating an event. So if you > spawned a new table process, you might send a message keyed with > "blackjack.table.setup". The message would contain information about > the table (number, capacity, etc.) > > Message consumers would each create a queue for receiving messages and > "bind" that queue using routing key patterns -- think of this as > subscribing to a message stream based on their keys. E.g. a player > process is interested when a table deal event occurs -- it would bind > using "blackjack.table.deal". You could "sniff" the messages for those > applying to your table(s) or push the routing keys further by > including table IDs (in which case you'd receive table events only for > the tables you bind to -- e.g. "blackjack.table.4.deal"). > Ah, I see. Thanks for the explanations. I had only briefly went over Queue systems before (only as much as I needed to use them) and never had to play with too complex pub/sub setups. This makes things clearer. > > As for the proliferation of queues, you'd need at least one queue per > participating process. I think it's fair to say that rabbitmq will > handily support tens of thousands of queues on a single machine > (throughput aside), though this is also something you'd want to pass > by the rabbit list. As always there are lots of variable when scale is > an issue. > > But in general, message brokers specialize in routing messages > involving lots and lots of producers and consumers. RabbitMQ happens > to be one excellent option (there are certainly others) and has the > benefit in your case of being very Erlang friendly. > > Garrett > This makes sense. Most of my [limited] experience with Queue systems were with ActiveMQ, where I guess the number of queue is more limited. I think RabbitMQ is a very interesting piece of architecture to add to an application, but I'm not too sure about the latency part. I guess this is a case of benchmarks, because as you said, it can be fine depending on the use case. Thanks for the explanation. From mononcqc@REDACTED Mon Apr 19 22:34:46 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 19 Apr 2010 16:34:46 -0400 Subject: [erlang-questions] Re: A discussion in application design and event propagation In-Reply-To: <330E578B-A0C0-4D3E-850B-6B5076C3D2AC@duomark.com> References: <9204A7C1-FCEA-4C0C-B390-62E3C9C3A818@duomark.com> <330E578B-A0C0-4D3E-850B-6B5076C3D2AC@duomark.com> Message-ID: On Mon, Apr 19, 2010 at 1:57 PM, Jay Nelson wrote: > It means 2 pg instances instead of one. If so, how are they related? One > gets game messages (and private deal, etc messages), and the other gets some > or all messages. Different rules for the 2 groups. gen_event is a good way > to forward select messages to both, thus my suggestion of using gen_event as > a dispatcher and pg for the group recipients. But you may find using a > pub/sub solution like RabbitMQ a better routing concentrator / filter. They > key feature in distributing the messages is latency depending on how > real-time the game is. > > That's a pretty interesting approach I hadn't thought of. Users could refine their own filtering whenever they receive a message in each process group, but the largest part of it could be done straight from an event manager where groups subscribe. This does seem to play on the strengths of both options. > jay > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mail@REDACTED Mon Apr 19 22:37:04 2010 From: mail@REDACTED (Tim Fletcher) Date: Mon, 19 Apr 2010 13:37:04 -0700 (PDT) Subject: Read from input without echo In-Reply-To: References: Message-ID: > I have limited knowledge of escript but have you tried io:get_password/0 ? That doesn't work with escript: http://groups.google.com/group/erlang-programming/browse_thread/thread/b6bb574d2fd9956b This functionality would be useful. Does the comment about escript using the old shell still stand? Tim From fritchie@REDACTED Mon Apr 19 22:48:28 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Mon, 19 Apr 2010 15:48:28 -0500 Subject: [erlang-questions] testing asynchronous code In-Reply-To: Message of "Mon, 19 Apr 2010 16:15:12 +0530." Message-ID: <97417.1271710108@snookles.snookles.com> Martin, you have another option: your test code calls YourModule:handle_cast() directly. If your client side stub looks like: client_side_stub(Server, Args) -> gen_server:cast(Server, {some_tag, Args}). Then refactor to this and export both client_side_stub() and make_some_tag(). client_side_stub(Server, Args) -> gen_server:cast(Server, make_some_tag(Args)). make_some_tag(Args) -> {some_tag, Args}. Now your testing code can do: OldState = YourModule:init(WhateverArgsYouNeed), case YourModule:handle_cast(YourModule:make_sometag(Args), OldState) of {noreply, State} -> verify_state_is_sane(State); {noreply, State, Timeout} -> %% Your code may not use this return tuple... verify_state_is_sane(State); {stop, Reason, State} -> %% Your code may not use this return tuple either verify_state_is_sane(State); The above code is has lots of cut-and-paste redundancy that you can factor out, if you wish. Or you can use the skeleton as-is, allowing the test code to check that the {noreply, State, Timeout} is indeed correct in some cases and {noreply, State} correct in other cases. There's only a little bit of glue code yet to write to grow into something like this: OldState = YourModule:init(WhateverArgsYouNeed), State = fold_async_messages(YourModule, OldState, [cast_message_number_1, {cast_message_2, Args}, {cast_message_foo, Args, More, Stuff}]), ok = verify_state_is_sane(State). Or perhaps you want to be able to pull the State out of an already-running gen_server process and test it like the above. Go ahead, create this: handle_call(get_internal_state, _From, State) -> {reply, State, State}; Now your testing code can use verify_state_is_sane(gen_server:call(Server, get_internal_state)). Or perhaps you'd rather embed verify_state_is_sane() inside the gen_server? Your test code can do this call, but the rest of the client API will ignore it. handle_call(check_internal_state, _From, State) -> ok = verify_internal_state(State), {reply, yup_looks_wonderful, State}; Hope this helps. -Scott From rvirding@REDACTED Mon Apr 19 23:16:02 2010 From: rvirding@REDACTED (Robert Virding) Date: Mon, 19 Apr 2010 23:16:02 +0200 Subject: floating point syntax In-Reply-To: References: Message-ID: IIRC it was probably more that we/I just didn't see the need to be able to write the shortened forms. It isn't that it is much extra you have to write. :-) Robert On 16 April 2010 18:26, Michael Turner wrote: > > I guess this has something to do with the fact that a period is also a > terminator in Erlang syntax. ?If you allowed "1." as a legal float, > for example, you're permitting this: > > ?fp1() -> 1.. > > And with > > ?f() -> 1.e5; > > you need a bit more token lookahead (since "e5" could be seen as the > start of a new function). ?My guess, as I say. > > -michael turner > > On 4/15/2010, "Robert Virding" wrote: > >>Yes, it is supposed to be like that, no, it is not a bug. Floats >>consist of both parts before and after the decimal point and an >>optional exponent. >> >>Robert >> >>On 15 April 2010 16:21, Anthony Shipman wrote: >>> The reference manual says "There are two types of numeric literals, integers >>> and floats. Besides the conventional notation,..." >>> >>> But what is the "conventional notation" exactly? I find that this fails: >>> 19> io_lib:fread("~f", "1e+2"). >>> {error,{fread,float}} >>> >>> Also >>> 1.e+2 fails >>> 1 ? ? fails >>> 1. ? ?fails >>> 1.0 ? succeeds >>> .1 ? ?fails >>> 0.1 ? succeeds >>> 1.0e+2 succeeds >>> >>> Is it supposed to be like that or is there a bug? >>> >>> -- >>> Anthony Shipman ? ? ? ? ? ? ? ? ? ?Mamas don't let your babies >>> als@REDACTED ? ? ? ? ? ? ? ? ? grow up to be outsourced. >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >>________________________________________________________________ >>erlang-questions (at) erlang.org mailing list. >>See http://www.erlang.org/faq.html >>To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From dennis.novikov@REDACTED Mon Apr 19 23:56:21 2010 From: dennis.novikov@REDACTED (Dennis Novikov) Date: Tue, 20 Apr 2010 00:56:21 +0300 Subject: [erlang-questions] shell-questions In-Reply-To: References: Message-ID: On Mon, Apr 19, 2010 at 10:51 PM, Dale Harvey wrote: > you can use to_erl / run_erl to use erlang through a named pipe > > http://stackoverflow.com/questions/2531150/how-to-receive-stdio-and-error-logger-messages-on-a-remote-shell > > I have found the ssh part redundant, but it seems to work well. It is there for "really remote shell" :) , as most of maintenance work I did this way was actually done from remote computer -- WBR, DN From ok@REDACTED Tue Apr 20 00:25:28 2010 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 20 Apr 2010 10:25:28 +1200 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> Message-ID: I pointed out that modules with parameters give us nothing that plain old closures don't, in response to which On Apr 19, 2010, at 3:43 PM, Bob Ippolito wrote: >> > > Trivial, but requires a lot of boilerplate code and certainly isn't > any easier to understand or debug than parameterized modules. It also > becomes impossible to write a useful type spec if you use closures > like that. (a) Such boilerplate code as is required can be automatically generated. (b) The possibility or otherwise of useful type specs depends on what the type language allows. Dependent type systems would have no trouble at all; the question of _how much_ of the machinery of a dependent type system one needs is an empirical matter. (c) A dismissal of closures on such grounds is ALSO a dismissal of modules with parameters since each can simulate the other. From bob@REDACTED Tue Apr 20 00:46:36 2010 From: bob@REDACTED (Bob Ippolito) Date: Mon, 19 Apr 2010 15:46:36 -0700 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> Message-ID: On Mon, Apr 19, 2010 at 3:25 PM, Richard O'Keefe wrote: > I pointed out that modules with parameters give us nothing > that plain old closures don't, in response to which > On Apr 19, 2010, at 3:43 PM, Bob Ippolito wrote: > >>> >> >> Trivial, but requires a lot of boilerplate code and certainly isn't >> any easier to understand or debug than parameterized modules. It also >> becomes impossible to write a useful type spec if you use closures >> like that. > > (a) Such boilerplate code as is required can be automatically generated. > (b) The possibility or otherwise of useful type specs depends on what > ? ?the type language allows. ?Dependent type systems would have no > ? ?trouble at all; the question of _how much_ of the machinery of a > ? ?dependent type system one needs is an empirical matter. > (c) A dismissal of closures on such grounds is ALSO a dismissal of > ? ?modules with parameters since each can simulate the other. I was speaking practically, not theoretically. The current implementation of Erlang, edoc, and related tools have much better support for parameterized modules than for using closures to emulate parameterized modules. We actually use the closure approach more often, because closures also have desirable properties with regard to module reloading. Our use of parameterized modules is almost entirely limited to mochiweb_request. However, when we do use closures to capture state and provide a module's worth of features it is not elegant due to boilerplate code and Erlang syntax "features". -bob From bernie@REDACTED Tue Apr 20 00:53:25 2010 From: bernie@REDACTED (Bernard Duggan) Date: Tue, 20 Apr 2010 08:53:25 +1000 Subject: [erlang-questions] shell-questions In-Reply-To: References: Message-ID: <4BCCDEE5.8000509@m5net.com> I did a bunch of reading on and tinkering with rlwrap after the last time this was raised as a (quite valid) issue. It looks, sadly, like the answer at the moment is 'no'. From memory, it was a limitation of the readline library and the configuration thereof. If anyone wants to correct me, though, I'd be more than happy for them to do so (I'd love to have tab-completion back) :) B Hynek Vychodil wrote: > Unfortunately rlwrap filters out ^G which is show stopper for me. Is > there any workaround? > > On Mon, Apr 19, 2010 at 4:52 PM, Evans, Matthew wrote: > >> Someone posted this a few weeks ago: >> >> >>> For anyone who hasn't found it, there's a great little utility (on >>> Linux, at least) called 'rlwrap' which gives you the full readline >>> feature set on any command line system (like, say, the erlang shell). >>> I simply have 'erl' aliased to 'rlwrap -a erl'. It gives you >>> everything provided by this patch as well as as a searchable history >>> (just like you get in bash with Ctrl-r). >>> >> -----Original Message----- >> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Jesper Pettersson >> Sent: Monday, April 19, 2010 2:42 AM >> To: erlang-questions@REDACTED >> Subject: [erlang-questions] shell-questions >> >> I have two questions about the erlang shell: >> >> 1) Is there any way to make the history persistent between sessions? >> 2) Is there any way to enable emacs-style backwards search in the command >> history using CTRL-R? >> >> Thanks in advance, >> >> Jesper Pettersson >> Klarna AB >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> > > > > From bernie@REDACTED Tue Apr 20 01:16:52 2010 From: bernie@REDACTED (Bernard Duggan) Date: Tue, 20 Apr 2010 09:16:52 +1000 Subject: [erlang-questions] performance vs msg queue length In-Reply-To: <4BCCA314.8020004@erlang-solutions.com> References: <4BCCA314.8020004@erlang-solutions.com> Message-ID: <4BCCE464.7090902@m5net.com> Ulf Wiger wrote: > John Erickson wrote: > >> Hello, I have noticed that the amount of work an erlang process gets done >> seems to decrease as its message queue grows. In extreme cases, this is >> understandable, as when the message queue fills up all available memory. >> However, even for more moderate cases such as ~100 messages each around >> 10kB there seems to be significant slowdown. These are in processes that >> are not scanning the msg queue; they have a single receive block which is >> expected to match every message. Is this expected? >> > > Are you sure that the work generated by a message doesn't lead to > e.g. gen_server calls to other processes. Any such call would lead > to a selective receive operation, which would have to scan the > message queue. > Just in support of what Ulf is saying, I was thinking exactly the same thing as you, John, when I was seeing massive performance problems with big queues (that is "I don't do any selective receives - why is my performance going to hell?"). Then I realised that the mnesia transaction operations I was doing internally do selective receives. There's no doubt a bunch of other cases too (in addition to the obvious gen_server:call already mentioned). > We happened to be discussing this at the SF Erlang Factory (Patrik, > here's your reminder): in a case such as > > call(P, Req) -> > MRef = erlang:monitor(process, P), > % Mark > P ! {'$call', {self(),MRef}, Req}, > receive > {MRef, Reply} -> Reply; > {'DOWN', MRef, _, _, Reason} -> erlang:error(Reason) > after 5000 -> > erlang:error(timeout) > end. > > the compiler could figure out that it could skip to the > end of the message queue (the end as of % Mark above), > since the following receive clause cannot possibly match any > messages that came in before this point (they all match on > MRef which is a new and unique object). > Oh, very clever :) (No sarcasm at all intended). Presumably that it could only be safely applied where MRef contained something guaranteed unique such as that returned by make_ref()? That would, I'd imagine, require some minor changes to things like mnesia:transaction() if they wanted to take advantage of it? Cheers, Bernard From john.hughes@REDACTED Tue Apr 20 02:26:10 2010 From: john.hughes@REDACTED (John Hughes) Date: Tue, 20 Apr 2010 02:26:10 +0200 Subject: [erlang-questions] Re: testing asynchronous code In-Reply-To: <07971735-2e27-4935-a546-4919c2150749@u21g2000yqc.googlegroups.com> References: <07971735-2e27-4935-a546-4919c2150749@u21g2000yqc.googlegroups.com> Message-ID: <5A06D659616E454E80DF631A0AECF596@JohnsTablet2009> We've actually just written a paper about asynchronous testing, based on testing ejabberd (where the message delivery events are asynchronous). The paper talks about how to analyze a recorded trace, without falling into the twin traps of complex code or expensive analysis. I've put a copy online at www.cs.chalmers.se/~rjmh/ast2010.pdf if you're interested. (The paper will be published at AST 2010 in Cape Town in May). John ----- Original Message ----- From: "Tim Fletcher" To: Sent: Monday, April 19, 2010 1:56 PM Subject: [erlang-questions] Re: testing asynchronous code >> Anyone have general strategies or advice for testing code that makes >> heavy use of gen_server:cast? The best I could come up with was to >> write _sync versions of all my functions, with gen_server:cast >> replaced by gen_server:call, and then test those, but that seems both >> clunky and fragile. > > How about using a "test spy" process to collect a trace of the cast > messages and then make assertions against the resulting trace? > > Tim > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From michael.santos@REDACTED Tue Apr 20 03:49:19 2010 From: michael.santos@REDACTED (Michael Santos) Date: Mon, 19 Apr 2010 21:49:19 -0400 Subject: [erlang-questions] Regular Expressions Problems In-Reply-To: References: Message-ID: <20100420014919.GA13474@ecn.lan> On Mon, Apr 19, 2010 at 01:03:13PM +0100, Gordon Guthrie wrote: > Folks > > I think I may have identified a regular expression bug in re. > > The following code never terminates in R13B-04: > > -module(fail). > > -export([fail/0]). > > fail() -> > Str = "http:/www.flickr.com/slideShow/index.gne?group_id=&user_id=69845378@REDACTED", > EMail_regex = "[a-z0-9!#$%&'*+/=?^_`{|}~-]+" > ++ "(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*" > ++ "@(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+" > ++ "(?:[a-zA-Z]{2}|com|org|net|gov|mil" > ++ "|biz|info|mobi|name|aero|jobs|museum)", > io:format("about to run...~n"), > Ret = re:run(Str, EMail_regex), > io:format("Ret is ~p~n", [Ret]). > > Eliminating the @ in either the string or the regex and it will > terminate - but if you don't it wont... $ pcretest PCRE version 7.4 2007-09-21 re> /[a-z0-9!#$%&'*+\/=?^_`{|}~-]+(?:.[a-z0-9!#$%&'*+\/=?^_`{|}~-]+)*@(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?.)+(?:[a-zA-Z]{2}|com|org|net|gov|mil|biz|info|mobi|name|aero|jobs|museum)/ data> http:/www.flickr.com/slideShow/index.gne?group_id=&user_id=69845378@REDACTED Error -8 "-8" will happen if the match() call counter reaches some limit (by default, 10000000). The comments in the header file explain that "the limit exists in order to catch runaway regular expressions that take for ever to determine that they do not match." In Erlang, after the regexp matching has performed a number of operations, it'll be swapped out. When the regexp matching is resumed, the match() counter is zero'ed. I'm not sure why this is done but removing it at least allows the match to return: 1> fail:fail(). about to run... Ret is nomatch ok diff --git a/erts/emulator/pcre/pcre_exec.c b/erts/emulator/pcre/pcre_exec.c index 5162513..3fe13ca 100644 --- a/erts/emulator/pcre/pcre_exec.c +++ b/erts/emulator/pcre/pcre_exec.c @@ -5191,7 +5191,6 @@ for(;;) EDEBUGF(("Loop limit break detected")); return PCRE_ERROR_LOOP_LIMIT; RESTART_INTERRUPTED: - md->match_call_count = 0; md->loop_limit = extra_data->loop_limit; rc = match(NULL,NULL,NULL,0,md,0,NULL,0,0); *extra_data->loop_counter_return = From dbarker@REDACTED Tue Apr 20 05:37:22 2010 From: dbarker@REDACTED (Deryk Barker) Date: Mon, 19 Apr 2010 20:37:22 -0700 Subject: Dynamic code reloading Message-ID: <4BCD2172.3010403@camosun.bc.ca> Please excuse me if this is a daft question...and I am sure there are OTP functions to do all this for you, but I wanted to show my students some features of "pure" erlang. Specifically, I'd like to have a server that can call the latest version of itself in response to a message. Based on my reading of the "old" erlang book (the OOP, sorry out of print, one), it seemed that having a receive pattern something like: {dbase, Pid, reload} -> reply (Pid, reloading), dbase:loop (DataBase); would work. (It's a *very* simple database server in a module called dbase). Unfortunately, this doesn't cause the new BEAM file, compiled outside the erlang shell, to be loaded. OK, try compiling in the erlang shell (server is running under same shell), but this takes down the server - silently. I know it's gone because its registered name (dbserver) is no longer in the registered name table. Perhaps I need to use code:load_file/1? But that tells me the module needs to be purged first. code:soft_purge/1 (which, according to the documentation, leaves existing processes running the module running) fails - with a not_purged error - and code:purge/1 works fine, but takes down the server. Can anyone tell me how I should be doing this and just how dim I am being? thanks, deryk -- |Deryk Barker, Computer Science Dept. | Music does not have to be understood| |Camosun College, Victoria, BC, Canada| It has to be listened to. | |email: dbarker@REDACTED | | |phone: +1 250 370 4452 | Hermann Scherchen. | From jeffm@REDACTED Tue Apr 20 06:11:30 2010 From: jeffm@REDACTED (Jeffm) Date: Tue, 20 Apr 2010 14:11:30 +1000 Subject: [erlang-questions] Dynamic code reloading In-Reply-To: <4BCD2172.3010403@camosun.bc.ca> References: <4BCD2172.3010403@camosun.bc.ca> Message-ID: <4BCD2972.4020007@ghostgun.com> When you loaded the new module in the shell did you use c() or l(dbase)? If you compiled external to the shell, as you stated, you'll need to use l(module_name) to tell the shell to load the new version. Jeff. On 20/04/10 1:37 PM, Deryk Barker wrote: > Please excuse me if this is a daft question...and I am sure there are > OTP functions to do all this for you, but I wanted to show my > students some features of "pure" erlang. > > Specifically, I'd like to have a server that can call the latest > version of itself in response to a message. > > Based on my reading of the "old" erlang book (the OOP, sorry out of > print, one), it seemed that having a receive pattern something like: > > {dbase, Pid, reload} -> > reply (Pid, reloading), > dbase:loop (DataBase); > > would work. (It's a *very* simple database server in a module called > dbase). > > Unfortunately, this doesn't cause the new BEAM file, compiled outside > the erlang shell, to be loaded. > > OK, try compiling in the erlang shell (server is running under same > shell), but this takes down the server - silently. I know it's gone > because its registered name (dbserver) is no longer in the registered > name table. > > Perhaps I need to use code:load_file/1? > > But that tells me the module needs to be purged first. > > code:soft_purge/1 (which, according to the documentation, leaves > existing processes running the module running) fails - with a > not_purged error - and code:purge/1 works fine, but takes down the > server. > > Can anyone tell me how I should be doing this and just how dim I am > being? > > thanks, > > deryk > From martindemello@REDACTED Tue Apr 20 08:27:22 2010 From: martindemello@REDACTED (Martin DeMello) Date: Tue, 20 Apr 2010 11:57:22 +0530 Subject: [erlang-questions] testing asynchronous code In-Reply-To: <8A5415EEB6C64512BCBB2A6A922AD9EE@SSI.CORP> References: <8A5415EEB6C64512BCBB2A6A922AD9EE@SSI.CORP> Message-ID: On Mon, Apr 19, 2010 at 8:21 PM, David Mercer wrote: > > I assume your issue is that you cannot test system state right after a cast > because its processing is asynchronous? Yes, exactly. > Alternatively, you might could[1] add a "sync" call into the server, which > does nothing, but you know when it returns that all prior casts have been > processed, and then you can make your asserts on the server or system state, > etc. ?(If the casts are passing processing off to other processes, the sync > call would have to "sync" with them before returning.) This was one of the things I thought about, but have no idea how to actually do. Where and how would I keep track of which casts are processed? Here's a concrete example - I have a tree of objects in mnesia, which I want to run some cleanup code on. So I send a cast to the root, which cleans up its associated resource, sends casts to all its children, and then deletes itself from the db, and so on recursively. How would I go about syncing on that? For example, I could wait for all the objects to be deleted from the db, but that's syncing on the "side effects" of the cast, rather than explicitly when the function it calls has returned. martin From dbarker@REDACTED Tue Apr 20 08:28:22 2010 From: dbarker@REDACTED (Deryk Barker) Date: Mon, 19 Apr 2010 23:28:22 -0700 Subject: [erlang-questions] Dynamic code reloading In-Reply-To: <4BCD2972.4020007@ghostgun.com> References: <4BCD2172.3010403@camosun.bc.ca> <4BCD2972.4020007@ghostgun.com> Message-ID: <4BCD4986.9000303@camosun.bc.ca> Jeffm wrote: > When you loaded the new module in the shell did you use c() or > l(dbase)? If you compiled external to the shell, as you stated, you'll > need to use l(module_name) to tell the shell to load the new version. I tried c() which took down the server process. At your suggestion, I just tried compiling externally and then used l(). Which worked - the first time, i.e. when all I had done since starting the erlang shell was spawn the server and register its name (which was definitely in the list returned by registered()). When I recompiled externally again and re-used l() it took down the server process...sigh. I don't understand what's going on here, my understanding of the manual was that you can have two versions of a module loaded, the current and the old, I should have expected, after the load, the running server to be using the old. It successfully switched to the newly loaded code, via the call dbase:loop(DataBase), the first time I tried this, so - again, as I understand it - it was then using the current. When I did another load of the module (again using l()), the old old (as it were) is purged and the current becomes the old. So why, if my server process was now running the (about to be old) current was it terminated? And why silently? Any insight gratefully received. deryk -- |Deryk Barker, Computer Science Dept. | Music does not have to be understood| |Camosun College, Victoria, BC, Canada| It has to be listened to. | |email: dbarker@REDACTED | | |phone: +1 250 370 4452 | Hermann Scherchen. | From martindemello@REDACTED Tue Apr 20 08:30:24 2010 From: martindemello@REDACTED (Martin DeMello) Date: Tue, 20 Apr 2010 12:00:24 +0530 Subject: [erlang-questions] Re: testing asynchronous code In-Reply-To: <07971735-2e27-4935-a546-4919c2150749@u21g2000yqc.googlegroups.com> References: <07971735-2e27-4935-a546-4919c2150749@u21g2000yqc.googlegroups.com> Message-ID: On Mon, Apr 19, 2010 at 5:26 PM, Tim Fletcher wrote: >> Anyone have general strategies or advice for testing code that makes >> heavy use of gen_server:cast? The best I could come up with was to >> write _sync versions of all my functions, with gen_server:cast >> replaced by gen_server:call, and then test those, but that seems both >> clunky and fragile. > > How about using a "test spy" process to collect a trace of the cast > messages and then make assertions against the resulting trace? Thanks, that's an interesting suggestion. I take it the basic idea is to unit test all the underlying synchronous code separately, and then assert that if the low-level code works, and all the casts are going out as specified, then the high-level code is correct? martin From igorrs@REDACTED Tue Apr 20 08:31:26 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Tue, 20 Apr 2010 03:31:26 -0300 Subject: [erlang-questions] Dynamic code reloading In-Reply-To: <4BCD2172.3010403@camosun.bc.ca> References: <4BCD2172.3010403@camosun.bc.ca> Message-ID: Once your server is up for the first time, you can use those steps to update the module after every change in the beam file: - Call code:soft_purge(dbase). - Call code:load_file(dbase). - Send a message {dbase, self(), reload} to the server. Igor. On Tue, Apr 20, 2010 at 12:37 AM, Deryk Barker wrote: > Please excuse me if this is a daft question...and I am sure there are OTP > functions to do all this for you, but I wanted ?to show my students some > features of "pure" erlang. > > Specifically, I'd like to have a server that can call the latest version of > itself in response to a message. > > Based on my reading of the "old" erlang book (the OOP, sorry out of print, > one), it seemed that having a receive pattern something like: > > ? {dbase, Pid, reload} -> > ? ? ? reply (Pid, reloading), > ? ? ? dbase:loop (DataBase); > > would work. (It's a *very* simple database server in a module called dbase). > > Unfortunately, this doesn't cause the new BEAM file, compiled outside the > erlang shell, to be loaded. > > OK, try compiling in the erlang shell (server is running under same shell), > but this takes down the server ?- silently. I know it's gone because its > registered name (dbserver) is no longer in the registered name table. > > Perhaps I need to use code:load_file/1? > > But that tells me the module needs to be purged first. > > code:soft_purge/1 (which, according to the documentation, leaves existing > processes running the module running) fails - with a not_purged error - and > code:purge/1 works fine, but takes down the server. > > Can anyone tell me how I should be doing this and just how dim I am being? > > thanks, > > deryk > > -- > |Deryk Barker, Computer Science Dept. | Music does not have to be > understood| > |Camosun College, Victoria, BC, Canada| It has to be listened to. > | > |email: dbarker@REDACTED ? ? ? ? | > | > |phone: +1 250 370 4452 ? ? ? ? ? ? ? | ? ? ? ? Hermann Scherchen. > ?| > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From jeffm@REDACTED Tue Apr 20 08:46:45 2010 From: jeffm@REDACTED (Jeffm) Date: Tue, 20 Apr 2010 16:46:45 +1000 Subject: [erlang-questions] Dynamic code reloading In-Reply-To: <4BCD4986.9000303@camosun.bc.ca> References: <4BCD2172.3010403@camosun.bc.ca> <4BCD2972.4020007@ghostgun.com> <4BCD4986.9000303@camosun.bc.ca> Message-ID: <4BCD4DD5.5040204@ghostgun.com> On 20/04/10 4:28 PM, Deryk Barker wrote: > Jeffm wrote: >> When you loaded the new module in the shell did you use c() or >> l(dbase)? If you compiled external to the shell, as you stated, >> you'll need to use l(module_name) to tell the shell to load the new >> version. > I tried c() which took down the server process. At your suggestion, I > just tried compiling externally and then used l(). > > Which worked - the first time, i.e. when all I had done since starting > the erlang shell was spawn the server and register its name (which was > definitely in the list returned by registered()). When I recompiled > externally again and re-used l() it took down the server process...sigh. At a guest... I saw a similar problem when I was developing a module. The problem is that the erlang vm only holds two versions of the code at once. This is perfectly alright until you have a mistake in your code like I did. The loop wasn't executing the new code and died when a third version was added. Try adding a message which will cause the module to print the current version of code. For example, -module(eg1) -export([ loop/0 ]). -define(V, 1). loop() receive print_version -> io:format("Current version of ~p is ~p~n", [?MODULE, ?V]), eg1:loop(); ... end. then after loading the new code sent it a message to see what the current version is. A thought just occurred to me. Your probably stuck in the old code as the program hasn't progressed around the loop. Load the new version, trigger it to go round the loop, then load the third version and trigger it again. By trigger, I'm suggesting a message, but any event which causes it to go round the loop should work. It only starts the newly loaded code once it re-enters to loop. Jeff. > I don't understand what's going on here, my understanding of the > manual was that you can have two versions of a module loaded, the > current and the old, I should have expected, after the load, the > running server to be using the old. It successfully switched to the > newly loaded code, via the call dbase:loop(DataBase), the first time I > tried this, so - again, as I understand it - it was then using the > current. > > When I did another load of the module (again using l()), the old old > (as it were) is purged and the current becomes the old. > So why, if my server process was now running the (about to be old) > current was it terminated? And why silently? > > Any insight gratefully received. > > deryk > From martindemello@REDACTED Tue Apr 20 08:48:26 2010 From: martindemello@REDACTED (Martin DeMello) Date: Tue, 20 Apr 2010 12:18:26 +0530 Subject: [erlang-questions] testing asynchronous code In-Reply-To: <97417.1271710108@snookles.snookles.com> References: <97417.1271710108@snookles.snookles.com> Message-ID: On Tue, Apr 20, 2010 at 2:18 AM, Scott Lystig Fritchie wrote: > Martin, you have another option: your test code calls > YourModule:handle_cast() directly. ?If your client side stub looks like: Thanks, some good ideas in there. Not sure I've absorbed it fully, but I'll stare at it till it clicks :) From what I can make out, though, it doesn't handle the case of handle_cast() itself calling cast(), since the intermediate states before the entire tree of casts has completed aren't really of interest. Is that correct, or have I missed something? martin From ulf.wiger@REDACTED Tue Apr 20 09:41:51 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 20 Apr 2010 09:41:51 +0200 Subject: suggestion: NIF_D Message-ID: <4BCD5ABF.5070900@erlang-solutions.com> Last week I was at the ACCU 2010 conference in Oxford, and had a short discussion with Walter Bright, the inventor of D. The discussion briefly touched the ErlOCaml project, and the idea that one would like to have the option to drop into a "safe" low-level language from Erlang. Walter (perhaps un- surprisingly) suggested that D might be a suitable candidate. I hadn't looked closer at D - only noted that it tended to do very well in the shootout (no more, it seems, as I couldn't find it there at all now). However, it does look very nice and accessible to C programmers: http://www.digitalmars.com/d/2.0/ctod.html Safety would come partly from the fact that it's a garbage- collected language, and partly because it fixes many of the idiosynchracies of C and C++. Another nice aspect of D is that it is designed to be relatively easy to analyse lexically, and compiles very quickly. This seems to open up for some fairly funky inlining or JIT code generation, but perhaps we're getting ahead of ourselves here... However, I realise that I spend far too little time coding C et al to be the one to try this integration, or indeed even to judge whether it would be worthwhile to do so. Any other takers? Opinions? BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From mazen.harake@REDACTED Tue Apr 20 09:52:47 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 20 Apr 2010 09:52:47 +0200 Subject: [erlang-questions] testing asynchronous code In-Reply-To: References: <8A5415EEB6C64512BCBB2A6A922AD9EE@SSI.CORP> Message-ID: <4BCD5D4F.4060300@erlang-solutions.com> Hi, Sorry for jumping in... This method (of using sync calls for testing async functionality) is sometimes dangerous, specially when you don't know the internals of the system. The simple reason is that a sync call can block something that wasn't suppose to be blocked and therefore you get 1) the possibility of deadlocks which wouldn't be a problem when run "live" and 2) a slight different semantics because you won't get the same behaviour of race-conditions which can appear due to the async nature of the processes communicating. My 2 cents :) /Mazen On 20/04/2010 08:27, Martin DeMello wrote: > On Mon, Apr 19, 2010 at 8:21 PM, David Mercer wrote: > >> I assume your issue is that you cannot test system state right after a cast >> because its processing is asynchronous? >> > Yes, exactly. > > >> Alternatively, you might could[1] add a "sync" call into the server, which >> does nothing, but you know when it returns that all prior casts have been >> processed, and then you can make your asserts on the server or system state, >> etc. (If the casts are passing processing off to other processes, the sync >> call would have to "sync" with them before returning.) >> > This was one of the things I thought about, but have no idea how to > actually do. Where and how would I keep track of which casts are > processed? Here's a concrete example - I have a tree of objects in > mnesia, which I want to run some cleanup code on. So I send a cast to > the root, which cleans up its associated resource, sends casts to all > its children, and then deletes itself from the db, and so on > recursively. How would I go about syncing on that? For example, I > could wait for all the objects to be deleted from the db, but that's > syncing on the "side effects" of the cast, rather than explicitly when > the function it calls has returned. > > martin > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From fabian.bergstrom@REDACTED Tue Apr 20 11:43:29 2010 From: fabian.bergstrom@REDACTED (=?ISO-8859-1?Q?Fabian_Bergstr=F6m?=) Date: Tue, 20 Apr 2010 11:43:29 +0200 Subject: [erlang-questions] Cross compilation and testing In-Reply-To: References: Message-ID: Den 19 april 2010 10.38 skrev Fabian Bergstr?m : > I ran otp_build tests and copied the tests over to the host I compiled > Elrang for but ts:install() doesn't seem to like it, it cannot find a > suitable compiler and a quick look at the code seems to indicate it > just > looks for win32 or unix as the target name to decide how to install. > Is > there a way I can do a staged install of the test server? Please disregard this, after I installed gcc-symlinks and binutils-symlinks, ts:install() worked. I had to make run_make executable before tests would run, though. -- Fabian Bergstr?m From als@REDACTED Tue Apr 20 11:47:07 2010 From: als@REDACTED (Anthony Shipman) Date: Tue, 20 Apr 2010 20:47:07 +1100 Subject: [erlang-questions] Re: floating point syntax In-Reply-To: References: Message-ID: <201004201947.07593.als@iinet.net.au> On Tue, 20 Apr 2010 07:16:02 am Robert Virding wrote: > IIRC it was probably more that we/I just didn't see the need to be > able to write the shortened forms. It isn't that it is much extra you > have to write. :-) > > Robert When parsing numbers for other languages it would be nice to be able to use fread() to parse them instead of writing my own. Some sort of ~N form to read a number, integer or floating point, in the usual syntax would be nice. -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From olopierpa@REDACTED Tue Apr 20 11:59:57 2010 From: olopierpa@REDACTED (Pierpaolo Bernardi) Date: Tue, 20 Apr 2010 11:59:57 +0200 Subject: [erlang-questions] Re: floating point syntax In-Reply-To: <201004201947.07593.als@iinet.net.au> References: <201004201947.07593.als@iinet.net.au> Message-ID: On Tue, Apr 20, 2010 at 11:47, Anthony Shipman wrote: > When parsing numbers for other languages it would be nice to be able to use > fread() to parse them instead of writing my own. Some sort of ~N form to read > a number, integer or floating point, in the usual syntax would be nice. This desire is expressed frequently on this m.l. Search for previous discussions, to see the arguments from the OTP team to reject this *improvement* 8^) Cheers P. From pan+eq@REDACTED Tue Apr 20 12:28:47 2010 From: pan+eq@REDACTED (pan+eq@REDACTED) Date: Tue, 20 Apr 2010 12:28:47 +0200 (CEST) Subject: [erlang-questions] Regular Expressions Problems In-Reply-To: <20100420014919.GA13474@ecn.lan> References: <20100420014919.GA13474@ecn.lan> Message-ID: Yes! You are absolutely right. You definitely saved me some time debugging this! A huge THANKS! I don't know why I did reset the variable, it seems like a silly thing to do now that You've pointed out the error :) I'll add the patch to GIT with you as the author if that's OK. /Patrik On Mon, 19 Apr 2010, Michael Santos wrote: > On Mon, Apr 19, 2010 at 01:03:13PM +0100, Gordon Guthrie wrote: >> Folks >> >> I think I may have identified a regular expression bug in re. >> >> The following code never terminates in R13B-04: >> >> -module(fail). >> >> -export([fail/0]). >> >> fail() -> >> Str = "http:/www.flickr.com/slideShow/index.gne?group_id=&user_id=69845378@REDACTED", >> EMail_regex = "[a-z0-9!#$%&'*+/=?^_`{|}~-]+" >> ++ "(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*" >> ++ "@(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+" >> ++ "(?:[a-zA-Z]{2}|com|org|net|gov|mil" >> ++ "|biz|info|mobi|name|aero|jobs|museum)", >> io:format("about to run...~n"), >> Ret = re:run(Str, EMail_regex), >> io:format("Ret is ~p~n", [Ret]). >> >> Eliminating the @ in either the string or the regex and it will >> terminate - but if you don't it wont... > > $ pcretest > PCRE version 7.4 2007-09-21 > > re> /[a-z0-9!#$%&'*+\/=?^_`{|}~-]+(?:.[a-z0-9!#$%&'*+\/=?^_`{|}~-]+)*@(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?.)+(?:[a-zA-Z]{2}|com|org|net|gov|mil|biz|info|mobi|name|aero|jobs|museum)/ > data> http:/www.flickr.com/slideShow/index.gne?group_id=&user_id=69845378@REDACTED > Error -8 > > "-8" will happen if the match() call counter reaches some limit (by > default, 10000000). The comments in the header file explain that "the > limit exists in order to catch runaway regular expressions that take > for ever to determine that they do not match." > > In Erlang, after the regexp matching has performed a number of operations, > it'll be swapped out. When the regexp matching is resumed, the match() > counter is zero'ed. I'm not sure why this is done but removing it at > least allows the match to return: > > 1> fail:fail(). > about to run... > Ret is nomatch > ok > > > diff --git a/erts/emulator/pcre/pcre_exec.c b/erts/emulator/pcre/pcre_exec.c > index 5162513..3fe13ca 100644 > --- a/erts/emulator/pcre/pcre_exec.c > +++ b/erts/emulator/pcre/pcre_exec.c > @@ -5191,7 +5191,6 @@ for(;;) > EDEBUGF(("Loop limit break detected")); > return PCRE_ERROR_LOOP_LIMIT; > RESTART_INTERRUPTED: > - md->match_call_count = 0; > md->loop_limit = extra_data->loop_limit; > rc = match(NULL,NULL,NULL,0,md,0,NULL,0,0); > *extra_data->loop_counter_return = > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From mail@REDACTED Tue Apr 20 13:57:00 2010 From: mail@REDACTED (Tim Fletcher) Date: Tue, 20 Apr 2010 04:57:00 -0700 (PDT) Subject: testing asynchronous code In-Reply-To: References: <07971735-2e27-4935-a546-4919c2150749@u21g2000yqc.googlegroups.com> Message-ID: <79cd7c55-9547-4dbb-8111-5782043c0790@m38g2000vbr.googlegroups.com> > I take it the basic idea is to unit test all the underlying > synchronous code separately, and then assert that if the > low-level code works, and all the casts are going out as > specified, then the high-level code is correct? Yes. By replacing your gen_server with a test double you can programmatically test your high-level calling code independently from your low-level synchronous code. Tim From psa@REDACTED Tue Apr 20 13:25:38 2010 From: psa@REDACTED (=?UTF-8?B?UGF1bG8gU8KOZXJnaW8gQWxtZWlkYQ==?=) Date: Tue, 20 Apr 2010 12:25:38 +0100 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> Message-ID: <4BCD8F32.2030606@di.uminho.pt> Hi all, I have not used much parameterized modules, but was very delighted when did do, and I think they gives us something, in practice. Richard O'Keefe wrote: > I pointed out that modules with parameters give us nothing > that plain old closures don't, in response to which Closures must be passed somehow, if they are to be used in a function. If I am writing many functions which use some nonrelated parameters, some closures are being passed around. That is already too much pollution for my tastes. >> Trivial, but requires a lot of boilerplate code and certainly isn't >> any easier to understand or debug than parameterized modules. It also >> becomes impossible to write a useful type spec if you use closures >> like that. > > (a) Such boilerplate code as is required can be automatically generated. If some code would need to be generated if I didn't have parameterized modules, then parameterized modules already give me something. Let's look at some examples (from a homemade "database" I wrote some time ago). The more simple use may be for substituting constants with something calculated at runtime; e.g. Instead of need_dump(Tab, LogOps) -> LogOps > ?DUMPLIMIT * ets:info(Tab, size). I can have need_dump(Tab, LogOps) -> LogOps > DumpLimit * ets:info(Tab, size). with no change in the interface of the function. But it becomes much more interesting when modules themselves are used as parameters of other modules. The first version had a constant directory path defined and used by a module pets_lib that served as a sort of general library. Then other modules used pets_lib. For example, a module pets_tm would have somewhere: Res = pets_lib:delete_table(Tab), How do I make the path (as well as many other configuration parameters) a value chosen at runtime, with little effort in rewriting the code which didn?t contemplate such possibility beforehand? Making pets_lib a parameterized module. What is the impact of that on client code? A simple change to: Res = Lib:delete_table(Tab), It looks pretty much the same, but now we have this Lib variable. If we were using closures, the closure would have to be passed somehow (who knows how many levels of invocations) until is was available to the function which performs this invocation. But if the pets_lib instantiation is a parameter of pets_tm, then I can use statements like the above all over pets_tm by doing a simple: :%s/pets_lib/Lib/g and making pets_tm parameterized; e.g. -module(pets_tm, [Lib, DumpLimit]). Nothing much changes, code is basically reused, we are programming mostly with plain old functions. The parameters that represent modules can even have pleasant names, as we do not need to worry about module namespace pollution in client code and we can rename modules easily without that having much impact on client code. Then we only need to glue modules together at service starting time; e.g. start_link() -> start_link(#pets_config{dir="PETS", gc={100,0.1,0.3}, sync_delay=1000}). init(#pets_config{dir=Dir, gc={MT, CR, PR}, sync_delay=SyncDelay}) -> process_flag(trap_exit, true), pets_locker:start_link(), Lib = pets_lib:new(Dir), Tabs = Lib:init(), (pets_gc:new(Lib, MT, CR, PR)):start_link(), Inserters = 2 * erlang:system_info(schedulers), LastTid = (pets_loader:new(Lib, ?FILE_READERS, Inserters)):load_tables(Tabs), (pets_writer:new(Lib, SyncDelay)):start_link(), (pets_tm:new(Lib, ?DUMP_LIMIT)):start_link(LastTid), {ok, nostate}. (Ironically DumpLimit is still a define here in the "main". ;)) I think all this is nice and very practical. If languages like ML have first class modules and that is ok, what is the problem in Erlang adopting this nice looking concept? Btw there is something which irritates me in the above: Having to write parenthesis as in: (pets_gc:new(Lib, MT, CR, PR)):start_link(), it should be possible to just write: pets_gc:new(Lib, MT, CR, PR):start_link(), (and I profoundly hate the "new" keyword, but that is another story) Best Regards, Paulo From bgustavsson@REDACTED Tue Apr 20 14:42:32 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 20 Apr 2010 14:42:32 +0200 Subject: [erlang-questions] Regular Expressions Problems In-Reply-To: References: <20100420014919.GA13474@ecn.lan> Message-ID: On Tue, Apr 20, 2010 at 12:28 PM, wrote: > Yes! You are absolutely right. You definitely saved me some time debugging > this! A huge THANKS! > > I don't know why I did reset the variable, it seems like a silly thing to do > now that You've pointed out the error :) > > I'll add the patch to GIT with you as the author if that's OK. Now included in 'pu': http://github.com/erlang/otp/commit/78a48004ed9e488de702baa63763ce6e6b602a93 Michael, Email me if you don't want it committed in your name or if you want to revise the commit message. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From paul.joseph.davis@REDACTED Tue Apr 20 14:46:25 2010 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Tue, 20 Apr 2010 08:46:25 -0400 Subject: [erlang-questions] suggestion: NIF_D In-Reply-To: <4BCD5ABF.5070900@erlang-solutions.com> References: <4BCD5ABF.5070900@erlang-solutions.com> Message-ID: On Tue, Apr 20, 2010 at 3:41 AM, Ulf Wiger wrote: > Last week I was at the ACCU 2010 conference in Oxford, and > had a short discussion with Walter Bright, the inventor of D. > The discussion briefly touched the ErlOCaml project, and the > idea that one would like to have the option to drop into a > "safe" low-level language from Erlang. Walter (perhaps un- > surprisingly) suggested that D might be a suitable candidate. > > I hadn't looked closer at D - only noted that it tended to > do very well in the shootout (no more, it seems, as I couldn't > find it there at all now). However, it does look very nice and > accessible to C programmers: > > http://www.digitalmars.com/d/2.0/ctod.html > > Safety would come partly from the fact that it's a garbage- > collected language, and partly because it fixes many of the > idiosynchracies of C and C++. > > Another nice aspect of D is that it is designed to be > relatively easy to analyse lexically, and compiles very > quickly. This seems to open up for some fairly funky inlining > or JIT code generation, but perhaps we're getting ahead of > ourselves here... > > However, I realise that I spend far too little time coding > C et al to be the one to try this integration, or indeed > even to judge whether it would be worthwhile to do so. > > Any other takers? Opinions? > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > http://www.erlang-solutions.com > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG > SOLUTIONS LTD. > > www.erlang-solutions.com > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > Ulf, Two issues come to mind. NIF execution ties up an entire scheduler, which means that embedded language environments are fairly limited in what they can do. The two JavaScript embeddings I know of both focus on use for small quickly executed scripts to serve merely as a convenience and/or sandbox for user code on servers. Secondly, both JavaScript interpreter embeddings use less than 1K lines of code each. One of the reasons I think that NIF's are so cool is that they really don't require a lot of code to plug in different libraries. So unless the D is really well integrated it most likely wouldn't be worth the hurdle as a user of the NIF. Two ways I can think of to make language embeddings more interesting would be to somehow allow NIF's to create a pid that could send and receive messages in a background thread or to have some sort of automagical dispatch to a thread pool. HTH, Paul Davis From dmercer@REDACTED Tue Apr 20 15:30:43 2010 From: dmercer@REDACTED (David Mercer) Date: Tue, 20 Apr 2010 08:30:43 -0500 Subject: [erlang-questions] performance vs msg queue length In-Reply-To: <4BCCA314.8020004@erlang-solutions.com> References: <4BCCA314.8020004@erlang-solutions.com> Message-ID: On Monday, April 19, 2010, Ulf Wiger wrote: > Are you sure that the work generated by a message doesn't lead to > e.g. gen_server calls to other processes. Any such call would lead > to a selective receive operation, which would have to scan the > message queue. > > We happened to be discussing this at the SF Erlang Factory (Patrik, > here's your reminder): in a case such as > > call(P, Req) -> > MRef = erlang:monitor(process, P), > % Mark > P ! {'$call', {self(),MRef}, Req}, > receive > {MRef, Reply} -> Reply; > {'DOWN', MRef, _, _, Reason} -> erlang:error(Reason) > after 5000 -> > erlang:error(timeout) > end. > > the compiler could figure out that it could skip to the > end of the message queue (the end as of % Mark above), > since the following receive clause cannot possibly match any > messages that came in before this point (they all match on > MRef which is a new and unique object). Does that handle all the OTP selective receives (e.g., gen_server, mnesia, etc.)? That is, do all OTP selective receives follow a call to erlang:monitor/2 and then receive matching on MRef in the same function? Am thinking that, if so, the rule could be made implicit regarding selective receives following erlang:monitor's in the same function. Otherwise, a more explicit approach might be necessary. Some sort of mark_queue function, which returns a reference which can then be inserted into a receive call -- something like: Qmark = erlang:mark_queue(), receive from Qmark . . . Cheers, DBM From vladdu55@REDACTED Tue Apr 20 16:04:47 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 20 Apr 2010 16:04:47 +0200 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: <4BCD8F32.2030606@di.uminho.pt> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> Message-ID: Hi, There have been many interesting arguments for and against parametrized modules and I will add a couple of my own, that reflect my feelings about them. I am lazy and will not quote the original text, I hope that won't create misunderstandings. * Of course it's easier to use module parameters instead of passing closures or state around. Just like it's easier to use global variables instead of function arguments. That is, as long as you can remember them all and until something breaks because of a change in a seemingly unrelated module... There's a reason some people like the functional style better. * Personally, I don't think it's a good idea to try to graft features on a language, for which it wasn't designed from the start. Most of the time the result is an ugly hybrid. It feels a lot better to implement different languages targeting the platform, each matching a different style. * Parametrized modules aren't the same thing as modules as a data type. At the moment, they are basically syntax sugar and there are some weird effects (bugs!) because of that: > {lists,hello,world}:reverse([1,2,3]). [3,2,1|{lists,hello,world}] The (higher-level) reason for this bug is just that there is no module data type to query if it is parametrized or not. best regards, Vlad From g@REDACTED Tue Apr 20 16:16:11 2010 From: g@REDACTED (Garrett Smith) Date: Tue, 20 Apr 2010 09:16:11 -0500 Subject: API design with parameterized modules Message-ID: The recent thread on parameterized modules is interesting. There appear to be camps :) Assuming that parameterized modules are here to stay (officially supported part of the language), what does this mean for API design? Are there any clear/obvious design guidelines that would help a designer know when to use a parameterized module over the form of module + state? It seems to me that the driver would be whether the traditional "module state" (e.g. the dict record passed around) can be divided into a static part that doesn't change with the instantiation of the parameterized module and a dynamic state that does. Given that you can fairly easily include both "types" of state in a single record, at what point does one decide to parameterize a module? Or is this just a matter of personal style that library users will have to adapt to? Garrett From ulf.wiger@REDACTED Tue Apr 20 16:27:54 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 20 Apr 2010 16:27:54 +0200 Subject: [erlang-questions] performance vs msg queue length In-Reply-To: References: <4BCCA314.8020004@erlang-solutions.com> Message-ID: <4BCDB9EA.8080206@erlang-solutions.com> David Mercer wrote: > Does that handle all the OTP selective receives (e.g., gen_server, mnesia, > etc.)? That is, do all OTP selective receives follow a call to > erlang:monitor/2 and then receive matching on MRef in the same function? > > Am thinking that, if so, the rule could be made implicit regarding selective > receives following erlang:monitor's in the same function. Otherwise, a more > explicit approach might be necessary. Some sort of mark_queue function, > which returns a reference which can then be inserted into a receive call -- > something like: > > Qmark = erlang:mark_queue(), > receive from Qmark > . . . One of the things being debated is how much rope should be given to the programmers, that they may hang themselves with. One great advantage of having the compiler do it internally is that it would only do so if in cases where it is absolutely safe to do so. I think the pattern would work in most, but not necessarily all, of the places where selective receive is required. For the remaining cases, I'm not convinced - off the top of my head - that an explicit mark_queue() function would help either (but that's from spending 10 seconds thinking about it, so I may well be wrong.) BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From dmercer@REDACTED Tue Apr 20 17:00:14 2010 From: dmercer@REDACTED (David Mercer) Date: Tue, 20 Apr 2010 10:00:14 -0500 Subject: [erlang-questions] performance vs msg queue length In-Reply-To: <4BCDB9EA.8080206@erlang-solutions.com> References: <4BCCA314.8020004@erlang-solutions.com> <4BCDB9EA.8080206@erlang-solutions.com> Message-ID: On Tuesday, April 20, 2010, Ulf Wiger wrote: > One of the things being debated is how much rope should be given > to the programmers, that they may hang themselves with. One great > advantage of having the compiler do it internally is that it > would only do so if in cases where it is absolutely safe to do > so. > > I think the pattern would work in most, but not necessarily all, > of the places where selective receive is required. For the > remaining cases, I'm not convinced - off the top of my head - > that an explicit mark_queue() function would help either > (but that's from spending 10 seconds thinking about it, so I > may well be wrong.) Unfortunately, my spot-check of about 30 selective receives within the OTP library indicates that a receive after an erlang:monitor is the exception rather than the rule. I may have just been unlucky, and I'm sure ROK could put his code-scanner to work to tell us exactly how often it occurs. Cheers, DBM From ulf.wiger@REDACTED Tue Apr 20 17:00:43 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 20 Apr 2010 17:00:43 +0200 Subject: [erlang-questions] suggestion: NIF_D In-Reply-To: References: <4BCD5ABF.5070900@erlang-solutions.com> Message-ID: <4BCDC19B.90505@erlang-solutions.com> Hi Paul, My assumption from what I've read is that the D runtime gives minimal overhead, and the integration is a matter of declaring the C API as EXTERNAL C (D-friendly declarations). But this is just from browsing. I am looking for concrete feedback if possible. BR, Ulf W Paul Davis wrote: > > Ulf, > > Two issues come to mind. NIF execution ties up an entire scheduler, > which means that embedded language environments are fairly limited in > what they can do. The two JavaScript embeddings I know of both focus > on use for small quickly executed scripts to serve merely as a > convenience and/or sandbox for user code on servers. > > Secondly, both JavaScript interpreter embeddings use less than 1K > lines of code each. One of the reasons I think that NIF's are so cool > is that they really don't require a lot of code to plug in different > libraries. So unless the D is really well integrated it most likely > wouldn't be worth the hurdle as a user of the NIF. > > Two ways I can think of to make language embeddings more interesting > would be to somehow allow NIF's to create a pid that could send and > receive messages in a background thread or to have some sort of > automagical dispatch to a thread pool. > > HTH, > Paul Davis -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From mazen.harake@REDACTED Tue Apr 20 17:15:30 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 20 Apr 2010 17:15:30 +0200 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> Message-ID: <4BCDC512.1060200@erlang-solutions.com> This boggles me: If you create a new "instance" of a module with a parameter, then you need to have that "instance variable" around to call it right? So you need to keep it somewhere... as a state. So in any case you need to keep a state, in other words you haven't achieved anything (except perhaps making your code more obfuscated). So _why_ is it ["of course"] "easier to use module parameters instead of passing closures or state around"? I must be missing some important detail somewhere... /Mazen On 20/04/2010 16:04, Vlad Dumitrescu wrote: > Hi, > > There have been many interesting arguments for and against > parametrized modules and I will add a couple of my own, that reflect > my feelings about them. I am lazy and will not quote the original > text, I hope that won't create misunderstandings. > > * Of course it's easier to use module parameters instead of passing > closures or state around. Just like it's easier to use global > variables instead of function arguments. That is, as long as you can > remember them all and until something breaks because of a change in a > seemingly unrelated module... There's a reason some people like the > functional style better. > > * Personally, I don't think it's a good idea to try to graft features > on a language, for which it wasn't designed from the start. Most of > the time the result is an ugly hybrid. It feels a lot better to > implement different languages targeting the platform, each matching a > different style. > > * Parametrized modules aren't the same thing as modules as a data > type. At the moment, they are basically syntax sugar and there are > some weird effects (bugs!) because of that: > > {lists,hello,world}:reverse([1,2,3]). > [3,2,1|{lists,hello,world}] > The (higher-level) reason for this bug is just that there is no module > data type to query if it is parametrized or not. > > best regards, > Vlad > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From dmercer@REDACTED Tue Apr 20 17:17:55 2010 From: dmercer@REDACTED (David Mercer) Date: Tue, 20 Apr 2010 10:17:55 -0500 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> Message-ID: On Tuesday, April 20, 2010, Vlad Dumitrescu wrote: > * Personally, I don't think it's a good idea to try to graft features > on a language, for which it wasn't designed from the start. Most of > the time the result is an ugly hybrid. It feels a lot better to > implement different languages targeting the platform, each matching a > different style. I agree, preferring a small, succinct language to a larger language providing many ways to do things (e.g., Scheme vs. Common Lisp). A smaller language results in a smaller set of more common idioms that are easily recognized and understood by programmers. It also reduces the barrier to entry for new programmers, who can easily grasp the whole language and easily read other people's code sooner. I would advocate, therefore, keeping parameterized modules on the fringes, without official blessing. By withholding official blessing, it makes programmers consider alternative -- more common and idiomatic -- ways of doing things. Steve Davis described a couple of months ago why he has tended to convert parameterized modules to unparameterized modules over time: . That was actually an interesting thread on this subject, and I gained some good insight into the use of parameterized modules. Cheers, DBM From roberto@REDACTED Tue Apr 20 17:53:44 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 20 Apr 2010 17:53:44 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: <8D18002A-6ADA-4122-983B-C1C636C700A6@rogvall.se> References: <19BD41F6-E388-4D73-9D5C-40350A4CE02F@rogvall.se> <8D18002A-6ADA-4122-983B-C1C636C700A6@rogvall.se> Message-ID: 2010/4/19 Tony Rogvall : > The packet type for http headers is 'httph' (not the 'h' appended) > So you need to switch packet type after you received the request in 'http' mode > > /Tony thank you tony, this does it. r. From rvirding@REDACTED Tue Apr 20 17:54:03 2010 From: rvirding@REDACTED (Robert Virding) Date: Tue, 20 Apr 2010 17:54:03 +0200 Subject: [erlang-questions] suggestion: NIF_D In-Reply-To: References: <4BCD5ABF.5070900@erlang-solutions.com> Message-ID: On 20 April 2010 14:46, Paul Davis wrote: > Ulf, > > Two issues come to mind. NIF execution ties up an entire scheduler, > which means that embedded language environments are fairly limited in > what they can do. The two JavaScript embeddings I know of both focus > on use for small quickly executed scripts to serve merely as a > convenience and/or sandbox for user code on servers. > > Secondly, both JavaScript interpreter embeddings use less than 1K > lines of code each. One of the reasons I think that NIF's are so cool > is that they really don't require a lot of code to plug in different > libraries. So unless the D is really well integrated it most likely > wouldn't be worth the hurdle as a user of the NIF. > > Two ways I can think of to make language embeddings more interesting > would be to somehow allow NIF's to create a pid that could send and > receive messages in a background thread or to have some sort of > automagical dispatch to a thread pool. Isn't this just for what you use ports, with or without linked drivers? The port abstraction *IS* a process with which you communicate using messages, although the process is not an erlang process but some external process. The port_command interface tends to hide this though. To me this is the distinction between NIFs and ports: NIFs behave like function calls and ports like asynchronous processes. N.B. many OTP interfaces hide the message communication behind a function call, for example gen_server, but the messages are still there. Robert From roberto@REDACTED Tue Apr 20 18:04:31 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 20 Apr 2010 18:04:31 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: 2010/4/19 Ingela Andin : > Hi! > > We recently accepted a patch for http packet mode e.i: > http://github.com/erlang/otp/commit/12da3796463c7d02999985ae15e98dcaa4d2bdb4 > > Do you have that patch? The offical ssl-release ssl-3.11 where this > patch has been > included is not yet released other then to git-repository. > > Regards Ingela Erlang/OTP team, Ericsson AB hi ingela, no i do not have this patch but i've decided to go for the httpd option in ssl. however, there's still a question i need to solve. 1. in https mode [ssl:transport_accept/1 + ssl:ssl_accept/1], if i receive a http request, i get a {error, closed} message, i.e. a socket is not created, thus the browser timeouts waiting for a response. 2. in http mode [gen_tcp:accept/1], if i receive a https request then i get a {http, Sock, {http_error, [22,3,1,0,157,1,0,0, ......]}} message, i.e. the socket is created but the data is not understood. i would like to respond with a 403 error in case 1 and a 404 error in case 2. what is the best way to handle this? thank you, r. From roberto@REDACTED Tue Apr 20 18:15:58 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 20 Apr 2010 18:15:58 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: > 1. > in https mode [ssl:transport_accept/1 + ssl:ssl_accept/1], if i > receive a http request, i get a {error, closed} message, i.e. a socket > is not created, thus the browser timeouts waiting for a response. > > 2. > in http mode [gen_tcp:accept/1], if i receive a https request then i get a > {http, Sock, {http_error, [22,3,1,0,157,1,0,0, ......]}} message, i.e. > the socket is created but the data is not understood. my bad, in situation 1 nothing happens, there's no {error, closed} message received. question still remains: what is the best way to handle this? r. From michael.santos@REDACTED Tue Apr 20 18:28:03 2010 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 20 Apr 2010 12:28:03 -0400 Subject: [erlang-questions] Regular Expressions Problems In-Reply-To: References: <20100420014919.GA13474@ecn.lan> Message-ID: <20100420162803.GB13474@ecn.lan> On Tue, Apr 20, 2010 at 02:42:32PM +0200, Bj??rn Gustavsson wrote: > On Tue, Apr 20, 2010 at 12:28 PM, wrote: > > Yes! You are absolutely right. You definitely saved me some time debugging > > this! A huge THANKS! > > > > I don't know why I did reset the variable, it seems like a silly thing to do > > now that You've pointed out the error :) > > > > I'll add the patch to GIT with you as the author if that's OK. > > Now included in 'pu': > > http://github.com/erlang/otp/commit/78a48004ed9e488de702baa63763ce6e6b602a93 > > Michael, > > Email me if you don't want it committed in your name or if you want to > revise the > commit message. Awesome, thanks! From psa@REDACTED Tue Apr 20 18:46:29 2010 From: psa@REDACTED (=?UTF-8?B?UGF1bG8gU8KOZXJnaW8gQWxtZWlkYQ==?=) Date: Tue, 20 Apr 2010 17:46:29 +0100 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> Message-ID: <4BCDDA65.5000706@di.uminho.pt> Hi, Vlad Dumitrescu wrote: > * Of course it's easier to use module parameters instead of passing > closures or state around. Just like it's easier to use global > variables instead of function arguments. That is, as long as you can > remember them all and until something breaks because of a change in a > seemingly unrelated module... There's a reason some people like the > functional style better. Some of the nice things in functional style are absence of side effects and referential transparency. Updating global vars may be a problem, but we are not talking about that. Using parameterized modules will not make us have global (mutable) variables or side effects (more than we already have in the language, which is not purely functional), neither will we loose referential transparency. This has nothing to do with loosing functional style. I am not talking about emulating objects and programming with a different style; I see an important use of parameterized modules to avoid having to thread all over things that for all practical purposes are "constants". People say the difference between: E:f(...) and m:f(..., E) is not much and so whats the fuss. Its not this difference that may be significant, but to have to propagate, split, and reconstruct the components that make up E until they reach this point in the program, meanwhile polluting and obfuscating "innocent" functions in the invocation chain. > * Personally, I don't think it's a good idea to try to graft features > on a language, for which it wasn't designed from the start. Most of > the time the result is an ugly hybrid. It feels a lot better to > implement different languages targeting the platform, each matching a > different style. From this reasoning we would not have funs as they did not exist from the start. But anyway, parameterized modules are pretty orthogonal to the main features of the language. > * Parametrized modules aren't the same thing as modules as a data > type. At the moment, they are basically syntax sugar and there are Why not? I can already pass them around and store them in data structures, like I do for closures; what remains to be added is proper support so that modules are a new runtime type and I can do type testing, e.g. is_module/1 and so on. > some weird effects (bugs!) because of that: > > {lists,hello,world}:reverse([1,2,3]). > [3,2,1|{lists,hello,world}] > The (higher-level) reason for this bug is just that there is no module > data type to query if it is parametrized or not. This is not a problem with the concept itself but with the (lack) of a proper implementation. Some strange things could also happen before proper support for funs was added. I don?t know what the plans are, but yes, it would be nice to have them properly supported and not as the hack they are now. Regards, Paulo From paul.joseph.davis@REDACTED Tue Apr 20 19:16:56 2010 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Tue, 20 Apr 2010 13:16:56 -0400 Subject: [erlang-questions] suggestion: NIF_D In-Reply-To: <4BCDC19B.90505@erlang-solutions.com> References: <4BCD5ABF.5070900@erlang-solutions.com> <4BCDC19B.90505@erlang-solutions.com> Message-ID: On Tue, Apr 20, 2010 at 11:00 AM, Ulf Wiger wrote: > > Hi Paul, > > My assumption from what I've read is that the D runtime gives > minimal overhead, and the integration is a matter of declaring > the C API as EXTERNAL C (D-friendly declarations). > > But this is just from browsing. I am looking for concrete > feedback if possible. Ulf, Sorry, I wrote that a bit too early. Reading it I realize I was being a bit schizophrenic on how I was considering the use of D in conjunction with Erlang. In the case of "lets take a look at writing NIF's with D" I was trying to say that the scope of a NIF is somewhat limited by the fact that it must return quickly or risk locking the entire Erlang VM. Due to this, the LOC in the NIF's I've seen is quite small (1K is a big NIF). With such a small foot print I wouldn't expect the benefits of using a language like D to outweigh the costs of depending on its build chain. In the case of "lets take a look at providing a platform to run arbitrary D in Erlang" I reckon that could be pretty awesome but I am unsure whether it would be enough awesome to depend on whatever build/runtime dependencies it required. Paul > BR, > Ulf W > > Paul Davis wrote: >> >> Ulf, >> >> Two issues come to mind. NIF execution ties up an entire scheduler, >> which means that embedded language environments are fairly limited in >> what they can do. The two JavaScript embeddings I know of both focus >> on use for small quickly executed scripts to serve merely as a >> convenience and/or sandbox for user code on servers. >> >> Secondly, both JavaScript interpreter embeddings use less than 1K >> lines of code each. One of the reasons I think that NIF's are so cool >> is that they really don't require a lot of code to plug in different >> libraries. So unless the D is really well integrated it most likely >> wouldn't be worth the hurdle as a user of the NIF. >> >> Two ways I can think of to make language embeddings more interesting >> would be to somehow allow NIF's to create a pid that could send and >> receive messages in a background thread or to have some sort of >> automagical dispatch to a thread pool. >> >> HTH, >> Paul Davis > > > -- > Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > http://www.erlang-solutions.com > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG > SOLUTIONS LTD. > > www.erlang-solutions.com > > From paul.joseph.davis@REDACTED Tue Apr 20 19:23:49 2010 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Tue, 20 Apr 2010 13:23:49 -0400 Subject: [erlang-questions] suggestion: NIF_D In-Reply-To: References: <4BCD5ABF.5070900@erlang-solutions.com> Message-ID: On Tue, Apr 20, 2010 at 11:54 AM, Robert Virding wrote: > On 20 April 2010 14:46, Paul Davis wrote: >> Ulf, >> >> Two issues come to mind. NIF execution ties up an entire scheduler, >> which means that embedded language environments are fairly limited in >> what they can do. The two JavaScript embeddings I know of both focus >> on use for small quickly executed scripts to serve merely as a >> convenience and/or sandbox for user code on servers. >> >> Secondly, both JavaScript interpreter embeddings use less than 1K >> lines of code each. One of the reasons I think that NIF's are so cool >> is that they really don't require a lot of code to plug in different >> libraries. So unless the D is really well integrated it most likely >> wouldn't be worth the hurdle as a user of the NIF. >> >> Two ways I can think of to make language embeddings more interesting >> would be to somehow allow NIF's to create a pid that could send and >> receive messages in a background thread or to have some sort of >> automagical dispatch to a thread pool. > > Isn't this just for what you use ports, with or without linked > drivers? The port abstraction *IS* a process with which you > communicate using messages, although the process is not an erlang > process but some external process. The port_command interface tends to > hide this though. > > To me this is the distinction between NIFs and ports: NIFs behave like > function calls and ports like asynchronous processes. N.B. many OTP > interfaces hide the message communication behind a function call, for > example gen_server, but the messages are still there. > > Robert > Robert, I was trying to come up with an argument or example of something that I'd want to be able to do from a NIF that I couldn't using a port and nothing came to mind. The only reason it ever occurred to me to want a pid-like capability was the availability of threads to NIF's. I had been trying to figure out how I might embed a scripting language yet avoid potential VM dead locks and the message passing interface would have been a nice solution. Though I don't think this is a particularly good reason to add such an interface, it was just one of those "so close, yet so very far away," moments. Paul From ulf.wiger@REDACTED Tue Apr 20 19:38:43 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 20 Apr 2010 19:38:43 +0200 Subject: [erlang-questions] suggestion: NIF_D In-Reply-To: References: <4BCD5ABF.5070900@erlang-solutions.com> <4BCDC19B.90505@erlang-solutions.com> Message-ID: <4BCDE6A3.1080908@erlang-solutions.com> Paul Davis wrote: > > In the case of "lets take a look at writing NIF's with D" I was trying > to say that the scope of a NIF is somewhat limited by the fact that it > must return quickly or risk locking the entire Erlang VM. Due to this, > the LOC in the NIF's I've seen is quite small (1K is a big NIF). With > such a small foot print I wouldn't expect the benefits of using a > language like D to outweigh the costs of depending on its build chain. Well, the requirement to return quickly doesn't necessarily mean that the code size will be small. The line count of beam/erl_db* (the ETS BIFs) sums up to 11K. :) I agree that the NIFs to-date have mainly been small. I fully expect to see some large ones in the near future. > In the case of "lets take a look at providing a platform to run > arbitrary D in Erlang" I reckon that could be pretty awesome but I am > unsure whether it would be enough awesome to depend on whatever > build/runtime dependencies it required. This is a real concern, of course. Not having used D, I have no clear intuition for the pain level this would cause, but I can definitely imagine that it could be significant. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From mikpe@REDACTED Tue Apr 20 20:40:07 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Tue, 20 Apr 2010 20:40:07 +0200 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: <4BCDC512.1060200@erlang-solutions.com> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> <4BCDC512.1060200@erlang-solutions.com> Message-ID: <19405.62727.635075.953915@pilspetsen.it.uu.se> Mazen Harake writes: > This boggles me: > > If you create a new "instance" of a module with a parameter, then you > need to have that "instance variable" around to call it right? So you > need to keep it somewhere... as a state. > > So in any case you need to keep a state, in other words you haven't > achieved anything (except perhaps making your code more obfuscated). > > So _why_ is it ["of course"] "easier to use module parameters instead of > passing closures or state around"? > > I must be missing some important detail somewhere... Consider the parts of the code that uses the module parameters. From jderick@REDACTED Tue Apr 20 20:41:47 2010 From: jderick@REDACTED (John Erickson) Date: Tue, 20 Apr 2010 11:41:47 -0700 Subject: [erlang-questions] performance vs msg queue length In-Reply-To: References: <4BCCA314.8020004@erlang-solutions.com> <4BCDB9EA.8080206@erlang-solutions.com> Message-ID: Good to know about mnesia and gen_server doing selective receives, but I am not using either. BTW, I found some other evidence of this slowdown from an earlier posting. Although the main post was about interpreter speed, I found these benchmark numbers quite strange for the compiled version as well. With message queues of 10k, it looks like about 3x slowdown. [begin [ self() ! a || _ <- lists:seq(1,N) ], > S = now(), > [ receive _ -> ok end || _ <- lists:seq(1,N) ], > io:format("~w~n", [timer:now_diff(now(), S) / N]) > end || N <- [1,10,100,1000,10000]]. > > ... > > If I put it into a module, add 1e5 and 1e6 then I get: > > 1> test:test(). > 1.0 > 0.1 > 0.06 > 0.114 > 0.1554 > 0.36398 > 0.358997 > On Tue, Apr 20, 2010 at 8:00 AM, David Mercer wrote: > On Tuesday, April 20, 2010, Ulf Wiger wrote: > > > One of the things being debated is how much rope should be given > > to the programmers, that they may hang themselves with. One great > > advantage of having the compiler do it internally is that it > > would only do so if in cases where it is absolutely safe to do > > so. > > > > I think the pattern would work in most, but not necessarily all, > > of the places where selective receive is required. For the > > remaining cases, I'm not convinced - off the top of my head - > > that an explicit mark_queue() function would help either > > (but that's from spending 10 seconds thinking about it, so I > > may well be wrong.) > > Unfortunately, my spot-check of about 30 selective receives within the OTP > library indicates that a receive after an erlang:monitor is the exception > rather than the rule. I may have just been unlucky, and I'm sure ROK could > put his code-scanner to work to tell us exactly how often it occurs. > > Cheers, > > DBM > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From vladdu55@REDACTED Tue Apr 20 21:04:43 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 20 Apr 2010 21:04:43 +0200 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: <4BCDC512.1060200@erlang-solutions.com> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> <4BCDC512.1060200@erlang-solutions.com> Message-ID: On Tue, Apr 20, 2010 at 17:15, Mazen Harake wrote: > If you create a new "instance" of a module with a parameter, then you need > to have that "instance variable" around to call it right? So you need to > keep it somewhere... as a state. > > So in any case you need to keep a state, in other words you haven't achieved > anything (except perhaps making your code more obfuscated). > > So _why_ is it ["of course"] "easier to use module parameters instead of > passing closures or state around"? Hi, If you send state around the functional way, it has to be a parameter to all the functions that use some part of that state. If the state data changes, it may be needed to update all these places (if the state representation wasn't flexible enough). If the state is globally accessible, you just use the appropriate variable where it is needed. regards, Vlad From mononcqc@REDACTED Tue Apr 20 21:09:38 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 20 Apr 2010 15:09:38 -0400 Subject: [erlang-questions] performance vs msg queue length In-Reply-To: References: <4BCCA314.8020004@erlang-solutions.com> <4BCDB9EA.8080206@erlang-solutions.com> Message-ID: On Tue, Apr 20, 2010 at 2:41 PM, John Erickson wrote: > Good to know about mnesia and gen_server doing selective receives, but I am > not using either. BTW, I found some other evidence of this slowdown from > an > earlier posting. Although the main post was about interpreter speed, I > found these benchmark numbers quite strange for the compiled version as > well. With message queues of 10k, it looks like about 3x slowdown. > > [begin [ self() ! a || _ <- lists:seq(1,N) ], > > S = now(), > > [ receive _ -> ok end || _ <- lists:seq(1,N) ], > > io:format("~w~n", [timer:now_diff(now(), S) / N]) > > end || N <- [1,10,100,1000,10000]]. > > > > ... > > > > If I put it into a module, add 1e5 and 1e6 then I get: > > > > 1> test:test(). > > 1.0 > > 0.1 > > 0.06 > > 0.114 > > 0.1554 > > 0.36398 > > 0.358997 > > > > I've ran your benchmark: 1.0 0.2 0.1 0.15 0.1913 0.31713 I've then reworked your benchmark, this time starting a new process for each one and adding a 100k messages round: [spawn(fun() -> [ self() ! a || _ <- lists:seq(1,N) ], S = now(), [ receive _ -> ok end || _ <- lists:seq(1,N) ], io:format("~w: ~w~n", [N,timer:now_diff(now(), S) / N]) end) || N <- [1,10,100,1000,10000,100000]]. If I reorder everything it outputs: 1: 1.0 10: 0.2 100: 0.18 1000: 0.173 10000: 0.1922 100000: 0.18945 In this case, the results are much more consistent and show no trace of degradation when adding messages. Someone who knows the innards of the VM better than me could explain, but it looks like running your test sequentially messes the results up. From matthew@REDACTED Tue Apr 20 21:13:01 2010 From: matthew@REDACTED (Matthew Sackman) Date: Tue, 20 Apr 2010 20:13:01 +0100 Subject: [erlang-questions] performance vs msg queue length In-Reply-To: <4BCCA314.8020004@erlang-solutions.com> References: <4BCCA314.8020004@erlang-solutions.com> Message-ID: <20100420191300.GA31807@wellquite.org> Hi John, Ulf et al, On Mon, Apr 19, 2010 at 08:38:12PM +0200, Ulf Wiger wrote: > John Erickson wrote: > >Hello, I have noticed that the amount of work an erlang process gets done > >seems to decrease as its message queue grows. In extreme cases, this is > >understandable, as when the message queue fills up all available memory. > > However, even for more moderate cases such as ~100 messages each around > >10kB there seems to be significant slowdown. These are in processes that > >are not scanning the msg queue; they have a single receive block which is > >expected to match every message. Is this expected? > > Are you sure that the work generated by a message doesn't lead to > e.g. gen_server calls to other processes. Any such call would lead > to a selective receive operation, which would have to scan the > message queue. Right, which is pretty much the reason why we at RabbitMQ extended gen_server with gen_server2 which, prior to the process_message loop, completely drains the mailbox, shoving everything into a queue. Messages are then pulled out of the queue and processed as required. The advantage is that any :call you subsequently make is likely to result in a selective receive on a short mailbox, thus it performs *much* better. http://hg.rabbitmq.com/rabbitmq-server/file/default/src/gen_server2.erl The things you lose are: a) deprioritisation of processes which send to an overwhelmed process b) potentially spending your entire process's CPU time on draining the mailbox and thus getting no real work done. There are a number of other enhancements for gen_server2 which we've added which are documented in the file, but are not really relevant for this discussion. Matthew From per.melin@REDACTED Tue Apr 20 21:15:12 2010 From: per.melin@REDACTED (Per Melin) Date: Tue, 20 Apr 2010 21:15:12 +0200 Subject: [erlang-questions] performance vs msg queue length In-Reply-To: References: <4BCCA314.8020004@erlang-solutions.com> <4BCDB9EA.8080206@erlang-solutions.com> Message-ID: On Tue, Apr 20, 2010 at 8:41 PM, John Erickson wrote: > [begin [ self() ! a || _ <- lists:seq(1,N) ], >> ? ? ? ?S = now(), >> ? ? ? ?[ receive _ -> ok end || _ <- lists:seq(1,N) ], >> ? ? ? ?io:format("~w~n", [timer:now_diff(now(), S) / N]) >> ?end || N <- [1,10,100,1000,10000]]. >> >> ... >> >> If I put it into a module, add 1e5 and 1e6 then I get: >> >> 1> test:test(). >> 1.0 >> 0.1 >> 0.06 >> 0.114 >> 0.1554 >> 0.36398 >> 0.358997 One little change: [begin L = lists:seq(1,N), [ self() ! a || _ <- L ], S = now(), [ receive _ -> ok end || _ <- L ], io:format("~w~n", [timer:now_diff(now(), S) / N]) end || N <- [1,10,100,1000,10000,100000,1000000]]. 1.0 0.1 0.06 0.091 0.1144 0.11898 0.120905 From fritchie@REDACTED Tue Apr 20 21:46:22 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 20 Apr 2010 14:46:22 -0500 Subject: [erlang-questions] ACM Erlang Workshop 2010: call for papers In-Reply-To: Message of "Mon, 19 Apr 2010 15:17:29 CDT." <95631.1271708249@snookles.snookles.com> Message-ID: <66830.1271792782@snookles.snookles.com> Hi, everyone, sorry about the little extra bit of spam. The HTML version of the call for papers at http://www.erlang.org/workshop/2010/ had last-minute edits that I overlooked when sending plain ASCII version yesterday. The final/finished/photo-ready deadline for papers is Monday, August 2nd, 2010. -Scott From fritchie@REDACTED Tue Apr 20 22:37:40 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 20 Apr 2010 15:37:40 -0500 Subject: [erlang-questions] testing asynchronous code In-Reply-To: Message of "Tue\, 20 Apr 2010 12\:18\:26 +0530." Message-ID: <69562.1271795860@snookles.snookles.com> Martin DeMello wrote: >> Martin, you have another option: your test code calls >> YourModule:handle_cast() directly. ?If your client side stub looks >> like: md> Thanks, some good ideas in there. Not sure I've absorbed it fully, md> but I'll stare at it till it clicks :) From what I can make out, md> though, it doesn't handle the case of handle_cast() itself calling md> cast(), since the intermediate states before the entire tree of md> casts has completed aren't really of interest. Is that correct, or md> have I missed something? You may or may not have missed something, it's tough to tell. If your test code calls YourModule:handle_cast() and checks the return result, that checking is easiest if YourModule:handle_cast() is purely functional. If it has side-effects, then checking for sanity is more difficult. As John Hughes suggested, trace-based tests might be easier if you're dealing with lots of side-effects. It's quite unusual to have a gen_server send itself a message using gen_server:cast(). I don't know why your code would do it... ... but I confess that it is very occasionally useful to use gen_server:cast() or "self() ! {some_message_tag, ReminderData}" to remind yourself to do something that is very inconvenient to deal with at this instant in time. When your call stack pops the normal way and returns control to gen_server's message handling loop, then you can conveniently handle that reminder data via YourModule:handle_cast() or YourModule:handle_info(). i_am_buried_seven_levels_deep_in_my_call_stack(Args, State) -> %% Weird args handling stuff can be better handled by sending %% a (non-blocking!) message to myself. Perhaps this reminder %% message can look the same as a cast message that I have to %% handle anyway. %% %% One case where this is useful is if this function needs to %% modify "State", but the structure of the code (including all %% the intermediate callers between here and handle_cast() is %% inconvenient to refactor to return a new state var. %% Refactoring is usually the better choice ... except when %% it isn't better. It depends. case is_arg_weird(Args, State) of true -> gen_server:cast(self(), {arg_was_weird, Args}); false -> ok end, %% Do normal, down-in-the-depths calculation here, then return %% only an atom. ok. It's like a continuation, except uglier, except when it's only slightly ugly. Depends on the context. :-) -Scott From jderick@REDACTED Wed Apr 21 00:25:44 2010 From: jderick@REDACTED (John Erickson) Date: Tue, 20 Apr 2010 15:25:44 -0700 Subject: [erlang-questions] performance vs msg queue length In-Reply-To: References: <4BCCA314.8020004@erlang-solutions.com> <4BCDB9EA.8080206@erlang-solutions.com> Message-ID: These results are clearly better, but still I'm a bit surprised there is a noticable slowdown, especially from 100-10k. Does anyone know the reason for this? It seems to imply we should try to keep message queues under 100 messages. I also wonder how message size plays into this. On Tue, Apr 20, 2010 at 12:15 PM, Per Melin wrote: > On Tue, Apr 20, 2010 at 8:41 PM, John Erickson wrote: > > [begin [ self() ! a || _ <- lists:seq(1,N) ], > >> S = now(), > >> [ receive _ -> ok end || _ <- lists:seq(1,N) ], > >> io:format("~w~n", [timer:now_diff(now(), S) / N]) > >> end || N <- [1,10,100,1000,10000]]. > >> > >> ... > >> > >> If I put it into a module, add 1e5 and 1e6 then I get: > >> > >> 1> test:test(). > >> 1.0 > >> 0.1 > >> 0.06 > >> 0.114 > >> 0.1554 > >> 0.36398 > >> 0.358997 > > One little change: > > [begin L = lists:seq(1,N), > [ self() ! a || _ <- L ], > S = now(), > [ receive _ -> ok end || _ <- L ], > io:format("~w~n", [timer:now_diff(now(), S) / N]) > end || N <- [1,10,100,1000,10000,100000,1000000]]. > > 1.0 > 0.1 > 0.06 > 0.091 > 0.1144 > 0.11898 > 0.120905 > From matthew@REDACTED Wed Apr 21 00:42:54 2010 From: matthew@REDACTED (Matthew Sackman) Date: Tue, 20 Apr 2010 23:42:54 +0100 Subject: [erlang-questions] performance vs msg queue length In-Reply-To: References: <4BCCA314.8020004@erlang-solutions.com> <4BCDB9EA.8080206@erlang-solutions.com> Message-ID: <20100420224254.GB31807@wellquite.org> On Tue, Apr 20, 2010 at 03:25:44PM -0700, John Erickson wrote: > These results are clearly better, but still I'm a bit surprised there is a > noticable slowdown, especially from 100-10k. Does anyone know the reason > for this? I don't *know*, but my guesses are: 1. The VM deschedules processes sending messages to processes with full mailboxes. I don't know if this extends to a process sending a message to itself - I would hope it didn't extend to that, but then again, sending messages to yourself is a reasonably rare thing to do, so it's possible they've not bothered optimising that case. 2. Allocation of a bigger mailbox. I would expect that it would probably be a standard "everytime we run out of space, double the size of the mailbox", but there are probably a few sensible policies for this. malloc won't be free. 3. You are constructing ever longer lists of 'ok' atoms. Doing a recursive loop with counter would avoid this, and so you would get down to timings of doing the actual receives. This list is probably in reverse order of liklihood. Matthew From bernie@REDACTED Wed Apr 21 00:53:55 2010 From: bernie@REDACTED (Bernard Duggan) Date: Wed, 21 Apr 2010 08:53:55 +1000 Subject: [erlang-questions] performance vs msg queue length In-Reply-To: References: <4BCCA314.8020004@erlang-solutions.com> <4BCDB9EA.8080206@erlang-solutions.com> Message-ID: <4BCE3083.2060608@m5net.com> Actually I'd be kind of inclined to put it down to experimental error, minor internal implementation and system details (size of memory blocks being allocated, use of processor cache and so on) and perhaps other stuff going on in the system. Because my results were: 5.0 0.6 0.24 0.295 0.1545 0.1631 0.160826 Looks like pretty reasonable performance characteristics to me. B John Erickson wrote: > These results are clearly better, but still I'm a bit surprised there is a > noticable slowdown, especially from 100-10k. Does anyone know the reason > for this? It seems to imply we should try to keep message queues under 100 > messages. I also wonder how message size plays into this. > > > On Tue, Apr 20, 2010 at 12:15 PM, Per Melin wrote: > > >> On Tue, Apr 20, 2010 at 8:41 PM, John Erickson wrote: >> >>> [begin [ self() ! a || _ <- lists:seq(1,N) ], >>> >>>> S = now(), >>>> [ receive _ -> ok end || _ <- lists:seq(1,N) ], >>>> io:format("~w~n", [timer:now_diff(now(), S) / N]) >>>> end || N <- [1,10,100,1000,10000]]. >>>> >>>> ... >>>> >>>> If I put it into a module, add 1e5 and 1e6 then I get: >>>> >>>> 1> test:test(). >>>> 1.0 >>>> 0.1 >>>> 0.06 >>>> 0.114 >>>> 0.1554 >>>> 0.36398 >>>> 0.358997 >>>> >> One little change: >> >> [begin L = lists:seq(1,N), >> [ self() ! a || _ <- L ], >> S = now(), >> [ receive _ -> ok end || _ <- L ], >> io:format("~w~n", [timer:now_diff(now(), S) / N]) >> end || N <- [1,10,100,1000,10000,100000,1000000]]. >> >> 1.0 >> 0.1 >> 0.06 >> 0.091 >> 0.1144 >> 0.11898 >> 0.120905 >> >> From bernie@REDACTED Wed Apr 21 00:58:31 2010 From: bernie@REDACTED (Bernard Duggan) Date: Wed, 21 Apr 2010 08:58:31 +1000 Subject: [erlang-questions] performance vs msg queue length In-Reply-To: <20100420224254.GB31807@wellquite.org> References: <4BCCA314.8020004@erlang-solutions.com> <4BCDB9EA.8080206@erlang-solutions.com> <20100420224254.GB31807@wellquite.org> Message-ID: <4BCE3197.20409@m5net.com> Matthew Sackman wrote: > 3. You are constructing ever longer lists of 'ok' atoms. Doing a > recursive loop with counter would avoid this, and so you would get down > to timings of doing the actual receives. > I'd considered that one too, but I think the inner list comprehension falls squarely into the "result obviously not used" category that allows the compiler to optimise away result storage (see the optimisation guide and a thread a month or so back discussing this). Out of curiosity I replaced the list comprehension with a lists:foreach() (which always doesn't store results)and the results were actually marginally (though not hugely) worse across the board. 6> test:comprehension(). 5.0 0.6 0.24 0.295 0.1545 0.1631 0.160826 [ok,ok,ok,ok,ok,ok,ok] 7> test:foreach(). 6.0 0.8 0.33 0.382 0.1846 0.1998 0.195081 Cheers, Bernard From ok@REDACTED Wed Apr 21 03:08:50 2010 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 21 Apr 2010 13:08:50 +1200 Subject: [erlang-questions] Re: floating point syntax In-Reply-To: <201004201947.07593.als@iinet.net.au> References: <201004201947.07593.als@iinet.net.au> Message-ID: On Apr 20, 2010, at 9:47 PM, Anthony Shipman wrote: > On Tue, 20 Apr 2010 07:16:02 am Robert Virding wrote: >> IIRC it was probably more that we/I just didn't see the need to be >> able to write the shortened forms. It isn't that it is much extra you >> have to write. :-) >> >> Robert > > When parsing numbers for other languages it would be nice to be able > to use > fread() to parse them instead of writing my own. Some sort of ~N > form to read > a number, integer or floating point, in the usual syntax would be > nice. What _is_ the "usual syntax"? Here's a floating point number: 1.23@REDACTED (Burroughs Algol) Here's a floating point number: 1.23E45_DOUBLE (Fortran) Here's a floating point number: 1.23D45 (Fortran, Lisp, Smalltalk) Here's a floating point number: 1.23e45L (C) Here's a floating point number: ~12.34 (ML) Here's a floating point number: 1_234_567.0 (Ada, OCaml) Here's a floating point number: 1 234 567.0 (Fortran, Algol 60, Algol 68) Here's a floating point number: 0x12.34p45 (C) Here's a floating point number: 101.101E2B (PL/I) Here's a floating point number: 1.23Q45 (ANSI Smalltalk) For what it's worth, here's the "usual syntax" for floats from the ANSI Smalltalk standard: float ::= mantissa [exponentLetter exponent] mantissa ::= digits ?.? digits digits ::= digit+ exponent ::= [?-?]decimalInteger exponentLetter ::= ?e? | ?d? | ?q? Except for letting the exponent letter vary to indicate the precision of the number (single, double, double extended), this is pretty traditional. (Even "d" is traditional.) Erlang is not alone in requiring a digit each side of the dot. There is so much variation between programming languages that it is extremely hard to cover all of them (and believe me, I've tried). True, Fortran allows numbers with a digit on only one side of the dot, but it also allowed spaces inside numbers; is _that_ "usual syntax"? One really important issue these days is allowing numbers written by people or programs using the comma as the radix point. There are obvious advantages to "being generous in what you accept", but we cannot just appeal to "usual syntax" to determine what that should be. We need something more explicit. From ok@REDACTED Wed Apr 21 04:01:24 2010 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 21 Apr 2010 14:01:24 +1200 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: <4BCD8F32.2030606@di.uminho.pt> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> Message-ID: <3FFFC7AE-F622-4C41-8BFE-95E9B8646535@cs.otago.ac.nz> On Apr 20, 2010, at 11:25 PM, Paulo S?ergio Almeida wrote: > Closures must be passed somehow, if they are to be used in a > function. If I am writing many functions which use some nonrelated > parameters, some closures are being passed around. That is already > too much pollution for my tastes. If they really *are* nonrelated, they should be separate parameters anyway. If they are related, they can be in one data structure (which might be a closure). "Pollution" is not a property of closures, but of a coding style. > >>> Trivial, but requires a lot of boilerplate code and certainly isn't >>> any easier to understand or debug than parameterized modules. It >>> also >>> becomes impossible to write a useful type spec if you use closures >>> like that. >> (a) Such boilerplate code as is required can be automatically >> generated. > > If some code would need to be generated if I didn't have > parameterized modules, then parameterized modules already give me > something. What? Automatic code generation isn't anything you need to be aware of, let alone involved with. > > Let's look at some examples (from a homemade "database" I wrote some > time ago). > > The more simple use may be for substituting constants with something > calculated at runtime; e.g. > > Instead of > > need_dump(Tab, LogOps) -> LogOps > ?DUMPLIMIT * ets:info(Tab, size). > > I can have > > need_dump(Tab, LogOps) -> LogOps > DumpLimit * ets:info(Tab, size). > > with no change in the interface of the function. But this is the Functional Programming Lesson: there *is* a change in the interface of the function! In the first case, the function uses nothing but its arguments. In the second case, the function has an extra parameter (DumpLimit). The function is _really_ need_dump(%Hidden%, Tab, LogOps) -> LogOps > %Hidden%#%hidden%.DumpLimit * ets:info(Tab, size). How it's _compiled_ is a separate issue; I'm arguing about what it _means_. > > But it becomes much more interesting when modules themselves are > used as parameters of other modules. > > The first version had a constant directory path defined and used by > a module pets_lib that served as a sort of general library. Then > other modules used pets_lib. > > For example, a module pets_tm would have somewhere: > > Res = pets_lib:delete_table(Tab), > > How do I make the path (as well as many other configuration > parameters) a value chosen at runtime, with little effort in > rewriting the code which didn?t contemplate such possibility > beforehand? The problem is that you can't. Yes, you *can* replace pets_lib: by Pets_Lib:, but now - either you have to pass Pets_Lib around all over the place, which doesn't count as "little effort", or - you have to pass Pets_Lib as a parameter to the module containing this call, which transitively affects its callers as well, ... - and you had better first take care to rename any existing occurrences of "Pets_Lib" to something else Perhaps we can name this a "module parameter cascade". > Making pets_lib a parameterized module. What is the impact of that > on client code? A simple change to: > > Res = Lib:delete_table(Tab), > > It looks pretty much the same, but now we have this Lib variable. Looks, as they say, can be deceiving. > If we were using closures, the closure would have to be passed > somehow (who knows how many levels of invocations) until is was > available to the function which performs this invocation. (a) People seriously underestimate what closures can do. (b) This is not an argument for modules with parameters, it is an argument for nested functions. > But if the pets_lib instantiation is a parameter of pets_tm, then I > can use statements like the above all over pets_tm by doing a simple: > > :%s/pets_lib/Lib/g You had better pray desperately to whatever god(s) you recognise that there are no other occurrences of Lib, and while you're at it, beg forgiveness for breaking the name link between the module pets_lib and the module instance variable Lib (which would be better as Pets_Lib, so that the only difference between the module and the instance variable is capitalisation). > > and making pets_tm parameterized; e.g. > > -module(pets_tm, [Lib, DumpLimit]). Can anyone explain to me why the syntax for this is not -module(pets_tm(Lib, DumpLimit)) ? Not a reason of the form "this was the quickest hack to the parser", but a reason why this is the _right_ syntax? > > Then we only need to glue modules together at service starting time; > e.g. That is, you are making a change to a module which requires - *remote* compensatory changes - to possibly *many* service startups - which previously never mentioned the module in question at all. And before you reply, no, I am *not* saying that using closures will fix these issues. But using closures *will* make you try hard to think of ways of addressing the problem-space issues that don't have these solution-space consequences in the first place. One thing that bothers me is that we used to have one kind of hot loading. Now we have at least three. (a) classic replacement of a plain module with another. (b) replacement of a parameterised module, with consequences on existing instances (c) replacement of an instance by another instance with different parameter values. How do you do (c)? The thing is that adding a parameter to a plain module converts all cases of (a) involving that module to cases of (b) or (c). In particular, replacing one ?DUMP_LIMIT by another turns an (a) -- handled by existing tools -- into a (c). I understand ML-style signatures, structures, and functors well enough to use them with some confidence. (Although I was _always_ surprised when a new problem with the framework was pointed out.) But they are *static*. I understand object orientation tolerable well. I can read and write C++ as long as it doesn't use too many templates. I'm fluent in Smalltalk, and I used to do a fair bit of Eiffel. So I understand constructing webs of objects and replacing one of them. Erlang modules with parameters are neither one thing nor the other. They are not pure static things like ML functors, because the underlying modules can be hot-reloaded. They are not quite OO because there doesn't seem to be any simple way to do (c). Reverting to the pets-lib example, - changing from a compile-time limit to a startup-configurable limit is not a trivial DESIGN change, so a technique that makes it look like a trivial CODE change makes me suspicious and anxious - it can obviously be done by querying a configuration file at run time (the way C programs call sysconf() instead of using templates or macros). Querying a configuration at run time is fully compatible with all the old Erlang tools. Indeed, from the point of view of system maintenance, being able to trace the catalogue lookups in someone else's large body of code and find out just who needs what when can be a help. The catalogue can be a module. Let's take the data base example. > need_dump(Tab, LogOps) -> LogOps > ?DUMPLIMIT * ets:info(Tab, size). We want to make DUMP_LIMIT something that can be configured at run time. But that's easy! need_dump(Tab, LogOps) -> my_config:dump_limit(Tab) * ets:info(Tab, size). where -module(my_config). -export([dump_limit/1]). dump_limit(_Tab) -> ?DUMP_LIMIT. To change the configured value, load a new version of my_config. To select a value at system startup, select which version of the configuration module to load. The use-case for modules with parameters (if there is one) is where there are grounds for believing that there may need to be multiple distinct instances of the same module at the same time AND where the module parameter cascade is tolerable. From jeffm@REDACTED Wed Apr 21 04:04:50 2010 From: jeffm@REDACTED (Jeffm) Date: Wed, 21 Apr 2010 12:04:50 +1000 Subject: [erlang-questions] Dynamic code reloading In-Reply-To: <4BCD4DD5.5040204@ghostgun.com> References: <4BCD2172.3010403@camosun.bc.ca> <4BCD2972.4020007@ghostgun.com> <4BCD4986.9000303@camosun.bc.ca> <4BCD4DD5.5040204@ghostgun.com> Message-ID: <4BCE5D42.9080502@ghostgun.com> Was sitting around waiting for a meeting this morning and thought I'd create this to clarify erlang hot code updates.: Summary: The following uses this example module to demonstrate hot code loading without the use of OTP and one possible reason why a process may die during a hot code update. %%%%% Start of file hot.erl %%%%% %% %% Simple example of hot code update. %% -module(hot). -export([ start/0, loop/0 ]). -define(V, 4). start() -> spawn(?MODULE, loop, []). loop() -> receive print_version -> io:format("~p current version is ~p~n", [?MODULE, ?V]), hot:loop(); Other -> io:format("~p got message ~p~n", [?MODULE, Other]), hot:loop() end. %%%%% End hot.erl %%%%% start erl at a unix shell and... compile and load code: 1> c(hot). {ok,hot} Start the process: 2> P = hot:start(). <0.42.0> check presence 3> i(). ... <0.42.0> hot:loop/0 233 1 0 hot:loop/0 1 Total 24633 706328 0 224 ok Say hello: 4> P ! "hello". hot got message "hello" "hello" check version: 5> P ! print_version. hot current version is 1 print_version Edit hot.erl in a text editor in another window and change -define(V, 1). to -define(V,2). Compile and load new code: 6> c(hot). {ok,hot} Check version: 7> P ! print_version. hot current version is 1 print_version Notice it hasn't changed. Check again: 8> P ! print_version. hot current version is 2 print_version It's now version two as it's re-entered the loop. Now to break it: Edit hot.erl again setting V to 3, then compile and load with 9> c(hot). {ok,hot} check still running with 10> i(). .... <0.42.0> hot:loop/0 233 61 0 hot:loop/0 1 Total 26607 749896 0 224 ok Edit hot.erl setting V to 4, then compile and load, 11> c(hot). {ok,hot} Check for presence of process, 12> i(). <0.35.0> erlang:apply/2 2584 47585 0 c:pinfo/1 49 Total 27361 780079 0 223 ok The process has died! This is due to version 2 of the code being unloaded to make room for version 4, version 3 and 4 now being in memory, before the process had looped and exited version 2 and started using version 3 as it is blocked on the receive waiting for a message. One possible fix for this is to have an 'update' message which does nothing except cause the process to loop in order to load new code when loaded by sending each process affected the update message after the new code is loaded. Hope that clarifies things, Jeff. From ok@REDACTED Wed Apr 21 04:07:10 2010 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 21 Apr 2010 14:07:10 +1200 Subject: [erlang-questions] performance vs msg queue length In-Reply-To: References: <4BCCA314.8020004@erlang-solutions.com> <4BCDB9EA.8080206@erlang-solutions.com> Message-ID: <293E9B31-E8FE-492D-920A-423BB544E399@cs.otago.ac.nz> On Apr 21, 2010, at 3:00 AM, David Mercer wrote: > Unfortunately, my spot-check of about 30 selective receives within > the OTP > library indicates that a receive after an erlang:monitor is the > exception > rather than the rule. I may have just been unlucky, and I'm sure > ROK could > put his code-scanner to work to tell us exactly how often it occurs. ROK is too busy trying to annotate two masters' theses, catch up on several days of e-mail, watch anxiously over a week-long computation, and cope with a maddeningly slow network to do this. Pity. From ok@REDACTED Wed Apr 21 04:44:25 2010 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 21 Apr 2010 14:44:25 +1200 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> <4BCDC512.1060200@erlang-solutions.com> Message-ID: On Apr 21, 2010, at 7:04 AM, Vlad Dumitrescu wrote: > On Tue, Apr 20, 2010 at 17:15, Mazen Harake > wrote: >> If you create a new "instance" of a module with a parameter, then >> you need >> to have that "instance variable" around to call it right? So you >> need to >> keep it somewhere... as a state. >> >> So in any case you need to keep a state, in other words you haven't >> achieved >> anything (except perhaps making your code more obfuscated). >> >> So _why_ is it ["of course"] "easier to use module parameters >> instead of >> passing closures or state around"? > > Hi, > > If you send state around the functional way, it has to be a parameter > to all the functions that use some part of that state. If the state > data changes, it may be needed to update all these places (if the > state representation wasn't flexible enough). > > If the state is globally accessible, you just use the appropriate > variable where it is needed. The "it has to be a parameter ..." part is true of Erlang right now, not a truth about "the functional way" in general. "if the state representation wasn't flexible enough" applies with equal, if not greater, force to module parameters: if we used to have a parameter Foo and now find that it needs to be split into Foo proper and Fooly, we still should _review_ each occurrence of Foo to see what to do about it. Actually, what we're discussing here is not STATE (which changes) but CONTEXT (which doesn't). Module parameters do not support state. Other functional languages usually handle this with nested functions, not with module parameters. From cowboymathu@REDACTED Wed Apr 21 09:01:33 2010 From: cowboymathu@REDACTED (=?UTF-8?B?TUF0aHV2YXRoYW5hbiBNT3UgfHwgIOCuruCupOCvgeCuteCupOCuqeCuqeCvjSDgrq7gr4w=?=) Date: Wed, 21 Apr 2010 12:31:33 +0530 Subject: Blacklist access from the specified set of nodes Message-ID: Hi all, net_kernal:allow(Nodes) will limit access to specified nodes. I need to know the way to block specified set of nodes or all the nodes from particular remote server. HOw can I do that? Thanks, -- Mathuvathanan Mou. From mail@REDACTED Wed Apr 21 09:19:50 2010 From: mail@REDACTED (Tim Fletcher) Date: Wed, 21 Apr 2010 00:19:50 -0700 (PDT) Subject: Parameterized module idioms In-Reply-To: <3FFFC7AE-F622-4C41-8BFE-95E9B8646535@cs.otago.ac.nz> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> <3FFFC7AE-F622-4C41-8BFE-95E9B8646535@cs.otago.ac.nz> Message-ID: <98e237bd-6a9c-475f-9f0c-6c8f427a81cf@x3g2000yqd.googlegroups.com> > The use-case for modules with parameters (if there is one) is > where there are grounds for believing that there may need to > be multiple distinct instances of the same module at the same > time AND where the module parameter cascade is tolerable. Are there any good examples of this? What about the motivation described in Richard Carlsson's paper (http://portal.acm.org/citation.cfm?id=940885) about callback module usage? From johanmunk@REDACTED Wed Apr 21 09:24:56 2010 From: johanmunk@REDACTED (johan munk) Date: Wed, 21 Apr 2010 09:24:56 +0200 Subject: compiler: internal consistency check failed In-Reply-To: References: Message-ID: This bug report apparently got lost when sent to erlang-bugs: On Tue, Apr 20, 2010 at 2:56 PM, johan munk wrote: > Compiling this: > > -module(foo). > > -export([foo_int/0]). > > foo_int() -> > foo_int(<<0:8,0:8>>). > > foo_int(V) -> > <> = V, > I. > > gave me this: > > foo: function foo_int/0+4: > Internal consistency check failed - please report this bug. > Instruction: {bs_context_to_binary,{literal,<<0,0>>}} > Error: {bad_source,{literal,<<0,0>>}}: > > It compiles when not using the compiler option 'inline'. > > (Erlang R13B03) > > > /JM > From mazen.harake@REDACTED Wed Apr 21 10:37:11 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Wed, 21 Apr 2010 10:37:11 +0200 Subject: [erlang-questions] Re: Parameterized module idioms In-Reply-To: <98e237bd-6a9c-475f-9f0c-6c8f427a81cf@x3g2000yqd.googlegroups.com> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> <3FFFC7AE-F622-4C41-8BFE-95E9B8646535@cs.otago.ac.nz> <98e237bd-6a9c-475f-9f0c-6c8f427a81cf@x3g2000yqd.googlegroups.com> Message-ID: <4BCEB937.7030903@erlang-solutions.com> I would rather focus on chapter 6 in that paper because that is what will happen when people think that this is "the Erlang way". Doom and destruction is coming with this :P /Mazen On 21/04/2010 09:19, Tim Fletcher wrote: >> The use-case for modules with parameters (if there is one) is >> where there are grounds for believing that there may need to >> be multiple distinct instances of the same module at the same >> time AND where the module parameter cascade is tolerable. >> > Are there any good examples of this? > > What about the motivation described in Richard Carlsson's paper > (http://portal.acm.org/citation.cfm?id=940885) about callback module > usage? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From hynek@REDACTED Wed Apr 21 11:12:52 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Wed, 21 Apr 2010 11:12:52 +0200 Subject: [erlang-questions] Re: floating point syntax In-Reply-To: References: <201004201947.07593.als@iinet.net.au> Message-ID: On Wed, Apr 21, 2010 at 3:08 AM, Richard O'Keefe wrote: > > On Apr 20, 2010, at 9:47 PM, Anthony Shipman wrote: > >> On Tue, 20 Apr 2010 07:16:02 am Robert Virding wrote: >>> >>> IIRC it was probably more that we/I just didn't see the need to be >>> able to write the shortened forms. It isn't that it is much extra you >>> have to write. :-) >>> >>> Robert >> >> When parsing numbers for other languages it would be nice to be able to >> use >> fread() to parse them instead of writing my own. Some sort of ~N form to >> read >> a number, integer or floating point, in the usual syntax would be nice. > > What _is_ the "usual syntax"? > Here's a floating point number: 1.23@REDACTED ? ? ? ? ? (Burroughs Algol) > Here's a floating point number: 1.23E45_DOUBLE ? ?(Fortran) > Here's a floating point number: 1.23D45 ? ? ? ? ? (Fortran, Lisp, Smalltalk) > Here's a floating point number: 1.23e45L ? ? ? ? ?(C) > Here's a floating point number: ~12.34 ? ? ? ? ? ?(ML) > Here's a floating point number: 1_234_567.0 ? ? ? (Ada, OCaml) > Here's a floating point number: 1 234 567.0 ? ? ? (Fortran, Algol 60, Algol > 68) > Here's a floating point number: 0x12.34p45 ? ? ? ?(C) > Here's a floating point number: 101.101E2B ? ? ? ?(PL/I) > Here's a floating point number: 1.23Q45 ? ? ? ? ? (ANSI Smalltalk) Here's a floating point number: _12.34 (J) to your impressive collection ;-) > > For what it's worth, here's the "usual syntax" for floats from the > ANSI Smalltalk standard: > > float ::= mantissa [exponentLetter exponent] > mantissa ::= digits ?.? digits > digits ::= digit+ > exponent ::= [?-?]decimalInteger > exponentLetter ::= ?e? | ?d? | ?q? > > Except for letting the exponent letter vary to indicate the > precision of the number (single, double, double extended), > this is pretty traditional. ?(Even "d" is traditional.) > Erlang is not alone in requiring a digit each side of the dot. > > There is so much variation between programming languages that > it is extremely hard to cover all of them (and believe me, I've > tried). ?True, Fortran allows numbers with a digit on only one > side of the dot, but it also allowed spaces inside numbers; is > _that_ "usual syntax"? > > One really important issue these days is allowing numbers written > by people or programs using the comma as the radix point. > > There are obvious advantages to "being generous in what you accept", > but we cannot just appeal to "usual syntax" to determine what that > should be. ?We need something more explicit. > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss. Be a data hero! Try GoodData now for free: www.gooddata.com From ingela@REDACTED Wed Apr 21 11:55:13 2010 From: ingela@REDACTED (Ingela Andin) Date: Wed, 21 Apr 2010 11:55:13 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: <8D18002A-6ADA-4122-983B-C1C636C700A6@rogvall.se> References: <19BD41F6-E388-4D73-9D5C-40350A4CE02F@rogvall.se> <8D18002A-6ADA-4122-983B-C1C636C700A6@rogvall.se> Message-ID: Acctualy the patch that I mentioned makes ssl behave as gen_tcp that will automaticaly change from http to httph. Regards Ingela Erlang OTP team Ericsson AB 2010/4/19 Tony Rogvall : > The packet type for http headers is 'httph' (not the 'h' appended) > So you need to switch packet type after you received the request in 'http' mode > > /Tony > > On 19 apr 2010, at 15.03, Roberto Ostinelli wrote: > >> 2010/4/19 Tony Rogvall : >>> You must switch to http header packet processing ! >>> Use {packet, httph} or {packet,httph_bin} after the initial request. >>> End of headers is signaled by http_eoh. >>> After http_eoh you must switch to content processing {packet, 0} and then read possible content. >>> The size of the content is givent by the header Content-Length or if chunked encoding is used >>> the size given before each chunk. >>> >>> Hope this helps >>> >>> /Tony >>> >> >> hi tony, >> >> well i am starting the server with the options: >> >> ? ? ? Options = [ >> ? ? ? ? ? ? ? binary, {packet, http}, {reuseaddr, true}, {active, false}, >> ? ? ? ? ? ? ? {backlog, 128}, {ssl_imp, new}, >> ? ? ? ? ? ? ? {certfile, "/Users/roberto/Code/sslbug/server.pem"}, >> ? ? ? ? ? ? ? {keyfile, "/Users/roberto/Code/sslbug/privkey.pem"}, >> ? ? ? ? ? ? ? {password, "roberto"} >> ? ? ? ], >> >> which does include {packet, http}. is this not enough? i normally >> switch to {packet, 0} only when i go passive to read body of requests, >> after eoh. >> >> r. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mail@REDACTED Wed Apr 21 12:09:30 2010 From: mail@REDACTED (Tim Fletcher) Date: Wed, 21 Apr 2010 03:09:30 -0700 (PDT) Subject: Parameterized module idioms In-Reply-To: <4BCEB937.7030903@erlang-solutions.com> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> <3FFFC7AE-F622-4C41-8BFE-95E9B8646535@cs.otago.ac.nz> <98e237bd-6a9c-475f-9f0c-6c8f427a81cf@x3g2000yqd.googlegroups.com> <4BCEB937.7030903@erlang-solutions.com> Message-ID: <383e97b0-c06f-406f-bf05-c0e79fd01158@r27g2000yqn.googlegroups.com> > I would rather focus on chapter 6 in that paper because that is what > will happen when people think that this is "the Erlang way". > > Doom and destruction is coming with this :P Agreed, I think it's a feature that's very likely to be over-used. My previous two questions were a bit devil's advocate-esque. They lead to: what use cases are parameterized modules really intended for (i.e., things that are impossible/awkward to do otherwise), and do the benefits of supporting these use cases outweigh the costs/ disadvantages of adding this feature to the language? I don't think the use case in that paper is sufficient to introduce the feature on its own, considering the costs. The use case that Richard O'Keefe describes previously makes sense, but needs concrete examples. Tim From ingela.andin@REDACTED Wed Apr 21 12:10:13 2010 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 21 Apr 2010 12:10:13 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: Hi! 2010/4/20 Roberto Ostinelli : > 2010/4/19 Ingela Andin : >> Hi! >> >> We recently accepted a patch for http packet mode e.i: >> http://github.com/erlang/otp/commit/12da3796463c7d02999985ae15e98dcaa4d2bdb4 >> >> Do you have that patch? The offical ssl-release ssl-3.11 where this >> patch has been >> included is not yet released other then to git-repository. >> >> Regards Ingela Erlang/OTP team, Ericsson AB > > hi ingela, > > no i do not have this patch Without the patch you need to switch headeras as Tony said with new ssl. > but i've decided to go for the httpd > option in ssl. Sorry I do not understand what you mean by that? > 1. > in https mode [ssl:transport_accept/1 + ssl:ssl_accept/1], if i > receive a http request, i get a {error, closed} message, i.e. a socket > is not created, thus the browser timeouts waiting for a response. > > 2. > in http mode [gen_tcp:accept/1], if i receive a https request then i get a > {http, Sock, {http_error, [22,3,1,0,157,1,0,0, ......]}} message, i.e. > the socket is created but the data is not understood. > > i would like to respond with a 403 error in case 1 and a 404 error in case 2. > > what is the best way to handle this? Are you writing your own http server? Could you explain more what yoy are trying to do? Regards Ingela Erlang OTP team Ericsson AB From bgustavsson@REDACTED Wed Apr 21 15:23:35 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 21 Apr 2010 15:23:35 +0200 Subject: [erlang-questions] Re: compiler: internal consistency check failed In-Reply-To: References: Message-ID: On Wed, Apr 21, 2010 at 9:24 AM, johan munk wrote: >> foo: function foo_int/0+4: >> Internal consistency check failed - please report this bug. >> Instruction: {bs_context_to_binary,{literal,<<0,0>>}} >> Error: {bad_source,{literal,<<0,0>>}}: >> >> It compiles when not using the compiler option 'inline'. >> >> (Erlang R13B03) The bug is fixed in the 'dev' branch in our git repository and the bug fix will be included in the R14 release. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From martindemello@REDACTED Wed Apr 21 15:59:52 2010 From: martindemello@REDACTED (Martin DeMello) Date: Wed, 21 Apr 2010 19:29:52 +0530 Subject: [erlang-questions] testing asynchronous code In-Reply-To: <69562.1271795860@snookles.snookles.com> References: <69562.1271795860@snookles.snookles.com> Message-ID: On Wed, Apr 21, 2010 at 2:07 AM, Scott Lystig Fritchie wrote: > > It's quite unusual to have a gen_server send itself a message using > gen_server:cast(). ?I don't know why your code would do it... It's not, it's sending a message to another gen_server. It does it that way simply because anyone calling the other gen_server directly wants to do it asynchronously, and so I had implemented handle_cast for the cascaded operations, and not handle_call. But now that you mention it, it does seem like a code smell anyway, since only the top level calling code should really need a cast, and can do everything synchronously "behind the scenes". I'm new to the language, and still in the process of getting a feel for what good and bad erlang code looks like, so I appreciate the tips. > ... but I confess that it is very occasionally useful to use > gen_server:cast() or "self() ! {some_message_tag, ReminderData}" to > remind yourself to do something that is very inconvenient to deal with > at this instant in time. ?When your call stack pops the normal way and > returns control to gen_server's message handling loop, then you can > conveniently handle that reminder data via YourModule:handle_cast() or > YourModule:handle_info(). That's a useful trick :) Bookmarked in case I ever need it. martin From psa@REDACTED Wed Apr 21 17:31:51 2010 From: psa@REDACTED (=?UTF-8?B?UGF1bG8gU8KOZXJnaW8gQWxtZWlkYQ==?=) Date: Wed, 21 Apr 2010 16:31:51 +0100 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: <3FFFC7AE-F622-4C41-8BFE-95E9B8646535@cs.otago.ac.nz> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> <3FFFC7AE-F622-4C41-8BFE-95E9B8646535@cs.otago.ac.nz> Message-ID: <4BCF1A67.5030302@di.uminho.pt> Hi again, Richard O'Keefe wrote: > If they really *are* nonrelated, they should be separate parameters anyway. > If they are related, they can be in one data structure (which might be > a closure). "Pollution" is not a property of closures, but of a coding > style. The pollution I mean is the passing around along the invocation chain from the "root" function up to the "leaf" functions, of several possibly unrelated parameters. >> If some code would need to be generated if I didn't have parameterized >> modules, then parameterized modules already give me something. > > What? Automatic code generation isn't anything you need to be aware of, > let alone involved with. Ok. I can buy that. It is some extra infrastructure but not my problem. >> Instead of >> >> need_dump(Tab, LogOps) -> LogOps > ?DUMPLIMIT * ets:info(Tab, size). >> >> I can have >> >> need_dump(Tab, LogOps) -> LogOps > DumpLimit * ets:info(Tab, size). >> >> with no change in the interface of the function. > > > But this is the Functional Programming Lesson: > there *is* a change in the interface of the function! The interface is what matters to client code, not internals. The interface has not changed: Before: I supply an atom and an integer and get a boolean. After: I supply an atom and an integer and get a boolean. The client code that performs the invocation of the function does not see a change in the interface and does not need to be changed. More precisely: the invocations in the same module (intra-module calls) do not need to be changed. Invocations in other modules will be changed in that now the module name in the call is a variable; i.e. from m:f(P) to M:f(P). > In the first case, the function uses nothing but its arguments. > In the second case, the function has an extra parameter (DumpLimit). > The function is _really_ > > need_dump(%Hidden%, Tab, LogOps) -> > LogOps > %Hidden%#%hidden%.DumpLimit * ets:info(Tab, size). > > How it's _compiled_ is a separate issue; I'm arguing about what it _means_. But it almost seems like you are talking about how it is compiled ;) What the function _uses_ is irrelevant to its interface. It is as if saying that the values captured by a closure are part of its interface, because the closure definition uses them. That now an extra parameter is used is just an implementation choice, not a fundamental thing in the concept. One could have another implementation (not that I am saying it would be a good idea) were the module source is compiled at runtime with the parameters substituted and then loaded, resulting in a module like m_instance_1213124 which has the same functions, with the exact same interface, and no hidden extra parameter. And when a client does M:f(P) what _really_ happens could be m_instance_1213:f(P) which would not be much different than doing now: M = lists, M:sort([3,1,2]). with no hidden extra parameters being passed. Such an implementation would have very different consequences, both disadvantages like runtime instantiation costs and possible errors, and advantages like possible optimizations using runtime knowledge, and I am not saying it would be realistic or in the spirit of what we expect. I am just saying that the interface remains the same, but now the function belongs to a module that is only computed at runtime and that for intra-module calls even that is irrelevant. >> For example, a module pets_tm would have somewhere: >> >> Res = pets_lib:delete_table(Tab), >> >> How do I make the path (as well as many other configuration >> parameters) a value chosen at runtime, with little effort in rewriting >> the code which didn?t contemplate such possibility beforehand? > > The problem is that you can't. Yes I can ;) I actually could. > Yes, you *can* replace pets_lib: by Pets_Lib:, but now > - either you have to pass Pets_Lib around all over the place, > which doesn't count as "little effort", or > - you have to pass Pets_Lib as a parameter to the module containing Exactly: if it is a paramter in the client mode. Then it is little effort. It really WAS little effort in rewriting my library. > this call, which transitively affects its callers as well, ... This is an interesting point, which I don?t know if it was much discussed here. I tended to notice that to be able to have nice little-effort changes, client modules end up having parameters. This is a sort of "viral" phenomena, which we want to contain. This viral aspect made me think that when building an abstraction, the module(s) that are exposed to the outside world should NOT be parameterized, while the modules used internally can be parameterized if it helps productivity in writing code. This is what I ended up with, looking at each module definition: pets.erl:-module(pets). pets_gc.erl:-module(pets_gc, [Lib, MaxTids, CollectRatio, PurgeRatio]). pets_lib.erl:-module(pets_lib, [PetsDir]). pets_loader.erl:-module(pets_loader, [Lib, MaxReaders, MaxInserters]). pets_locker.erl:-module(pets_locker). pets_tm.erl:-module(pets_tm, [Lib, DumpLimit]). pets_writer.erl:-module(pets_writer, [Lib, SyncDelay]). test.erl:-module(test). The only module that clients use, "pets", is not parameterized. The modules used internally are either parameterized: pets_gc, pets_lib, pets_loader, pets_tm, pets_writer or not parameterized: pets_locker test > - and you had better first take care to rename any existing occurrences > of "Pets_Lib" to something else Of course. But it is easy to: invent a nice name; then check that it doesn?t occur in any module. > > Perhaps we can name this a "module parameter cascade". > >> Making pets_lib a parameterized module. What is the impact of that on >> client code? A simple change to: >> >> Res = Lib:delete_table(Tab), >> >> It looks pretty much the same, but now we have this Lib variable. > > Looks, as they say, can be deceiving. > >> If we were using closures, the closure would have to be passed somehow >> (who knows how many levels of invocations) until is was available to >> the function which performs this invocation. > > (a) People seriously underestimate what closures can do. > (b) This is not an argument for modules with parameters, > it is an argument for nested functions. Not sure what you mean. Using closures will have a greater impact on client code, and also on the implementation code that I am trying to reuse. > >> But if the pets_lib instantiation is a parameter of pets_tm, then I >> can use statements like the above all over pets_tm by doing a simple: >> >> :%s/pets_lib/Lib/g > > You had better pray desperately to whatever god(s) you recognise > that there are no other occurrences of Lib, and while you're at easy: grep Lib *erl and look at the result. > it, beg forgiveness for breaking the name link between the module > pets_lib and the module instance variable Lib (which would be > better as Pets_Lib, so that the only difference between the module > and the instance variable is capitalisation). This is a curious point. I am aware that I broke the connection. It was intentional. Before it was pets_lib only because the application is called pets. But I see it as my general library from this application. I preferably wouldn?t want to worry about what the application is going to be called and to have to change all over from "pets_lib" to "amnesia_lib" if I decide to rename it. This kind of use is the normal convention in Erlang, having to worry about global module namespace pollution. Another advantage of using variable for modules, like "Lib" above, is having to worry less about that kind of pollution, and what the application is going to be called. ;) >> Then we only need to glue modules together at service starting time; e.g. > > That is, you are making a change to a module which requires > - *remote* compensatory changes > - to possibly *many* service startups > - which previously never mentioned the module in question at all. I dont't understand what you mean. But all this gluing is done in the "main" for the internal modules of the application. As I exemplified, it was easy to do. The result is also easy to reason about. After instantiation, everything will behave as if the values passed to the glued modules had been -define'ed constants; no strange side-effects; referential transparency; functional style using "POF"s: plain old functions. (Does this term exist? ;)) > Let's take the data base example. > >> need_dump(Tab, LogOps) -> LogOps > ?DUMPLIMIT * ets:info(Tab, size). > > We want to make DUMP_LIMIT something that can be configured at > run time. But that's easy! > > need_dump(Tab, LogOps) -> > my_config:dump_limit(Tab) * ets:info(Tab, size). > > where > -module(my_config). > -export([dump_limit/1]). > > dump_limit(_Tab) -> ?DUMP_LIMIT. > > To change the configured value, load a new version of my_config. > To select a value at system startup, select which version of the > configuration module to load. Here there is a recompilation and selection of module. But if we want to compute at startup time some value to serve as "parameter" , that value will have to be stored somewhere in a globally acessible data structure like ets, to be consulted by my_config:dumplimit/1. That can sometimes be slow. Now I remember that one of the "permissible" uses for the process dictionary is to store "parameters" written once but never changed later. This kind of use ties code with the process structure and the use of get/1 can be slow. Parameterized modules can make some of these uses avoidable. > The use-case for modules with parameters (if there is one) is > where there are grounds for believing that there may need to > be multiple distinct instances of the same module at the same > time AND where the module parameter cascade is tolerable. A quite common example would be several instances of a web server together listening in different ports. But in this case the module(s) exposed to the client being parameterized, contrary to my "guideline" above, would lead to the client possibly storing instances in some data structures to avoid being itself parameterized and containing the "viral" impact of parameterized modules. Best regards, Paulo From roberto@REDACTED Wed Apr 21 19:08:39 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 21 Apr 2010 19:08:39 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: hi! >> but i've decided to go for the httpd >> option in ssl. > > Sorry I do not ?understand what you mean by that? i meant that i'm handling the switch myself to httph, so patch is currenlty not needed. > > Are you writing your own http server? Could you explain more what yoy > are trying to do? i was adding SSL support to misultin http://code.google.com/p/misultin/w/list as said, what i'm still missing is some kind of response in the case: . a http request is done on a https server [currently, it timeouts] . a https request is done on a http server [currently, it shuts down the connection]. it is not clear how to provide some kind of http response on a SSL socket, since the socket is not created unless SSL is used [and thus, it timeouts]. in the same way, a non-SSL socket will simply do not understand incoming tcp data which is part of the SSL negotiation [and thus, only available option is to send a 403 error which is of course mis-interpreted by the browser which initiated a SSL negitiation. hope this is more clear :) r. From roberto@REDACTED Wed Apr 21 19:26:43 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 21 Apr 2010 19:26:43 +0200 Subject: monitor message queue of a process Message-ID: dear list, i would like to know if there's a native way to monitor and react on a message queue of a process, without the need to modify the process itself. it's quite easy to get the message queue length using erlang:process_info/2, however i would need this monitoring done not in a polling-way [which would periodically check for this information], but i would love to have some kind of a message when the queue length is modified and satisfies some requisites. i'm trying to figure out a way to provide some kind of generic buffering technique to solve selective receives' snowball effect that is being discussed so much lately. ideas welcome. r. From kiszl@REDACTED Wed Apr 21 21:10:54 2010 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Wed, 21 Apr 2010 21:10:54 +0200 Subject: [erlang-questions] monitor message queue of a process In-Reply-To: References: Message-ID: <4BCF4DBE.3040000@tmit.bme.hu> Definitely not generic... But as a quick hack you can set a trace flag on your process in order to receive a message whenever the process receives one: erlang:trace(Pid, true, ['receive', {tracer, MonitorPid}]). This way you only need to check the queue length after the first n trace messages, then after n-m messages, etc... Regards, Zoltan. On 4/21/2010 7:26 PM, Roberto Ostinelli wrote: > dear list, > > i would like to know if there's a native way to monitor and react on a > message queue of a process, without the need to modify the process > itself. > > it's quite easy to get the message queue length using > erlang:process_info/2, however i would need this monitoring done not > in a polling-way [which would periodically check for this > information], but i would love to have some kind of a message when the > queue length is modified and satisfies some requisites. > > i'm trying to figure out a way to provide some kind of generic > buffering technique to solve selective receives' snowball effect that > is being discussed so much lately. > > ideas welcome. > > r. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From johanmunk@REDACTED Wed Apr 21 22:05:19 2010 From: johanmunk@REDACTED (johan munk) Date: Wed, 21 Apr 2010 22:05:19 +0200 Subject: timer server timeout Message-ID: Following the control flow in the gen_server, sys, and timer modules I see this: Calling sys:get_status(timer_server) will "reset" the "after time" in the loop function of gen_server. As the timer server uses that time to get a timeout if no other messages are received, one can potentially cause timers to never trigger. Is this a known issue/feature? /JM From jay@REDACTED Wed Apr 21 21:09:09 2010 From: jay@REDACTED (Jay Nelson) Date: Wed, 21 Apr 2010 12:09:09 -0700 Subject: testing asynchronous code Message-ID: <328D34C0-2133-4D3E-958F-B7C9CA307D52@duomark.com> Scott wrote: > It's quite unusual to have a gen server send itself a message using > gen server:cast(). I don't know why your code would do it... I do find it very useful during initialization. As you said in a different thread, a gen_server should start quickly to avoid causing a supervisor timeout when it launches too many slow children. start_link(...) -> {ok, Pid} = gen_server:start_link(...), gen_server:cast(Pid, finish_init), {ok, Pid}. The keys to the technique are: 1) Make sure you cast before the Pid leaks out to any callers 2) Make sure enough of the state is initialized for finish_init to run 3) Any callers are safely in the queue after finish_init jay From kenneth.lundin@REDACTED Wed Apr 21 22:40:52 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 21 Apr 2010 22:40:52 +0200 Subject: [erlang-questions] monitor message queue of a process In-Reply-To: <4BCF4DBE.3040000@tmit.bme.hu> References: <4BCF4DBE.3040000@tmit.bme.hu> Message-ID: On Wed, Apr 21, 2010 at 9:10 PM, Zoltan Lajos Kis wrote: > Definitely not generic... > But as a quick hack you can set a trace flag on your process in order to > receive a message whenever the process receives one: > erlang:trace(Pid, true, ['receive', {tracer, MonitorPid}]). > This way you only need to check the queue length after the first n trace > messages, then after n-m messages, etc... Using the tracing mechanisms as part of the normal execution will destroy the possibilities to trace on the system without destroying the function. Therefore this is REALLY NOT RECOMMENDED. By the way , we will most likely optimize the selective receive for the result of a gen_server call and similar until the next release (in the compiler and the Erlang VM). /Kenneth Erlang/OTP, Ericsson > Regards, > Zoltan. > > > > On 4/21/2010 7:26 PM, Roberto Ostinelli wrote: >> >> dear list, >> >> i would like to know if there's a native way to monitor and react on a >> message queue of a process, without the need to modify the process >> itself. >> >> it's quite easy to get the message queue length using >> erlang:process_info/2, however i would need this monitoring done not >> in a polling-way [which would periodically check for this >> information], but i would love to have some kind of a message when the >> queue length is modified and satisfies some requisites. >> >> i'm trying to figure out a way to provide some kind of generic >> buffering technique to solve selective receives' snowball effect that >> is being discussed so much lately. >> >> ideas welcome. >> >> r. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mevans@REDACTED Wed Apr 21 23:33:59 2010 From: mevans@REDACTED (Evans, Matthew) Date: Wed, 21 Apr 2010 17:33:59 -0400 Subject: mnesia ram cache Message-ID: Hi, I am about to implement an mnesia database for a project that is expected the database to grow into the millions of records. I will be using fragmented tables for this, so am confident that the only space limit will be what the disc has available. Due to the immense size of the database I will have to create the table as disc_only_copies since we do not have anywhere enough RAM available for disc_copies/ram_copies. It goes without saying that lookup (and write times) will be severely hit by this limit. I need to support in excess of 300,000 lookups per second, which I know that ETS can easily manage with enough room to spare. To implement this I intend to write my own ETS cache on top of mnesia, and use table subscription to the mnesia table (mnesia:subscribe/1) so that the cache can be updated when inserts and updates occur to the master mnesia database. The cache shall be managed by a gen_server that contains LRU or similar rules to purge data to avoid it growing too much. This is fine, but got me wondering if it would be appropriate for mnesia to implement similar features? Perhaps the addition of a new set of configuration options: [{ram_cache, NodeList}, {ram_cache_max_size,SizeInBytes}, {ram_cache_max_records, SizeInRecords}, {ram_cache_purge_fun, Fun()}] Obviously ram_cache will setup the cache on the list of NodeList. ram_cache_max_size / ram_cache_max_records will be the limit causing the cache to be purged, and ram_cache_purge_fun will be a Fun() that provides a set of rules to manage the purge. For example: fun() -> [ets:delete(my_table_ram_cache,Rec) || [{Rec,Stuff,Inserted} <- ets:tab2list(my_table_ram_cache), timer:now_diff(erlang:now(),Inserted) > 10000000] end. Obviously in the example above I am assuming the cache is an ETS table, maybe some better abstraction is desirable. Is such a feature on the roadmap anywhere? Regards Matt From alessandro.sivieri@REDACTED Wed Apr 21 23:50:14 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Wed, 21 Apr 2010 23:50:14 +0200 Subject: Calling external modules from a fun Message-ID: Hi all, I am working on an application which, for some reasons, creates a fun (probably something more like a continuation) and then sends it to a different node for execution; the sender node is launched with my application code path (the ebin directory), and so the receiver one. Now, if my code path contains some module which I have wrote, I have found that in a fun I cannot invoke it (as module:function), because if I do so, the remote machine (which actually receives and executes the fun itself) answers with an undefined_lambda error; so my question is: is there a way to call a function from a fun remotely received, if it is not from the standard Erlang distribution? -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From bernie@REDACTED Thu Apr 22 00:29:57 2010 From: bernie@REDACTED (Bernard Duggan) Date: Thu, 22 Apr 2010 08:29:57 +1000 Subject: [erlang-questions] mnesia ram cache In-Reply-To: References: Message-ID: <4BCF7C65.3070100@m5net.com> Evans, Matthew wrote: > To implement this I intend to write my own ETS cache on top of mnesia, and use table subscription to the mnesia table (mnesia:subscribe/1) so that the cache can be updated when inserts and updates occur to the master mnesia database. The cache shall be managed by a gen_server that contains LRU or similar rules to purge data to avoid it growing too much. > Something to be aware of before you get too attached to this plan: you won't be able to maintain transactional atomicity in the cache. That is, if, for example, some node makes changes to two tables within a single transaction, the subscription system as of R13B04 does not have a nice way for a subscriber to group them together. The result could be that someone reading the cache may get one change but not the other. This may not matter for your case if any single write leaves your database in a consistent (or "consistent enough") state, but it's important to know that you'll be losing that feature of mnesia. (There is, however, a patch I wrote in R14 which addresses this exact limitation). Cheers, Bernard From matthew@REDACTED Thu Apr 22 00:38:51 2010 From: matthew@REDACTED (Matthew Sackman) Date: Wed, 21 Apr 2010 23:38:51 +0100 Subject: [erlang-questions] Re: testing asynchronous code In-Reply-To: <328D34C0-2133-4D3E-958F-B7C9CA307D52@duomark.com> References: <328D34C0-2133-4D3E-958F-B7C9CA307D52@duomark.com> Message-ID: <20100421223851.GB16488@wellquite.org> On Wed, Apr 21, 2010 at 12:09:09PM -0700, Jay Nelson wrote: > 1) Make sure you cast before the Pid leaks out to any callers Why? Do you think that gives you any guarantee that that message will arrive in the mailbox before a message sent from a different process? I would be a little surprised if that's guaranteed. Matthew From mevans@REDACTED Thu Apr 22 00:40:48 2010 From: mevans@REDACTED (Evans, Matthew) Date: Wed, 21 Apr 2010 18:40:48 -0400 Subject: [erlang-questions] mnesia ram cache In-Reply-To: <4BCF7C65.3070100@m5net.com> References: <4BCF7C65.3070100@m5net.com> Message-ID: Thanks for that, it's good to know. In this case I should be fine since there should be no cases where a single transaction will make changes to > 1 table in a single transaction. But in future your change will help for another project I am working on. -----Original Message----- From: Bernard Duggan [mailto:bernie@REDACTED] Sent: Wednesday, April 21, 2010 6:30 PM To: Evans, Matthew Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] mnesia ram cache Evans, Matthew wrote: > To implement this I intend to write my own ETS cache on top of mnesia, and use table subscription to the mnesia table (mnesia:subscribe/1) so that the cache can be updated when inserts and updates occur to the master mnesia database. The cache shall be managed by a gen_server that contains LRU or similar rules to purge data to avoid it growing too much. > Something to be aware of before you get too attached to this plan: you won't be able to maintain transactional atomicity in the cache. That is, if, for example, some node makes changes to two tables within a single transaction, the subscription system as of R13B04 does not have a nice way for a subscriber to group them together. The result could be that someone reading the cache may get one change but not the other. This may not matter for your case if any single write leaves your database in a consistent (or "consistent enough") state, but it's important to know that you'll be losing that feature of mnesia. (There is, however, a patch I wrote in R14 which addresses this exact limitation). Cheers, Bernard From ok@REDACTED Thu Apr 22 01:03:44 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 22 Apr 2010 11:03:44 +1200 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: <4BCF1A67.5030302@di.uminho.pt> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> <3FFFC7AE-F622-4C41-8BFE-95E9B8646535@cs.otago.ac.nz> <4BCF1A67.5030302@di.uminho.pt> Message-ID: There's quite a lot to reply to, and I don't have the time right now. I'll just make a few points. > Here there is a recompilation and selection of module. But if we > want to compute at startup time some value to serve as "parameter" , > that value will have to be stored somewhere in a globally acessible > data structure like ets, to be consulted by my_config:dumplimit/1. > That can sometimes be slow. If we want to compute at startup time some value to serve as a "parameter", that value will have to be stored somewhere, AND THAT SOMEWHERE CAN BE A MODULE. There is no law that says a module can't be written by a program. Here's an actual transcript. Eshell V5.7.2 (abort with ^G) 1> c(reload). {ok,reload} 2> reload:put(27). {module,reloadable} 3> reload:get(). 27 4> reload:put(42). {module,reloadable} 5> reload:get(). 42 6> reload:put(137). {module,reloadable} 7> reload:get(). 137 8> reload:put([<<1>>,<<2,3>>,<<4,5,6>>]). {module,reloadable} 9> reload:get(). [<<1>>,<<2,3>>,<<4,5,6>>] Now here's the proof-of-concept implementation. Using compile:forms(..., [binary,...]) we can get the same effect without touching the file system, and in a "real" implementation that's what I'd do. -module(reload). -export([put/1,get/0]). put(Datum) -> {ok,Device} = file:open("reloadable.erl", [write]), io:format(Device, "-module(reloadable).~n-export([datum/0]).~ndatum() ->~n ~p .~n", [Datum]), ok = file:close(Device), compile:file("reloadable", []), code:purge(reloadable), code:load_file(reloadable). get() -> reloadable:datum(). And yes, I do realise that what we have here is a global mutable variable with an amazingly slow assignment statement, but that's precisely what a changeable configuration parameter IS. The overhead of of setting the parameter up (or changing it) is moderately high (although using compile:forms(..., [binary,...]) would be more efficient as well as safer). But the overhead of *using* the parameter is the same as the overhead of any cross-module function call. And if that weren't tolerable, we wouldn't be using Erlang. And of course this can be generalised in a whole lot of ways. > > Now I remember that one of the "permissible" uses for the process > dictionary is to store "parameters" written once but never changed > later. This kind of use ties code with the process structure and the > use of get/1 can be slow. Slow? Time for some numbers. 6> getput:k(100000000). [{variant,constant},{result,100000000},{time,530}] 7> getput:b(100000000). [{variant,direct},{result,100000000},{time,1730}] 8> getput:t(100000000). [{variant,dictionary},{result,100000000},{time,2290}] Here's the code that produced those: -module(getput). -export([t/1, b/1, k/1]). t(N) -> put(key, 1), {T0,_} = statistics(runtime), R = loop(N, 0), {T1,_} = statistics(runtime), [{variant,dictionary},{result,R}, {time,T1-T0}]. loop(0, R) -> R; loop(N, R) -> loop(N-1, R+get(key)). b(N) -> {T0,_} = statistics(runtime), R = loup(N, 0), {T1,_} = statistics(runtime), [{variant,direct},{result,R}, {time,T1-T0}]. loup(0, R) -> R; loup(N, R) -> loup(N-1, R+(N div N)). k(N) -> {T0,_} = statistics(runtime), R = lowp(N, 0), {T1,_} = statistics(runtime), [{variant,constant},{result,R}, {time,T1-T0}]. lowp(0, R) -> R; lowp(N, R) -> lowp(N-1, R+1). So - the loop that just adds 1 takes 5.3 ns per iteration - the loop that adds N div N takes 17.3 ns per iteration - the loop that uses get() takes 22.9 ns per iteration We conclude that - N div N takes 12 ns - get(key) takes 17.6 ns and therefore EITHER my benchmark and interpretation are hopelessly fouled up OR using get/1 is NOT particularly slow. To be honest, I incline to the former; how can looking something up in a hash table be so good compared with an integer division? While it may be true that get/1 _can_ be slow (I'd need to see the numbers), you should never just _assume_ that get/1 is slow for the use you intend to make of it. > > A quite common example would be several instances of a web server > together listening in different ports. It's not clear why the port should be part of a web server's context rather than part of its state, or why these instances need to be all together in a single Erlang node (because if they aren't, module parameters offer us no convenience), or why if they are all in a single node "they" shouldn't be "it", a single system listening on several ports and doing load balancing of some sort. > From hd2010@REDACTED Thu Apr 22 01:10:14 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 22 Apr 2010 01:10:14 +0200 Subject: [erlang-questions] The If expression Message-ID: <4BCF85D6.3070205@eonblast.com> Did anything come from the discussion about the if expression ( http://www.erlang.org/pipermail/erlang-questions/2009-January/040808.html )? I remember reading that one-branched ifs shouldn't be necessary. And that if they are needed it's a sign of bad structure / code in the first place. I'm a serial offender. Is there a specific tip on how to think differently when one runs into frequently using one-armed ifs? I.e., writing a lot of true -> nil for the empty second branch? Thanks! Henning From zabrane3@REDACTED Thu Apr 22 01:12:14 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Thu, 22 Apr 2010 01:12:14 +0200 Subject: can "gen_tcp:recv" blocks my entire server? Message-ID: Hi List, I'm impleting a simple Web server using gen_tcp:listen(Port, [...,{active, false}]) options. For each request, a child process is spawned to handle it. Then, each child process use gen_tcp:recv/2 to read data from the client socket. Very basic design! My question is simple. If one of my child processes calls gen_tcp:recv/2 and blocks (due to a slow client), does this also blocks all other spawned childs? If yes, what's the best way to read from a socket without blocking the entire server (i.e the other spawned processes)? Regards Zabrane From ok@REDACTED Thu Apr 22 01:24:41 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 22 Apr 2010 11:24:41 +1200 Subject: [erlang-questions] The If expression In-Reply-To: <4BCF85D6.3070205@eonblast.com> References: <4BCF85D6.3070205@eonblast.com> Message-ID: On Apr 22, 2010, at 11:10 AM, Henning Diedrich wrote: > Did anything come from the discussion about the if expression ( http://www.erlang.org/pipermail/erlang-questions/2009-January/040808.html > )? (1) The "undocumented?" comment in that article is just plain wrong. This is a property of all guards and is documented. (2) As for if Expression -> Body else -> ElseBody end, (A) It is just _too_ confusing to have 'if ' and 'if ' in the same language. (B) General Erlang "style" for 'else' is '; true ->' (C) It has been argued at great length over many years in the Software Engineering community that 'boolean' is almost always the wrong type to use. In too many situations, it is not clear which convention is followed: does 'true' mean connected or disconnected? In too many other situations, there are more than two possibilities. In this mailing list, the latter case has very often been substantiated. > > I remember reading that one-branched ifs shouldn't be necessary. (3) It's not that they SHOULDN'T be necessary. It's not even the obvious fact that they AREN'T necessary. The point is (2,C) that they are often WRONG. > And that if they are needed it's a sign of bad structure / code in > the first place. Not "if they are NEEDED", because they never are. "If they are USED". > > Is there a specific tip on how to think differently when one runs > into frequently using one-armed ifs? I.e., writing a lot of true -> > nil for the empty second branch? For one thing, this is a clear sign of writing imperative code. Pure functional languages like Haskell and Clean don't have one-armed ifs because every expression has to have a value, and both arms of an if have to have values of the same type. The best thing to do is to show us some code and invite us to refactor it. From dmurray@REDACTED Thu Apr 22 01:48:13 2010 From: dmurray@REDACTED (David N Murray) Date: Wed, 21 Apr 2010 19:48:13 -0400 (EDT) Subject: [erlang-questions] The If expression In-Reply-To: References: <4BCF85D6.3070205@eonblast.com> Message-ID: On Apr 22, Richard O'Keefe scribed: > > > The best thing to do is to show us some code and invite us to refactor > it. > I'm coming from an imperative background and constantly run into one-armed ifs, or more usually, one-armed cases: init([]) -> case cfg:get(ibr_connect) of ok -> gen_server:cast(self(), connect); _ -> ok end, {ok, {init, [], []}}. % State, MsgId, Buf When this gen_server is started, I have other housekeeping to do before it can actually start doing it's thing, and the supervisor casts a connect message when it's ok to connect. When I process the connect in this server, I store {ibr_connect, ok} in the cfg module. This is my breadcrumb that I've been here before and everything is good to go. If the gen_server gets restarted, I want it to connect right away and not wait for someone to tell it to (automatic reconnect logic, the module is a tcp client to another server). Usually, my tests are "has this happened, or is this a that" and I really don't care about the "false" side of that test. I'm having a hard time getting my head around this. TIA, Dave From jay@REDACTED Thu Apr 22 02:55:08 2010 From: jay@REDACTED (Jay Nelson) Date: Wed, 21 Apr 2010 17:55:08 -0700 Subject: [erlang-questions] Re: testing asynchronous code In-Reply-To: <20100421223851.GB16488@wellquite.org> References: <328D34C0-2133-4D3E-958F-B7C9CA307D52@duomark.com> <20100421223851.GB16488@wellquite.org> Message-ID: <670EF79B-D523-48EA-9039-ED6DFD989CE9@duomark.com> On Apr 21, 2010, at 3:38 PM, Matthew Sackman wrote: > On Wed, Apr 21, 2010 at 12:09:09PM -0700, Jay Nelson wrote: >> 1) Make sure you cast before the Pid leaks out to any callers > > Why? Do you think that gives you any guarantee that that message will > arrive in the mailbox before a message sent from a different > process? I > would be a little surprised if that's guaranteed. You're right, the cast call spawns a process with a single call to erlang:send/2, but that process may not get a chance to run before the initializing process makes a gen_server:call. I have done this in past for a large data load initialization, and I need to find and fix that, because it can cause a race condition error between the final initialization and the first gen_server:call made by the process which created the gen_server. What is guaranteed is if you make a gen_server:call (which is what I should have said in the first place, and therefore doesn't apply to Scott's original comment about casting to self) inside the start function after init returns the Pid, but before the start function hands it back (provided another process is not able to guess the Pid before it is told what it is). The language does guarantee that two calls from the same process to another process will maintain their order in the message queue. If the route of a message goes through an intermediate process (as cast does), there is no ordering guarantee at the final recipient. jay From hd2010@REDACTED Thu Apr 22 03:08:53 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 22 Apr 2010 03:08:53 +0200 Subject: [erlang-questions] The If expression In-Reply-To: References: <4BCF85D6.3070205@eonblast.com> Message-ID: <4BCFA1A5.9000900@eonblast.com> Hi Richard, thanks a lot for the lightning-speed & sorrow reply! Please be gentle, my eyes are sore already :-D I also realize I was somewhat hysteric, maybe. Looking for examples now, I found 5 true->nil in a thousands lines. So maybe that absolves a bit of my guilt of doing imperative code. No matter, the question remains as sincere as before. That was exactly what I wanted to address and what the 5 places - given below - totally look like. > >> Did anything come from the discussion about the if expression ( >> http://www.erlang.org/pipermail/erlang-questions/2009-January/040808.html )? >> > > (1) The "undocumented?" comment in that article is just plain wrong. I meant to basically give the head of that related thread. > (B) General Erlang "style" for 'else' is '; true ->' It's not hard to get used to. I wasn't going to complain about that oddity actually, but wondered what to do different, to not have to write it 'empty' now and then. Regarding 'have', see below please. > (C) It has been argued at great length over many years in the > Software Engineering community that 'boolean' is almost always > the wrong type to use. In too many situations, it is not clear > which convention is followed: does 'true' mean connected or > disconnected? In too many other situations, there are more than > two possibilities. In this mailing list, the latter case has > very often been substantiated. I think it's exactly the (few) cases where 'yes or no' is what matters, see samples at bottom, thanks. The rest of the paragraph, I won't dispute. AND I'd happily avoid the (few) cases left. >> I remember reading that one-branched ifs shouldn't be necessary. > (3) It's not that they SHOULDN'T be necessary. > It's not even the obvious fact that they AREN'T necessary. > The point is (2,C) that they are often WRONG. Right, it sure looks like they /are/ necessary, to me. Which /is/ why I started wondering if I am doing sth. wrong here. >> >> Is there a specific tip on how to think differently when one runs >> into frequently using one-armed ifs? I.e., writing a lot of true -> >> nil for the empty second branch? > > For one thing, this is a clear sign of writing imperative code. Exactly, that's why I am asking about /thinking/ differently. > Pure functional languages like Haskell and Clean don't have one-armed > ifs because every expression has to have a value, and both arms of an > if have to have values of the same type. It's funny you say that because it seems like /all/ spots where I ran into this issue are one of * output * message sending * global names registration ... David's example adds * creating a connection ... also an external state issue. All samples seem to be about /creating a side effect/ --- or, under certain conditions, not creating it. Is that a legitimate opening for a one-branched if? Exactly for the less functionally-clean stuff, on Erlang's more practical leanings? As I said, the rest of the 1000 lines, with ~15 more ifs, didn't need empty 'else' clauses. Way rare enough to be a non-issue as matter of style, typing or readability. But for the principle. > > The best thing to do is to show us some code and invite us to refactor > it. Thanks a lot for the offer! With a tad of context. Sorry if there should be a typo, I sanitized them to make them more concise. (1) print -> if % suppress printing the default suite when unused >>> (Nom == "Default") and (Passed+Failed+Crashed == 0) -> >>> nil; % print all other true -> echo("~s~n~s ~s - Tests: ~p, Passed: ~p, Failed: ~p, Crashes: ~p~n~s", [Line, Nom, Verdict, Passed+Failed, Passed, Failed, Crashed, Line]) end, main ! { printed, self() }, suite_loop(Nom, Caller, Sub, Passed, Failed, Crashed); (2) verbose_echo(Verbose, Format, Para) -> % Don't print if verbosity is off (false). >>> if Verbose -> echo(Format, Para); true -> nil end. (3) % If exists, drop old before adding new element Prev = gl:get(suite, Suite), >>> if Prev /= undefined -> gl:drop(suite, Suite); true -> nil end, gl:add(suite, Suite), (4) glist() -> spawn(fun() -> glist_loop([], nil, nil) end). glist_loop(List, PrevCaller, Return) -> % call back to originally caller, to deliver result. % (so: not on very first call, if no Caller is given, or it died (?) in the meantime. if is_pid(PrevCaller) -> vecho(?D4, "call ~p ~p", [PrevCaller, Return]), PrevCaller ! Return; >>> true -> nil end, receive { add, Element, Caller } -> glist_loop([Element | List], Caller, ok); { drop, Key, Caller } -> glist_loop(lists:keydelete(Key,1,List), Caller, ok); { get, Key, Caller } -> glist_loop(List, Caller, lists:keyfind(Key,1,List)); E -> throw(E) end. (5) safe_unregister(Name) -> Registered = whereis(Name) /= undefined, >>> if Registered -> unregister(Name); true -> nil end. Thanks you very much for looking at this! Henning From bernie@REDACTED Thu Apr 22 03:12:18 2010 From: bernie@REDACTED (Bernard Duggan) Date: Thu, 22 Apr 2010 11:12:18 +1000 Subject: [erlang-questions] Re: testing asynchronous code In-Reply-To: <670EF79B-D523-48EA-9039-ED6DFD989CE9@duomark.com> References: <328D34C0-2133-4D3E-958F-B7C9CA307D52@duomark.com> <20100421223851.GB16488@wellquite.org> <670EF79B-D523-48EA-9039-ED6DFD989CE9@duomark.com> Message-ID: <4BCFA272.3040801@m5net.com> Jay Nelson wrote: > On Apr 21, 2010, at 3:38 PM, Matthew Sackman wrote: > > >> On Wed, Apr 21, 2010 at 12:09:09PM -0700, Jay Nelson wrote: >> >>> 1) Make sure you cast before the Pid leaks out to any callers >>> >> Why? Do you think that gives you any guarantee that that message will >> arrive in the mailbox before a message sent from a different >> process? I >> would be a little surprised if that's guaranteed. >> > > You're right, the cast call spawns a process with a single call to > erlang:send/2, but that process may not get a chance to run before > the initializing process makes a gen_server:call. That's not my reading of the gen_server code at all. The do_send() function (which is what ultimately does the erlang:send() of the message) /only/ spawns a new function if erlang:send() returns 'noconnect', and that will only happen (if my understanding is correct) if the process in question resides on a currently unconnected node. In the case of the example code given, that's not going to be the case since the gen_server was started by the same process doing the cast, and therefore even if it was started on a remote node we must already be connected to that node. To address the earlier question that lead to this, my understanding is that if you have the following message sending: A -> B A -> C C -> B (dependant on message from A) then the message from A to B /is/ guaranteed to arrive before the message from C. Feel free to correct me on that though. Cheers, Bernard From rvirding@REDACTED Thu Apr 22 03:39:12 2010 From: rvirding@REDACTED (Robert Virding) Date: Thu, 22 Apr 2010 03:39:12 +0200 Subject: [erlang-questions] The If expression In-Reply-To: <4BCFA1A5.9000900@eonblast.com> References: <4BCF85D6.3070205@eonblast.com> <4BCFA1A5.9000900@eonblast.com> Message-ID: On 22 April 2010 03:08, Henning Diedrich wrote: >> >> Pure functional languages like Haskell and Clean don't have one-armed >> ifs because every expression has to have a value, and both arms of an >> if have to have values of the same type. > > It's funny you say that because it seems like /all/ spots where I ran into > this issue are one of > > ? * output > ? * message sending > ? * global names registration > > ... David's example adds > > ? * creating a connection > > ... also an external state issue. > > All samples seem to be about /creating a side effect/ --- ?or, under certain > conditions, not creating it. > > Is that a legitimate opening for a one-branched if? Exactly for the less > functionally-clean stuff, on Erlang's more practical leanings? The problem is that however you choose to use 'if', or any other construction for that matter, Erlang is a functional language and everything returns, and must return, a value. So even if 'if' didn't need to have an explicit default or else case then it would still have to return a value. We would never be able agree on a default value anyway. :-) 'nil' is not a good value here as it has no inherent meaning in Erlang as it does in Lisp. It would also mean that 'if' would not behave consistently with 'case' and function clauses which generate an error if there is no matching clause. This I think would be a serious mistake. Robert From jay@REDACTED Thu Apr 22 06:59:46 2010 From: jay@REDACTED (Jay Nelson) Date: Wed, 21 Apr 2010 21:59:46 -0700 Subject: The If expression Message-ID: Henning Deidrich offered one-armed ifs to rewrite: I never use the if statement. It doesn't even occur to me to use it, although writing Java or something else I use it all the time naturally. Here's the style I would use for some of your cases (some of the others may not have enough context to be good examples for rewriting): > (2) > verbose_echo(Verbose, Format, Para) -> > > % Don't print if verbosity is off (false). > >>> if Verbose -> echo(Format, Para); true -> nil end. verbose_echo(true, Format, Para) -> echo(Format, Para); verbose_echo(_, _Format, _Para) -> nil. It doesn't do what the comment says, and you have to consider whether nil is the value you want back. It depends on what echo/2 returns as to what is appropriate (and whether the caller even cares about the return value). With this approach it is easier to test and easier to add new cases (verbosity levels) or change the existing one. > (3) > % If exists, drop old before adding new element > Prev = gl:get(suite, Suite), > >>> if Prev /= undefined -> gl:drop(suite, Suite); true -> > nil end, > gl:add(suite, Suite), Prev = case gl:get(suite, Suite) -> undefined -> nil; Defined -> gl:drop(suite, Suite), Defined end, gl:add(suite, Suite), Here I made the assumption that you needed the value of Prev later. If you don't, you can drop it out and not care what the return value of the case statement is. This is actually a common code smell to watch for. You are setting a variable, then testing for a negative value using /= constant. Instead, use case and make the first clause the constant you want to exclude, everything else will match the catch-all clause. Even when I have case ... of true -> ...; false -> ... end I find it more comforting in erlang to know I covered the cases I care about. You can even use a single armed case if you want a freak value to crash your process. > (5) > safe_unregister(Name) -> > Registered = whereis(Name) /= undefined, > >>> if Registered -> unregister(Name); true -> nil end. safe_unregister(Name) -> case whereis(Name) of undefined -> ok; Other -> unregister(Name), ok end. Another instance of #3. You can easily fix the return values to different ones, or have a return value after the case ... end. If you need to know the value of whereis, you can modify it to this: safe_unregister(Name) -> Loc = case whereis(Name) of undefined -> undefined; Other -> unregister(Name), Other end, ... computations involving Loc ... %% or even return it... Loc. Any time you find yourself binding a variable, and then soon after wanting to modify it or do an if based on a small set of values, stop, back up and try using a case statement. It has the advantage of giving you multiple variables with delayed binding of the correct one as your chosen value: Chosen = case multi_var_fun() of Val1 -> Val1; Val2 -> f(Val2); Val3 when Val3 > 0, Val3 < 5 -> g(Val3); Other -> h(Other) end, Here Chosen is a late-bound variable with several values visited as intermediate bindings before arriving at a final choice. An imperative programmer tries to do something like: Chosen = multi_var_fun(), if (Chosen == Val1) ... else ... The erlang way is to avoid binding the variable you care about until you are absolutely sure you have the _value_ you care about, rather than modifying the value along the way. A one-armed if is semantically like saying I want Val1 most of the time, and in a few cases I want to change it to a different value. jay From jay@REDACTED Thu Apr 22 07:06:31 2010 From: jay@REDACTED (Jay Nelson) Date: Wed, 21 Apr 2010 22:06:31 -0700 Subject: The If expression Message-ID: <59CC3133-75E2-4955-82A0-F758D6B6D73B@duomark.com> Henning Deidrich offered one-armed ifs to rewrite: > (4) > glist() -> > spawn(fun() -> glist_loop([], nil, nil) end). > > glist_loop(List, PrevCaller, Return) -> > > % call back to originally caller, to deliver result. > % (so: not on very first call, if no Caller is given, or it > % died (?) in the meantime. > if > is_pid(PrevCaller) -> > vecho(?D4, "call ~p ~p", [PrevCaller, Return]), > PrevCaller ! Return; > >>> true -> nil > end, > > receive > { add, Element, Caller } -> glist_loop([Element | List], > Caller, ok); > { drop, Key, Caller } -> > glist_loop(lists:keydelete(Key,1,List), Caller, ok); > { get, Key, Caller } -> glist_loop(List, Caller, > lists:keyfind(Key,1,List)); > E -> throw(E) > end. glist_loop(List, PrevCaller, Return) when is_pid(PrevCaller) -> vecho(?D4, ...), PrevCaller ! Return, glist_response(List, PrevCaller, Return); glist_loop(_List, PrevCaller, _Return) -> {error, {not_a_pid, PrevCaller}}. Your drop through branch looks like it gets caught in a receive statement with no timeout, here I explicitly return an error value. You could choose to do something different. glist_response(List, PrevCaller, Return) -> receive ... same as above ... end. Sometimes factoring out a different function gives you the pattern match that you need to break out the cases. Don't forget that when clauses can be as useful as argument values to differentiate the cases. jay From jay@REDACTED Thu Apr 22 06:16:50 2010 From: jay@REDACTED (Jay Nelson) Date: Wed, 21 Apr 2010 21:16:50 -0700 Subject: [erlang-questions] Re: testing asynchronous code In-Reply-To: <4BCFA272.3040801@m5net.com> References: <328D34C0-2133-4D3E-958F-B7C9CA307D52@duomark.com> <20100421223851.GB16488@wellquite.org> <670EF79B-D523-48EA-9039-ED6DFD989CE9@duomark.com> <4BCFA272.3040801@m5net.com> Message-ID: <0B7F5CEB-8C2A-4114-9AC3-035E507CF0A2@duomark.com> >> You're right, the cast call spawns a process with a single call to >> erlang:send/2, but that process may not get a chance to run before >> the initializing process makes a gen_server:call. > That's not my reading of the gen_server code at all. The do_send() > function (which is what ultimately does the erlang:send() of the > message) /only/ spawns a new function if erlang:send() returns > 'noconnect', and that will only happen (if my understanding is > correct) > if the process in question resides on a currently unconnected > node. In > the case of the example code given, that's not going to be the case > since the gen_server was started by the same process doing the > cast, and > therefore even if it was started on a remote node we must already be > connected to that node. You _read_ the code, while I just glanced at it to double-check if was providing misleading information. Your reading is correct, the sight of the spawn just gave me a spasm of remorse and I retracted too quickly. Thankfully, my old code is correct and it is safe to cast directly after receiving the Pid from init, before returning it to the caller. > > To address the earlier question that lead to this, my understanding is > that if you have the following message sending: > > A -> B > A -> C > C -> B (dependant on message from A) > > then the message from A to B /is/ guaranteed to arrive before the > message from C. Feel free to correct me on that though. That is correct. And I am going to step off this thread before I create any more misleading asides. jay From ok@REDACTED Thu Apr 22 07:56:59 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 22 Apr 2010 17:56:59 +1200 Subject: [erlang-questions] Re: The If expression In-Reply-To: References: Message-ID: On Apr 22, 2010, at 4:59 PM, Jay Nelson wrote: > Henning Deidrich offered one-armed ifs to rewrite: [I haven't seen that one yet] > > (2) > > verbose_echo(Verbose, Format, Para) -> > > > > % Don't print if verbosity is off (false). > > >>> if Verbose -> echo(Format, Para); true -> nil end. > > verbose_echo(true, Format, Para) -> > echo(Format, Para); > verbose_echo(_, _Format, _Para) -> > nil. This is a TEXTBOOK example of where using Boolean is wrong. It is normal to have more than two levels of verbosity. syslog has Emergency (level 0) Alert (level 1) Critical (level 2) Error (level 3) Warning (level 4) Notice (level 5) Info (level 6) Debug (level 7) Which subset of these corresponds to 'true' and which to 'false'? If you aren't using all of the syslog levels now, you will very likely be using more than two of them in the not too distant future. There is also a classic trap: you talk about one of the verbosity levels being "off". Sooner or later, someone who reads about verbosity being "off" will try verbose_echo(off, Format, Para) or perhaps verbose_echo(on, Format, Para). So why not use 'off' and 'on' as the levels in the first place? Better still, why not use level names like 'verbose' and 'laconic'? And in the spirit of making broken code crash it would be even better to write verbose_echo(verbose, Format, Para) -> echo(Format, Para); verbose_echo(laconic, _Format, _Para) -> nil. > From ulf.wiger@REDACTED Thu Apr 22 09:38:21 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 22 Apr 2010 09:38:21 +0200 Subject: [erlang-questions] Re: testing asynchronous code In-Reply-To: <4BCFA272.3040801@m5net.com> References: <328D34C0-2133-4D3E-958F-B7C9CA307D52@duomark.com> <20100421223851.GB16488@wellquite.org> <670EF79B-D523-48EA-9039-ED6DFD989CE9@duomark.com> <4BCFA272.3040801@m5net.com> Message-ID: <4BCFFCED.2070804@erlang-solutions.com> Bernard Duggan wrote: > To address the earlier question that lead to this, my understanding is > that if you have the following message sending: > > A -> B > A -> C > C -> B (dependant on message from A) > > then the message from A to B /is/ guaranteed to arrive before the > message from C. Feel free to correct me on that though. Although most conceivable implementations ought to give the expected order, I am not aware of any explicit guarantee in this case. Specifically, in the distributed case, I believe it not to be true at all. Imagine that A, B and C are all on different nodes. We can then easily imagine the connection between node(A) and node(B) to be significantly slower than the two others, or perhaps even temporarily congested. In this case, a message could travel from A to C to B faster than A->B. This was in fact one of the things covered by Hans Svensson at the Erlang Workshop in Freiburg 2007 (http://www.erlang.org/workshop/2007/proceedings/06svenss.ppt) For now, one can make a distinction between inter-node messaging semantics and intra-node semantics. When two local pids are talking to each other, message delivery is atomic. This is not true in the distributed case. Even inside a VM to day (with SMP) exit signals are asynchronous, and there is good reason to make normal message passing more asynchronous as well. This would improve many-core scalability, but would have the effect that the behaviour you mention can not be assumed to hold. The one ordering guarantee that exists is that if P1 sends a sequence of messages to P2, they will arrive in the same order as they were sent. (Unfortunately, Hans Svensson was able to devise a test with two nodes forcing the disclaimer "...provided they arrive at all") BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From zabrane3@REDACTED Thu Apr 22 10:11:42 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Thu, 22 Apr 2010 10:11:42 +0200 Subject: [erlang-questions] can "gen_tcp:recv" blocks my entire server? In-Reply-To: References: Message-ID: 2010/4/22 Paul Fisher > Have you tried it? > Yes of course I have. This is why I'm asking if recv/2 is the problem in my server design? > > -- > paul > > On Apr 21, 2010, at 6:16 PM, "zabrane Mikael" > wrote: > > > Hi List, > > > > I'm impleting a simple Web server using gen_tcp:listen(Port, [..., > > {active, > > false}]) options. > > For each request, a child process is spawned to handle it. > > Then, each child process use gen_tcp:recv/2 to read data from the > > client > > socket. > > Very basic design! > > > > My question is simple. If one of my child processes calls > > gen_tcp:recv/2 and > > blocks (due to a slow client), > > does this also blocks all other spawned childs? > > > > If yes, what's the best way to read from a socket without blocking the > > entire server (i.e the other spawned processes)? > > > > Regards > > Zabrane > From roberto@REDACTED Thu Apr 22 10:43:00 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 22 Apr 2010 10:43:00 +0200 Subject: [erlang-questions] monitor message queue of a process In-Reply-To: References: <4BCF4DBE.3040000@tmit.bme.hu> Message-ID: 2010/4/21 Kenneth Lundin : > On Wed, Apr 21, 2010 at 9:10 PM, Zoltan Lajos Kis wrote: >> Definitely not generic... >> But as a quick hack you can set a trace flag on your process in order to >> receive a message whenever the process receives one: >> erlang:trace(Pid, true, ['receive', {tracer, MonitorPid}]). >> This way you only need to check the queue length after the first n trace >> messages, then after n-m messages, etc... > > Using the tracing mechanisms as part of the normal execution will > destroy the possibilities to > trace on the system without destroying the function. Therefore this is > REALLY NOT RECOMMENDED. > > By the way , we will most likely optimize the selective receive for > the result of ?a gen_server call and similar until the next release > (in the compiler and the Erlang VM). > > /Kenneth Erlang/OTP, Ericsson thank you for the responses. this leads me into thinking that the only possible solution is therefore to have a message queuer buffer which must be actively interrogated by the process when it is ready to receive new messages. before i completely give up the idea of having some kind of automation without compromising efficiency, are there any other ideas? cheers, r. From roberto@REDACTED Thu Apr 22 10:56:59 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 22 Apr 2010 10:56:59 +0200 Subject: [erlang-questions] monitor message queue of a process In-Reply-To: References: <4BCF4DBE.3040000@tmit.bme.hu> Message-ID: just another thing. can someone who knows the deep magic behind gen_server calls answer this simple question: can you build in gen_server a service that provides buffering to a process performing selective receives, and in this case should i be better off with async or sync calls? thank you :) r. From mazen.harake@REDACTED Thu Apr 22 11:01:02 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Thu, 22 Apr 2010 11:01:02 +0200 Subject: [erlang-questions] can "gen_tcp:recv" blocks my entire server? In-Reply-To: References: Message-ID: <4BD0104E.6070603@erlang-solutions.com> Have you looked into {active,once} ? I think it is more suitable in your case. My suggestion would be to open an accepted socket with {active, false}, then setopts(Sock,[{active, once}]) when you are ready to receive the request (in your code) and then handle that and go back to setopts(Sock,[{active,once}]) every time you want to get a packet in your code. It will be received as a message and is much easier to handler. recv is IMHO only used when you are very specific about the way the data should be read and to keep tighter flow control. If you combine {active, once} with {packet, http} then you will have a much more stable ground to stand on, unless you want to hack-to-learn which I guess is good enough reason to do it any way you like :) My 2 cents :) GL, /Mazen On 22/04/2010 10:11, zabrane Mikael wrote: > 2010/4/22 Paul Fisher > > >> Have you tried it? >> >> > Yes of course I have. > This is why I'm asking if recv/2 is the problem in my server design? > > > > >> -- >> paul >> >> On Apr 21, 2010, at 6:16 PM, "zabrane Mikael" >> wrote: >> >> >>> Hi List, >>> >>> I'm impleting a simple Web server using gen_tcp:listen(Port, [..., >>> {active, >>> false}]) options. >>> For each request, a child process is spawned to handle it. >>> Then, each child process use gen_tcp:recv/2 to read data from the >>> client >>> socket. >>> Very basic design! >>> >>> My question is simple. If one of my child processes calls >>> gen_tcp:recv/2 and >>> blocks (due to a slow client), >>> does this also blocks all other spawned childs? >>> >>> If yes, what's the best way to read from a socket without blocking the >>> entire server (i.e the other spawned processes)? >>> >>> Regards >>> Zabrane >>> >> > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From roberto@REDACTED Thu Apr 22 11:06:25 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 22 Apr 2010 11:06:25 +0200 Subject: [erlang-questions] can "gen_tcp:recv" blocks my entire server? In-Reply-To: <4BD0104E.6070603@erlang-solutions.com> References: <4BD0104E.6070603@erlang-solutions.com> Message-ID: >>> On Apr 21, 2010, at 6:16 PM, "zabrane Mikael" >>> wrote: >>>> My question is simple. If one of my child processes calls >>>> gen_tcp:recv/2 and >>>> blocks (due to a slow client), >>>> does this also blocks all other spawned childs? >>>> >>>> If yes, what's the best way to read from a socket without blocking the >>>> entire server (i.e the other spawned processes)? this shouldn't be the case. that's one of the main reasons why it is a common pattern to spawn a process for every accepted socket. there's most probably something wrong somewhere in your server design. 2010/4/22 Mazen Harake : > Have you looked into {active,once} ? I think it is more suitable in your > case. My suggestion would be to open an accepted socket with {active, > false}, then setopts(Sock,[{active, once}]) when you are ready to receive > the request (in your code) and then handle that and go back to > setopts(Sock,[{active,once}]) every time you want to get a packet in your > code. pardon me, why shouldn't it be {active, once} from the beginning? just curious.. r. From mazen.harake@REDACTED Thu Apr 22 11:13:34 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Thu, 22 Apr 2010 11:13:34 +0200 Subject: [erlang-questions] can "gen_tcp:recv" blocks my entire server? In-Reply-To: References: <4BD0104E.6070603@erlang-solutions.com> Message-ID: <4BD0133E.3070004@erlang-solutions.com> On 22/04/2010 11:06, Roberto Ostinelli wrote: > pardon me, why shouldn't it be {active, once} from the beginning? just curious.. > > r. > It could be, if you prefer that, I guess... I was more giving the way I personally usually do it. Don't recall why I started doing things that way but it is the same really. Never said "it shouldn't be active once from the beginning", but you're right :) /M --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From rvirding@REDACTED Thu Apr 22 11:35:33 2010 From: rvirding@REDACTED (Robert Virding) Date: Thu, 22 Apr 2010 11:35:33 +0200 Subject: Clojure and OO, and Erlang Message-ID: Debasish Gosh (@debasishg) tweeted this very interesting link to a discussion on clojure, objects and OO. I think many of the arguments here also apply to erlang. "Do objects imply mutability ? an interesting discussion on #clojure .. http://clojure-log.n01se.net/date/2010-04-21.html#i28" Robert From s.merle@REDACTED Thu Apr 22 11:48:59 2010 From: s.merle@REDACTED (Sebastien Merle) Date: Thu, 22 Apr 2010 11:48:59 +0200 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> <3FFFC7AE-F622-4C41-8BFE-95E9B8646535@cs.otago.ac.nz> <4BCF1A67.5030302@di.uminho.pt> Message-ID: Taking about numbers, I was interested in the difference between parametrized modules and closures call overhead. And the result is that parametrized modules seems to be nearly two times faster than closure: 1> test:test(100000000). [{closure,10756202},{module,6001309}] Here's the code: ---------------------------------------- -module(test). -export([test/1, call/3]). -record(?MODULE, {count = 0}). test(Count) -> C = time(fun test_closure/1, Count), M = time(fun test_module/1, Count), [{closure, C}, {module, M}]. time(F, A) -> T1 = erlang:now(), F(A), T2 = erlang:now(), timer:now_diff(T2, T1). test_closure(Count) -> C1 = new_closure(42), C2 = new_closure(18), C1(call, {C2, Count}). test_module(Count) -> M1 = new_module(42), M2 = new_module(18), M1:call(M2, Count). new_closure(Start) -> make_closure(#?MODULE{count = Start}). new_module(Start) -> #?MODULE{count = Start}. call_closure(_Other, 0, State) -> State#?MODULE.count; call_closure(Other, Count, State) -> NewState = State#?MODULE{count = State#?MODULE.count + 1}, Other(call, {make_closure(NewState), Count - 1}). call_module(_Other, 0, State) -> State#?MODULE.count; call_module(Other, Count, State) -> NewState = State#?MODULE{count = State#?MODULE.count + 1}, Other:call(NewState, Count - 1). call(Other, Count, State) -> call_module(Other, Count, State). make_closure(State) -> fun(call, {Other, Count}) -> call_closure(Other, Count, State) end. ---------------------------------------- And yes I know it's not a real parametrized module, but the function calls are equivalent, isn't it ? -- Sebastien Merle. From rtrlists@REDACTED Thu Apr 22 12:41:08 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 22 Apr 2010 11:41:08 +0100 Subject: [erlang-questions] Clojure and OO, and Erlang In-Reply-To: References: Message-ID: On Thu, Apr 22, 2010 at 10:35 AM, Robert Virding wrote: > Debasish Gosh (@debasishg) tweeted this very interesting link to a > discussion on clojure, objects and OO. I think many of the arguments > here also apply to erlang. > > "Do objects imply mutability ? an interesting discussion on #clojure > .. http://clojure-log.n01se.net/date/2010-04-21.html#i28" > > Robert > > Reminds me a bit of William Cook's excellent "On Understanding Data Absraction, Revisited", talking about the differences between ADT's and Objects: http://wcook.blogspot.com/2009/11/on-understanding-data-absraction.html Robby From erlang@REDACTED Thu Apr 22 12:48:23 2010 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 22 Apr 2010 12:48:23 +0200 Subject: [erlang-questions] can "gen_tcp:recv" blocks my entire server? In-Reply-To: References: Message-ID: On Thu, Apr 22, 2010 at 1:12 AM, zabrane Mikael wrote: > Hi List, > > I'm impleting a simple Web server using gen_tcp:listen(Port, [...,{active, > false}]) options. > For each request, a child process is spawned to handle it. > Then, each child process use gen_tcp:recv/2 to read data from the client > socket. > Very basic design! > > My question is simple. If one of my child processes calls gen_tcp:recv/2 and > blocks (due to a slow client), > does this also blocks all other spawned childs? No You'd better post the code - perhaps your spawns don't do what you think they do, or some process has died, or something. The code should not block so if it does something else is wrong. Difficult to say without peering at the code. /Joe > > If yes, what's the best way to read from a socket without blocking the > entire server (i.e the other spawned processes)? > > Regards > Zabrane > From ulf.wiger@REDACTED Thu Apr 22 12:51:17 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 22 Apr 2010 12:51:17 +0200 Subject: [erlang-questions] monitor message queue of a process In-Reply-To: References: <4BCF4DBE.3040000@tmit.bme.hu> Message-ID: <4BD02A25.40902@erlang-solutions.com> Roberto Ostinelli wrote: > just another thing. > > can someone who knows the deep magic behind gen_server calls answer > this simple question: can you build in gen_server a service that > provides buffering to a process performing selective receives, and in > this case should i be better off with async or sync calls? I think the high-level way to approach this problem is to have job-based rate regulation at the inputs of the system. I have been working on such a component, which I am now getting very happy with. Our own tests have been extremely promising, and overall, it builds on our experiences from the AXD 301 and its derivatives, our instant messaging systems. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From rumata-estor@REDACTED Thu Apr 22 13:42:24 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Thu, 22 Apr 2010 15:42:24 +0400 Subject: dialyzer: user-defined types just synonyms? Message-ID: <4BD03620.5060709@nm.ru> I want to define my type as a synonym for built-in type. But I want dialyzer to warn me if a function is called with wrong type. I have following code and dialyzer says everything is nice. But I want it to warn about "StrFromStr = my_to_str(Str)" line. Can I make it? How? I remember Haskell's "newtype" would be of desired behaviour. -module(test). -export([start/1]). -type my_str() :: string(). -spec start(string()) -> tuple(). start(Str0) -> MyStr = str_to_my(Str0), Str = my_to_str(MyStr), MyFromMy = str_to_my(MyStr), StrFromStr = my_to_str(Str), {Str, MyFromMy, StrFromStr}. -spec str_to_my(string()) -> my_str(). str_to_my(Str) -> Str. -spec my_to_str(MyStr) -> string() when is_subtype(MyStr, my_str()). my_to_str(MyStr) -> MyStr. From roberto@REDACTED Thu Apr 22 13:48:51 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 22 Apr 2010 13:48:51 +0200 Subject: [erlang-questions] monitor message queue of a process In-Reply-To: <4BD02A25.40902@erlang-solutions.com> References: <4BCF4DBE.3040000@tmit.bme.hu> <4BD02A25.40902@erlang-solutions.com> Message-ID: 2010/4/22 Ulf Wiger : > I think the high-level way to approach this problem is to have > job-based rate regulation at the inputs of the system. > > I have been working on such a component, which I am now getting > very happy with. Our own tests have been extremely promising, and > overall, it builds on our experiences from the AXD 301 and its > derivatives, our instant messaging systems. > > BR, > Ulf W hi ulf, would you care to expand a little? of course, if this is compatible with your time/nda. i, for one, believe your experience could enlighten quite a lot in this domain. cheers, r. From zabrane3@REDACTED Thu Apr 22 14:23:21 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Thu, 22 Apr 2010 14:23:21 +0200 Subject: [erlang-questions] can "gen_tcp:recv" blocks my entire server? In-Reply-To: <4BD0104E.6070603@erlang-solutions.com> References: <4BD0104E.6070603@erlang-solutions.com> Message-ID: I'll try the {active,once} advice. Thanks! 2010/4/22 Mazen Harake > Have you looked into {active,once} ? I think it is more suitable in your > case. My suggestion would be to open an accepted socket with {active, > false}, then setopts(Sock,[{active, once}]) when you are ready to receive > the request (in your code) and then handle that and go back to > setopts(Sock,[{active,once}]) every time you want to get a packet in your > code. It will be received as a message and is much easier to handler. recv > is IMHO only used when you are very specific about the way the data should > be read and to keep tighter flow control. > > If you combine {active, once} with {packet, http} then you will have a much > more stable ground to stand on, unless you want to hack-to-learn which I > guess is good enough reason to do it any way you like :) > > My 2 cents :) > > GL, > > /Mazen > > > On 22/04/2010 10:11, zabrane Mikael wrote: > >> 2010/4/22 Paul Fisher >> >> >> >>> Have you tried it? >>> >>> >>> >> Yes of course I have. >> This is why I'm asking if recv/2 is the problem in my server design? >> >> >> >> >> >>> -- >>> paul >>> >>> On Apr 21, 2010, at 6:16 PM, "zabrane Mikael" >>> wrote: >>> >>> >>> >>>> Hi List, >>>> >>>> I'm impleting a simple Web server using gen_tcp:listen(Port, [..., >>>> {active, >>>> false}]) options. >>>> For each request, a child process is spawned to handle it. >>>> Then, each child process use gen_tcp:recv/2 to read data from the >>>> client >>>> socket. >>>> Very basic design! >>>> >>>> My question is simple. If one of my child processes calls >>>> gen_tcp:recv/2 and >>>> blocks (due to a slow client), >>>> does this also blocks all other spawned childs? >>>> >>>> If yes, what's the best way to read from a socket without blocking the >>>> entire server (i.e the other spawned processes)? >>>> >>>> Regards >>>> Zabrane >>>> >>>> >>> >>> >> >> > > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become > ERLANG SOLUTIONS LTD. > > www.erlang-solutions.com > > From zabrane3@REDACTED Thu Apr 22 14:26:23 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Thu, 22 Apr 2010 14:26:23 +0200 Subject: [erlang-questions] can "gen_tcp:recv" blocks my entire server? In-Reply-To: References: Message-ID: > > You'd better post the code - perhaps your spawns don't do what you think they do, or some > process has died, or something. The code should not block so if it > does something else is wrong. > Difficult to say without peering at the code. Nothing secret with my code. Here it is. - tws.erl is the main gen_server listeing for incoming connections. - echo.erl only echo back what it gets To build, simply: $ make $ ./start.sh 1> echo:start(). 2> echo:stop(). Regards Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: tws.tgz Type: application/x-gzip Size: 1697 bytes Desc: not available URL: From mononcqc@REDACTED Thu Apr 22 14:26:38 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 22 Apr 2010 08:26:38 -0400 Subject: [erlang-questions] Clojure and OO, and Erlang In-Reply-To: References: Message-ID: I'd like to link to the Scheme wiki, particularly the section "What aspects has the term 'object oriented' been assigned to?": http://community.schemewiki.org/?object-oriented-programming I think that unless we specify what kind of terms we include in the definition of OO, we're bound to hit our heads debating different concepts at once. On Thu, Apr 22, 2010 at 5:35 AM, Robert Virding wrote: > Debasish Gosh (@debasishg) tweeted this very interesting link to a > discussion on clojure, objects and OO. I think many of the arguments > here also apply to erlang. > > "Do objects imply mutability ? an interesting discussion on #clojure > .. http://clojure-log.n01se.net/date/2010-04-21.html#i28" > > Robert > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From matthew@REDACTED Thu Apr 22 13:41:51 2010 From: matthew@REDACTED (Matthew Sackman) Date: Thu, 22 Apr 2010 12:41:51 +0100 Subject: [erlang-questions] Re: testing asynchronous code In-Reply-To: <4BCFFCED.2070804@erlang-solutions.com> References: <328D34C0-2133-4D3E-958F-B7C9CA307D52@duomark.com> <20100421223851.GB16488@wellquite.org> <670EF79B-D523-48EA-9039-ED6DFD989CE9@duomark.com> <4BCFA272.3040801@m5net.com> <4BCFFCED.2070804@erlang-solutions.com> Message-ID: <20100422114150.GA2762@mrnibble.lshift.net> On Thu, Apr 22, 2010 at 09:38:21AM +0200, Ulf Wiger wrote: > The one ordering guarantee that exists is that if P1 sends a > sequence of messages to P2, they will arrive in the same order > as they were sent. Absolutely, and *even* if the VM happens to implement something stronger, there's no way you should trust it unless it also appears in some formal semantics. It's pretty well known that the Sun JVM used to (may still?) implement much stronger semantics than the JLS, leading to an awful lot of applications which worked fine on the Sun JVM but not on the IBM JVM which, whilst still compliant, was much less generous with its guarantees. In particular, we've always written Rabbit so that the *only* assumption it makes wrt ordering is the above. There are some interesting other things that we rely on by inspection of code, say, gen_server: eg, and this one turns out to be very useful: "if you return 'stop' from some handle_* callback *and* a reply term, gen_server will wait until *after* the terminate callback has completed *before* sending the reply", but things which are in the language itself or the VM, we're very careful to avoid false assumptions about. In fact, there are a lot of questions about exit signals being converted to messages, the ordering thereof (i.e. can the message caused by the exit signal of a process dying overtake other messages that process has sent?), what ordering is there on multiple monitors on a process on process death etc etc, which we just work around, assuming the worst, given that signals and especially monitors don't seem to be mentioned at all in the formal semantics as they stand. Matthew From rumata-estor@REDACTED Thu Apr 22 15:18:26 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Thu, 22 Apr 2010 17:18:26 +0400 Subject: [erlang-questions] dialyzer: user-defined types just synonyms? In-Reply-To: <4BD03620.5060709@nm.ru> References: <4BD03620.5060709@nm.ru> Message-ID: <4BD04CA2.80306@nm.ru> And one more question. I have a code: -module(test). -export([start/0]). -spec start() -> ok. start() -> %%check_atom(bad), case check_atom(good) of ok -> ok; error -> error end. -spec check_atom(good | nice) -> ok | error. check_atom(Atom) -> ok. Dialyzer shows: test.erl:10: The pattern 'error' can never match the type 'ok' I'd like it to pay more attention to -spec than to function's body so I wouldn't get this message. Is it possible? I understand that I can filter these messages with grep but this way I may throw away something important. Dmitry Belyaev wrote: > > I want to define my type as a synonym for built-in type. But I want > dialyzer to warn me if a function is called with wrong type. > I have following code and dialyzer says everything is nice. But I want > it to warn about "StrFromStr = my_to_str(Str)" line. > Can I make it? How? > > I remember Haskell's "newtype" would be of desired behaviour. > > -module(test). > > -export([start/1]). > > -type my_str() :: string(). > > -spec start(string()) -> tuple(). > start(Str0) -> > MyStr = str_to_my(Str0), > Str = my_to_str(MyStr), > MyFromMy = str_to_my(MyStr), > StrFromStr = my_to_str(Str), > {Str, MyFromMy, StrFromStr}. > > -spec str_to_my(string()) -> my_str(). > str_to_my(Str) -> > Str. > > -spec my_to_str(MyStr) -> string() when is_subtype(MyStr, my_str()). > my_to_str(MyStr) -> > MyStr. > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kostis@REDACTED Thu Apr 22 15:47:41 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 22 Apr 2010 16:47:41 +0300 Subject: [erlang-questions] dialyzer: user-defined types just synonyms? In-Reply-To: <4BD04CA2.80306@nm.ru> References: <4BD03620.5060709@nm.ru> <4BD04CA2.80306@nm.ru> Message-ID: <4BD0537D.9080703@cs.ntua.gr> Dmitry Belyaev wrote: > And one more question. I have a code: > -module(test). > > -export([start/0]). > > -spec start() -> ok. > start() -> > %%check_atom(bad), > case check_atom(good) of > ok -> ok; > error -> error > end. > > -spec check_atom(good | nice) -> ok | error. > check_atom(Atom) -> > ok. > > Dialyzer shows: > test.erl:10: The pattern 'error' can never match the type 'ok' > I'd like it to pay more attention to -spec than to function's body so I > wouldn't get this message. Is it possible? You've defined two functions in your example, one that returns either 'ok' or 'error' and you give it a spec that says it only returns 'ok' and one that clearly returns 'ok' only and your spec claims it returns 'error' also. Why on earth you would you want to do that in your program? If you want obfuscated code, write in C instead. Dialyzer has detected that you are clearly an evil programmer :-) Why should it be nice to you? Kostis From rumata-estor@REDACTED Thu Apr 22 15:57:09 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Thu, 22 Apr 2010 17:57:09 +0400 Subject: [erlang-questions] dialyzer: user-defined types just synonyms? In-Reply-To: <4BD0537D.9080703@cs.ntua.gr> References: <4BD03620.5060709@nm.ru> <4BD04CA2.80306@nm.ru> <4BD0537D.9080703@cs.ntua.gr> Message-ID: <4BD055B5.9070607@nm.ru> Cause I may extend this function in the future. Or got this version after refactoring and don't want to change code that uses it. Kostis Sagonas wrote: > > Dmitry Belyaev wrote: >> And one more question. I have a code: >> -module(test). >> >> -export([start/0]). >> >> -spec start() -> ok. >> start() -> >> %%check_atom(bad), >> case check_atom(good) of >> ok -> ok; >> error -> error >> end. >> >> -spec check_atom(good | nice) -> ok | error. >> check_atom(Atom) -> >> ok. >> >> Dialyzer shows: >> test.erl:10: The pattern 'error' can never match the type 'ok' >> I'd like it to pay more attention to -spec than to function's body so >> I wouldn't get this message. Is it possible? > > You've defined two functions in your example, one that returns either > 'ok' or 'error' and you give it a spec that says it only returns 'ok' > and one that clearly returns 'ok' only and your spec claims it returns > 'error' also. Why on earth you would you want to do that in your > program? If you want obfuscated code, write in C instead. > > Dialyzer has detected that you are clearly an evil programmer :-) > Why should it be nice to you? > > Kostis > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kostis@REDACTED Thu Apr 22 15:58:27 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 22 Apr 2010 16:58:27 +0300 Subject: [erlang-questions] dialyzer: user-defined types just synonyms? In-Reply-To: <4BD03620.5060709@nm.ru> References: <4BD03620.5060709@nm.ru> Message-ID: <4BD05603.8010904@cs.ntua.gr> Dmitry Belyaev wrote: > I want to define my type as a synonym for built-in type. But I want > dialyzer to warn me if a function is called with wrong type. > I have following code and dialyzer says everything is nice. But I want > it to warn about "StrFromStr = my_to_str(Str)" line. > Can I make it? How? You cannot make it. The notion of equality between types is structural, not nominative. (http://en.wikipedia.org/wiki/Structural_type_system) Kostis > I remember Haskell's "newtype" would be of desired behaviour. > > -module(test). > > -export([start/1]). > > -type my_str() :: string(). > > -spec start(string()) -> tuple(). > start(Str0) -> > MyStr = str_to_my(Str0), > Str = my_to_str(MyStr), > MyFromMy = str_to_my(MyStr), > StrFromStr = my_to_str(Str), > {Str, MyFromMy, StrFromStr}. > > -spec str_to_my(string()) -> my_str(). > str_to_my(Str) -> > Str. > > -spec my_to_str(MyStr) -> string() when is_subtype(MyStr, my_str()). > my_to_str(MyStr) -> > MyStr. > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From rumata-estor@REDACTED Thu Apr 22 16:16:01 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Thu, 22 Apr 2010 18:16:01 +0400 Subject: [erlang-questions] dialyzer: user-defined types just synonyms? In-Reply-To: References: <4BD03620.5060709@nm.ru> <4BD04CA2.80306@nm.ru> Message-ID: <4BD05A21.5040705@nm.ru> I thought type specs are made for external typechecks. Ok. Now I have this code: -module(test). -export([start/0]). -spec start() -> ok. start() -> AtomBad = get_atom(1), AtomGood = get_atom(2), check_atom(AtomBad), case check_atom(AtomGood) of ok -> ok; error -> error end. -spec get_atom(any()) -> good | bad. get_atom(1) -> bad; get_atom(_) -> good. -spec check_atom(good | nice) -> ok | error. check_atom(Atom) -> ok. It doesn't warn me about check_atom(AtomBad). Tobias Lindahl wrote: > 2010/4/22 Dmitry Belyaev : > >> And one more question. I have a code: >> -module(test). >> >> -export([start/0]). >> >> -spec start() -> ok. >> start() -> >> %%check_atom(bad), >> case check_atom(good) of >> ok -> ok; >> error -> error >> end. >> >> -spec check_atom(good | nice) -> ok | error. >> check_atom(Atom) -> >> ok. >> >> Dialyzer shows: >> test.erl:10: The pattern 'error' can never match the type 'ok' >> I'd like it to pay more attention to -spec than to function's body so I >> wouldn't get this message. Is it possible? >> > > Once again, no. You cannot widen a success typing of a function using > a spec. The type of the function will be the intersection of the type > in the spec and whatever type Dialyzer infers for the function. In > this case it finds that the error clause is unreachable since the > function unconditionally returns ok. > > Tobias > > From rumata-estor@REDACTED Thu Apr 22 16:19:32 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Thu, 22 Apr 2010 18:19:32 +0400 Subject: [erlang-questions] dialyzer: user-defined types just synonyms? In-Reply-To: <4BD05603.8010904@cs.ntua.gr> References: <4BD03620.5060709@nm.ru> <4BD05603.8010904@cs.ntua.gr> Message-ID: <4BD05AF4.2050505@nm.ru> So is it impossible to distinguish plain string came from user and string prepared for database (escaped)? I don't want to put it in some container like tuple. Kostis Sagonas wrote: > > Dmitry Belyaev wrote: >> I want to define my type as a synonym for built-in type. But I want >> dialyzer to warn me if a function is called with wrong type. >> I have following code and dialyzer says everything is nice. But I >> want it to warn about "StrFromStr = my_to_str(Str)" line. >> Can I make it? How? > > You cannot make it. The notion of equality between types is > structural, not nominative. > (http://en.wikipedia.org/wiki/Structural_type_system) > > Kostis > >> I remember Haskell's "newtype" would be of desired behaviour. >> >> -module(test). >> >> -export([start/1]). >> >> -type my_str() :: string(). >> >> -spec start(string()) -> tuple(). >> start(Str0) -> >> MyStr = str_to_my(Str0), >> Str = my_to_str(MyStr), >> MyFromMy = str_to_my(MyStr), >> StrFromStr = my_to_str(Str), >> {Str, MyFromMy, StrFromStr}. >> >> -spec str_to_my(string()) -> my_str(). >> str_to_my(Str) -> >> Str. >> >> -spec my_to_str(MyStr) -> string() when is_subtype(MyStr, my_str()). >> my_to_str(MyStr) -> >> MyStr. >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kostis@REDACTED Thu Apr 22 16:18:58 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 22 Apr 2010 17:18:58 +0300 Subject: [erlang-questions] dialyzer: user-defined types just synonyms? In-Reply-To: <4BD05A21.5040705@nm.ru> References: <4BD03620.5060709@nm.ru> <4BD04CA2.80306@nm.ru> <4BD05A21.5040705@nm.ru> Message-ID: <4BD05AD2.900@cs.ntua.gr> Dmitry Belyaev wrote: > I thought type specs are made for external typechecks. Ok. Now I have > this code: > -module(test). > > -export([start/0]). > > -spec start() -> ok. > start() -> > AtomBad = get_atom(1), > AtomGood = get_atom(2), > check_atom(AtomBad), > case check_atom(AtomGood) of > ok -> ok; > error -> error > end. > > -spec get_atom(any()) -> good | bad. > get_atom(1) -> > bad; > get_atom(_) -> > good. > > -spec check_atom(good | nice) -> ok | error. > check_atom(Atom) -> > ok. > > It doesn't warn me about check_atom(AtomBad). It does; see below. You are using an older Erlang/OTP. Upgrade to R13B04. Kostis test.erl:12: The pattern 'error' can never match the type 'ok' test.erl:21: The specification for test:check_atom/1 states that the function might also return 'error' but the inferred return is 'ok' From rumata-estor@REDACTED Thu Apr 22 16:26:09 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Thu, 22 Apr 2010 18:26:09 +0400 Subject: [erlang-questions] dialyzer: user-defined types just synonyms? In-Reply-To: References: <4BD03620.5060709@nm.ru> <4BD04CA2.80306@nm.ru> Message-ID: <4BD05C81.1020409@nm.ru> It would be nice if it would intersect spec parameters and check spec result contain what dialyzer infers. So, if I have -spec function(A) -> B. and dialyzer finds that real type as function(C) -> D then it would check that C contains A and B contains D and its final type would be function(C) -> B. Tobias Lindahl wrote: > 2010/4/22 Dmitry Belyaev : > >> And one more question. I have a code: >> -module(test). >> >> -export([start/0]). >> >> -spec start() -> ok. >> start() -> >> %%check_atom(bad), >> case check_atom(good) of >> ok -> ok; >> error -> error >> end. >> >> -spec check_atom(good | nice) -> ok | error. >> check_atom(Atom) -> >> ok. >> >> Dialyzer shows: >> test.erl:10: The pattern 'error' can never match the type 'ok' >> I'd like it to pay more attention to -spec than to function's body so I >> wouldn't get this message. Is it possible? >> > > Once again, no. You cannot widen a success typing of a function using > a spec. The type of the function will be the intersection of the type > in the spec and whatever type Dialyzer infers for the function. In > this case it finds that the error clause is unreachable since the > function unconditionally returns ok. > > Tobias > > From james.aimonetti@REDACTED Thu Apr 22 16:28:03 2010 From: james.aimonetti@REDACTED (James Aimonetti) Date: Thu, 22 Apr 2010 10:28:03 -0400 Subject: [erlang-questions] Re: The If expression In-Reply-To: References: Message-ID: <4BD05CF3.3070908@gmail.com> This thread is well-timed as I have an issue I wanted some help on. I have two lists, one of keys and one of values. I want to zip them together, but I have one specific key/value combo that I want to exclude from the zipped list. For context, this is creating an appropriate {proplist} for couchbeam. to_couchbeam(Fields, Values) -> { lists:flatten( lists:zipwith(fun to_couchbeam_zipper/2, Fields, Values) ) }. to_couchbeam_zipper(F, V) -> K = to_couchbeam_key(F), Val = to_couchbeam_value(V), case Key == <<"_rev">> andalso Val == undefined of true -> []; false -> {K, Val} end. to_couchbeam_* converts various terms to formats suitable for couchbeam to save the document. I only want to remove the '_rev' key when it's value is undefined, otherwise couchbeam fails to save the document. If '_rev' has a defined value, great, keep it in the generated proplist. I refactored according to some of the advice earlier, so now to_couchbeam_zipper is to_couchbeam_zipper('_rev', undefined) -> []; to_couchbeam_zipper(F, V) -> { to_couchbeam_key(F), to_couchbeam_value(V) }. So I may have answered my own question here, as concerns the case statement. Is there a cleaner way to zip two lists in this fashion without resorting to inserting an empty list for unwanted k/v pairs and flattening the resulting proplist? Thanks, James Jay Nelson wrote: > Henning Deidrich offered one-armed ifs to rewrite: > > I never use the if statement. It doesn't even occur to me to use it, > although writing Java or something else I use it all the time > naturally. Here's the style I would use for some of your cases (some > of the others may not have enough context to be good examples for > rewriting): > > > (2) > > verbose_echo(Verbose, Format, Para) -> > > > > % Don't print if verbosity is off (false). > > >>> if Verbose -> echo(Format, Para); true -> nil end. > > verbose_echo(true, Format, Para) -> > echo(Format, Para); > verbose_echo(_, _Format, _Para) -> > nil. > > It doesn't do what the comment says, and you have to consider whether > nil is the value you want back. It depends on what echo/2 returns as > to what is appropriate (and whether the caller even cares about the > return value). With this approach it is easier to test and easier to > add new cases (verbosity levels) or change the existing one. > > > > (3) > > % If exists, drop old before adding new element > > Prev = gl:get(suite, Suite), > > >>> if Prev /= undefined -> gl:drop(suite, Suite); true -> > > nil end, > > gl:add(suite, Suite), > > Prev = case gl:get(suite, Suite) -> > undefined -> nil; > Defined -> gl:drop(suite, Suite), Defined > end, > gl:add(suite, Suite), > > > Here I made the assumption that you needed the value of Prev later. > If you don't, you can drop it out and not care what the return value > of the case statement is. > > This is actually a common code smell to watch for. You are setting a > variable, then testing for a negative value using /= constant. > Instead, use case and make the first clause the constant you want to > exclude, everything else will match the catch-all clause. > > Even when I have case ... of true -> ...; false -> ... end I find it > more comforting in erlang to know I covered the cases I care about. > You can even use a single armed case if you want a freak value to > crash your process. > > > (5) > > safe_unregister(Name) -> > > Registered = whereis(Name) /= undefined, > > >>> if Registered -> unregister(Name); true -> nil end. > > safe_unregister(Name) -> > case whereis(Name) of > undefined -> ok; > Other -> unregister(Name), ok > end. > > Another instance of #3. You can easily fix the return values to > different ones, or have a return value after the case ... end. If you > need to know the value of whereis, you can modify it to this: > > safe_unregister(Name) -> > Loc = case whereis(Name) of > undefined -> undefined; > Other -> unregister(Name), Other > end, > ... computations involving Loc ... > %% or even return it... > Loc. > > > Any time you find yourself binding a variable, and then soon after > wanting to modify it or do an if based on a small set of values, stop, > back up and try using a case statement. It has the advantage of > giving you multiple variables with delayed binding of the correct one > as your chosen value: > > Chosen = case multi_var_fun() of > Val1 -> Val1; > Val2 -> f(Val2); > Val3 when Val3 > 0, Val3 < 5 -> g(Val3); > Other -> h(Other) > end, > > Here Chosen is a late-bound variable with several values visited as > intermediate bindings before arriving at a final choice. An > imperative programmer tries to do something like: > > Chosen = multi_var_fun(), > if (Chosen == Val1) ... > else ... > > The erlang way is to avoid binding the variable you care about until > you are absolutely sure you have the _value_ you care about, rather > than modifying the value along the way. A one-armed if is > semantically like saying I want Val1 most of the time, and in a few > cases I want to change it to a different value. > > jay > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- James Aimonetti mobile: 314.809.6307 work: 540.459.2220 email: james.aimonetti@REDACTED website: http://jamesaimonetti.com From rumata-estor@REDACTED Thu Apr 22 16:32:18 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Thu, 22 Apr 2010 18:32:18 +0400 Subject: [erlang-questions] dialyzer: user-defined types just synonyms? In-Reply-To: <4BD05AD2.900@cs.ntua.gr> References: <4BD03620.5060709@nm.ru> <4BD04CA2.80306@nm.ru> <4BD05A21.5040705@nm.ru> <4BD05AD2.900@cs.ntua.gr> Message-ID: <4BD05DF2.1010701@nm.ru> Yes, I use R1303. But the desired warning is about line 11: check_atom(AtomBad) would fail because AtomBad type is 'bad'. Or at least ('good'|'bad'). Kostis Sagonas wrote: > > Dmitry Belyaev wrote: >> I thought type specs are made for external typechecks. Ok. Now I have >> this code: >> -module(test). >> >> -export([start/0]). >> >> -spec start() -> ok. >> start() -> >> AtomBad = get_atom(1), >> AtomGood = get_atom(2), >> check_atom(AtomBad), >> case check_atom(AtomGood) of >> ok -> ok; >> error -> error >> end. >> >> -spec get_atom(any()) -> good | bad. >> get_atom(1) -> >> bad; >> get_atom(_) -> >> good. >> >> -spec check_atom(good | nice) -> ok | error. >> check_atom(Atom) -> >> ok. >> >> It doesn't warn me about check_atom(AtomBad). > > It does; see below. You are using an older Erlang/OTP. > Upgrade to R13B04. > > Kostis > > test.erl:12: The pattern 'error' can never match the type 'ok' > test.erl:21: The specification for test:check_atom/1 states that the > function might also return 'error' but the inferred return is 'ok' > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From anders@REDACTED Thu Apr 22 16:54:21 2010 From: anders@REDACTED (Anders Dahlin) Date: Thu, 22 Apr 2010 16:54:21 +0200 Subject: [erlang-questions] Re: The If expression In-Reply-To: <4BD05CF3.3070908@gmail.com> References: <4BD05CF3.3070908@gmail.com> Message-ID: <4BD0631D.5060408@dahlinenergy.se> Alt using lists:foldl/3 to_couchbeam(Fields, Values) -> Fun = fun ('_rev', {[undefined| Vs], Acc}) -> {Vs, Acc}; (F, {[V| Vs], Acc}) -> {Vs, Acc ++ [{to_couchbeam_key(F), to_couchbeam_value(V)}]} end, {_, Res} = lists:foldl(Fun, {Values, []}, Fields), Res. Alt using functions: to_couchbeam(Fields, Values) -> to_couchbeam(Fields, Values, []). to_couchbeam([], [], Acc) -> Acc; to_couchbeam(['_rev'| Fields], [undefined| Values], Acc) -> to_couchbeam(Fields, Values, Acc); to_couchbeam([F| Fields], [V| Values], Acc) -> to_couchbeam(Fields, Values, Acc ++ [{to_couchbeam_key(F), to_couchbeam_value(V)}]). Completely untested... On 2010-04-22 16:28, James Aimonetti wrote: > This thread is well-timed as I have an issue I wanted some help on. > > I have two lists, one of keys and one of values. I want to zip them > together, but I have one specific key/value combo that I want to exclude > from the zipped list. For context, this is creating an appropriate > {proplist} for couchbeam. > > to_couchbeam(Fields, Values) -> > { lists:flatten( lists:zipwith(fun to_couchbeam_zipper/2, Fields, > Values) ) }. > > to_couchbeam_zipper(F, V) -> > K = to_couchbeam_key(F), > Val = to_couchbeam_value(V), > case Key == <<"_rev">> andalso Val == undefined of > true -> []; > false -> {K, Val} > end. > > to_couchbeam_* converts various terms to formats suitable for couchbeam > to save the document. I only want to remove the '_rev' key when it's > value is undefined, otherwise couchbeam fails to save the document. If > '_rev' has a defined value, great, keep it in the generated proplist. > > I refactored according to some of the advice earlier, so now > to_couchbeam_zipper is > > to_couchbeam_zipper('_rev', undefined) -> []; > to_couchbeam_zipper(F, V) -> { to_couchbeam_key(F), > to_couchbeam_value(V) }. > > So I may have answered my own question here, as concerns the case > statement. Is there a cleaner way to zip two lists in this fashion > without resorting to inserting an empty list for unwanted k/v pairs and > flattening the resulting proplist? > > Thanks, > > James > > Jay Nelson wrote: >> Henning Deidrich offered one-armed ifs to rewrite: >> >> I never use the if statement. It doesn't even occur to me to use it, >> although writing Java or something else I use it all the time >> naturally. Here's the style I would use for some of your cases (some >> of the others may not have enough context to be good examples for >> rewriting): >> >> > (2) >> > verbose_echo(Verbose, Format, Para) -> >> > >> > % Don't print if verbosity is off (false). >> > >>> if Verbose -> echo(Format, Para); true -> nil end. >> >> verbose_echo(true, Format, Para) -> >> echo(Format, Para); >> verbose_echo(_, _Format, _Para) -> >> nil. >> >> It doesn't do what the comment says, and you have to consider whether >> nil is the value you want back. It depends on what echo/2 returns as >> to what is appropriate (and whether the caller even cares about the >> return value). With this approach it is easier to test and easier to >> add new cases (verbosity levels) or change the existing one. >> >> >> > (3) >> > % If exists, drop old before adding new element >> > Prev = gl:get(suite, Suite), >> > >>> if Prev /= undefined -> gl:drop(suite, Suite); true -> >> > nil end, >> > gl:add(suite, Suite), >> >> Prev = case gl:get(suite, Suite) -> >> undefined -> nil; >> Defined -> gl:drop(suite, Suite), Defined >> end, >> gl:add(suite, Suite), >> >> >> Here I made the assumption that you needed the value of Prev later. >> If you don't, you can drop it out and not care what the return value >> of the case statement is. >> >> This is actually a common code smell to watch for. You are setting a >> variable, then testing for a negative value using /= constant. >> Instead, use case and make the first clause the constant you want to >> exclude, everything else will match the catch-all clause. >> >> Even when I have case ... of true -> ...; false -> ... end I find it >> more comforting in erlang to know I covered the cases I care about. >> You can even use a single armed case if you want a freak value to >> crash your process. >> >> > (5) >> > safe_unregister(Name) -> >> > Registered = whereis(Name) /= undefined, >> > >>> if Registered -> unregister(Name); true -> nil end. >> >> safe_unregister(Name) -> >> case whereis(Name) of >> undefined -> ok; >> Other -> unregister(Name), ok >> end. >> >> Another instance of #3. You can easily fix the return values to >> different ones, or have a return value after the case ... end. If you >> need to know the value of whereis, you can modify it to this: >> >> safe_unregister(Name) -> >> Loc = case whereis(Name) of >> undefined -> undefined; >> Other -> unregister(Name), Other >> end, >> ... computations involving Loc ... >> %% or even return it... >> Loc. >> >> >> Any time you find yourself binding a variable, and then soon after >> wanting to modify it or do an if based on a small set of values, stop, >> back up and try using a case statement. It has the advantage of >> giving you multiple variables with delayed binding of the correct one >> as your chosen value: >> >> Chosen = case multi_var_fun() of >> Val1 -> Val1; >> Val2 -> f(Val2); >> Val3 when Val3 > 0, Val3 < 5 -> g(Val3); >> Other -> h(Other) >> end, >> >> Here Chosen is a late-bound variable with several values visited as >> intermediate bindings before arriving at a final choice. An >> imperative programmer tries to do something like: >> >> Chosen = multi_var_fun(), >> if (Chosen == Val1) ... >> else ... >> >> The erlang way is to avoid binding the variable you care about until >> you are absolutely sure you have the _value_ you care about, rather >> than modifying the value along the way. A one-armed if is >> semantically like saying I want Val1 most of the time, and in a few >> cases I want to change it to a different value. >> >> jay >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > From ulf.wiger@REDACTED Thu Apr 22 17:07:31 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 22 Apr 2010 17:07:31 +0200 Subject: [erlang-questions] monitor message queue of a process In-Reply-To: References: <4BCF4DBE.3040000@tmit.bme.hu> <4BD02A25.40902@erlang-solutions.com> Message-ID: <4BD06633.2010709@erlang-solutions.com> Roberto Ostinelli wrote: > > hi ulf, > > would you care to expand a little? of course, if this is compatible > with your time/nda. > > i, for one, believe your experience could enlighten quite a lot in this domain. That is my hope too. :) I hope to be able to expand on this very soon. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From rumata-estor@REDACTED Thu Apr 22 17:13:17 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Thu, 22 Apr 2010 19:13:17 +0400 Subject: [erlang-questions] dialyzer: user-defined types just synonyms? In-Reply-To: References: <4BD03620.5060709@nm.ru> <4BD04CA2.80306@nm.ru> <4BD05C81.1020409@nm.ru> Message-ID: <4BD0678D.6080400@nm.ru> I saw those options already but they don't do what I'd like to see. I have very complex system. And I am _heavily_ refactoring it. And thought dialyzer would be of great help for me. Now I see many functions that long ago had (and may will have in the future) wider result type than now dialyzer infers. I get many messages like those (currently I grep them from output): .*: The pattern .* can never match since previous clauses completely covered the type .* .*: The pattern .* can never match the type .* .*: The variable .* can never match since previous clauses completely covered the type .* I don't want to change this code (it may be needed in future). I'd better make result type of function wider, so dialyzer won't warn about superflous (by this moment) code. Tobias Lindahl wrote: > 2010/4/22 Dmitry Belyaev : > >> It would be nice if it would intersect spec parameters and check spec result >> contain what dialyzer infers. >> >> So, if I have >> -spec function(A) -> B. >> and dialyzer finds that real type as function(C) -> D then it would check >> that C contains A and B contains D and its final type would be >> function(C) -> B. >> >> > > Dialyzer checks that there is a non-empty intersection between the > spec type and the infered type. In other words if the spec is possible > at all. If not, you will get a warning. > > You might want to have a look at the options -Wunderspecs and > -Woverspecs. They give you some more information about how the specs > relate to the inferred types. They are off by default since you > typically get a lot of warnings from them. > > Tobias > > From rumata-estor@REDACTED Thu Apr 22 17:35:56 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Thu, 22 Apr 2010 19:35:56 +0400 Subject: [erlang-questions] dialyzer: user-defined types just synonyms? In-Reply-To: References: <4BD03620.5060709@nm.ru> <4BD04CA2.80306@nm.ru> <4BD05C81.1020409@nm.ru> <4BD0678D.6080400@nm.ru> Message-ID: <4BD06CDC.8080701@nm.ru> Good point about unused=untested. Ok, but what about the other problem? http://pastebin.com/q7muyTnz There must be warning on line 9. I get nothing with R13b4. Tobias Lindahl wrote: > 2010/4/22 Dmitry Belyaev : > >> I saw those options already but they don't do what I'd like to see. >> >> I have very complex system. And I am _heavily_ refactoring it. And thought >> dialyzer would be of great help for me. >> Now I see many functions that long ago had (and may will have in the future) >> wider result type than now dialyzer infers. I get many messages like those >> (currently I grep them from output): >> >> .*: The pattern .* can never match since previous clauses completely covered >> the type .* >> .*: The pattern .* can never match the type .* >> .*: The variable .* can never match since previous clauses completely >> covered the type .* >> >> I don't want to change this code (it may be needed in future). I'd better >> make result type of function wider, so dialyzer won't warn about superflous >> (by this moment) code. >> > > I'm sorry that I can't help you with this. > > However, if you are refactoring your code and creating a lot of dead > code as you do it, it might not be a bad idea to remove the dead code. > The argument that you might need it in the future is a bit weak since > the code will be lying around and being untested until that. > > Personally, I would have removed the code since it then becomes clear > that whatever functionality it implements has not been needed, nor > tested, since the refactoring. > > Tobias > > > >> Tobias Lindahl wrote: >> >>> 2010/4/22 Dmitry Belyaev : >>> >>> >>>> It would be nice if it would intersect spec parameters and check spec >>>> result >>>> contain what dialyzer infers. >>>> >>>> So, if I have >>>> -spec function(A) -> B. >>>> and dialyzer finds that real type as function(C) -> D then it would check >>>> that C contains A and B contains D and its final type would be >>>> function(C) -> B. >>>> >>>> >>>> >>> Dialyzer checks that there is a non-empty intersection between the >>> spec type and the infered type. In other words if the spec is possible >>> at all. If not, you will get a warning. >>> >>> You might want to have a look at the options -Wunderspecs and >>> -Woverspecs. They give you some more information about how the specs >>> relate to the inferred types. They are off by default since you >>> typically get a lot of warnings from them. >>> >>> Tobias >>> >>> >>> >> > > From psa@REDACTED Thu Apr 22 19:21:49 2010 From: psa@REDACTED (=?UTF-8?B?UGF1bG8gU8KOZXJnaW8gQWxtZWlkYQ==?=) Date: Thu, 22 Apr 2010 18:21:49 +0100 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> <3FFFC7AE-F622-4C41-8BFE-95E9B8646535@cs.otago.ac.nz> <4BCF1A67.5030302@di.uminho.pt> Message-ID: <4BD085AD.7050602@di.uminho.pt> > If we want to compute at startup time some value to serve as a "parameter", > that value will have to be stored somewhere, > AND THAT SOMEWHERE CAN BE A MODULE. > > There is no law that says a module can't be written by a program. Your are right, but it is more unpleasant/cumbersome than using parameterized modules. > -module(reload). > -export([put/1,get/0]). > > put(Datum) -> > {ok,Device} = file:open("reloadable.erl", [write]), > io:format(Device, > "-module(reloadable).~n-export([datum/0]).~ndatum() ->~n ~p > .~n", > [Datum]), > ok = file:close(Device), > compile:file("reloadable", []), > code:purge(reloadable), > code:load_file(reloadable). Interesting that this reminds the hypothetical implementation for parameterized modules I sketched in the previous mail, in which a module would be compiled with the parameter substituted. But here we use a well-know name for the module, and do not have to propagate the generated module name in a variable. > And yes, I do realise that what we have here is a global mutable > variable with an amazingly slow assignment statement, > but that's precisely what a changeable configuration parameter IS. > > The overhead of of setting the parameter up (or changing it) is > moderately high (although using compile:forms(..., [binary,...]) > would be more efficient as well as safer). But the overhead of > *using* the parameter is the same as the overhead of any > cross-module function call. And if that weren't tolerable, we > wouldn't be using Erlang. In this case I agree that the overhead of using it is negligible. But while it may be common that a configuration is something done at start time and we can afford it to be slow (like in my database); not always may that be the case, and having to resort to a mechamism (like module compilation) that implies slow (re)configuration may sometimes be a problem. >> Now I remember that one of the "permissible" uses for the process >> dictionary is to store "parameters" written once but never changed >> later. This kind of use ties code with the process structure and the >> use of get/1 can be slow. > > Slow? Time for some numbers. > > 6> getput:k(100000000). > [{variant,constant},{result,100000000},{time,530}] > 7> getput:b(100000000). > [{variant,direct},{result,100000000},{time,1730}] > 8> getput:t(100000000). > [{variant,dictionary},{result,100000000},{time,2290}] > > Here's the code that produced those: > > -module(getput). > -export([t/1, b/1, k/1]). > > t(N) -> > put(key, 1), > {T0,_} = statistics(runtime), > R = loop(N, 0), > {T1,_} = statistics(runtime), > [{variant,dictionary},{result,R}, {time,T1-T0}]. > > loop(0, R) -> R; > loop(N, R) -> loop(N-1, R+get(key)). > > b(N) -> > {T0,_} = statistics(runtime), > R = loup(N, 0), > {T1,_} = statistics(runtime), > [{variant,direct},{result,R}, {time,T1-T0}]. > > loup(0, R) -> R; > loup(N, R) -> loup(N-1, R+(N div N)). > > k(N) -> > {T0,_} = statistics(runtime), > R = lowp(N, 0), > {T1,_} = statistics(runtime), > [{variant,constant},{result,R}, {time,T1-T0}]. > > lowp(0, R) -> R; > lowp(N, R) -> lowp(N-1, R+1). > > So > - the loop that just adds 1 takes 5.3 ns per iteration > - the loop that adds N div N takes 17.3 ns per iteration > - the loop that uses get() takes 22.9 ns per iteration > We conclude that > - N div N takes 12 ns > - get(key) takes 17.6 ns > and therefore > EITHER my benchmark and interpretation are hopelessly fouled up > OR using get/1 is NOT particularly slow. > > To be honest, I incline to the former; how can looking something up > in a hash table be so good compared with an integer division? I also found these numbers fishy. I thought about it and wondered if a considerable part of time is being spent dealing with operations on integers. I have a doubt: do big integers start larger in 64 bits than in 32? The efficiency guide still just says for small integers: Integer (-16#7FFFFFF < i <16#7FFFFFF). So either things are still the same or the guide has not been updated. If things are the same as for 32 bits, the time may be spent manipulating big integers. To test this hypothesis I added another variant: s(N) -> {T0,_} = statistics(runtime), R = losp(N, 0), {T1,_} = statistics(runtime), [{variant,small_numbers},{result,R}, {time,T1-T0}]. losp(0, R) -> R; losp(N, R) -> losp(N-1, (R+1) band 255). This variant also does some manipulation of a parameter, but the result is always a small integer. Running: air:tmp psa$ erlc +native getput.erl air:tmp psa$ erl Erlang R13B01 (erts-5.7.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) 1> getput:k(100000000). [{variant,constant},{result,100000000},{time,610}] 2> getput:s(100000000). [{variant,small_numbers},{result,0},{time,340}] 3> getput:b(100000000). [{variant,direct},{result,100000000},{time,3620}] 5> getput:t(100000000). [{variant,dictionary},{result,100000000},{time,7110}] Here, the "s" variant is event faster than the "k" one. This version is also accessing a parameter and doing some computation, but only doing an increment at most on a big integer. But this does not explain the large time in the "b" variant. I though the problem could be the "div" operation. Added the variant: d(N) -> {T0,_} = statistics(runtime), R = lodp(N, 0), {T1,_} = statistics(runtime), [{variant,no_div},{result,R}, {time,T1-T0}]. lodp(0, R) -> R; lodp(N, R) -> lodp(N-1, R + ((N+1) - N)). which results in: 1> getput:d(100000000). [{variant,no_div},{result,100000000},{time,730}] Conclusion: the time for the non-get versions comes from the use of "div" and the number of times a possibly big integer is manipulated. To measure more accurately what time "get" takes. I wrote: -module(m). -export([v/1, g/1]). loop(_F, 0) -> ok; loop(F, N) -> F(), loop(F, N-1). run(F, N) -> T1 = now(), loop(F, N), T2 = now(), timer:now_diff(T2, T1) * 1000 div N. v(N) -> run(fun() -> 1 end, N). g(N) -> put(key, 1), run(fun() -> get(key) end, N). Here the "v" function loops a function which just returns 1, and the "g" version is only different in that a get is performed. The result is the time elapsed in nanoseconds. erlc +native m.erl air:code psa$ erl Erlang R13B01 (erts-5.7.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) 1> m:v(100000000). 8 2> m:g(100000000). 73 So, the get takes around 65 nanosecs. To test the use of parameters of parameterized modules I wrote a module: -module(mp, [P]). exactly the same as "m", with an extra function: p(N) -> run(fun() -> P end, N). Running again: air:code psa$ erlc +native mp.erl air:code psa$ erl Erlang R13B01 (erts-5.7.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) 1> M = mp:new(10). {mp,10} 2> M:v(100000000). 8 3> M:p(100000000). 8 4> M:g(100000000). 71 Conclusion: using a parameter is negligible; using a parameterized module in this case is negligible compared to what the module did before. I remain with the opinion I had before, that get is too slow (for this purpose, compared with using parameters). If I need to lookup up a dozen parameters to serve a request, there goes almost 1 microsecond of wasted CPU. And I use the process dictionary. I basically ignore all the fuss about how bad it is to use it. It is the fastest hash table we have in Erlang, appropriate to store large terms with possible substructure sharing, which would grind ets to a halt or possibly make memory usage explode. But they are not a substitute for "instantly" acessible parameters. > > While it may be true that get/1 _can_ be slow (I'd need to see the > numbers), you should never just _assume_ that get/1 is slow for > the use you intend to make of it. I didn?t. ;) >> A quite common example would be several instances of a web server >> together listening in different ports. > > It's not clear why the port should be part of a web server's context > rather than part of its state, or why these instances need to be all > together in a single Erlang node (because if they aren't, module > parameters offer us no convenience), or why if they are all in a single > node "they" shouldn't be "it", a single system listening on several > ports and doing load balancing of some sort. Ok. Thinking about it, it was a bad example. It may be common, but in this case one would expect port numbers to change and plan ahead accordingly; and as you say, they would belong naturally to its state. Regards, Paulo From jay@REDACTED Thu Apr 22 19:25:02 2010 From: jay@REDACTED (Jay Nelson) Date: Thu, 22 Apr 2010 10:25:02 -0700 Subject: [erlang-questions] Re: The If expression In-Reply-To: References: Message-ID: <55E4B828-F128-4264-B9BE-1B797CDCD8D1@duomark.com> >> > (2) >> > verbose_echo(Verbose, Format, Para) -> >> > >> > % Don't print if verbosity is off (false). >> > >>> if Verbose -> echo(Format, Para); true -> nil end. >> >> verbose_echo(true, Format, Para) -> >> echo(Format, Para); >> verbose_echo(_, _Format, _Para) -> >> nil. > > This is a TEXTBOOK example of where using Boolean is wrong. When using the 'if' statement, the issue wasn't quite so apparent. Using the function clause approach it sticks you in the eye, because you have to write a function that matches something (_ instead of true) and returns something. Your hand hovers over the keyboard wondering what to type. Another code smell is a two branched function with true and _: foo(true) -> ...; foo(_) -> ... Why _? Why not just let it crash? If there is only one value, why is it true? jay From roberto@REDACTED Thu Apr 22 19:24:28 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 22 Apr 2010 19:24:28 +0200 Subject: [erlang-questions] monitor message queue of a process In-Reply-To: <4BD06633.2010709@erlang-solutions.com> References: <4BCF4DBE.3040000@tmit.bme.hu> <4BD02A25.40902@erlang-solutions.com> <4BD06633.2010709@erlang-solutions.com> Message-ID: 2010/4/22 Ulf Wiger : > I hope to be able to expand on this very soon. i sure hope so. i feel there are so many 'deep' topics which are scarcely available outside the otp/erlang team, or people working in close contact with them. 'undocumented feature' are two words which i have never heard so often when applied to other languages/frameworks. don't get me wrong: i think the guys of the erlang/otp team are doing an amazing job, and they even are still willing to find the time to participate in this list. i can only thank them all for all of this goodness. it's just that i always have the feeling that there are things that you are easily unaware of in erlang until you bang your head on them. and sometimes, it's a *hard* bang. selective receives is one of them, for instance. i would simply love to find a way to dig more, which -of course- is a matter of effort i'm totally willing to furnish. cheers to you all, and ulf, this one i'll be gladly waiting for it. :) r. From mevans@REDACTED Thu Apr 22 20:10:05 2010 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 22 Apr 2010 14:10:05 -0400 Subject: [erlang-questions] monitor message queue of a process In-Reply-To: References: <4BCF4DBE.3040000@tmit.bme.hu> <4BD02A25.40902@erlang-solutions.com> Message-ID: Another suggestion would be the possibility to provide a max_message_queue_size option when you start a process (i.e. in spawn, gen_server:start etc).. When this size is exceeded new messages are rejected. Of course, the sender needs to be aware that this has occurred, perhaps by sending a message of type: {error,sender_busy,Name,OriginalMessage} Where Name is the pid or registered name of the sender. That way the producer can invoke any kind of recovery it sees fit. Of course, if the message is sent via a gen_server:call then the call would return with an exception. We recently ran into a problem where an application was sending too many error/log messages which caused our event manager process' (gen_event) message queue to fill up. We eventually ran out of memory causing the VM to grind to a halt and die. We fixed this by applying a form of asynchronous back pressure to the producer to prevent this. Regards Matt -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Roberto Ostinelli Sent: Thursday, April 22, 2010 7:49 AM To: Ulf Wiger Cc: Erlang Subject: Re: [erlang-questions] monitor message queue of a process 2010/4/22 Ulf Wiger : > I think the high-level way to approach this problem is to have > job-based rate regulation at the inputs of the system. > > I have been working on such a component, which I am now getting > very happy with. Our own tests have been extremely promising, and > overall, it builds on our experiences from the AXD 301 and its > derivatives, our instant messaging systems. > > BR, > Ulf W hi ulf, would you care to expand a little? of course, if this is compatible with your time/nda. i, for one, believe your experience could enlighten quite a lot in this domain. cheers, r. ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From mevans@REDACTED Thu Apr 22 20:22:27 2010 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 22 Apr 2010 14:22:27 -0400 Subject: [erlang-questions] monitor message queue of a process In-Reply-To: References: <4BCF4DBE.3040000@tmit.bme.hu> <4BD02A25.40902@erlang-solutions.com> Message-ID: I mean "Where Name is the pid or registered name of the process that received the message but was too busy". -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Evans, Matthew Sent: Thursday, April 22, 2010 2:10 PM To: Roberto Ostinelli; Ulf Wiger Cc: Erlang Subject: RE: [erlang-questions] monitor message queue of a process Another suggestion would be the possibility to provide a max_message_queue_size option when you start a process (i.e. in spawn, gen_server:start etc).. When this size is exceeded new messages are rejected. Of course, the sender needs to be aware that this has occurred, perhaps by sending a message of type: {error,sender_busy,Name,OriginalMessage} Where Name is the pid or registered name of the sender. That way the producer can invoke any kind of recovery it sees fit. Of course, if the message is sent via a gen_server:call then the call would return with an exception. We recently ran into a problem where an application was sending too many error/log messages which caused our event manager process' (gen_event) message queue to fill up. We eventually ran out of memory causing the VM to grind to a halt and die. We fixed this by applying a form of asynchronous back pressure to the producer to prevent this. Regards Matt -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Roberto Ostinelli Sent: Thursday, April 22, 2010 7:49 AM To: Ulf Wiger Cc: Erlang Subject: Re: [erlang-questions] monitor message queue of a process 2010/4/22 Ulf Wiger : > I think the high-level way to approach this problem is to have > job-based rate regulation at the inputs of the system. > > I have been working on such a component, which I am now getting > very happy with. Our own tests have been extremely promising, and > overall, it builds on our experiences from the AXD 301 and its > derivatives, our instant messaging systems. > > BR, > Ulf W hi ulf, would you care to expand a little? of course, if this is compatible with your time/nda. i, for one, believe your experience could enlighten quite a lot in this domain. cheers, r. ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From francesco@REDACTED Thu Apr 22 21:16:51 2010 From: francesco@REDACTED (Francesco Cesarini (Erlang Solutions)) Date: Thu, 22 Apr 2010 20:16:51 +0100 Subject: CUFP Call For Erlang Presentations (Baltimore, October 2nd) Message-ID: <4BD0A0A3.10803@erlang-solutions.com> Commercial Users of Functional Programming Workshop (CUFP) 2010 Functional Programming As a Means, Not an End Call for Presentations Sponsored by SIGPLAN Co-located with ICFP 2010 Baltimore (MD), USA, October 1-2, 2010 _________________________________________________________ Presentation proposals due 30 May 2009 http://www.cufp.org/ _________________________________________________________ Functional programming languages have been a hot topic of academic research for over 35 years, and are rapidly being adopted in diverse real-world settings ranging from from tech startups to financial and biomedical firms. A vigorous community of practically-minding functional programmers has come into existence, using them as tools to build reliable and fast systems. CUFP is designed to serve this community. The aim is for CUFP to be a place where people can see how others are using functional programming to solve real world problems; where practitioners meet and collaborate; where language designers and users can share ideas about the future of their favorite language; and where one can learn practical techniques and approaches for putting functional programming to work. ## Giving a CUFP Talk ## If you have experience using functional languages in a practical setting, we invite you to submit a proposal to give a talk at the workshop. We're looking for two kinds of talks: _Experience reports_ are typically 25 minutes long, and aim to inform participants about how functional programming plays out in real-world applications, focusing especially on lessons learned and insights gained. Experience reports don't need to be highly technical; reflections on the commercial, management, or software engineering aspects are, if anything, more important. You do not need to submit a paper! _Technical talks_ are expected to be 30-45 minutes long, and should focus on teaching the audience something about a technical technique or methodology, from the point of view of someone who has seen it play out in practice. These talks could cover anything from techniques for building functional concurrent applications, to managing dynamic reconfigurations, to design recipes for using types effectively in large-scale applications. While these talks will often be based on a particular language, they should be accessible to a broad range of functional programmers. If you are interested in offering a talk, or nominating someone to do so, send an e-mail to francesco(at)erlang-consulting(dot)com or yminsky(at)janestreet(dot)com by 30 May 2010 with a short description of what you'd like to talk about or what you think your nominee should give a talk about. Such descriptions should be about one page long and include a short biography of the speaker. The outcome of your proposal will be sent to you no later than the 15th of June. Note that presenters, like other attendees, need to register for the event. Presentations will be video taped and those presenters who want their video released will be expected to sign an ACM copyright release form. ## Program Plans ## CUFP 2010 will last for two days, and will consist of three components: CUFP Talks, CUFP Tutorials and CUFP BOFs. CUFP Talks will consist of a series of talks given by practitioners in the field. The talks will be split between shorter experience reports, describing ways in which functional programming has been used, either successfully or unsuccessfully, in the real world; and longer technical talks, which will detail techniques and approaches for applying functional programming in practical settings. CUFP Tutorials will be made up of in-depth, hands-on sessions for learning particular technologies and techniques in functional programming. These will range from language introductions for beginners to more advanced and focused sessions on particular language features, techniques or toolkits. CUFP BOFs will consist of so-called "birds-of-a-feather" sessions, which will create spaces for informal conversations on a variety of topics. Among other things, this will be a good venue for language designers and practitioners to exchange ideas about the future of functional programming. As is typical with BOFs, topics will be chosen beforehand based on a public process for gathering proposals, organized via our website and a mailing list. There will be no published proceedings, as the meeting is intended to be more a discussion forum than a technical interchange. --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From mevans@REDACTED Thu Apr 22 22:46:47 2010 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 22 Apr 2010 16:46:47 -0400 Subject: mnesiaex (was mnesia ram cache) In-Reply-To: References: <4BCF7C65.3070100@m5net.com> Message-ID: Hi, I was wondering if anyone had had any luck using mnesiaex I am reasonably confident that I can get the required performance from plain old mnesia+DETS, but it seems that mnesiaex could be a good standby if I can not get the desired performance (perhaps using SleepyCat or similar as the backend). Thanks Matt -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Evans, Matthew Sent: Wednesday, April 21, 2010 6:41 PM To: Bernard Duggan Cc: erlang-questions@REDACTED Subject: RE: [erlang-questions] mnesia ram cache Thanks for that, it's good to know. In this case I should be fine since there should be no cases where a single transaction will make changes to > 1 table in a single transaction. But in future your change will help for another project I am working on. -----Original Message----- From: Bernard Duggan [mailto:bernie@REDACTED] Sent: Wednesday, April 21, 2010 6:30 PM To: Evans, Matthew Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] mnesia ram cache Evans, Matthew wrote: > To implement this I intend to write my own ETS cache on top of mnesia, and use table subscription to the mnesia table (mnesia:subscribe/1) so that the cache can be updated when inserts and updates occur to the master mnesia database. The cache shall be managed by a gen_server that contains LRU or similar rules to purge data to avoid it growing too much. > Something to be aware of before you get too attached to this plan: you won't be able to maintain transactional atomicity in the cache. That is, if, for example, some node makes changes to two tables within a single transaction, the subscription system as of R13B04 does not have a nice way for a subscriber to group them together. The result could be that someone reading the cache may get one change but not the other. This may not matter for your case if any single write leaves your database in a consistent (or "consistent enough") state, but it's important to know that you'll be losing that feature of mnesia. (There is, however, a patch I wrote in R14 which addresses this exact limitation). Cheers, Bernard ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From fritchie@REDACTED Thu Apr 22 22:47:32 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 22 Apr 2010 15:47:32 -0500 Subject: [erlang-questions] Re: testing asynchronous code In-Reply-To: Message of "Thu, 22 Apr 2010 12:41:51 BST." <20100422114150.GA2762@mrnibble.lshift.net> Message-ID: <15148.1271969252@snookles.snookles.com> Matthew Sackman wrote: ms> [...] gen_server: eg, and this one turns out to be very useful: "if you ms> return 'stop' from some handle_* callback *and* a reply term, ms> gen_server will wait until *after* the terminate callback has ms> completed *before* sending the reply", [...] Oh, I didn't know about that. Then again, I think I can count in many years the number of times that I've used the 'stop' tuple, in either form, on only two or three fingers. Regarding sending messages to yourself, in general, you should worry about message order/interleaving with messages from other parties. In the specific case of a long-running init phase (longer than a supervisor will tolerate), interleaving isn't a worry. If I have a long-running init phase, the server sends itself a message from inside the MyCallBackModule:init() function. Jay's approach, sending the message after MyCallBackModule:init() has returned but before start_link() or start() returns the server's PID to the outside world, is almost the same thing. -Scott From ingela@REDACTED Thu Apr 22 23:29:08 2010 From: ingela@REDACTED (Ingela Andin) Date: Thu, 22 Apr 2010 23:29:08 +0200 Subject: [erlang-questions] Re: ssl and {active, once} - bug? In-Reply-To: References: Message-ID: Hi 2010/4/21 Roberto Ostinelli : > hi! > >>> but i've decided to go for the httpd >>> option in ssl. >> >> Sorry I do not ?understand what you mean by that? > > i meant that i'm handling the switch myself to httph, so patch is > currenlty not needed. Ahh, I see. >> >> Are you writing your own http server? Could you explain more what yoy >> are trying to do? > > i was adding SSL support to misultin http://code.google.com/p/misultin/w/list > > as said, what i'm still missing is some kind of response in the case: > > . a http request is done on a https server [currently, it timeouts] > . a https request is done on a http server [currently, it shuts down > the connection]. > > it is not clear how to provide some kind of http response on a SSL > socket, since the socket is not created unless SSL is used [and thus, > it timeouts]. > > in the same way, a non-SSL socket will simply do not understand > incoming tcp data which is part of the SSL negotiation [and thus, only > available option is to send a 403 error which is of course > mis-interpreted by the browser which initiated a SSL negitiation. > > hope this is more clear :) > > r. Yes I understand better now. You should look at the RFC2817. You have to use upgrade from tcp to tls(ssl) to be able to handle http and https request on the same port. The old erlang ssl implementation does not support that. But new ssl does support socket upgrade :) Regards Ingela Erlang/OTP team Ericsson AB From ok@REDACTED Fri Apr 23 00:14:40 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 23 Apr 2010 10:14:40 +1200 Subject: [erlang-questions] Re: The If expression In-Reply-To: <4BD05CF3.3070908@gmail.com> References: <4BD05CF3.3070908@gmail.com> Message-ID: <18C12D27-43A6-459C-9BD7-89F78C463BC9@cs.otago.ac.nz> On Apr 23, 2010, at 2:28 AM, James Aimonetti wrote: > This thread is well-timed as I have an issue I wanted some help on. > > I have two lists, one of keys and one of values. I want to zip them > together, but I have one specific key/value combo that I want to > exclude from the zipped list. For context, this is creating an > appropriate {proplist} for couchbeam. > > to_couchbeam(Fields, Values) -> > { lists:flatten( lists:zipwith(fun to_couchbeam_zipper/2, Fields, > Values) ) }. > > to_couchbeam_zipper(F, V) -> > K = to_couchbeam_key(F), > Val = to_couchbeam_value(V), > case Key == <<"_rev">> andalso Val == undefined of > true -> []; > false -> {K, Val} > end. to_couchbeam([], []) -> []; to_couchbeam([Field|Fields], [Value|Values]) -> Key = to_couchbeam_key(Field), Val = to_couchbeam_value(Value), Tail = to_couchbeam(Fields, Values) case {Key,Val} of {<<"_rev">>, undefined} -> Tail ; Pair -> [Pair|Tail] end. What have 'true' or 'false' to do with it? It's <<"_rev">> and 'undefined' you want to watch out for, so it's they that belong in the case. Why build a list including stuff you don't want, and then strip it out? What we really want here is EEP 19 (http://www.erlang.org/eeps/eep-0019.html) to_couchbeam(Fields, Values) -> [Pair || F <- Fields && V <- Values, Pair = {to_couchbeam_key(F), to_couchbeam_value(V)}, Pair =/= {<<"_rev">>, undefined}]. (THAT IS NOT LEGAL ERLANG RIGHT NOW.) If you are willing to build a list and then squish out the element you don't want, then to_couchbeam(Fields, Values) -> P = fun (K,V) -> {to_couchbeam_key(K), to_couchbeam_value(V)} end, lists:delete({<<"_rev">>,undefined}, lists:zipwith(P, Fields, Values)). seems like the best that can be done. From james.aimonetti@REDACTED Fri Apr 23 01:25:16 2010 From: james.aimonetti@REDACTED (James Aimonetti) Date: Thu, 22 Apr 2010 19:25:16 -0400 Subject: [erlang-questions] Re: The If expression In-Reply-To: <18C12D27-43A6-459C-9BD7-89F78C463BC9@cs.otago.ac.nz> References: <4BD05CF3.3070908@gmail.com> <18C12D27-43A6-459C-9BD7-89F78C463BC9@cs.otago.ac.nz> Message-ID: Thanks Richard, the feedback is much appreciated. I have a couple different places where your multi-generator syntax would indeed be useful. James On Thu, Apr 22, 2010 at 6:14 PM, Richard O'Keefe wrote: > > On Apr 23, 2010, at 2:28 AM, James Aimonetti wrote: > >> This thread is well-timed as I have an issue I wanted some help on. >> >> I have two lists, one of keys and one of values. I want to zip them >> together, but I have one specific key/value combo that I want to exclude >> from the zipped list. For context, this is creating an appropriate >> {proplist} for couchbeam. >> >> to_couchbeam(Fields, Values) -> >> { lists:flatten( lists:zipwith(fun to_couchbeam_zipper/2, Fields, Values) >> ) }. >> >> to_couchbeam_zipper(F, V) -> >> K = to_couchbeam_key(F), >> Val = to_couchbeam_value(V), >> case Key == <<"_rev">> andalso Val == undefined of >> ?true -> []; >> ?false -> {K, Val} >> end. > > to_couchbeam([], []) -> > ? ?[]; > to_couchbeam([Field|Fields], [Value|Values]) -> > ? ?Key = to_couchbeam_key(Field), > ? ?Val = to_couchbeam_value(Value), > ? ?Tail = to_couchbeam(Fields, Values) > ? ?case {Key,Val} > ? ? ?of {<<"_rev">>, undefined} -> Tail > ? ? ? ; Pair ? ? ? ? ? ? ? ? ? ?-> [Pair|Tail] > ? ?end. > > What have 'true' or 'false' to do with it? > It's <<"_rev">> and 'undefined' you want to watch out > for, so it's they that belong in the case. > Why build a list including stuff you don't want, > and then strip it out? > > What we really want here is EEP 19 > (http://www.erlang.org/eeps/eep-0019.html) > > to_couchbeam(Fields, Values) -> > ? ?[Pair || F <- Fields && V <- Values, > ? ? ? ? ? ? Pair = {to_couchbeam_key(F), to_couchbeam_value(V)}, > ? ? ? ? ? ? Pair =/= {<<"_rev">>, ? ? ? ?undefined}]. > > (THAT IS NOT LEGAL ERLANG RIGHT NOW.) > > If you are willing to build a list and then squish out the > element you don't want, then > > to_couchbeam(Fields, Values) -> > ? ?P = fun (K,V) -> {to_couchbeam_key(K), to_couchbeam_value(V)} end, > ? ?lists:delete({<<"_rev">>,undefined}, lists:zipwith(P, Fields, Values)). > > seems like the best that can be done. > > > From ok@REDACTED Fri Apr 23 05:58:21 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 23 Apr 2010 15:58:21 +1200 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: <4BD085AD.7050602@di.uminho.pt> References: <06f39d05-8aaf-4140-b606-24358d8768aa@j21g2000yqh.googlegroups.com> <5E7B0672-D467-459F-B3CC-9650CFC635F8@souja.net> <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> <3FFFC7AE-F622-4C41-8BFE-95E9B8646535@cs.otago.ac.nz> <4BCF1A67.5030302@di.uminho.pt> <4BD085AD.7050602@di.uminho .pt> Message-ID: <63870159-27AD-4ECB-BA05-BBFA7A9E6B9F@cs.otago.ac.nz> A module with parameters is basically a surpassingly strange way to write a function that returns a record of functions. Right now, we can do exactly that: new(V1, ..., Vk) -> #record{ f1 = fun (...) -> ... end, ... fm = fun (...) -> ... end }. This thing, which I regard as an aggregate of closures, can be passed around (just as an instance of a module with parameters can), and the functions can be invoked: R#record.fi(...) There are precisely two differences between this application of the closure idea and modules with parameters. (A) functions defined in the record like that cannot be recursive (B) the invocation doesn't look like a remote function call. We could make it look _more_ like a remote function call if only my "frames" proposal were adopted, a proposal which has been around for several years and addresses a lot of issues, and which has the merit of being basically the same as Joe Armstrong's "proper structs" proposal. With frames, we'd write new(V1, ..., Vk) -> <{ f1 ~ fun (...) -> ... end , ... , fm ~ fun (...) -> ... end }>. We now add one new feature: if E0 evaluates to a frame, and F is an atom and F is a key in the frame E0 then E0:F(E1, ..., En) means case E0 of <{F ~ Fun}> -> Fun(E1, ..., En) end This fixes (B). We can fix (A) without requiring cyclic data structures in the VM with a change to that definition: case E0 of V0 = <{F ~ Fun}> -> Fun(V0, E1, ..., En) end and then we'd have to write <{ f1 ~ fun (M, ...) -> ... M:fk(...) ... end , ... , fm ~ fun (M, ...) -> ... M:fj(...) ... end }> which is a bit of a pain, but doable. My argument here is - we want frames for a lot of other reasons, - once we have frames, they can do what modules with parameters do - modules with parameters can't do all the things frames can - so who needs modules with parameters? One thing that's easy with frames is to check that all the functions we want are there: client(Frame ~ <{ f1 ~ F1, ..., fk ~ Fk }>, ...) when is_function(F1, n1), ..., is_function(Fk, nk) -> ... use Frame ... From vladdu55@REDACTED Fri Apr 23 08:28:20 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 23 Apr 2010 08:28:20 +0200 Subject: [erlang-questions] Parameterized module idioms In-Reply-To: <63870159-27AD-4ECB-BA05-BBFA7A9E6B9F@cs.otago.ac.nz> References: <1DB5B964-3F23-43BB-A641-58BB5D923954@cs.otago.ac.nz> <4BCD8F32.2030606@di.uminho.pt> <3FFFC7AE-F622-4C41-8BFE-95E9B8646535@cs.otago.ac.nz> <4BCF1A67.5030302@di.uminho.pt> <4BD085AD.7050602@di.uminho.pt> <63870159-27AD-4ECB-BA05-BBFA7A9E6B9F@cs.otago.ac.nz> Message-ID: Hi, 2010/4/23 Richard O'Keefe : > A module with parameters is basically a surpassingly strange > way to write a function that returns a record of functions. > > Right now, we can do exactly that: > > ? ?new(V1, ..., Vk) -> > ? ? ? ?#record{ > ? ? ? ? ? ?f1 = fun (...) -> ... end, > ? ? ? ? ? ?... > ? ? ? ? ? ?fm = fun (...) -> ... end > ? ? ? ?}. > > This thing, which I regard as an aggregate of closures, > can be passed around (just as an instance of a module with > parameters can), and the functions can be invoked: > > ? ? ? ?R#record.fi(...) Cool! This record is then a representation of an anonymous module instance. Taken a step longer and adding some "normal" data fields, it becomes an object of the OO-kind, with the funs building the virtual method table. A classless object, JavaScript-like, and immutable. Then we can relate this to the discussion recently brought to our attention by Robert Virding in the thread named "Clojure and OO, and Erlang"... best regards, Vlad From rumata-estor@REDACTED Fri Apr 23 10:10:48 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Fri, 23 Apr 2010 12:10:48 +0400 Subject: [erlang-questions] dialyzer: user-defined types just synonyms? In-Reply-To: References: <4BD03620.5060709@nm.ru> <4BD04CA2.80306@nm.ru> <4BD05C81.1020409@nm.ru> <4BD0678D.6080400@nm.ru> <4BD06CDC.8080701@nm.ru> Message-ID: <4BD15608.2040200@nm.ru> So it will warn only if the code will always fail, but will not if it just _may_ fail. And this behaviour is what it was made for. I see, thank you. Tobias Lindahl wrote: > 2010/4/22 Dmitry Belyaev : > >> Good point about unused=untested. >> Ok, but what about the other problem? >> http://pastebin.com/q7muyTnz >> >> There must be warning on line 9. I get nothing with R13b4. >> > > The return type for get_atom is 'bad' | 'good'. > > The analysis is not strong enough to find that the call at line 7 is > returning 'bad', so when the return value is used in the call on line > 9, Dialyzer finds that the intersection between the argument type and > the success typing is 'good' which means that the call can succeed and > no warning is emitted. > > The behavior is correct, but one could always whish that the analysis > was stronger. Remember that Dialyzer is conservative in its warnings. > It will only warn when it can find that something definitely goes > wrong. > > Tobias > > Tobias > > From johanmon@REDACTED Fri Apr 23 12:53:44 2010 From: johanmon@REDACTED (Johan Montelius) Date: Fri, 23 Apr 2010 12:53:44 +0200 Subject: Group communication in Erlang Message-ID: Group communication is tricky to handle in the face of failing nodes. The modules in the Erlang standard library do not help much. The pg module does not handle the case where the leader of the group dies and in the worst case only part of the group will receive a multicast message. The module pg2 is not much better since it leaves it up to the user to implement the communication layer. I've implemented a module, called view, that provides (hopefully) an improvement. The service provides uniform atomic multicast and also some other communication primitives. It will not survive a network failure so it should not be used if you think that the monitor functionality will deliver "noconnection" messages that are not truly failures. If you're interested there is more information on: http://web.it.kth.se/~johanmon/view.html Happy to receive comments. Johan -- Dr Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From mail@REDACTED Fri Apr 23 14:16:39 2010 From: mail@REDACTED (Tim Fletcher) Date: Fri, 23 Apr 2010 05:16:39 -0700 (PDT) Subject: monitor message queue of a process In-Reply-To: References: <4BCF4DBE.3040000@tmit.bme.hu> Message-ID: <9ddc30c3-cf2b-4a4a-a2fd-0321caf8ff2e@b6g2000yqi.googlegroups.com> > this leads me into thinking that the only possible solution is > therefore to have a message queuer buffer which must be actively > interrogated by the process when it is ready to receive new messages. Might be some similar ideas in Ada we can steal. IIRC, task entries (like handle_call, but one for each message type) can each have a barrier (like a guard), which can query the number of operations queued on the entry. So you could optionally add a restriction to turn away entry calls if the queue was too big. Tim From dmurray@REDACTED Fri Apr 23 15:08:52 2010 From: dmurray@REDACTED (David N Murray) Date: Fri, 23 Apr 2010 09:08:52 -0400 (EDT) Subject: Supervisor Linked to Shell Message-ID: Hi all, Newbie question. I have: -module(t). -behavior(supervisor). -export([start/0, init/1]). start() -> supervisor:start_link(?MODULE, []). init(_) -> {ok, {{one_for_one, 3, 3600}, []}}. 128> t:start(). {ok,<0.683.0>} I'm developing in Emacs. If I make a mistake in the shell, it brings down the supervisor: 129> crash(). ** exception error: undefined shell command crash/0 in function erlang:error/1 called as erlang:error({shell_undef,crash,0}) =ERROR REPORT==== 23-Apr-2010::09:06:08 === ** Generic server <0.683.0> terminating ** Last message in was {'EXIT',<0.676.0>, {{shell_undef,crash,0}, [{erlang,error,[{shell_undef,crash,0}]}, {shell,shell_undef,2}, {erl_eval,local_func,5}, {shell,exprs,7}, {shell,eval_exprs,7}, {shell,eval_loop,3}]}} ** When Server state == {state,{<0.683.0>,t}, one_for_one,[], {dict,0,16,16,8,80,48, {[],[],[],[],[],[],[],[],[],[],[],[],[], [],[],[]}, {{[],[],[],[],[],[],[],[],[],[],[],[],[], [],[],[]}}}, 3,3600,[],t,[]} ** Reason for termination == ** {{shell_undef,crash,0}, [{erlang,error,[{shell_undef,crash,0}]}, {shell,shell_undef,2}, {erl_eval,local_func,5}, {shell,exprs,7}, {shell,eval_exprs,7}, {shell,eval_loop,3}]} 130> How can I prevent this? From the book I'm using, it looks like supervisor:start() was removed. I'm using R13B04. TIA, Dave From james.aimonetti@REDACTED Fri Apr 23 15:47:13 2010 From: james.aimonetti@REDACTED (James Aimonetti) Date: Fri, 23 Apr 2010 09:47:13 -0400 Subject: Help with Webmachine/Couchbeam combo Message-ID: <4BD1A4E1.1060505@gmail.com> I have a Webmachine app that is using Couchdb, and I'm using Couchbeam to retrieve and store documents and views. I start couchbeam up in a module that up to this point I called in my wm_app.erl (assuming I created the webmachine project with the name wm_app). The module called looks like this: % wm_app_couch.erl start() -> couchbeam:start(), Conn = couchbeam_server:start_connection_link(#couchdb_params{}), couchbeam_db:open_or_create(Conn, {?DB, atom_to_list(?DB)}). As I've tested storing and retrieving docs and views in the shell, on occasion the Db process or Conn process will die and I have to manually restart the processes. How do I add couchbeam, the Conn, and the Db pids to the wm_app_sup.erl supervisor to make sure they're automatically restarted should they die? I guess I'm not sure how to structure the ChildSpec for each, especially when the Db pid is generated using the Conn pid as an argument. Or is there a better, more appropriate solution. Sorry if this is n00b territory. -- James Aimonetti mobile: 314.809.6307 work: 540.459.2220 email: james.aimonetti@REDACTED website: http://jamesaimonetti.com From dmercer@REDACTED Fri Apr 23 16:09:05 2010 From: dmercer@REDACTED (David Mercer) Date: Fri, 23 Apr 2010 09:09:05 -0500 Subject: [erlang-questions] Calling external modules from a fun In-Reply-To: References: Message-ID: <4301ECAF454148CAA9FCCCD09892276C@SSI.CORP> On Wednesday, April 21, 2010, Alessandro Sivieri wrote: > I am working on an application which, for some reasons, creates a fun > (probably something more like a continuation) and then sends it to a > different node for execution; the sender node is launched with my > application code path (the ebin directory), and so the receiver one. > Now, if my code path contains some module which I have wrote, I have found > that in a fun I cannot invoke it (as module:function), because if I do so, > the remote machine (which actually receives and executes the fun itself) > answers with an undefined_lambda error; so my question is: is there a way > to > call a function from a fun remotely received, if it is not from the > standard > Erlang distribution? Since no-one smarter than me responded, here's my take. In order for a fun to work on a remote node, the module that created the fun must also exist on the remote node. I think of funs less as self-contained lambdas, but more of pointers to the code in the module that created it, along with any bound values required for the closure. Thus, if the module is not present on the remote node (or if it is a different version of the same), I believe the fun cannot exist on the remote node. Not sure exactly what happens on the remote node when it is received -- whether it crashes the node, crashes the process that receives it, or just crashes any process that tries to execute it. Cheers, DBM From erlangy@REDACTED Fri Apr 23 16:30:51 2010 From: erlangy@REDACTED (Michael McDaniel) Date: Fri, 23 Apr 2010 07:30:51 -0700 Subject: [erlang-questions] Help with Webmachine/Couchbeam combo In-Reply-To: <4BD1A4E1.1060505@gmail.com> References: <4BD1A4E1.1060505@gmail.com> Message-ID: <20100423143050.GD15986@delora.autosys.us> On Fri, Apr 23, 2010 at 09:47:13AM -0400, James Aimonetti wrote: > I have a Webmachine app that is using Couchdb, and I'm using Couchbeam to > retrieve and store documents and views. > > I start couchbeam up in a module that up to this point I called in my > wm_app.erl (assuming I created the webmachine project with the name > wm_app). The module called looks like this: > > % wm_app_couch.erl > start() -> > couchbeam:start(), > Conn = couchbeam_server:start_connection_link(#couchdb_params{}), > couchbeam_db:open_or_create(Conn, {?DB, atom_to_list(?DB)}). > > As I've tested storing and retrieving docs and views in the shell, on > occasion the Db process or Conn process will die and I have to manually > restart the processes. > > How do I add couchbeam, the Conn, and the Db pids to the wm_app_sup.erl > supervisor to make sure they're automatically restarted should they die? > I guess I'm not sure how to structure the ChildSpec for each, especially > when the Db pid is generated using the Conn pid as an argument. Or is > there a better, more appropriate solution. > ________________________________________________________________ not familiar with couchbeam though if it is has behaviour gen_* then the following will likely work. wm_sup.erl ... init([]) -> ... CB = {couchbeam, {couchbeam, start, []}, permanent, 5000, worker, dynamic} , Processes = [Web, ... , CB] , {ok, { {one_for_one, 3, 10}, Processes} } . ~Michael From dmurray@REDACTED Fri Apr 23 16:48:27 2010 From: dmurray@REDACTED (David N Murray) Date: Fri, 23 Apr 2010 10:48:27 -0400 (EDT) Subject: [erlang-questions] Supervisor Linked to Shell In-Reply-To: References: Message-ID: figured it out: -module(t). -behavior(supervisor). -export([start/0, init/1]). start() -> - supervisor:start_link(?MODULE, []). + supervisor:start_link(?MODULE, self()). - init(_) -> + init(ShellPid) -> + unlink(ShellPid), {ok, {{one_for_one, 3, 3600}, []}}. Note to self: Search the archives first. Dave From mevans@REDACTED Fri Apr 23 18:49:19 2010 From: mevans@REDACTED (Evans, Matthew) Date: Fri, 23 Apr 2010 12:49:19 -0400 Subject: Process Message Queues Message-ID: Hi, Question for you all. The message queue in Erlang follows the following rule. Messages are put in to the end of the queue, when the receive is processed the message is tested against the first pattern. If this matches this pattern it is removed from the queue, if it doesn't match then the message is matched against the second pattern and so on. If no patterns are matched then the message is put back on the queue and the next message is taken from the queue and so on... So I spawn the following process, and send the 4 messages below: ============================================== (scm@REDACTED)1> Pid = spawn(fun() -> receive message_a -> error_logger:info_msg("Got message_a~n"); message_b -> error_logger:info_msg("got message_b~n"); message_c -> receive message_d -> error_logger:info_msg("Got message_d~n") end end end). <0.19842.0> (scm@REDACTED)2> Pid ! hello. hello (scm@REDACTED)3> Pid ! goodbye. goodbye (scm@REDACTED)4> Pid ! message_d. message_d (scm@REDACTED)5> Pid ! message_e. message_e (scm@REDACTED)6> erlang:process_info(Pid,message_queue_len). {message_queue_len,0} =============================================== At this point wouldn't the message queue length be 4? I know that message_d is in the queue since this works: =============================================== (scm@REDACTED)7> Pid ! message_c. =INFO REPORT==== 23-Apr-2010::16:39:47 === Got message_d ** at node scm@REDACTED ** message_c (scm@REDACTED)56> ============================================== I guess it's not too critical, but it makes it hard(er) to figure out if a message queue is filling up. Thanks Matt From fritchie@REDACTED Fri Apr 23 20:00:46 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Fri, 23 Apr 2010 13:00:46 -0500 Subject: [erlang-questions] Calling external modules from a fun In-Reply-To: Message of "Fri, 23 Apr 2010 09:09:05 CDT." <4301ECAF454148CAA9FCCCD09892276C@SSI.CORP> Message-ID: <76853.1272045646@snookles.snookles.com> David Mercer wrote: dm> Since no-one smarter than me responded, here's my take. I don't claim to be any smarter on this subject, but there's some empirical study possible while we wait for wisdom. dm> In order for a fun to work on a remote node, the module that created dm> the fun must also exist on the remote node. True and false. If you use this syntax, yes, there's a module dependency: spawn(Node, fun some_module:foo_func/0). There's a noticable difference in the sizes of: size(term_to_binary(fun some_module:foo_func/0)). size(term_to_binary(fun() -> some_module:foo_func() end)). If the fun uses the fun() -> ... end syntax, then any function calls made externally must be resolvable by the VM, the code server, etc. So "fun() -> erlang:display(42) end" will work because the BIF definitely exists over on the remote node ... because it's a BIF. The same resolvability property can be seen when using intra-module calls. Using size(term_to_binary(Fun)), we see create experiments that show that all of the transitively dependent functions are not included in the fun because the size won't change. -Scott From per.melin@REDACTED Fri Apr 23 21:52:16 2010 From: per.melin@REDACTED (Per Melin) Date: Fri, 23 Apr 2010 21:52:16 +0200 Subject: [erlang-questions] Process Message Queues In-Reply-To: References: Message-ID: On Fri, Apr 23, 2010 at 6:49 PM, Evans, Matthew wrote: > (scm@REDACTED)1> Pid = spawn(fun() -> receive message_a -> error_logger:info_msg("Got message_a~n"); message_b -> error_logger:info_msg("got message_b~n"); message_c -> receive message_d -> error_logger:info_msg("Got message_d~n") end end end). > <0.19842.0> > (scm@REDACTED)2> Pid ! hello. > hello > (scm@REDACTED)3> Pid ! goodbye. > goodbye > (scm@REDACTED)4> Pid ! message_d. > message_d > (scm@REDACTED)5> Pid ! message_e. > message_e > (scm@REDACTED)6> erlang:process_info(Pid,message_queue_len). > {message_queue_len,0} Put the first line with the spawn in a function in a module. Compile it and start your experiment by calling the function. The rest should then behave as expected. From nem@REDACTED Fri Apr 23 22:40:56 2010 From: nem@REDACTED (Geoff Cant) Date: Fri, 23 Apr 2010 13:40:56 -0700 Subject: [erlang-questions] Process Message Queues In-Reply-To: References: Message-ID: <9DA9FF5F-D598-4F01-B8E4-CF2D5947DE6E@erlang.geek.nz> On 2010-04-23, at 12:52 , Per Melin wrote: > On Fri, Apr 23, 2010 at 6:49 PM, Evans, Matthew wrote: >> (scm@REDACTED)1> Pid = spawn(fun() -> receive message_a -> error_logger:info_msg("Got message_a~n"); message_b -> error_logger:info_msg("got message_b~n"); message_c -> receive message_d -> error_logger:info_msg("Got message_d~n") end end end). >> <0.19842.0> >> (scm@REDACTED)2> Pid ! hello. >> hello >> (scm@REDACTED)3> Pid ! goodbye. >> goodbye >> (scm@REDACTED)4> Pid ! message_d. >> message_d >> (scm@REDACTED)5> Pid ! message_e. >> message_e >> (scm@REDACTED)6> erlang:process_info(Pid,message_queue_len). >> {message_queue_len,0} > > Put the first line with the spawn in a function in a module. Compile > it and start your experiment by calling the function. The rest should > then behave as expected. Am I right in saying that this interesting result is due to the fact that shell code (evaluated by erl_eval) is different to module code (evaluated by beam)? If I remember correctly, to implement receive erl_eval drains the message queue into a list and then simulates what beam would do when evaluating a receive statement. Hence the oddness with message_queue_len, and why it behaves as expected when written as module code. The differences between shell code behaviour and module code behaviour often seem to be a source of confusion. Even though I know they're different, I was bitten myself recently by unicode handling differences between the two and spent a few frustrating hours figuring it out :) --Geoff From rvirding@REDACTED Fri Apr 23 22:56:15 2010 From: rvirding@REDACTED (Robert Virding) Date: Fri, 23 Apr 2010 22:56:15 +0200 Subject: [erlang-questions] Process Message Queues In-Reply-To: <9DA9FF5F-D598-4F01-B8E4-CF2D5947DE6E@erlang.geek.nz> References: <9DA9FF5F-D598-4F01-B8E4-CF2D5947DE6E@erlang.geek.nz> Message-ID: On 23 April 2010 22:40, Geoff Cant wrote: > On 2010-04-23, at 12:52 , Per Melin wrote: > >> On Fri, Apr 23, 2010 at 6:49 PM, Evans, Matthew wrote: >>> (scm@REDACTED)1> Pid = spawn(fun() -> receive message_a -> error_logger:info_msg("Got message_a~n"); message_b -> error_logger:info_msg("got message_b~n"); message_c -> receive message_d -> error_logger:info_msg("Got message_d~n") end end end). >>> <0.19842.0> >>> (scm@REDACTED)2> Pid ! hello. >>> hello >>> (scm@REDACTED)3> Pid ! goodbye. >>> goodbye >>> (scm@REDACTED)4> Pid ! message_d. >>> message_d >>> (scm@REDACTED)5> Pid ! message_e. >>> message_e >>> (scm@REDACTED)6> erlang:process_info(Pid,message_queue_len). >>> {message_queue_len,0} >> >> Put the first line with the spawn in a function in a module. Compile >> it and start your experiment by calling the function. The rest should >> then behave as expected. > > Am I right in saying that this interesting result is due to the fact that shell code (evaluated by erl_eval) is different to module code (evaluated by beam)? If I remember correctly, to implement receive erl_eval drains the message queue into a list and then simulates what beam would do when evaluating a receive statement. Hence the oddness with message_queue_len, and why it behaves as expected when written as module code. I think I remember reading somewhere that process_info doesn't always return the true current status of the process, apparently some of the data isn't updated continually. Or something like that. However, when I checked the docs I couldn't find anything about it, so it has either gone away or was never there. > The differences between shell code behaviour and module code behaviour often seem to be a source of confusion. Even though I know they're different, I was bitten myself recently by unicode handling differences between the two and spent a few frustrating hours figuring it out :) Didn't know about unicode differences, only that you can get slightly different handling of receive because of the way the message buffer is handled. I would think it very strange if you get different behaviour from somethin that was called. Robert From comptekki@REDACTED Sat Apr 24 00:24:44 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 23 Apr 2010 16:24:44 -0600 Subject: ibrowse errors Message-ID: I'm running the unit tests with ibrowse like this: c(ibrowse_test,[debug_info]). c(ibrowse,[debug_info]). c(ibrowse_lib,[debug_info]). im(). ii(ibrowse_test). ii(ibrowse). ii(ibrowse_lib). iaa([init]). ibrowse_test:unit_tests(). I can see that during the test that http://intranet/messenger should give an error like the one below for google, but I would think that www.google.com would pass. get, http://www.google.com : Err -> {'EXIT',{badarg,[{ets,lookup,[ibrowse_lb,{"www.google.com",80}]}, {ibrowse,send_req, ["http://www.google.com",[],get,[], [{connect_timeout,5000}], 30000]}, {ibrowse,send_req,5}, {ibrowse_test,execute_req,3}]}} Why is it failing on www.google.com? thx, -wes From comptekki@REDACTED Sat Apr 24 00:35:39 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 23 Apr 2010 16:35:39 -0600 Subject: ibrowse try #2 Message-ID: I also tried this and got an error: 2> ibrowse:send_req("http://www.google.com/", [], get). ** exception error: bad argument in function ets:lookup/2 called as ets:lookup(ibrowse_lb,{"www.google.com",80}) in call from ibrowse:send_req/6 Any idea why? thx, -wes From comptekki@REDACTED Sat Apr 24 00:47:01 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 23 Apr 2010 16:47:01 -0600 Subject: [erlang-questions] ibrowse try #2 In-Reply-To: References: Message-ID: On Fri, Apr 23, 2010 at 4:41 PM, Andrei Soroker wrote: > Make sure to start it first: ibrowse:start(). > > Andrei Andrei, thx - I'm getting closer: 1> ibrowse:start(). {ok,<0.34.0>} 2> ibrowse:send_req("http://www.google.com/", [], get). =ERROR REPORT==== 23-Apr-2010::16:43:16 === ** Generic server <0.36.0> terminating ** Last message in was {spawn_connection, {url,"http://www.google.com/","www.google.com",80, undefined,undefined,"/",http}, 10,10, {[],false}} ** When Server state == {state,<0.34.0>,16403,"www.google.com",80,10,10,0} ** Reason for termination == ** {'module could not be loaded', [{ibrowse_http_client,start_link, [{16403, {url,"http://www.google.com/","www.google.com",80,undefined, undefined,"/",http}, {[],false}}]}, {ibrowse_lb,handle_call,3}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} ** exception exit: {{undef, [{ibrowse_http_client,start_link, [{16403, {url,"http://www.google.com/","www.google.com",80,undefined, undefined,"/",http}, {[],false}}]}, {ibrowse_lb,handle_call,3}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]}, {gen_server,call, [<0.36.0>, {spawn_connection, {url,"http://www.google.com/","www.google.com",80,undefined, undefined,"/",http}, 10,10, {[],false}}]}} in function gen_server:call/2 in call from ibrowse:send_req/6 3> Do I need to start something else? Is there a place for this list for send code/errors so the whole chunk is not in the email? thx, -wes From comptekki@REDACTED Sat Apr 24 03:40:52 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 23 Apr 2010 19:40:52 -0600 Subject: parse html Message-ID: ibrowse works good for getting pages, is there an erlang library that parses html? Someone recommended ibrowse for parsing html, but it seems to only get html?? thx, -wes From bob@REDACTED Sat Apr 24 03:47:05 2010 From: bob@REDACTED (Bob Ippolito) Date: Fri, 23 Apr 2010 18:47:05 -0700 Subject: [erlang-questions] parse html In-Reply-To: References: Message-ID: On Fri, Apr 23, 2010 at 6:40 PM, Wes James wrote: > ibrowse works good for getting pages, is there an erlang library that > parses html? ?Someone recommended ibrowse for parsing html, but it > seems to only get html?? There's a mochiweb_html module in mochiweb for that purpose. You might be able to find something better, but it works for us. http://code.google.com/p/mochiweb/ -bob From karthik@REDACTED Sat Apr 24 21:55:10 2010 From: karthik@REDACTED (Karthik Kailash) Date: Sat, 24 Apr 2010 12:55:10 -0700 Subject: Problems Compiling Erlang R12B-5 Message-ID: Hi, I am trying to compile Erlang on a debian system. This is the output of the Makefile: http://kpaste.net/fc4 -- the errors are at the bottom of the pasted output. My arguments to the configure script were the following: ./configure --enable-threads --enable-smp-support --enable-kernel-poll --prefix=/opt/erlang If posting the output of configure would be helpful I can do that as well. Any help would be appreciated! Thanks, Karthik -- Karthik Kailash SocialVision Inc. "Online Television Becomes a Social Experience" www.socialvisioninc.com Phone: (408) 768-7704 From mpalmer@REDACTED Sat Apr 24 22:21:22 2010 From: mpalmer@REDACTED (Matthew Palmer) Date: Sun, 25 Apr 2010 06:21:22 +1000 Subject: Problems Compiling Erlang R12B-5 In-Reply-To: References: Message-ID: <20100424202122.GW20959@hezmatt.org> On Sat, Apr 24, 2010 at 12:55:10PM -0700, Karthik Kailash wrote: > I am trying to compile Erlang on a debian system. This is the output of the > Makefile: http://kpaste.net/fc4 -- the errors are at the bottom of the > pasted output. Try giving your machine some (more) swap; something killed the build, and unless someone was being prolific with the kill command, the OOM killer would be the best bet. dmesg will also provide more supporting evidence of that theory. - Matt -- I'm not sure which upsets me more: that people are so unwilling to accept responsibility for their own actions, or that they are so eager to regulate everyone else's. From karthik@REDACTED Sat Apr 24 22:48:12 2010 From: karthik@REDACTED (Karthik Kailash) Date: Sat, 24 Apr 2010 13:48:12 -0700 Subject: [erlang-questions] Re: Problems Compiling Erlang R12B-5 In-Reply-To: <20100424202122.GW20959@hezmatt.org> References: <20100424202122.GW20959@hezmatt.org> Message-ID: Hi Matt, I followed output of dmesg while I ran the build but there were no messages at all. I also ran top during the make process and saw that the memory usage never went above 200M (out of 300M available). Karthik On Sat, Apr 24, 2010 at 1:21 PM, Matthew Palmer wrote: > On Sat, Apr 24, 2010 at 12:55:10PM -0700, Karthik Kailash wrote: > > I am trying to compile Erlang on a debian system. This is the output of > the > > Makefile: http://kpaste.net/fc4 -- the errors are at the bottom of the > > pasted output. > > Try giving your machine some (more) swap; something killed the build, and > unless someone was being prolific with the kill command, the OOM killer > would be the best bet. dmesg will also provide more supporting evidence of > that theory. > > - Matt > > -- > I'm not sure which upsets me more: that people are so unwilling to accept > responsibility for their own actions, or that they are so eager to regulate > everyone else's. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Karthik Kailash SocialVision Inc. "Online Television Becomes a Social Experience" www.socialvisioninc.com Phone: (408) 768-7704 From karthik@REDACTED Sun Apr 25 00:02:57 2010 From: karthik@REDACTED (Karthik Kailash) Date: Sat, 24 Apr 2010 15:02:57 -0700 Subject: [erlang-questions] Problems Compiling Erlang R12B-5 In-Reply-To: References: Message-ID: Hi Roger, Here is the configure output: http://kpaste.net/4649c0b Thanks, Karthik On Sat, Apr 24, 2010 at 2:40 PM, Roger Critchlow wrote: > The last page of the output of configure should identify any configuration > problems. > > Your make is failing on the very first attempt to compile an erlang source > file, so the problem isn't minor. > > I compiled 12B-5 successfully on Lucid Lynx this week. > > -- rec -- > > -- Karthik Kailash SocialVision Inc. "Online Television Becomes a Social Experience" www.socialvisioninc.com Phone: (408) 768-7704 From chandrashekhar.mullaparthi@REDACTED Sun Apr 25 06:15:03 2010 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Sun, 25 Apr 2010 05:15:03 +0100 Subject: [erlang-questions] ibrowse try #2 In-Reply-To: References: Message-ID: On 23 April 2010 23:47, Wes James wrote: > On Fri, Apr 23, 2010 at 4:41 PM, Andrei Soroker wrote: > > Make sure to start it first: ibrowse:start(). > > > > Andrei > > > Andrei, thx - I'm getting closer: > > 1> ibrowse:start(). > {ok,<0.34.0>} > 2> ibrowse:send_req("http://www.google.com/", [], get). > > =ERROR REPORT==== 23-Apr-2010::16:43:16 === > ** Generic server <0.36.0> terminating > ** Last message in was {spawn_connection, > {url,"http://www.google.com/","www.google.com > ",80, > undefined,undefined,"/",http}, > 10,10, > {[],false}} > ** When Server state == {state,<0.34.0>,16403,"www.google.com",80,10,10,0} > ** Reason for termination == > ** {'module could not be loaded', > [{ibrowse_http_client,start_link, > Looks like the erlang node can't find the ibrowse_http_client.beam file. Have you compiled all the modules? What does m(ibrowse_http_client) give? Do I need to start something else? Is there a place for this list for > send code/errors so the whole chunk is not in the email? > Not really, just post it here. cheers Chandru From chandrashekhar.mullaparthi@REDACTED Sun Apr 25 06:16:22 2010 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Sun, 25 Apr 2010 05:16:22 +0100 Subject: [erlang-questions] ibrowse errors In-Reply-To: References: Message-ID: On 23 April 2010 23:24, Wes James wrote: > I'm running the unit tests with ibrowse like this: > > c(ibrowse_test,[debug_info]). > c(ibrowse,[debug_info]). > c(ibrowse_lib,[debug_info]). > im(). > ii(ibrowse_test). > ii(ibrowse). > ii(ibrowse_lib). > iaa([init]). > ibrowse_test:unit_tests(). > > > I can see that during the test that http://intranet/messenger should > give an error like the one below for google, but I would think that > www.google.com would pass. > > get, http://www.google.com : Err -> > {'EXIT',{badarg,[{ets,lookup,[ibrowse_lb,{"www.google.com",80}]}, > {ibrowse,send_req, > ["http://www.google.com",[],get,[], > [{connect_timeout,5000}], > 30000]}, > {ibrowse,send_req,5}, > {ibrowse_test,execute_req,3}]}} > > > Why is it failing on www.google.com? > Make sure you run ibrowse:start() before running the tests. cheers Chandru From zabrane3@REDACTED Sun Apr 25 12:55:31 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Sun, 25 Apr 2010 12:55:31 +0200 Subject: Stabe Erlang to SQLite3 binding ? Message-ID: Hi List, Is there any stable (I mean, ready for production) binding to SQLite3? Regards Zabrane From max.lapshin@REDACTED Sun Apr 25 12:58:19 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 25 Apr 2010 14:58:19 +0400 Subject: [erlang-questions] Stabe Erlang to SQLite3 binding ? In-Reply-To: References: Message-ID: My branch: http://github.com/maxlapshin/erlang-sqlite3 (using linked-in driver with async threads) was working, however I would not call it "ready for production". From dmurray@REDACTED Sun Apr 25 15:22:50 2010 From: dmurray@REDACTED (David N Murray) Date: Sun, 25 Apr 2010 09:22:50 -0400 (EDT) Subject: [erlang-questions] Problems Compiling Erlang R12B-5 In-Reply-To: References: Message-ID: On Apr 24, Karthik Kailash scribed: > Hi Roger, > > Here is the configure output: http://kpaste.net/4649c0b > > Thanks, > Karthik I recall running into something similar, but it wasn't while building erlang. It was gcc running out of memory and it manifested itself the same way (i.e. "killed"). IIRC, I may have fixed it by changing one of these (which all look fine, now): $ ulimit -a # bash internal command core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited pending signals (-i) 7679 max locked memory (kbytes, -l) 32 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 7679 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited proabably either -d, -m, or -v. If you're running out of swap space, you can add more swap (check mkswap and swapon). You can monitor both memory and swap usage with 'top' while it's building and see if that's your problem. hth, Dave From hans.bolinder@REDACTED Mon Apr 26 08:49:53 2010 From: hans.bolinder@REDACTED (Hans Bolinder) Date: Mon, 26 Apr 2010 08:49:53 +0200 Subject: [erlang-questions] Supervisor Linked to Shell In-Reply-To: References: Message-ID: <19413.14225.787345.703015@ornendil.du.uab.ericsson.se> [David N Murray :] > I'm developing in Emacs. If I make a mistake in the shell, it brings > down the supervisor: > > 129> crash(). > ** exception error: undefined shell command crash/0 > in function erlang:error/1 > called as erlang:error({shell_undef,crash,0}) ... > How can I prevent this? You could try the function shell:catch_exception/1 http://www.erlang.org/doc/man/shell.html#catch_exception-1 or the STDLIB configuration parameter 'shell_catch_exception' http://www.erlang.org/doc/man/STDLIB_app.html The latter could for instance be set by export ERL_FLAGS="-stdlib shell_catch_exception true" Best regards, Hans Bolinder, Erlang/OTP team, Ericsson From johanmunk@REDACTED Mon Apr 26 09:34:08 2010 From: johanmunk@REDACTED (johan munk) Date: Mon, 26 Apr 2010 09:34:08 +0200 Subject: timer server timeout In-Reply-To: References: Message-ID: No-one has a view on this? On Wed, Apr 21, 2010 at 10:05 PM, johan munk wrote: > > Following the control flow in the gen_server, sys, and timer modules I see > this: > Calling sys:get_status(timer_server) will "reset" the "after time" in the > loop function of gen_server. As the timer server uses that time to get a > timeout if no other messages are received, one can potentially cause timers > to never trigger. > Is this a known issue/feature? > > /JM > From ulf.wiger@REDACTED Mon Apr 26 09:57:26 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 26 Apr 2010 09:57:26 +0200 Subject: [erlang-questions] Re: timer server timeout In-Reply-To: References: Message-ID: <4BD54766.8080106@erlang-solutions.com> johan munk wrote: > No-one has a view on this? Yes, indeed, but not always the time to answer questions. :) I never use the internal timeout in gen_server for anything much, for this very reason. It can be useful as an inactivity timeout, but if you really want to get a reliable timeout since a specific point in time, you have to set a timer. Fortunately, the timer BIFs are very cheap (don't use the functions in the timer module; use erlang:send_after/3 or erlang:start_timer/3.) BR, Ulf W > On Wed, Apr 21, 2010 at 10:05 PM, johan munk wrote: > >> Following the control flow in the gen_server, sys, and timer modules I see >> this: >> Calling sys:get_status(timer_server) will "reset" the "after time" in the >> loop function of gen_server. As the timer server uses that time to get a >> timeout if no other messages are received, one can potentially cause timers >> to never trigger. >> Is this a known issue/feature? >> >> /JM >> > -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From johanmunk@REDACTED Mon Apr 26 10:17:54 2010 From: johanmunk@REDACTED (johan munk) Date: Mon, 26 Apr 2010 10:17:54 +0200 Subject: [erlang-questions] Re: timer server timeout In-Reply-To: <4BD54766.8080106@erlang-solutions.com> References: <4BD54766.8080106@erlang-solutions.com> Message-ID: On Mon, Apr 26, 2010 at 9:57 AM, Ulf Wiger wrote: > > > I never use the internal timeout in gen_server for anything > much, for this very reason. It can be useful as an inactivity > timeout, but if you really want to get a reliable timeout since > a specific point in time, you have to set a timer. Fortunately, > the timer BIFs are very cheap (don't use the functions in the > timer module; use erlang:send_after/3 or erlang:start_timer/3.) > ah OK - thanks. Only my one-shot timers needs to be more reliable than what the timer module provide. I suggest that the doc of the timer module is extended with info about this gotcha. It says: "The timeouts are not exact, but should be at least as long as requested." which holds but one gets the impression that timeouts are guaranteed (in finite time). > BR, > Ulf W > > > > On Wed, Apr 21, 2010 at 10:05 PM, johan munk wrote: >> >> Following the control flow in the gen_server, sys, and timer modules I >>> see >>> this: >>> Calling sys:get_status(timer_server) will "reset" the "after time" in the >>> loop function of gen_server. As the timer server uses that time to get a >>> timeout if no other messages are received, one can potentially cause >>> timers >>> to never trigger. >>> Is this a known issue/feature? >>> >>> /JM >>> >>> >> > > -- > Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > http://www.erlang-solutions.com > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become > ERLANG SOLUTIONS LTD. > > www.erlang-solutions.com > > From kenneth.lundin@REDACTED Mon Apr 26 10:19:45 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Mon, 26 Apr 2010 10:19:45 +0200 Subject: [erlang-questions] timer server timeout In-Reply-To: References: Message-ID: This is not a known issue. After a quick glance I agree with you that a sys:get_status call to the timer_server or any other "gen_server" using the gen_server timeout feature can disturb the timing either by delaying the timeout message or by not letting any timeout message to be sent. I will put this on the list of things to look at. It is whoever very unlikely that this happens in normal operation since there is no reason to call sys:get_status other than for debugging purposes. I have never heard of any real problems caused by this. /Kenneth Erlang/OTP Ericsson On Wed, Apr 21, 2010 at 10:05 PM, johan munk wrote: > Following the control flow in the gen_server, sys, and timer modules I see > this: > Calling sys:get_status(timer_server) will "reset" the "after time" in the > loop function of gen_server. As the timer server uses that time to get a > timeout if no other messages are received, one can potentially cause timers > to never trigger. > Is this a known issue/feature? > > /JM > From johanmunk@REDACTED Mon Apr 26 10:31:32 2010 From: johanmunk@REDACTED (johan munk) Date: Mon, 26 Apr 2010 10:31:32 +0200 Subject: [erlang-questions] timer server timeout In-Reply-To: References: Message-ID: Thanks, But attaching to a production node and periodically asking a gen_server for it's state for raw monitoring of it is now potentially causing that server to malfunction. I would say that at least calls for an NB in doc somewhere. On Mon, Apr 26, 2010 at 10:19 AM, Kenneth Lundin wrote: > This is not a known issue. > After a quick glance I agree with you that a sys:get_status call to > the timer_server or any other > "gen_server" using the gen_server timeout feature can disturb the > timing either by delaying > the timeout message or by not letting any timeout message to be sent. > > I will put this on the list of things to look at. > > It is whoever very unlikely that this happens in normal operation > since there is no reason to call > sys:get_status other than for debugging purposes. > > I have never heard of any real problems caused by this. > > /Kenneth Erlang/OTP Ericsson > > On Wed, Apr 21, 2010 at 10:05 PM, johan munk wrote: > > Following the control flow in the gen_server, sys, and timer modules I > see > > this: > > Calling sys:get_status(timer_server) will "reset" the "after time" in the > > loop function of gen_server. As the timer server uses that time to get a > > timeout if no other messages are received, one can potentially cause > timers > > to never trigger. > > Is this a known issue/feature? > > > > /JM > > > From koops.j@REDACTED Mon Apr 26 11:13:24 2010 From: koops.j@REDACTED (Jeroen Koops) Date: Mon, 26 Apr 2010 11:13:24 +0200 Subject: [JOB] Texprezzo is looking for an Erlang/OTP developer Message-ID: Texprezzo, a privately owned startup based in Amsterdam, the Netherlands, is looking for a full-time Erlang/OTP developer. What we ask: - Several years of professional programming experience, not necessarily in Erlang. - Strong interest in, and some experience with, developing in and learning about Erlang/OTP. - Ability to work unsupervised - good problem-solving skills. - Good communication skills in English. - Legally able to work in the Netherlands - at this point Texprezzo is unable to aid with sponsorship and/or visa applications. Knowledge of any of the following is a plus: - HTML, Javascript, PHP, Linux, Solaris - Telecom-related technologies, especially relating to SMS What we offer: - Opportunity to be among the first employees in a startup - Competitive renumeration package - Earn a living doing what you love doing: programming Erlang! Note: This is a permanent position, at this moment we are not looking for contractors/freelancers. To apply, please send a CV to jeroen.koops@REDACTED You can also contact me for more information at the same address, or by telephone on +31-20-3120528. *Company information:* *The Texprezzo Group was founded in 2008 by a team of experienced telecom professionals with a wide variety of backgrounds with experience in various areas related to mobile messaging. Their goal was to create new value with the existing messaging infrastructures as currently deployed by mobile operators today, thus stimulating a new wave of revenues and end user experience.* * * *The main investors behind Texprezzo Group BV have a long track record in establishing and creating new technology businesses. An example of which is the recently sold mBalance Group, the leading SMS Routing infrastructure company.* From ulf.wiger@REDACTED Mon Apr 26 11:46:03 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 26 Apr 2010 11:46:03 +0200 Subject: [erlang-questions] timer server timeout In-Reply-To: References: Message-ID: <4BD560DB.90404@erlang-solutions.com> Kenneth, I think we could put this down as one of the things that people at Ericsson have known for years but never formally raised a complaint about. Mea Culpa, I've avoided the use of the internal gen_server timeouts for at least a decade for this reason, and have certainly advised others to do it as well. I guess I assumed that it was common knowledge by now and didn't reflect on whether or not it was in the docs. I am almost sure that it has been mentioned a number of times on the mailing list, but that's neither here nor there. It should be documented. It's not just debugging that can cause this. Code change and stray messages can do it too. For the same reason that a process should be prepared to throw away unrecognized messages in some top-level state, it should not allow such messages to interfere with timeout handling. BR, Ulf W Kenneth Lundin wrote: > This is not a known issue. > After a quick glance I agree with you that a sys:get_status call to > the timer_server or any other > "gen_server" using the gen_server timeout feature can disturb the > timing either by delaying > the timeout message or by not letting any timeout message to be sent. > > I will put this on the list of things to look at. > > It is whoever very unlikely that this happens in normal operation > since there is no reason to call > sys:get_status other than for debugging purposes. > > I have never heard of any real problems caused by this. > > /Kenneth Erlang/OTP Ericsson > > On Wed, Apr 21, 2010 at 10:05 PM, johan munk wrote: >> Following the control flow in the gen_server, sys, and timer modules I see >> this: >> Calling sys:get_status(timer_server) will "reset" the "after time" in the >> loop function of gen_server. As the timer server uses that time to get a >> timeout if no other messages are received, one can potentially cause timers >> to never trigger. >> Is this a known issue/feature? >> >> /JM >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From kenneth.lundin@REDACTED Mon Apr 26 12:36:20 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Mon, 26 Apr 2010 12:36:20 +0200 Subject: [erlang-questions] timer server timeout In-Reply-To: <4BD560DB.90404@erlang-solutions.com> References: <4BD560DB.90404@erlang-solutions.com> Message-ID: At least I did not know about this potential probem with gen_sever timeouts and I think it should have been reported as a bug. I think this bad beaviour should be corrected and only if that is not possible there is reason to document this little nasty side-effect. /Kenneth On Mon, Apr 26, 2010 at 11:46 AM, Ulf Wiger wrote: > > Kenneth, > > I think we could put this down as one of the things that > people at Ericsson have known for years but never formally > raised a complaint about. Mea Culpa, I've avoided the use > of the internal gen_server timeouts for at least a decade > for this reason, and have certainly advised others to do it > as well. I guess I assumed that it was common knowledge by > now and didn't reflect on whether or not it was in the docs. > > I am almost sure that it has been mentioned a number of times > on the mailing list, but that's neither here nor there. It > should be documented. > > It's not just debugging that can cause this. Code change and > stray messages can do it too. For the same reason that a > process should be prepared to throw away unrecognized > messages in some top-level state, it should not allow such > messages to interfere with timeout handling. > > BR, > Ulf W > > Kenneth Lundin wrote: >> >> This is not a known issue. >> After a quick glance I agree with you that a sys:get_status call to >> the timer_server or any other >> "gen_server" using the gen_server timeout feature can disturb the >> timing either by delaying >> the timeout message or by not letting any timeout message to be sent. >> >> I will put this on the list of things to ?look at. >> >> It is whoever very unlikely that this happens in normal operation >> since there is no reason to call >> sys:get_status other than for debugging purposes. >> >> I have never heard of any real problems caused by this. >> >> /Kenneth ?Erlang/OTP Ericsson >> >> On Wed, Apr 21, 2010 at 10:05 PM, johan munk wrote: >>> >>> Following the control flow in the gen_server, sys, and timer modules I >>> see >>> this: >>> Calling sys:get_status(timer_server) will "reset" the "after time" in the >>> loop function of gen_server. As the timer server uses that time to get a >>> timeout if no other messages are received, one can potentially cause >>> timers >>> to never trigger. >>> Is this a known issue/feature? >>> >>> /JM >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > -- > Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > http://www.erlang-solutions.com > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG > SOLUTIONS LTD. > > www.erlang-solutions.com > > From maruthavanan_s@REDACTED Mon Apr 26 13:44:20 2010 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Mon, 26 Apr 2010 07:44:20 -0400 Subject: SQLite ODBC Message-ID: Hi, I want to connect SQLite to ODBC drivers. Is this possible with latest version? I was able to successfully connection but when I try to get the columns I fail with "no column supported" error. Thanks, Marutha From rtrlists@REDACTED Mon Apr 26 14:10:05 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 26 Apr 2010 13:10:05 +0100 Subject: [erlang-questions] SQLite ODBC In-Reply-To: References: Message-ID: On Mon, Apr 26, 2010 at 12:44 PM, maruthavanan s wrote: > > Hi, > > I want to connect SQLite to ODBC drivers. Is this possible with latest > version? > > I was able to successfully connection but when I try to get the columns I > fail with "no column supported" error. > > Thanks, > Marutha > The Erlang ODBC lib does not currently support Unicode. You will need to convert Unicode columns as part of your query. For example, by using ODBC conversion functions: SELECT {fn CONVERT(unicode_column, SQL_VARCHAR)} "char_column" FROM table BTW, blobs are also not in there at the moment. Robby From rtrlists@REDACTED Mon Apr 26 17:20:36 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 26 Apr 2010 16:20:36 +0100 Subject: [erlang-questions] SQLite ODBC In-Reply-To: References: Message-ID: On Mon, Apr 26, 2010 at 4:09 PM, maruthavanan s wrote: > Hi, > > Thanks a lot got the problem, > > When I select INT column it is working fine, but when I try to get VARCHAR > still could not make it up. > > Do I need to write any custom function. > > I am using the below query. and get the below result. > > > 11> odbc:sql_query(Pid,"select SERVER_ID from Server"). > {selected,["Server_ID"],[{2}]} > > 12>odbc:sql_query(Pid,"select {fn CONVERT(SERVER_IP, SQL_VARCHAR)} "har" > from Server"). > * 1: syntax error before: har > > I think I dont know to proceed with ODBC functions. Please help. > > Thanks, > Marutha > > > Ah, you'll need to escape internal " characters. Try: odbc:sql_query(Pid,"select {fn CONVERT(SERVER_IP, SQL_VARCHAR)} \"har\" from Server"). Robby From maruthavanan_s@REDACTED Mon Apr 26 17:09:59 2010 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Mon, 26 Apr 2010 11:09:59 -0400 Subject: [erlang-questions] SQLite ODBC In-Reply-To: References: , Message-ID: Hi, Thanks a lot got the problem, When I select INT column it is working fine, but when I try to get VARCHAR still could not make it up. Do I need to write any custom function. I am using the below query. and get the below result. 11> odbc:sql_query(Pid,"select SERVER_ID from Server"). {selected,["Server_ID"],[{2}]} 12>odbc:sql_query(Pid,"select {fn CONVERT(SERVER_IP, SQL_VARCHAR)} "har" from Server"). * 1: syntax error before: har I think I dont know to proceed with ODBC functions. Please help. Thanks, Marutha On Mon, Apr 26, 2010 at 12:44 PM, maruthavanan s wrote: Hi, I want to connect SQLite to ODBC drivers. Is this possible with latest version? I was able to successfully connection but when I try to get the columns I fail with "no column supported" error. Thanks, Marutha The Erlang ODBC lib does not currently support Unicode. You will need to convert Unicode columns as part of your query. For example, by using ODBC conversion functions: SELECT {fn CONVERT(unicode_column, SQL_VARCHAR)} "char_column" FROM table BTW, blobs are also not in there at the moment. Robby From ingela.andin@REDACTED Mon Apr 26 21:57:57 2010 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 26 Apr 2010 21:57:57 +0200 Subject: [erlang-questions] SQLite ODBC In-Reply-To: References: Message-ID: 2010/4/26 Robert Raschke : > On Mon, Apr 26, 2010 at 12:44 PM, maruthavanan s > wrote: > >> >> Hi, >> >> I want to connect SQLite to ODBC drivers. Is this possible with latest >> version? >> >> I was able to successfully connection but when I try to get the columns I >> fail with "no column supported" error. >> >> Thanks, >> Marutha >> > > > The ?Erlang ODBC lib does not currently support Unicode. You will need to > convert Unicode columns as part of your query. For example, by using ODBC > conversion functions: > SELECT {fn CONVERT(unicode_column, SQL_VARCHAR)} "char_column" FROM table There will be some unicode support in the upcoming release, now available in github if you are interested. Regards Ingela, Erlang OTP team, Ericsson AB From garret.smith@REDACTED Tue Apr 27 01:16:04 2010 From: garret.smith@REDACTED (Garret Smith) Date: Mon, 26 Apr 2010 16:16:04 -0700 Subject: supervisor process not responding to messages ('EXIT', which_children, etc) Message-ID: The system is Windows 2k3 service pack 2, running as a VMWare guest. I am running Erlang as a Windows service via erlsrv, so boot script and all. Memory consumption is stable around 25MB, CPU usage seldom exceeds 10%. The application seems to run fine for a while, then "strange" things start happening. I have finally been able to catch it. I have a main application supervisor, a couple child supervisors, and many workers under each 2nd level supervisor. All 2nd level supervisors are permanent. Restart strategy is one_for_one. In this case, 3 workers died within a few seconds of each other, causing the 2nd level supervisor to die. The application supervisor did not restart the 2nd level supervisor. I started an interactive VM and connected to the service to investigate. This is a copy of "Info" from appmon of the application supervisor: ------------------------------------- Node: 'n1@REDACTED, Process: <0.97.0> [{registered_name,ac_supervisor}, {current_function,{proc_lib,sync_wait,2}}, {initial_call,{proc_lib,init_p,5}}, {status,waiting}, {message_queue_len,3}, {messages,[{'EXIT',<0.114.0>,shutdown}, {'EXIT',<0.101.0>,shutdown}, {'$gen_call',{<0.6225.0>,#Ref<0.0.167.237088>},which_children}]}, {links,[<0.104.0>,<0.117.0>,<0.2213.0>,<0.98.0>,<0.95.0>]}, {dictionary,[{'$ancestors',[<0.95.0>]}, {'$initial_call',{supervisor,ac_supervisor,1}}]}, {trap_exit,true}, {error_handler,error_handler}, {priority,normal}, {group_leader,<0.94.0>}, {total_heap_size,754}, {heap_size,377}, {stack_size,21}, {reductions,468}, {garbage_collection,[{fullsweep_after,65535},{minor_gcs,4}]}, {suspending,[]}] ------------------------------------------- The messages queue concerns me. The last message, which_children, was me running rpc:call(n1@REDACTED, supervisor, which_children, []) from my interactive VM. The rpc call did not return after many seconds, so I killed the interactive VM. The first 2 messages appear to be the 2nd level supervisors dying. According to the SASL log, these supervisors died over 2 days ago and have never been restarted. In testing, they have always been restarted immediately up to the restart limit. For comparison, I got the same info from a "non-hung" supervisor. The biggest difference, besides the empty message queue, was that current_function was {gen_server, loop, 6} instead of {proc_lib,sync_wait,2}. I found the same on another stalled supervisor. I would love any help on what the problem is, what I can do to continue to diagnose, what more data I can provide. Thanks, Garret Smith From garret.smith@REDACTED Tue Apr 27 01:26:37 2010 From: garret.smith@REDACTED (Garret Smith) Date: Mon, 26 Apr 2010 16:26:37 -0700 Subject: Fwd: supervisor process not responding to messages ('EXIT', which_children, etc) In-Reply-To: References: Message-ID: Forgot the important detail, running R13B03. Thanks, Garret Smith From caox@REDACTED Tue Apr 27 04:00:32 2010 From: caox@REDACTED (caox) Date: Tue, 27 Apr 2010 10:00:32 +0800 Subject: heap allocate error Message-ID: hi I got an heap allocate error when using binary_to_term to operate received binary from c driver : Crash dump was written to: erl_crash.dump eheap_alloc: Cannot allocate 1824525600 bytes of memory (of type "heap"). And I found the memory size being allocated was about 10 times lager than the actual size of the binary. How could I avoid the crash of system in this situation? From max.lapshin@REDACTED Tue Apr 27 07:25:36 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 27 Apr 2010 09:25:36 +0400 Subject: cron inside erlang Message-ID: Hi, I have a problem: I need to receive message each minute. Jitter about 1 second is ok, but I need a strict warranty, that there will be 86400 messages per day. Thus I'm afraid that simple timer:send_interval can be not-monotonic, because it can do following: Pid ! Message receive after Timeout -> repeat end Will send_interval fit my needs, or I have to make own mechanism with synchronization from wall_clock? From rtrlists@REDACTED Tue Apr 27 10:20:23 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 27 Apr 2010 09:20:23 +0100 Subject: [erlang-questions] cron inside erlang In-Reply-To: References: Message-ID: On Tue, Apr 27, 2010 at 6:25 AM, Max Lapshin wrote: > Hi, I have a problem: I need to receive message each minute. Jitter > about 1 second is ok, > but I need a strict warranty, that there will be 86400 messages per day. > > Thus I'm afraid that simple timer:send_interval can be not-monotonic, > because it can do following: > > Pid ! Message > receive > after > Timeout -> repeat > end > > Will send_interval fit my needs, or I have to make own mechanism with > synchronization from wall_clock? > > Not sure I understand the "receive message each minute". Do you want to essentially limit your reception capacity to one per minute? Or did you mean "send one message every minute"? It might be that crone (http://catseye.tc/projects/crone/) could help you with the wall clock side of things. Robby From max.lapshin@REDACTED Tue Apr 27 10:25:07 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 27 Apr 2010 12:25:07 +0400 Subject: [erlang-questions] cron inside erlang In-Reply-To: References: Message-ID: Yes, thanks, I will look at it. I mean, that I need to perform some action in the end of each minute and it is very important to trigger this action only 1440 times a day (not 86000 of course =). I think, that crone meets my requirements. From Antonio.Musumeci@REDACTED Tue Apr 27 16:12:54 2010 From: Antonio.Musumeci@REDACTED (Musumeci, Antonio S) Date: Tue, 27 Apr 2010 10:12:54 -0400 Subject: Deployment sizes of erlang nodes Message-ID: <01489E237C2C8F45AF7898E135F48E80034BDE758E@NYWEXMBX2129.msad.ms.com> How large have people deployed pools of networked Erlang nodes? Dozens, hundreds, thousands? Thanks. -------------------------------------------------------------------------- NOTICE: If received in error, please destroy, and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error. We may monitor and store emails to the extent permitted by applicable law. From comptekki@REDACTED Tue Apr 27 20:08:54 2010 From: comptekki@REDACTED (Wes James) Date: Tue, 27 Apr 2010 12:08:54 -0600 Subject: compiling erlang Message-ID: I'm noticing a lot of "erlc -W +debug_info" during the "make" phase of erlang. Is debug info being generated for all of erlang - if so, won't this make erlang slower? -wes From ulf.wiger@REDACTED Tue Apr 27 21:04:33 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 27 Apr 2010 21:04:33 +0200 Subject: [erlang-questions] compiling erlang In-Reply-To: References: Message-ID: <4BD73541.1030803@erlang-solutions.com> Wes James wrote: > I'm noticing a lot of "erlc -W +debug_info" during the "make" phase > of erlang. Is debug info being generated for all of erlang - if so, > won't this make erlang slower? No. Debug_info is stored in the beam file and can be retrieved using the function beam_lib:chunks(BeamFile, [abstract_code]). It is ignored by the code loader and has no effect on the runtime behaviour. Debug_info can be used by offline analysis tools, such as dialyzer, and also by runtime analysis tools like cover, which reads the debug_info, instruments the code, compiles it and loads it, replacing the original module. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From fritchie@REDACTED Tue Apr 27 21:05:15 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 27 Apr 2010 14:05:15 -0500 Subject: [erlang-questions] supervisor process not responding to messages ('EXIT', which_children, etc) In-Reply-To: Message of "Mon, 26 Apr 2010 16:16:04 PDT." Message-ID: <55916.1272395115@snookles.snookles.com> Garret Smith wrote: gs> For comparison, I got the same info from a "non-hung" supervisor. gs> The biggest difference, besides the empty message queue, was that gs> current_function was {gen_server, loop, 6} instead of gs> {proc_lib,sync_wait,2}. I found the same on another stalled gs> supervisor. Perhaps erlang:process_info(Pid, backtrace) or erlang:process_display(Pid, backtrace) would help figure out where the supervisor is hung. My guess is that it's dealing with a previous failure and/or restart. -Scott From comptekki@REDACTED Tue Apr 27 21:09:33 2010 From: comptekki@REDACTED (Wes James) Date: Tue, 27 Apr 2010 13:09:33 -0600 Subject: [erlang-questions] compiling erlang In-Reply-To: <4BD73541.1030803@erlang-solutions.com> References: <4BD73541.1030803@erlang-solutions.com> Message-ID: On Tue, Apr 27, 2010 at 1:04 PM, Ulf Wiger wrote: > Wes James wrote: >> >> I'm noticing a lot of "erlc -W +debug_info" during the "make" phase >> of erlang. Is debug info being generated for all of erlang - if so, >> won't this make erlang slower? > > No. Debug_info is stored in the beam file and can be retrieved using > the function beam_lib:chunks(BeamFile, [abstract_code]). It is > ignored by the code loader and has no effect on the runtime behaviour. > > Debug_info can be used by offline analysis tools, such as dialyzer, > and also by runtime analysis tools like cover, which reads the > debug_info, instruments the code, compiles it and loads it, replacing > the original module. > > BR, > Ulf W OK - thanks for the info. -wes From kostis@REDACTED Tue Apr 27 21:38:08 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 27 Apr 2010 22:38:08 +0300 Subject: [erlang-questions] compiling erlang In-Reply-To: References: Message-ID: <4BD73D20.3080800@cs.ntua.gr> Wes James wrote: > I'm noticing a lot of "erlc -W +debug_info" during the "make" phase > of erlang. Is debug info being generated for all of erlang - if so, > won't this make erlang slower? No. The presence of debug info has no effect on runtime performance. It only makes the .beam file a bit bigger but it's not really a problem. Kostis From dougedmunds@REDACTED Tue Apr 27 22:50:48 2010 From: dougedmunds@REDACTED (Doug Edmunds (gmail)) Date: Tue, 27 Apr 2010 13:50:48 -0700 Subject: tail recursion question Message-ID: <4BD74E28.9030300@gmail.com> In this example module, loop1 is tail recursive, but are loop2 and loop3 equally tail recursive? or do loop2 and loop3 leave something on the stack? Thanks. --dae-- -module(loops). -compile(export_all). start() -> register(loop1, spawn(?MODULE, loop1,[10])), register(loop2, spawn(?MODULE, loop2,[10])), register(loop3, spawn(?MODULE, loop3,[10])). loop1(State) -> io:format("loop1:~p~n",[State]), receive a -> State2 = State - 1; b -> State2 = State + 1; _ -> State2 = State end, loop1(State2). loop2(State) -> io:format("loop2:~p~n",[State]), receive a -> State2 = State - 1 , loop2(State2); b -> State2 = State + 1 , loop2(State2); _ -> State2 = State, loop2(State2) end. loop3(State) -> io:format("loop3:~p~n",[State]), receive a -> State2 = State - 1 , loop3(State2); b -> State2 = State + 1 , loop3(State2); _ -> State2 = State, loop3(State2) end, never_reach_this(). never_reach_this() -> io:format("Am I tail recursive or not?"). From v@REDACTED Wed Apr 28 00:22:14 2010 From: v@REDACTED (Valentin Micic) Date: Wed, 28 Apr 2010 00:22:14 +0200 Subject: [erlang-questions] tail recursion question In-Reply-To: <4BD74E28.9030300@gmail.com> Message-ID: loop2 - yes, it is tail-recursive. loop3 - following an approach that compiler should not second-guess a programmer, I'd say *no*, this is not a tail-recursive function. This should be relatively simple to test -- run the code by calling loop3/1, and see if the memory increases over time. If it does (eventually dumping the core), the loop3/1 is definitely *not* tail recursive :-) V/ On 2010/04/27 10:50 PM, "Doug Edmunds (gmail)" wrote: > In this example module, loop1 is tail recursive, but > are loop2 and loop3 equally tail recursive? > or do loop2 and loop3 leave something on the stack? > > Thanks. > --dae-- > > -module(loops). > -compile(export_all). > > start() -> > register(loop1, spawn(?MODULE, loop1,[10])), > register(loop2, spawn(?MODULE, loop2,[10])), > register(loop3, spawn(?MODULE, loop3,[10])). > > loop1(State) -> > io:format("loop1:~p~n",[State]), > receive > a -> > State2 = State - 1; > b -> > State2 = State + 1; > _ -> > State2 = State > end, > loop1(State2). > > > loop2(State) -> > io:format("loop2:~p~n",[State]), > receive > a -> > State2 = State - 1 , > loop2(State2); > b -> > State2 = State + 1 , > loop2(State2); > _ -> > State2 = State, > loop2(State2) > end. > > loop3(State) -> > io:format("loop3:~p~n",[State]), > receive > a -> > State2 = State - 1 , > loop3(State2); > b -> > State2 = State + 1 , > loop3(State2); > _ -> > State2 = State, > loop3(State2) > end, > never_reach_this(). > > never_reach_this() -> > io:format("Am I tail recursive or not?"). > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From garret.smith@REDACTED Wed Apr 28 01:46:14 2010 From: garret.smith@REDACTED (Garret Smith) Date: Tue, 27 Apr 2010 18:46:14 -0500 Subject: [erlang-questions] supervisor process not responding to messages ('EXIT', which_children, etc) In-Reply-To: <55916.1272395115@snookles.snookles.com> References: <55916.1272395115@snookles.snookles.com> Message-ID: The results of 'erlang:process_info(Pid, backtrace)' below as you suggested. It seems that the supervisor was trying to restart a child, the child took too long to start so it was killed, but then the supervisor hung. At this point, I can have the child start faster, but why is the supervisor hung? Thanks, -Garret Smith $Program counter: 0x00a55114 (proc_lib:sync_wait/2 + 28) CP: 0x00000000 (invalid) arity = 0 0x00b27d18 Return addr 0x00acbd9c (supervisor:do_start_child/2 + 156) y(0) infinity y(1) <0.2213.0> 0x00b27d24 Return addr 0x00ace9d4 (supervisor:restart/3 + 1772) y(0) {child,<0.111.0>,set_supervisor,{set_supervisor,start_link,[]},permanent,1000,supervisor,[set_supervisor]} y(1) {local,ac_supervisor} y(2) Catch 0x00acbd9c (supervisor:do_start_child/2 + 156) 0x00b27d34 Return addr 0x00accf18 (supervisor:handle_info/2 + 76) y(0) {state,{local,ac_supervisor},one_for_one,[{child,<0.117.0>,es_syncer,{es_syncer,start_link,[]},permanent,1000,worker,[es_syncer]},{child,<0.114.0>,algorithm_supervisor,{algorithm_supervisor,start_link,[]},permanent,1000,supervisor,[algorithm_supervisor]},{child,<0.111.0>,set_supervisor,{set_supervisor,start_link,[]},permanent,1000,supervisor,[set_supervisor]},{child,<0.104.0>,input_supervisor,{input_supervisor,start_link,[]},permanent,1000,supervisor,[input_supervisor]},{child,<0.101.0>,output_supervisor,{output_supervisor,start_link,[]},permanent,1000,supervisor,[output_supervisor]},{child,<0.98.0>,mbgw_supervisor,{mbgw_supervisor,start_link,[]},permanent,1000,supervisor,[mbgw_supervisor]}],{dict,0,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},2,3600,[{1272,125254,287042}],ac_supervisor,[]} y(1) {child,<0.111.0>,set_supervisor,{set_supervisor,start_link,[]},permanent,1000,supervisor,[set_supervisor]} y(2) {local,ac_supervisor} 0x00b27d44 Return addr 0x00a8b128 (gen_server:handle_msg/5 + 932) 0x00b27d48 Return addr 0x00a54ce0 (proc_lib:init_p_do_apply/3 + 28) y(0) supervisor y(1) {state,{local,ac_supervisor},one_for_one,[{child,<0.117.0>,es_syncer,{es_syncer,start_link,[]},permanent,1000,worker,[es_syncer]},{child,<0.114.0>,algorithm_supervisor,{algorithm_supervisor,start_link,[]},permanent,1000,supervisor,[algorithm_supervisor]},{child,<0.111.0>,set_supervisor,{set_supervisor,start_link,[]},permanent,1000,supervisor,[set_supervisor]},{child,<0.104.0>,input_supervisor,{input_supervisor,start_link,[]},permanent,1000,supervisor,[input_supervisor]},{child,<0.101.0>,output_supervisor,{output_supervisor,start_link,[]},permanent,1000,supervisor,[output_supervisor]},{child,<0.98.0>,mbgw_supervisor,{mbgw_supervisor,start_link,[]},permanent,1000,supervisor,[mbgw_supervisor]}],{dict,0,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},2,3600,[],ac_supervisor,[]} y(2) ac_supervisor y(3) <0.95.0> y(4) {'EXIT',<0.111.0>,shutdown} y(5) Catch 0x00a8b128 (gen_server:handle_msg/5 + 932) 0x00b27d64 Return addr 0x00773ea4 () y(0) Catch 0x00a54cf0 (proc_lib:init_p_do_apply/3 + 44) On Tue, Apr 27, 2010 at 2:05 PM, Scott Lystig Fritchie wrote: > Garret Smith wrote: > > gs> For comparison, I got the same info from a "non-hung" supervisor. > gs> The biggest difference, besides the empty message queue, was that > gs> current_function was {gen_server, loop, 6} instead of > gs> {proc_lib,sync_wait,2}. ?I found the same on another stalled > gs> supervisor. > > Perhaps erlang:process_info(Pid, backtrace) or > erlang:process_display(Pid, backtrace) would help figure out where the > supervisor is hung. ?My guess is that it's dealing with a previous > failure and/or restart. > > -Scott > From ok@REDACTED Wed Apr 28 03:58:52 2010 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 28 Apr 2010 13:58:52 +1200 Subject: [erlang-questions] tail recursion question In-Reply-To: References: Message-ID: <54CABBF9-5281-4496-8B07-2FE0E9EB921B@cs.otago.ac.nz> The issue is whether a function f(...) -> ... all cases have f(...) g(...). is tail recursive. More generally, it's whether calls to any function that is known not to terminate can be treated as tail calls. The quick answer is 'who cares? almost all functions should be *able* to terminate'. This is not to say that almost all functions should *have* to terminate, just that there should be some possibly rare situation where they can be shut down. The longer answer is that if a compiler can prove that a function call cannot terminate, then it doesn't have to arrange for anything to happen when that call returns, and it _may_ compile that call as a tail call. However, it would be far more useful for the compiler to report a "dead code" warning: the existence of the call to g(...) is strong evidence that some programmer at some time in the past thought the calls to f(...) WOULD return, and if the compiler has proven that they don't, that's evidence of at least a quality issue. A human programmer can remove the call to g(...), or, more plausibly, figure out what the base case of f should be and ensure that f *can* return. An Erlang compiler is not required to check termination or non-termination, and so is not *required* to compile such weird code as tail recursive. From dmurray@REDACTED Wed Apr 28 04:29:43 2010 From: dmurray@REDACTED (David N Murray) Date: Tue, 27 Apr 2010 22:29:43 -0400 (EDT) Subject: Thinking in Erlang Message-ID: Hi, I'm one of those people that learned lisp and moved on, but am a better programmer for what I learned from lisp. One of the things I picked up was function returning functions and the lisp version of closures. In my Erlang reading, I tend to think this way when I read about anonymous functions and closures. I have a function: (defun make-fqueue (n) (let ((queue '())) (lambda (e) (push e queue) (if (> (length queue) n) (setf queue (subseq queue 0 n))) queue))) The above function returns a function that maintains a fixed-length queue, pushing each element it receives onto the queue and popping the head off when the queue reaches a certain size and always returning the queue. Multiple calls to make-fqueue return multiple queue "objects." What I'm realling doing above is encapsulating a mutable list and carrying it around inside the function, hidden from the user. If I try and implement something like this in Erlang, and don't want to be passing the list around to whomever may use it, I'm doing it with a gen_server, correct? TIA, Dave From jacob.vorreuter@REDACTED Wed Apr 28 05:29:49 2010 From: jacob.vorreuter@REDACTED (Jacob Vorreuter) Date: Tue, 27 Apr 2010 20:29:49 -0700 Subject: determining a socket's file descriptor integer value Message-ID: When a socket is opened is it possible to get any information about its file descriptor? I have an Erlang proxy server that accepts incoming HTTP connections, determines an appropriate backend server, opens a socket to that server and then proxies the HTTP request and response data between the backend server socket and the accepted client socket. I'm curious if I can accept the client socket in Erlang and then pass it to a NIF along with the ip and port of the backend server and do the actual proxying of data in C. Is that crazy talk? If I could determine the integer value of the client socket file descriptor that would be enough information to read and write from/to that socket in the C code, right? The Erlang socket is a tcp_inet port driver from what I can tell. Is there some way to query it for the information I'm looking for? Thanks, Jacob Vorreuter From vinoski@REDACTED Wed Apr 28 05:47:54 2010 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 27 Apr 2010 23:47:54 -0400 Subject: [erlang-questions] determining a socket's file descriptor integer value In-Reply-To: References: Message-ID: On Tue, Apr 27, 2010 at 11:29 PM, Jacob Vorreuter wrote: > When a socket is opened is it possible to get any information about its file descriptor? > > I have an Erlang proxy server that accepts incoming HTTP connections, determines an appropriate backend server, opens a socket to that server and then proxies the HTTP request and response data between the backend server socket and the accepted client socket. ?I'm curious if I can accept the client socket in Erlang and then pass it to a NIF along with the ip and port of the backend server and do the actual proxying of data in C. ?Is that crazy talk? ?If I could determine the integer value of the client socket file descriptor that would be enough information to read and write from/to that socket in the C code, right? > > The Erlang socket is a tcp_inet port driver from what I can tell. ?Is there some way to query it for the information I'm looking for? You can use prim_inet:getfd(Socket), with the caveat that prim_inet is an internal module. You might want to look at the Yaws sendfile linked-in driver I wrote a couple years ago, since it does similar things to what you're talking about. http://github.com/klacke/yaws/blob/master/c_src/yaws_sendfile_drv.c http://github.com/klacke/yaws/blob/master/src/yaws_sendfile.erl Not sure a NIF will work for you, though, given that you need to avoid blocking the VM. --steve From ok@REDACTED Wed Apr 28 06:20:02 2010 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 28 Apr 2010 16:20:02 +1200 Subject: [erlang-questions] Thinking in Erlang In-Reply-To: References: Message-ID: On Apr 28, 2010, at 2:29 PM, David N Murray wrote: [object creation in Lisp] > What I'm realling doing above is encapsulating a mutable list and > carrying > it around inside the function, hidden from the user. If I try and > implement something like this in Erlang, and don't want to be passing > the list around to whomever may use it, I'm doing it with a > gen_server, > correct? There may be two different questions lurking here, and they have different answers. (1) CAN you use gen_server to do this. Yes, of course. (2) That's rather heavyweight compared with the simplicity of the Lisp version, do you HAVE to do that? No, you don't. loop(State, N) -> receive {push,Item,Sender} -> Sender!{ok,self()} loop(lists:sublist(State, 2, N-1) ++ [Item], N) ; {pop,Sender} when State =/= [] -> Sender!{here,self(),head(State)}, loop(tail(State), N) ; quit -> ok end. push(Pid, Item) -> Pid!{push,Item,self()}, receive {ok,Pid} -> ok end. pop(Pid) -> Pid!{pop,self()}, receive {here,Pid,Item} -> Item end. quit(Pid) -> Pid!quit. new(N) when is_integer(N), N > 0 -> spawn(fun () -> loop([], N) end). There are three relevant differences between Lisp and Erlang. (1) Lisp has mutable data structures, Erlang doesn't. (2) Erlang has lightweight processes, Lisp doesn't. (3) Lisp has nested recursive functions, Erlang doesn't. The "loop" function cannot be inside the "new" function because it has to call itself, and Erlang has no LABELS form. I would not be surprised to see that change some day. Modules with parameters can be seen as an embarassingly clumsy way to _sort of_ get nested functions, except in that case it's the outer function that's crippled. From jacob.vorreuter@REDACTED Wed Apr 28 06:59:14 2010 From: jacob.vorreuter@REDACTED (Jacob Vorreuter) Date: Tue, 27 Apr 2010 21:59:14 -0700 Subject: [erlang-questions] determining a socket's file descriptor integer value In-Reply-To: References: Message-ID: Thanks Steve, that's perfect! On Apr 27, 2010, at 8:47 PM, Steve Vinoski wrote: > On Tue, Apr 27, 2010 at 11:29 PM, Jacob Vorreuter > wrote: >> When a socket is opened is it possible to get any information about its file descriptor? >> >> I have an Erlang proxy server that accepts incoming HTTP connections, determines an appropriate backend server, opens a socket to that server and then proxies the HTTP request and response data between the backend server socket and the accepted client socket. I'm curious if I can accept the client socket in Erlang and then pass it to a NIF along with the ip and port of the backend server and do the actual proxying of data in C. Is that crazy talk? If I could determine the integer value of the client socket file descriptor that would be enough information to read and write from/to that socket in the C code, right? >> >> The Erlang socket is a tcp_inet port driver from what I can tell. Is there some way to query it for the information I'm looking for? > > You can use prim_inet:getfd(Socket), with the caveat that prim_inet is > an internal module. > > You might want to look at the Yaws sendfile linked-in driver I wrote a > couple years ago, since it does similar things to what you're talking > about. > > http://github.com/klacke/yaws/blob/master/c_src/yaws_sendfile_drv.c > http://github.com/klacke/yaws/blob/master/src/yaws_sendfile.erl > > Not sure a NIF will work for you, though, given that you need to avoid > blocking the VM. > > --steve From v@REDACTED Wed Apr 28 10:01:02 2010 From: v@REDACTED (Valentin Micic) Date: Wed, 28 Apr 2010 10:01:02 +0200 Subject: [erlang-questions] tail recursion question In-Reply-To: <54CABBF9-5281-4496-8B07-2FE0E9EB921B@cs.otago.ac.nz> Message-ID: On 2010/04/28 3:58 AM, "Richard O'Keefe" wrote: > However, it would be far > more useful for the compiler to report a "dead code" warning: As far as I can recollect -- this is what Erlang compiler does, right? V/ From gary@REDACTED Wed Apr 28 10:05:46 2010 From: gary@REDACTED (Gary Hai) Date: Wed, 28 Apr 2010 16:05:46 +0800 Subject: Nagative integer number encode/decode error. Message-ID: <000401cae6a9$a0ec4260$e2c4c720$@com> Is it a bug? Change the code of complex.c list in http://erlang.org/doc/tutorial/example.html /* complex.c */ #include int foo(int x) { char* p = "abc"; printf("x-p = %d\n", x-(int)p); return x - (int)p; } int bar(int y) { printf("y = %d\n", (-y)); printf("y = %s\n", (char *)(-y)); return y; // return y*2; } Then compile and run the sample as http://erlang.org/doc/tutorial/cnode.html . Output of cserver: $ ./cserver 1234 Node server: [c1@REDACTED][vbox][c1] test erl_errno:0 Connected to test@REDACTED x-p = -134597552 y = 134597553 y = bc Output of Eshell V5.7.5 (test@REDACTED)8> AA = complex3:foo(0). -134597552 (test@REDACTED)9> complex3:bar(AA). -134597553 The complex3:bar(AA) should return the same value as AA. But it return the value decreased 1 to AA. Gary Hai From raimo+erlang-questions@REDACTED Wed Apr 28 12:21:34 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 28 Apr 2010 12:21:34 +0200 Subject: [erlang-questions] Calling external modules from a fun In-Reply-To: <76853.1272045646@snookles.snookles.com> References: <4301ECAF454148CAA9FCCCD09892276C@SSI.CORP> <76853.1272045646@snookles.snookles.com> Message-ID: <20100428102134.GA13251@erix.ericsson.se> On Fri, Apr 23, 2010 at 01:00:46PM -0500, Scott Lystig Fritchie wrote: > David Mercer wrote: > > dm> Since no-one smarter than me responded, here's my take. > > I don't claim to be any smarter on this subject, but there's some > empirical study possible while we wait for wisdom. > > dm> In order for a fun to work on a remote node, the module that created > dm> the fun must also exist on the remote node. > > True and false. If you use this syntax, yes, there's a module > dependency: > > spawn(Node, fun some_module:foo_func/0). I'd say it is entirely true, no false case. The module that created the fun must also exist on the remote node. The fun's code is not transfered to the other node when the fun is sent to the other node. The module that implements the fun has to exist on the other node, and in the same version as on the originating node. Quite some MD5 and other numbering magic is done to ensure that. > > There's a noticable difference in the sizes of: > > size(term_to_binary(fun some_module:foo_func/0)). > size(term_to_binary(fun() -> some_module:foo_func() end)). > > If the fun uses the fun() -> ... end syntax, then any function calls > made externally must be resolvable by the VM, the code server, etc. So > "fun() -> erlang:display(42) end" will work because the BIF definitely > exists over on the remote node ... because it's a BIF. No, that would not work since the code that calls erlang:display(42) must exist on the other node. I.e the fun code body. Sending fun erlang:display/0 would work since what is sent in this case is just the entry point, which exists on the other node. Sending fun some_module:foo_func/0 works up to the point where it is called. It is just the entry point that is transferred. In this case some_module:foo_func() is called regardless of which code implements it. If there is such a function it is called. Sending fun () -> some_module:foo_func() end also will work up to the point where it is called. Same as always it is only the entry point that is transferred. This entry point has a larger encoding since it contains module MD5 sums and other stuff to ensure that exactly that the code that the originating node expects is implementing the fun on te other node. When it is called, the module in which "fun () -> ... end" was compiled must exist on the other node. If it does it is executed and calls whatever function that is registered as some_module:foo_func/0 so such a module/function must also exist or that call will fail. > > The same resolvability property can be seen when using intra-module > calls. Using size(term_to_binary(Fun)), we see create experiments that > show that all of the transitively dependent functions are not included > in the fun because the size won't change. > > -Scott > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From johanmon@REDACTED Wed Apr 28 11:39:13 2010 From: johanmon@REDACTED (Johan Montelius) Date: Wed, 28 Apr 2010 11:39:13 +0200 Subject: [erlang-questions] Thinking in Erlang In-Reply-To: References: Message-ID: > it around inside the function, hidden from the user. If I try and > implement something like this in Erlang, and don't want to be passing > the list around to whomever may use it, I'm doing it with a gen_server, > correct? You could try something like this but the process way described by Richard O'Keefe is of course the Erlang way of doing it. Johan 8<---------------------------------------- -module(fqueue). -export([new/1, push/2]). new(N) -> fun(Elm) -> push(Elm, N, []) end. push(Queue, Element) -> Queue(Element). push(Element, N, All) -> Popped = pop([Element|All], N), fun(Elm) -> push(Elm, N, Popped) end. pop(All, N) -> if length(All) > N -> {Rest, Pop} = lists:split(N, All), io:format("popping ~w~n", [Pop]), Rest; true -> All end. 8<---------------------------------------- 24> Q0 = fqueue:new(2). #Fun 25> Q1 = fqueue:push(Q0, foo). #Fun 26> Q2 = fqueue:push(Q1, bar). #Fun 27> Q3 = fqueue:push(Q2, zot). popping [foo] #Fun 28> -- Dr Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From alessandro.sivieri@REDACTED Wed Apr 28 12:56:38 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Wed, 28 Apr 2010 12:56:38 +0200 Subject: [erlang-questions] Calling external modules from a fun In-Reply-To: <20100428102134.GA13251@erix.ericsson.se> References: <4301ECAF454148CAA9FCCCD09892276C@SSI.CORP> <76853.1272045646@snookles.snookles.com> <20100428102134.GA13251@erix.ericsson.se> Message-ID: 2010/4/28 Raimo Niskanen > > I'd say it is entirely true, no false case. The module that > created the fun must also exist on the remote node. > > The fun's code is not transfered to the other node > when the fun is sent to the other node. The module > that implements the fun has to exist on the other > node, and in the same version as on the originating > node. Quite some MD5 and other numbering magic > is done to ensure that. > Ah, thank you for your precise answer, even if it screws up some of my intentions... So if I want to send a function which does not exist in the receiving node, I need to send the source code, because the fun itself is not enough; so, in a way, I cannot create a closure in the sender, send it to a different node, and make the latter execute the function (a remote evaluation in the end), right? -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From johanmon@REDACTED Wed Apr 28 13:31:13 2010 From: johanmon@REDACTED (Johan Montelius) Date: Wed, 28 Apr 2010 13:31:13 +0200 Subject: Semantics of send Message-ID: What is the semantics of send? In the following example: to_a_or_to_ab_or_to_none(A, B) -> A ! foo, B ! bar, crash(). If B receives bar is it then so that A receives foo? And where is the behavior stated? Is it guaranteed by the Erlang implementation in a distributed setting? If the sender crashes, is disconnected ... Johan -- Dr Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From rumata-estor@REDACTED Wed Apr 28 13:53:48 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Wed, 28 Apr 2010 15:53:48 +0400 Subject: [erlang-questions] Semantics of send In-Reply-To: References: Message-ID: <4BD821CC.6070701@nm.ru> The code will be executed without checks are those messages delivered. "Pid ! Msg" doesn't check whether process with that Pid exists. The only check is made if you use no pid() but atom() for registered process. Johan Montelius wrote: > > > > What is the semantics of send? In the following example: > > to_a_or_to_ab_or_to_none(A, B) -> > A ! foo, > B ! bar, > crash(). > > If B receives bar is it then so that A receives foo? > > And where is the behavior stated? > > Is it guaranteed by the Erlang implementation in a distributed setting? > > If the sender crashes, is disconnected ... > > Johan > > > From rumata-estor@REDACTED Wed Apr 28 13:59:53 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Wed, 28 Apr 2010 15:59:53 +0400 Subject: [erlang-questions] Semantics of send In-Reply-To: <4BD821CC.6070701@nm.ru> References: <4BD821CC.6070701@nm.ru> Message-ID: <4BD82339.1060203@nm.ru> Maybe I answered not clear enough. If you use atom() instead of pid() it is like whereis(Atom) ! Msg. So the code will fail if no process is registered with that name. Still, no checks that the message is delivered. Dmitry Belyaev wrote: > The code will be executed without checks are those messages delivered. > "Pid ! Msg" doesn't check whether process with that Pid exists. > > The only check is made if you use no pid() but atom() for registered > process. > > Johan Montelius wrote: >> >> >> >> What is the semantics of send? In the following example: >> >> to_a_or_to_ab_or_to_none(A, B) -> >> A ! foo, >> B ! bar, >> crash(). >> >> If B receives bar is it then so that A receives foo? >> >> And where is the behavior stated? >> >> Is it guaranteed by the Erlang implementation in a distributed setting? >> >> If the sender crashes, is disconnected ... >> >> Johan >> >> >> > > From rumata-estor@REDACTED Wed Apr 28 14:09:38 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Wed, 28 Apr 2010 16:09:38 +0400 Subject: [erlang-questions] Semantics of send In-Reply-To: <4BD82339.1060203@nm.ru> References: <4BD821CC.6070701@nm.ru> <4BD82339.1060203@nm.ru> Message-ID: <4BD82582.3070600@nm.ru> Maybe I was too hurry to answer. http://erlang.org/doc/reference_manual/processes.html#id2279644 Message sending is asynchronous and safe, the message is guaranteed to eventually reach the recipient, provided that the recipient exists. "eventually" =) Dmitry Belyaev wrote: > > Maybe I answered not clear enough. > > If you use atom() instead of pid() it is like whereis(Atom) ! Msg. > So the code will fail if no process is registered with that name. > Still, no checks that the message is delivered. > > Dmitry Belyaev wrote: >> The code will be executed without checks are those messages delivered. >> "Pid ! Msg" doesn't check whether process with that Pid exists. >> >> The only check is made if you use no pid() but atom() for registered >> process. >> >> Johan Montelius wrote: >>> >>> >>> >>> What is the semantics of send? In the following example: >>> >>> to_a_or_to_ab_or_to_none(A, B) -> >>> A ! foo, >>> B ! bar, >>> crash(). >>> >>> If B receives bar is it then so that A receives foo? >>> >>> And where is the behavior stated? >>> >>> Is it guaranteed by the Erlang implementation in a distributed setting? >>> >>> If the sender crashes, is disconnected ... >>> >>> Johan >>> >>> >>> >> >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From dmurray@REDACTED Wed Apr 28 14:58:28 2010 From: dmurray@REDACTED (David N Murray) Date: Wed, 28 Apr 2010 08:58:28 -0400 (EDT) Subject: [erlang-questions] Thinking in Erlang In-Reply-To: References: Message-ID: On Apr 28, Richard O'Keefe scribed: > > > There may be two different questions lurking here, > and they have different answers. > > (1) CAN you use gen_server to do this. > Yes, of course. > (2) That's rather heavyweight compared with the simplicity of the > Lisp version, do you HAVE to do that? > No, you don't. > Thanks Richard and John. I appreciate the thorough responses. My small example of course have bigger implications. That style was a common theme in my Lisp programming: returning a function that carried whatever it needed. I'm trying to get a handle on the best way to approach my problem. It looks like either way I'm passing around additional data (either the queue or the pid) in Erlang. Thanks, Dave From johanmon@REDACTED Wed Apr 28 15:49:26 2010 From: johanmon@REDACTED (Johan Montelius) Date: Wed, 28 Apr 2010 15:49:26 +0200 Subject: [erlang-questions] Semantics of send In-Reply-To: <4BD82582.3070600@nm.ru> References: <4BD821CC.6070701@nm.ru> <4BD82339.1060203@nm.ru> <4BD82582.3070600@nm.ru> Message-ID: >>> to_a_or_to_ab_or_to_none(A, B) -> >>> A ! foo, >>> B ! bar, >>> crash(). On Wed, 28 Apr 2010 14:09:38 +0200, Dmitry Belyaev wrote: > Maybe I was too hurry to answer. > > http://erlang.org/doc/reference_manual/processes.html#id2279644 > Message sending is asynchronous and safe, the message is guaranteed to > eventually reach the recipient, provided that the recipient exists. Yes, according to the Reference Manual, messages will arrive (provided that A and B exists). This is probably also true if A resides on the same node as the sender. However If the sender does not reside on the same node things are not that simple. To start with we could have a temporary network problem causing the message foo to be lost (it is in the senders buffer until it gives up and throws the message away). The first send operation (as far as I know) does not wait for the foo message to be acknowledged and continues with sending bar to B. Another problem is if the foo message is placed in the out buffer waiting to be sent and bar is placed in another out buffer. What if bar is sent and then the sender node crashes. Maybe the reference manual should say: Message sending is asynchronous, the message is guaranteed to eventually reach the recipient, provided that the recipient exists and there is no problem. Is this the proper description of send? Johan -- Dr Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From erlang@REDACTED Wed Apr 28 16:43:45 2010 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 28 Apr 2010 16:43:45 +0200 Subject: [erlang-questions] Semantics of send In-Reply-To: References: <4BD821CC.6070701@nm.ru> <4BD82339.1060203@nm.ru> <4BD82582.3070600@nm.ru> Message-ID: On Wed, Apr 28, 2010 at 3:49 PM, Johan Montelius wrote: > > >>>> to_a_or_to_ab_or_to_none(A, B) -> >>>> A ! foo, >>>> B ! bar, >>>> crash(). > > On Wed, 28 Apr 2010 14:09:38 +0200, Dmitry Belyaev > wrote: > >> Maybe I was too hurry to answer. >> >> http://erlang.org/doc/reference_manual/processes.html#id2279644 >> Message sending is asynchronous and safe, the message is guaranteed to >> eventually reach the recipient, provided that the recipient exists. > > Yes, according to the Reference Manual, messages will arrive (provided that > A and B exists). This is probably also true if A resides on the same node as > the sender. However If the sender does not reside on the same node things > are not that simple. > > To start with we could have a temporary network problem causing the message > foo to be lost (it is in the senders buffer until it gives up and throws the > message away). The first send operation (as far as I know) does not wait for > the foo message to be acknowledged and continues with sending bar to B. > > Another problem is if the foo message is placed in the out buffer waiting to > be sent and bar is placed in another out buffer. What if bar is sent and > then the sender node crashes. > > Maybe the reference manual should say: Message sending is asynchronous, the > message is guaranteed to > eventually reach the recipient, provided that the recipient exists and > there is no problem. > > Is this the proper description of send? Sort of. It depends upon which level of abstraction you are talking about - as a first approximation its ok - but if you dig under the surface you'll find some problems. A guarantee is only any good if there is a method for determining if what you want has happened or not. The message could reach the recipient but you might never know. The only way to know is to send a confirmation back, you might not get the confirmation because of a machine failure or communication failure. So you can never confirm what happened. - -- 000 --- At the point in time when A sends the message B is alive. B is on a different machine. Just before the message was supposed to be delivered to B - B terminates normally. In this case there is "no problem" - exit(normal) means I'm terminating and there is no problem. But the message does not reach the recipient. Even if the message *is* received by the recipient this is no guarantee that the recipient does anything sensible with the message. Immediately after the message is received the process might die for a totally unrelated reason. If the sender needed to know that the receiver had not only received the message but done something sensible with it it would have to send back a confirmation message. "What happens if the confirmation message isn't received," asked the Byzantine General ..."send a confirmation of the confirmation ..." I guess an exact description of the semantics of send would involve a model of what A believes about the world and what B believes about the world, together with a state machine. A's belief as to whether B is alive and has received a message has nothing to do with whether B actually *is* alive. A only knows about how B *was* not how B *is* (based on the last message sent by B to A). In such a model you'd also have to model time. Where is the message when its in transit before it gets in the mailbox. Is TCP reliable, in the absence of a socket error and given that we have written all data to a socket, can we assume that the data is read from the socket? I have no idea what the answers to these questions are - pragmatically I mostly use a "if you want to guarantee that a message is processed, send back a reply after the procssing has worked" style of programming. Then I use links as a "catch all" - an invariant to kill off processes that have died. Links between nodes are ultimately are layered on top of messaging - so even if a process dies, you cannot assume the exit will always be delivered. I think it's valid to assume that between any two processes if you send a sequence of message to a process and get a sequence of replies that the message are processed in-order, so it should suffice to check the first and last message for liveness and assume that all the ones in the middle were OK. /Joe > > Johan > > > > -- > Dr Johan Montelius > Royal Institute of Technology - KTH > School of Information and Communication Technology - ICT > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From raimo+erlang-questions@REDACTED Wed Apr 28 17:19:18 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 28 Apr 2010 17:19:18 +0200 Subject: [erlang-questions] Calling external modules from a fun In-Reply-To: References: <4301ECAF454148CAA9FCCCD09892276C@SSI.CORP> <76853.1272045646@snookles.snookles.com> <20100428102134.GA13251@erix.ericsson.se> Message-ID: <20100428151918.GA20300@erix.ericsson.se> On Wed, Apr 28, 2010 at 12:56:38PM +0200, Alessandro Sivieri wrote: > 2010/4/28 Raimo Niskanen > > > > > > I'd say it is entirely true, no false case. The module that > > created the fun must also exist on the remote node. > > > > The fun's code is not transfered to the other node > > when the fun is sent to the other node. The module > > that implements the fun has to exist on the other > > node, and in the same version as on the originating > > node. Quite some MD5 and other numbering magic > > is done to ensure that. > > > > Ah, thank you for your precise answer, even if it screws up some of my > intentions... > So if I want to send a function which does not exist in the receiving node, > I need to send the source code, because the fun itself is not enough; so, in > a way, I cannot create a closure in the sender, send it to a different node, > and make the latter execute the function (a remote evaluation in the end), > right? Right! Unless you have a server on the other node that is prepared to scan and parse arbitrary source code... Or use erl_scan and erl_parse to make an rpc call to erl_eval, which is the same thing utilizing only core modules and servers: Try this on any two nodes (substitute Node2): (Node1)1> rpc:call(Node2, erl_eval, exprs, [[{call,1,{remote,1,{atom,1,io},{atom,1,format}}, {string,1,"~p~n"}, {cons,1, {call,1,{remote,1,{atom,1,erlang},{atom,1,nodes}}, []}, {nil,1}}]}], []]). [Node1] {value,ok,[]} (Node1)2> Where I got the long awful list (Parsed) from: {ok,Scanned,_} = erl_scan:string("io:format(\"~p~n\", [erlang:nodes()])."). {ok,Parsed} = erl_parse:parse_exprs(Scanned). Bindings = erl_eval:new_bindings(). and then rpc:call(Node2, erl_eval, exprs, Parsed, Bindings). There is more in the man pages for erl_eval, erl_parse and erl_scan... NOTE: rpc makes the printout [Node1] appear on this node (Node1) since it sets the process group leader for the rpc process on Node2 that executes erlang:nodes() to be the process group leader of the shell process on Node1. I.e erlang:nodes() is executed on Node2 so it says [Node1] and that is printed on the console of Node1 :-) > > -- > Sivieri Alessandro > alessandro.sivieri@REDACTED > http://www.chimera-bellerofonte.eu/ > http://www.poul.org/ -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From per@REDACTED Wed Apr 28 17:59:30 2010 From: per@REDACTED (Per Hedeland) Date: Wed, 28 Apr 2010 17:59:30 +0200 (CEST) Subject: What is a tid()? Message-ID: <201004281559.o3SFxUSh069872@pluto.hedeland.org> $ erlc test.erl ./test.erl:5: referring to built-in type tid as a remote type; please take out the module name --Per Hedeland test.erl------------------ -module(test). -export([test/1]). -spec test(cs:tid()) -> ok. test(_) -> ok. From mononcqc@REDACTED Wed Apr 28 18:06:08 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 28 Apr 2010 12:06:08 -0400 Subject: [erlang-questions] What is a tid()? In-Reply-To: <201004281559.o3SFxUSh069872@pluto.hedeland.org> References: <201004281559.o3SFxUSh069872@pluto.hedeland.org> Message-ID: If I recall correctly, they're table IDs used by ETS tables. You're basically using it as a remote type, but dialyzer seems to focus on the built-in part of it. On Wed, Apr 28, 2010 at 11:59 AM, Per Hedeland wrote: > $ erlc test.erl > ./test.erl:5: referring to built-in type tid as a remote type; please take > out the module name > > --Per Hedeland > > test.erl------------------ > > -module(test). > > -export([test/1]). > > -spec test(cs:tid()) -> ok. > test(_) -> ok. > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kostis@REDACTED Wed Apr 28 18:09:36 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 28 Apr 2010 19:09:36 +0300 Subject: [erlang-questions] What is a tid()? In-Reply-To: <201004281559.o3SFxUSh069872@pluto.hedeland.org> References: <201004281559.o3SFxUSh069872@pluto.hedeland.org> Message-ID: <4BD85DC0.7060608@cs.ntua.gr> Per Hedeland wrote: > $ erlc test.erl > ./test.erl:5: referring to built-in type tid as a remote type; please take out the module name tid() is a built-in opaque type for ETS table identifiers. These cannot be redefined. However, the warning you have run into predates the introduction of remote types and is probably a bug. Although one should currently use tid() to refer to ets:tid(), it should be possible to use a tid() type that one of their modules export. I'll put looking into this issue on my todo list. Kostis From fritchie@REDACTED Wed Apr 28 20:12:25 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 28 Apr 2010 13:12:25 -0500 Subject: [erlang-questions] supervisor process not responding to messages ('EXIT', which_children, etc) In-Reply-To: Message of "Tue, 27 Apr 2010 18:46:14 CDT." Message-ID: <26872.1272478345@snookles.snookles.com> gs> The results of 'erlang:process_info(Pid, backtrace)' below as you gs> suggested. It seems that the supervisor was trying to restart a gs> child, the child took too long to start so it was killed, but then gs> the supervisor hung. At this point, I can have the child start gs> faster, but why is the supervisor hung? The supervisor's behavior must be deterministic, so it starts children synchronously. (More on that in a little bit.) >From the supervisor:start_link() manual: The created supervisor process calls Module:init/1 to find out about restart strategy, maximum restart frequency and child processes. To ensure a synchronized start-up procedure, start_link/2,3 does not return until Module:init/1 has returned and all child processes have been started. You have to read between the lines to see that the above paragraph applies to you. A child's init func is handled synchronously. During the supervisor's start, Module:init/1 won't return until all the children are started. All restart strategies require that children be started in the order that they are specified. App developers rely on this child start order to preserve inter-process/service dependencies. If child processes were started in random order, application dependencies could be broken, and the app can run incorrectly or, perhaps worse yet, even fail to start at all. In your case, when a single worker has died and requires restarting, the supervisor is using the same method synchronous method of restarting the child. If a child can't start in a predictable (and hopefully very short) amount of time, then the variable-time work needs to be done after the child's init function returns. The strategies mentioned a few days ago in the "Subject: testing asynchronous code" thread can be very useful. -Scott From rvirding@REDACTED Wed Apr 28 22:17:34 2010 From: rvirding@REDACTED (Robert Virding) Date: Wed, 28 Apr 2010 22:17:34 +0200 Subject: [erlang-questions] Calling external modules from a fun In-Reply-To: <20100428151918.GA20300@erix.ericsson.se> References: <4301ECAF454148CAA9FCCCD09892276C@SSI.CORP> <76853.1272045646@snookles.snookles.com> <20100428102134.GA13251@erix.ericsson.se> <20100428151918.GA20300@erix.ericsson.se> Message-ID: On 28 April 2010 17:19, Raimo Niskanen wrote: > Try this on any two nodes (substitute Node2): > > ? ?(Node1)1> rpc:call(Node2, erl_eval, exprs, > ? ? ? ? ? ? ? ? ? ? ? [[{call,1,{remote,1,{atom,1,io},{atom,1,format}}, > ? ? ? ? ? ? ? ? ? ? ? ? ?{string,1,"~p~n"}, > ? ? ? ? ? ? ? ? ? ? ? ? ?{cons,1, > ? ? ? ? ? ? ? ? ? ? ? ? ? {call,1,{remote,1,{atom,1,erlang},{atom,1,nodes}}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ?[]}, > ? ? ? ? ? ? ? ? ? ? ? ? ? {nil,1}}]}], > ? ? ? ? ? ? ? ? ? ? ? []]). > ? ?[Node1] > ? ?{value,ok,[]} > ? ?(Node1)2> > > Where I got the long awful list (Parsed) from: > > ? ?{ok,Scanned,_} = erl_scan:string("io:format(\"~p~n\", [erlang:nodes()])."). > ? ?{ok,Parsed} = erl_parse:parse_exprs(Scanned). > ? ?Bindings = erl_eval:new_bindings(). > > and then > > ? ?rpc:call(Node2, erl_eval, exprs, Parsed, Bindings). Of course if you are going to be doing a lot of sending of source code to be evaluated on a remote machine then it would be easier and shorter to use LFE. The equivalent to the above in vanilla erlang is: Expr = [':',io,format,[quote,"~p~n"],[list,[':',erlang,nodes]]], rpc:call(Node2, lfe_eval, expr, [Expr]). Doing it all in LFE if course even nicer: (let ((e '(: io format '"~p~n" (list (: erlang nodes))))) (: rpc call node 'lfe_eval 'expr (list e))) Robert From per@REDACTED Wed Apr 28 22:41:53 2010 From: per@REDACTED (Per Hedeland) Date: Wed, 28 Apr 2010 22:41:53 +0200 (CEST) Subject: [erlang-questions] What is a tid()? In-Reply-To: <4BD85DC0.7060608@cs.ntua.gr> Message-ID: <201004282041.o3SKfrHO075654@pluto.hedeland.org> Kostis Sagonas wrote: > >Per Hedeland wrote: >> $ erlc test.erl >> ./test.erl:5: referring to built-in type tid as a remote type; please take out the module name > >tid() is a built-in opaque type for ETS table identifiers. Thanks - maybe it's already in the queue for an update to http://www.erlang.org/doc/reference_manual/typespec.html? >These cannot be redefined. > >However, the warning you have run into predates the introduction of >remote types and is probably a bug. Although one should currently use >tid() to refer to ets:tid(), it should be possible to use a tid() type >that one of their modules export. Well, it can't be defined in the other module anyway, can it? If so that's perfectly OK by me, and the error is appropriate - it's just a documentation omission. --Per From alessandro.sivieri@REDACTED Wed Apr 28 23:01:42 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Wed, 28 Apr 2010 23:01:42 +0200 Subject: [erlang-questions] Calling external modules from a fun In-Reply-To: References: <4301ECAF454148CAA9FCCCD09892276C@SSI.CORP> <76853.1272045646@snookles.snookles.com> <20100428102134.GA13251@erix.ericsson.se> <20100428151918.GA20300@erix.ericsson.se> Message-ID: 2010/4/28 Robert Virding > Of course if you are going to be doing a lot of sending of source code > to be evaluated on a remote machine then it would be easier and > shorter to use LFE. The equivalent to the above in vanilla erlang is: > > Expr = [':',io,format,[quote,"~p~n"],[list,[':',erlang,nodes]]], > rpc:call(Node2, lfe_eval, expr, [Expr]). > > Doing it all in LFE if course even nicer: > > (let ((e '(: io format '"~p~n" (list (: erlang nodes))))) > (: rpc call node 'lfe_eval 'expr (list e))) > > Well, this goes a little beyond my functional languages knowledge... :) -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From zabrane3@REDACTED Wed Apr 28 23:21:05 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Wed, 28 Apr 2010 23:21:05 +0200 Subject: http or httpc? Message-ID: Hi List, There's 2 http clients coming Erlang: * http * httpc Which one is recommended? Regards From mevans@REDACTED Wed Apr 28 23:51:33 2010 From: mevans@REDACTED (Evans, Matthew) Date: Wed, 28 Apr 2010 17:51:33 -0400 Subject: pg2...a warning Message-ID: Hi, We have a specific configuration where there are 2 special hardware nodes that perform a specific task. We have an instance of a gen_server process on each of these nodes that acts as a gateway for other processes to request this task. This request is done by doing a call to pg2:get_closest_pid/1. The problem is that an asynchronous operation, beyond our control, can cause pg2:join/2 to be called many times for the same process. The result of which is that pg2:get_closest_pid/1 will not be random (e.g. process on node 1 gets 5 "joins", and node 2 gets 3 "joins"). Or rather it will not be random in how we consider it to be (i.e. we only want a process to join a group a single time). The documentation implies that this is the case: join(Name, Pid) -> ok | {error, Reason} Types: Name = term() Pid = pid() Reason = {no_such_group, Name} Joins the process Pid to the group Name. A process can join a group several times; it must then leave the group the same number of times. We solved this by doing: Self = self(), ok = case lists:member(Self, pg2:get_members(?MODULE)) of true -> ok; _ -> pg2:join(?MODULE, Self) end; To ensure we join only once. I'm sure this is intentional, but perhaps the documentation should be more specific? Or even have an option to allow only a single join. Regards Matt From ok@REDACTED Thu Apr 29 00:12:37 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 29 Apr 2010 10:12:37 +1200 Subject: [erlang-questions] tail recursion question In-Reply-To: References: Message-ID: <4F70DBF7-D5C8-4BAC-AB3D-6BBB62AA5A6E@cs.otago.ac.nz> On Apr 28, 2010, at 8:01 PM, Valentin Micic wrote: > On 2010/04/28 3:58 AM, "Richard O'Keefe" wrote: > >> However, it would be far >> more useful for the compiler to report a "dead code" warning: > > As far as I can recollect -- this is what Erlang compiler does, right? Wrong. Try typing in this little module. -module(foo). -export([f/1]). f(N) -> f(N+1), g(N). g(N) -> exit(N). This is an example of the kind of code we're talking about, where the g(N) call is in fact dead because the f(N+1) call cannot return. In order to report the g(N) call as dead, the compiler would have to *notice* that f/1 never returns. It doesn't. The compiler is completely silent about this. That's not a complaint. From ok@REDACTED Thu Apr 29 00:48:18 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 29 Apr 2010 10:48:18 +1200 Subject: [erlang-questions] Semantics of send In-Reply-To: References: Message-ID: <75581F42-FEB2-4328-83B2-12212368B7E8@cs.otago.ac.nz> On Apr 28, 2010, at 11:31 PM, Johan Montelius wrote: > > > What is the semantics of send? In the following example: > > to_a_or_to_ab_or_to_none(A, B) -> > A ! foo, > B ! bar, > crash(). > > If B receives bar is it then so that A receives foo? Consider the following scenario: (1) The message for A is scheduled for delivery in the VM (2) The message for B is scheduled for delivery in the VM (3) The sender crashes. (4) The VM puts the message for B on a wire, which happens to be routed through a microwave link. (5) The VM puts the message for A on another wire, which happens to be routed through an RFC1149 [%] link. (6) The message for B arrives. (7) The RFC1149 carrier is eaten by a sparrowhawk. [%] Internet Protocol over Avian Carriers (pigeons). According to the Wikipedia, it has been implemented. The Norwegian version, where the avian carriers carried some sort of memory chip, actually beat ADSL for throughput, albeit with terrible latency. > > And where is the behavior stated? Both the "Erlang Programming" book and the "Programming Erlang" book explain it, it's in the documentation, ... > > Is it guaranteed by the Erlang implementation in a distributed > setting? As explained above, how _could_ it be? > > If the sender crashes, is disconnected ... Once a process has launched a message, there is no further relationship whatsoever between that process and that message, unless the programmer explicitly placed the process's PID as part of the message data. You could replace "processes" by "people" and "messages" by "snail-mail letters", and nothing would change. Message transport between nodes is inherently unreliable. Message transport _within_ a node is normally taken as reliable, but even then, it's possible for process A itself to die before all messages sent to it are delivered. > From ok@REDACTED Thu Apr 29 00:55:18 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 29 Apr 2010 10:55:18 +1200 Subject: [erlang-questions] Thinking in Erlang In-Reply-To: References: Message-ID: On Apr 29, 2010, at 12:58 AM, David N Murray wrote: > It looks like either way I'm passing around additional data (either > the > queue or the pid) in Erlang. If you wanted to use a mutable queue in Lisp, you would have to pass around *SOME* kind of reference to it. In the Erlang world, all that changes is that the reference is a process ID instead of the address of a mutable object. Let me show another example, assuming my previous code: newt(N) -> Queue = new(N), fun ({push,X}) -> push(Queue, X) ; (pop) -> pop(Queue, X) ; (quit) -> quit(Queue) end. > F = newt(10). > F({push,1}). > F({push,2}). > F(pop). ==> 1 > F(pop). ==> 2 > F(quit). Whether you return an "object identifier" which can then be passed to various functions, or you return a "handler function" which can then be passed "messages", is really quite unimportant. You can take EITHER approach in Lisp. You can take EITHER approach in Erlang. You have to pass ONE thing around, either way, with either language. You don't "pass" anything "additional". > > Thanks, > Dave > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jay@REDACTED Thu Apr 29 03:25:05 2010 From: jay@REDACTED (Jay Nelson) Date: Wed, 28 Apr 2010 18:25:05 -0700 Subject: Calling external modules from a fun Message-ID: <0697586F-DD89-464A-9F7E-BE5715421B00@duomark.com> > Doing it all in LFE if course even nicer: > (let ((e '(: io format '"~p~n" (list (: erlang nodes))))) > (: rpc call node 'lfe eval 'expr (list e))) > Robert Boy, that's some happy looking code! (: jay From dmurray@REDACTED Thu Apr 29 01:50:16 2010 From: dmurray@REDACTED (David N Murray) Date: Wed, 28 Apr 2010 19:50:16 -0400 (EDT) Subject: [erlang-questions] Thinking in Erlang In-Reply-To: References: Message-ID: On Apr 29, Richard O'Keefe scribed: > > F(quit). > > Whether you return an "object identifier" which can then be > passed to various functions, or you return a "handler function" > which can then be passed "messages", is really quite unimportant. > You can take EITHER approach in Lisp. > You can take EITHER approach in Erlang. > You have to pass ONE thing around, either way, with either language. > You don't "pass" anything "additional". > Thanks Richard. This last example was the way I was thinking about it in Lisp. I didn't consider making the function call as carrying something around. I think I may be thinking more of scheme where I can do this: (define q (make-queue 10)) (q elem) whereas the lisp version would be more along the lines of (defparameter q (make-queue 10)) (funcall q elem) Which is exactly what you were pointing out and I though I was trying to avoid. Thanks for the tutorial. Dave From garret.smith@REDACTED Thu Apr 29 05:54:37 2010 From: garret.smith@REDACTED (Garret Smith) Date: Wed, 28 Apr 2010 20:54:37 -0700 Subject: [erlang-questions] supervisor process not responding to messages ('EXIT', which_children, etc) In-Reply-To: <26872.1272478345@snookles.snookles.com> References: <26872.1272478345@snookles.snookles.com> Message-ID: Thanks a lot for your help working through this Scott. First off, I need to move any work that could take more than a few ms out of the init/1 function. This makes sense from an initial startup as well as restart case. Would the best way be to move any long-running tasks to handle_cast with a gen_server:cast, or handle_event with a gen_fsm:send_all_state_event in init/1 depending on worker type? Second, the initial start order and synchronous start makes sense. I am relying on this behavior myself. Third, I'm not sure I understand the restart case. If I was using the rest_for_one restart strategy, I would expect all children after the failed child to be killed and restarted synchronously and in order. However, I am using the one_for_one restart strategy, so I expect the supervisor to restart only the failed child without restarting any other children (but still synchronously). An example of my structure for conversation, which seems like a common pattern app_supervisor child_sup1 child_sup_2 child_sup_3 worker1 worker2 w1 w2 w3 w1 w2 w3 Say enough workers under child_sup_2 die in a short time to exceed the restart limit. child_sup_2 then exits as expected. app_supervisor then restarts child_sup_2 as expected. child_sup_2 takes too long to restart, so app_supervisor kills it during init, also terminating any workers that had started. At this point, what should happen? Without digging into documentation right now, I would expect either app_supervisor to immediately exit, or to continue trying to restart child_sup_2 until it succeeds or reaches the max restart count and exits. What I have observed is that app_supervisor is deadlocked in proc_lib:sync_wait/2. It no longer responds to any messages: 'EXIT' signals from other children, which_children messages from supervisor:which_children, etc. I am pretty sure that this is not intended behavior... To summarize, I can fix the problem by moving any long-running tasks out of init/1, and should do this regardless, but I would expect OTP to do something other than hang. Thoughts? -Garret Smith On Wed, Apr 28, 2010 at 11:12 AM, Scott Lystig Fritchie wrote: > gs> The results of 'erlang:process_info(Pid, backtrace)' below as you > gs> suggested. ?It seems that the supervisor was trying to restart a > gs> child, the child took too long to start so it was killed, but then > gs> the supervisor hung. ?At this point, I can have the child start > gs> faster, but why is the supervisor hung? > > The supervisor's behavior must be deterministic, so it starts children > synchronously. ?(More on that in a little bit.) > > From the supervisor:start_link() manual: > > ? ?The created supervisor process calls Module:init/1 to find out about > ? ?restart strategy, maximum restart frequency and child processes. To > ? ?ensure a synchronized start-up procedure, start_link/2,3 does not > ? ?return until Module:init/1 has returned and all child processes have > ? ?been started. > > You have to read between the lines to see that the above paragraph > applies to you. ?A child's init func is handled synchronously. ?During > the supervisor's start, Module:init/1 won't return until all the > children are started. ?All restart strategies require that children be > started in the order that they are specified. > > App developers rely on this child start order to preserve > inter-process/service dependencies. ?If child processes were started in > random order, application dependencies could be broken, and the app can > run incorrectly or, perhaps worse yet, even fail to start at all. > > In your case, when a single worker has died and requires restarting, the > supervisor is using the same method synchronous method of restarting the > child. ?If a child can't start in a predictable (and hopefully very > short) amount of time, then the variable-time work needs to be done > after the child's init function returns. ?The strategies mentioned a few > days ago in the "Subject: testing asynchronous code" thread can be very > useful. > > -Scott > From emmiller@REDACTED Thu Apr 29 08:39:07 2010 From: emmiller@REDACTED (Evan Miller) Date: Thu, 29 Apr 2010 01:39:07 -0500 Subject: Chicago Boss 0.3 is available Message-ID: Greetings! Chicago Boss is an Erlang MVC that tries to mimic the best of Ruby on Rails and Python's Django. I've put together the first beta release tonight, which you can download here: http://www.chicagoboss.org/download.html (Version 0.1 sounds wimpy so I skipped straight to 0.3.) To celebrate, I wrote a semi-epic beginner's guide that takes you through everything from installing a Tokyo Tyrant database to implementing a log-in system with Chicago Boss. The guide is called "An Evening With Chicago Boss": http://www.evanmiller.org/chicago-boss-guide.html Finally I've set up a mailing list for the project on Google groups. If you act fast, you can be the second member! Direct any Boss-related questions there: http://groups.google.com/group/chicagoboss Erlang technologies included with Boss: * ErlyDTL * Etcher * SimpleBridge * Mochiweb * Medici * Lots of Boss-only magic :-) Cheers, Evan -- Evan Miller http://www.evanmiller.org/ From mail@REDACTED Thu Apr 29 08:55:33 2010 From: mail@REDACTED (Tim Fletcher) Date: Wed, 28 Apr 2010 23:55:33 -0700 (PDT) Subject: http or httpc? In-Reply-To: References: Message-ID: <89298b8d-5ecd-4fde-9a21-d1b23582d140@g21g2000yqk.googlegroups.com> > There's 2 http clients coming Erlang: > * http > * httpc The httpc module is the actual http client implementation, the http module is a thin API wrapper. > Which one is recommended? Use the http module unless you have a good reason not to: http://www.erlang.org/doc/apps/inets/http_client.html#id2260841 Tim From ulf.wiger@REDACTED Thu Apr 29 09:29:22 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 29 Apr 2010 09:29:22 +0200 Subject: [erlang-questions] pg2...a warning In-Reply-To: References: Message-ID: <4BD93552.5090809@erlang-solutions.com> Evans, Matthew wrote: > > The problem is that an asynchronous operation, beyond our control, > can cause pg2:join/2 to be called many times for the same process. > The result of which is that pg2:get_closest_pid/1 will not be random > (e.g. process on node 1 gets 5 "joins", and node 2 gets 3 "joins"). > Or rather it will not be random in how we consider it to be (i.e. we > only want a process to join a group a single time). This made me curious. I will admit to not having used pg2, but the other day I was inspired to explore how to emulate pg2's behaviour using gproc [1]. I noted the part in the documentation stating that you can join several times, but didn't catch the fact that get_members/1 would include each pid once for each time it has joined. This seems to imply that joining several times serves an entirely different purpose than relieving the programmer of the trouble of keeping track of whether or not it has joined the group before. OTOH, the man page doesn't mention this at all, which makes me believe that it's a bug rather than a feature. It talks about how you should use pg2:get_members/1 when you want to send a message to all members of a group. This would be a good place to highlight the fact that you need to remove duplicates from the list if you want the message to be sent only once to each member. BR, Ulf W [1] Making a pg2-like module on top of gproc is actually quite easy, but requires the distributed gproc to work, which has not been the case until now. I am in the process of verifying it, and will hopefully be able to push a new version very soon. -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From alessandro.sivieri@REDACTED Thu Apr 29 10:10:45 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Thu, 29 Apr 2010 10:10:45 +0200 Subject: [erlang-questions] Re: Calling external modules from a fun In-Reply-To: <0697586F-DD89-464A-9F7E-BE5715421B00@duomark.com> References: <0697586F-DD89-464A-9F7E-BE5715421B00@duomark.com> Message-ID: Ok, now I'm playing a bit with the "code" module: I have found out that I can send and load a module binary code more than once, without any problem, so I wanted to ask what happens if I load a module with the same name twice, but each time with different source code (for example, an updated module): does this work like when a module is recompiled during execution? That is, the new code is used only if invoked as module:function? And what if I load a module (always through the code:load_binary) already present in the code path of the interpreter? Thanks for your answers so far! -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From johanmon@REDACTED Thu Apr 29 10:23:04 2010 From: johanmon@REDACTED (Johan Montelius) Date: Thu, 29 Apr 2010 10:23:04 +0200 Subject: [erlang-questions] Semantics of send In-Reply-To: <4BD8A52F.7010008@gmail.com> References: <4BD8A52F.7010008@gmail.com> Message-ID: On Wed, 28 Apr 2010 23:14:23 +0200, Richard Carlsson wrote: >> What is the semantics of send? In the following example: >> >> to_a_or_to_ab_or_to_none(A, B) -> >> A ! foo, >> B ! bar, >> crash(). >> >> If B receives bar is it then so that A receives foo? >There are no such guarantees unless A=B (that is, if both > messages are from the same sender to the same receiver). Not even then does it hold true. On Thu, 29 Apr 2010 00:48:18 +0200, Richard O'Keefe wrote: >> And where is the behavior stated? > > Both the "Erlang Programming" book and the "Programming Erlang" book > explain it, it's in the documentation, ... Yes but not in the Reference Manual. > transport between nodes is inherently unreliable. Message > transport _within_ a node is normally taken as reliable, > but even then, it's possible for process A itself to die > before all messages sent to it are delivered. There is certainly a difference in reliability in the implementation of in-node message sending and ex-node sending. I (without having looked in the src) would think that it is guaranteed in the case above that if bar reached the message queue of B then foo reaches the message queue of A. On Wed, 28 Apr 2010 16:43:45 +0200, Joe Armstrong wrote: > ... Links between nodes are ultimately are > layered on top of messaging - so even if a process dies, you cannot > assume the exit will always be delivered. I think we should be able to reply on some signals, signals that can be generated locally i.e. DOWN/noconnection. I know that reliable message sending is not guaranteed by the implementation but from the specification in the Reference Manual you would think it so. The statement in the Reference Manual is quite different from Barklund and Virding draft specification of Erlang 10.6.2: "There are situations where it is not reasonable to require that all signals arrive at their destination, ..." It is also different from Svensson and Fredlunds description in "A More Accurate Semantics for Distributed Erlang" where it is made clear that message sending between nodes is unreliable and that messages can be dropped. In the end there is of course always situations where you don't know if a message arrived or not but there are situations where you do know and you can also determine if it could be that a message was lost. I think the language should take advantage of this. The semantic description of send should be tied to signals and especially the noconnection signal. If a message is possibly dropped then a noconnection signal should be sent to both the sender and receiver. In the case above, if the sender receives a noconnection signal it knows that foo might not have arrived. A is also informed that messages could have been dropped (even if foo actually did arrive). It could be that this is guaranteed by the system today and that is of course good but it should be in the specification of the language. If we had this then I think implementing fault tolerant distributed algorithms would be much easier. In stead of sending ack-messages on the application level one could rely on the fact that messages did arrive or a noconnection signal is generated. Johan -- Dr Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From per@REDACTED Thu Apr 29 10:53:37 2010 From: per@REDACTED (Per Hedeland) Date: Thu, 29 Apr 2010 10:53:37 +0200 (CEST) Subject: [erlang-questions] What is a tid()? In-Reply-To: <201004282041.o3SKfrHO075654@pluto.hedeland.org> Message-ID: <201004290853.o3T8rb9s092242@pluto.hedeland.org> Per Hedeland wrote: > >Kostis Sagonas wrote: >> >>Per Hedeland wrote: >>> $ erlc test.erl >>> ./test.erl:5: referring to built-in type tid as a remote type; please take out the module name >> >>tid() is a built-in opaque type for ETS table identifiers. > >Thanks - maybe it's already in the queue for an update to >http://www.erlang.org/doc/reference_manual/typespec.html? > >>These cannot be redefined. >> >>However, the warning you have run into predates the introduction of >>remote types and is probably a bug. Although one should currently use >>tid() to refer to ets:tid(), it should be possible to use a tid() type >>that one of their modules export. > >Well, it can't be defined in the other module anyway, can it? If so >that's perfectly OK by me, and the error is appropriate - it's just a >documentation omission. Actually, I take that back - thinking about it, it seems very wrong that this tid() should be built-in. Or rather, that it has the "global" name tid() instead of only ets:tid(), which would be both clearer and more logical (currently ets:tid() can't even be used). It's even listed along with match_spec() (which *requires* the ets: prefix) in ets(3). As far as I can see (only looking in the documentation....), all the other built-in / "global" types are defined in terms of the language, either as fundamental types or derived from other types - whereas tid() can only be defined in terms of a specific module (that the module happens to be partially implemented in the VM shouldn't be relevant). Surely this is actually a bug, and the typespec documentation that omits tid() is correct? --Per From ulf.wiger@REDACTED Thu Apr 29 11:17:08 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 29 Apr 2010 11:17:08 +0200 Subject: RFC: exprecs and record versions Message-ID: <4BD94E94.4010701@erlang-solutions.com> Inspired by a recent discussion on the riak mailing list, I went back to my exprecs module and started playing with a very obscure feature supporting conversions between different versions of records. http://github.com/uwiger/parse_trans I implemented it quite some time ago, but never documented it; the 'playing' was needed in order to remind myself of how it actually worked. My question is whether this might be useful as-is (i.e. needing only some documentation), or whether some vital piece is missing in order to make it useful. My initial intent was for it to be used in code_change callbacks, rather than having to explitly pattern-match on the old instance of a record using its tuple representation. Here is an example: -module(test_exprecs). -export([f/0]). -compile({parse_transform, exprecs}). -record(r, {a, b, c}). -record(foo, {a}). -record(foo__v1, {a,b}). -record(foo__v2, {a,b,c = 17}). -export_records([r,foo]). f() -> {new, '#new-r'([])}. To see how the code is transformed, parse_trans:pp_beam(Mod) can be helpful. Mod is either the module name (if the beam file is in the CWD), or the full beam file name. Here's a small shell interaction: Eshell V5.7.5 (abort with ^G) 1> Foo = test_exprecs:'#new-foo'([{a,17}]). {foo,17} 2> Foo_v1 = {foo,17,a}. % no API for creating old versions {foo,17,a} 3> Foo_v2 = {foo,17,a,b}. {foo,17,a,b} 4> test_exprecs:'#info-foo'(fields). [a] 5> test_exprecs:'#info-'(foo,fields,"v1"). [a,b] 6> test_exprecs:'#info-'(foo,fields,"v2"). [a,b,c] 7> test_exprecs:'#convert-'("v2",Foo_v2). {{foo,17},[{b,a},{c,b}]} The last command (7) converts an old record to the new format (record name without the __Vsn suffix is always the newest definition). It returns {Rec,Discarded}, where Discarded are the {AttrName, Value} pairs that were in the old record, but not in the new. Attributes that are in the new record but not the old will get default values, as per the record definition. Again, the original intent was for code_change/3 functions: code_change(FromVsn, State, _Extra) -> {NewState,[]} = '#convert-'(FromVsn,State), {ok, NewState}. For the above to work, you would need to quote the versioned records: -record('state__1.5', {...}), because of the dots in the version string. You could also make sure that the current record definition also exists with a version suffix. Given the complete lack of documentation, I would guess that the opportunity for backwards-incompatible changes is good, if you feel that the concept needs major surgery. :) BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From attila.r.nohl@REDACTED Thu Apr 29 11:30:00 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 29 Apr 2010 11:30:00 +0200 Subject: Fwd: [erlang-questions] Re: http or httpc? In-Reply-To: References: <89298b8d-5ecd-4fde-9a21-d1b23582d140@g21g2000yqk.googlegroups.com> Message-ID: I wanted to send this e-mail to the list... ---------- Forwarded message ---------- From: Attila Rajmund Nohl Date: Thu, 29 Apr 2010 11:29:30 +0200 Subject: Re: [erlang-questions] Re: http or httpc? To: Tim Fletcher 2010/4/29, Tim Fletcher : >> There's 2 http clients coming Erlang: >> * http >> * httpc > > The httpc module is the actual http client implementation, the http > module is a thin API wrapper. > >> Which one is recommended? > > Use the http module unless you have a good reason not to: > > http://www.erlang.org/doc/apps/inets/http_client.html#id2260841 I think that's documentation that wasn't updated. The http module is not even documented in the module list, and as far as I know, it's just a backward-compatible interface. New code should use httpc. From zabrane3@REDACTED Thu Apr 29 13:55:05 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Thu, 29 Apr 2010 13:55:05 +0200 Subject: [erlang-questions] Re: http or httpc? In-Reply-To: References: <89298b8d-5ecd-4fde-9a21-d1b23582d140@g21g2000yqk.googlegroups.com> Message-ID: I'll give httpc a try. Thanks guys ! 2010/4/29 Attila Rajmund Nohl > I wanted to send this e-mail to the list... > > ---------- Forwarded message ---------- > From: Attila Rajmund Nohl > Date: Thu, 29 Apr 2010 11:29:30 +0200 > Subject: Re: [erlang-questions] Re: http or httpc? > To: Tim Fletcher > > 2010/4/29, Tim Fletcher : > >> There's 2 http clients coming Erlang: > >> * http > >> * httpc > > > > The httpc module is the actual http client implementation, the http > > module is a thin API wrapper. > > > >> Which one is recommended? > > > > Use the http module unless you have a good reason not to: > > > > http://www.erlang.org/doc/apps/inets/http_client.html#id2260841 > > I think that's documentation that wasn't updated. The http module is > not even documented in the module list, and as far as I know, it's > just a backward-compatible interface. New code should use httpc. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From johanmon@REDACTED Thu Apr 29 16:35:14 2010 From: johanmon@REDACTED (Johan Montelius) Date: Thu, 29 Apr 2010 16:35:14 +0200 Subject: [erlang-questions] Semantics of send In-Reply-To: <4BD93C1F.6070600@chalmers.se> References: <4BD821CC.6070701@nm.ru> <4BD82339.1060203@nm.ru> <4BD82582.3070600@nm.ru> <4BD93C1F.6070600@chalmers.se> Message-ID: On Thu, 29 Apr 2010 09:58:23 +0200, Hans Svensson wrote: > Joe basically says it all, but if anyone wants a longer story I'll > insert a shameless plug for a paper we wrote some years ago discussing > exactly this kind of problems: > http://www.cs.chalmers.se/~hanssv/doc/ew07-dist.pdf > > If you read it, you'll learn that messages can indeed get lost, and the > only safe way to communicate is by using links *and* confirmation > messages! > > /Hans > I'm aware of the paper and also the paper "A more accurate Semantics for Distributed Erlang". And there the delivery of messages is connected to delivering of exit/noconnection signals. This is in my view crucial but the Erlang Reference manual makes no such connections. Johan -- Dr Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From mail@REDACTED Thu Apr 29 17:10:23 2010 From: mail@REDACTED (Tim Fletcher) Date: Thu, 29 Apr 2010 08:10:23 -0700 (PDT) Subject: Fwd: [erlang-questions] Re: http or httpc? In-Reply-To: References: <89298b8d-5ecd-4fde-9a21-d1b23582d140@g21g2000yqk.googlegroups.com> Message-ID: <43cfc227-546a-4c74-9835-2d9f9120949f@q32g2000yqb.googlegroups.com> > I think that's documentation that wasn't updated. The http module is > not even documented in the module list, and as far as I know, it's > just a backward-compatible interface. New code should use httpc. My mistake, I thought using the http module was the correct API. A lot of new-ish code uses the http module: http://google.com/codesearch?q=http%3Arequest+lang%3Aerlang In addition to the inets docs, some of the inets code also uses the http module (src/inets_app/inets_service.erl, test/httpc_SUITE.erl, and test/httpc_cookie_SUITE.erl). Does that need patching? Tim From g9414002.pccu.edu.tw@REDACTED Thu Apr 29 17:46:13 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Thu, 29 Apr 2010 23:46:13 +0800 Subject: [erlang-questions] Thinking in Erlang In-Reply-To: References: Message-ID: On Wed, Apr 28, 2010 at 10:29 AM, David N Murray wrote: > I have a function: > > (defun make-fqueue (n) > (let ((queue '())) > (lambda (e) > (push e queue) > (if (> (length queue) n) > (setf queue (subseq queue 0 n))) > queue))) > > The above function returns a function that maintains a fixed-length queue, > pushing each element it receives onto the queue and popping the head off > when the queue reaches a certain size and always returning the queue. > Multiple calls to make-fqueue return multiple queue "objects." > I reviewed and recalled some Lisp keywords while reading your program. I didn't understand why it is a Lambda function which will receive an element, input it into the inner queue, and end up as a queue but queue function. I supposed that the make-fqueue/1 returns a queue function which accepts an element and make another queue function. And the following is an Erlang translation of you make-fqueue/1: --------------------------------------------- %% In module "test" queue(N, Q_) -> fun (E) -> Q = push(E, Q_), if (length(Q) > N) -> Q1 = lists:sublist(Q, 1, N), E1 = lists:sublist(Q, N+1, N+1); true -> Q1 = Q, E1 = [] end, {queue(N, Q1), E1} end. push(A, Xs) -> [A|Xs]. ------------------------------------------- Ouput: 4> Q_ = test:queue(3,[]). #Fun 5> {Q1,O1} = Q_(a). {#Fun,[]} 6> {Q2,O2} = Q1(a). {#Fun,[]} 7> {Q3,O3} = Q2(a). {#Fun,[]} 8> {Q4,O4} = Q3(a). {#Fun,[a]} 9> {Q5,O5} = Q4(a). {#Fun,[a]} 10> {Q6,O6} = Q5(b). {#Fun,[a]} 11> {Q7,O7} = Q6(c). {#Fun,[a]} 12> {Q8,O8} = Q7(d). {#Fun,[a]} 13> {Q9,O9} = Q8(e). {#Fun,[b]} 14> {Q10,O10} = Q9(f). {#Fun,[c]} 15> Right? From mog-lists@REDACTED Thu Apr 29 17:45:48 2010 From: mog-lists@REDACTED (mog) Date: Thu, 29 Apr 2010 10:45:48 -0500 Subject: unforseen behavior Message-ID: <20100429154548.GI26687@woodman.lan> Hello I was playing with some code to cache results of some ldap queries and in doing so I am getting something odd attached is sample code that shows my problem. Is there a reason the vm wont call my unexported module? -- Matthew O'Gorman xim: mog@REDACTED email: mog@REDACTED site: http://blog.rldn.net +-----------------------------------------------------------------------+ It is easier for a camel to pass through the eye of a needle if it is lightly greased. -- Kehlog Albran, "The Profit" +-----------------------------------------------------------------------+ -------------- next part -------------- -module(test). -export([works/0, fails/0, exported_func/1]). exported_func(Subtract) -> 3 - Subtract. random_func(Add) -> 3 + Add. works() -> abstract_works(exported_func, 3). abstract_works(Func, Arg) -> io:format("Result is ~p~n", [test:Func(Arg)]). fails() -> abstract_fails(random_func, 3). abstract_fails(Func, Arg) -> io:format("Result is ~p~n", [Func(Arg)]). -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: Digital signature URL: From attila.r.nohl@REDACTED Thu Apr 29 17:49:56 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 29 Apr 2010 17:49:56 +0200 Subject: Fwd: [erlang-questions] Re: http or httpc? In-Reply-To: <43cfc227-546a-4c74-9835-2d9f9120949f@q32g2000yqb.googlegroups.com> References: <89298b8d-5ecd-4fde-9a21-d1b23582d140@g21g2000yqk.googlegroups.com> <43cfc227-546a-4c74-9835-2d9f9120949f@q32g2000yqb.googlegroups.com> Message-ID: 2010/4/29, Tim Fletcher : >> I think that's documentation that wasn't updated. The http module is >> not even documented in the module list, and as far as I know, it's >> just a backward-compatible interface. New code should use httpc. > > My mistake, I thought using the http module was the correct API. > > A lot of new-ish code uses the http module: > > http://google.com/codesearch?q=http%3Arequest+lang%3Aerlang I believe the httpc module is from 2009, so it was introduced only in inets-5.3.1 (the current version), so it's pretty new. > In addition to the inets docs, some of the inets code also uses the > http module (src/inets_app/inets_service.erl, test/httpc_SUITE.erl, > and test/httpc_cookie_SUITE.erl). Does that need patching? I don't know, it's wise to keep testing the legacy interface, the OTP team can't expect everyone to update immediately. From comptekki@REDACTED Thu Apr 29 20:39:38 2010 From: comptekki@REDACTED (Wes James) Date: Thu, 29 Apr 2010 12:39:38 -0600 Subject: parsing text Message-ID: I have a function grabbing a page and I'm pulling text out of the result. I can get the line: lists:nth(424,B). <<"Page Counter4880">> but 4880 will eventually get to 10000, etc. I can do string:substr(binary_to_list(lists:nth(424,B)), 34, 4). to get the 4880 out, but to do a check for > 9999 would I grab position 34 + the rest of the text then test positions after that for int? or maybe something like this? A=string:substr(binary_to_list(lists:nth(424,B)), 34, 6) B=string:substr(binary_to_list(lists:nth(424,B)), 34, 5) C=string:substr(binary_to_list(lists:nth(424,B)), 34, 4) case is_integer(A) -> A; is_integer(B) -> B; is_integer(C) -> C; end. Hmm - A, B, and C will be strings though so is_integer won't work. If I do list_to_integer(A), I guess I can write something like this: try list_to_integer(A) catch throw:Term -> Term; exit:Reason -> {'EXIT',Reason}; error:Reason -> {'EXIT',{Reason,erlang:get_stacktrace()}} end. and so forth.... -wes From attila.r.nohl@REDACTED Thu Apr 29 20:53:07 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 29 Apr 2010 20:53:07 +0200 Subject: [erlang-questions] parsing text In-Reply-To: References: Message-ID: 2010/4/29, Wes James : > I have a function grabbing a page and I'm pulling text out of the > result. I can get the line: > > lists:nth(424,B). > <<"Page Counter4880">> Try something like re:run(lists:nth(424,B), "(\\d+)", [{capture, first, list}]). From rvirding@REDACTED Thu Apr 29 20:57:04 2010 From: rvirding@REDACTED (Robert Virding) Date: Thu, 29 Apr 2010 20:57:04 +0200 Subject: [erlang-questions] Thinking in Erlang In-Reply-To: References: Message-ID: 2010/4/29 ??? (Yau-Hsien Huang) : > On Wed, Apr 28, 2010 at 10:29 AM, David N Murray wrote: > >> I have a function: >> >> (defun make-fqueue (n) >> ?(let ((queue '())) >> ? ?(lambda (e) >> ? ? ?(push e queue) >> ? ? ?(if (> (length queue) n) >> ? ? ? ?(setf queue (subseq queue 0 n))) >> ? ? ?queue))) >> >> The above function returns a function that maintains a fixed-length queue, >> pushing each element it receives onto the queue and popping the head off >> when the queue reaches a certain size and always returning the queue. >> Multiple calls to make-fqueue return multiple queue "objects." >> > > I reviewed and recalled some Lisp keywords while reading your program. > I didn't understand why it is a Lambda function which will receive an > element, > input it into the inner queue, and end up as a queue but queue function. > I supposed that the make-fqueue/1 returns a queue function which accepts > an element and make another queue function. And the following is an Erlang > translation of you make-fqueue/1: > > --------------------------------------------- > %% In module "test" > queue(N, Q_) -> > ? ?fun (E) -> > ? ? ? ? ? ?Q = push(E, Q_), > ? ?if (length(Q) > N) -> > ? ?Q1 = lists:sublist(Q, 1, N), > ? ?E1 = lists:sublist(Q, N+1, N+1); > ? ? ? true -> Q1 = Q, > ? ? ? E1 = [] > ? ?end, > ? ?{queue(N, Q1), E1} > ? ?end. > > push(A, Xs) -> > ? ?[A|Xs]. > ------------------------------------------- > > Ouput: > > 4> Q_ = test:queue(3,[]). > #Fun > 5> {Q1,O1} = Q_(a). > {#Fun,[]} > ... > 14> {Q10,O10} = Q9(f). > {#Fun,[c]} > 15> > > Right? The really big difference is that Scheme has mutable data while Erlang has immutable data. The Scheme version can destructively update a local variable within the lambda so you can keep calling the same lambda while in the Erlang version, yours, you must return a new fun each time to contain the updated data. That is one reason for using a process in this case, a pid is "mutable" in this sense in that you can send many messages to it. Robert From erlangy@REDACTED Thu Apr 29 21:00:09 2010 From: erlangy@REDACTED (Michael McDaniel) Date: Thu, 29 Apr 2010 12:00:09 -0700 Subject: [erlang-questions] parsing text In-Reply-To: References: Message-ID: <20100429190008.GT15986@delora.autosys.us> if I am understanding correctly what you want to do, how about something like ... X = <<"Page Counter4880">> , {match, [Val]} = re:run(X, ".*(?.*).*", [caseless,{capture,['VAL'],list}]) ... presuming your string is a constant format ~Michael On Thu, Apr 29, 2010 at 12:39:38PM -0600, Wes James wrote: > I have a function grabbing a page and I'm pulling text out of the > result. I can get the line: > > lists:nth(424,B). > <<"Page Counter4880">> > > > but 4880 will eventually get to 10000, etc. > > I can do > > string:substr(binary_to_list(lists:nth(424,B)), 34, 4). > > to get the 4880 out, but to do a check for > 9999 would I grab > position 34 + the rest of the text then test positions after that for > int? > > or maybe something like this? > > > A=string:substr(binary_to_list(lists:nth(424,B)), 34, 6) > B=string:substr(binary_to_list(lists:nth(424,B)), 34, 5) > C=string:substr(binary_to_list(lists:nth(424,B)), 34, 4) > case > is_integer(A) -> A; > is_integer(B) -> B; > is_integer(C) -> C; > end. > > > Hmm - A, B, and C will be strings though so is_integer won't work. If > I do list_to_integer(A), I guess I can write something like this: > > try list_to_integer(A) > catch > throw:Term -> Term; > exit:Reason -> {'EXIT',Reason}; > error:Reason -> {'EXIT',{Reason,erlang:get_stacktrace()}} > end. > > > and so forth.... > > -wes > > ________________________________________________________________ From dougedmunds@REDACTED Thu Apr 29 21:02:20 2010 From: dougedmunds@REDACTED (Doug Edmunds (gmail)) Date: Thu, 29 Apr 2010 12:02:20 -0700 Subject: emacs: moving to shell after compile Message-ID: <4BD9D7BC.60807@gmail.com> I have erlang-mode in .emacs: (setq load-path (cons "C:/home/erl5.7.5/lib/tools-2.6.5.1/emacs" load-path)) (setq erlang-root-dir "C:/home/erl5.7.5") (setq exec-path (cons "C:/home/erl5.7.5/bin" exec-path)) (require 'erlang-start) /lib/tools-2.6.5.1/emacs has: erlang.el erlang-eunit.el erlang-skels.el erlang-start.el I want C-C C-K to jump to the shell after compiling. I included (global-set-key [f12] 'other-window) so I can do this manually, but I'd like it to be automatic. What is the lisp code to use and which file should it go in? --dae From fritchie@REDACTED Thu Apr 29 21:11:51 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 29 Apr 2010 14:11:51 -0500 Subject: [erlang-questions] supervisor process not responding to messages ('EXIT', which_children, etc) In-Reply-To: Message of "Wed, 28 Apr 2010 20:54:37 PDT." Message-ID: <2870.1272568311@snookles.snookles.com> Garret Smith wrote: gs> Say enough workers under child_sup_2 die in a short time to exceed gs> the restart limit. child_sup_2 then exits as expected. gs> app_supervisor then restarts child_sup_2 as expected. child_sup_2 gs> takes too long to restart, so app_supervisor kills it during init, gs> also terminating any workers that had started. Er, but the last round of correspondence on this topic found that there isn't a timeout during worker init, so how that child_sup_2 get killed during its init? Am I misunderstanding something? gs> What I have observed is that app_supervisor is deadlocked in gs> proc_lib:sync_wait/2. It no longer responds to any messages: 'EXIT' gs> signals from other children, which_children messages from gs> supervisor:which_children, etc. I am pretty sure that this is not gs> intended behavior... It's a case of "Doctor, it hurts when I do this". The supervisor will become responsive again when the child's init is finished. The child's init is taking too long, so make it shorter. -Scott From dmurray@REDACTED Thu Apr 29 18:10:16 2010 From: dmurray@REDACTED (David N Murray) Date: Thu, 29 Apr 2010 12:10:16 -0400 (EDT) Subject: [erlang-questions] Thinking in Erlang In-Reply-To: References: Message-ID: On Apr 29, ??^C?^@^@??? (Yau-Hsien Huang) scribed: > On Wed, Apr 28, 2010 at 10:29 AM, David N Murray wrote: > > {#Fun,[c]} > 15> > > Right? > No, the lisp returns the queue (list). * (defparameter q (make-fqueue 3)) Q * (funcall q 1) (1) * (funcall q 2) (2 1) * (funcall q 3) (3 2 1) * (funcall q 4) (4 3 2) * Dave From mevans@REDACTED Thu Apr 29 21:23:51 2010 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 29 Apr 2010 15:23:51 -0400 Subject: [erlang-questions] pg2...a warning In-Reply-To: <4BD93552.5090809@erlang-solutions.com> References: <4BD93552.5090809@erlang-solutions.com> Message-ID: Thanks Ulf, Steve told mea bout your gproc work, It looks interesting. I actually ran into another pg2 strangeness today on another application. This process does a pg2:join in the init function. This is the ONLY place where this occurs. I would therefore like to know why pg2:get_members/2 reports two entries for that process? Very strange. -----Original Message----- From: Ulf Wiger [mailto:ulf.wiger@REDACTED] Sent: Thursday, April 29, 2010 3:29 AM To: Evans, Matthew Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] pg2...a warning Evans, Matthew wrote: > > The problem is that an asynchronous operation, beyond our control, > can cause pg2:join/2 to be called many times for the same process. > The result of which is that pg2:get_closest_pid/1 will not be random > (e.g. process on node 1 gets 5 "joins", and node 2 gets 3 "joins"). > Or rather it will not be random in how we consider it to be (i.e. we > only want a process to join a group a single time). This made me curious. I will admit to not having used pg2, but the other day I was inspired to explore how to emulate pg2's behaviour using gproc [1]. I noted the part in the documentation stating that you can join several times, but didn't catch the fact that get_members/1 would include each pid once for each time it has joined. This seems to imply that joining several times serves an entirely different purpose than relieving the programmer of the trouble of keeping track of whether or not it has joined the group before. OTOH, the man page doesn't mention this at all, which makes me believe that it's a bug rather than a feature. It talks about how you should use pg2:get_members/1 when you want to send a message to all members of a group. This would be a good place to highlight the fact that you need to remove duplicates from the list if you want the message to be sent only once to each member. BR, Ulf W [1] Making a pg2-like module on top of gproc is actually quite easy, but requires the distributed gproc to work, which has not been the case until now. I am in the process of verifying it, and will hopefully be able to push a new version very soon. -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From rvirding@REDACTED Thu Apr 29 21:42:23 2010 From: rvirding@REDACTED (Robert Virding) Date: Thu, 29 Apr 2010 21:42:23 +0200 Subject: [erlang-questions] Thinking in Erlang In-Reply-To: References: Message-ID: On 29 April 2010 18:10, David N Murray wrote: > On Apr 29, ??^C?^@^@??? (Yau-Hsien Huang) scribed: > >> On Wed, Apr 28, 2010 at 10:29 AM, David N Murray wrote: >> >> {#Fun,[c]} >> 15> >> >> Right? >> > > No, the lisp returns the queue (list). > > * (defparameter q (make-fqueue 3)) > Q > * (funcall q 1) > (1) > * (funcall q 2) > (2 1) > * (funcall q 3) > (3 2 1) > * (funcall q 4) > (4 3 2) > * Yes it returns the queue but it also destructively modifies a local variable inside the lambda so you can reuse the lambda, without having to return a new one each go. This you must do in Erlang so you must return a new fun each time as well. Robert From rvirding@REDACTED Thu Apr 29 21:56:27 2010 From: rvirding@REDACTED (Robert Virding) Date: Thu, 29 Apr 2010 21:56:27 +0200 Subject: [erlang-questions] unforseen behavior In-Reply-To: <20100429154548.GI26687@woodman.lan> References: <20100429154548.GI26687@woodman.lan> Message-ID: Func is an atom which is the name of a function and not a fun, not a function object. In abstract_fails/2 then Func(Arg) is not a valid function call as Func is not a fun. In exported_works/2 in test:Func(Arg), it is the name of an exported function in a module and is used together with the module name to call the function. This is legal. Calling exported functions in other modules has slightly different semantics. Robert On 29 April 2010 17:45, mog wrote: > Hello I was playing with some code to cache results of some ldap > queries and in doing so I am getting something odd attached is sample > code that shows my problem. ?Is there a reason the vm wont call my > unexported module? > > > -- > Matthew O'Gorman > xim: mog@REDACTED > email: mog@REDACTED > site: http://blog.rldn.net > +-----------------------------------------------------------------------+ > ?It is easier for a camel to pass through the eye of a needle if it is > ?lightly greased. > ? ? ? ? ? ? ? ? ?-- Kehlog Albran, "The Profit" > +-----------------------------------------------------------------------+ > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (GNU/Linux) > > iQIcBAEBAgAGBQJL2amrAAoJEPcfMjht6uSy8AMP/iyzfFJNKQdtdnQFdZStkITU > OoU3RiRoSRctKF6jKtBpMnbSPZ1LMwgD//SeW2SPKVXajckPK/cmMqLAC3hHEzIE > Oa85RyrhI3MSo1lHCYP5jAsztlSrxBbAcsfr2ag+rDx9ypp6S3VCAWBMs231Gzm3 > gZJ6AAr5YRdu6B4GNxyB06fuDV9giEBZHYwwey8LF0jAs/1+UIm03+SKOFoLloGy > oCWviLVcwfQclm2aJ1FNDyK093lmCpYwM2V8elPpzkruYnbDSLLJDvIlg+bkR6Mx > bDvIBqMawPm2ayRAB+9o1mFFlbOELE8vDonfLAmWCz9lK5lQrky/SEUHVPVKoooK > /887zvPTlbecagk4RpuZjaABaTqr53ryqvjJlBdqFNcoLGFf9cq8m+6ma+5MHkoY > wxfmvWAyrCbvywY5tl1NkqMZ4JRWtABYWMo++57OyC43zE9ZuLYq8+V0HwrVhzdU > 0V60uvpYvaXFWHEe+bmiMbL8d6Av2QNifjok2Y3UcJkRqYVrdMAScVK01rKF28Ki > B0xP+HT63YkwkhypaI/4UtM2YbRfBV53Zr8rkmkjvCSEqoDNzcv+a5iNBr5TEyVy > mgMcTtJlxxKdVbHjCnDGVYm3WQLFbcgsSdR6ItUtZYjqaBWBFBRmBOiwjCw+msDV > lOSFJfNPndB6SspvONR9 > =cUkA > -----END PGP SIGNATURE----- > > From comptekki@REDACTED Thu Apr 29 22:06:15 2010 From: comptekki@REDACTED (Wes James) Date: Thu, 29 Apr 2010 14:06:15 -0600 Subject: parsing text In-Reply-To: References: Message-ID: On Thu, Apr 29, 2010 at 12:39 PM, Wes James wrote: > I have a function grabbing a page and I'm pulling text out of the > result. ?I can get the line: Attila and Michael, Thx - looks like I've got some more studying to do on erlang usage. -wes From nem@REDACTED Thu Apr 29 22:11:26 2010 From: nem@REDACTED (Geoff Cant) Date: Thu, 29 Apr 2010 13:11:26 -0700 Subject: [erlang-questions] pg2...a warning In-Reply-To: References: <4BD93552.5090809@erlang-solutions.com> Message-ID: <9B73EB73-8E1E-4BF2-A973-76D5911A3FF7@erlang.geek.nz> I've also had some trouble with pg2 recently (the R13B04 version). It seems we have some code that joins groups repeatedly (or some other network condition - nodes parting/joining the cluster?) causes pg2 to think that pids have joined groups lots of times (for 180K values of lots). This causes spectacularly pathological behaviour in a pg2 cluster - you can get to the point where new nodes are no longer able to start pg2 as they run out of ram and abort when exchanging group definitions with other nodes. (pg2 does something like: all_members() -> [[G, group_members(G)] || G <- all_groups()]. group_members(Name) -> [P || [P, N] <- ets:match(pg2_table, {{member, Name, '$1'},'$2'}), _ <- lists:seq(1, N)]. all_groups() -> [N || [N] <- ets:match(pg2_table, {{group,'$1'}})]. when exchanging group membership information. In the pathological case I've run into it means sending [group_name, [ 180000 x Pid1, 180000 x Pid2, ... ] ] at startup. Clearly our code has some bugs as we're joining too often, but this behaviour is just nuts - it'd be cheaper to send our entire ets pg2_table over the wire) I'm pretty sure in our use of pg2 we want the 2nd and subsequent joins to be nops, and it's almost tempting to write 'pg3' just for that. Cheers, -Geoff On 2010-04-29, at 12:23 , Evans, Matthew wrote: > Thanks Ulf, > > Steve told mea bout your gproc work, It looks interesting. > > I actually ran into another pg2 strangeness today on another application. > > This process does a pg2:join in the init function. This is the ONLY place where this occurs. > > I would therefore like to know why pg2:get_members/2 reports two entries for that process? > > Very strange. > > -----Original Message----- > From: Ulf Wiger [mailto:ulf.wiger@REDACTED] > Sent: Thursday, April 29, 2010 3:29 AM > To: Evans, Matthew > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] pg2...a warning > > Evans, Matthew wrote: >> >> The problem is that an asynchronous operation, beyond our control, >> can cause pg2:join/2 to be called many times for the same process. >> The result of which is that pg2:get_closest_pid/1 will not be random >> (e.g. process on node 1 gets 5 "joins", and node 2 gets 3 "joins"). >> Or rather it will not be random in how we consider it to be (i.e. we >> only want a process to join a group a single time). > > This made me curious. > > I will admit to not having used pg2, but the other day I was inspired > to explore how to emulate pg2's behaviour using gproc [1]. > > I noted the part in the documentation stating that you can join > several times, but didn't catch the fact that get_members/1 would > include each pid once for each time it has joined. This seems to imply > that joining several times serves an entirely different purpose than > relieving the programmer of the trouble of keeping track of whether > or not it has joined the group before. > > OTOH, the man page doesn't mention this at all, which makes me believe > that it's a bug rather than a feature. It talks about how you should > use pg2:get_members/1 when you want to send a message to all members > of a group. This would be a good place to highlight the fact that > you need to remove duplicates from the list if you want the message > to be sent only once to each member. > > BR, > Ulf W > > [1] Making a pg2-like module on top of gproc is actually quite easy, > but requires the distributed gproc to work, which has not been the > case until now. I am in the process of verifying it, and will > hopefully be able to push a new version very soon. A solid, documented version of gproc would be a most welcome addition to OTP in my opinion. From mevans@REDACTED Thu Apr 29 22:59:59 2010 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 29 Apr 2010 16:59:59 -0400 Subject: [erlang-questions] pg2...a warning In-Reply-To: <9B73EB73-8E1E-4BF2-A973-76D5911A3FF7@erlang.geek.nz> References: <4BD93552.5090809@erlang-solutions.com> <9B73EB73-8E1E-4BF2-A973-76D5911A3FF7@erlang.geek.nz> Message-ID: I agree, and was going to reply with almost what you wrote. It seems that this occurs when we reboot a card that is part of an Erlang cluster. There is also the following code: handle_info({nodeup, Node}, S) -> gen_server:cast({?MODULE, Node}, {exchange, node(), all_members()}), {noreply, S}; Correct me if I'm wrong, but in a large cluster, won't 'Node' get this message "number of nodes in cluster (-1)" times? Basically, I'm seeing very strange behavior here. I'll dig a little deeper. I too am tempted to modify pg2 to only allow a process to join once. Matt -----Original Message----- From: Geoff Cant [mailto:nem@REDACTED] Sent: Thursday, April 29, 2010 4:11 PM To: Evans, Matthew Cc: Ulf Wiger; erlang-questions@REDACTED Subject: Re: [erlang-questions] pg2...a warning I've also had some trouble with pg2 recently (the R13B04 version). It seems we have some code that joins groups repeatedly (or some other network condition - nodes parting/joining the cluster?) causes pg2 to think that pids have joined groups lots of times (for 180K values of lots). This causes spectacularly pathological behaviour in a pg2 cluster - you can get to the point where new nodes are no longer able to start pg2 as they run out of ram and abort when exchanging group definitions with other nodes. (pg2 does something like: all_members() -> [[G, group_members(G)] || G <- all_groups()]. group_members(Name) -> [P || [P, N] <- ets:match(pg2_table, {{member, Name, '$1'},'$2'}), _ <- lists:seq(1, N)]. all_groups() -> [N || [N] <- ets:match(pg2_table, {{group,'$1'}})]. when exchanging group membership information. In the pathological case I've run into it means sending [group_name, [ 180000 x Pid1, 180000 x Pid2, ... ] ] at startup. Clearly our code has some bugs as we're joining too often, but this behaviour is just nuts - it'd be cheaper to send our entire ets pg2_table over the wire) I'm pretty sure in our use of pg2 we want the 2nd and subsequent joins to be nops, and it's almost tempting to write 'pg3' just for that. Cheers, -Geoff On 2010-04-29, at 12:23 , Evans, Matthew wrote: > Thanks Ulf, > > Steve told mea bout your gproc work, It looks interesting. > > I actually ran into another pg2 strangeness today on another application. > > This process does a pg2:join in the init function. This is the ONLY place where this occurs. > > I would therefore like to know why pg2:get_members/2 reports two entries for that process? > > Very strange. > > -----Original Message----- > From: Ulf Wiger [mailto:ulf.wiger@REDACTED] > Sent: Thursday, April 29, 2010 3:29 AM > To: Evans, Matthew > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] pg2...a warning > > Evans, Matthew wrote: >> >> The problem is that an asynchronous operation, beyond our control, >> can cause pg2:join/2 to be called many times for the same process. >> The result of which is that pg2:get_closest_pid/1 will not be random >> (e.g. process on node 1 gets 5 "joins", and node 2 gets 3 "joins"). >> Or rather it will not be random in how we consider it to be (i.e. we >> only want a process to join a group a single time). > > This made me curious. > > I will admit to not having used pg2, but the other day I was inspired > to explore how to emulate pg2's behaviour using gproc [1]. > > I noted the part in the documentation stating that you can join > several times, but didn't catch the fact that get_members/1 would > include each pid once for each time it has joined. This seems to imply > that joining several times serves an entirely different purpose than > relieving the programmer of the trouble of keeping track of whether > or not it has joined the group before. > > OTOH, the man page doesn't mention this at all, which makes me believe > that it's a bug rather than a feature. It talks about how you should > use pg2:get_members/1 when you want to send a message to all members > of a group. This would be a good place to highlight the fact that > you need to remove duplicates from the list if you want the message > to be sent only once to each member. > > BR, > Ulf W > > [1] Making a pg2-like module on top of gproc is actually quite easy, > but requires the distributed gproc to work, which has not been the > case until now. I am in the process of verifying it, and will > hopefully be able to push a new version very soon. A solid, documented version of gproc would be a most welcome addition to OTP in my opinion. From mevans@REDACTED Thu Apr 29 23:05:00 2010 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 29 Apr 2010 17:05:00 -0400 Subject: [erlang-questions] pg2...a warning In-Reply-To: References: <4BD93552.5090809@erlang-solutions.com> <9B73EB73-8E1E-4BF2-A973-76D5911A3FF7@erlang.geek.nz> Message-ID: One additional thing. Different nodes are reporting different values (i.e. quantity) for pg2:get_members(Group). -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Evans, Matthew Sent: Thursday, April 29, 2010 5:00 PM To: Geoff Cant Cc: Ulf Wiger; erlang-questions@REDACTED Subject: RE: [erlang-questions] pg2...a warning I agree, and was going to reply with almost what you wrote. It seems that this occurs when we reboot a card that is part of an Erlang cluster. There is also the following code: handle_info({nodeup, Node}, S) -> gen_server:cast({?MODULE, Node}, {exchange, node(), all_members()}), {noreply, S}; Correct me if I'm wrong, but in a large cluster, won't 'Node' get this message "number of nodes in cluster (-1)" times? Basically, I'm seeing very strange behavior here. I'll dig a little deeper. I too am tempted to modify pg2 to only allow a process to join once. Matt -----Original Message----- From: Geoff Cant [mailto:nem@REDACTED] Sent: Thursday, April 29, 2010 4:11 PM To: Evans, Matthew Cc: Ulf Wiger; erlang-questions@REDACTED Subject: Re: [erlang-questions] pg2...a warning I've also had some trouble with pg2 recently (the R13B04 version). It seems we have some code that joins groups repeatedly (or some other network condition - nodes parting/joining the cluster?) causes pg2 to think that pids have joined groups lots of times (for 180K values of lots). This causes spectacularly pathological behaviour in a pg2 cluster - you can get to the point where new nodes are no longer able to start pg2 as they run out of ram and abort when exchanging group definitions with other nodes. (pg2 does something like: all_members() -> [[G, group_members(G)] || G <- all_groups()]. group_members(Name) -> [P || [P, N] <- ets:match(pg2_table, {{member, Name, '$1'},'$2'}), _ <- lists:seq(1, N)]. all_groups() -> [N || [N] <- ets:match(pg2_table, {{group,'$1'}})]. when exchanging group membership information. In the pathological case I've run into it means sending [group_name, [ 180000 x Pid1, 180000 x Pid2, ... ] ] at startup. Clearly our code has some bugs as we're joining too often, but this behaviour is just nuts - it'd be cheaper to send our entire ets pg2_table over the wire) I'm pretty sure in our use of pg2 we want the 2nd and subsequent joins to be nops, and it's almost tempting to write 'pg3' just for that. Cheers, -Geoff On 2010-04-29, at 12:23 , Evans, Matthew wrote: > Thanks Ulf, > > Steve told mea bout your gproc work, It looks interesting. > > I actually ran into another pg2 strangeness today on another application. > > This process does a pg2:join in the init function. This is the ONLY place where this occurs. > > I would therefore like to know why pg2:get_members/2 reports two entries for that process? > > Very strange. > > -----Original Message----- > From: Ulf Wiger [mailto:ulf.wiger@REDACTED] > Sent: Thursday, April 29, 2010 3:29 AM > To: Evans, Matthew > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] pg2...a warning > > Evans, Matthew wrote: >> >> The problem is that an asynchronous operation, beyond our control, >> can cause pg2:join/2 to be called many times for the same process. >> The result of which is that pg2:get_closest_pid/1 will not be random >> (e.g. process on node 1 gets 5 "joins", and node 2 gets 3 "joins"). >> Or rather it will not be random in how we consider it to be (i.e. we >> only want a process to join a group a single time). > > This made me curious. > > I will admit to not having used pg2, but the other day I was inspired > to explore how to emulate pg2's behaviour using gproc [1]. > > I noted the part in the documentation stating that you can join > several times, but didn't catch the fact that get_members/1 would > include each pid once for each time it has joined. This seems to imply > that joining several times serves an entirely different purpose than > relieving the programmer of the trouble of keeping track of whether > or not it has joined the group before. > > OTOH, the man page doesn't mention this at all, which makes me believe > that it's a bug rather than a feature. It talks about how you should > use pg2:get_members/1 when you want to send a message to all members > of a group. This would be a good place to highlight the fact that > you need to remove duplicates from the list if you want the message > to be sent only once to each member. > > BR, > Ulf W > > [1] Making a pg2-like module on top of gproc is actually quite easy, > but requires the distributed gproc to work, which has not been the > case until now. I am in the process of verifying it, and will > hopefully be able to push a new version very soon. A solid, documented version of gproc would be a most welcome addition to OTP in my opinion. ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From mevans@REDACTED Fri Apr 30 00:16:34 2010 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 29 Apr 2010 18:16:34 -0400 Subject: [erlang-questions] pg2...a warning In-Reply-To: References: <4BD93552.5090809@erlang-solutions.com> <9B73EB73-8E1E-4BF2-A973-76D5911A3FF7@erlang.geek.nz> Message-ID: Ok, It does appear that processes will join in a very haphazard way. I created new Erlang node. Did a pg2:start() and then net_adm:ping(SomeNode) to join this new node to the rest of the pool of Erlang nodes. I can see on the new node 140 members of that process (when there are only 2), 72 on another node, and 52 on another. Presumably start order is important here. A few things I've noticed: 1) The pg2:init function will do net_kernel:monitor_nodes(). This means that whenever a node starts {nodeup,Node} will be sent to itself causing the pg2 process to 'exchange' it's list of processes and groups to the new pg2 process on the new node. The new node will then create an entry for those processes. Problem is, as I see things quickly, is that all other nodes in the pool of nodes will do the same thing - causing the counters to be increased (ets:update) many times. 2) The init function also sends {new_pg2,Node} to all nodes in the pool. This causes an additional exchange of member data (same logic as in 1, above). init([]) -> Ns = nodes(), net_kernel:monitor_nodes(true), lists:foreach(fun(N) -> {?MODULE, N} ! {new_pg2, node()}, self() ! {nodeup, N} end, Ns), pg2_table = ets:new(pg2_table, [ordered_set, protected, named_table]), {ok, #state{}}. It then sends a {nodeup,Node} to itself causing it to send its member data to other nodes in the group. Is there a possible race condition here whereby it has some member data from other nodes in the pool, and it will then send that data to all the other nodes? This probably needs looking into. Regards Matt -----Original Message----- From: Evans, Matthew Sent: Thursday, April 29, 2010 5:05 PM To: Evans, Matthew; Geoff Cant Cc: Ulf Wiger; erlang-questions@REDACTED Subject: RE: [erlang-questions] pg2...a warning One additional thing. Different nodes are reporting different values (i.e. quantity) for pg2:get_members(Group). -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Evans, Matthew Sent: Thursday, April 29, 2010 5:00 PM To: Geoff Cant Cc: Ulf Wiger; erlang-questions@REDACTED Subject: RE: [erlang-questions] pg2...a warning I agree, and was going to reply with almost what you wrote. It seems that this occurs when we reboot a card that is part of an Erlang cluster. There is also the following code: handle_info({nodeup, Node}, S) -> gen_server:cast({?MODULE, Node}, {exchange, node(), all_members()}), {noreply, S}; Correct me if I'm wrong, but in a large cluster, won't 'Node' get this message "number of nodes in cluster (-1)" times? Basically, I'm seeing very strange behavior here. I'll dig a little deeper. I too am tempted to modify pg2 to only allow a process to join once. Matt -----Original Message----- From: Geoff Cant [mailto:nem@REDACTED] Sent: Thursday, April 29, 2010 4:11 PM To: Evans, Matthew Cc: Ulf Wiger; erlang-questions@REDACTED Subject: Re: [erlang-questions] pg2...a warning I've also had some trouble with pg2 recently (the R13B04 version). It seems we have some code that joins groups repeatedly (or some other network condition - nodes parting/joining the cluster?) causes pg2 to think that pids have joined groups lots of times (for 180K values of lots). This causes spectacularly pathological behaviour in a pg2 cluster - you can get to the point where new nodes are no longer able to start pg2 as they run out of ram and abort when exchanging group definitions with other nodes. (pg2 does something like: all_members() -> [[G, group_members(G)] || G <- all_groups()]. group_members(Name) -> [P || [P, N] <- ets:match(pg2_table, {{member, Name, '$1'},'$2'}), _ <- lists:seq(1, N)]. all_groups() -> [N || [N] <- ets:match(pg2_table, {{group,'$1'}})]. when exchanging group membership information. In the pathological case I've run into it means sending [group_name, [ 180000 x Pid1, 180000 x Pid2, ... ] ] at startup. Clearly our code has some bugs as we're joining too often, but this behaviour is just nuts - it'd be cheaper to send our entire ets pg2_table over the wire) I'm pretty sure in our use of pg2 we want the 2nd and subsequent joins to be nops, and it's almost tempting to write 'pg3' just for that. Cheers, -Geoff On 2010-04-29, at 12:23 , Evans, Matthew wrote: > Thanks Ulf, > > Steve told mea bout your gproc work, It looks interesting. > > I actually ran into another pg2 strangeness today on another application. > > This process does a pg2:join in the init function. This is the ONLY place where this occurs. > > I would therefore like to know why pg2:get_members/2 reports two entries for that process? > > Very strange. > > -----Original Message----- > From: Ulf Wiger [mailto:ulf.wiger@REDACTED] > Sent: Thursday, April 29, 2010 3:29 AM > To: Evans, Matthew > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] pg2...a warning > > Evans, Matthew wrote: >> >> The problem is that an asynchronous operation, beyond our control, >> can cause pg2:join/2 to be called many times for the same process. >> The result of which is that pg2:get_closest_pid/1 will not be random >> (e.g. process on node 1 gets 5 "joins", and node 2 gets 3 "joins"). >> Or rather it will not be random in how we consider it to be (i.e. we >> only want a process to join a group a single time). > > This made me curious. > > I will admit to not having used pg2, but the other day I was inspired > to explore how to emulate pg2's behaviour using gproc [1]. > > I noted the part in the documentation stating that you can join > several times, but didn't catch the fact that get_members/1 would > include each pid once for each time it has joined. This seems to imply > that joining several times serves an entirely different purpose than > relieving the programmer of the trouble of keeping track of whether > or not it has joined the group before. > > OTOH, the man page doesn't mention this at all, which makes me believe > that it's a bug rather than a feature. It talks about how you should > use pg2:get_members/1 when you want to send a message to all members > of a group. This would be a good place to highlight the fact that > you need to remove duplicates from the list if you want the message > to be sent only once to each member. > > BR, > Ulf W > > [1] Making a pg2-like module on top of gproc is actually quite easy, > but requires the distributed gproc to work, which has not been the > case until now. I am in the process of verifying it, and will > hopefully be able to push a new version very soon. A solid, documented version of gproc would be a most welcome addition to OTP in my opinion. ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From dmurray@REDACTED Thu Apr 29 21:09:32 2010 From: dmurray@REDACTED (David N Murray) Date: Thu, 29 Apr 2010 15:09:32 -0400 (EDT) Subject: [erlang-questions] emacs: moving to shell after compile In-Reply-To: <4BD9D7BC.60807@gmail.com> References: <4BD9D7BC.60807@gmail.com> Message-ID: you could just create a small function that does both in your .emacs file: (defun erlang-compile-and-jump () (interactive) (erlang-compile) (other-window)) (global-set-key [f12] 'erlang-compile-and-jump) or bind the key just in the erlang-mode rather than a global mode (I've never bothered to do that with any mode so I don't know how). That way, it doesn't break when you upgrade erlang. hth, Dave On Apr 29, Doug Edmunds (gmail) scribed: > I have erlang-mode in .emacs: > > (setq load-path (cons "C:/home/erl5.7.5/lib/tools-2.6.5.1/emacs" > load-path)) > (setq erlang-root-dir "C:/home/erl5.7.5") > (setq exec-path (cons "C:/home/erl5.7.5/bin" exec-path)) > (require 'erlang-start) > > /lib/tools-2.6.5.1/emacs has: > erlang.el > erlang-eunit.el > erlang-skels.el > erlang-start.el > > I want C-C C-K to jump to the shell after compiling. > > I included (global-set-key [f12] 'other-window) so I can do > this manually, but I'd like it to be automatic. > > What is the lisp code to use and which file should it go in? > > --dae > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ulf.wiger@REDACTED Fri Apr 30 00:21:18 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 30 Apr 2010 00:21:18 +0200 Subject: [erlang-questions] pg2...a warning In-Reply-To: References: <4BD93552.5090809@erlang-solutions.com> Message-ID: <4BDA065E.5040606@erlang-solutions.com> Evans, Matthew wrote: > Thanks Ulf, > > Steve told mea bout your gproc work, It looks interesting. > > I actually ran into another pg2 strangeness today on another application. > [...] I have now shelved my idea of emulating pg2 on top of gproc. It seems as if most people are attracted to the basic idea of pg2 much more than the exact semantics and the implementation. I pushed a new version of gproc that works well in a distributed setting. I will qualify that statement by saying that there is not yet any test suite for the global registration part (we use QuickCheck for the local part, and that still works). http://github.com/uwiger/gproc/ It requires my own version of the hanssv+serge/gen_leader.erl, as I found a bug in how gen_leader handled DOWN messages. http://github.com/uwiger/gen_leader_revival/commit/e682e979ebade10b61d35f37b85e2f7b6a953af2 It would be great to have some more people take it for a spin and report any bugs. A simple process group implementation on top of gproc could look like this (warning: mailing-list-quality code, not compiled): -module(gproc_pg). -export([join/1, leave/1, get_members/1]). join(Group) -> Key = key(Group), try gproc:reg(Key) % fails if already registered catch error:badarg -> gproc:get_value(Key) % fails if not already registered end, true. leave(Group) -> gproc:unreg(key(Group)). get_members(Group) -> gproc:lookup_pids(key(Group)). key(Group) -> {p,g,{?MODULE,Group}}. % a global property named {?MODULE,Group} One problem with the above is that there is no check to see if you misspelled the group name and ended up alone in a newly created group. If your application uses hard-coded group names, this is fairly easily addressed. Otherwise, some persistent table of valid group names could be used to check against. As stated before, gen_leader is a bit clumsy in environments where nodes are dynamically added and removed. Changing the participants of a gen_leader cluster is fairly difficult. The gen_leader_revival project tries to merge a number of existing modifications to gen_leader to make it more flexible. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From comptekki@REDACTED Fri Apr 30 00:56:43 2010 From: comptekki@REDACTED (Wes James) Date: Thu, 29 Apr 2010 16:56:43 -0600 Subject: repeat function every hour Message-ID: I see that there is a timer module. Is there something where I can start an erl program and it will run a certain function at some given time on a given interval - kind of like cron? thx, -wes From mevans@REDACTED Fri Apr 30 01:01:22 2010 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 29 Apr 2010 19:01:22 -0400 Subject: [erlang-questions] repeat function every hour In-Reply-To: References: Message-ID: Yes. timer:apply_after(Time,M,F,A). For example, to execute test_mod:test_fun(hello,1,2) every hour: timer:apply_after(3600000, test_mod, test_fun, [hello,1,2]). Of course, your function "test_fun" should call timer:apply_afer(Time,M,F,A) again. Matt -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Wes James Sent: Thursday, April 29, 2010 6:57 PM To: erlang-questions@REDACTED Subject: [erlang-questions] repeat function every hour I see that there is a timer module. Is there something where I can start an erl program and it will run a certain function at some given time on a given interval - kind of like cron? thx, -wes ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From yrashk@REDACTED Fri Apr 30 01:38:36 2010 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Thu, 29 Apr 2010 16:38:36 -0700 Subject: [erlang-questions] repeat function every hour In-Reply-To: References: Message-ID: <81C68517-F999-425C-9A32-13532A1849EC@gmail.com> From erl -man timer : apply_interval(Time, Module, Function, Arguments) -> {ok, TRef} | {error, Reason} Types Time = integer() in milliseconds Module = Function = atom() Arguments = [term()] Evaluates apply(Module, Function, Arguments) repeatedly at intervals of Time. Returns {ok, TRef}, or {error, Reason}. On 2010-04-29, at 3:56 PM, Wes James wrote: > I see that there is a timer module. Is there something where I can > start an erl program and it will run a certain function at some given > time on a given interval - kind of like cron? > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From max.lapshin@REDACTED Fri Apr 30 07:07:07 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 30 Apr 2010 09:07:07 +0400 Subject: [erlang-questions] repeat function every hour In-Reply-To: <81C68517-F999-425C-9A32-13532A1849EC@gmail.com> References: <81C68517-F999-425C-9A32-13532A1849EC@gmail.com> Message-ID: I had to make my own ticker: it runs erlang:now() and calculates how many milliseconds left till the end of current minute: minute_end({Mega,Sec,Micro}) -> Number = (Mega * 1000000 + Sec) div 60, Seconds = (Mega * 1000000 + Sec) rem 60, Milli = Seconds * 1000 + (Micro div 1000), {?INTERVAL - Milli, Number}. From ok@REDACTED Fri Apr 30 07:07:37 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 30 Apr 2010 17:07:37 +1200 Subject: [erlang-questions] parsing text In-Reply-To: References: Message-ID: <7F3F254C-517D-4D20-895A-AEDFAEBA5099@cs.otago.ac.nz> On Apr 30, 2010, at 6:39 AM, Wes James wrote: > I have a function grabbing a page and I'm pulling text out of the > result. I can get the line: > > lists:nth(424,B). > <<"Page Counter4880">> > > > but 4880 will eventually get to 10000, etc. It's not clear exactly how much else about the data will vary. My take on this is that you want the stuff between and . I'm actually typing this on a machine that doesn't have Erlang installed, so I'm going to do this from first principles. % begins_with(Prefix, String) :: (list(T), list(T)) -> list(T) | no begins_with([X|Prefix], [X|String]) -> begins_with(Prefix, String); begins_with([_|_], _) -> no; begins_with([], String) -> String. % decat(B, ABC) :: (list(T), list(T)) -> {list(T),list(T)} | no % decat(B, ABC) -> {A,B}, anything else -> no. decat(Infix, String) when Infix =/= [] -> decat_loop(Infix, String, []). decat_loop(_, [], _) -> no; decat_loop(Infix, String = [Head|Tail], Rev) -> case begins_with(Infix, String) of After when is_list(After) -> {lists:reverse(Rev),After} ; no -> decat_loop(Infix, Tail, [Head|Rev]) end. This is all we need now. Binary = <<"Page Counter4880">>, String = binary_to_list(Binary), {_, After_TD} = decat("", String), {Wanted, _} = decat("", After_TD), list_to_integer(Wanted) The next step for improving this would be to work directly on the binary instead of turning it into a list. It's not particularly hard; binary_decat_loop is closely analogous to C's strstr(). Or of course you could use regular expressions. If I were doing this in AWK I'd just do S = "Page Counter4880" if (S ~ /[0-9]*<\/TD>/) { N = substr(S, RSTART+4, RLENGTH-9)+0 ... } Erlang has a couple of regular expression packages, including one that works on strings. Of course, the question is *how much* variation in the input can there be. XML tags may have any amount of white space except inside identifiers and strings, so if "Page Counter4880" is legal input, then "Page Counter4880" is also legal input with *exactly* the same meaning, and an XML-aware application (at any rate a structure-controlled one) should not treat them at all differently. In that case you want an XML parser, and then to match by tree position. (There's a reason why XPath exists; I'm not sure there's a reason why it's so horrible.) From g9414002.pccu.edu.tw@REDACTED Fri Apr 30 07:35:59 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Fri, 30 Apr 2010 13:35:59 +0800 Subject: [erlang-questions] Thinking in Erlang In-Reply-To: References: Message-ID: On Fri, Apr 30, 2010 at 12:10 AM, David N Murray wrote: > On Apr 29, ??^C?^@^@??? (Yau-Hsien Huang) scribed: > > > On Wed, Apr 28, 2010 at 10:29 AM, David N Murray >wrote: > > > > {#Fun,[c]} > > 15> > > > > Right? > > > > No, the lisp returns the queue (list). > > * (defparameter q (make-fqueue 3)) > Q > * (funcall q 1) > (1) > * (funcall q 2) > (2 1) > * (funcall q 3) > (3 2 1) > * (funcall q 4) > (4 3 2) > * > > Dave > Interesting, mutable variables. Two important features of Erlang are process-communication modelling and immutable objects. Thus, one of same implementations can be: ---------------------------------------------------------------- %% In module "test" % A client queue functions. enq(Fq, E) -> Fq ! {self(), E}, receive {Q,O} -> {Q,O}; Q -> Q end. fqueue(N) -> spawn(test, q, [N, []]). % Function for maintaining a queue while manipulating a queue function. q(N, Q) -> receive {Pid, break} -> Pid ! Q; {Pid, E} -> Q1 = [E|Q], if length(Q1) > N -> Pid ! {lists:sublist(Q1, 1, N), lists:sublist(Q1, N+1, N+1)}, q(N, lists:sublist(Q1, 1, N)); true -> Pid ! {Q1, []}, q(N, Q1) end end. -------------------------------------------------- Output: 8> Q_ = test:fqueue(2). <0.77.0> 9> test:enq(Q_,a). {[a],[]} 10> test:enq(Q_,b). {[b,a],[]} 11> test:enq(Q_,c). {[c,b],[a]} 12> From g9414002.pccu.edu.tw@REDACTED Fri Apr 30 09:07:23 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Fri, 30 Apr 2010 15:07:23 +0800 Subject: [erlang-questions] Thinking in Erlang In-Reply-To: References: Message-ID: 2010/4/30 ??? (Yau-Hsien Huang) > q(N, Q) -> > receive > {Pid, break} -> > Pid ! Q; > {Pid, E} -> > Q1 = [E|Q], > if length(Q1) > N -> > Pid ! {lists:sublist(Q1, 1, N), > lists:sublist(Q1, N+1, N+1)}, > q(N, lists:sublist(Q1, 1, N)); > true -> > Pid ! {Q1, []}, > q(N, Q1) > end > end. > A question. In q/2, it takes a list of length N and makes another one of length N+1. Then it sinks into a recursive call with at most N elements of the list Q1. Does it use redundant data for recursion? Or there's some smart way in the abstract machine to trim redundancy? From mats.westin@REDACTED Fri Apr 30 09:54:37 2010 From: mats.westin@REDACTED (Mats) Date: Fri, 30 Apr 2010 09:54:37 +0200 Subject: FYI, Erlang reference in conference about the economy Message-ID: This conference[1] at the University of Cambridge "Network and Systemic Risks" is using Erlang to explain how we may save the world.. I love this quote[2]: "..the telephony network is arguably at the moment the largest machine that the human race has constructed, arguably because the Internet is bigger now with the same issues.." What do you think, interesting? :) Mats Westin [1] http://www.youtube.com/watch?v=bz6z3BEI-1E#t=3m45s [2] http://www.youtube.com/watch?v=bz6z3BEI-1E#t=6m10s -> http://se.linkedin.com/in/matswestin From comptekki@REDACTED Fri Apr 30 17:04:02 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 30 Apr 2010 09:04:02 -0600 Subject: [erlang-questions] parsing text In-Reply-To: <7F3F254C-517D-4D20-895A-AEDFAEBA5099@cs.otago.ac.nz> References: <7F3F254C-517D-4D20-895A-AEDFAEBA5099@cs.otago.ac.nz> Message-ID: On Thu, Apr 29, 2010 at 11:07 PM, Richard O'Keefe wrote: > > On Apr 30, 2010, at 6:39 AM, Wes James wrote: > >> I have a function grabbing a page and I'm pulling text out of the >> result. ?I can get the line: >> >> lists:nth(424,B). >> <<"Page Counter4880">> >> >> >> but 4880 will eventually get to 10000, etc. > > It's not clear exactly how much else about the data will > vary. ?My take on this is that you want the stuff between > and . Richard, Thanks for your input on this. I tested it and it worked. I messed around with xmerl_scan:string, but "Page Counter4880" doesn't seem to be well formed xml - I kept getting errors. xmerl_scan:string("" ++ 11> "x" ++ 11> "x" ++ 11> "y" ++ 11> ""). works, but xmerl_scan:string("Page Counter4880"). 2711- fatal: {unknown_entity_ref,nbsp} 2621- fatal: error_scanning_entity_ref ** exception exit: {fatal,{error_scanning_entity_ref,{file,file_name_unknown}, {line,1}, {col,10}}} in function xmerl_scan:fatal/2 in call from xmerl_scan:scan_content/11 in call from xmerl_scan:scan_element/12 in call from xmerl_scan:scan_document/2 in call from xmerl_scan:string/2 not.... -wes From anthonym@REDACTED Fri Apr 30 18:51:15 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Fri, 30 Apr 2010 09:51:15 -0700 Subject: [erlang-questions] parsing text In-Reply-To: References: <7F3F254C-517D-4D20-895A-AEDFAEBA5099@cs.otago.ac.nz> Message-ID: <20100430165115.GA83987@alumni.caltech.edu> On Fri, Apr 30, 2010 at 09:04:02AM -0600, Wes James wrote: > On Thu, Apr 29, 2010 at 11:07 PM, Richard O'Keefe wrote: > > > > On Apr 30, 2010, at 6:39 AM, Wes James wrote: > > > >> I have a function grabbing a page and I'm pulling text out of the > >> result. ?I can get the line: > >> > >> lists:nth(424,B). > >> <<"Page Counter4880">> > >> > >> > >> but 4880 will eventually get to 10000, etc. > > > > It's not clear exactly how much else about the data will > > vary. ?My take on this is that you want the stuff between > > and . > > > > > Richard, > > Thanks for your input on this. I tested it and it worked. I messed > around with xmerl_scan:string, but > "Page Counter4880" doesn't seem to be > well formed xml - I kept getting errors. > > xmerl_scan:string("" ++ > 11> "x" ++ > 11> "x" ++ > 11> "y" ++ > 11> ""). > > works, but > > xmerl_scan:string("Page Counter4880"). > 2711- fatal: {unknown_entity_ref,nbsp} > 2621- fatal: error_scanning_entity_ref > ** exception exit: {fatal,{error_scanning_entity_ref,{file,file_name_unknown}, > {line,1}, > {col,10}}} > in function xmerl_scan:fatal/2 > in call from xmerl_scan:scan_content/11 > in call from xmerl_scan:scan_element/12 > in call from xmerl_scan:scan_document/2 > in call from xmerl_scan:string/2 > > not.... Your string contains an HTML entity   but that is not a valid xml entity (there are only 5 of those http://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references) So if you tried 1> xmerl_scan:string("Page&nbsp;Counter4880"). {{xmlElement,'B','B',[], {xmlNamespace,[],[]}, [],1,[], [{xmlText,[{'B',1}],1,[],"Page",text}, {xmlText,[{'B',1}],2,[]," Counter",text}], [],"/tmp",undeclared}, "4880"} You can see it does better, but still not what you want as it can only parse part of the structure (only ... can be parsed, then you hit an end element without a start and the parsing stops). Your best bet might be to attempt to parse the entire file and not just part of it. But you'd still need a way to escape html entities so they can be parsed by an xml parser. -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From comptekki@REDACTED Fri Apr 30 19:10:43 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 30 Apr 2010 11:10:43 -0600 Subject: [erlang-questions] parsing text In-Reply-To: <20100430165115.GA83987@alumni.caltech.edu> References: <7F3F254C-517D-4D20-895A-AEDFAEBA5099@cs.otago.ac.nz> <20100430165115.GA83987@alumni.caltech.edu> Message-ID: On Fri, Apr 30, 2010 at 10:51 AM, Anthony Molinaro wrote: > > Your string contains an HTML entity   but that is not a valid xml > entity (there are only 5 of those > http://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references) > > So if you tried > > 1> xmerl_scan:string("Page&nbsp;Counter4880"). > {{xmlElement,'B','B',[], > ? ? ? ? ? ? {xmlNamespace,[],[]}, > ? ? ? ? ? ? [],1,[], > ? ? ? ? ? ? [{xmlText,[{'B',1}],1,[],"Page",text}, > ? ? ? ? ? ? ?{xmlText,[{'B',1}],2,[]," Counter",text}], > ? ? ? ? ? ? [],"/tmp",undeclared}, > ?"4880"} > > You can see it does better, but still not what you want as it can only parse > part of the structure (only ... can be parsed, then you hit an end > element without a start and the parsing stops). > > Your best bet might be to attempt to parse the entire file and not just part > of it. ?But you'd still need a way to escape html entities so they can be > parsed by an xml parser. > > -Anthony Anthony, Thx for this. This give me an idea on this. Since the line will mostly be the same: All I need to really do is chop off the chars before the 4 and then chop off the last 10 chars that would give me the result I need. Something like: S=string:substr("Page Counter4880", 34). "4880" list_to_integer(string:substr(S, 1, string:len(S)-10)). 4880 Thx all! -wes From maruthavanan_s@REDACTED Fri Apr 30 19:21:33 2010 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Fri, 30 Apr 2010 13:21:33 -0400 Subject: Increasing proceses Message-ID: Hi, I have a situation where I need to get response from URL and save it when a event occurs in process. To make the other requests non blocking and to hit the URL immediately when ever a new request comes, I spawn a process that invoked HTTP, this keeps on increasing the process and says system limit is reached. Is it possible to avoid such errors? Thanks, Marutha From jeraymond@REDACTED Fri Apr 30 19:51:22 2010 From: jeraymond@REDACTED (Jeremy Raymond) Date: Fri, 30 Apr 2010 13:51:22 -0400 Subject: [erlang-questions] Increasing proceses In-Reply-To: References: Message-ID: Are you running out of Erlang processes? By default the vm has a max of 32768 processes. You can increase this when starting Erlang via the +P parameter. E.g.) erl +P 100000. Reference: http://www3.erlang.org/doc/man/erl.html -- Jeremy Raymond On Fri, Apr 30, 2010 at 1:21 PM, maruthavanan s wrote: > > Hi, > > > > I have a situation where I need to get response from URL and save it when a > event occurs in process. > > > > To make the other requests non blocking and to hit the URL immediately when > ever a new request comes, I spawn a process that invoked HTTP, this keeps on > increasing the process and says system limit is reached. > > > > Is it possible to avoid such errors? > > > > Thanks, > > Marutha > From comptekki@REDACTED Fri Apr 30 19:57:02 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 30 Apr 2010 11:57:02 -0600 Subject: calling timer_interval from module Message-ID: If I have: -module('gpc'). -export([start/0]). start() -> %{ok,R}= timer:apply_interval(3000, gpc, getc, []). getc() -> "yep". when I run: gpc:start(). I get the error: =ERROR REPORT==== 30-Apr-2010::11:39:15 === Error in process <0.70.0> with exit value: {undef,[{gpc,getc,[]}]} If I change -export to: -export([start/0, getc/0]). If I run: {ok,R}=timer:apply_interval(3000, gpc, getc, []). I get {ok,{interval,#Ref<0.0.0.461>}} but nothing ever happens. If I do the same thing with: -module(tst). -export([tst_fun/0]). tst_fun() -> io:format("~n~w~n", [erlang:localtime()]). tst:tst_fun(). It works?? Any idea why the first doesn't work? The difference is running timer:apply_interval from the cli vs from inside a module, it looks like. thx, -wes From maruthavanan_s@REDACTED Fri Apr 30 20:21:53 2010 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Fri, 30 Apr 2010 14:21:53 -0400 Subject: [erlang-questions] Increasing proceses In-Reply-To: References: , Message-ID: HI, But I will still fall under a situation when number of process exceeds that limit. Thanks, Marutha > From: jeraymond@REDACTED > Date: Fri, 30 Apr 2010 13:51:22 -0400 > To: maruthavanan_s@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Increasing proceses > > Are you running out of Erlang processes? By default the vm has a max of 32768 > processes. You can increase this when starting Erlang via the +P parameter. > E.g.) erl +P 100000. Reference: http://www3.erlang.org/doc/man/erl.html > -- > Jeremy Raymond > > > On Fri, Apr 30, 2010 at 1:21 PM, maruthavanan s > wrote: > > > > > Hi, > > > > > > > > I have a situation where I need to get response from URL and save it when a > > event occurs in process. > > > > > > > > To make the other requests non blocking and to hit the URL immediately when > > ever a new request comes, I spawn a process that invoked HTTP, this keeps on > > increasing the process and says system limit is reached. > > > > > > > > Is it possible to avoid such errors? > > > > > > > > Thanks, > > > > Marutha > > From comptekki@REDACTED Fri Apr 30 20:36:42 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 30 Apr 2010 12:36:42 -0600 Subject: calling timer_interval from module In-Reply-To: References: Message-ID: On Fri, Apr 30, 2010 at 11:57 AM, Wes James wrote: > Any idea why the first doesn't work? > > The difference is running timer:apply_interval from the cli vs from > inside a module, it looks like. > > thx, > > -wes > I see now. Where there seemed to be nothing happening - it actually was. I put in an in:format on the final item and it now is repeating. I guess there is a difference on cli vs. output called from a module when the function is in the module?? -wes From kiszl@REDACTED Fri Apr 30 20:38:01 2010 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Fri, 30 Apr 2010 20:38:01 +0200 Subject: [erlang-questions] calling timer_interval from module In-Reply-To: References: Message-ID: <4BDB2389.5080200@tmit.bme.hu> On 4/30/2010 7:57 PM, Wes James wrote: > If I have: > > -module('gpc'). > > -export([start/0]). > > start() -> > %{ok,R}= > timer:apply_interval(3000, gpc, getc, []). > > getc() -> "yep". > > when I run: > > gpc:start(). > > I get the error: > > =ERROR REPORT==== 30-Apr-2010::11:39:15 === > Error in process<0.70.0> with exit value: {undef,[{gpc,getc,[]}]} > > If I change -export to: > > -export([start/0, getc/0]). > > If I run: > > {ok,R}=timer:apply_interval(3000, gpc, getc, []). > > I get > > {ok,{interval,#Ref<0.0.0.461>}} > > but nothing ever happens. > > If I do the same thing with: > > > -module(tst). > > -export([tst_fun/0]). > > tst_fun() -> > io:format("~n~w~n", [erlang:localtime()]). > > > tst:tst_fun(). > > > It works?? > > Any idea why the first doesn't work? > > The difference is running timer:apply_interval from the cli vs from > inside a module, it looks like. > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > If getc is exported it is being called every three seconds with your code. It just does not produce any visible output (returns a string, but how would you see that?). Change your code to getc() -> io:format("yep~n"). to see stuff happen. Regards, Zoltan. From comptekki@REDACTED Fri Apr 30 20:43:39 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 30 Apr 2010 12:43:39 -0600 Subject: [erlang-questions] calling timer_interval from module In-Reply-To: <4BDB2389.5080200@tmit.bme.hu> References: <4BDB2389.5080200@tmit.bme.hu> Message-ID: On Fri, Apr 30, 2010 at 12:38 PM, Zoltan Lajos Kis wrote: >> >> > > If getc is exported it is being called every three seconds with your code. > It just does not produce any visible output (returns a string, but how would > you see that?). Change your code to getc() -> io:format("yep~n"). to see > stuff happen. > > Regards, > Zoltan. > Zoltan, You are right - I just sent a note to the list that I had just figured that out. It seems output happens differently depending on where it is called on the cli. thx, -wes From comptekki@REDACTED Fri Apr 30 20:54:08 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 30 Apr 2010 12:54:08 -0600 Subject: Autoreply: Re: [erlang-questions] calling timer_in In-Reply-To: <15fadd01cae895$87b92770$0165010a@mail2world.com> References: <15fadd01cae895$87b92770$0165010a@mail2world.com> Message-ID: On Fri, Apr 30, 2010 at 12:47 PM, Anders Leven wrote: > Hi! > > I am on vacation, if it is urgent send me an SMS. > > R/ Anders > > _______________________________________________________________ > > _______________________________________________________________ > Every time I send a message to the list today, I get this persons autoreply. What happens on this list when this happens - list manager boots them or ignore? just wondering - this is a great list! thx, -wes From comptekki@REDACTED Fri Apr 30 21:17:51 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 30 Apr 2010 13:17:51 -0600 Subject: page counter - thx for everyones help Message-ID: Here is the code so far: -module('gpc'). -export([start/0, start/1, getc/0]). start() -> ibrowse:start(), timer:apply_interval(3000, gpc, getc, []). start(Interval) -> ibrowse:start(), timer:apply_interval(Interval*1000, gpc, getc, []). getc() -> S = string:substr( binary_to_list( lists:nth( 424, re:replace( erlang:element( 4, ibrowse:send_req("http://192.168.1.1/printer/configu.html?autoref=0&weblang=0", [], get) ), "\\n", "", [global] ) ) ), 34), S2=list_to_integer(string:substr(S, 1, string:len(S)-10)), io:format("~w~n", [S2]). So this by default gives me a page count every 3 seconds or I can put in gpc:start(10) for 10 second intervals. What I would like to do is take daily page counts and stick them some where. I could make a file on the system with the date as the name (20100430.txt) and just write the value there or I could write the value to mnesia and keep track via the data base. I would be doing a current count - yesterday count to get a total count for just each day. I'm doing this so I can keep track of what the students in a university computer lab are printing each day. If you have any suggestions, please drop them in this list :) I'll be experimenting anyway since I've yet to read or write to a file with erlang on a disc or gone beyond just reading what mnesia can do via web page info. thx, -wes From jeraymond@REDACTED Fri Apr 30 21:19:03 2010 From: jeraymond@REDACTED (Jeremy Raymond) Date: Fri, 30 Apr 2010 15:19:03 -0400 Subject: [erlang-questions] Increasing proceses In-Reply-To: References: Message-ID: Don't the processes end once you have retrieved the item from the URL? Perhaps post your code so we can see in more detail what you're trying to accomplish. -- Jeremy Raymond On Fri, Apr 30, 2010 at 2:21 PM, maruthavanan s wrote: > HI, > > But I will still fall under a situation when number of process exceeds that > limit. > > Thanks, > Marutha > > > From: jeraymond@REDACTED > > Date: Fri, 30 Apr 2010 13:51:22 -0400 > > To: maruthavanan_s@REDACTED > > CC: erlang-questions@REDACTED > > Subject: Re: [erlang-questions] Increasing proceses > > > > > Are you running out of Erlang processes? By default the vm has a max of > 32768 > > processes. You can increase this when starting Erlang via the +P > parameter. > > E.g.) erl +P 100000. Reference: http://www3.erlang.org/doc/man/erl.html > > -- > > Jeremy Raymond > > > > > > On Fri, Apr 30, 2010 at 1:21 PM, maruthavanan s > > wrote: > > > > > > > > Hi, > > > > > > > > > > > > I have a situation where I need to get response from URL and save it > when a > > > event occurs in process. > > > > > > > > > > > > To make the other requests non blocking and to hit the URL immediately > when > > > ever a new request comes, I spawn a process that invoked HTTP, this > keeps on > > > increasing the process and says system limit is reached. > > > > > > > > > > > > Is it possible to avoid such errors? > > > > > > > > > > > > Thanks, > > > > > > Marutha > > > > From colm.dougan@REDACTED Fri Apr 30 21:46:26 2010 From: colm.dougan@REDACTED (Colm Dougan) Date: Fri, 30 Apr 2010 12:46:26 -0700 Subject: [erlang-questions] parsing text In-Reply-To: References: Message-ID: If you already found the offset of the start of the number then you can use string:to_inetger (or more generally io_lib:fread) to extract the number, regardless of it being variable length. Eshell V5.7.5 (abort with ^G) 1> string:to_integer("4880"). {4880,""} Eshell V5.7.5 (abort with ^G) 1> io_lib:fread("~d", "4880"). {ok,[4880],[]} Colm On Thu, Apr 29, 2010 at 11:39 AM, Wes James wrote: > I have a function grabbing a page and I'm pulling text out of the > result. ?I can get the line: > > lists:nth(424,B). > <<"Page Counter4880">> > > > but 4880 will eventually get to 10000, etc. > > I can do > > string:substr(binary_to_list(lists:nth(424,B)), 34, 4). > > to get the 4880 out, but to do a check for > 9999 would I grab > position 34 + the rest of the text then test positions after that for > int? > > or maybe something like this? > > > A=string:substr(binary_to_list(lists:nth(424,B)), 34, 6) > B=string:substr(binary_to_list(lists:nth(424,B)), 34, 5) > C=string:substr(binary_to_list(lists:nth(424,B)), 34, 4) > case > ? ?is_integer(A) -> A; > ? ?is_integer(B) -> B; > ? ?is_integer(C) -> C; > end. > > > Hmm - A, B, and C will be strings though so is_integer won't work. ?If > I do list_to_integer(A), I guess I can write something like this: > > try list_to_integer(A) > catch > ? ?throw:Term -> Term; > ? ?exit:Reason -> {'EXIT',Reason}; > ? ?error:Reason -> {'EXIT',{Reason,erlang:get_stacktrace()}} > end. > > > and so forth.... > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From comptekki@REDACTED Fri Apr 30 21:49:07 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 30 Apr 2010 13:49:07 -0600 Subject: [erlang-questions] parsing text In-Reply-To: References: Message-ID: On Fri, Apr 30, 2010 at 1:46 PM, Colm Dougan wrote: > If you already found the offset of the start of the number then you > can use string:to_inetger (or more generally io_lib:fread) to extract > the number, regardless of it being variable length. > > Eshell V5.7.5 ?(abort with ^G) > 1> string:to_integer("4880"). > {4880,""} > > Eshell V5.7.5 ?(abort with ^G) > 1> io_lib:fread("~d", "4880"). > {ok,[4880],[]} > > Colm Hey that's cool! Thx Colm! -wes