From antoine.koener@REDACTED Sat Dec 1 00:15:51 2012 From: antoine.koener@REDACTED (Antoine Koener) Date: Sat, 1 Dec 2012 00:15:51 +0100 Subject: [erlang-questions] Tcp repair mode Message-ID: <4435F7A7-82EC-4EAB-AF88-2072E68F35FC@gmail.com> Hi everyone ! This feature would be "nice" for handling failovers :-) lwn.net/Articles/495304 A remote node may handle, as a reloaded module an existing connection ! -- Was on the go -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Sun Dec 2 03:27:36 2012 From: robert.virding@REDACTED (Robert Virding) Date: Sun, 2 Dec 2012 02:27:36 +0000 (GMT) Subject: [erlang-questions] Is using gen_server behaviour appropriate for his usecase? In-Reply-To: Message-ID: <1855257212.251860.1354415256937.JavaMail.root@erlang-solutions.com> Sorry, I don't agree with you here. While using OTP is generally a Good Thing, using OTP because you know WHY you need to use use it is a Much Much Much Better Thing! You see cases of where people have blindly used gen_servers then struggle and produce convoluted code to get around the very features which gen_servers provide. So I would say that you should know what the basic Erlang provides and how you use it, know what OTP provides and how you use it and choose that which is closest to what you need. I am not saying that you shouldn't use OTP, I am saying you should know why you are using OTP. Actually it is not very difficult to roll your own processes which fit into OTP supervision trees. It is done using the proc_lib and sys modules and it describe quite well here http://www.erlang.org/doc/design_principles/spec_proc.html . Robert ----- Original Message ----- > From: "Garrett Smith" > To: "Vineet Naik" > Cc: erlang-questions@REDACTED > Sent: Friday, 30 November, 2012 2:48:54 PM > Subject: Re: [erlang-questions] Is using gen_server behaviour appropriate for his usecase? > > Roman is spot on correct. > > There's a school of thought that says new developers should learn > core > Erlang and master it before tackling OTP. IMHO this is wrong. OTP > *is* > core Erlang at this point.[1] > > Just use OTP, right away. If the many moving parts of the standard > OTP > behaviors seems like a steep learning curve, take a look at e2: > > http://e2project.org > > e2 is a veneer on top of OTP that, at least in my opinion (I wrote it > for this reason :) makes canonical development a lot easier. You > might > start by reading/following the tutorial: > > http://e2project.org/tutorial.html > > The tutorial has more to do with general application development in > Erlang than it does in with the specifics of the e2 library. > > Garrett > > [1] There's a long history of Erlang that separates the core from > OTP. > I understand how books, tutorials, teaching philosophies, and common > wisdom would treat OTP as an advanced topic. But as a relative > newcomer to Erlang, not having any historical context, etc. I found > this treatment unhelpful in my own learning. IMO "gen_server" (e2 > calls these "services" and "tasks" depending on the scenario) is a > starting point, after one has learned the basics of the language > itself. > > On Fri, Nov 30, 2012 at 2:36 AM, Roman Gafiyatullin > wrote: > > Hi, > > > > In most of the cases one shall not use "simple loop functions": raw > > process > > cannot participate properly in the supervision tree. > > > > If you need some new tricky specific model of behaving - better > > implement it > > over gen_server. > > For instance supervisor is a gen_server :) > > > > If you need something more low level - use 'gen' module (see how > > gen_fsm is > > implemented). > > > > -- > > Regards > > RG ( +375 33 602 5080, UTC+3 ) > > > > On Friday, November 30, 2012 at 11:25 am, Vineet Naik wrote: > > > > Hello, > > > > I am an Erlang newbie and trying to write a simple chat bot as an > > XMPP external component. I am using exmpp library and following > > along these tutorials[1] > > > > So far I have one module `bot_server` that uses the gen_server > > interface. Inside it's `handle_info` callback, incoming messages > > from various client will be received. To handle and reply to the > > these messages, I am thinking of spawning a "bot" process per > > client. It will stay alive as long as the client is available > > ie. when the client sends "unavailable" presence, it will die. I > > also need to keep a list of all the alive bot processes in the > > bot_server's state. > > > > My question is, would it be appropriate to implement the bot as a > > gen_server too considering that it needs to handle two calls, one > > for handling incoming message (asynchronous) and second for > > killing itself (synchronous)? > > > > In general, when should one use gen_server and when should > > one write a simple loop function? > > > > [1] exmpp tutorials: > > http://blog.process-one.net/scalable_xmpp_bots_with_erlang_and_exmpp_part_i/ > > > > Thanks, > > Vineet > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ali.sabil@REDACTED Sun Dec 2 18:06:22 2012 From: ali.sabil@REDACTED (Ali Sabil) Date: Sun, 2 Dec 2012 18:06:22 +0100 Subject: [erlang-questions] Is using gen_server behaviour appropriate for his usecase? In-Reply-To: <1855257212.251860.1354415256937.JavaMail.root@erlang-solutions.com> References: <1855257212.251860.1354415256937.JavaMail.root@erlang-solutions.com> Message-ID: Hi all, In case this would be useful, sometimes ago I put together a behavior module for implementing generic processes that fit into the OTP supervision trees. I didn't have much time to document it yet, but it's very stable and has been used in production. You can find the code here: https://github.com/asabil/gen_process On Sun, Dec 2, 2012 at 3:27 AM, Robert Virding < robert.virding@REDACTED> wrote: > Sorry, I don't agree with you here. While using OTP is generally a Good > Thing, using OTP because you know WHY you need to use use it is a Much Much > Much Better Thing! You see cases of where people have blindly used > gen_servers then struggle and produce convoluted code to get around the > very features which gen_servers provide. So I would say that you should > know what the basic Erlang provides and how you use it, know what OTP > provides and how you use it and choose that which is closest to what you > need. > > I am not saying that you shouldn't use OTP, I am saying you should know > why you are using OTP. > > Actually it is not very difficult to roll your own processes which fit > into OTP supervision trees. It is done using the proc_lib and sys modules > and it describe quite well here > http://www.erlang.org/doc/design_principles/spec_proc.html . > > Robert > > ----- Original Message ----- > > From: "Garrett Smith" > > To: "Vineet Naik" > > Cc: erlang-questions@REDACTED > > Sent: Friday, 30 November, 2012 2:48:54 PM > > Subject: Re: [erlang-questions] Is using gen_server behaviour > appropriate for his usecase? > > > > Roman is spot on correct. > > > > There's a school of thought that says new developers should learn > > core > > Erlang and master it before tackling OTP. IMHO this is wrong. OTP > > *is* > > core Erlang at this point.[1] > > > > Just use OTP, right away. If the many moving parts of the standard > > OTP > > behaviors seems like a steep learning curve, take a look at e2: > > > > http://e2project.org > > > > e2 is a veneer on top of OTP that, at least in my opinion (I wrote it > > for this reason :) makes canonical development a lot easier. You > > might > > start by reading/following the tutorial: > > > > http://e2project.org/tutorial.html > > > > The tutorial has more to do with general application development in > > Erlang than it does in with the specifics of the e2 library. > > > > Garrett > > > > [1] There's a long history of Erlang that separates the core from > > OTP. > > I understand how books, tutorials, teaching philosophies, and common > > wisdom would treat OTP as an advanced topic. But as a relative > > newcomer to Erlang, not having any historical context, etc. I found > > this treatment unhelpful in my own learning. IMO "gen_server" (e2 > > calls these "services" and "tasks" depending on the scenario) is a > > starting point, after one has learned the basics of the language > > itself. > > > > On Fri, Nov 30, 2012 at 2:36 AM, Roman Gafiyatullin > > wrote: > > > Hi, > > > > > > In most of the cases one shall not use "simple loop functions": raw > > > process > > > cannot participate properly in the supervision tree. > > > > > > If you need some new tricky specific model of behaving - better > > > implement it > > > over gen_server. > > > For instance supervisor is a gen_server :) > > > > > > If you need something more low level - use 'gen' module (see how > > > gen_fsm is > > > implemented). > > > > > > -- > > > Regards > > > RG ( +375 33 602 5080, UTC+3 ) > > > > > > On Friday, November 30, 2012 at 11:25 am, Vineet Naik wrote: > > > > > > Hello, > > > > > > I am an Erlang newbie and trying to write a simple chat bot as an > > > XMPP external component. I am using exmpp library and following > > > along these tutorials[1] > > > > > > So far I have one module `bot_server` that uses the gen_server > > > interface. Inside it's `handle_info` callback, incoming messages > > > from various client will be received. To handle and reply to the > > > these messages, I am thinking of spawning a "bot" process per > > > client. It will stay alive as long as the client is available > > > ie. when the client sends "unavailable" presence, it will die. I > > > also need to keep a list of all the alive bot processes in the > > > bot_server's state. > > > > > > My question is, would it be appropriate to implement the bot as a > > > gen_server too considering that it needs to handle two calls, one > > > for handling incoming message (asynchronous) and second for > > > killing itself (synchronous)? > > > > > > In general, when should one use gen_server and when should > > > one write a simple loop function? > > > > > > [1] exmpp tutorials: > > > > http://blog.process-one.net/scalable_xmpp_bots_with_erlang_and_exmpp_part_i/ > > > > > > Thanks, > > > Vineet > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sun Dec 2 18:41:17 2012 From: g@REDACTED (Garrett Smith) Date: Sun, 2 Dec 2012 11:41:17 -0600 Subject: [erlang-questions] Is using gen_server behaviour appropriate for his usecase? In-Reply-To: <1855257212.251860.1354415256937.JavaMail.root@erlang-solutions.com> References: <1855257212.251860.1354415256937.JavaMail.root@erlang-solutions.com> Message-ID: I'm not advocating that one should use proc_lib/gen_server facilities blindly -- or incorrectly :) The OP asked about roll-your-own message loops vs OTP -- and I'd cite that as an example of why we should teach OTP as fundamental to correct Erlang application development. My motivation for writing e2 was that I observed new users struggling with the complexity of the gen_server interface (primarily) and, given the importance of it, wanted to make it easier to use correctly. But, as you suggest, we should perhaps talk more about proc_lib and how to use it when you're tempted to otherwise run a process outside OTP facilities. On Sat, Dec 1, 2012 at 8:27 PM, Robert Virding wrote: > Sorry, I don't agree with you here. While using OTP is generally a Good Thing, using OTP because you know WHY you need to use use it is a Much Much Much Better Thing! You see cases of where people have blindly used gen_servers then struggle and produce convoluted code to get around the very features which gen_servers provide. So I would say that you should know what the basic Erlang provides and how you use it, know what OTP provides and how you use it and choose that which is closest to what you need. > > I am not saying that you shouldn't use OTP, I am saying you should know why you are using OTP. > > Actually it is not very difficult to roll your own processes which fit into OTP supervision trees. It is done using the proc_lib and sys modules and it describe quite well here http://www.erlang.org/doc/design_principles/spec_proc.html . > > Robert > > ----- Original Message ----- >> From: "Garrett Smith" >> To: "Vineet Naik" >> Cc: erlang-questions@REDACTED >> Sent: Friday, 30 November, 2012 2:48:54 PM >> Subject: Re: [erlang-questions] Is using gen_server behaviour appropriate for his usecase? >> >> Roman is spot on correct. >> >> There's a school of thought that says new developers should learn >> core >> Erlang and master it before tackling OTP. IMHO this is wrong. OTP >> *is* >> core Erlang at this point.[1] >> >> Just use OTP, right away. If the many moving parts of the standard >> OTP >> behaviors seems like a steep learning curve, take a look at e2: >> >> http://e2project.org >> >> e2 is a veneer on top of OTP that, at least in my opinion (I wrote it >> for this reason :) makes canonical development a lot easier. You >> might >> start by reading/following the tutorial: >> >> http://e2project.org/tutorial.html >> >> The tutorial has more to do with general application development in >> Erlang than it does in with the specifics of the e2 library. >> >> Garrett >> >> [1] There's a long history of Erlang that separates the core from >> OTP. >> I understand how books, tutorials, teaching philosophies, and common >> wisdom would treat OTP as an advanced topic. But as a relative >> newcomer to Erlang, not having any historical context, etc. I found >> this treatment unhelpful in my own learning. IMO "gen_server" (e2 >> calls these "services" and "tasks" depending on the scenario) is a >> starting point, after one has learned the basics of the language >> itself. >> >> On Fri, Nov 30, 2012 at 2:36 AM, Roman Gafiyatullin >> wrote: >> > Hi, >> > >> > In most of the cases one shall not use "simple loop functions": raw >> > process >> > cannot participate properly in the supervision tree. >> > >> > If you need some new tricky specific model of behaving - better >> > implement it >> > over gen_server. >> > For instance supervisor is a gen_server :) >> > >> > If you need something more low level - use 'gen' module (see how >> > gen_fsm is >> > implemented). >> > >> > -- >> > Regards >> > RG ( +375 33 602 5080, UTC+3 ) >> > >> > On Friday, November 30, 2012 at 11:25 am, Vineet Naik wrote: >> > >> > Hello, >> > >> > I am an Erlang newbie and trying to write a simple chat bot as an >> > XMPP external component. I am using exmpp library and following >> > along these tutorials[1] >> > >> > So far I have one module `bot_server` that uses the gen_server >> > interface. Inside it's `handle_info` callback, incoming messages >> > from various client will be received. To handle and reply to the >> > these messages, I am thinking of spawning a "bot" process per >> > client. It will stay alive as long as the client is available >> > ie. when the client sends "unavailable" presence, it will die. I >> > also need to keep a list of all the alive bot processes in the >> > bot_server's state. >> > >> > My question is, would it be appropriate to implement the bot as a >> > gen_server too considering that it needs to handle two calls, one >> > for handling incoming message (asynchronous) and second for >> > killing itself (synchronous)? >> > >> > In general, when should one use gen_server and when should >> > one write a simple loop function? >> > >> > [1] exmpp tutorials: >> > http://blog.process-one.net/scalable_xmpp_bots_with_erlang_and_exmpp_part_i/ >> > >> > Thanks, >> > Vineet >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From ok@REDACTED Sun Dec 2 23:02:16 2012 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 3 Dec 2012 11:02:16 +1300 Subject: [erlang-questions] -spec tuple variable size In-Reply-To: <618AC0AF-1EBB-433E-B40F-978AB2745B95@gmail.com> References: <757B8B50-CCFC-4CCB-8976-2ABFE587DFBE@gmail.com> <5A969370-3E73-46B2-8427-5298281E10DF@gmail.com> <2ADC11CB-DA02-490E-BFDA-689EB841D8B5@khandkar.net> <79F1C2F0-9178-4701-A7D6-5A2D1725FAAC@gmail.com> <618AC0AF-1EBB-433E-B40F-978AB2745B95@gmail.com> Message-ID: <702E085A-B7E3-4FB2-9A82-1131971EDDD5@cs.otago.ac.nz> On 1/12/2012, at 11:03 AM, Dmitry Kolesnikov wrote: > Hello, > > On Nov 30, 2012, at 2:26 AM, "Richard O'Keefe" wrote: > >> Let me put it this way: >> why do YOU care whether all the elements of the tuple have >> the same type? > > One of my current limitation is that tuple elements MUST be a _particular_ data type. That really doesn't answer the question, although it may clarify it. You don't want "any size tuple all of whose elements are the same type, whatever that type is", but "any size tuple all of whose elements are of _this_ specific type", but you have not as yet told us what _this_ specific type happens to be. However, the question stands: - why does it _have_ to be a tuple? - why do _you_ care about the types? > I simply want to validate that client writes tuples with supported data types. - why can't the _clients_ validate? - _who_ is restricting the types? > E.g. I do not have a support for nested tuples yet. - what are _you_ doing that cares what the elements of a tuple are? You're hinting at a setup that sounds quite un-natural in Erlang, and as yet we have no clear idea of what the application-level problem is. From codeithere@REDACTED Mon Dec 3 08:47:45 2012 From: codeithere@REDACTED (Code Box) Date: Sun, 2 Dec 2012 23:47:45 -0800 Subject: [erlang-questions] Good Parser that works with Erlang 13B Message-ID: I have a streaming parser in my erlang code base which is causing a lot of problems to me. It is such bad that it make so many multiple copies of strings that it has to parse and it eats up all the memory of my stack and causes the whole system to crash. I am looking at changing the JSON parser from the current crappy implementation that I have with a good one. I have certain limitations on the Erlang Version i am using. I am using Erlang 13 B right now. Moving to the newer version of Erlang is a lot more complicated as i will have to test it first, that is like a month of work. Being in a startup i have very little time to get this done. I am seeking suggestion around good performing JSON parsers that work with Erlang 13B and are easy to integrate. Looking for C parsers needs Erlang 14B that is also complicated so i am not considering any option that wants me to upgrade the Erlang version. I want to get these things out of this exercise :- 1. Plug the new Parser into the current System ( Can be a good Erlang Parser ) 2. Relieve memory pressure on the system 3. Get a faster JSON Parser. 4. Keep the Erlang Version to 13B Looking around i did find Mochi Erlang Parser. I am seeking suggestion around the Parser that i can use for this? Appreciate your suggestions on this. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrkhoza@REDACTED Mon Dec 3 09:20:53 2012 From: mrkhoza@REDACTED (Lucky Khoza) Date: Mon, 3 Dec 2012 10:20:53 +0200 Subject: [erlang-questions] Kerl Message-ID: Hi Guys, I used kerl to build and install Erlang versions: R14B01 and R14B03, and I also used the kerl commands to delete both builds: ./kerl delete build r14b01 and /kerl delete build r14b03, and now when I try to delete the installation of both versions using kerl commands: ./kerl delete installation r14b01and ./kerl delete installation r14b0, I got this message and I don't understand: r14b01/r14b03 is not a kerl-managed Erlang/OTP installation, same message for each deletion. Please help guys and your help is highly appreciated Kindest Regards Lucky -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon Dec 3 09:33:22 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 3 Dec 2012 08:33:22 +0000 Subject: [erlang-questions] Good Parser that works with Erlang 13B In-Reply-To: References: Message-ID: <4CFE44AA-D31D-481B-BFF7-8710EE3C0EEA@gmail.com> Try JSX - it's on GitHub can't remember the author but it's fast lightweight and pure erlang. Has a nice API and streaming support too. On 3 Dec 2012, at 07:47, Code Box wrote: > I have a streaming parser in my erlang code base which is causing a lot of problems to me. It is such bad that it make so many multiple copies of strings that it has to parse and it eats up all the memory of my stack and causes the whole system to crash. I am looking at changing the JSON parser from the current crappy implementation that I have with a good one. I have certain limitations on the Erlang Version i am using. I am using Erlang 13 B right now. Moving to the newer version of Erlang is a lot more complicated as i will have to test it first, that is like a month of work. Being in a startup i have very little time to get this done. I am seeking suggestion around good performing JSON parsers that work with Erlang 13B and are easy to integrate. Looking for C parsers needs Erlang 14B that is also complicated so i am not considering any option that wants me to upgrade the Erlang version. > > I want to get these things out of this exercise :- > > 1. Plug the new Parser into the current System ( Can be a good Erlang Parser ) > 2. Relieve memory pressure on the system > 3. Get a faster JSON Parser. > 4. Keep the Erlang Version to 13B > > Looking around i did find Mochi Erlang Parser. I am seeking suggestion around the Parser that i can use for this? Appreciate your suggestions on this. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From develop7@REDACTED Mon Dec 3 10:12:50 2012 From: develop7@REDACTED (Andrei Dziahel) Date: Mon, 3 Dec 2012 12:12:50 +0300 Subject: [erlang-questions] Kerl In-Reply-To: References: Message-ID: Hello, Looks like a bug to me. You may want to report it at https://github.com/spawngrid/kerl/issues. On Mon, Dec 3, 2012 at 11:20 AM, Lucky Khoza wrote: > Hi Guys, > > I used kerl to build and install Erlang versions: R14B01 and R14B03, and I > also used the kerl commands to delete both builds: ./kerl delete build > r14b01 and /kerl delete build r14b03, > and now when I try to delete the installation of both versions using kerl > commands: ./kerl delete installation r14b01and ./kerl delete installation > r14b0, I got this message and I don't understand: > > r14b01/r14b03 is not a kerl-managed Erlang/OTP installation, > > same message for each deletion. > > Please help guys and your help is highly appreciated > > Kindest Regards > Lucky > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Regards, Andrei Dziahel -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Mon Dec 3 10:57:16 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 3 Dec 2012 10:57:16 +0100 Subject: [erlang-questions] using bundled certificates in the ssl:ssl_accept upgrade function In-Reply-To: References: Message-ID: Hi! 2012/11/30, Daniel Barney : > Hello, > > I am using erlang on a server that has quite a few ips, and I need to > serve out a different certificate based on which ip the client > connects to. > > So i am upgrading a tcp socket based on the ip that the client is > connecting to. Unfortunatly i have only managed to get this to work > with certificates that don't have another certificates bundled with > them. > > Am i just doing this wrong? > > so here is how I tried the first time, this establishes the encrypted > connection, but it doesn't serve the bundled certificates. just the > first one. > > {ok,BundleFileData} = file:read_file("/mnt/ssl/mycert.bundle.crt"), > > [{_,TheCert,_} | _] = public_key:pem_decode(CertFIleData), %% notice > how I only grab the first one, which is why it can only serve the > first one in the chain > Certs = [{cert,TheCert},{keyfile = "/mnt/ssl/mycert.key"}], > {ok,SslSocket} = ssl:ssl_accept(Socket,[{active,false},{verify, > verify_none}] ++ Certs) %% Socket is opened somewhere else > > I tried to verify the connection with the following command: openssl > s_client -showcerts -connect 127.0.0.1:4430 > > but the cert is never trusted because only it is served and never the > bundled certs.and I expect that the cert will not be trusted because I > am only giving it the first one. > > and I get this: > CONNECTED(00000003) > depth=0 %% removed because it is not my cert > verify error:num=20:unable to get local issuer certificate > verify return:1 > depth=0 %% removed because it is not my cert > verify error:num=27:certificate not trusted > verify return:1 > depth=0 %% removed because it is not my cert > verify error:num=21:unable to verify the first certificate > verify return:1 > --- > Certificate chain > 0 %% removed because it is not my cert > i:/C=US/O=GeoTrust, Inc./CN=RapidSSL CA > %% some more stuff removed > Verify return code: 21 (unable to verify the first certificate) > > > and when I switch to just trying to load the file from the disk, which > returns an ecertfile error > > > Certs = [{certfile,"/mnt/ssl/mycert.bundle.crt"},{keyfile = > "/mnt/ssl/mycert.key"}], The option certifile expects a pem-file that only holds one certificate the peer certificate. Other certificates in the chain should be specified by the option cacertfile option. > {error,ecertfile} = ssl:ssl_accept(Socket,[{active,false},{verify, > verify_none}] ++ Certs) %% Socket is opened somewhere else > > I've checked and the file does exist at the path, and I have used the > same cert bundle in a Node.js project before so I know its not the > certificate. > > > So my question is am I doing this completely wrong? I can't manage to > find anything on the manual page for ssl to indicate that bundled > certs wouldn't work in erlang, and I can't imagine that erlang does > not support bundled certificates. So i have to be doing this wrong. > > any help would be much appreciated on figuring this out. So yes and no we support bundled certificates but not in the way you expected. Only cacertfile can be a bundle not certfile! This is a very old API and if there is a good reason too we could extend it, you are welcome to make suggestions. Regards Ingela Erlang/OTP team - Ericsson AB From dmkolesnikov@REDACTED Mon Dec 3 13:25:13 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Mon, 3 Dec 2012 14:25:13 +0200 Subject: [erlang-questions] Good Parser that works with Erlang 13B In-Reply-To: <4CFE44AA-D31D-481B-BFF7-8710EE3C0EEA@gmail.com> References: <4CFE44AA-D31D-481B-BFF7-8710EE3C0EEA@gmail.com> Message-ID: <-9025498646794268830@unknownmsgid> JSX is way to go... Best Regards, Dmitry >-|-|-*> On 3.12.2012, at 10.33, Tim Watson wrote: > Try JSX - it's on GitHub can't remember the author but it's fast lightweight and pure erlang. Has a nice API and streaming support too. > > On 3 Dec 2012, at 07:47, Code Box wrote: > >> I have a streaming parser in my erlang code base which is causing a lot of problems to me. It is such bad that it make so many multiple copies of strings that it has to parse and it eats up all the memory of my stack and causes the whole system to crash. I am looking at changing the JSON parser from the current crappy implementation that I have with a good one. I have certain limitations on the Erlang Version i am using. I am using Erlang 13 B right now. Moving to the newer version of Erlang is a lot more complicated as i will have to test it first, that is like a month of work. Being in a startup i have very little time to get this done. I am seeking suggestion around good performing JSON parsers that work with Erlang 13B and are easy to integrate. Looking for C parsers needs Erlang 14B that is also complicated so i am not considering any option that wants me to upgrade the Erlang version. >> >> I want to get these things out of this exercise :- >> >> 1. Plug the new Parser into the current System ( Can be a good Erlang Parser ) >> 2. Relieve memory pressure on the system >> 3. Get a faster JSON Parser. >> 4. Keep the Erlang Version to 13B >> >> Looking around i did find Mochi Erlang Parser. I am seeking suggestion around the Parser that i can use for this? Appreciate your suggestions on this. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From n.oxyde@REDACTED Mon Dec 3 13:27:25 2012 From: n.oxyde@REDACTED (Anthony Ramine) Date: Mon, 3 Dec 2012 13:27:25 +0100 Subject: [erlang-questions] "Makefile.in has been updated, please re-run configure!" Message-ID: <78603B4F-9E48-4226-9095-2DC0D8835A80@gmail.com> Hi, When erts/Makefile.in is updated, make fails with the following message: Makefile.in has been updated, please re-run configure! Is there a reason why config.status isn't just called to regenerate Makefile from its template? This tool is very useful when it comes to templates' update and reconfiguring [1]. Regards, [1] http://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/config_002estatus-Invocation.html -- Anthony Ramine From max.lapshin@REDACTED Tue Dec 4 10:56:44 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 4 Dec 2012 12:56:44 +0300 Subject: [erlang-questions] record2ei: automated ei decoding from Erlang records to C structs Message-ID: When you write a C port you may experience some problems with unpacking erlang records, that were sent to port with term_to_binary/1 I thought that it would be very nice to write: port_command(Port, term_to_binary(#trade_event{price = Price, volume = V})) and then write in C: struct trade_event *evt = read_trade_event(buf, &idx); handle(evt->price, evt->volume); So I've written https://github.com/maxlapshin/record2ei This is a small and very dumb generator of C header and C source that generates C structures and their encoding/decoding functions that calls boilerplate code like ei_decode_long or ei_encode_binary. It is very limited, yet it helped to me. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pan@REDACTED Tue Dec 4 11:48:06 2012 From: pan@REDACTED (Patrik Nyblom) Date: Tue, 4 Dec 2012 11:48:06 +0100 Subject: [erlang-questions] Problem with io monitor - a bug? In-Reply-To: References: Message-ID: <50BDD4E6.1050004@erlang.org> Hi, No, it's not a bug. It's a race in the calling code. The spawned process sends the first io-request and the shell sends the second. The first "read" will be overridden while the shell's request is served. They are so to say stacked. Having two processes (the shell and your newly spawned process) reading from the same io-device is generally not a good idea, they are competing for a single resource that is kind of hard to share... You have to keep other processes from tampering with your terminal while you're reading it, in this case by letting the shell wait for your process. In fact, you actually get your prompt from the spawned process written, but the shell will override/overwrite it. Try sleeping the shell for a second and your prompt will temporarily be shown, but then the shell comes back and your prompt disappears again. If you sleep for a while in the child process before reading, the child will instead override the shell. There's no telling which one will be stacked upon the other. It's kind of impossible to determine which order is preferred by the caller... So, your program has to decide on the order by synchronizing it's activities. Example: 1> Prompt = "Please enter data: ". 2> F = fun() -> try Result = io:read(Prompt), io:format("Res: ~p~n",[Result]) catch A:B -> io:format("Read error: ~p:~p~n",[A,B]) end end. 3> {Pid,Ref} = spawn_monitor(F), receive after 1000 -> ok end. %% "Please Enter data: " will be shown for a second, then disappears and the shell prompt comes back. 4> receive after 1000 -> ok end. %% Please enter data will appear again, but disappears after a second again. 5> receive {'DOWN',Ref,process,Pid,_} -> ok end. Please enter data: hi. Res: {ok,hi} %% Now try waiting a second to make the shell send the io-request before your spawned process 6> G = fun() -> try receive after 1000 -> ok end, Result = io:read(Prompt), io:format("Res: ~p~n",[Result]) catch A:B -> io:format("Read error: ~p:~p~n",[A,B]) end end. 7> spawn(G). %% First you will see the shell prompt for ~ a second, then you will get: Please enter data: hello. Res: {ok,hello} 8> In other words, do not have two processes competing tor one terminal, it will only give confusing results :) Cheers, /Patrik, OTP On 11/30/2012 01:57 PM, ?lvaro wrote: > Dear all, > > Every time I run this simple code, my program gets stuck: > > > Pid = spawn (moduleName,myread,["hello"]). > > where: > > myread(Prompt)-> > try > Result = io:read(Prompt), > io:format("Res: ~p~n",[Result]), > catch > A:B-> > io:format("Read error: ~p:~p~n",[A,B]), > false > end. > > > if I check the status of the process: > > > erlang:process_info(Pid). > > [{current_function,{io,wait_io_mon_reply,2}}, > {initial_call,{default_environment,myread,1}}, > {status,waiting}, > {message_queue_len,0}, > {messages,[]}, > {links,[]}, > {dictionary,[]}, > {trap_exit,false}, > {error_handler,error_handler}, > {priority,normal}, > {group_leader,<0.25.0>}, > {total_heap_size,233}, > {heap_size,233}, > {stack_size,7}, > {reductions,28}, > {garbage_collection,[{min_bin_vheap_size,46368}, > {min_heap_size,233}, > {fullsweep_after,65535}, > {minor_gcs,0}]}, > {suspending,[]}] > > > It seems to be stuck in the function io:wait_io_mon_reply/2 (surfing > through io.erl I found that this function waits for a response from > the process managing the io ). This process is not answering, hence > my code stops working. > I've checked that the "group_leader()" once I spawn the reading > process is the same used by my shell. > > Any ideas on how to deal with it? > > Regards, > ?lvaro > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From barcojie@REDACTED Tue Dec 4 13:21:41 2012 From: barcojie@REDACTED (Barco You) Date: Tue, 4 Dec 2012 20:21:41 +0800 Subject: [erlang-questions] [gproc] How to broadcast message to all other processes except self In-Reply-To: References: Message-ID: Dear All, I use gproc:reg to register all the processes to a KEY and through gproc:send(KEY, MSG) I can broadcast message to all the processes, including myself. But now I hope to send message to all the other processes except myself. How can I do it? Could you please illuminate ? Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Tue Dec 4 14:31:35 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 4 Dec 2012 14:31:35 +0100 Subject: [erlang-questions] [gproc] How to broadcast message to all other processes except self In-Reply-To: References: Message-ID: <7BD18BC2-3C82-4D40-A309-29BCB0E6C1BD@feuerlabs.com> The function gproc:send/2, when used on a non-unique key, in essence does this: lists:foreach(fun(Pid) -> Pid ! Msg end, gproc:lookup_pids(Key)), So you can take that as a starting point, and add a clause that pattern-matches on the own pid, and then doesn't send a message in that case. BR, Ulf W On 4 Dec 2012, at 13:21, Barco You wrote: > Dear All, > > I use gproc:reg to register all the processes to a KEY and through gproc:send(KEY, MSG) I can broadcast message to all the processes, including myself. But now I hope to send message to all the other processes except myself. How can I do it? Could you please illuminate ? Thank you. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From barcojie@REDACTED Tue Dec 4 17:19:43 2012 From: barcojie@REDACTED (Barco You) Date: Wed, 5 Dec 2012 00:19:43 +0800 Subject: [erlang-questions] [gproc] How to broadcast message to all other processes except self In-Reply-To: <7BD18BC2-3C82-4D40-A309-29BCB0E6C1BD@feuerlabs.com> References: <7BD18BC2-3C82-4D40-A309-29BCB0E6C1BD@feuerlabs.com> Message-ID: Thank you for replying. But I tried to use Pid ! Msg, but seemingly can not receive message as gproc:send(Key, Msg). On Tue, Dec 4, 2012 at 9:31 PM, Ulf Wiger wrote: > > The function gproc:send/2, when used on a non-unique key, in essence does > this: > > lists:foreach(fun(Pid) -> > Pid ! Msg > end, gproc:lookup_pids(Key)), > > So you can take that as a starting point, and add a clause that > pattern-matches on the own pid, and then doesn't send a message in that > case. > > BR, > Ulf W > > On 4 Dec 2012, at 13:21, Barco You wrote: > > Dear All, > > I use gproc:reg to register all the processes to a KEY and through gproc:send(KEY, > MSG) I can broadcast message to all the processes, including myself. But > now I hope to send message to all the other processes except myself. How > can I do it? Could you please illuminate ? Thank you. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From barcojie@REDACTED Tue Dec 4 17:35:06 2012 From: barcojie@REDACTED (Barco You) Date: Wed, 5 Dec 2012 00:35:06 +0800 Subject: [erlang-questions] [gproc] How to broadcast message to all other processes except self In-Reply-To: References: <7BD18BC2-3C82-4D40-A309-29BCB0E6C1BD@feuerlabs.com> Message-ID: Hi Ulf, it's solved. thank you! Barco On Wed, Dec 5, 2012 at 12:19 AM, Barco You wrote: > Thank you for replying. But I tried to use Pid ! Msg, but seemingly can > not receive message as gproc:send(Key, Msg). > > > On Tue, Dec 4, 2012 at 9:31 PM, Ulf Wiger wrote: > >> >> The function gproc:send/2, when used on a non-unique key, in essence does >> this: >> >> lists:foreach(fun(Pid) -> >> Pid ! Msg >> end, gproc:lookup_pids(Key)), >> >> So you can take that as a starting point, and add a clause that >> pattern-matches on the own pid, and then doesn't send a message in that >> case. >> >> BR, >> Ulf W >> >> On 4 Dec 2012, at 13:21, Barco You wrote: >> >> Dear All, >> >> I use gproc:reg to register all the processes to a KEY and through gproc:send(KEY, >> MSG) I can broadcast message to all the processes, including myself. But >> now I hope to send message to all the other processes except myself. How >> can I do it? Could you please illuminate ? Thank you. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jrosenblum@REDACTED Tue Dec 4 18:29:36 2012 From: jrosenblum@REDACTED (James Rosenblum) Date: Tue, 4 Dec 2012 09:29:36 -0800 (PST) Subject: [erlang-questions] displaying the globally, registered name in Observer In-Reply-To: <13e058f0-56e0-404c-a6c6-abc8b0dee8a6@knuth> References: <1351465423.72517.YahooMailNeo@web181203.mail.ne1.yahoo.com> <13e058f0-56e0-404c-a6c6-abc8b0dee8a6@knuth> Message-ID: <1354642176.31382.YahooMailNeo@web181203.mail.ne1.yahoo.com> I am writing a distributed application and could not seem to find a way to get Observer's Applications pane to show the registered name of a process when that process is globally registered (as in gen_server:start_link({global, Name}, ?MODULE, [],[])). I also couldn't find a global method which returned a registered name given a pid. So, I added a function go the global module pid_2_name(Pid) -> ? ? case ets:lookup(global_pid_names, Pid) of [{Pid, Name}] ->? ? ?if node(Pid) == node() -> ? ?case is_process_alive(Pid) of true -> Name; false -> undefined ? ?end; ? ? ? true -> ? ?Name ? ?end; [] -> undefined ? ? end. and then I modified appmon_info and observer_procinfo to make use of this function when process_info failed to return a {registered_info, _} tuple. I did this sort of thing: ?case process_info(P, registered_name) of {registered_name, Name} -> atom_to_list(Name); _ -> ? ?case global:pid_2_name(P) of undefined -> pid_to_list(P); Name -> "*" ++ atom_to_list(Name) ? ?end ? ? end; So was this a totally unsafe and / or ?unnecessary?thing to do; or, is this for the most part an okay thing to do? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbothma@REDACTED Tue Dec 4 19:09:08 2012 From: jbothma@REDACTED (JD Bothma) Date: Tue, 4 Dec 2012 19:09:08 +0100 Subject: [erlang-questions] Re: confusing returns from httpc:request/4 Message-ID: I'm experiencing the same thing, fixed by the same workaround: httpc:set_options([{max_keep_alive_length, 0},{max_pipeline_length, 0}, {max_sessions, 0} ]), I'm on R15B02, testing against Cowboy 0.6.1 Has your fix gone in yet or has it perhaps regressed? Regards, JD >* On Oct 21, 8:54 pm, Ingela Andin <> wrote:*>>* Hi!*>>**>>* Yes thank you for the information. I think I have located the bug. It seems*>>* that to avoid crash reports the return value when receiving a tcp_closed*>>* was changed and this altered the terminate clause that is run, alas breaking*>>* the automatic retries that should be done if the server terminates a connection*>>* before serving all pipelined requests. This will be fixed in an*>>* upcoming release.*>>**>>* Regards Ingela Erlang/OTP-team, Ericsson AB* -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Tue Dec 4 22:12:01 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Tue, 4 Dec 2012 22:12:01 +0100 Subject: [erlang-questions] Proper/EQC parallel statem test question Message-ID: Hi list! Does postcondition of the statem test has to be a pure function? Is it OK to do a request to a processe of the tested system from postcondition like I do here [1] ? This becomes a problem for me when running degenerated parallel test where commands seems to be run in a single process, while postcondition is being checked in another test. Example: Sequential part: [actor_up(v01), actor_up(v02)] Parallel part: Process one: [] Process two: [ actor_search(v01), actor_search(v02)] So essentially it is still a sequential test, since only one of the two parallel testers is running any code. But due to postconditions of actor_search operation being run in yet another process (probably main test process) they are getting mixed up. Thanks! Gleb P.S. This is an academic project, hence I am using Proper and can't test if it behaves differently in full version EQC. 1: https://github.com/gleber/dstree/blob/master/test/dstree_statem.erl#L129 From daniel.goertzen@REDACTED Tue Dec 4 22:21:02 2012 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Tue, 4 Dec 2012 15:21:02 -0600 Subject: [erlang-questions] upcoming R16 features? Message-ID: I can find little information on R16. Could someone in the know tell us what's coming? (or point me to such info if I've missed the obvious) I am specifically interested in NIF processes and frames/structs/dicts. Thanks, Dan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonstl@REDACTED Tue Dec 4 23:28:46 2012 From: simonstl@REDACTED (Simon St.Laurent) Date: Tue, 04 Dec 2012 17:28:46 -0500 Subject: [erlang-questions] process manager, R16, and 64-bit Macs Message-ID: <50BE791E.1050202@simonstl.com> The question about R16 features reminded me that one of the promises of R16 is the removal of Process Manager: "The Pman application has been superseded by the Observer application. Pman will be removed in R16." - http://www.erlang.org/doc/man/pman.html However, Observer does not run at all on 64-bit Macs, because of that platform's lack of support for wxWidgets. (The other Erlang GUI apps I've tried do run, though the debugger is a bit crash-prone.) R15B3 seems to report the same "wxWidgets not found, wx will NOT be usable" message. Is there any chance of Pman getting a delay until support is better? Or is this just the cost of progress? (I'm writing Introducing Erlang for O'Reilly, and need to decide if I'm ripping out the process manager coverage and replacing it, and if I need to keep that coverage around online for customers who happen to use recent Macs.) If there's a better place to ask this, please let me know! Thanks, -- Simon St.Laurent http://simonstl.com/ From torben.lehoff@REDACTED Wed Dec 5 09:22:06 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Wed, 05 Dec 2012 09:22:06 +0100 Subject: [erlang-questions] Proper/EQC parallel statem test question In-Reply-To: References: Message-ID: <50BF042E.9030200@gmail.com> Hi Gleb, On 2012-12-04 22:12, Gleb Peregud wrote: > Hi list! > > Does postcondition of the statem test has to be a pure function? Is it > OK to do a request to a processe of the tested system from > postcondition like I do here [1] ? If you have a sufficiently complicated system you might have to do a number of things to validate what you have done is actually correct, so I think what you are doing is in principle okay. The trick is that you have make sure that you are fully done with a test step before you proceed. To exemplify what you might have to deal with: I had a system where I had to observe multiple messages in the system to be sure that everything was okay. For that I spawned a process for each message and if the message occoured before some reasonable timeout the process reported true back to a collector process. The collector process checked all responses and replied back to the test. That worked. I knew how long it was reasonable to wait and then I could introduce the synchronisation point needed by - in my case - QuickCheck. > > This becomes a problem for me when running degenerated parallel test > where commands seems to be run in a single process, while > postcondition is being checked in another test. Example: > > Sequential part: > [actor_up(v01), > actor_up(v02)] > > Parallel part: > Process one: [] > Process two: [ > actor_search(v01), > actor_search(v02)] > > So essentially it is still a sequential test, since only one of the > two parallel testers is running any code. But due to postconditions of > actor_search operation being run in yet another process (probably main > test process) they are getting mixed up. This I do not understand - could you elaborate on what is going on? Cheers, ___ /orben > > Thanks! > Gleb > > P.S. This is an academic project, hence I am using Proper and can't > test if it behaves differently in full version EQC. > > 1: https://github.com/gleber/dstree/blob/master/test/dstree_statem.erl#L129 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- http://www.linkedin.com/in/torbenhoffmann From gustav.simonsson@REDACTED Wed Dec 5 09:34:29 2012 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Wed, 5 Dec 2012 09:34:29 +0100 Subject: [erlang-questions] upcoming R16 features? In-Reply-To: References: Message-ID: OTP Technical Board - Decisions affecting R16: http://www.erlang.org/news/35 Cheers, Gustav Simonsson On Tue, Dec 4, 2012 at 10:21 PM, Daniel Goertzen wrote: > I can find little information on R16. Could someone in the know tell us > what's coming? (or point me to such info if I've missed the obvious) I am > specifically interested in NIF processes and frames/structs/dicts. > > Thanks, > Dan. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pan@REDACTED Wed Dec 5 11:15:01 2012 From: pan@REDACTED (Patrik Nyblom) Date: Wed, 5 Dec 2012 11:15:01 +0100 Subject: [erlang-questions] process manager, R16, and 64-bit Macs In-Reply-To: <50BE791E.1050202@simonstl.com> References: <50BE791E.1050202@simonstl.com> Message-ID: <50BF1EA5.3040201@erlang.org> On 12/04/2012 11:28 PM, Simon St.Laurent wrote: > The question about R16 features reminded me that one of the promises > of R16 is the removal of Process Manager: > > "The Pman application has been superseded by the Observer application. > Pman will be removed in R16." - http://www.erlang.org/doc/man/pman.html > > However, Observer does not run at all on 64-bit Macs, because of that > platform's lack of support for wxWidgets. (The other Erlang GUI apps > I've tried do run, though the debugger is a bit crash-prone.) R15B3 > seems to report the same "wxWidgets not found, wx will NOT be usable" > message. We're working on using the wxWidgets 2.9 "beta-line" for macs, which would fix the 64bit problem (it's cocoa-based). 2.8 is also almost impossible to build on mountain lion, so we need to up wxWidgets version on mac anyway. Unfortunately we have some rather unpleasant stability problems with 2.9 on the mac, so we might need to reconsider... > > Is there any chance of Pman getting a delay until support is better? > Or is this just the cost of progress? If we do not have a cross platform wx solution that works for R16, we will definitely "think again"! > > (I'm writing Introducing Erlang for O'Reilly, and need to decide if > I'm ripping out the process manager coverage and replacing it, and if > I need to keep that coverage around online for customers who happen to > use recent Macs.) Might be a good idea to keep it online... > > If there's a better place to ask this, please let me know! > Nope, erlang-questions is the right place :) > Thanks, Cheers, /Patrik, OTP From pan@REDACTED Wed Dec 5 11:21:59 2012 From: pan@REDACTED (Patrik Nyblom) Date: Wed, 5 Dec 2012 11:21:59 +0100 Subject: [erlang-questions] upcoming R16 features? In-Reply-To: References: Message-ID: <50BF2047.7000307@erlang.org> Just to emphasize... As you can see from the linked text, there will be no maps (or frames) in R16. Cheers, /Patrik On 12/05/2012 09:34 AM, Gustav Simonsson wrote: > OTP Technical Board - Decisions affecting R16: > http://www.erlang.org/news/35 > > Cheers, > Gustav Simonsson > > > On Tue, Dec 4, 2012 at 10:21 PM, Daniel Goertzen > > wrote: > > I can find little information on R16. Could someone in the know > tell us what's coming? (or point me to such info if I've missed > the obvious) I am specifically interested in NIF processes and > frames/structs/dicts. > > Thanks, > Dan. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From pan@REDACTED Wed Dec 5 11:23:07 2012 From: pan@REDACTED (Patrik Nyblom) Date: Wed, 5 Dec 2012 11:23:07 +0100 Subject: [erlang-questions] upcoming R16 features? In-Reply-To: References: Message-ID: <50BF208B.3080001@erlang.org> ...oh - and no native processes either. Sorry :( /Patrik On 12/05/2012 09:34 AM, Gustav Simonsson wrote: > OTP Technical Board - Decisions affecting R16: > http://www.erlang.org/news/35 > > Cheers, > Gustav Simonsson > > > On Tue, Dec 4, 2012 at 10:21 PM, Daniel Goertzen > > wrote: > > I can find little information on R16. Could someone in the know > tell us what's coming? (or point me to such info if I've missed > the obvious) I am specifically interested in NIF processes and > frames/structs/dicts. > > Thanks, > Dan. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngocdaothanh@REDACTED Wed Dec 5 11:31:00 2012 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Wed, 5 Dec 2012 19:31:00 +0900 Subject: [erlang-questions] process manager, R16, and 64-bit Macs In-Reply-To: <50BF1EA5.3040201@erlang.org> References: <50BE791E.1050202@simonstl.com> <50BF1EA5.3040201@erlang.org> Message-ID: About the future, what do you think about giving up wxWidgets, and using Nodejs? There are many GUI libraries for Nodejs right now. On Wed, Dec 5, 2012 at 7:15 PM, Patrik Nyblom wrote: > On 12/04/2012 11:28 PM, Simon St.Laurent wrote: >> >> The question about R16 features reminded me that one of the promises of >> R16 is the removal of Process Manager: >> >> "The Pman application has been superseded by the Observer application. >> Pman will be removed in R16." - http://www.erlang.org/doc/man/pman.html >> >> However, Observer does not run at all on 64-bit Macs, because of that >> platform's lack of support for wxWidgets. (The other Erlang GUI apps I've >> tried do run, though the debugger is a bit crash-prone.) R15B3 seems to >> report the same "wxWidgets not found, wx will NOT be usable" message. > > We're working on using the wxWidgets 2.9 "beta-line" for macs, which would > fix the 64bit problem (it's cocoa-based). 2.8 is also almost impossible to > build on mountain lion, so we need to up wxWidgets version on mac anyway. > Unfortunately we have some rather unpleasant stability problems with 2.9 on > the mac, so we might need to reconsider... >> >> >> Is there any chance of Pman getting a delay until support is better? Or >> is this just the cost of progress? > > If we do not have a cross platform wx solution that works for R16, we will > definitely "think again"! >> >> >> (I'm writing Introducing Erlang for O'Reilly, and need to decide if I'm >> ripping out the process manager coverage and replacing it, and if I need to >> keep that coverage around online for customers who happen to use recent >> Macs.) > > Might be a good idea to keep it online... >> >> >> If there's a better place to ask this, please let me know! >> > Nope, erlang-questions is the right place :) >> >> Thanks, > > Cheers, > /Patrik, OTP > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From pan@REDACTED Wed Dec 5 12:31:50 2012 From: pan@REDACTED (Patrik Nyblom) Date: Wed, 5 Dec 2012 12:31:50 +0100 Subject: [erlang-questions] process manager, R16, and 64-bit Macs In-Reply-To: References: <50BE791E.1050202@simonstl.com> <50BF1EA5.3040201@erlang.org> Message-ID: <50BF30A6.3030608@erlang.org> On 12/05/2012 11:31 AM, Ngoc Dao wrote: > About the future, what do you think about giving up wxWidgets, and using Nodejs? > There are many GUI libraries for Nodejs right now. No such plans for the moment. Such a user contribution would be nice though... hint, hint... /Patrik > > On Wed, Dec 5, 2012 at 7:15 PM, Patrik Nyblom wrote: >> On 12/04/2012 11:28 PM, Simon St.Laurent wrote: >>> The question about R16 features reminded me that one of the promises of >>> R16 is the removal of Process Manager: >>> >>> "The Pman application has been superseded by the Observer application. >>> Pman will be removed in R16." - http://www.erlang.org/doc/man/pman.html >>> >>> However, Observer does not run at all on 64-bit Macs, because of that >>> platform's lack of support for wxWidgets. (The other Erlang GUI apps I've >>> tried do run, though the debugger is a bit crash-prone.) R15B3 seems to >>> report the same "wxWidgets not found, wx will NOT be usable" message. >> We're working on using the wxWidgets 2.9 "beta-line" for macs, which would >> fix the 64bit problem (it's cocoa-based). 2.8 is also almost impossible to >> build on mountain lion, so we need to up wxWidgets version on mac anyway. >> Unfortunately we have some rather unpleasant stability problems with 2.9 on >> the mac, so we might need to reconsider... >>> >>> Is there any chance of Pman getting a delay until support is better? Or >>> is this just the cost of progress? >> If we do not have a cross platform wx solution that works for R16, we will >> definitely "think again"! >>> >>> (I'm writing Introducing Erlang for O'Reilly, and need to decide if I'm >>> ripping out the process manager coverage and replacing it, and if I need to >>> keep that coverage around online for customers who happen to use recent >>> Macs.) >> Might be a good idea to keep it online... >>> >>> If there's a better place to ask this, please let me know! >>> >> Nope, erlang-questions is the right place :) >>> Thanks, >> Cheers, >> /Patrik, OTP >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Wed Dec 5 12:46:04 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 05 Dec 2012 12:46:04 +0100 Subject: [erlang-questions] process manager, R16, and 64-bit Macs In-Reply-To: <50BF30A6.3030608@erlang.org> References: <50BE791E.1050202@simonstl.com> <50BF1EA5.3040201@erlang.org> <50BF30A6.3030608@erlang.org> Message-ID: <50BF33FC.5080608@ninenines.eu> On 12/05/2012 12:31 PM, Patrik Nyblom wrote: > On 12/05/2012 11:31 AM, Ngoc Dao wrote: >> About the future, what do you think about giving up wxWidgets, and >> using Nodejs? >> There are many GUI libraries for Nodejs right now. > No such plans for the moment. Such a user contribution would be nice > though... hint, hint... Sure but no nodejs please? Or I wouldn't be able to run it. :) -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From jvalduvieco@REDACTED Wed Dec 5 13:17:48 2012 From: jvalduvieco@REDACTED (Joan Valduvieco Llopart) Date: Wed, 5 Dec 2012 13:17:48 +0100 Subject: [erlang-questions] process manager, R16, and 64-bit Macs In-Reply-To: <50BF33FC.5080608@ninenines.eu> References: <50BE791E.1050202@simonstl.com> <50BF1EA5.3040201@erlang.org> <50BF30A6.3030608@erlang.org> <50BF33FC.5080608@ninenines.eu> Message-ID: I am using observer with mountain lion with wxgtk + xquartz without a glitch. :) I have a macports repo with my packages. https://github.com/jvalduvieco/macports 2012/12/5 Lo?c Hoguin > On 12/05/2012 12:31 PM, Patrik Nyblom wrote: > >> On 12/05/2012 11:31 AM, Ngoc Dao wrote: >> >>> About the future, what do you think about giving up wxWidgets, and >>> using Nodejs? >>> There are many GUI libraries for Nodejs right now. >>> >> No such plans for the moment. Such a user contribution would be nice >> though... hint, hint... >> > > Sure but no nodejs please? Or I wouldn't be able to run it. :) > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth.lundin@REDACTED Wed Dec 5 17:30:33 2012 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 5 Dec 2012 17:30:33 +0100 Subject: [erlang-questions] Erlang/OTP R15B03 has been released In-Reply-To: References: Message-ID: After the release of R15B03 a very unfortunate bug in ssl:ssl_accept/2 (accept with timeout) was detected. This bug will have negative impact on a number of popular http servers and server frameworks written in Erlang. We have because of this decided to release a corrected R15B03 release which will be visible under the tag OTP_R15B03-1 in the source repository at Github. We will also produce new source tar files and new pre-built executables for Windows. In addition to this we will ask Erlang Solutions to produce new packages for Mac OS X and a number of Linux distributions. When we make a new release we will also take the opportunity to make a small change regarding the generation of crashdumps which makes it 100% compatible for users not using heart. More about that in the release notes. We expect to have this ready later this week and will of course announce it on this list. /Kenneth , Erlang/OTP Ericsson On 11/28/12, Kenneth Lundin wrote: > Erlang/OTP R15B03 has been released. This is the third service release for > R15B > > This release mainly contains a number of bug fixes as well as smaller user > contributions (but as > usual there are some new functionality as well). > > One thing worth to mention is a bugfix regarding the use of the "-heart" > in combination with the Erlang crash dump function. > To solve the problem that heart quite often will kill the Erlang VM before > the crashdump is completed we have been forced to > introduce a small potential incompatibility. The incompatibility is that > the environment > variable ERL_CRASH_DUMP_SECONDS must be set in order to get any Erlang > crash dump at all. > See the readme file and the documentation for more details. > > You can find the README file with more detailed info at > http://www.erlang.org/download/otp_src_R15B03.readme > > You can download the full source distribution from > > http://www.erlang.org/download/otp_src_R15B03.tar.gz > http://www.erlang.org/download/otp_src_R15B02.readme (this file) > > Note: To unpack the TAR archive you need a GNU TAR compatible program. > > For installation instructions please read the README that is part of > the distribution. > > You can also find this release at the official Erlang/OTP Git-repository at > Github here: > https://github.com/erlang/otp/tree/OTP_R15B03 (i.e. on the maint branch > tag= OTP_R15B02) > > The Windows binary distribution can be downloaded from > > http://www.erlang.org/download/otp_win32_R15B03.exe > http://www.erlang.org/download/otp_win64_R15B03.exe > > > On-line documentation can be found at http://www.erlang.org/doc/. > You can also download the complete HTML documentation or the Unix manual > files > > http://www.erlang.org/download/otp_doc_html_R15B03.tar.gz > http://www.erlang.org/download/otp_doc_man_R15B03.tar.gz > > We also want to thank those that sent us patches, suggestions and bug > reports, > > The Erlang/OTP Team at Ericsson > From mattevans123@REDACTED Wed Dec 5 19:42:50 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Wed, 5 Dec 2012 13:42:50 -0500 Subject: [erlang-questions] upcoming R16 features? In-Reply-To: <50BF208B.3080001@erlang.org> References: , , <50BF208B.3080001@erlang.org> Message-ID: Any idea when native processes may happen? We have a number of projects that could make use of this (I'd even be willing to play with an experimental version). Thanks Matt Date: Wed, 5 Dec 2012 11:23:07 +0100 From: pan@REDACTED To: erlang-questions@REDACTED Subject: Re: [erlang-questions] upcoming R16 features? ...oh - and no native processes either. Sorry :( /Patrik On 12/05/2012 09:34 AM, Gustav Simonsson wrote: OTP Technical Board - Decisions affecting R16: http://www.erlang.org/news/35 Cheers, Gustav Simonsson On Tue, Dec 4, 2012 at 10:21 PM, Daniel Goertzen wrote: I can find little information on R16. Could someone in the know tell us what's coming? (or point me to such info if I've missed the obvious) I am specifically interested in NIF processes and frames/structs/dicts. Thanks, Dan. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From sadie@REDACTED Thu Dec 6 03:34:14 2012 From: sadie@REDACTED (Sunao Furukawa) Date: Thu, 06 Dec 2012 11:34:14 +0900 Subject: [erlang-questions] Wrangler Problem Message-ID: <50C00426.8060300@kind.ocn.ne.jp> Hello Everyone! It is nice to meet you. I have a problem of Wrangler1.0. I installed Wrangler, but Installer emitted, "No Erlang installation was found on your system.Please ensure that Erlang in installed on your computer before installing Wrangler." I use Erlang(64bit) on Windows8(64bit). I want to use Wrangler in emacs 24.2(Windows 32bit). I installed Erlang c:\erl5.9.3. I will install Wrangler(c:\wrangler). I tried something,but I do not know what to do. Please help me. Thanks. From dch@REDACTED Thu Dec 6 08:18:36 2012 From: dch@REDACTED (Dave Cottlehuber) Date: Thu, 6 Dec 2012 08:18:36 +0100 Subject: [erlang-questions] Wrangler Problem In-Reply-To: <50C00426.8060300@kind.ocn.ne.jp> References: <50C00426.8060300@kind.ocn.ne.jp> Message-ID: On 6 December 2012 03:34, Sunao Furukawa wrote: > Hello Everyone! > It is nice to meet you. > I have a problem of Wrangler1.0. > I installed Wrangler, > but Installer emitted, > "No Erlang installation was found on your system.Please ensure > that Erlang in installed on your computer before installing Wrangler." > I use Erlang(64bit) on Windows8(64bit). > I want to use Wrangler in emacs 24.2(Windows 32bit). > I installed Erlang c:\erl5.9.3. > I will install Wrangler(c:\wrangler). > I tried something,but > I do not know what to do. > Please help me. You may want to add c:\erl5.9.3\bin to your user PATH so that wrangler can find it. From rustompmody@REDACTED Thu Dec 6 08:49:37 2012 From: rustompmody@REDACTED (Rustom Mody) Date: Thu, 6 Dec 2012 13:19:37 +0530 Subject: [erlang-questions] Wrangler Problem In-Reply-To: <50C00426.8060300@kind.ocn.ne.jp> References: <50C00426.8060300@kind.ocn.ne.jp> Message-ID: On Thu, Dec 6, 2012 at 8:04 AM, Sunao Furukawa wrote: > Hello Everyone! > It is nice to meet you. > I have a problem of Wrangler1.0. > I installed Wrangler, > but Installer emitted, > "No Erlang installation was found on your system.Please ensure > that Erlang in installed on your computer before installing Wrangler." > I use Erlang(64bit) on Windows8(64bit). > I want to use Wrangler in emacs 24.2(Windows 32bit). > I installed Erlang c:\erl5.9.3. > I will install Wrangler(c:\wrangler). > I tried something,but > I do not know what to do. > Please help me. > > Thanks. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Good to know about wrangler. Tried it on my debian box, emacs 23.4 erlang-mode version 2.7 Then I do C-c C-r Th echo-area shows a lot of loading this and that from wrangler but the wrangler menu does not appear. This is the elisp setup (for erlang) (add-to-list 'load-path "~/local/lib/erlang/lib/wrangler-1.0/elisp") (require 'wrangler) -- http://www.the-magus.in http://blog.languager.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From H.Li@REDACTED Thu Dec 6 10:28:30 2012 From: H.Li@REDACTED (H.Li@REDACTED) Date: Thu, 6 Dec 2012 09:28:30 -0000 (GMT) Subject: [erlang-questions] Wrangler Problem In-Reply-To: <50C00426.8060300@kind.ocn.ne.jp> References: <50C00426.8060300@kind.ocn.ne.jp> Message-ID: <1c36ca23be568d6b4f866e4341bb16f3.squirrel@webmail.cs.kent.ac.uk> Hi Sunao, I've just updated the Wrangler Windows installer so that the installation of erl5.9.3 can be detected. Here is the link: https://github.com/downloads/RefactoringTools/wrangler/Wrangler_Setup.exe Please let me know if it works. Thanks! Kind Regards, Huiqing > Hello Everyone! > It is nice to meet you. > I have a problem of Wrangler1.0. > I installed Wrangler, > but Installer emitted, > "No Erlang installation was found on your system.Please ensure > that Erlang in installed on your computer before installing Wrangler." > I use Erlang(64bit) on Windows8(64bit). > I want to use Wrangler in emacs 24.2(Windows 32bit). > I installed Erlang c:\erl5.9.3. > I will install Wrangler(c:\wrangler). > I tried something,but > I do not know what to do. > Please help me. > > Thanks. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From pan@REDACTED Thu Dec 6 13:32:21 2012 From: pan@REDACTED (Patrik Nyblom) Date: Thu, 6 Dec 2012 13:32:21 +0100 Subject: [erlang-questions] upcoming R16 features? In-Reply-To: References: , , <50BF208B.3080001@erlang.org> Message-ID: <50C09055.2020706@erlang.org> On 12/05/2012 07:42 PM, Matthew Evans wrote: > Any idea when native processes may happen? No, not really - to far down in the backlog to say anything for sure. > > We have a number of projects that could make use of this (I'd even be > willing to play with an experimental version). Godd, I'll remember that when the time comes - a real world application and beta tester using it is always a good thing! > > Thanks > > Matt > Cheers, /Patrik > ------------------------------------------------------------------------ > Date: Wed, 5 Dec 2012 11:23:07 +0100 > From: pan@REDACTED > To: erlang-questions@REDACTED > Subject: Re: [erlang-questions] upcoming R16 features? > > ...oh - and no native processes either. Sorry :( > > /Patrik > > On 12/05/2012 09:34 AM, Gustav Simonsson wrote: > > OTP Technical Board - Decisions affecting R16: > http://www.erlang.org/news/35 > > Cheers, > Gustav Simonsson > > > On Tue, Dec 4, 2012 at 10:21 PM, Daniel Goertzen > > wrote: > > I can find little information on R16. Could someone in the > know tell us what's coming? (or point me to such info if I've > missed the obvious) I am specifically interested in NIF > processes and frames/structs/dicts. > > Thanks, > Dan. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ erlang-questions > mailing list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Thu Dec 6 14:02:49 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 6 Dec 2012 13:02:49 +0000 Subject: [erlang-questions] upcoming R16 features? In-Reply-To: <50C09055.2020706@erlang.org> References: , , <50BF208B.3080001@erlang.org> <50C09055.2020706@erlang.org> Message-ID: <401CFED2-5BEA-4D44-BFFB-70F667206540@gmail.com> Patrik On 6 Dec 2012, at 12:32, Patrik Nyblom wrote: > On 12/05/2012 07:42 PM, Matthew Evans wrote: >> Any idea when native processes may happen? > No, not really - to far down in the backlog to say anything for sure. >> >> We have a number of projects that could make use of this (I'd even be willing to play with an experimental version). > Godd, I'll remember that when the time comes - a real world application and beta tester using it is always a good thing! >> +1 You can add me to that list too - I have a number of drivers that I'll be keen to convert to native processes asap and working with an experimental brach of OTP is fine. >> Thanks >> >> Matt >> > Cheers, > /Patrik From anthonym@REDACTED Thu Dec 6 20:38:09 2012 From: anthonym@REDACTED (Anthony Molinaro) Date: Thu, 6 Dec 2012 11:38:09 -0800 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? Message-ID: <20121206193809.GC27411@alumni.caltech.edu> Hi, I've been testing out R15B02 on one of our production machines (CentOS 5, erlang compiled from source), and comparing it to the same system running on R14B04. The code is the same (it was all compiled with R14B04, and deployed on both systems). On the R15B02 I see this error crop up every so often 2012-12-01 00:01:33 =CRASH REPORT==== crasher: initial call: mochiweb_acceptor:init/3 pid: <0.2942.3599> registered_name: [] exception error: {{badmatch,{error,einval}},[{mochiweb_http,new_request,3,[]},{mochiweb_http,handle_invalid_request,3,[]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} ancestors: [webmachine_mochiweb,web_serve_sup,<0.619.0>] messages: [{tcp_closed,#Port<0.222190389>}] links: [<0.873.0>] dictionary: [] trap_exit: false status: running heap_size: 987 stack_size: 24 reductions: 541 neighbours: It doesn't happen extremely often (once or twice a minute), but its enough to make me wary of an upgrade. I looked through all the readme's for R15B, R15B01 and R15B02 but didn't find anything which looked like it could be a cause. Anyone have any ideas? Thanks, -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From comptekki@REDACTED Thu Dec 6 20:48:07 2012 From: comptekki@REDACTED (Wes James) Date: Thu, 6 Dec 2012 12:48:07 -0700 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? In-Reply-To: <20121206193809.GC27411@alumni.caltech.edu> References: <20121206193809.GC27411@alumni.caltech.edu> Message-ID: I thought code compiled, say for R14 would work on all R14 releases, but if you want to run on R15, you would need to recompile on R15 for it to work properly. wes On Thu, Dec 6, 2012 at 12:38 PM, Anthony Molinaro < anthonym@REDACTED> wrote: > Hi, > > I've been testing out R15B02 on one of our production machines > (CentOS 5, erlang compiled from source), and comparing it to the > same system running on R14B04. The code is the same (it was all > compiled with R14B04, and deployed on both systems). > > On the R15B02 I see this error crop up every so often > > 2012-12-01 00:01:33 =CRASH REPORT==== > crasher: > initial call: mochiweb_acceptor:init/3 > pid: <0.2942.3599> > registered_name: [] > exception error: > {{badmatch,{error,einval}},[{mochiweb_http,new_request,3,[]},{mochiweb_http,handle_invalid_request,3,[]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} > ancestors: [webmachine_mochiweb,web_serve_sup,<0.619.0>] > messages: [{tcp_closed,#Port<0.222190389>}] > links: [<0.873.0>] > dictionary: [] > trap_exit: false > status: running > heap_size: 987 > stack_size: 24 > reductions: 541 > neighbours: > > It doesn't happen extremely often (once or twice a minute), but its enough > to make me wary of an upgrade. > > I looked through all the readme's for R15B, R15B01 and R15B02 but didn't > find anything which looked like it could be a cause. > > Anyone have any ideas? > > Thanks, > > -Anthony > > -- > ------------------------------------------------------------------------ > Anthony Molinaro > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Thu Dec 6 20:58:36 2012 From: anthonym@REDACTED (Anthony Molinaro) Date: Thu, 6 Dec 2012 11:58:36 -0800 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? In-Reply-To: References: <20121206193809.GC27411@alumni.caltech.edu> Message-ID: <20121206195836.GA27514@alumni.caltech.edu> No, according to Kenneth http://erlang.org/pipermail/erlang-questions/2011-June/059464.html they support at least 2 versions back, so R15B should load and run code compiled with R14B and R13B. And none of my code is compiled with HiPE or native so that's not an issue here. -Anthony On Thu, Dec 06, 2012 at 12:48:07PM -0700, Wes James wrote: > I thought code compiled, say for R14 would work on all R14 releases, but if > you want to run on R15, you would need to recompile on R15 for it to work > properly. > > wes > > > On Thu, Dec 6, 2012 at 12:38 PM, Anthony Molinaro < > anthonym@REDACTED> wrote: > > > Hi, > > > > I've been testing out R15B02 on one of our production machines > > (CentOS 5, erlang compiled from source), and comparing it to the > > same system running on R14B04. The code is the same (it was all > > compiled with R14B04, and deployed on both systems). > > > > On the R15B02 I see this error crop up every so often > > > > 2012-12-01 00:01:33 =CRASH REPORT==== > > crasher: > > initial call: mochiweb_acceptor:init/3 > > pid: <0.2942.3599> > > registered_name: [] > > exception error: > > {{badmatch,{error,einval}},[{mochiweb_http,new_request,3,[]},{mochiweb_http,handle_invalid_request,3,[]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} > > ancestors: [webmachine_mochiweb,web_serve_sup,<0.619.0>] > > messages: [{tcp_closed,#Port<0.222190389>}] > > links: [<0.873.0>] > > dictionary: [] > > trap_exit: false > > status: running > > heap_size: 987 > > stack_size: 24 > > reductions: 541 > > neighbours: > > > > It doesn't happen extremely often (once or twice a minute), but its enough > > to make me wary of an upgrade. > > > > I looked through all the readme's for R15B, R15B01 and R15B02 but didn't > > find anything which looked like it could be a cause. > > > > Anyone have any ideas? > > > > Thanks, > > > > -Anthony > > > > -- > > ------------------------------------------------------------------------ > > Anthony Molinaro > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ------------------------------------------------------------------------ Anthony Molinaro From anders.nygren@REDACTED Thu Dec 6 21:02:20 2012 From: anders.nygren@REDACTED (Anders Nygren) Date: Thu, 6 Dec 2012 14:02:20 -0600 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? In-Reply-To: References: <20121206193809.GC27411@alumni.caltech.edu> Message-ID: No, its the other way around, that code compiled with a newer version is not sure to work on an older system. /Anders On Thu, Dec 6, 2012 at 1:48 PM, Wes James wrote: > I thought code compiled, say for R14 would work on all R14 releases, but if > you want to run on R15, you would need to recompile on R15 for it to work > properly. > > wes > > > > On Thu, Dec 6, 2012 at 12:38 PM, Anthony Molinaro > wrote: >> >> Hi, >> >> I've been testing out R15B02 on one of our production machines >> (CentOS 5, erlang compiled from source), and comparing it to the >> same system running on R14B04. The code is the same (it was all >> compiled with R14B04, and deployed on both systems). >> >> On the R15B02 I see this error crop up every so often >> >> 2012-12-01 00:01:33 =CRASH REPORT==== >> crasher: >> initial call: mochiweb_acceptor:init/3 >> pid: <0.2942.3599> >> registered_name: [] >> exception error: >> {{badmatch,{error,einval}},[{mochiweb_http,new_request,3,[]},{mochiweb_http,handle_invalid_request,3,[]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} >> ancestors: [webmachine_mochiweb,web_serve_sup,<0.619.0>] >> messages: [{tcp_closed,#Port<0.222190389>}] >> links: [<0.873.0>] >> dictionary: [] >> trap_exit: false >> status: running >> heap_size: 987 >> stack_size: 24 >> reductions: 541 >> neighbours: >> >> It doesn't happen extremely often (once or twice a minute), but its enough >> to make me wary of an upgrade. >> >> I looked through all the readme's for R15B, R15B01 and R15B02 but didn't >> find anything which looked like it could be a cause. >> >> Anyone have any ideas? >> >> Thanks, >> >> -Anthony >> >> -- >> ------------------------------------------------------------------------ >> Anthony Molinaro >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From david@REDACTED Thu Dec 6 22:43:22 2012 From: david@REDACTED (David Fox) Date: Thu, 06 Dec 2012 15:43:22 -0600 Subject: [erlang-questions] Help creating distributed server cache Message-ID: <50C1117A.4000008@davidjfox.com> I'm currently developing a gaming server which stores player information that can be accessed from any of our games via a REST API. So far I've thought of two ways to structure and cache player data: 1. When a client requests data on a player, spawn 1 player process. This process handles: all subsequent requests from clients for this player, retrieving the player data from the DB when created and periodically updating the DB with any new data from clients. If the player is not requested by another client within... say 30 minutes, the player process will terminate. 2. Just keep previously requested data in a distributed LRU cache (e.g., memcached, redis, mnesia) Out of the two, I prefer #1 since it would allow me to separate the functionality of different "data types" (e.g., player data, game data). There are just 2 problems with doing it this way that I'd like your thoughts and help with: I. I would have to implement some sort of "LRU process cache" so I could terminate processes to free memory for new ones. II. If a load balancer connects a client to node #1, but the process for the requested player is on node #2, how can the player process on node #2 send the data to the socket opened for the client on node #1. Is it possible to somehow send an socket across nodes? The reason I ask, is that I'd like to prevent sending big messages across nodes. Thanks for the help! From anthonym@REDACTED Thu Dec 6 22:53:21 2012 From: anthonym@REDACTED (Anthony Molinaro) Date: Thu, 6 Dec 2012 13:53:21 -0800 Subject: [erlang-questions] Help creating distributed server cache In-Reply-To: <50C1117A.4000008@davidjfox.com> References: <50C1117A.4000008@davidjfox.com> Message-ID: <20121206215321.GB27911@alumni.caltech.edu> You might consider just using riak with the memory backend. It already has a REST API http://docs.basho.com/riak/latest/tutorials/querying/Basic-Operations/ and with the memory backend you can set a ttl http://docs.basho.com/riak/latest/tutorials/choosing-a-backend/Memory/ There's no need to keep processes running or do much of anything other than install it on a few machines, configure it and go. -Anthony On Thu, Dec 06, 2012 at 03:43:22PM -0600, David Fox wrote: > I'm currently developing a gaming server which stores player > information that can be accessed from any of our games via a REST > API. > > So far I've thought of two ways to structure and cache player data: > > 1. When a client requests data on a player, spawn 1 player process. > This process handles: all subsequent requests from clients for this > player, retrieving the player data from the DB when created and > periodically updating the DB with any new data from clients. If the > player is not requested by another client within... say 30 minutes, > the player process will terminate. > > 2. Just keep previously requested data in a distributed LRU cache > (e.g., memcached, redis, mnesia) > > Out of the two, I prefer #1 since it would allow me to separate the > functionality of different "data types" (e.g., player data, game > data). > > There are just 2 problems with doing it this way that I'd like your > thoughts and help with: > I. I would have to implement some sort of "LRU process cache" so I > could terminate processes to free memory for new ones. > II. If a load balancer connects a client to node #1, but the process > for the requested player is on node #2, how can the player process > on node #2 send the data to the socket opened for the client on node > #1. Is it possible to somehow send an socket across nodes? The > reason I ask, is that I'd like to prevent sending big messages > across nodes. > > Thanks for the help! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ------------------------------------------------------------------------ Anthony Molinaro From david@REDACTED Thu Dec 6 23:26:43 2012 From: david@REDACTED (David Fox) Date: Thu, 06 Dec 2012 16:26:43 -0600 Subject: [erlang-questions] Help creating distributed server cache In-Reply-To: <20121206215321.GB27911@alumni.caltech.edu> References: <50C1117A.4000008@davidjfox.com> <20121206215321.GB27911@alumni.caltech.edu> Message-ID: <50C11BA3.3040200@davidjfox.com> So you'd suggest going with #2 instead of #1? Could you tell me why? Thanks for pointing out riak's memory backend. Had forgotten about that :) -David On 12/6/2012 15:53, Anthony Molinaro wrote: > You might consider just using riak with the memory backend. It already > has a REST API > > http://docs.basho.com/riak/latest/tutorials/querying/Basic-Operations/ > > and with the memory backend you can set a ttl > > http://docs.basho.com/riak/latest/tutorials/choosing-a-backend/Memory/ > > There's no need to keep processes running or do much of anything > other than install it on a few machines, configure it and go. > > -Anthony > > On Thu, Dec 06, 2012 at 03:43:22PM -0600, David Fox wrote: >> I'm currently developing a gaming server which stores player >> information that can be accessed from any of our games via a REST >> API. >> >> So far I've thought of two ways to structure and cache player data: >> >> 1. When a client requests data on a player, spawn 1 player process. >> This process handles: all subsequent requests from clients for this >> player, retrieving the player data from the DB when created and >> periodically updating the DB with any new data from clients. If the >> player is not requested by another client within... say 30 minutes, >> the player process will terminate. >> >> 2. Just keep previously requested data in a distributed LRU cache >> (e.g., memcached, redis, mnesia) >> >> Out of the two, I prefer #1 since it would allow me to separate the >> functionality of different "data types" (e.g., player data, game >> data). >> >> There are just 2 problems with doing it this way that I'd like your >> thoughts and help with: >> I. I would have to implement some sort of "LRU process cache" so I >> could terminate processes to free memory for new ones. >> II. If a load balancer connects a client to node #1, but the process >> for the requested player is on node #2, how can the player process >> on node #2 send the data to the socket opened for the client on node >> #1. Is it possible to somehow send an socket across nodes? The >> reason I ask, is that I'd like to prevent sending big messages >> across nodes. >> >> Thanks for the help! >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Thu Dec 6 23:34:18 2012 From: g@REDACTED (Garrett Smith) Date: Thu, 6 Dec 2012 22:34:18 +0000 Subject: [erlang-questions] Help creating distributed server cache In-Reply-To: <50C1117A.4000008@davidjfox.com> References: <50C1117A.4000008@davidjfox.com> Message-ID: Hi David, On Thu, Dec 6, 2012 at 9:43 PM, David Fox wrote: > I'm currently developing a gaming server which stores player information > that can be accessed from any of our games via a REST API. > > So far I've thought of two ways to structure and cache player data: > > 1. When a client requests data on a player, spawn 1 player process. This > process handles: all subsequent requests from clients for this player, > retrieving the player data from the DB when created and periodically > updating the DB with any new data from clients. If the player is not > requested by another client within... say 30 minutes, the player process > will terminate. > > 2. Just keep previously requested data in a distributed LRU cache (e.g., > memcached, redis, mnesia) > > Out of the two, I prefer #1 since it would allow me to separate the > functionality of different "data types" (e.g., player data, game data). > > There are just 2 problems with doing it this way that I'd like your thoughts > and help with: > I. I would have to implement some sort of "LRU process cache" so I could > terminate processes to free memory for new ones. > II. If a load balancer connects a client to node #1, but the process for the > requested player is on node #2, how can the player process on node #2 send > the data to the socket opened for the client on node #1. Is it possible to > somehow send an socket across nodes? The reason I ask, is that I'd like to > prevent sending big messages across nodes. It's tough to answer high level "approach" style questions, especially without some hands on work (tinkering) to help you understand the problem. Limiting yourself to either/or options at this stage might also be premature. Do you have a first pass at the public API for this service? If you have an idea of the functions that could define the interface, you can ask, for each unimplemented function: - Can I make this side-effect free -- i.e. calling the function doesn't change state or otherwise tamper with the universe? - Does the function read from or write to long running state? Side effect free functions are easy, which is why you try to solve problems using them exclusively whenever possible. For long running state, you can use a simple gen_server to implement state initialization and mutation. If you have questions about what I mean here, you'll need to bone up on gen_server, or alternatively look at e2 services (see http://e2project.org) as they're simpler to write. Once you have something very basic working, see if you're done! It might just work for you as is, at least for the short term. If it doesn't work, address the specific problem. E.g. if your problem is "I loose my state when the VM crahses," you'll need to implement persistence in some fashion. Questions at that level are much easier to answer :) Garrett From david@REDACTED Fri Dec 7 00:01:40 2012 From: david@REDACTED (David Fox) Date: Thu, 06 Dec 2012 17:01:40 -0600 Subject: [erlang-questions] Help creating distributed server cache In-Reply-To: References: <50C1117A.4000008@davidjfox.com> Message-ID: <50C123D4.1080008@davidjfox.com> Hi Garret, thanks for the response. I have not yet finished implementing the API; I'm still in the design phase figuring out how everything should be hooked up. Totally agree on not limiting yourself to options this early on, I'm just asking for some help and opinions on some problems I saw in potential implementations :) David Fox m: 630 930 9219 Chicago On 12/6/2012 16:34, Garrett Smith wrote: > Hi David, > > On Thu, Dec 6, 2012 at 9:43 PM, David Fox wrote: >> I'm currently developing a gaming server which stores player information >> that can be accessed from any of our games via a REST API. >> >> So far I've thought of two ways to structure and cache player data: >> >> 1. When a client requests data on a player, spawn 1 player process. This >> process handles: all subsequent requests from clients for this player, >> retrieving the player data from the DB when created and periodically >> updating the DB with any new data from clients. If the player is not >> requested by another client within... say 30 minutes, the player process >> will terminate. >> >> 2. Just keep previously requested data in a distributed LRU cache (e.g., >> memcached, redis, mnesia) >> >> Out of the two, I prefer #1 since it would allow me to separate the >> functionality of different "data types" (e.g., player data, game data). >> >> There are just 2 problems with doing it this way that I'd like your thoughts >> and help with: >> I. I would have to implement some sort of "LRU process cache" so I could >> terminate processes to free memory for new ones. >> II. If a load balancer connects a client to node #1, but the process for the >> requested player is on node #2, how can the player process on node #2 send >> the data to the socket opened for the client on node #1. Is it possible to >> somehow send an socket across nodes? The reason I ask, is that I'd like to >> prevent sending big messages across nodes. > It's tough to answer high level "approach" style questions, especially > without some hands on work (tinkering) to help you understand the > problem. > > Limiting yourself to either/or options at this stage might also be premature. > > Do you have a first pass at the public API for this service? > > If you have an idea of the functions that could define the interface, > you can ask, for each unimplemented function: > > - Can I make this side-effect free -- i.e. calling the function > doesn't change state or otherwise tamper with the universe? > > - Does the function read from or write to long running state? > > Side effect free functions are easy, which is why you try to solve > problems using them exclusively whenever possible. > > For long running state, you can use a simple gen_server to implement > state initialization and mutation. If you have questions about what I > mean here, you'll need to bone up on gen_server, or alternatively look > at e2 services (see http://e2project.org) as they're simpler to write. > > Once you have something very basic working, see if you're done! It > might just work for you as is, at least for the short term. If it > doesn't work, address the specific problem. E.g. if your problem is "I > loose my state when the VM crahses," you'll need to implement > persistence in some fashion. > > Questions at that level are much easier to answer :) > > Garrett From g@REDACTED Fri Dec 7 00:26:54 2012 From: g@REDACTED (Garrett Smith) Date: Thu, 6 Dec 2012 17:26:54 -0600 Subject: [erlang-questions] Help creating distributed server cache In-Reply-To: <50C123D4.1080008@davidjfox.com> References: <50C1117A.4000008@davidjfox.com> <50C123D4.1080008@davidjfox.com> Message-ID: On Thu, Dec 6, 2012 at 5:01 PM, David Fox wrote: > Hi Garret, thanks for the response. > > I have not yet finished implementing the API; I'm still in the design phase > figuring out how everything should be hooked up. Right. though I think you can start to build actual pieces (i.e. compiling/running code) based on what's the most obvious to you, even if it's just super stupidly simple. It will give you a basis for iteration, which will further you help your understanding. You may find yourself spending almost no time designing :) > Totally agree on not limiting yourself to options this early on, I'm just > asking for some help and opinions on some problems I saw in potential > implementations :) > > David Fox > m: 630 930 9219 > Chicago Ah, I'm also in Chicago. Do you know about the Chicago Erlang User Group: http://www.meetup.com/ErlangChicago/ We haven't met the last few months, but I'd like to do an informal meetup before years end! > On 12/6/2012 16:34, Garrett Smith wrote: >> >> Hi David, >> >> On Thu, Dec 6, 2012 at 9:43 PM, David Fox wrote: >>> >>> I'm currently developing a gaming server which stores player information >>> that can be accessed from any of our games via a REST API. >>> >>> So far I've thought of two ways to structure and cache player data: >>> >>> 1. When a client requests data on a player, spawn 1 player process. This >>> process handles: all subsequent requests from clients for this player, >>> retrieving the player data from the DB when created and periodically >>> updating the DB with any new data from clients. If the player is not >>> requested by another client within... say 30 minutes, the player process >>> will terminate. >>> >>> 2. Just keep previously requested data in a distributed LRU cache (e.g., >>> memcached, redis, mnesia) >>> >>> Out of the two, I prefer #1 since it would allow me to separate the >>> functionality of different "data types" (e.g., player data, game data). >>> >>> There are just 2 problems with doing it this way that I'd like your >>> thoughts >>> and help with: >>> I. I would have to implement some sort of "LRU process cache" so I could >>> terminate processes to free memory for new ones. >>> II. If a load balancer connects a client to node #1, but the process for >>> the >>> requested player is on node #2, how can the player process on node #2 >>> send >>> the data to the socket opened for the client on node #1. Is it possible >>> to >>> somehow send an socket across nodes? The reason I ask, is that I'd like >>> to >>> prevent sending big messages across nodes. >> >> It's tough to answer high level "approach" style questions, especially >> without some hands on work (tinkering) to help you understand the >> problem. >> >> Limiting yourself to either/or options at this stage might also be >> premature. >> >> Do you have a first pass at the public API for this service? >> >> If you have an idea of the functions that could define the interface, >> you can ask, for each unimplemented function: >> >> - Can I make this side-effect free -- i.e. calling the function >> doesn't change state or otherwise tamper with the universe? >> >> - Does the function read from or write to long running state? >> >> Side effect free functions are easy, which is why you try to solve >> problems using them exclusively whenever possible. >> >> For long running state, you can use a simple gen_server to implement >> state initialization and mutation. If you have questions about what I >> mean here, you'll need to bone up on gen_server, or alternatively look >> at e2 services (see http://e2project.org) as they're simpler to write. >> >> Once you have something very basic working, see if you're done! It >> might just work for you as is, at least for the short term. If it >> doesn't work, address the specific problem. E.g. if your problem is "I >> loose my state when the VM crahses," you'll need to implement >> persistence in some fashion. >> >> Questions at that level are much easier to answer :) >> >> Garrett > > From steven.charles.davis@REDACTED Fri Dec 7 01:42:05 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 6 Dec 2012 16:42:05 -0800 (PST) Subject: [erlang-questions] Help creating distributed server cache In-Reply-To: <50C1117A.4000008@davidjfox.com> References: <50C1117A.4000008@davidjfox.com> Message-ID: <51c212fa-185d-45e1-ad97-815ce1cde94f@googlegroups.com> >From what you've said, I would guess that the correct answer is: 2) memcached protocol Since Your solution (1) starts with "When a client requests data on a player, spawn 1 player process." If that had been "when a client requests data on themselves from another game" then it could have been in the running... A memcached implementation will sort out LRU without you having to reinvent (stabilize, test) a wheel. Not sure why there's a REST requirement. If this MUST be HTTP then I see it, otherwise what does it do for you? My 2c, /s On Thursday, December 6, 2012 3:43:22 PM UTC-6, David Fox wrote: > > I'm currently developing a gaming server which stores player information > that can be accessed from any of our games via a REST API. > > So far I've thought of two ways to structure and cache player data: > > 1. When a client requests data on a player, spawn 1 player process. This > process handles: all subsequent requests from clients for this player, > retrieving the player data from the DB when created and periodically > updating the DB with any new data from clients. If the player is not > requested by another client within... say 30 minutes, the player process > will terminate. > > 2. Just keep previously requested data in a distributed LRU cache (e.g., > memcached, redis, mnesia) > > Out of the two, I prefer #1 since it would allow me to separate the > functionality of different "data types" (e.g., player data, game data). > > There are just 2 problems with doing it this way that I'd like your > thoughts and help with: > I. I would have to implement some sort of "LRU process cache" so I could > terminate processes to free memory for new ones. > II. If a load balancer connects a client to node #1, but the process for > the requested player is on node #2, how can the player process on node > #2 send the data to the socket opened for the client on node #1. Is it > possible to somehow send an socket across nodes? The reason I ask, is > that I'd like to prevent sending big messages across nodes. > > Thanks for the help! > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Fri Dec 7 02:20:55 2012 From: anthonym@REDACTED (Anthony Molinaro) Date: Thu, 6 Dec 2012 17:20:55 -0800 Subject: [erlang-questions] Help creating distributed server cache In-Reply-To: <50C11BA3.3040200@davidjfox.com> References: <50C1117A.4000008@davidjfox.com> <20121206215321.GB27911@alumni.caltech.edu> <50C11BA3.3040200@davidjfox.com> Message-ID: I mention riak because it gets you furthest quickest. It solves the problems you outlined (REST API and LRU style caching), while avoiding the problems (assuming you use a player id for your key riak will route requests to the appropriate node, and it has LRU style caching). In addition you get distributed caching which works even when nodes go down. So it just seemed to fit your problem as described. -Anthony On Dec 6, 2012, at 2:26 PM, David Fox wrote: > So you'd suggest going with #2 instead of #1? Could you tell me why? > > Thanks for pointing out riak's memory backend. Had forgotten about that :) > > -David > > On 12/6/2012 15:53, Anthony Molinaro wrote: >> You might consider just using riak with the memory backend. It already >> has a REST API >> >> http://docs.basho.com/riak/latest/tutorials/querying/Basic-Operations/ >> >> and with the memory backend you can set a ttl >> >> http://docs.basho.com/riak/latest/tutorials/choosing-a-backend/Memory/ >> >> There's no need to keep processes running or do much of anything >> other than install it on a few machines, configure it and go. >> >> -Anthony >> >> On Thu, Dec 06, 2012 at 03:43:22PM -0600, David Fox wrote: >>> I'm currently developing a gaming server which stores player >>> information that can be accessed from any of our games via a REST >>> API. >>> >>> So far I've thought of two ways to structure and cache player data: >>> >>> 1. When a client requests data on a player, spawn 1 player process. >>> This process handles: all subsequent requests from clients for this >>> player, retrieving the player data from the DB when created and >>> periodically updating the DB with any new data from clients. If the >>> player is not requested by another client within... say 30 minutes, >>> the player process will terminate. >>> >>> 2. Just keep previously requested data in a distributed LRU cache >>> (e.g., memcached, redis, mnesia) >>> >>> Out of the two, I prefer #1 since it would allow me to separate the >>> functionality of different "data types" (e.g., player data, game >>> data). >>> >>> There are just 2 problems with doing it this way that I'd like your >>> thoughts and help with: >>> I. I would have to implement some sort of "LRU process cache" so I >>> could terminate processes to free memory for new ones. >>> II. If a load balancer connects a client to node #1, but the process >>> for the requested player is on node #2, how can the player process >>> on node #2 send the data to the socket opened for the client on node >>> #1. Is it possible to somehow send an socket across nodes? The >>> reason I ask, is that I'd like to prevent sending big messages >>> across nodes. >>> >>> Thanks for the help! >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From sadie@REDACTED Fri Dec 7 05:43:10 2012 From: sadie@REDACTED (Sunao Furukawa) Date: Fri, 07 Dec 2012 13:43:10 +0900 Subject: [erlang-questions] Wrangler Problem In-Reply-To: References: <50C00426.8060300@kind.ocn.ne.jp> Message-ID: <50C173DE.7010402@kind.ocn.ne.jp> (2012/12/06 16:49), Rustom Mody wrote: > Good to know about wrangler. > Tried it on my debian box, > emacs 23.4 > erlang-mode version 2.7 > > Then I do C-c C-r > Th echo-area shows a lot of loading this and that from wrangler but > the wrangler menu does not appear. > > This is the elisp setup (for erlang) > > (add-to-list 'load-path > "~/local/lib/erlang/lib/wrangler-1.0/elisp") > (require 'wrangler) I have the same problem. I installed R14B04(32bit) on windows8(64bit). Wrangler's installer discovered R14B04. I did C-c C-r on emacs24.2(32bit) ,but the wrangler menu does not appear. And also elisp setup have problem. Wrangler's installer add .emacs (add-to-list 'load-path "/elisp") (setq exec-path (cons "/bin" exec-path)) (require 'wrangler) . But Wrangler directory does not have bin. Alternatively,It has ebin. Installer mistakes its directory and internal error exists. I have no idea. Please help me. Thanks! From rustompmody@REDACTED Fri Dec 7 07:30:04 2012 From: rustompmody@REDACTED (Rustom Mody) Date: Fri, 7 Dec 2012 12:00:04 +0530 Subject: [erlang-questions] Wrangler Problem In-Reply-To: <50C173DE.7010402@kind.ocn.ne.jp> References: <50C00426.8060300@kind.ocn.ne.jp> <50C173DE.7010402@kind.ocn.ne.jp> Message-ID: On Fri, Dec 7, 2012 at 10:13 AM, Sunao Furukawa wrote: > (2012/12/06 16:49), Rustom Mody wrote: > >> Good to know about wrangler. >> Tried it on my debian box, >> emacs 23.4 >> erlang-mode version 2.7 >> >> Then I do C-c C-r >> Th echo-area shows a lot of loading this and that from wrangler but the >> wrangler menu does not appear. >> >> This is the elisp setup (for erlang) >> >> (add-to-list 'load-path >> "~/local/lib/erlang/lib/**wrangler-1.0/elisp") >> (require 'wrangler) >> > > I have the same problem. > I installed R14B04(32bit) on windows8(64bit). > Wrangler's installer discovered R14B04. > I did C-c C-r on emacs24.2(32bit) ,but the wrangler menu does not appear. > And also elisp setup have problem. > Wrangler's installer add .emacs > (add-to-list 'load-path "/elisp") > (setq exec-path (cons "/bin" exec-path)) > (require 'wrangler) > . > But Wrangler directory does not have bin. > Alternatively,It has ebin. > Installer mistakes its directory and internal error exists. > I have no idea. > Please help me. > > Thanks! > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > On Thu, Dec 6, 2012 at 9:31 PM, Rustom Mody wrote: > > > On Thu, Dec 6, 2012 at 2:29 PM, wrote: > >> >> Hi Rustom, >> >> What about is you try the command: >> >> Do M-X erlang-wrangler-on Did you get load messages? Do M-X wrangler-menu-init Does the wrangler menu entry appear? [After that it does not work for me either!!] -------------- next part -------------- An HTML attachment was scrubbed... URL: From yashgt@REDACTED Fri Dec 7 10:46:01 2012 From: yashgt@REDACTED (Yash Ganthe) Date: Fri, 07 Dec 2012 04:46:01 -0500 Subject: [erlang-questions] Erlang and Websphere MQ Message-ID: Hi, What is the best way to read from and write to IBM Websphere MQ using Erlang. Are there any modules available for doing that? We would have chosen Rabbit MQ. However the customer already has Websphere MQ in their existing systems. Thanks and regards, Yash -- Using Opera's revolutionary email client: http://www.opera.com/mail/ From vinayakapawar@REDACTED Fri Dec 7 11:00:57 2012 From: vinayakapawar@REDACTED (Vinayak Pawar) Date: Fri, 7 Dec 2012 15:30:57 +0530 Subject: [erlang-questions] Erlang and Websphere MQ In-Reply-To: References: Message-ID: How about using Websphere MQ HTTP bridge? The other option is writing Erlang port driver using their C/C++ API. Regards, Vinayak On Fri, Dec 7, 2012 at 3:16 PM, Yash Ganthe wrote: > Hi, > > What is the best way to read from and write to IBM Websphere MQ using > Erlang. Are there any modules available for doing that? We would have > chosen Rabbit MQ. However the customer already has Websphere MQ in their > existing systems. > > Thanks and regards, > Yash > > -- > Using Opera's revolutionary email client: http://www.opera.com/mail/ > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From naikvin@REDACTED Fri Dec 7 14:43:48 2012 From: naikvin@REDACTED (Vineet Naik) Date: Fri, 7 Dec 2012 19:13:48 +0530 Subject: [erlang-questions] Recommended approach for dependency management during development Message-ID: Hello, I trying to write my first non trivial Erlang/OTP application and looking for the best approach for organizing and managing dependencies during development. So far there has been a single dependency for the project which is the exmpp library[1]. Upon installing exmpp, it copied compiled code to the search path (/usr/lib/erlang/lib/) so I did not have to care while using it in my code. But now I need to use erlang-mysql-driver[2] too and it doesn't come with an installation script like exmpp that would copy compiled files to the search path. Presently I can think of following ways to fix this: 1. Add hard coded paths of src & include dirs of the libraries in my Emakefile (disadvantage: wouldn't work on other machines) 2. Compile the code from the libraries and copy the beam files over to the ebin dir of my project. I need to do this only once as I don't plan to change their code. 3. Add search paths to the command every time while starting the shell. Does rebar handle this and should I be using it instead of the Emakefile? Sorry if my questions are not clear, I am really confused about this :-) [1] https://github.com/processone/exmpp [2] https://github.com/dizzyd/erlang-mysql-driver Thanks, Vineet -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Fri Dec 7 15:10:13 2012 From: g@REDACTED (Garrett Smith) Date: Fri, 7 Dec 2012 08:10:13 -0600 Subject: [erlang-questions] Recommended approach for dependency management during development In-Reply-To: References: Message-ID: Hi Vineet, On Fri, Dec 7, 2012 at 7:43 AM, Vineet Naik wrote: > Hello, > > I trying to write my first non trivial Erlang/OTP application and > looking for the best approach for organizing and managing > dependencies during development. Good question! > So far there has been a single dependency for the project which > is the exmpp library[1]. Upon installing exmpp, it copied > compiled code to the search path (/usr/lib/erlang/lib/) so I did > not have to care while using it in my code. > > But now I need to use erlang-mysql-driver[2] too and it doesn't > come with an installation script like exmpp that would copy > compiled files to the search path. > > Presently I can think of following ways to fix this: > > 1. Add hard coded paths of src & include dirs of the libraries in > my Emakefile (disadvantage: wouldn't work on other machines) > > 2. Compile the code from the libraries and copy the beam files > over to the ebin dir of my project. I need to do this only once > as I don't plan to change their code. > > 3. Add search paths to the command every time while starting the > shell. > > Does rebar handle this and should I be using it instead of the > Emakefile? I would approach this in two passes: 1. Start using rebar to manage your dependencies for local development 2. Use Erlang releases to bundle your application for deployment At build time, rebar will download and build dependencies in a local "deps" dir. You can configure your shell start script to set ERL_LIBS to deps to make those applications available from the shell. There's no need to touch your system wide Erlang/OTP installation. But that's strictly for development. For production (deployment outside your dev environment) you really want releases. Releases work very well. The main Erlang distribution itself is a release. The problem is building them. I hear rebar works for this now, but I haven't use it. I use a Makefile and Erlang reltool, systools -- after a lot of work hacking through what works and what doesn't, total PITA. If rebar creates correct, clean releases now, I'd definitely use that instead. This is a good overview of releases: http://www.erlang.org/doc/design_principles/release_structure.html Garrett From desired.mta@REDACTED Fri Dec 7 15:30:40 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Fri, 7 Dec 2012 16:30:40 +0200 Subject: [erlang-questions] Recommended approach for dependency management during development In-Reply-To: References: Message-ID: And to list. On Fri, Dec 7, 2012 at 4:30 PM, Motiejus Jak?tys wrote: > On Fri, Dec 7, 2012 at 4:10 PM, Garrett Smith wrote: >> >> The problem is building them. >> >> I hear rebar works for this now, but I haven't use it. I use a >> Makefile and Erlang reltool, systools -- after a lot of work hacking >> through what works and what doesn't, total PITA. If rebar creates >> correct, clean releases now, I'd definitely use that instead. >> >> This is a good overview of releases: >> >> http://www.erlang.org/doc/design_principles/release_structure.html > > Releases with rebar are embarrassingly easy, given you structure your > program properly. Look for guides on the net -- I am sure learning it > from scratch will take less than a few hours, but it will definitely > be worth the effort. > > -- > Motiejus Jak?tys -- Motiejus Jak?tys From naikvin@REDACTED Fri Dec 7 15:32:54 2012 From: naikvin@REDACTED (Vineet Naik) Date: Fri, 7 Dec 2012 20:02:54 +0530 Subject: [erlang-questions] Recommended approach for dependency management during development In-Reply-To: References: Message-ID: Hi Garrett, On Fri, Dec 7, 2012 at 7:40 PM, Garrett Smith wrote: > [...] > > > At build time, rebar will download and build dependencies in a local > "deps" dir. > Does it require that the dependencies be rebar compatible as well. Right now I am having exmpp which doesn't use rebar and I guess doesn't have the directory structure suitable for it. With exmpp installed in default erlang search path, it's not an issue for me at least during development but frankly I would be more happy with a sandboxed environment similar to Python's virtualenv. Glad that rebar downloads and builds dependencies local to the project. Will try if it works for non-rebar-compatible libs too. Thanks for your reply, it helped a lot :-) Regards, Vineet -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Fri Dec 7 15:48:03 2012 From: g@REDACTED (Garrett Smith) Date: Fri, 7 Dec 2012 08:48:03 -0600 Subject: [erlang-questions] Recommended approach for dependency management during development In-Reply-To: References: Message-ID: On Fri, Dec 7, 2012 at 8:32 AM, Vineet Naik wrote: > Hi Garrett, > > On Fri, Dec 7, 2012 at 7:40 PM, Garrett Smith wrote: >> >> [...] >> >> >> At build time, rebar will download and build dependencies in a local >> "deps" dir. > > > Does it require that the dependencies be rebar compatible as well. Right > now I am having exmpp which doesn't use rebar and I guess doesn't have > the directory structure suitable for it. Briefly looking at this, you'll have a problem following my advice with rebar for this case -- that project has a complex build. I spotted this thread: https://support.process-one.net/browse/EXMPP-51 which points to this project: https://github.com/archaelus/exmpp I tried building that using "rebar compile" but hit some errors. If that works for you, just list that fork for your rebar dependency. Otherwise, you could troubleshoot the problem or, just build the project yourself using whatever works from a Makefile -- and put it into a local "deps" dir. Other required apps will land there from rebar builds and you'll have a single dir for ERL_LIBS. As for building a release, it sounds like rebar has come of age. But exmpp may not be friendly. Could be a good area to help contribute to with patches. > With exmpp installed in default erlang search path, it's not an issue for > me at least during development but frankly I would be more happy with a > sandboxed environment similar to Python's virtualenv. Glad that rebar > downloads and builds dependencies local to the project. Will try if it works > for non-rebar-compatible libs too. > > Thanks for your reply, it helped a lot :-) You're welcome! Garrett From pan@REDACTED Fri Dec 7 15:58:45 2012 From: pan@REDACTED (Patrik Nyblom) Date: Fri, 7 Dec 2012 15:58:45 +0100 Subject: [erlang-questions] Erlang/OTP R15B03 has been released In-Reply-To: References: Message-ID: <50C20425.8080509@erlang.org> Hi all! The update of R15B03, dubbed R15B03-1, is now available for download at erlang.org. The tag OTP_R15B03-1 is also on github. Note that this update does not change the system version, it's still R15B03, but the ssl and erts applications have new version numbers. From the README: --- ssl-5.1.2 ----------------------------------------------------------- OTP-10600 ssl:ssl_accept/2 timeout is no longer ignored --- erts-5.9.3.1 -------------------------------------------------------- OTP-10602 Create an erl_crash.dump if no heart exists and no ERL_CRASH_DUMP_SECONDS is set (behaviour changed). Don't create an erl_crash.dump if heart do exists and no ERL_CRASH_DUMP_SECONDS is set (behaviour not changed). This changes the behaviour back to the R15B02 default considering if a beam was running with no heart. As stated before, the ssl bug is the reason for this unusual update of the release. Please update your installations to avoid unpleasant surprises when dealing with ssl (or crashdumps)! Best regards, The Erlang/OTP team On 12/05/2012 05:30 PM, Kenneth Lundin wrote: > After the release of R15B03 a very unfortunate bug in ssl:ssl_accept/2 > (accept with timeout) was detected. This bug will have negative impact > on a number of popular http servers and server frameworks written in > Erlang. > > We have because of this decided to release a corrected R15B03 release > which will be > visible under the tag OTP_R15B03-1 in the source repository at Github. > > We will also produce new source tar files and new pre-built > executables for Windows. > In addition to this we will ask Erlang Solutions to produce new > packages for Mac OS X > and a number of Linux distributions. > > When we make a new release we will also take the opportunity to make a > small change regarding the generation of crashdumps which makes it > 100% compatible for users not using heart. More about that in the > release notes. > > We expect to have this ready later this week and will of course > announce it on this list. > > /Kenneth , Erlang/OTP Ericsson > > On 11/28/12, Kenneth Lundin wrote: >> Erlang/OTP R15B03 has been released. This is the third service release for >> R15B >> >> This release mainly contains a number of bug fixes as well as smaller user >> contributions (but as >> usual there are some new functionality as well). >> >> One thing worth to mention is a bugfix regarding the use of the "-heart" >> in combination with the Erlang crash dump function. >> To solve the problem that heart quite often will kill the Erlang VM before >> the crashdump is completed we have been forced to >> introduce a small potential incompatibility. The incompatibility is that >> the environment >> variable ERL_CRASH_DUMP_SECONDS must be set in order to get any Erlang >> crash dump at all. >> See the readme file and the documentation for more details. >> >> You can find the README file with more detailed info at >> http://www.erlang.org/download/otp_src_R15B03.readme >> >> You can download the full source distribution from >> >> http://www.erlang.org/download/otp_src_R15B03.tar.gz >> http://www.erlang.org/download/otp_src_R15B02.readme (this file) >> >> Note: To unpack the TAR archive you need a GNU TAR compatible program. >> >> For installation instructions please read the README that is part of >> the distribution. >> >> You can also find this release at the official Erlang/OTP Git-repository at >> Github here: >> https://github.com/erlang/otp/tree/OTP_R15B03 (i.e. on the maint branch >> tag= OTP_R15B02) >> >> The Windows binary distribution can be downloaded from >> >> http://www.erlang.org/download/otp_win32_R15B03.exe >> http://www.erlang.org/download/otp_win64_R15B03.exe >> >> >> On-line documentation can be found at http://www.erlang.org/doc/. >> You can also download the complete HTML documentation or the Unix manual >> files >> >> http://www.erlang.org/download/otp_doc_html_R15B03.tar.gz >> http://www.erlang.org/download/otp_doc_man_R15B03.tar.gz >> >> We also want to thank those that sent us patches, suggestions and bug >> reports, >> >> The Erlang/OTP Team at Ericsson >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From naikvin@REDACTED Fri Dec 7 16:48:58 2012 From: naikvin@REDACTED (Vineet Naik) Date: Fri, 7 Dec 2012 21:18:58 +0530 Subject: [erlang-questions] Recommended approach for dependency management during development In-Reply-To: References: Message-ID: Just found out that rebar supports "raw" repositories[1] which will only be fetched but ignored during compilation. So after running `get-deps`, I had to manually compile deps/exmpp and now I `rebar compile` works fine. Looks like my problem is solved! Thanks [1] https://github.com/rebar/rebar/blob/master/rebar.config.sample#L160-L175 On Fri, Dec 7, 2012 at 8:18 PM, Garrett Smith wrote: > On Fri, Dec 7, 2012 at 8:32 AM, Vineet Naik wrote: > > Hi Garrett, > > > > On Fri, Dec 7, 2012 at 7:40 PM, Garrett Smith wrote: > >> > >> [...] > >> > >> > >> At build time, rebar will download and build dependencies in a local > >> "deps" dir. > > > > > > Does it require that the dependencies be rebar compatible as well. Right > > now I am having exmpp which doesn't use rebar and I guess doesn't have > > the directory structure suitable for it. > > Briefly looking at this, you'll have a problem following my advice > with rebar for this case -- that project has a complex build. > > I spotted this thread: > > https://support.process-one.net/browse/EXMPP-51 > > which points to this project: > > https://github.com/archaelus/exmpp > > I tried building that using "rebar compile" but hit some errors. > > If that works for you, just list that fork for your rebar dependency. > Otherwise, you could troubleshoot the problem or, just build the > project yourself using whatever works from a Makefile -- and put it > into a local "deps" dir. Other required apps will land there from > rebar builds and you'll have a single dir for ERL_LIBS. > > As for building a release, it sounds like rebar has come of age. But > exmpp may not be friendly. Could be a good area to help contribute to > with patches. > > > With exmpp installed in default erlang search path, it's not an issue for > > me at least during development but frankly I would be more happy with a > > sandboxed environment similar to Python's virtualenv. Glad that rebar > > downloads and builds dependencies local to the project. Will try if it > works > > for non-rebar-compatible libs too. > > > > Thanks for your reply, it helped a lot :-) > > You're welcome! > > Garrett > -- Vineet Naik -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Dec 7 16:51:31 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Fri, 07 Dec 2012 16:51:31 +0100 Subject: [erlang-questions] Recommended approach for dependency management during development In-Reply-To: References: Message-ID: <50C21083.2010603@ninenines.eu> Oh thanks, I didn't know this got merged. By the way I think the cannon rebar repository is https://github.com/basho/rebar ? On 12/07/2012 04:48 PM, Vineet Naik wrote: > Just found out that rebar supports "raw" repositories[1] which will only be > fetched but ignored during compilation. So after running `get-deps`, I > had to manually compile deps/exmpp and now I `rebar compile` works fine. > > Looks like my problem is solved! > > Thanks > > [1] https://github.com/rebar/rebar/blob/master/rebar.config.sample#L160-L175 > > > On Fri, Dec 7, 2012 at 8:18 PM, Garrett Smith > wrote: > > On Fri, Dec 7, 2012 at 8:32 AM, Vineet Naik > wrote: > > Hi Garrett, > > > > On Fri, Dec 7, 2012 at 7:40 PM, Garrett Smith > wrote: > >> > >> [...] > >> > >> > >> At build time, rebar will download and build dependencies in a local > >> "deps" dir. > > > > > > Does it require that the dependencies be rebar compatible as > well. Right > > now I am having exmpp which doesn't use rebar and I guess doesn't > have > > the directory structure suitable for it. > > Briefly looking at this, you'll have a problem following my advice > with rebar for this case -- that project has a complex build. > > I spotted this thread: > > https://support.process-one.net/browse/EXMPP-51 > > which points to this project: > > https://github.com/archaelus/exmpp > > I tried building that using "rebar compile" but hit some errors. > > If that works for you, just list that fork for your rebar dependency. > Otherwise, you could troubleshoot the problem or, just build the > project yourself using whatever works from a Makefile -- and put it > into a local "deps" dir. Other required apps will land there from > rebar builds and you'll have a single dir for ERL_LIBS. > > As for building a release, it sounds like rebar has come of age. But > exmpp may not be friendly. Could be a good area to help contribute to > with patches. > > > With exmpp installed in default erlang search path, it's not an > issue for > > me at least during development but frankly I would be more happy > with a > > sandboxed environment similar to Python's virtualenv. Glad that rebar > > downloads and builds dependencies local to the project. Will try > if it works > > for non-rebar-compatible libs too. > > > > Thanks for your reply, it helped a lot :-) > > You're welcome! > > Garrett > > > > > -- > Vineet Naik > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From daniel@REDACTED Fri Dec 7 17:43:27 2012 From: daniel@REDACTED (Daniel Luna) Date: Fri, 7 Dec 2012 11:43:27 -0500 Subject: [erlang-questions] Recommended approach for dependency management during development In-Reply-To: <50C21083.2010603@ninenines.eu> References: <50C21083.2010603@ninenines.eu> Message-ID: On 7 December 2012 10:51, Lo?c Hoguin wrote: > Oh thanks, I didn't know this got merged. > > By the way I think the cannon rebar repository is > https://github.com/basho/rebar ? This changed a few weeks ago. It's https://github.com/rebar/rebar now. Cheers, Daniel From wojtek@REDACTED Fri Dec 7 18:09:44 2012 From: wojtek@REDACTED (=?UTF-8?B?V29qdGVrIE5hcmN6ecWEc2tp?=) Date: Fri, 07 Dec 2012 18:09:44 +0100 Subject: [erlang-questions] Erlang and Websphere MQ In-Reply-To: References: Message-ID: <50C222D8.50905@power.com.pl> On 2012-12-07 10:46, Yash Ganthe wrote: > > What is the best way to read from and write to IBM Websphere MQ using > Erlang. Are there any modules available for doing that? We have a port program in C++. Will investigate its quality / releasability next week. --Regards, Wojtek N. From jeremy@REDACTED Fri Dec 7 19:13:12 2012 From: jeremy@REDACTED (Jeremy Ong) Date: Fri, 7 Dec 2012 10:13:12 -0800 Subject: [erlang-questions] Recommended approach for dependency management during development In-Reply-To: References: <50C21083.2010603@ninenines.eu> Message-ID: I wrote http://www.jeremyong.com/blog/2012/12/02/setting-up-an-erlang-cluster-on-ec2/ a short while ago while I was figuring out how to structure OTP projects with several applications and release them with rebar. Maybe somebody will find it helpful. Better yet, if you know of a practice or trick that I'm unaware of, feel free to comment there and I will make edits. On Fri, Dec 7, 2012 at 8:43 AM, Daniel Luna wrote: > On 7 December 2012 10:51, Lo?c Hoguin wrote: > > Oh thanks, I didn't know this got merged. > > > > By the way I think the cannon rebar repository is > > https://github.com/basho/rebar ? > > This changed a few weeks ago. It's https://github.com/rebar/rebar now. > > Cheers, > > Daniel > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david@REDACTED Fri Dec 7 19:49:59 2012 From: david@REDACTED (David Fox) Date: Fri, 07 Dec 2012 12:49:59 -0600 Subject: [erlang-questions] Help creating distributed server cache In-Reply-To: <51c212fa-185d-45e1-ad97-815ce1cde94f@googlegroups.com> References: <50C1117A.4000008@davidjfox.com> <51c212fa-185d-45e1-ad97-815ce1cde94f@googlegroups.com> Message-ID: <50C23A57.10805@davidjfox.com> There is no hard requirement for a RESTful API, but since this API will be used in a wide variety of places (e.g., web/html5 games, mobile, flash, etc) and not just internally, we decided having a RESTful API would be a good idea and make using the API in development quicker/easier. On 12/6/2012 18:42, Steve Davis wrote: > From what you've said, I would guess that the correct answer is: > 2) memcached protocol > > Since > > Your solution (1) starts with "When a client requests data on a > player, spawn 1 player process." If that had been "when a client > requests data on themselves from another game" then it could have been > in the running... > > A memcached implementation will sort out LRU without you having to > reinvent (stabilize, test) a wheel. > > Not sure why there's a REST requirement. If this MUST be HTTP then I > see it, otherwise what does it do for you? > > My 2c, > /s > > On Thursday, December 6, 2012 3:43:22 PM UTC-6, David Fox wrote: > > I'm currently developing a gaming server which stores player > information > that can be accessed from any of our games via a REST API. > > So far I've thought of two ways to structure and cache player data: > > 1. When a client requests data on a player, spawn 1 player > process. This > process handles: all subsequent requests from clients for this > player, > retrieving the player data from the DB when created and periodically > updating the DB with any new data from clients. If the player is not > requested by another client within... say 30 minutes, the player > process > will terminate. > > 2. Just keep previously requested data in a distributed LRU cache > (e.g., > memcached, redis, mnesia) > > Out of the two, I prefer #1 since it would allow me to separate the > functionality of different "data types" (e.g., player data, game > data). > > There are just 2 problems with doing it this way that I'd like your > thoughts and help with: > I. I would have to implement some sort of "LRU process cache" so I > could > terminate processes to free memory for new ones. > II. If a load balancer connects a client to node #1, but the > process for > the requested player is on node #2, how can the player process on > node > #2 send the data to the socket opened for the client on node #1. > Is it > possible to somehow send an socket across nodes? The reason I ask, is > that I'd like to prevent sending big messages across nodes. > > Thanks for the help! > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david@REDACTED Fri Dec 7 19:52:38 2012 From: david@REDACTED (David Fox) Date: Fri, 07 Dec 2012 12:52:38 -0600 Subject: [erlang-questions] Help creating distributed server cache In-Reply-To: References: <50C1117A.4000008@davidjfox.com> <50C123D4.1080008@davidjfox.com> Message-ID: <50C23AF6.40105@davidjfox.com> I did not know about the Chicago group. Thanks for the heads up, I'll make sure to look into it :) On 12/6/2012 17:26, Garrett Smith wrote: > On Thu, Dec 6, 2012 at 5:01 PM, David Fox wrote: >> Hi Garret, thanks for the response. >> >> I have not yet finished implementing the API; I'm still in the design phase >> figuring out how everything should be hooked up. > Right. though I think you can start to build actual pieces (i.e. > compiling/running code) based on what's the most obvious to you, even > if it's just super stupidly simple. It will give you a basis for > iteration, which will further you help your understanding. You may > find yourself spending almost no time designing :) > >> Totally agree on not limiting yourself to options this early on, I'm just >> asking for some help and opinions on some problems I saw in potential >> implementations :) >> >> David Fox >> m: 630 930 9219 >> Chicago > Ah, I'm also in Chicago. Do you know about the Chicago Erlang User Group: > > http://www.meetup.com/ErlangChicago/ > > We haven't met the last few months, but I'd like to do an informal > meetup before years end! > >> On 12/6/2012 16:34, Garrett Smith wrote: >>> Hi David, >>> >>> On Thu, Dec 6, 2012 at 9:43 PM, David Fox wrote: >>>> I'm currently developing a gaming server which stores player information >>>> that can be accessed from any of our games via a REST API. >>>> >>>> So far I've thought of two ways to structure and cache player data: >>>> >>>> 1. When a client requests data on a player, spawn 1 player process. This >>>> process handles: all subsequent requests from clients for this player, >>>> retrieving the player data from the DB when created and periodically >>>> updating the DB with any new data from clients. If the player is not >>>> requested by another client within... say 30 minutes, the player process >>>> will terminate. >>>> >>>> 2. Just keep previously requested data in a distributed LRU cache (e.g., >>>> memcached, redis, mnesia) >>>> >>>> Out of the two, I prefer #1 since it would allow me to separate the >>>> functionality of different "data types" (e.g., player data, game data). >>>> >>>> There are just 2 problems with doing it this way that I'd like your >>>> thoughts >>>> and help with: >>>> I. I would have to implement some sort of "LRU process cache" so I could >>>> terminate processes to free memory for new ones. >>>> II. If a load balancer connects a client to node #1, but the process for >>>> the >>>> requested player is on node #2, how can the player process on node #2 >>>> send >>>> the data to the socket opened for the client on node #1. Is it possible >>>> to >>>> somehow send an socket across nodes? The reason I ask, is that I'd like >>>> to >>>> prevent sending big messages across nodes. >>> It's tough to answer high level "approach" style questions, especially >>> without some hands on work (tinkering) to help you understand the >>> problem. >>> >>> Limiting yourself to either/or options at this stage might also be >>> premature. >>> >>> Do you have a first pass at the public API for this service? >>> >>> If you have an idea of the functions that could define the interface, >>> you can ask, for each unimplemented function: >>> >>> - Can I make this side-effect free -- i.e. calling the function >>> doesn't change state or otherwise tamper with the universe? >>> >>> - Does the function read from or write to long running state? >>> >>> Side effect free functions are easy, which is why you try to solve >>> problems using them exclusively whenever possible. >>> >>> For long running state, you can use a simple gen_server to implement >>> state initialization and mutation. If you have questions about what I >>> mean here, you'll need to bone up on gen_server, or alternatively look >>> at e2 services (see http://e2project.org) as they're simpler to write. >>> >>> Once you have something very basic working, see if you're done! It >>> might just work for you as is, at least for the short term. If it >>> doesn't work, address the specific problem. E.g. if your problem is "I >>> loose my state when the VM crahses," you'll need to implement >>> persistence in some fashion. >>> >>> Questions at that level are much easier to answer :) >>> >>> Garrett >> From david@REDACTED Fri Dec 7 23:55:17 2012 From: david@REDACTED (David Fox) Date: Fri, 07 Dec 2012 16:55:17 -0600 Subject: [erlang-questions] Help creating distributed server cache In-Reply-To: <269eb6b1-f17a-4ea9-b5f5-02117cc20b2e@googlegroups.com> References: <50C1117A.4000008@davidjfox.com> <269eb6b1-f17a-4ea9-b5f5-02117cc20b2e@googlegroups.com> Message-ID: <50C273D5.2060703@davidjfox.com> I've never heard of this tech before. Thanks for the heads up, looks quite interesting :) On 12/7/2012 13:34, Arthur Ingram wrote: > Take a look at the following > > > https://github.com/ztmr/egtm > > http://robtweed.wordpress.com/2012/10/22/natively-stateless/ > > > > On Thursday, December 6, 2012 3:43:22 PM UTC-6, David Fox wrote: > > I'm currently developing a gaming server which stores player > information > that can be accessed from any of our games via a REST API. > > So far I've thought of two ways to structure and cache player data: > > 1. When a client requests data on a player, spawn 1 player > process. This > process handles: all subsequent requests from clients for this > player, > retrieving the player data from the DB when created and periodically > updating the DB with any new data from clients. If the player is not > requested by another client within... say 30 minutes, the player > process > will terminate. > > 2. Just keep previously requested data in a distributed LRU cache > (e.g., > memcached, redis, mnesia) > > Out of the two, I prefer #1 since it would allow me to separate the > functionality of different "data types" (e.g., player data, game > data). > > There are just 2 problems with doing it this way that I'd like your > thoughts and help with: > I. I would have to implement some sort of "LRU process cache" so I > could > terminate processes to free memory for new ones. > II. If a load balancer connects a client to node #1, but the > process for > the requested player is on node #2, how can the player process on > node > #2 send the data to the socket opened for the client on node #1. > Is it > possible to somehow send an socket across nodes? The reason I ask, is > that I'd like to prevent sending big messages across nodes. > > Thanks for the help! > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Sat Dec 8 01:52:42 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Fri, 7 Dec 2012 16:52:42 -0800 (PST) Subject: [erlang-questions] Help creating distributed server cache In-Reply-To: <50C23A57.10805@davidjfox.com> References: <50C1117A.4000008@davidjfox.com> <51c212fa-185d-45e1-ad97-815ce1cde94f@googlegroups.com> <50C23A57.10805@davidjfox.com> Message-ID: <52b60d93-b120-45c1-aaa5-dd53e1aafa57@googlegroups.com> Then it does sound like either one of: - Couchbase http://www.couchbase.com - which is what I had in mind as a memcached supporting cache/db - Riak (as Anthony had suggested) ...would suit your needs. Riak is mature and massively scalable by design. Couchbase is also highly scalable, offers document-oriented storage and a js query interface. It's hard to predict which would suit you best, as that would depend on your data and current infrastructure. Both are definitely worth doing the "due diligence" on. /s On Friday, December 7, 2012 12:49:59 PM UTC-6, David Fox wrote: > > There is no hard requirement for a RESTful API, but since this API will > be used in a wide variety of places (e.g., web/html5 games, mobile, flash, > etc) and not just internally, we decided having a RESTful API would be a > good idea and make using the API in development quicker/easier. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Sat Dec 8 02:44:25 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Fri, 7 Dec 2012 17:44:25 -0800 (PST) Subject: [erlang-questions] Recommended approach for dependency management during development In-Reply-To: References: Message-ID: <653c9650-c3d9-4c1e-bb64-259c09797f6a@googlegroups.com> I'm somewhat intrigued about why "dependency/package management" isn't a resolved issue. For sure rebar is a big and useful step towards that, and for erlang apps it is a great solution, but the true panacea has still not arrived. What is really intriguing is that the dependency management issue, at least by my observation, is not truly resolved for any language/platform i can think of (ruby gems strikes me as perhaps closest, but still not the true resolution). One very interesting approach has been maven, but the lessons from that project are multitudinous, e.g.: http://stackoverflow.com/questions/861382/why-does-maven-have-such-a-bad-rep I personally suspect that the solution may be deep in our accepted software assumptions. Specifically, the unresolved issue is not dependency management per se, but the impedance mismatch and inherent state of accepting and declaring what we know as "relative paths". If that is true, then dependency management is simply another case where hidden state is the enemy of any attempt at rationalizing behavior. /s On Friday, December 7, 2012 7:43:48 AM UTC-6, Vineet Naik wrote: > > I trying to write my first non trivial Erlang/OTP application and > looking for the best approach for organizing and managing > dependencies during development. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Dec 8 10:58:24 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 8 Dec 2012 12:58:24 +0300 Subject: [erlang-questions] Erlang NFS client Message-ID: Hi. I've took code from jungerl ( https://github.com/msantos/erpcgen is actual copy on github), generated stub code for mount3 and nfs_prot3 protocols and written a small wrapper around them. It works like a charm. Erlang can work with remote NFS storage (such as enterprise NAS) faster and better than kernel driver. For example scanning whole remote directory is done in async manner which is much faster than plain find /mount/remote -type f https://github.com/maxlapshin/nfs3 Thanks for erpcgen! -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Sat Dec 8 12:12:18 2012 From: tony@REDACTED (Tony Rogvall) Date: Sat, 8 Dec 2012 12:12:18 +0100 Subject: [erlang-questions] Erlang NFS client In-Reply-To: References: Message-ID: Just a historical note. This code was initially written by me. With improvements by Martin Bj?rklund, Luke Gorrie wrote the NFS examples. Later the code was "stolen" by Sendmail. I was really upset when I heard that Sendmail had put copyright on "my" code, the reason was very business driven "The boss decided that we had to do it" :-) (anyone with the more info about this could fill in the embarrassing facts) I will probably steel it back again, but for now just ignore the Copyright. Well well /Tony On 8 dec 2012, at 10:58, Max Lapshin wrote: > Hi. > > I've took code from jungerl ( https://github.com/msantos/erpcgen is actual copy on github), generated stub code for mount3 and nfs_prot3 protocols and written a small wrapper around them. > > It works like a charm. Erlang can work with remote NFS storage (such as enterprise NAS) faster and better than kernel driver. > > For example scanning whole remote directory is done in async manner which is much faster than plain find /mount/remote -type f > > https://github.com/maxlapshin/nfs3 > > Thanks for erpcgen! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Dec 8 12:36:59 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 8 Dec 2012 14:36:59 +0300 Subject: [erlang-questions] Erlang NFS client In-Reply-To: References: Message-ID: It is a bad idea to remove copyrights. I've just copied rpc_client from that repo to my example and I'm eager to change to your copyright. What should I write? And where sendmail uses erlang? -------------- next part -------------- An HTML attachment was scrubbed... URL: From yashgt@REDACTED Sat Dec 8 16:18:09 2012 From: yashgt@REDACTED (Yash Ganthe) Date: Sat, 8 Dec 2012 07:18:09 -0800 (PST) Subject: [erlang-questions] Erlang and Websphere MQ In-Reply-To: <50C222D8.50905@power.com.pl> References: <50C222D8.50905@power.com.pl> Message-ID: <6eb1f411-f5b7-4bb5-afce-043a033eece5@googlegroups.com> Please keep us informed. On Friday, December 7, 2012 12:09:44 PM UTC-5, Wojtek Narczy?ski wrote: > > On 2012-12-07 10:46, Yash Ganthe wrote: > > > > What is the best way to read from and write to IBM Websphere MQ using > > Erlang. Are there any modules available for doing that? > We have a port program in C++. Will investigate its quality / > releasability next week. > > --Regards, > Wojtek N. > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fritchie@REDACTED Sun Dec 9 02:37:49 2012 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Sat, 08 Dec 2012 19:37:49 -0600 Subject: [erlang-questions] Erlang NFS client In-Reply-To: Message of "Sat\, 08 Dec 2012 12\:12\:18 +0100." Message-ID: <41045.1355017069@snookles.snookles.com> Tony Rogvall wrote: tr> This code was initially written by me. With improvements by Martin tr> Bj?rklund, Luke Gorrie wrote the NFS examples. Later the code was tr> "stolen" by Sendmail. I was really upset when I heard that Sendmail tr> had put copyright on "my" code, the reason was very business driven tr> "The boss decided that we had to do it" :-) Hi, Tony. As one of the guilty back then, I don't recall it being a matter of clobbering copyright but a matter of adding explicit license terms on the modifications that Sendmail, Inc. people made (including myself at the time). You certainly still have copyright claims on that code. Max, we found that code immensely useful a decade ago. I'm glad it's still in use. -Scott From corticalcomputer@REDACTED Sun Dec 9 11:55:14 2012 From: corticalcomputer@REDACTED (Gene Sher) Date: Sun, 9 Dec 2012 05:55:14 -0500 Subject: [erlang-questions] eBook version of the Handbook of Neuroevolution Through Erlang, is now available from Springer. In-Reply-To: References: Message-ID: Hello Everyone, Quick update: I've recently convinced Springer to provide a discount. They are offering 20% discount, with the discount code embedded in the following link, good until 12/31: http://www.springer.com/computer/swe/book/978-1-4614-4462-6?token=DWw53HF6DtywP8y Also, if you have a University nearby, you can get a copy for just 25$, since Springer primarily deals with Universities and their Libraries, your library and SpringerLink might already have access to the eBook. If you have access to SpringerLink, you can alternatively order a "MyCopy", a soft-cover version (black & white, rather than color) for $24.95. Best regards, -Gene On Sun, Nov 11, 2012 at 12:02 AM, Gene Sher wrote: > Hello Erlangers, > > The eBook version of my the Handbook of Neuroevolution Through Erlang, is > now in print: http://www.springer.com/computer/swe/book/978-1-4614-4462-6 > The Hardcover book will be available within the next 2-3 weeks from > Amazon, Barnes & Noble, and Springer directly. > > Book overview: > > - Provides a friendly step-by-step guide on the construction of > Topology and Weight Evolving Artificial Neural Network systems from start > to finish > - Covers novel material for using Erlang in the construction of TWEANN > systems > - Explains why Neural Network based Computational Intelligence systems > map perfectly to Erlang?s architecture, and the importance of this > programming language to the future of computational intelligence > - Introduces new TWEANN algorithms, with the final result being a > concurrent, cutting edge, direct and indirect encoded, plasticity enabled, > TWEANN platform > > *Handbook of Neuroevolution Through Erlang* presents both the theory > behind, and the methodology of, developing a neuroevolutionary-based > computational intelligence system using Erlang. With a foreword written by > Joe Armstrong, this handbook offers an extensive tutorial for creating a > state of the art Topology and Weight Evolving Artificial Neural Network > (TWEANN) platform. In a step-by-step format, the reader is guided from a > single simulated neuron to a complete system. By following these steps, the > reader will be able to use novel technology to build a TWEANN system, which > can be applied to Artificial Life simulation, and Forex trading. Because of > Erlang?s architecture, it perfectly matches that of evolutionary and > neurocomptational systems. As a programming language, it is a concurrent, > message passing paradigm which allows the developers to make full use of > the multi-core & multi-cpu systems. *Handbook of Neuroevolution Through > Erlang* explains how to leverage Erlang?s features in the field of > machine learning, and the system?s real world applications, ranging from > algorithmic financial trading to artificial life and robotics. > > It covers in detail the subject of Neuroevolution, its applications, why > Erlang is the quintessential neural network programming language, and the > construction of DXNN2: https://github.com/CorticalComputer/DXNN2 > A robust, purely Erlang, Topology and Weight Evolving Artificial Neural > Network platform, capable of evolving direct and indirect systems, with and > without plasticity, and a hierarchical structure that yields easily to > allow one to develop self-repairing intelligent agents. > > Best regards, > -Gene > -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco@REDACTED Sun Dec 9 12:02:54 2012 From: francesco@REDACTED (Francesco Cesarini) Date: Sun, 09 Dec 2012 12:02:54 +0100 Subject: [erlang-questions] eBook version of the Handbook of Neuroevolution Through Erlang, is now available from Springer. In-Reply-To: References: Message-ID: Well done! As someone who always felt Erlang was a perfect match for certain branches of AI, I'm really excited to see this work in print. Hope others follow your example and get even more Erlang books written. F Gene Sher wrote: >Hello Everyone, > >Quick update: I've recently convinced Springer to provide a discount. >They >are offering 20% discount, with the discount code embedded in the >following >link, good until 12/31: >http://www.springer.com/computer/swe/book/978-1-4614-4462-6?token=DWw53HF6DtywP8y > >Also, if you have a University nearby, you can get a copy for just 25$, >since Springer primarily deals with Universities and their Libraries, >your >library and SpringerLink might already have access to the eBook. If you >have access to SpringerLink, you can alternatively order a "MyCopy", a >soft-cover version (black & white, rather than color) for $24.95. > >Best regards, >-Gene > >On Sun, Nov 11, 2012 at 12:02 AM, Gene Sher >wrote: > >> Hello Erlangers, >> >> The eBook version of my the Handbook of Neuroevolution Through >Erlang, is >> now in print: >http://www.springer.com/computer/swe/book/978-1-4614-4462-6 >> The Hardcover book will be available within the next 2-3 weeks from >> Amazon, Barnes & Noble, and Springer directly. >> >> Book overview: >> >> - Provides a friendly step-by-step guide on the construction of >> Topology and Weight Evolving Artificial Neural Network systems >from start >> to finish >> - Covers novel material for using Erlang in the construction of >TWEANN >> systems >> - Explains why Neural Network based Computational Intelligence >systems >> map perfectly to Erlang?s architecture, and the importance of this >> programming language to the future of computational intelligence >> - Introduces new TWEANN algorithms, with the final result being a >> concurrent, cutting edge, direct and indirect encoded, plasticity >enabled, >> TWEANN platform >> >> *Handbook of Neuroevolution Through Erlang* presents both the theory >> behind, and the methodology of, developing a neuroevolutionary-based >> computational intelligence system using Erlang. With a foreword >written by >> Joe Armstrong, this handbook offers an extensive tutorial for >creating a >> state of the art Topology and Weight Evolving Artificial Neural >Network >> (TWEANN) platform. In a step-by-step format, the reader is guided >from a >> single simulated neuron to a complete system. By following these >steps, the >> reader will be able to use novel technology to build a TWEANN system, >which >> can be applied to Artificial Life simulation, and Forex trading. >Because of >> Erlang?s architecture, it perfectly matches that of evolutionary and >> neurocomptational systems. As a programming language, it is a >concurrent, >> message passing paradigm which allows the developers to make full use >of >> the multi-core & multi-cpu systems. *Handbook of Neuroevolution >Through >> Erlang* explains how to leverage Erlang?s features in the field of >> machine learning, and the system?s real world applications, ranging >from >> algorithmic financial trading to artificial life and robotics. >> >> It covers in detail the subject of Neuroevolution, its applications, >why >> Erlang is the quintessential neural network programming language, and >the >> construction of DXNN2: https://github.com/CorticalComputer/DXNN2 >> A robust, purely Erlang, Topology and Weight Evolving Artificial >Neural >> Network platform, capable of evolving direct and indirect systems, >with and >> without plasticity, and a hierarchical structure that yields easily >to >> allow one to develop self-repairing intelligent agents. >> >> Best regards, >> -Gene >> > > >------------------------------------------------------------------------ > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From yashgt@REDACTED Sun Dec 9 14:35:32 2012 From: yashgt@REDACTED (Yash Ganthe) Date: Sun, 09 Dec 2012 08:35:32 -0500 Subject: [erlang-questions] Logging interprocess communication Message-ID: Hi, In an Erlang VM when one process send a message to another using: Pid ! MyMessage we would like to log this interaction to a log file. We need to do this without having the developer introduce explicit log statements all over the code wherever one process communicates with another. What is the best way to achieve this? Thanks, Yash -- Using Opera's revolutionary email client: http://www.opera.com/mail/ From essen@REDACTED Sun Dec 9 14:44:07 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Sun, 09 Dec 2012 14:44:07 +0100 Subject: [erlang-questions] Logging interprocess communication In-Reply-To: References: Message-ID: <50C495A7.2010804@ninenines.eu> On 12/09/2012 02:35 PM, Yash Ganthe wrote: > > Hi, > > In an Erlang VM when one process send a message to another using: Pid ! > MyMessage we would like to log this interaction to a log file. We need > to do this without having the developer introduce explicit log > statements all over the code wherever one process communicates with > another. > > What is the best way to achieve this? I would look at Erlang support for DTrace/Systemtap depending on the system you are running. For Linux, Systemtap needs a special kernel build but that's not hard to setup (at least on Arch Linux). -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From freza@REDACTED Sun Dec 9 14:39:51 2012 From: freza@REDACTED (Jachym Holecek) Date: Sun, 9 Dec 2012 08:39:51 -0500 Subject: [erlang-questions] Logging interprocess communication In-Reply-To: References: Message-ID: <20121209133950.GA16063@circlewave.net> # Yash Ganthe 2012-12-09: > In an Erlang VM when one process send a message to another using: Pid ! > MyMessage we would like to log this interaction to a log file. We need to > do this without having the developer introduce explicit log statements all > over the code wherever one process communicates with another. > > What is the best way to achieve this? There's a builtin tracing facility inside the VM, controlled by erlang:trace/3 and friends. The dbg(3) module builds on top of it, supports dumping traces to file among other things. Not sure if there are more elaborate frontends or step-by-step documents. BR, -- Jachym From a.shneyderman@REDACTED Sun Dec 9 15:11:06 2012 From: a.shneyderman@REDACTED (Alex Shneyderman) Date: Sun, 9 Dec 2012 15:11:06 +0100 Subject: [erlang-questions] Logging interprocess communication In-Reply-To: References: Message-ID: You might find parse_transforms useful for this sort of things: http://stackoverflow.com/questions/2416192/is-there-a-good-complete-tutorial-on-erlang-parse-transforms-available I am not sure your particular use case can be weaved around but it would be hard to believe if it is not. Cheers. On Sun, Dec 9, 2012 at 2:35 PM, Yash Ganthe wrote: > > Hi, > > In an Erlang VM when one process send a message to another using: Pid ! > MyMessage we would like to log this interaction to a log file. We need to do > this without having the developer introduce explicit log statements all over > the code wherever one process communicates with another. > > What is the best way to achieve this? > > Thanks, > Yash > -- > Using Opera's revolutionary email client: http://www.opera.com/mail/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Sun Dec 9 15:13:31 2012 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 9 Dec 2012 15:13:31 +0100 Subject: [erlang-questions] Logging interprocess communication In-Reply-To: References: Message-ID: This is *exactly* what the trace BIFs are for erlang:trace(PidSpec, Bool, TraceFlags) does what you want. Here is an example: -module(tracer). -export([start/0, loop/0, watch/1]). start() -> Pid = spawn(tracer, loop, []), spawn(tracer, watch, [Pid]), Pid. loop() -> receive Any -> io:format("I got:~p~n",[Any]), loop() end. watch(Pid) -> erlang:trace(Pid, true, [send, timestamp]), watch(). watch() -> receive Any -> io:format("watch got:~p~n",[Any]), loop() end. In the shell 2> c(tracer). {ok,tracer} 3> Pid = tracer:start(). <0.43.0> 4> Pid ! hello. I got:hello hello watch got:{trace_ts,<0.43.0>,send, {io_request,<0.43.0>,<0.24.0>, {put_chars,unicode,io_lib,format,["I got:~p~n",[hello]]}}, <0.24.0>, {1355,61952,458716}} The line erlang:trace(Pid, true, [send, timestamp]), means "trace the process Pid and tell me about all messages it sends, an include a time stamp. "Tell me" here means 'send me a message' This is described (with examples) in my book and in Francesco and Simon's book, and at http://www.erlang.org/doc/man/erlang.html#trace-3 Cheers /Joe On Sun, Dec 9, 2012 at 2:35 PM, Yash Ganthe wrote: > > Hi, > > In an Erlang VM when one process send a message to another using: Pid ! > MyMessage we would like to log this interaction to a log file. We need to > do this without having the developer introduce explicit log statements all > over the code wherever one process communicates with another. > > What is the best way to achieve this? > > Thanks, > Yash > -- > Using Opera's revolutionary email client: http://www.opera.com/mail/ > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From heinz@REDACTED Sun Dec 9 18:24:31 2012 From: heinz@REDACTED (Heinz Nikolaus Gies) Date: Sun, 9 Dec 2012 18:24:31 +0100 Subject: [erlang-questions] ANN: Erlang DTrace consumer Message-ID: <38798195-68BD-4157-93E0-681B975D71E3@licenser.net> Hi everyone, I've some new toy to play with: an Erlang DTrace consumer, it's still rough around the edges and probably has bug but if you want to try it out and let me know what problems you run into please go ahead and enjoy: https://github.com/project-fifo/erltrace Regards, Heinz -------------- next part -------------- An HTML attachment was scrubbed... URL: From zephyr.pellerin@REDACTED Mon Dec 10 02:30:58 2012 From: zephyr.pellerin@REDACTED (Zephyr Pellerin) Date: Sun, 9 Dec 2012 17:30:58 -0800 Subject: [erlang-questions] eBook version of the Handbook of Neuroevolution Through Erlang, is now available from Springer. In-Reply-To: References: Message-ID: Amazing! I will gladly buy this book from Amazon! On Sun, Dec 9, 2012 at 3:02 AM, Francesco Cesarini wrote: > Well done! As someone who always felt Erlang was a perfect match for certain > branches of AI, I'm really excited to see this work in print. Hope others > follow your example and get even more Erlang books written. > > F > > Gene Sher wrote: >> >> Hello Everyone, >> >> Quick update: I've recently convinced Springer to provide a discount. They >> are offering 20% discount, with the discount code embedded in the following >> link, good until 12/31: >> http://www.springer.com/computer/swe/book/978-1-4614-4462-6?token=DWw53HF6DtywP8y >> >> Also, if you have a University nearby, you can get a copy for just 25$, >> since Springer primarily deals with Universities and their Libraries, your >> library and SpringerLink might already have access to the eBook. If you have >> access to SpringerLink, you can alternatively order a "MyCopy", a soft-cover >> version (black & white, rather than color) for $24.95. >> >> Best regards, >> -Gene >> >> On Sun, Nov 11, 2012 at 12:02 AM, Gene Sher >> wrote: >>> >>> Hello Erlangers, >>> >>> The eBook version of my the Handbook of Neuroevolution Through Erlang, is >>> now in print: http://www.springer.com/computer/swe/book/978-1-4614-4462-6 >>> The Hardcover book will be available within the next 2-3 weeks from >>> Amazon, Barnes & Noble, and Springer directly. >>> >>> Book overview: >>> >>> Provides a friendly step-by-step guide on the construction of Topology >>> and Weight Evolving Artificial Neural Network systems from start to finish >>> Covers novel material for using Erlang in the construction of TWEANN >>> systems >>> Explains why Neural Network based Computational Intelligence systems map >>> perfectly to Erlang?s architecture, and the importance of this programming >>> language to the future of computational intelligence >>> Introduces new TWEANN algorithms, with the final result being a >>> concurrent, cutting edge, direct and indirect encoded, plasticity enabled, >>> TWEANN platform >>> >>> Handbook of Neuroevolution Through Erlang presents both the theory >>> behind, and the methodology of, developing a neuroevolutionary-based >>> computational intelligence system using Erlang. With a foreword written by >>> Joe Armstrong, this handbook offers an extensive tutorial for creating a >>> state of the art Topology and Weight Evolving Artificial Neural Network >>> (TWEANN) platform. In a step-by-step format, the reader is guided from a >>> single simulated neuron to a complete system. By following these steps, the >>> reader will be able to use novel technology to build a TWEANN system, which >>> can be applied to Artificial Life simulation, and Forex trading. Because of >>> Erlang?s architecture, it perfectly matches that of evolutionary and >>> neurocomptational systems. As a programming language, it is a concurrent, >>> message passing paradigm which allows the developers to make full use of the >>> multi-core & multi-cpu systems. Handbook of Neuroevolution Through Erlang >>> explains how to leverage Erlang?s features in the field of machine learning, >>> and the system?s real world applications, ranging from algorithmic financial >>> trading to artificial life and robotics. >>> >>> It covers in detail the subject of Neuroevolution, its applications, why >>> Erlang is the quintessential neural network programming language, and the >>> construction of DXNN2: https://github.com/CorticalComputer/DXNN2 >>> A robust, purely Erlang, Topology and Weight Evolving Artificial Neural >>> Network platform, capable of evolving direct and indirect systems, with and >>> without plasticity, and a hierarchical structure that yields easily to allow >>> one to develop self-repairing intelligent agents. >>> >>> Best regards, >>> -Gene >> >> >> ________________________________ >> >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mrkhoza@REDACTED Mon Dec 10 11:08:24 2012 From: mrkhoza@REDACTED (Lucky Khoza) Date: Mon, 10 Dec 2012 12:08:24 +0200 Subject: [erlang-questions] (no subject) Message-ID: Hi Erlang Developers, I'm actually using emacs text editor to code, however when i compiled my code, I got this list below: [{i,os:getenv("HOME")++"/svn/modules/"},debug_info,warnings_as_errors]). And i'm not sure what is the cause of these two atoms: debug_info, and warnings_as_errors. Kindest Regards Lucky -------------- next part -------------- An HTML attachment was scrubbed... URL: From jobiscuit@REDACTED Mon Dec 10 11:12:40 2012 From: jobiscuit@REDACTED (Joseph Dunne) Date: Mon, 10 Dec 2012 12:12:40 +0200 Subject: [erlang-questions] (no subject) In-Reply-To: References: Message-ID: <552F3E3FF08F4AA0A1431E8883074FFD@gmail.com> Hi Lucky, these are not errors. They are the compiler options that are sent to erlang. They are setup in your .emacs.d config files. Kind Regards Joseph Dunne | Developer +27794906833 On Monday 10 December 2012 at 12:08 PM, Lucky Khoza wrote: > Hi Erlang Developers, > > I'm actually using emacs text editor to code, however when i compiled my code, I got this list below: > > [{i,os:getenv("HOME")++"/svn/modules/"},debug_info,warnings_as_errors]). > > And i'm not sure what is the cause of these two atoms: debug_info, and warnings_as_errors. > > > > Kindest Regards > Lucky > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsvancara@REDACTED Mon Dec 10 11:37:37 2012 From: bsvancara@REDACTED (Bohuslav Svancara) Date: Mon, 10 Dec 2012 11:37:37 +0100 Subject: [erlang-questions] ASN1 - DEFAULT clause problem Message-ID: Hello! I have some problem with ASN1. - I have this ASN1 definition in the Test.asn file: Test DEFINITIONS IMPLICIT TAGS ::= BEGIN EXPORTS Test; Test ::= SEQUENCE { a INTEGER, b INTEGER DEFAULT 0 } END - I am using this comand: erl -eval asn1ct:compile(\"Test.asn\"). It works fine. But when I add a "DEFAULT 0" clause to the line "a INTEGER," so it looks like this: a INTEGER DEFAULT 0, then asn1ct fails: 1> asn1error:7:'Test':'Test' {asn1,{duplicates_of_the_tags,[{'UNIVERSAL','INTEGER'}]}} 1> Why please? - Release of Erlang 5.9/OTP R15B - Windows 7 Prof 64-bit Sincerely, Bob ---------------------------------------------------- Just for recap: Failing Test.asn looks like this: Test DEFINITIONS IMPLICIT TAGS ::= BEGIN EXPORTS Test; Test ::= SEQUENCE { a INTEGER DEFAULT 0, b INTEGER DEFAULT 0 } END -------------- next part -------------- An HTML attachment was scrubbed... URL: From seand-erlang@REDACTED Mon Dec 10 12:30:28 2012 From: seand-erlang@REDACTED (Sean D) Date: Mon, 10 Dec 2012 11:30:28 +0000 Subject: [erlang-questions] Using Key/Value store with shared storage Message-ID: <20121210113027.GA18095@mryaffle.seand.me.uk> Hi all, We are currently running an application for a customer that stores a large number of key/value pairs. Performance is important for us as we need to maintain a write rate of at least 10,000 keys/second on one server. After evaluating various key/value stores, we found Bitcask worked extremely well for us and we went with this. The solution currently has multiple servers working independently of each and we would like to make the solution more resilient by using shared storage. I.e. If one of the servers goes down, the others can pick up the work load and add to/read from the same store. I am aware that Riak seems to be the standard solution for a resilient key-value store in the Erlang world, however from my initial investigations, this seems to work by duplicating the data between Riak nodes, and this is something I want to avoid as the number of keys we are storing will be in the range of 100s of GB and I would prefer that the shared storage is used rather than data needing to be duplicated. I am also concerned that the overhead of Riak may prove a bottle-neck, however this isn't something that I have tested. If anyone here has used a key/value store with a SAN or similar in this way, I'd be very keen to hear your experiences. Many thanks in advance, Sean From kenneth.lundin@REDACTED Mon Dec 10 12:45:26 2012 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Mon, 10 Dec 2012 12:45:26 +0100 Subject: [erlang-questions] ASN1 - DEFAULT clause problem In-Reply-To: References: Message-ID: See embedded comments below On Mon, Dec 10, 2012 at 11:37 AM, Bohuslav Svancara wrote: > Hello! > > I have some problem with ASN1. > > - I have this ASN1 definition in the Test.asn file: > > Test DEFINITIONS IMPLICIT TAGS ::= > > BEGIN > EXPORTS Test; > > Test ::= SEQUENCE > { > a INTEGER, > b INTEGER DEFAULT 0 > } > > END > > - I am using this comand: > > erl -eval asn1ct:compile(\"Test.asn\"). > Recommended way to invoke the ASN.1 compiler is erlc Test.asn # this is equivalent to what you did above. This will generate code for BER encoding rules with input/output as a list och integers 0..255 The most modern and fastest BER backend is invoked like this: erlc -bber_bin +driver Test.asn This will generate code for BER encoding rules with input/output as binaries > It works fine. > > But when I add a "DEFAULT 0" clause to the line "a INTEGER," so it looks > like this: > > a INTEGER DEFAULT 0, > > then asn1ct fails: > > 1> asn1error:7:'Test':'Test' > {asn1,{duplicates_of_the_tags,[{'UNIVERSAL','INTEGER'}]}} > 1> > > Why please? > > - Release of Erlang 5.9/OTP R15B > - Windows 7 Prof 64-bit > > Sincerely, > Bob > > ---------------------------------------------------- > Just for recap: Failing Test.asn looks like this: > > Test DEFINITIONS IMPLICIT TAGS ::= > > BEGIN > EXPORTS Test; > > Test ::= SEQUENCE > { > a INTEGER DEFAULT 0, > b INTEGER DEFAULT 0 > } > > END > > The DEFAULT property on a component in a SEQUENCE makes that property optional , meaning that it can be omitted by the sender. Since the BER encoding rules are used and the module default is IMPLICIT TAGS the tags for the components in a SEQUENCE must be unique (if there are 2 or more optionals in a row). In this case , since no tags are introduced in the syntax the tags for both a and b component will be UNIVERSAL, INTEGER and the 2 components can not be distinguished from each other. That's why the compiler reports an error. In order to correct this you can either change module default to AUTOMATIC TAGS or add you own tags into the declaration like this: Test ::= SEQUENCE { a [0] INTEGER DEFAULT 0, b [1] INTEGER DEFAULT 0 } Now you have two context specific unique tags 0 and 1 for the components which will make it possible to distinguish as a receiver which component (or both) that are part of the message. This is how ASN.1 works , it is not specific to the ASN.1 compiler in OTP /Kenneth, Erlang/OTP, Ericsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsvancara@REDACTED Mon Dec 10 13:13:54 2012 From: bsvancara@REDACTED (Bohuslav Svancara) Date: Mon, 10 Dec 2012 13:13:54 +0100 Subject: [erlang-questions] ASN1 - DEFAULT clause problem In-Reply-To: References: Message-ID: Thank you Kenneth. I never noticed IMPLICIT / AUTOMATIC TAGS clause. :-) Thank you many times. Bob 2012/12/10 Kenneth Lundin > See embedded comments below > > On Mon, Dec 10, 2012 at 11:37 AM, Bohuslav Svancara wrote: > >> Hello! >> >> I have some problem with ASN1. >> >> - I have this ASN1 definition in the Test.asn file: >> >> Test DEFINITIONS IMPLICIT TAGS ::= >> >> BEGIN >> EXPORTS Test; >> >> Test ::= SEQUENCE >> { >> a INTEGER, >> b INTEGER DEFAULT 0 >> } >> >> END >> >> - I am using this comand: >> >> erl -eval asn1ct:compile(\"Test.asn\"). >> > > Recommended way to invoke the ASN.1 compiler is > erlc Test.asn # this is equivalent to what you did above. > > This will generate code for BER encoding rules with input/output as a list > och integers 0..255 > > The most modern and fastest BER backend is invoked like this: > erlc -bber_bin +driver Test.asn > > This will generate code for BER encoding rules with input/output as > binaries > > >> It works fine. >> >> But when I add a "DEFAULT 0" clause to the line "a INTEGER," so it looks >> like this: >> >> a INTEGER DEFAULT 0, >> >> then asn1ct fails: >> >> 1> asn1error:7:'Test':'Test' >> {asn1,{duplicates_of_the_tags,[{'UNIVERSAL','INTEGER'}]}} >> 1> >> >> Why please? >> >> - Release of Erlang 5.9/OTP R15B >> - Windows 7 Prof 64-bit >> >> Sincerely, >> Bob >> >> ---------------------------------------------------- >> Just for recap: Failing Test.asn looks like this: >> >> Test DEFINITIONS IMPLICIT TAGS ::= >> >> BEGIN >> EXPORTS Test; >> >> Test ::= SEQUENCE >> { >> a INTEGER DEFAULT 0, >> b INTEGER DEFAULT 0 >> } >> >> END >> >> > The DEFAULT property on a component in a SEQUENCE makes that property > optional , meaning that it can be omitted by the sender. > > Since the BER encoding rules are used and the module default is IMPLICIT > TAGS > the tags for the components in a SEQUENCE must be unique (if there are 2 > or more optionals in a row). In this case , since no tags are introduced in > the syntax the > tags for both a and b component will be UNIVERSAL, INTEGER and the 2 > components can not be distinguished from each other. That's why the > compiler reports an error. > > In order to correct this you can either change module default to AUTOMATIC > TAGS or > add you own tags into the declaration like this: > > Test ::= SEQUENCE > { > a [0] INTEGER DEFAULT 0, > b [1] INTEGER DEFAULT 0 > } > > Now you have two context specific unique tags 0 and 1 for the components > which will make it possible to distinguish as a receiver which component > (or both) that are > part of the message. > > This is how ASN.1 works , it is not specific to the ASN.1 compiler in OTP > > > /Kenneth, Erlang/OTP, Ericsson > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Mon Dec 10 13:20:03 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 10 Dec 2012 13:20:03 +0100 Subject: [erlang-questions] Ets bag complexity Message-ID: Hello list As far as I can understand from paper [1] by Scott Lystic Fritchie ETS tables of 'bag' type are implemented as a hash table with linear arrays of values attached to each key? So, are the following statements correct for bag ETSes? - lookup and delete operations for key which has K values is of average complexity O(1+K) - insert operations are amortized O(1) (sometimes resizing of arrays is needed) - select operations are O(N+K) where N is a number of keys, K is number of values Cheers, Gleb Peregud 1: www.erlang.se/workshop/2003/paper/p43-fritchie.pdf From sverker.eriksson@REDACTED Mon Dec 10 15:12:10 2012 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Mon, 10 Dec 2012 15:12:10 +0100 Subject: [erlang-questions] Ets bag complexity In-Reply-To: References: Message-ID: <50C5EDBA.3010205@erix.ericsson.se> Gleb Peregud wrote: > Hello list > > As far as I can understand from paper [1] by Scott Lystic Fritchie ETS > tables of 'bag' type are implemented as a hash table with linear > arrays of values attached to each key? > > So, are the following statements correct for bag ETSes? > - lookup and delete operations for key which has K values is of > average complexity O(1+K) > - insert operations are amortized O(1) (sometimes resizing of arrays is needed) > insert in bag is also O(1+K) as we have to make sure that we do not insert duplicates. > - select operations are O(N+K) where N is a number of keys, K is > number of values > > I guess you mean O(N*K) if K is average number of values per key. In other words traversal of the entire table. /Sverker, Erlang/OTP From gleber.p@REDACTED Mon Dec 10 15:24:39 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 10 Dec 2012 15:24:39 +0100 Subject: [erlang-questions] Ets bag complexity In-Reply-To: <50C5EDBA.3010205@erix.ericsson.se> References: <50C5EDBA.3010205@erix.ericsson.se> Message-ID: >> - insert operations are amortized O(1) (sometimes resizing of arrays is >> needed) > insert in bag is also O(1+K) as we have to make sure that we do not insert > duplicates. Good point! >> - select operations are O(N+K) where N is a number of keys, K is >> number of values > > I guess you mean O(N*K) if K is average number of values per key. In other > words traversal of the entire table. Yes, in general case. I assumed (and didn't inform you) a specific case when only a single key matches the select match specification. From sverker.eriksson@REDACTED Mon Dec 10 15:46:15 2012 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Mon, 10 Dec 2012 15:46:15 +0100 Subject: [erlang-questions] Ets bag complexity In-Reply-To: References: <50C5EDBA.3010205@erix.ericsson.se> Message-ID: <50C5F5B7.5090806@erix.ericsson.se> Gleb Peregud wrote: >>> - insert operations are amortized O(1) (sometimes resizing of arrays is >>> needed) >>> >> insert in bag is also O(1+K) as we have to make sure that we do not insert >> duplicates. >> > > Good point! > > Also to be clear, there is no heavy resizing of entire hash array. Linear hashing is used which means table grows by splitting and rehashing one bucket at a time. >>> - select operations are O(N+K) where N is a number of keys, K is >>> number of values >>> >> I guess you mean O(N*K) if K is average number of values per key. In other >> words traversal of the entire table. >> > > Yes, in general case. I assumed (and didn't inform you) a specific > case when only a single key matches the select match specification. > > If the key in match-spec is totally bound, the select implementation does a hash lookup and thereby get O(1+K). Otherwise we get O(N*K) from total table traversal. /Sverker From jesper.louis.andersen@REDACTED Mon Dec 10 15:50:31 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 10 Dec 2012 15:50:31 +0100 Subject: [erlang-questions] Ets bag complexity In-Reply-To: References: Message-ID: <2E8554A1-0141-4E1A-B921-316155CB5E92@erlang-solutions.com> On Dec 10, 2012, at 1:20 PM, Gleb Peregud wrote: > As far as I can understand from paper [1] by Scott Lystic Fritchie ETS > tables of 'bag' type are implemented as a hash table with linear > arrays of values attached to each key? This reminds me of this weeks funny revelation. I have an ETS duplicate_bag of the form {Key, Value}. Now I thought that I could save something by changing the code into using {Key, [Value, ?]} in a set since I use lookup_element/3 on it to pick out the list of values. Surely, I can avoid the linear walk of the tuples and the gathering of the list, which would be much faster, right? The speed was the same as before. Though I did not check the memory usage :) I guess that the price of copying the data set into the Process is around the same as before and this dwarfs the run time. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From greg@REDACTED Mon Dec 10 17:33:37 2012 From: greg@REDACTED (Greg Burd) Date: Mon, 10 Dec 2012 11:33:37 -0500 Subject: [erlang-questions] Google+ Erlang Advocacy Community Message-ID: Please join this Google+ community as a way to highlight the growing adoption of Erlang and draw others into the fold. https://plus.google.com/communities/106420982717344643903 best, @gregburd From essen@REDACTED Mon Dec 10 17:36:15 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 10 Dec 2012 17:36:15 +0100 Subject: [erlang-questions] Google+ Erlang Advocacy Community In-Reply-To: References: Message-ID: <50C60F7F.3030001@ninenines.eu> On 12/10/2012 05:33 PM, Greg Burd wrote: > Please join this Google+ community as a way to highlight the growing > adoption of Erlang and draw others into the fold. > > https://plus.google.com/communities/106420982717344643903 I wanted to but you can't do communities from the Android app. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From mononcqc@REDACTED Mon Dec 10 17:42:07 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 10 Dec 2012 11:42:07 -0500 Subject: [erlang-questions] Google+ Erlang Advocacy Community In-Reply-To: References: Message-ID: <20121210164206.GA1664@ferdmbp.local> Is it intended to be in any way different from: https://plus.google.com/communities/116304504166177020233 which already counts over a hundred members? BR, Fred. On 12/10, Greg Burd wrote: > Please join this Google+ community as a way to highlight the growing > adoption of Erlang and draw others into the fold. > > https://plus.google.com/communities/106420982717344643903 > > best, > > @gregburd > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From greg@REDACTED Mon Dec 10 17:43:51 2012 From: greg@REDACTED (Greg Burd) Date: Mon, 10 Dec 2012 11:43:51 -0500 Subject: [erlang-questions] Google+ Erlang Advocacy Community In-Reply-To: References: Message-ID: Doh! I didn't find that the other day when I went looking. I don't want to fracture the community. Now all we need is a "merge" button... @gregburd | Basho Technologies | Riak | http://basho.com | @basho On Mon, Dec 10, 2012 at 11:36 AM, Micka?l R?mond wrote: > Hello, > > On Mon, Dec 10, 2012 at 5:33 PM, Greg Burd wrote: >> >> Please join this Google+ community as a way to highlight the growing >> adoption of Erlang and draw others into the fold. >> >> https://plus.google.com/communities/106420982717344643903 >> > > Actually, there is actually a quite big community here: > http://plus.google.com/u/0/communities/116304504166177020233 > > That would be great to focus on that one instead (already 158 members and > growing :) > > -- > Micka?l R?mond > http://www.process-one.net From max.lapshin@REDACTED Mon Dec 10 17:59:26 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 10 Dec 2012 19:59:26 +0300 Subject: [erlang-questions] Using Key/Value store with shared storage In-Reply-To: <20121210113027.GA18095@mryaffle.seand.me.uk> References: <20121210113027.GA18095@mryaffle.seand.me.uk> Message-ID: You speak about many servers but one SAN. What for? Real throughput is limited about 3 gbps. It means that you will be limited in writing no more than 10000 values ok 30kb per second. There will be no cheap way to scale after this limit if you use storage, but if you shard and throw away your raid, you can scale. Maybe yo On Monday, December 10, 2012, Sean D wrote: > Hi all, > > We are currently running an application for a customer that stores a large > number of key/value pairs. Performance is important for us as we need to > maintain a write rate of at least 10,000 keys/second on one server. After > evaluating various key/value stores, we found Bitcask worked extremely well > for us and we went with this. > > The solution currently has multiple servers working independently of each > and we would like to make the solution more resilient by using shared > storage. I.e. If one of the servers goes down, the others can pick up the > work load and add to/read from the same store. > > I am aware that Riak seems to be the standard solution for a resilient > key-value store in the Erlang world, however from my initial > investigations, > this seems to work by duplicating the data between Riak nodes, and this is > something I want to avoid as the number of keys we are storing will be in > the range of 100s of GB and I would prefer that the shared storage is used > rather than data needing to be duplicated. I am also concerned that the > overhead of Riak may prove a bottle-neck, however this isn't something that > I have tested. > > If anyone here has used a key/value store with a SAN or similar in this > way, > I'd be very keen to hear your experiences. > > Many thanks in advance, > Sean > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Mon Dec 10 18:18:47 2012 From: g@REDACTED (Garrett Smith) Date: Mon, 10 Dec 2012 11:18:47 -0600 Subject: [erlang-questions] Using Key/Value store with shared storage In-Reply-To: References: <20121210113027.GA18095@mryaffle.seand.me.uk> Message-ID: Aye, as well, this is curious: > we would like to make the solution more resilient by using shared storage This seems flawed to me. Do you want resiliency, or shared storage? On Mon, Dec 10, 2012 at 10:59 AM, Max Lapshin wrote: > You speak about many servers but one SAN. > What for? Real throughput is limited about 3 gbps. It means that you will be > limited in writing no more than 10000 values ok 30kb per second. > > There will be no cheap way to scale after this limit if you use storage, but > if you shard and throw away your raid, you can scale. > > > > Maybe yo > > On Monday, December 10, 2012, Sean D wrote: >> >> Hi all, >> >> We are currently running an application for a customer that stores a large >> number of key/value pairs. Performance is important for us as we need to >> maintain a write rate of at least 10,000 keys/second on one server. After >> evaluating various key/value stores, we found Bitcask worked extremely >> well >> for us and we went with this. >> >> The solution currently has multiple servers working independently of each >> and we would like to make the solution more resilient by using shared >> storage. I.e. If one of the servers goes down, the others can pick up the >> work load and add to/read from the same store. >> >> I am aware that Riak seems to be the standard solution for a resilient >> key-value store in the Erlang world, however from my initial >> investigations, >> this seems to work by duplicating the data between Riak nodes, and this is >> something I want to avoid as the number of keys we are storing will be in >> the range of 100s of GB and I would prefer that the shared storage is used >> rather than data needing to be duplicated. I am also concerned that the >> overhead of Riak may prove a bottle-neck, however this isn't something >> that >> I have tested. >> >> If anyone here has used a key/value store with a SAN or similar in this >> way, >> I'd be very keen to hear your experiences. >> >> Many thanks in advance, >> Sean >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mahesh@REDACTED Mon Dec 10 19:21:32 2012 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Mon, 10 Dec 2012 10:21:32 -0800 Subject: [erlang-questions] Using Key/Value store with shared storage In-Reply-To: References: <20121210113027.GA18095@mryaffle.seand.me.uk> Message-ID: <9C0EB22C-AC1B-4421-A3C4-8FCE6D1E19DC@dieswaytoofast.com> > This seems flawed to me. Do you want resiliency, or shared storage? Ooooh. Troll! :-) Seriously though - as Max Lapshin points out, w/ the exception of doing some very interesting (read complex, and potentially destabilizing) architecting w/ infiniband, multiple links and cascade setups, you are going to be seriously bottle-necking on your shared storage. And, to Garret's point, resiliency and shared storage are (kinda) orthogonal. An appropriately spilled can of coke can wreak havoc to your shared storage, and BigCouch/Riak/Voldemort/... can be remarkably resilient. A few years back, we migrated out of the "eggs in one basket" SAN approach to BigCouch. Our SAN setup was increasingly starting to look something that would give even Rube Goldberg nightmares, and the sheer amount of hackery associated with this was starting to keep me up at nights. Anyhow, just my two bits... Mahesh Paolini-Subramanya That Tall Bald Indian Guy... Google+ | Blog | Twitter | LinkedIn On Dec 10, 2012, at 9:18 AM, Garrett Smith wrote: > Aye, as well, this is curious: > >> we would like to make the solution more resilient by using shared storage > > > > On Mon, Dec 10, 2012 at 10:59 AM, Max Lapshin wrote: >> You speak about many servers but one SAN. >> What for? Real throughput is limited about 3 gbps. It means that you will be >> limited in writing no more than 10000 values ok 30kb per second. >> >> There will be no cheap way to scale after this limit if you use storage, but >> if you shard and throw away your raid, you can scale. >> >> >> >> Maybe yo >> >> On Monday, December 10, 2012, Sean D wrote: >>> >>> Hi all, >>> >>> We are currently running an application for a customer that stores a large >>> number of key/value pairs. Performance is important for us as we need to >>> maintain a write rate of at least 10,000 keys/second on one server. After >>> evaluating various key/value stores, we found Bitcask worked extremely >>> well >>> for us and we went with this. >>> >>> The solution currently has multiple servers working independently of each >>> and we would like to make the solution more resilient by using shared >>> storage. I.e. If one of the servers goes down, the others can pick up the >>> work load and add to/read from the same store. >>> >>> I am aware that Riak seems to be the standard solution for a resilient >>> key-value store in the Erlang world, however from my initial >>> investigations, >>> this seems to work by duplicating the data between Riak nodes, and this is >>> something I want to avoid as the number of keys we are storing will be in >>> the range of 100s of GB and I would prefer that the shared storage is used >>> rather than data needing to be duplicated. I am also concerned that the >>> overhead of Riak may prove a bottle-neck, however this isn't something >>> that >>> I have tested. >>> >>> If anyone here has used a key/value store with a SAN or similar in this >>> way, >>> I'd be very keen to hear your experiences. >>> >>> Many thanks in advance, >>> Sean >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Mon Dec 10 19:34:37 2012 From: g@REDACTED (Garrett Smith) Date: Mon, 10 Dec 2012 12:34:37 -0600 Subject: [erlang-questions] Using Key/Value store with shared storage In-Reply-To: <9C0EB22C-AC1B-4421-A3C4-8FCE6D1E19DC@dieswaytoofast.com> References: <20121210113027.GA18095@mryaffle.seand.me.uk> <9C0EB22C-AC1B-4421-A3C4-8FCE6D1E19DC@dieswaytoofast.com> Message-ID: On Mon, Dec 10, 2012 at 12:21 PM, Mahesh Paolini-Subramanya wrote: > This seems flawed to me. Do you want resiliency, or shared storage? > > Ooooh. Troll! :-) I wasn't trying, but it is admittedly my nature. But really, shared anything, almost by definition, is a compromise on resiliency, no? Or do I need to talk to some sales guys to get my thinking straight? > Seriously though - as Max Lapshin points out, w/ the exception of doing some > very interesting (read complex, and potentially destabilizing) architecting > w/ infiniband, multiple links and cascade setups, you are going to be > seriously bottle-necking on your shared storage. > And, to Garret's point, resiliency and shared storage are (kinda) > orthogonal. An appropriately spilled can of coke can wreak havoc to your > shared storage, and BigCouch/Riak/Voldemort/... can be remarkably resilient. > > A few years back, we migrated out of the "eggs in one basket" SAN approach > to BigCouch. Our SAN setup was increasingly starting to look something that > would give even Rube Goldberg nightmares, and the sheer amount of hackery > associated with this was starting to keep me up at nights. And?? You never said what happened! Garrett From seand-erlang@REDACTED Mon Dec 10 21:54:19 2012 From: seand-erlang@REDACTED (Sean D) Date: Mon, 10 Dec 2012 20:54:19 +0000 Subject: [erlang-questions] Using Key/Value store with shared storage In-Reply-To: <9C0EB22C-AC1B-4421-A3C4-8FCE6D1E19DC@dieswaytoofast.com> References: <20121210113027.GA18095@mryaffle.seand.me.uk> <9C0EB22C-AC1B-4421-A3C4-8FCE6D1E19DC@dieswaytoofast.com> Message-ID: <20121210205418.GA4507@mryaffle.seand.me.uk> Thanks for the comments. I have included a few points inline. Cheers, Sean On Mon, Dec 10, 2012 at 10:21:32AM -0800, Mahesh Paolini-Subramanya wrote: > This seems flawed to me. Do you want resiliency, or shared storage? > > Ooooh. Troll! :-) > > Seriously though - as Max Lapshin points out, w/ the exception of doing > some very interesting (read complex, and potentially destabilizing) > architecting w/ infiniband, multiple links and cascade setups, you are > going to be seriously bottle-necking on your shared storage. I'm wondering if the term "Shared Storage" has caused some confusion. When I was talking about shared storage, I was talking about using a high-end SAN. The reason for this is because I believe they are designed to deal with this type of scenario. This is obviously dependent on the quality of the SAN though. Budgetary constraints are likely to determine whether or not this will be worth my while. Does anyone have any views on how much overhead there is in keeping nodes in sync in a Riak-type solution. Does this affect I/O performance or does it simply require extra more processing power? > And, to Garret's point, resiliency and shared storage are (kinda) > orthogonal. An appropriately spilled can of coke can wreak havoc to > your shared storage, and BigCouch/Riak/Voldemort/... can be remarkably > resilient. Again, SANs are designed to be highly resilient. I would hope that the appropriate spilling of a can of coke would need to be highly malicious in order to bring down a SAN. I am not doubting any of these technologies are not resilient. I would just rather avoid duplicating data. > A few years back, we migrated out of the "eggs in one basket" SAN > approach to BigCouch. Our SAN setup was increasingly starting to look > something that would give even Rube Goldberg nightmares, and the sheer > amount of hackery associated with this was starting to keep me up at > nights. > > Anyhow, just my two bits... Now this is interesting to me. High maintenance overheads would be offputting. Was your SAN configuration particularly complex? What sort of issues were you having? > [1]Mahesh Paolini-Subramanya > That Tall Bald Indian Guy... > [2]Google+ | [3]Blog | [4]Twitter | [5]LinkedIn > On Dec 10, 2012, at 9:18 AM, Garrett Smith <[6]g@REDACTED> wrote: > > Aye, as well, this is curious: > > we would like to make the solution more resilient by using shared > storage > > On Mon, Dec 10, 2012 at 10:59 AM, Max Lapshin > <[7]max.lapshin@REDACTED> wrote: > > You speak about many servers but one SAN. > What for? Real throughput is limited about 3 gbps. It means that you > will be > limited in writing no more than 10000 values ok 30kb per second. > There will be no cheap way to scale after this limit if you use > storage, but > if you shard and throw away your raid, you can scale. > Maybe yo > On Monday, December 10, 2012, Sean D wrote: > > Hi all, > We are currently running an application for a customer that stores a > large > number of key/value pairs. Performance is important for us as we > need to > maintain a write rate of at least 10,000 keys/second on one server. > After > evaluating various key/value stores, we found Bitcask worked > extremely > well > for us and we went with this. > The solution currently has multiple servers working independently of > each > and we would like to make the solution more resilient by using > shared > storage. I.e. If one of the servers goes down, the others can pick > up the > work load and add to/read from the same store. > I am aware that Riak seems to be the standard solution for a > resilient > key-value store in the Erlang world, however from my initial > investigations, > this seems to work by duplicating the data between Riak nodes, and > this is > something I want to avoid as the number of keys we are storing will > be in > the range of 100s of GB and I would prefer that the shared storage > is used > rather than data needing to be duplicated. I am also concerned that > the > overhead of Riak may prove a bottle-neck, however this isn't > something > that > I have tested. > If anyone here has used a key/value store with a SAN or similar in > this > way, > I'd be very keen to hear your experiences. > Many thanks in advance, > Sean > _______________________________________________ > erlang-questions mailing list > [8]erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > [9]erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > [10]erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > References > > 1. http://www.gravatar.com/avatar/204a87f81a0d9764c1f3364f53e8facf.png > 2. https://plus.google.com/u/0/108074935470209044442/posts > 3. http://dieswaytoofast.blogspot.com/ > 4. https://twitter.com/dieswaytoofast > 5. http://www.linkedin.com/in/dieswaytoofast > 6. mailto:g@REDACTED > 7. mailto:max.lapshin@REDACTED > 8. mailto:erlang-questions@REDACTED > 9. mailto:erlang-questions@REDACTED > 10. mailto:erlang-questions@REDACTED > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mahesh@REDACTED Mon Dec 10 22:29:56 2012 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Mon, 10 Dec 2012 13:29:56 -0800 Subject: [erlang-questions] Using Key/Value store with shared storage In-Reply-To: <20121210205418.GA4507@mryaffle.seand.me.uk> References: <20121210113027.GA18095@mryaffle.seand.me.uk> <9C0EB22C-AC1B-4421-A3C4-8FCE6D1E19DC@dieswaytoofast.com> <20121210205418.GA4507@mryaffle.seand.me.uk> Message-ID: > Now this is interesting to me. High maintenance overheads would be > offputting. Was your SAN configuration particularly complex? What > sort of issues were you having? Once you actually get your fibre/infiniband/10G/whatever adapters, you'll be surprised at how much lower your throughput will be compared to what the spec's say they should be. Once you (and your vendor) are done tuning, reconfiguring, etc., You'll almost certainly end up sharding across different zones, oh, other sorts of fun stuff. The end result is complexity - complex hardware, complex maintenance/monitoring issues, and complex intellectual grappling w/ the various edge-cases that you'll be dealing with. SANs are remarkably good for many things, but for the 'firehose' like throughput that you want/need, they're quite probably not feasible? cheers Mahesh Paolini-Subramanya That Tall Bald Indian Guy... Google+ | Blog | Twitter | LinkedIn On Dec 10, 2012, at 12:54 PM, Sean D wrote: > Thanks for the comments. I have included a few points inline. > > Cheers, > Sean > > On Mon, Dec 10, 2012 at 10:21:32AM -0800, Mahesh Paolini-Subramanya wrote: >> This seems flawed to me. Do you want resiliency, or shared storage? >> >> Ooooh. Troll! :-) >> >> Seriously though - as Max Lapshin points out, w/ the exception of doing >> some very interesting (read complex, and potentially destabilizing) >> architecting w/ infiniband, multiple links and cascade setups, you are >> going to be seriously bottle-necking on your shared storage. > > I'm wondering if the term "Shared Storage" has caused some confusion. When > I was talking about shared storage, I was talking about using a high-end > SAN. > > The reason for this is because I believe they are designed to deal with > this type of scenario. This is obviously dependent on the quality of the > SAN though. Budgetary constraints are likely to determine whether or not > this will be worth my while. > > Does anyone have any views on how much overhead there is in keeping nodes in > sync in a Riak-type solution. Does this affect I/O performance or does it > simply require extra more processing power? > >> And, to Garret's point, resiliency and shared storage are (kinda) >> orthogonal. An appropriately spilled can of coke can wreak havoc to >> your shared storage, and BigCouch/Riak/Voldemort/... can be remarkably >> resilient. > > Again, SANs are designed to be highly resilient. I would hope that the > appropriate spilling of a can of coke would need to be highly malicious in > order to bring down a SAN. > > I am not doubting any of these technologies are not resilient. I would just > rather avoid duplicating data. > >> A few years back, we migrated out of the "eggs in one basket" SAN >> approach to BigCouch. Our SAN setup was increasingly starting to look >> something that would give even Rube Goldberg nightmares, and the sheer >> amount of hackery associated with this was starting to keep me up at >> nights. >> >> Anyhow, just my two bits... > > Now this is interesting to me. High maintenance overheads would be > offputting. Was your SAN configuration particularly complex? What > sort of issues were you having? > >> [1]Mahesh Paolini-Subramanya >> That Tall Bald Indian Guy... >> [2]Google+ | [3]Blog | [4]Twitter | [5]LinkedIn >> On Dec 10, 2012, at 9:18 AM, Garrett Smith <[6]g@REDACTED> wrote: >> >> Aye, as well, this is curious: >> >> we would like to make the solution more resilient by using shared >> storage >> >> On Mon, Dec 10, 2012 at 10:59 AM, Max Lapshin >> <[7]max.lapshin@REDACTED> wrote: >> >> You speak about many servers but one SAN. >> What for? Real throughput is limited about 3 gbps. It means that you >> will be >> limited in writing no more than 10000 values ok 30kb per second. >> There will be no cheap way to scale after this limit if you use >> storage, but >> if you shard and throw away your raid, you can scale. >> Maybe yo >> On Monday, December 10, 2012, Sean D wrote: >> >> Hi all, >> We are currently running an application for a customer that stores a >> large >> number of key/value pairs. Performance is important for us as we >> need to >> maintain a write rate of at least 10,000 keys/second on one server. >> After >> evaluating various key/value stores, we found Bitcask worked >> extremely >> well >> for us and we went with this. >> The solution currently has multiple servers working independently of >> each >> and we would like to make the solution more resilient by using >> shared >> storage. I.e. If one of the servers goes down, the others can pick >> up the >> work load and add to/read from the same store. >> I am aware that Riak seems to be the standard solution for a >> resilient >> key-value store in the Erlang world, however from my initial >> investigations, >> this seems to work by duplicating the data between Riak nodes, and >> this is >> something I want to avoid as the number of keys we are storing will >> be in >> the range of 100s of GB and I would prefer that the shared storage >> is used >> rather than data needing to be duplicated. I am also concerned that >> the >> overhead of Riak may prove a bottle-neck, however this isn't >> something >> that >> I have tested. >> If anyone here has used a key/value store with a SAN or similar in >> this >> way, >> I'd be very keen to hear your experiences. >> Many thanks in advance, >> Sean >> _______________________________________________ >> erlang-questions mailing list >> [8]erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> [9]erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> [10]erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> References >> >> 1. http://www.gravatar.com/avatar/204a87f81a0d9764c1f3364f53e8facf.png >> 2. https://plus.google.com/u/0/108074935470209044442/posts >> 3. http://dieswaytoofast.blogspot.com/ >> 4. https://twitter.com/dieswaytoofast >> 5. http://www.linkedin.com/in/dieswaytoofast >> 6. mailto:g@REDACTED >> 7. mailto:max.lapshin@REDACTED >> 8. mailto:erlang-questions@REDACTED >> 9. mailto:erlang-questions@REDACTED >> 10. mailto:erlang-questions@REDACTED > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Dec 10 23:00:22 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 11 Dec 2012 01:00:22 +0300 Subject: [erlang-questions] Using Key/Value store with shared storage In-Reply-To: References: <20121210113027.GA18095@mryaffle.seand.me.uk> <9C0EB22C-AC1B-4421-A3C4-8FCE6D1E19DC@dieswaytoofast.com> <20121210205418.GA4507@mryaffle.seand.me.uk> Message-ID: No. SANs and other "enterprise" hardware are designed for cases, when vertical scaling is cheaper than horizontal scaling. This cases usually include fixed and predictionable growth of data flows, for example calculating salary or something else. Usually it happens when you cannot control software, for example you need to install Oracle with SAP and it is easier to buy most expensive raid array than to get into SAP sources and ask it to shard (hehe). It is very, very hard to scale such "enterprise" installation 10 times, because scaling price for such devices is not linear. This is why people make "webscale" software, that avoids all that costy hardware. What for to buy RAID controller if you can just copy data to another server and another datacenter? Yes, this HDD will break, but our data is copied to another location. If you stick to SAN device, you loose ability to scale 10 times on slashdot effect during one day, but using 20 slow amazon nodes can give you such possibility. You cannot just go and buy another SAN in wallmart or make your existing device 2 times faster. But of course, expensive SAN is a very nice toy to play if customer is going to pay for it =)) -------------- next part -------------- An HTML attachment was scrubbed... URL: From amit@REDACTED Tue Dec 11 00:46:40 2012 From: amit@REDACTED (Amit Kapoor) Date: Mon, 10 Dec 2012 15:46:40 -0800 Subject: [erlang-questions] erl_ddll:load_driver behaviour. Message-ID: <20121210234640.GD26955@gopher> Hi folks, Trying to understand load_drive behaviour. The test code I am using is at the end of this email. The driver being loaded is a dummy driver with a printf message in the DRIVER_INIT code. If I call spawn:test(true,3), the printf message in DRIVER_INIT code gets displayed 3 times, whereas spawn:test(false,3) displays the same message only once. The erl_ddll manual is a bit lacking in this area, so wanted to get some clarification. 1. Is this the expected behaviour? 2. Child processes seem to 'inherit' drivers loaded by parent process? regards Amit =================================================== -module(spawn). -export([test/2, load_driver/0]). test(_, 0) -> ok; test(Spawn, Count) -> case Spawn of true -> spawn(fun() -> load_driver() end); false -> load_driver() end, timer:sleep(5000), test(Spawn, Count - 1). load_driver() -> case erl_ddll:load_driver("/home/user", test_drv) of ok -> ok; {error, already_loaded} -> ok end. From dan@REDACTED Tue Dec 11 01:12:02 2012 From: dan@REDACTED (Daniel Dormont) Date: Mon, 10 Dec 2012 19:12:02 -0500 Subject: [erlang-questions] Mnesia cluster across OTP Releases Message-ID: Hey gang, Is it possible and/or safe to run a Mnesia cluster across two Erlang nodes running different releases? In particular I have one node on R14B04 and another on R13B03, and when I tried to set up a cluster I got this rather nasty error: ** FATAL ** Failed to merge schema: Incompatible schema storage types (remote). on 'ejabberd@REDACTED' cs [{name,schema},{type,set},{ram_copies,['ejabberd@REDACTED']},{disc_copies,['ejabberd@REDACTED']} ,{disc_only_copies,[]},{load_order,0},{access_mode,read_write},{index,[]},{snmp,[]},{local_content,false},{record_name,schema}, {attributes,[table,cstruct]},{user_properties,[]},{frag_properties,[]},{cookie,{{1351,554882,753639},'ejabberd@REDACTED'}}, {version,{{4,0},{'ejabberd@REDACTED',{1355,182745,138103}}}}], on 'ejabberd@REDACTED' rcs [{name,schema},{type,set},{ram_copies,[]},{disc_copies,['ejabberd@REDACTED','ejabberd@REDACTED']}, {disc_only_copies,[]},{load_order,0},{access_mode,read_write},{index,[]},{snmp,[]},{local_content,false},{record_name,schema},{attributes,[table,cstruct]}, {user_properties,[]},{frag_properties,[]},{cookie,{{1351,554882,753639},'ejabberd@REDACTED'}}, {version,{{5,0},{'ejabberd@REDACTED',{1355,182745,345805}}}}] But I wonder if there was something else wrong in my cluster configuration that caused this. thanks, Dan From vinoski@REDACTED Tue Dec 11 01:27:31 2012 From: vinoski@REDACTED (Steve Vinoski) Date: Mon, 10 Dec 2012 19:27:31 -0500 Subject: [erlang-questions] erl_ddll:load_driver behaviour. In-Reply-To: <20121210234640.GD26955@gopher> References: <20121210234640.GD26955@gopher> Message-ID: On Mon, Dec 10, 2012 at 6:46 PM, Amit Kapoor wrote: > Hi folks, > > Trying to understand load_drive behaviour. The test code I am using is at > the > end of this email. The driver being loaded is a dummy driver with a printf > message in the DRIVER_INIT code. If I call spawn:test(true,3), the printf > message in DRIVER_INIT code gets displayed 3 times, whereas > spawn:test(false,3) > displays the same message only once. The erl_ddll manual is a bit lacking > in > this area, so wanted to get some clarification. > > 1. Is this the expected behaviour? > This is expected because in the true case you're spawning a new process that loads the driver, and when that process exits, the driver gets unloaded. Near the top of the erl_ddll man page you can find this passage: "The driver is always reference counted and as long as a process keeping the driver loaded is still alive, the driver is present in the system." If you go into an erlang shell and run the false case first, then run the true case, the 3 true attempts will all get already_loaded because your shell process already loaded the driver in the first false case, and assuming that shell process is still alive, the true cases have no need to load it. > 2. Child processes seem to 'inherit' drivers loaded by parent process? > I don't think so. If the parent loads the driver and doesn't die, then the driver remains loaded and the child will also be able to see the driver. You can use erl_ddll:info(test_drv, processes) anytime after loading it to see what processes have a ref count on it. For example, after running spawn:test(false, 3) in your shell you'll see from erl_ddll:info(test_drv, processes) that the shell process has a ref count of 3 on test_drv. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From sven.heyll@REDACTED Tue Dec 11 02:36:23 2012 From: sven.heyll@REDACTED (Sven Heyll) Date: Tue, 11 Dec 2012 02:36:23 +0100 Subject: [erlang-questions] erlymock update Message-ID: Hi TDD-Junkies https://github.com/sheyll/erlymock I recently polished it a little, removed maven support and finally added an alternative to 'em:verify/1' call 'em:await_expectations' that actually blocks until all expected invokations have happed. This removes the need for clumsy unit test hacks where one would block the test process until a message is received, that is sent by some '{function, fun(_) -> TestProc ! go_on end}'. I created a gh-pages branch so there is actually documentation at http://sheyll.github.com/erlymock/ Maven support has been removed for now, unless someone want to use it, I will not add it back again, because the maven-erlang plugin includes erlymock. This release totally relies on rebar for building and dependency management. Thanks for reading, Sven -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Tue Dec 11 08:14:49 2012 From: dangud@REDACTED (Dan Gudmundsson) Date: Tue, 11 Dec 2012 08:14:49 +0100 Subject: [erlang-questions] Mnesia cluster across OTP Releases In-Reply-To: References: Message-ID: Yes should be possible except when I (and Ulf :-) mess things up, which we did in mnesia-4.4.19. In which mnesia couldn't talk to older versions. I don't think that is not your problem here: You have two different versions of the schema content here: old: {version,{{4,0},{'ejabberd@REDACTED',{1355,182745,138103}} new: {version,{{5,0},{'ejabberd@REDACTED',{1355,182745,345805}} Mnesia will not allow them to be merged, you have done something with the table def before the merge. /Dan On Tue, Dec 11, 2012 at 1:12 AM, Daniel Dormont wrote: > Hey gang, > > Is it possible and/or safe to run a Mnesia cluster across two Erlang > nodes running different releases? In particular I have one node on > R14B04 and another on R13B03, and when I tried to set up a cluster I > got this rather nasty error: > > ** FATAL ** Failed to merge schema: Incompatible schema storage types > (remote). on 'ejabberd@REDACTED' cs > [{name,schema},{type,set},{ram_copies,['ejabberd@REDACTED > ']},{disc_copies,['ejabberd@REDACTED']} > > ,{disc_only_copies,[]},{load_order,0},{access_mode,read_write},{index,[]},{snmp,[]},{local_content,false},{record_name,schema}, > > {attributes,[table,cstruct]},{user_properties,[]},{frag_properties,[]},{cookie,{{1351,554882,753639},'ejabberd@REDACTED > '}}, > {version,{{4,0},{'ejabberd@REDACTED',{1355,182745,138103}}}}], > on 'ejabberd@REDACTED' rcs > [{name,schema},{type,set},{ram_copies,[]},{disc_copies,['ejabberd@REDACTED > ','ejabberd@REDACTED']}, > > {disc_only_copies,[]},{load_order,0},{access_mode,read_write},{index,[]},{snmp,[]},{local_content,false},{record_name,schema},{attributes,[table,cstruct]}, > > {user_properties,[]},{frag_properties,[]},{cookie,{{1351,554882,753639},'ejabberd@REDACTED > '}}, > {version,{{5,0},{'ejabberd@REDACTED',{1355,182745,345805}}}}] > > But I wonder if there was something else wrong in my cluster > configuration that caused this. > > thanks, > Dan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyron.zerafa@REDACTED Tue Dec 11 11:40:51 2012 From: tyron.zerafa@REDACTED (Tyron Zerafa) Date: Tue, 11 Dec 2012 11:40:51 +0100 Subject: [erlang-questions] Local vs external function calls Message-ID: Hey, I was reading about Erjang (here) and it implements an interesting design in which it differentiates between local and external function calls. Is there anything similar in Erlang? What is the main difference between local and external functions? I have been trying some things out and it seems that they have identical semantics when passed to a remote node. Is there anything I am missing? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From Tobias.Schlager@REDACTED Tue Dec 11 14:54:32 2012 From: Tobias.Schlager@REDACTED (Tobias Schlager) Date: Tue, 11 Dec 2012 13:54:32 +0000 Subject: [erlang-questions] [ANN] eipmi - Erlang and IPMI Message-ID: <12F2115FD1CCEE4294943B2608A18FA335AF496A@MAIL01.win.lbaum.eu> Hi all, anybody interested in an implementation of IPMI in Erlang? http://www.intel.com/design/servers/ipmi/ IPMI is an industry standard for hardware management and monitoring originally released by Intel. Moreover, the PICMG consortium adopted it for their set of hardware standards (CompactPCI, AdvancedTCA, MicroTCA, etc.) which are widely in use in the Telecom industry. https://github.com/lindenbaum/eipmi The last weeks I've worked on an Erlang implementation of an IPMI "Remote Console". This is basically the "client side" of the IPMI over LAN specification. It's implemented as a rebar-built Erlang/OTP application including documentation & basic examples. If this sounds interesting to you, be sure to check it out. Contributions and/or bug reports are welcome. Regards Tobias From pan@REDACTED Tue Dec 11 16:59:19 2012 From: pan@REDACTED (Patrik Nyblom) Date: Tue, 11 Dec 2012 16:59:19 +0100 Subject: [erlang-questions] Local vs external function calls In-Reply-To: References: Message-ID: <50C75857.1050803@erlang.org> Hi! On 12/11/2012 11:40 AM, Tyron Zerafa wrote: > Hey, > > I was reading about Erjang (here) > and > it implements an interesting design in which it differentiates between > local and external function calls. Is there anything similar in > Erlang? What is the main difference between local and external functions? > I have been trying some things out and it seems that they have > identical semantics when passed to a remote node. Is there anything I > am missing? > Erjang *is* Erlang, or one implementation of it. The "regular" beam VM implements the exact semantics described in the article. > Thanks > > Cheers, /Patrik > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Wed Dec 12 10:27:42 2012 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Wed, 12 Dec 2012 11:27:42 +0200 Subject: [erlang-questions] time consuming operations inside gen_server Message-ID: <50C84E0E.7070901@gmail.com> Hi all, In our application, we have a gen_server that does a time consuming operation. The message is of type cast thus the caller process doesn't sit and wait for the operation to finish. But while the gen_server is busy with the cast message, it doesn't serve any other call, right? So, would it be appropriate to create a process that will do the time consuming operation and then notify the gen_server? Thanks for looking into this. Best regards, Martin From attila.r.nohl@REDACTED Wed Dec 12 10:42:02 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 12 Dec 2012 10:42:02 +0100 Subject: [erlang-questions] time consuming operations inside gen_server In-Reply-To: <50C84E0E.7070901@gmail.com> References: <50C84E0E.7070901@gmail.com> Message-ID: 2012/12/12 Martin Dimitrov : > Hi all, > > In our application, we have a gen_server that does a time consuming > operation. The message is of type cast thus the caller process doesn't > sit and wait for the operation to finish. But while the gen_server is > busy with the cast message, it doesn't serve any other call, right? > > So, would it be appropriate to create a process that will do the time > consuming operation and then notify the gen_server? The pattern I use in this case is to still use gen_server:call (if the caller needs to be blocked), start a separate process for the time-consuming stuff, return {noreply, ...} from the handle_call (so the gen_server can handle other calls), then call gen_server:reply from the started process when the time-consuming operation finished. Of course, the applicability of this pattern depends on what you actually do. From mrtndimitrov@REDACTED Wed Dec 12 10:46:59 2012 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Wed, 12 Dec 2012 11:46:59 +0200 Subject: [erlang-questions] time consuming operations inside gen_server In-Reply-To: References: <50C84E0E.7070901@gmail.com> Message-ID: <50C85293.3050202@gmail.com> Thanks, this sounds very cool. On 12/12/2012 11:42 AM, Attila Rajmund Nohl wrote: > 2012/12/12 Martin Dimitrov : >> Hi all, >> >> In our application, we have a gen_server that does a time consuming >> operation. The message is of type cast thus the caller process doesn't >> sit and wait for the operation to finish. But while the gen_server is >> busy with the cast message, it doesn't serve any other call, right? >> >> So, would it be appropriate to create a process that will do the time >> consuming operation and then notify the gen_server? > > The pattern I use in this case is to still use gen_server:call (if the > caller needs to be blocked), start a separate process for the > time-consuming stuff, return {noreply, ...} from the handle_call (so > the gen_server can handle other calls), then call gen_server:reply > from the started process when the time-consuming operation finished. > Of course, the applicability of this pattern depends on what you > actually do. > From r.gafiyatullin@REDACTED Wed Dec 12 10:58:00 2012 From: r.gafiyatullin@REDACTED (Roman Gafiyatullin) Date: Wed, 12 Dec 2012 12:58:00 +0300 Subject: [erlang-questions] time consuming operations inside gen_server In-Reply-To: <50C85293.3050202@gmail.com> References: <50C84E0E.7070901@gmail.com> <50C85293.3050202@gmail.com> Message-ID: <0C8C24A36E064937AC0908051F51D54C@me.com> Hello Martin, In this cases the problem is usually solved as Attila Rajmund Nohl said. Since it's the common situation I wrote a lib for that: https://github.com/RGafiyatullin/gen_wp See src/gen_wp_example.erl (https://github.com/RGafiyatullin/gen_wp/blob/master/src/gen_wp_example.erl) for how to use it. Questions and constructive critics are welcome and appreciated. -- RG On Wednesday, December 12, 2012 at 12:46 pm, Martin Dimitrov wrote: > Thanks, this sounds very cool. > > On 12/12/2012 11:42 AM, Attila Rajmund Nohl wrote: > > 2012/12/12 Martin Dimitrov : > > > Hi all, > > > > > > In our application, we have a gen_server that does a time consuming > > > operation. The message is of type cast thus the caller process doesn't > > > sit and wait for the operation to finish. But while the gen_server is > > > busy with the cast message, it doesn't serve any other call, right? > > > > > > So, would it be appropriate to create a process that will do the time > > > consuming operation and then notify the gen_server? > > > > > > > > > The pattern I use in this case is to still use gen_server:call (if the > > caller needs to be blocked), start a separate process for the > > time-consuming stuff, return {noreply, ...} from the handle_call (so > > the gen_server can handle other calls), then call gen_server:reply > > from the started process when the time-consuming operation finished. > > Of course, the applicability of this pattern depends on what you > > actually do. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Wed Dec 12 11:04:18 2012 From: bourinov@REDACTED (Max Bourinov) Date: Wed, 12 Dec 2012 14:04:18 +0400 Subject: [erlang-questions] time consuming operations inside gen_server In-Reply-To: <50C85293.3050202@gmail.com> References: <50C84E0E.7070901@gmail.com> <50C85293.3050202@gmail.com> Message-ID: Hi Martin, First of all I try to avoid calls in my systems. I believe that is there is a call - there is a potential lock. Casts are friends. In your case I would suggest you to have "special" workers for time consuming operations. Workers should be gen_servers and should be a part of your supervisors tree. This is just a general suggestion - of course each case is unique. Please adapt it to your case if possible. Best regards, Max On Wed, Dec 12, 2012 at 1:46 PM, Martin Dimitrov wrote: > Thanks, this sounds very cool. > > On 12/12/2012 11:42 AM, Attila Rajmund Nohl wrote: > > 2012/12/12 Martin Dimitrov : > >> Hi all, > >> > >> In our application, we have a gen_server that does a time consuming > >> operation. The message is of type cast thus the caller process doesn't > >> sit and wait for the operation to finish. But while the gen_server is > >> busy with the cast message, it doesn't serve any other call, right? > >> > >> So, would it be appropriate to create a process that will do the time > >> consuming operation and then notify the gen_server? > > > > The pattern I use in this case is to still use gen_server:call (if the > > caller needs to be blocked), start a separate process for the > > time-consuming stuff, return {noreply, ...} from the handle_call (so > > the gen_server can handle other calls), then call gen_server:reply > > from the started process when the time-consuming operation finished. > > Of course, the applicability of this pattern depends on what you > > actually do. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Wed Dec 12 11:11:25 2012 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Wed, 12 Dec 2012 12:11:25 +0200 Subject: [erlang-questions] time consuming operations inside gen_server In-Reply-To: References: <50C84E0E.7070901@gmail.com> <50C85293.3050202@gmail.com> Message-ID: <50C8584D.8080108@gmail.com> Hi, This is actually a cast but since it takes long the gen_server will be blocked and I want to avoid this. Thanks for the replies, Martin On 12/12/2012 12:04 PM, Max Bourinov wrote: > Hi Martin, > > First of all I try to avoid calls in my systems. I believe that is there is > a call - there is a potential lock. Casts are friends. > > In your case I would suggest you to have "special" workers for time > consuming operations. Workers should be gen_servers and should be a part of > your supervisors tree. > > This is just a general suggestion - of course each case is unique. Please > adapt it to your case if possible. > > Best regards, > Max > > > > > On Wed, Dec 12, 2012 at 1:46 PM, Martin Dimitrov wrote: > >> Thanks, this sounds very cool. >> >> On 12/12/2012 11:42 AM, Attila Rajmund Nohl wrote: >>> 2012/12/12 Martin Dimitrov : >>>> Hi all, >>>> >>>> In our application, we have a gen_server that does a time consuming >>>> operation. The message is of type cast thus the caller process doesn't >>>> sit and wait for the operation to finish. But while the gen_server is >>>> busy with the cast message, it doesn't serve any other call, right? >>>> >>>> So, would it be appropriate to create a process that will do the time >>>> consuming operation and then notify the gen_server? >>> >>> The pattern I use in this case is to still use gen_server:call (if the >>> caller needs to be blocked), start a separate process for the >>> time-consuming stuff, return {noreply, ...} from the handle_call (so >>> the gen_server can handle other calls), then call gen_server:reply >>> from the started process when the time-consuming operation finished. >>> Of course, the applicability of this pattern depends on what you >>> actually do. >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From mrtndimitrov@REDACTED Wed Dec 12 11:27:23 2012 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Wed, 12 Dec 2012 12:27:23 +0200 Subject: [erlang-questions] time consuming operations inside gen_server In-Reply-To: <0C8C24A36E064937AC0908051F51D54C@me.com> References: <50C84E0E.7070901@gmail.com> <50C85293.3050202@gmail.com> <0C8C24A36E064937AC0908051F51D54C@me.com> Message-ID: <50C85C0B.9030608@gmail.com> I browsed through the files and looks very interesting. What does "wp" stand for? Martin On 12/12/2012 11:58 AM, Roman Gafiyatullin wrote: > Hello Martin, > > In this cases the problem is usually solved as Attila Rajmund Nohl said. > Since it's the common situation I wrote a lib for that: https://github.com/RGafiyatullin/gen_wp > See src/gen_wp_example.erl (https://github.com/RGafiyatullin/gen_wp/blob/master/src/gen_wp_example.erl) for how to use it. > > Questions and constructive critics are welcome and appreciated. > > -- > RG > > > On Wednesday, December 12, 2012 at 12:46 pm, Martin Dimitrov wrote: > >> Thanks, this sounds very cool. >> >> On 12/12/2012 11:42 AM, Attila Rajmund Nohl wrote: >>> 2012/12/12 Martin Dimitrov : >>>> Hi all, >>>> >>>> In our application, we have a gen_server that does a time consuming >>>> operation. The message is of type cast thus the caller process doesn't >>>> sit and wait for the operation to finish. But while the gen_server is >>>> busy with the cast message, it doesn't serve any other call, right? >>>> >>>> So, would it be appropriate to create a process that will do the time >>>> consuming operation and then notify the gen_server? >>>> >>> >>> >>> The pattern I use in this case is to still use gen_server:call (if the >>> caller needs to be blocked), start a separate process for the >>> time-consuming stuff, return {noreply, ...} from the handle_call (so >>> the gen_server can handle other calls), then call gen_server:reply >>> from the started process when the time-consuming operation finished. >>> Of course, the applicability of this pattern depends on what you >>> actually do. >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > From mjtruog@REDACTED Wed Dec 12 11:28:52 2012 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 12 Dec 2012 02:28:52 -0800 Subject: [erlang-questions] time consuming operations inside gen_server In-Reply-To: <50C84E0E.7070901@gmail.com> References: <50C84E0E.7070901@gmail.com> Message-ID: <50C85C64.5060706@gmail.com> On 12/12/2012 01:27 AM, Martin Dimitrov wrote: > So, would it be appropriate to create a process that will do the time > consuming operation and then notify the gen_server? Yes, processes are cheap in Erlang. There is no reason not to create a separate process for a task, that runs for awhile, to make sure the task doesn't stop your gen_server from processing its message queue. You could do it with a erlang:spawn_link or supervisor:start_child, both of which are typically used in this type of a situation. You want to avoid using erlang:process_flag(trap_exit, true), because it is simpler to have a supervisor manage the process, than have manual, custom supervisor-like functionality (within your gen_server). There is another reason which long-running tasks often need separate processes, which is a separate concern. If memory is used quickly, in a large quantity, you want the garbage collector to run quickly to free memory for other processes. The short-lived process causes the garbage collector to collect when it dies (all the memory from the task), which would otherwise be a problem in a longer-lived process (since the memory would just accumulate, and would not be collected quickly). From max.lapshin@REDACTED Wed Dec 12 11:29:24 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 12 Dec 2012 13:29:24 +0300 Subject: [erlang-questions] time consuming operations inside gen_server In-Reply-To: <50C8584D.8080108@gmail.com> References: <50C84E0E.7070901@gmail.com> <50C85293.3050202@gmail.com> <50C8584D.8080108@gmail.com> Message-ID: Call is a throttling and overload control mechanism. If your worker is too slow you will ruine your erlang vm with several millions of casts in the message box On Wednesday, December 12, 2012, Martin Dimitrov wrote: > Hi, > > This is actually a cast but since it takes long the gen_server will be > blocked and I want to avoid this. > > Thanks for the replies, > > Martin > > On 12/12/2012 12:04 PM, Max Bourinov wrote: > > Hi Martin, > > > > First of all I try to avoid calls in my systems. I believe that is there > is > > a call - there is a potential lock. Casts are friends. > > > > In your case I would suggest you to have "special" workers for time > > consuming operations. Workers should be gen_servers and should be a part > of > > your supervisors tree. > > > > This is just a general suggestion - of course each case is unique. Please > > adapt it to your case if possible. > > > > Best regards, > > Max > > > > > > > > > > On Wed, Dec 12, 2012 at 1:46 PM, Martin Dimitrov > >wrote: > > > >> Thanks, this sounds very cool. > >> > >> On 12/12/2012 11:42 AM, Attila Rajmund Nohl wrote: > >>> 2012/12/12 Martin Dimitrov >: > >>>> Hi all, > >>>> > >>>> In our application, we have a gen_server that does a time consuming > >>>> operation. The message is of type cast thus the caller process doesn't > >>>> sit and wait for the operation to finish. But while the gen_server is > >>>> busy with the cast message, it doesn't serve any other call, right? > >>>> > >>>> So, would it be appropriate to create a process that will do the time > >>>> consuming operation and then notify the gen_server? > >>> > >>> The pattern I use in this case is to still use gen_server:call (if the > >>> caller needs to be blocked), start a separate process for the > >>> time-consuming stuff, return {noreply, ...} from the handle_call (so > >>> the gen_server can handle other calls), then call gen_server:reply > >>> from the started process when the time-consuming operation finished. > >>> Of course, the applicability of this pattern depends on what you > >>> actually do. > >>> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Dec 12 11:30:05 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 12 Dec 2012 11:30:05 +0100 Subject: [erlang-questions] time consuming operations inside gen_server In-Reply-To: References: <50C84E0E.7070901@gmail.com> <50C85293.3050202@gmail.com> Message-ID: <50C85CAD.9070106@ninenines.eu> Workers shouldn't be gen_servers. If they are, they are quite poor ones considering you'll have a hard time inspecting it, debugging it or tracing it because your sys:get_status/1 call will simply timeout while the gen_server is busy doing whatever it's doing. You also likely have a single call/cast for it, so you don't use much of gen_server itself. It's OK to write processes that aren't gen_servers, especially in cases like this. All you have to do is spawn a supervised worker passing it your Pid, monitor it and then wait for either it to go down or finish the task and give a result which you'll get back as a message. The worker can simply stop after it sends said message. If you don't expect a result, it's even simpler! On 12/12/2012 11:04 AM, Max Bourinov wrote: > Hi Martin, > > First of all I try to avoid calls in my systems. I believe that is there > is a call - there is a potential lock. Casts are friends. > > In your case I would suggest you to have "special" workers for time > consuming operations. Workers should be gen_servers and should be a part > of your supervisors tree. > > This is just a general suggestion - of course each case is unique. > Please adapt it to your case if possible. > > Best regards, > Max > > > > > On Wed, Dec 12, 2012 at 1:46 PM, Martin Dimitrov > wrote: > > Thanks, this sounds very cool. > > On 12/12/2012 11:42 AM, Attila Rajmund Nohl wrote: > > 2012/12/12 Martin Dimitrov >: > >> Hi all, > >> > >> In our application, we have a gen_server that does a time consuming > >> operation. The message is of type cast thus the caller process > doesn't > >> sit and wait for the operation to finish. But while the > gen_server is > >> busy with the cast message, it doesn't serve any other call, right? > >> > >> So, would it be appropriate to create a process that will do the > time > >> consuming operation and then notify the gen_server? > > > > The pattern I use in this case is to still use gen_server:call > (if the > > caller needs to be blocked), start a separate process for the > > time-consuming stuff, return {noreply, ...} from the handle_call (so > > the gen_server can handle other calls), then call gen_server:reply > > from the started process when the time-consuming operation finished. > > Of course, the applicability of this pattern depends on what you > > actually do. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From mononcqc@REDACTED Wed Dec 12 14:23:26 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 12 Dec 2012 08:23:26 -0500 Subject: [erlang-questions] time consuming operations inside gen_server In-Reply-To: <50C84E0E.7070901@gmail.com> References: <50C84E0E.7070901@gmail.com> Message-ID: <20121212132324.GA10495@ferdmbp.local> Hi Martin, I gave a quick read to this thread and there are a few things I think should be mentioned in order to make a decision. I'm writing this as some kind of general guide I follow mentally, so please do not feel patronized if you find I approach things at a basic level that's too simple for your level of expertise. I'm writing it for you, but also for myself (or anyone else finding it over google or whatever). I believe you won't solve this problem by leaving things as they are, but there are properties to figure out regarding the kind of work you're doing: 1. Is this queue build-up related to temporary overflow? Does it happen at peak time, in bursts, or is it a continuous overflow? 2. Are the tasks you're running in any way bound by time? What I mean here is to ask how long you're allowed to wait. Is it milliseconds, seconds, or hours, before a cast is a problem? 3. Are you in charge of producing the events in-system, or is it something triggered by user actions, outside of your control? 4. Why does it take long to process? Is it a problem due to CPU-bound problems, depending on other workers, I/O bound problems (disk, network) slowing your server down? 5. What's the nature of events you're handling? Answering each of these questions will be the first step to being able to pick an adequate solution. Here are a few possibilities: - if you're in charge of producing events (your system creates them from some static data source, for example) and can regulate them, by having a fixed number of producers and synchronous calls to put back pressure from your server to the workers. They won't do more work than the consuming part of your system can handle. In general terms, applying back-pressure this way is the most efficient way to solve and survive all overload issues. It's a bit tricky because it means you're pushing the problem up a level in your stack, until at some point you lower your issues with pressure or that at some point you push the backpressure back to users, and that's sometimes not acceptable. Pushing it back to some load-balancing mechanism that dispatches through more instances is often acceptable as an alternative. - You may expect tasks to be long to run, but to be fast to be acknowledged. In this case, moving to an asynchronous model makes sense. This can be done by spawning workers to do tasks while the server simply accepts the queries, responds to them, and queues up answers that have yet to come. This form of concurrency is different from adding processes for the sake of parallelism. We don't expect to handle more requests (or at least more than Original*NumberOfCores), we just want to make the accepting/response of events and theur handling disjoint, not happening in the same timeline to avoid blocking. - If requests are to be very short-lived, it may be interesting to just not handle them when the system is very busy, and fail. This is a bit more tricky to put in place, and I believe it's rarely a good solution based on the nature of the problem you solve. Systems where you're allowed to give up and not do handle things are a bit rare, I believe. - If your handling of events is slow due to the task simply being long to handle, you could try to figure the ideal rate at which you process data and the overload you could handle in peak hours. This means figuring out how many requests per second (or whatever time slice) you can handle, how much you receive, and then finding a way to raise this value through optimization, or adding more processes or more machines to handle it. This is often a good way to proceed, as far as I know, when you deal with predictable levels of overload or a constant level of overload. If the problem is being CPU-bound, then there's an upper limit to how much parallelism will help you. Better algorithm or data structure choice, going down to HiPE or C, or finally using more computers to do the work can all be considered. If it's network or disk bound, then you have plenty of different options to try. SSDs, compressing data, buffering before pushing it around, possibly merging events or dropping non-vital ones, adding more end-points (similar to sharding) may all help reduce that cost. - It's possible you have different kind of events, either from different sources or to different endpoints. If that happens, it may be interesting to quickly dispatch events from your central process to workers dedicated to a source and/or an endpoint. This will naturally divide the workload done and may solve your problem to some extent. If what you get is extremely uneven distribution of events (for example, 95% of them are from one source to one endpoint), then divinding your dispatching and handling is likely to only help about 5% of requests. - Dropping out of OTP is a possibility, as Lo?c suggested, but I would personally only do it once you know for a fact OTP is one of the bottleneck causing problems. I think OTP-by-default is a sane thing to have, and while dropping down is always an option, I think you should be able to debate and prove why it made sense to do it before doing it. - If you manage to fix your sequential bottleneck, you'll possibly find out you're creating a new one further down the system, up until you either get rid of all of them, or you reach a point where you need to apply back-pressure at a hard limit. This one is particularly painful because it may mean parts of your hard work need to be undone to start bubbling the back-pressure mechanisms up until a higher level. It may be interesting to make sure you know the true underlying cause of your problem there, to avoid optimizing towards a wall that way. That's about what I can manage to think of this morning. I hope this proves helpful to anybody out there and that I didn't insert to many typos or errors. Regards, Fred. On 12/12, Martin Dimitrov wrote: > Hi all, > > In our application, we have a gen_server that does a time consuming > operation. The message is of type cast thus the caller process doesn't > sit and wait for the operation to finish. But while the gen_server is > busy with the cast message, it doesn't serve any other call, right? > > So, would it be appropriate to create a process that will do the time > consuming operation and then notify the gen_server? > > Thanks for looking into this. > > Best regards, > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From serge@REDACTED Wed Dec 12 14:47:20 2012 From: serge@REDACTED (Serge Aleynikov) Date: Wed, 12 Dec 2012 08:47:20 -0500 Subject: [erlang-questions] time consuming operations inside gen_server In-Reply-To: References: <50C84E0E.7070901@gmail.com> <50C85293.3050202@gmail.com> Message-ID: <50C88AE8.20901@aleynikov.org> Max, If the system implementation is entirely based on casts, then it would only work property under small loads, otherwise its lack of congestion control would be subject to "fast producer slow consumer" problems. In the later case a gen_server that for one reason or another is not fetching messages from its message queue fast enough would accumulate many messages in its queue which will degrade performance of the entire system. On the other hand, the use of calls removes this problem by slowing down the producer, and the gen_server's implementer has a design decision to make in terms of how to ensure that producer is only minimally delayed and the server is scalable. This may indeed involve spawning processes from gen_server or using a pool of resources, and returning an error when all resources are busy, or some other design. Serge On 12/12/2012 5:04 AM, Max Bourinov wrote: > Hi Martin, > > First of all I try to avoid calls in my systems. I believe that is there > is a call - there is a potential lock. Casts are friends. > > In your case I would suggest you to have "special" workers for time > consuming operations. Workers should be gen_servers and should be a part > of your supervisors tree. > > This is just a general suggestion - of course each case is unique. > Please adapt it to your case if possible. > > Best regards, > Max > > > > > On Wed, Dec 12, 2012 at 1:46 PM, Martin Dimitrov > wrote: > > Thanks, this sounds very cool. > > On 12/12/2012 11:42 AM, Attila Rajmund Nohl wrote: > > 2012/12/12 Martin Dimitrov >: > >> Hi all, > >> > >> In our application, we have a gen_server that does a time consuming > >> operation. The message is of type cast thus the caller process > doesn't > >> sit and wait for the operation to finish. But while the gen_server is > >> busy with the cast message, it doesn't serve any other call, right? > >> > >> So, would it be appropriate to create a process that will do the time > >> consuming operation and then notify the gen_server? > > > > The pattern I use in this case is to still use gen_server:call (if the > > caller needs to be blocked), start a separate process for the > > time-consuming stuff, return {noreply, ...} from the handle_call (so > > the gen_server can handle other calls), then call gen_server:reply > > from the started process when the time-consuming operation finished. > > Of course, the applicability of this pattern depends on what you > > actually do. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From paulperegud@REDACTED Wed Dec 12 16:20:19 2012 From: paulperegud@REDACTED (Paul Peregud) Date: Wed, 12 Dec 2012 16:20:19 +0100 Subject: [erlang-questions] simple_one_for_one supervisor, transient restart strategy Message-ID: I've searched mailing list for this question and found nothing. There is this one peculiar behavior when you use simple_one_for_one with transient strategy. You do supervisor:start_child(Ref, Args), the child executes and if it dies gracefully everything is fine, child is removed from supervisor tree. However, if it fails, it is restarted WITHOUT Args. This combination of type/strategy would be a great tool for fire-and-forget tasks if it was not for that small issue. Is it a bug or is it a feature? -- Best regards, Paul Peregud +48602112091 From naikvin@REDACTED Wed Dec 12 16:31:48 2012 From: naikvin@REDACTED (Vineet Naik) Date: Wed, 12 Dec 2012 21:01:48 +0530 Subject: [erlang-questions] Running rebar generated release failing with "Kernel pid terminated" Message-ID: Hello, I am trying to package my app using rebar. `rebar generate` command runs without any problem (no errors at least). But when I try to start the console by running the script generated by rebar in bin directory, it fails with "Kernel pid terminated" A huge crash_dump is also created but I am not able to gather any info from it. I checked the erlang docs[1], I found this paragraph about the error "Kernel pid terminated (Who) (Exit-reason)" - The kernel supervisor has > detected a failure, usually that the application_controller has shut down > (Who = application_controller, Why = shutdown). The application controller > may have shut down for a number of reasons, the most usual being that the > node name of the distributed Erlang node is already in use. A complete > supervisor tree "crash" (i.e., the top supervisors have exited) will give > about the same result. This message comes from the Erlang code and not from > the virtual machine itself. It is always due to some kind of failure in an > application, either within OTP or a "user-written" one. Looking at the > error log for your application is probably the first step to take. If I run `console_clean` command, then the console start in the correct node. Does this rule out the "Node is already in use" case above? I also tried running the code from an erlang shell with the ebin dir paths of all required libs added manually to sys path and it works perfectly. How can I go about debugging this? [1] http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 Thanks, Vineet -------------- next part -------------- An HTML attachment was scrubbed... URL: From r.gafiyatullin@REDACTED Wed Dec 12 16:57:16 2012 From: r.gafiyatullin@REDACTED (Roman Gafiyatullin) Date: Wed, 12 Dec 2012 18:57:16 +0300 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: References: Message-ID: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> Vineet, most probably something went wrong during startup. Can anything similar to sasl-error.log be found in the /log/ directory? -- RG On Wednesday, December 12, 2012 at 6:31 pm, Vineet Naik wrote: > Hello, > > I am trying to package my app using rebar. `rebar generate` command runs without any > problem (no errors at least). But when I try to start the console by running the script generated > by rebar in bin directory, it fails with "Kernel pid terminated" > > A huge crash_dump is also created but I am not able to gather any info from it. > > I checked the erlang docs[1], I found this paragraph about the error > > > "Kernel pid terminated (Who) (Exit-reason)" - The kernel supervisor has detected a failure, usually that the application_controller has shut down (Who = application_controller, Why = shutdown). The application controller may have shut down for a number of reasons, the most usual being that the node name of the distributed Erlang node is already in use. A complete supervisor tree "crash" (i.e., the top supervisors have exited) will give about the same result. This message comes from the Erlang code and not from the virtual machine itself. It is always due to some kind of failure in an application, either within OTP or a "user-written" one. Looking at the error log for your application is probably the first step to take. > > If I run `console_clean` command, then the console start in the correct node. Does > this rule out the "Node is already in use" case above? > > I also tried running the code from an erlang shell with the ebin dir paths of all required > libs added manually to sys path and it works perfectly. > > How can I go about debugging this? > > [1] http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 > > Thanks, > Vineet > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Dec 12 17:18:57 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 12 Dec 2012 17:18:57 +0100 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? In-Reply-To: <20121206193809.GC27411@alumni.caltech.edu> References: <20121206193809.GC27411@alumni.caltech.edu> Message-ID: <15C0D662-A77C-4858-AD93-3306E72317F6@erlang-solutions.com> On Dec 6, 2012, at 8:38 PM, Anthony Molinaro wrote: > 2012-12-01 00:01:33 =CRASH REPORT==== > crasher: > initial call: mochiweb_acceptor:init/3 > pid: <0.2942.3599> > registered_name: [] > exception error: {{badmatch,{error,einval}},[{mochiweb_http,new_request,3,[]},{mochiweb_http,handle_invalid_request,3,[]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} > ancestors: [webmachine_mochiweb,web_serve_sup,<0.619.0>] > messages: [{tcp_closed,#Port<0.222190389>}] > links: [<0.873.0>] > dictionary: [] > trap_exit: false > status: running > heap_size: 987 > stack_size: 24 > reductions: 541 > neighbours: If you read through the Mochi code, you will see this: https://github.com/mochi/mochiweb/blob/master/src/mochiweb_http.erl#L125 ok = mochiweb_socket:setopts(Socket, [{packet, raw}]), So assuming that is true, the problem is a posix() message einval (Invalid Argument), probably because the socket is long dead and gone. It may be that R15 is more accurate at reporting here than R14, so the older version just accepts the result blindly and then have the no change, whereas R15 actually reports the correct error. A thing supporting my hypothesis is that there is a tcp_closed message in your mailbox :) Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From DanielLewis@REDACTED Wed Dec 12 17:42:02 2012 From: DanielLewis@REDACTED (Daniel Lewis) Date: Wed, 12 Dec 2012 16:42:02 +0000 Subject: [erlang-questions] Help running Erlang as a Service on XP Embedded Message-ID: <11FAA6E3726EDF479B4D97FC1C14B9039855C027@USASHEXMB05.LYV.LiveNation.com> Hi, I need some help getting Erlang to run as a Windows service on XP Embedded. I am using RabbitMQ for a project I have been assigned, which has Elang as a dependency. This works well on Windows 7, and Windows XP, but I also have to target XP Embedded. This is a non-clustered configuration (currently all components are on one machine) Everything runs well when I start all the components in the foreground. If I start it as a service, I see epmd.exe, erl.exe, and erlsrv.exe running, but it seems like the broker does not respond: C:\PROGRA~1\TICKET~1\TMDS\rabbitmq\sbin>rabbitmqctl.bat status "c:\progra~1\ticketmaster\tmds\erlang\bin\erl.exe" -pa "C:\PROGRA~1\TICKET~1\TMDS\rabbitmq\sbin\..\ebin" -noinput -hidden -sname rabbitmqctl -s rabbit_control -nodename rabbit@REDACTED -extra status Status of node 'rabbit@REDACTED' ... Error: unable to connect to node 'rabbit@REDACTED': nodedown diagnostics: - nodes and their ports on DANIEL-WYSE: [{'RabbitMQ',1052},{rabbitmqctl,3072}] - current node: 'rabbitmqctl@REDACTED' - current node home dir: C:\Documents and Settings\Administrator - current node cookie hash: ak3qKWQRwDfzEGk9nwYiVQ== Are there possibly permissions issues? I can grant the service permissions if I knew which ones applied. Any other ideas? I'd appreciate any help I can get Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From rustompmody@REDACTED Wed Dec 12 18:58:36 2012 From: rustompmody@REDACTED (Rustom Mody) Date: Wed, 12 Dec 2012 23:28:36 +0530 Subject: [erlang-questions] noob namespace question Message-ID: In Erlang variables start with a capital letter, atoms start with a lowercase letter. Functions also start with lowercase. How does Erlang distinguish atoms and functions? Especially in the context of higher order functions.? [Disclaimer: noob to Erlang but not functional programming or Lisp. Erlang atoms look quite close to Lisp atoms though also different...] -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Dec 12 19:00:25 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 12 Dec 2012 19:00:25 +0100 Subject: [erlang-questions] noob namespace question In-Reply-To: References: Message-ID: <50C8C639.6060006@ninenines.eu> On 12/12/2012 06:58 PM, Rustom Mody wrote: > In Erlang variables start with a capital letter, atoms start with a > lowercase letter. > Functions also start with lowercase. > > How does Erlang distinguish atoms and functions? Especially in the > context of higher order functions.? > > [Disclaimer: noob to Erlang but not functional programming or Lisp. > Erlang atoms look quite close to Lisp atoms though also different...] Function names are atoms. Funs don't have a name (yet) so there's no naming happening there. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From bourinov@REDACTED Wed Dec 12 19:08:35 2012 From: bourinov@REDACTED (Max Bourinov) Date: Wed, 12 Dec 2012 22:08:35 +0400 Subject: [erlang-questions] noob namespace question In-Reply-To: <50C8C639.6060006@ninenines.eu> References: <50C8C639.6060006@ninenines.eu> Message-ID: Hi Rustom, note that 'CapitelLetter' is also an atom. On Wed, Dec 12, 2012 at 10:00 PM, Lo?c Hoguin wrote: > On 12/12/2012 06:58 PM, Rustom Mody wrote: > >> In Erlang variables start with a capital letter, atoms start with a >> lowercase letter. >> Functions also start with lowercase. >> >> How does Erlang distinguish atoms and functions? Especially in the >> context of higher order functions.? >> >> [Disclaimer: noob to Erlang but not functional programming or Lisp. >> Erlang atoms look quite close to Lisp atoms though also different...] >> > > Function names are atoms. > > Funs don't have a name (yet) so there's no naming happening there. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rustompmody@REDACTED Wed Dec 12 19:14:43 2012 From: rustompmody@REDACTED (Rustom Mody) Date: Wed, 12 Dec 2012 23:44:43 +0530 Subject: [erlang-questions] noob namespace question In-Reply-To: <50C8C639.6060006@ninenines.eu> References: <50C8C639.6060006@ninenines.eu> Message-ID: On Wed, Dec 12, 2012 at 11:30 PM, Lo?c Hoguin wrote: > On 12/12/2012 06:58 PM, Rustom Mody wrote: > >> In Erlang variables start with a capital letter, atoms start with a >> lowercase letter. >> Functions also start with lowercase. >> >> How does Erlang distinguish atoms and functions? Especially in the >> context of higher order functions.? >> >> [Disclaimer: noob to Erlang but not functional programming or Lisp. >> Erlang atoms look quite close to Lisp atoms though also different...] >> > > Function names are atoms. > > Funs don't have a name (yet) so there's no naming happening there. > I dont understand. Let me try and be more explicit about my concern. Let h be a higher-order function of one argument (h/1) Now in the call h(foo) How does Erlang know whether foo is a function or an atom? -------------- next part -------------- An HTML attachment was scrubbed... URL: From james@REDACTED Wed Dec 12 19:24:53 2012 From: james@REDACTED (James Aimonetti) Date: Wed, 12 Dec 2012 10:24:53 -0800 Subject: [erlang-questions] noob namespace question In-Reply-To: References: <50C8C639.6060006@ninenines.eu> Message-ID: <20121212102453.515a2053@thinky64.2600hz.com> On Wed, 12 Dec 2012 23:44:43 +0530 Rustom Mody wrote: > On Wed, Dec 12, 2012 at 11:30 PM, Lo?c Hoguin wrote: > > > On 12/12/2012 06:58 PM, Rustom Mody wrote: > > > >> In Erlang variables start with a capital letter, atoms start with a > >> lowercase letter. > >> Functions also start with lowercase. > >> > >> How does Erlang distinguish atoms and functions? Especially in the > >> context of higher order functions.? > >> > >> [Disclaimer: noob to Erlang but not functional programming or Lisp. > >> Erlang atoms look quite close to Lisp atoms though also different...] > >> > > > > Function names are atoms. > > > > Funs don't have a name (yet) so there's no naming happening there. > > > > I dont understand. Let me try and be more explicit about my concern. > > Let h be a higher-order function of one argument (h/1) > > Now in the call h(foo) > How does Erlang know whether foo is a function or an atom? fun h/1 is the function h of arity 1 foo is an atom fun foo/2 is the function foo of arity 2 So to pass the function foo to h, h(fun foo/2). Does that help? -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: not available URL: From rtrlists@REDACTED Wed Dec 12 19:27:21 2012 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 12 Dec 2012 18:27:21 +0000 Subject: [erlang-questions] Help running Erlang as a Service on XP Embedded In-Reply-To: <11FAA6E3726EDF479B4D97FC1C14B9039855C027@USASHEXMB05.LYV.LiveNation.com> References: <11FAA6E3726EDF479B4D97FC1C14B9039855C027@USASHEXMB05.LYV.LiveNation.com> Message-ID: It is probably an Erlang cookie mismatch. I can't off the top of my head remember where Erlang stores the cookies by default. I always set them explicitly. The mismatch is likely, since your Service will be running in a different context to your status invocation. Hope this helps, Robby On Dec 12, 2012 4:43 PM, "Daniel Lewis" wrote: > Hi,**** > > ** ** > > I need some help getting Erlang to run as a Windows service on XP Embedded. > **** > > ** ** > > I am using RabbitMQ for a project I have been assigned, which has Elang as > a dependency. This works well on Windows 7, and Windows XP, but I also have > to target XP Embedded. This is a non-clustered configuration (currently > all components are on one machine)**** > > ** ** > > Everything runs well when I start all the components in the foreground.*** > * > > ** ** > > If I start it as a service, I see epmd.exe, erl.exe, and erlsrv.exe > running, but it seems like the broker does not respond:**** > > ** ** > > **** > > C:\PROGRA~1\TICKET~1\TMDS\rabbitmq\sbin>rabbitmqctl.bat status**** > > ** ** > > "c:\progra~1\ticketmaster\tmds\erlang\bin\erl.exe" -pa > "C:\PROGRA~1\TICKET~1\TMDS\rabbitmq\sbin\..\ebin" -noinput -hidden -sname > rabbitmqctl -s rabbit_control**** > > -nodename rabbit@REDACTED -extra status**** > > ** ** > > Status of node 'rabbit@REDACTED' ...**** > > Error: unable to connect to node 'rabbit@REDACTED': nodedown**** > > diagnostics:**** > > - nodes and their ports on DANIEL-WYSE: > [{'RabbitMQ',1052},{rabbitmqctl,3072}]**** > > - current node: 'rabbitmqctl@REDACTED'**** > > - current node home dir: C:\Documents and Settings\Administrator**** > > - current node cookie hash: ak3qKWQRwDfzEGk9nwYiVQ==**** > > ** ** > > ** ** > > Are there possibly permissions issues? I can grant the service permissions > if I knew which ones applied. Any other ideas? I'd appreciate any help I > can get**** > > ** ** > > Thanks!**** > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Wed Dec 12 19:27:44 2012 From: siraaj@REDACTED (Siraaj Khandkar) Date: Wed, 12 Dec 2012 13:27:44 -0500 Subject: [erlang-questions] noob namespace question In-Reply-To: References: <50C8C639.6060006@ninenines.eu> Message-ID: <0A0C98A3-6AC9-4EE0-B893-A87776E0928D@khandkar.net> On Dec 12, 2012, at 13:14, Rustom Mody wrote: > > > On Wed, Dec 12, 2012 at 11:30 PM, Lo?c Hoguin wrote: > On 12/12/2012 06:58 PM, Rustom Mody wrote: > In Erlang variables start with a capital letter, atoms start with a > lowercase letter. > Functions also start with lowercase. > > How does Erlang distinguish atoms and functions? Especially in the > context of higher order functions.? > > [Disclaimer: noob to Erlang but not functional programming or Lisp. > Erlang atoms look quite close to Lisp atoms though also different...] > > Function names are atoms. > > Funs don't have a name (yet) so there's no naming happening there. > > I dont understand. Let me try and be more explicit about my concern. > > Let h be a higher-order function of one argument (h/1) > > Now in the call h(foo) > How does Erlang know whether foo is a function or an atom? Syntactic context :) Parser is a state machine that interprets tokens based on current parse state/context. Generally, I like to think of atoms as first class building blocks that references to modules and functions (among other things) can be constructed out of. -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Wed Dec 12 19:26:32 2012 From: anthonym@REDACTED (Anthony Molinaro) Date: Wed, 12 Dec 2012 10:26:32 -0800 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? In-Reply-To: <15C0D662-A77C-4858-AD93-3306E72317F6@erlang-solutions.com> References: <20121206193809.GC27411@alumni.caltech.edu> <15C0D662-A77C-4858-AD93-3306E72317F6@erlang-solutions.com> Message-ID: <20121212182632.GB72594@alumni.caltech.edu> Well, yes, I did see the tcp_closed message, my real question is I guess related to your statement "It may be that R15 is more accurate at reporting here than R14, so the older version just accepts the result blindly and then have the no change, whereas R15 actually reports the correct error." Can anyone from OTP clarify if any work was done with regards to error messages? I did not find anything in the release notes which is why I'm wondering. I suppose I could attempt to git bisect this or something, but I'm not sure how difficult that might be. Thanks, -Anthony On Wed, Dec 12, 2012 at 05:18:57PM +0100, Jesper Louis Andersen wrote: > > On Dec 6, 2012, at 8:38 PM, Anthony Molinaro wrote: > > > 2012-12-01 00:01:33 =CRASH REPORT==== > > crasher: > > initial call: mochiweb_acceptor:init/3 > > pid: <0.2942.3599> > > registered_name: [] > > exception error: {{badmatch,{error,einval}},[{mochiweb_http,new_request,3,[]},{mochiweb_http,handle_invalid_request,3,[]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} > > ancestors: [webmachine_mochiweb,web_serve_sup,<0.619.0>] > > messages: [{tcp_closed,#Port<0.222190389>}] > > links: [<0.873.0>] > > dictionary: [] > > trap_exit: false > > status: running > > heap_size: 987 > > stack_size: 24 > > reductions: 541 > > neighbours: > > If you read through the Mochi code, you will see this: > > https://github.com/mochi/mochiweb/blob/master/src/mochiweb_http.erl#L125 > > ok = mochiweb_socket:setopts(Socket, [{packet, raw}]), > > So assuming that is true, the problem is a posix() message einval (Invalid Argument), probably because the > socket is long dead and gone. It may be that R15 is more accurate at reporting here than R14, so the older version just accepts the result blindly and then have the no change, whereas R15 actually reports the correct error. > > A thing supporting my hypothesis is that there is a tcp_closed message in your mailbox :) > > > Jesper Louis Andersen > Erlang Solutions Ltd., Copenhagen > > > -- ------------------------------------------------------------------------ Anthony Molinaro From thomas@REDACTED Wed Dec 12 19:30:03 2012 From: thomas@REDACTED (Thomas Allen) Date: Wed, 12 Dec 2012 13:30:03 -0500 Subject: [erlang-questions] noob namespace question In-Reply-To: References: <50C8C639.6060006@ninenines.eu> Message-ID: <20121212183003.GB13660@members.linode.com> On Wed, Dec 12, 2012 at 11:44:43PM +0530, Rustom Mody wrote: > I dont understand. Let me try and be more explicit about my concern. > > Let h be a higher-order function of one argument (h/1) > > Now in the call h(foo) > How does Erlang know whether foo is a function or an atom? This is not valid, and you will get a "bad function" error if you try to pass your function this way. You must create a "fun" like so: h(fun foo/0) Similarly, you could pass an anonymous fun (like fun(X) -> X end), or a module-function combination (like fun lists:reverse/1). Thomas Allen From thomas@REDACTED Wed Dec 12 19:38:56 2012 From: thomas@REDACTED (Thomas Allen) Date: Wed, 12 Dec 2012 13:38:56 -0500 Subject: [erlang-questions] noob namespace question In-Reply-To: <20121212183003.GB13660@members.linode.com> References: <50C8C639.6060006@ninenines.eu> <20121212183003.GB13660@members.linode.com> Message-ID: <20121212183856.GA15600@members.linode.com> On Wed, Dec 12, 2012 at 01:30:03PM -0500, Thomas Allen wrote: > This is not valid, and you will get a "bad function" error if you try to > pass your function this way. You must create a "fun" like so: Note that there are ways to make atoms as functions usable. You will have to export functions you are calling this way, even if they are internal to your module: -module(foo). %% API. -export([main/0]). %% Private. -export([a/0, b/0]). %% API. main() -> test1(a) + test2(b). %% Private. test1(Fn) -> ?MODULE:Fn(). test2(Fn) -> apply(?MODULE, Fn, []). a() -> 1. b() -> 2. You will see that: 1> foo:main(). 3 These are two ways to call a function with nothing but an atom. Thomas Allen From rustompmody@REDACTED Wed Dec 12 19:40:26 2012 From: rustompmody@REDACTED (Rustom Mody) Date: Thu, 13 Dec 2012 00:10:26 +0530 Subject: [erlang-questions] noob namespace question In-Reply-To: <20121212183003.GB13660@members.linode.com> References: <50C8C639.6060006@ninenines.eu> <20121212183003.GB13660@members.linode.com> Message-ID: On Thu, Dec 13, 2012 at 12:00 AM, Thomas Allen wrote: > On Wed, Dec 12, 2012 at 11:44:43PM +0530, Rustom Mody wrote: > > I dont understand. Let me try and be more explicit about my concern. > > > > Let h be a higher-order function of one argument (h/1) > > > > Now in the call h(foo) > > How does Erlang know whether foo is a function or an atom? > > This is not valid, and you will get a "bad function" error if you try to > pass your function this way. You must create a "fun" like so: > > h(fun foo/0) > > Similarly, you could pass an anonymous fun (like fun(X) -> X end), or > a module-function combination (like fun lists:reverse/1). > > Thanks Thomas that answers succinctly. In Lisp terminology Erlang is more like a Lisp-2 than a Lisp-1 http://en.wikipedia.org/wiki/Common_Lisp#The_function_namespace -- http://www.the-magus.in http://blog.languager.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rustompmody@REDACTED Wed Dec 12 19:45:34 2012 From: rustompmody@REDACTED (Rustom Mody) Date: Thu, 13 Dec 2012 00:15:34 +0530 Subject: [erlang-questions] doc question Message-ID: At the head of http://www.erlang.org/doc/design_principles/applications.html it says "read in conjunction with app(4) and application(3)" The "app(4)" and "application(3)" are not links. What/where to search for?? Rusi -------------- next part -------------- An HTML attachment was scrubbed... URL: From peralta.alejandro@REDACTED Wed Dec 12 19:51:29 2012 From: peralta.alejandro@REDACTED (Ale) Date: Wed, 12 Dec 2012 15:51:29 -0300 Subject: [erlang-questions] doc question In-Reply-To: References: Message-ID: 2012/12/12 Rustom Mody > At the head of > http://www.erlang.org/doc/design_principles/applications.html > it says "read in conjunction with app(4) and application(3)" > > The "app(4)" and "application(3)" are not links. What/where to search > for?? > > erl -man app erl -man application is my best guess (because app is in section 5) > Rusi > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Ale. -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Wed Dec 12 19:52:16 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Wed, 12 Dec 2012 19:52:16 +0100 Subject: [erlang-questions] doc question In-Reply-To: References: Message-ID: On Wed, Dec 12, 2012 at 7:45 PM, Rustom Mody wrote: > At the head of http://www.erlang.org/doc/design_principles/applications.html > it says "read in conjunction with app(4) and application(3)" > > The "app(4)" and "application(3)" are not links. What/where to search for?? Manual pages: $ man 4 app (although only man 5 app works for me) $ man 3 application You can get Erlang manual by using "erl -man Something", where Something is an Erlang-related thing you are looking for. This saves you from remembering numbers and getting unexpected results. Compare in your shell: $ man file $ erl -man file -- Motiejus Jak?tys From dan@REDACTED Wed Dec 12 20:01:39 2012 From: dan@REDACTED (Daniel Dormont) Date: Wed, 12 Dec 2012 14:01:39 -0500 Subject: [erlang-questions] doc question In-Reply-To: References: Message-ID: Web version: http://www.erlang.org/doc/apps/kernel/application.html http://www.erlang.org/doc/man/app.html On Wed, Dec 12, 2012 at 1:52 PM, Motiejus Jak?tys wrote: > On Wed, Dec 12, 2012 at 7:45 PM, Rustom Mody wrote: >> At the head of http://www.erlang.org/doc/design_principles/applications.html >> it says "read in conjunction with app(4) and application(3)" >> >> The "app(4)" and "application(3)" are not links. What/where to search for?? > > Manual pages: > $ man 4 app (although only man 5 app works for me) > $ man 3 application > > You can get Erlang manual by using "erl -man Something", where > Something is an Erlang-related thing you are looking for. This saves > you from remembering numbers and getting unexpected results. Compare > in your shell: > > $ man file > $ erl -man file > > -- > Motiejus Jak?tys > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From DanielLewis@REDACTED Wed Dec 12 20:41:21 2012 From: DanielLewis@REDACTED (Daniel Lewis) Date: Wed, 12 Dec 2012 19:41:21 +0000 Subject: [erlang-questions] Help running Erlang as a Service on XP Embedded In-Reply-To: References: <11FAA6E3726EDF479B4D97FC1C14B9039855C027@USASHEXMB05.LYV.LiveNation.com> Message-ID: <11FAA6E3726EDF479B4D97FC1C14B9039855C1DB@USASHEXMB05.LYV.LiveNation.com> Hi Robby, Thanks for the suggestion. I searched for files called ".erlang.cookie" and I only see two of them - should there be more? I checked and these two files are identical, including contents, size, and date. Any other ideas that may help? Best Regards, Daniel From: Robert Raschke [mailto:rtrlists@REDACTED] Sent: Wednesday, December 12, 2012 11:27 AM To: Daniel Lewis Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Help running Erlang as a Service on XP Embedded It is probably an Erlang cookie mismatch. I can't off the top of my head remember where Erlang stores the cookies by default. I always set them explicitly. The mismatch is likely, since your Service will be running in a different context to your status invocation. Hope this helps, Robby On Dec 12, 2012 4:43 PM, "Daniel Lewis" > wrote: Hi, I need some help getting Erlang to run as a Windows service on XP Embedded. I am using RabbitMQ for a project I have been assigned, which has Elang as a dependency. This works well on Windows 7, and Windows XP, but I also have to target XP Embedded. This is a non-clustered configuration (currently all components are on one machine) Everything runs well when I start all the components in the foreground. If I start it as a service, I see epmd.exe, erl.exe, and erlsrv.exe running, but it seems like the broker does not respond: C:\PROGRA~1\TICKET~1\TMDS\rabbitmq\sbin>rabbitmqctl.bat status "c:\progra~1\ticketmaster\tmds\erlang\bin\erl.exe" -pa "C:\PROGRA~1\TICKET~1\TMDS\rabbitmq\sbin\..\ebin" -noinput -hidden -sname rabbitmqctl -s rabbit_control -nodename rabbit@REDACTED -extra status Status of node 'rabbit@REDACTED' ... Error: unable to connect to node 'rabbit@REDACTED': nodedown diagnostics: - nodes and their ports on DANIEL-WYSE: [{'RabbitMQ',1052},{rabbitmqctl,3072}] - current node: 'rabbitmqctl@REDACTED' - current node home dir: C:\Documents and Settings\Administrator - current node cookie hash: ak3qKWQRwDfzEGk9nwYiVQ== Are there possibly permissions issues? I can grant the service permissions if I knew which ones applied. Any other ideas? I'd appreciate any help I can get Thanks! _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dch@REDACTED Wed Dec 12 20:52:41 2012 From: dch@REDACTED (Dave Cottlehuber) Date: Wed, 12 Dec 2012 20:52:41 +0100 Subject: [erlang-questions] Help running Erlang as a Service on XP Embedded In-Reply-To: <11FAA6E3726EDF479B4D97FC1C14B9039855C1DB@USASHEXMB05.LYV.LiveNation.com> References: <11FAA6E3726EDF479B4D97FC1C14B9039855C027@USASHEXMB05.LYV.LiveNation.com> <11FAA6E3726EDF479B4D97FC1C14B9039855C1DB@USASHEXMB05.LYV.LiveNation.com> Message-ID: On 12 December 2012 20:41, Daniel Lewis wrote: > Hi Robby, > > > > Thanks for the suggestion. > > > > I searched for files called ".erlang.cookie" and I only see two of them - > should there be more? I checked and these two files are identical, > including contents, size, and date. Likely if you are running this as a service the home directory will not be where you are expecting it to be. This was covered in erlang-question archives in the last few months, I can't recall the specifics off the top of my head. Try adding `-setcookie yumyumyum` to the batch file as suggested earlier. You might be able to try setting this up with erlsrv.exe and adding `-debugtype console` to the parameters. You can then interact with the hidden service console (winsta0) directly. I don't know if this works on embedded XP but its a good way to try. I'm not sure on the rabbit parameters but your shortname probably should be "erl -sname rabbitmqctl@REDACTED" if you want to set the @?. section directly. A+ Dave From rtrlists@REDACTED Wed Dec 12 21:55:39 2012 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 12 Dec 2012 20:55:39 +0000 Subject: [erlang-questions] Help running Erlang as a Service on XP Embedded In-Reply-To: References: <11FAA6E3726EDF479B4D97FC1C14B9039855C027@USASHEXMB05.LYV.LiveNation.com> <11FAA6E3726EDF479B4D97FC1C14B9039855C1DB@USASHEXMB05.LYV.LiveNation.com> Message-ID: Hi Daniel, a few things to verify: Is RabbitMQ actually up and running? That is, can you use it normally, connect, create/use exchanges, queues etc.? Is the REST API running? Or the managment plugin? If yes, then the next step would be to fix the cookie in the bat files. For that you'll need to unregister the service again, then edit the register service bat to include the -setcookie argument, and as Dave mentions it might be worth ensuring that your local name is set to include the host part explicitly as well here. Also add the -setcookie into the rabbitmqctl.bat. If your service is in fact not usable at all, then having a closer look at the log files is in order. Actually, looking at the log files is a good idea in any case. Regards, Robby On Wed, Dec 12, 2012 at 7:52 PM, Dave Cottlehuber wrote: > On 12 December 2012 20:41, Daniel Lewis > wrote: > > Hi Robby, > > > > > > > > Thanks for the suggestion. > > > > > > > > I searched for files called ".erlang.cookie" and I only see two of them - > > should there be more? I checked and these two files are identical, > > including contents, size, and date. > > Likely if you are running this as a service the home directory will > not be where you are expecting it to be. This was covered in > erlang-question archives in the last few months, I can't recall the > specifics off the top of my head. > > Try adding `-setcookie yumyumyum` to the batch file as suggested earlier. > > You might be able to try setting this up with erlsrv.exe and adding > `-debugtype console` to the parameters. You can then interact with the > hidden service console (winsta0) directly. I don't know if this works > on embedded XP but its a good way to try. > > I'm not sure on the rabbit parameters but your shortname probably > should be "erl -sname rabbitmqctl@REDACTED" if you want to set the > @?. section directly. > > A+ > Dave > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From DanielLewis@REDACTED Wed Dec 12 23:36:31 2012 From: DanielLewis@REDACTED (Daniel Lewis) Date: Wed, 12 Dec 2012 22:36:31 +0000 Subject: [erlang-questions] Help running Erlang as a Service on XP Embedded In-Reply-To: References: <11FAA6E3726EDF479B4D97FC1C14B9039855C027@USASHEXMB05.LYV.LiveNation.com> <11FAA6E3726EDF479B4D97FC1C14B9039855C1DB@USASHEXMB05.LYV.LiveNation.com> Message-ID: <11FAA6E3726EDF479B4D97FC1C14B9039855C235@USASHEXMB05.LYV.LiveNation.com> Hi Robby and Dave I tried Dave's suggestions - I notice no difference when I set the cookie to "yumyumyum" in the batch files (I still get the "nodedown" message for rabbit@REDACTED) The things Robby wanted to verify do not work. I cannot use it normally, when I try to connect I get a "Could not connect to Broker" message. http://localhost:55672/ returns a message that "Internet Explorer cannot display the webpage" Also, I do not see any log files with today's date on them... they all have old dates, and are probably from when I was running this in the foreground From: Robert Raschke [mailto:rtrlists@REDACTED] Sent: Wednesday, December 12, 2012 1:56 PM To: Daniel Lewis Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Help running Erlang as a Service on XP Embedded Hi Daniel, a few things to verify: Is RabbitMQ actually up and running? That is, can you use it normally, connect, create/use exchanges, queues etc.? Is the REST API running? Or the managment plugin? If yes, then the next step would be to fix the cookie in the bat files. For that you'll need to unregister the service again, then edit the register service bat to include the -setcookie argument, and as Dave mentions it might be worth ensuring that your local name is set to include the host part explicitly as well here. Also add the -setcookie into the rabbitmqctl.bat. If your service is in fact not usable at all, then having a closer look at the log files is in order. Actually, looking at the log files is a good idea in any case. Regards, Robby On Wed, Dec 12, 2012 at 7:52 PM, Dave Cottlehuber > wrote: On 12 December 2012 20:41, Daniel Lewis > wrote: > Hi Robby, > > > > Thanks for the suggestion. > > > > I searched for files called ".erlang.cookie" and I only see two of them - > should there be more? I checked and these two files are identical, > including contents, size, and date. Likely if you are running this as a service the home directory will not be where you are expecting it to be. This was covered in erlang-question archives in the last few months, I can't recall the specifics off the top of my head. Try adding `-setcookie yumyumyum` to the batch file as suggested earlier. You might be able to try setting this up with erlsrv.exe and adding `-debugtype console` to the parameters. You can then interact with the hidden service console (winsta0) directly. I don't know if this works on embedded XP but its a good way to try. I'm not sure on the rabbit parameters but your shortname probably should be "erl -sname rabbitmqctl@REDACTED" if you want to set the @?. section directly. A+ Dave _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Thu Dec 13 00:48:03 2012 From: anthonym@REDACTED (Anthony Molinaro) Date: Wed, 12 Dec 2012 15:48:03 -0800 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? In-Reply-To: <20121212182632.GB72594@alumni.caltech.edu> References: <20121206193809.GC27411@alumni.caltech.edu> <15C0D662-A77C-4858-AD93-3306E72317F6@erlang-solutions.com> <20121212182632.GB72594@alumni.caltech.edu> Message-ID: <20121212234803.GA75710@alumni.caltech.edu> Okay, here's some more information, seems like the real error is {tcp_error,#Port<0.98735301>,emsgsize} earlier. So maybe sometime between R14B04 and R15B02 the handling of emsgsize changed? I think what happens with mochiweb is it has a receive which matches several message types, then has a catch all. The catch all gets the emsgsize error and the tcp_closed error is in the mailbox, then the code handling the catch all attempts to setopts on the socket and gets the einval error because the socket is closed. So I'm really curious what might have changed in the tcp driver that might have changed this behavior. I'm trying to figure out how to reproduce so I can patch mochiweb to work around this (most likely by just adding a clause to the receive). -Anthony On Wed, Dec 12, 2012 at 10:26:32AM -0800, Anthony Molinaro wrote: > Well, yes, I did see the tcp_closed message, my real question is I guess > related to your statement > > "It may be that R15 is more accurate at reporting here than R14, so the older version just accepts the result blindly and then have the no change, whereas R15 actually reports the correct error." > > Can anyone from OTP clarify if any work was done with regards to error > messages? I did not find anything in the release notes which is why > I'm wondering. > > I suppose I could attempt to git bisect this or something, but I'm not > sure how difficult that might be. > > Thanks, > > -Anthony > > On Wed, Dec 12, 2012 at 05:18:57PM +0100, Jesper Louis Andersen wrote: > > > > On Dec 6, 2012, at 8:38 PM, Anthony Molinaro wrote: > > > > > 2012-12-01 00:01:33 =CRASH REPORT==== > > > crasher: > > > initial call: mochiweb_acceptor:init/3 > > > pid: <0.2942.3599> > > > registered_name: [] > > > exception error: {{badmatch,{error,einval}},[{mochiweb_http,new_request,3,[]},{mochiweb_http,handle_invalid_request,3,[]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} > > > ancestors: [webmachine_mochiweb,web_serve_sup,<0.619.0>] > > > messages: [{tcp_closed,#Port<0.222190389>}] > > > links: [<0.873.0>] > > > dictionary: [] > > > trap_exit: false > > > status: running > > > heap_size: 987 > > > stack_size: 24 > > > reductions: 541 > > > neighbours: > > > > If you read through the Mochi code, you will see this: > > > > https://github.com/mochi/mochiweb/blob/master/src/mochiweb_http.erl#L125 > > > > ok = mochiweb_socket:setopts(Socket, [{packet, raw}]), > > > > So assuming that is true, the problem is a posix() message einval (Invalid Argument), probably because the > > socket is long dead and gone. It may be that R15 is more accurate at reporting here than R14, so the older version just accepts the result blindly and then have the no change, whereas R15 actually reports the correct error. > > > > A thing supporting my hypothesis is that there is a tcp_closed message in your mailbox :) > > > > > > Jesper Louis Andersen > > Erlang Solutions Ltd., Copenhagen > > > > > > > > -- > ------------------------------------------------------------------------ > Anthony Molinaro > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ------------------------------------------------------------------------ Anthony Molinaro From tristan.sloughter@REDACTED Thu Dec 13 01:05:31 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Wed, 12 Dec 2012 18:05:31 -0600 Subject: [erlang-questions] SSL record overflow Message-ID: This issue appeared using couchbeam to connect to a Cloudant database. But it appears to be related to the ssl app, not couchbeam or ibrowse. When trying to make a request to the server in R15B03 I get: ** exception error: no match of right hand side value {error,{conn_failed,{error,"record overflow"}}} and in R14 its: SSL: hello: ./ssl_record.erl:364:Fatal error: record overflow ** exception error: no match of right hand side value {error,{conn_failed,{error,"record overflow"}}} Has anyone experienced a similar issue? Suggestions on what I should do to debug this issue? Thanks, Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Thu Dec 13 01:22:37 2012 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 12 Dec 2012 19:22:37 -0500 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? In-Reply-To: <20121212234803.GA75710@alumni.caltech.edu> References: <20121206193809.GC27411@alumni.caltech.edu> <15C0D662-A77C-4858-AD93-3306E72317F6@erlang-solutions.com> <20121212182632.GB72594@alumni.caltech.edu> <20121212234803.GA75710@alumni.caltech.edu> Message-ID: On Wed, Dec 12, 2012 at 6:48 PM, Anthony Molinaro < anthonym@REDACTED> wrote: > Okay, here's some more information, seems like the real error is > > {tcp_error,#Port<0.98735301>,emsgsize} > > earlier. So maybe sometime between R14B04 and R15B02 the handling of > emsgsize changed? > Hi Anthony, any chance your app is dealing with some very long HTTP header lines? In November 2011 I submitted a patch that was intended to stop HTTP DOS attacks consisting of huge header lines. It's commit 5984409d, which came just after R14B04 was released in October 2011. --steve > > I think what happens with mochiweb is it has a receive which matches > several > message types, then has a catch all. The catch all gets the emsgsize error > and the tcp_closed error is in the mailbox, then the code handling the > catch > all attempts to setopts on the socket and gets the einval error because > the socket is closed. > > So I'm really curious what might have changed in the tcp driver that might > have changed this behavior. I'm trying to figure out how to reproduce so > I can patch mochiweb to work around this (most likely by just adding a > clause to the receive). > > -Anthony > > On Wed, Dec 12, 2012 at 10:26:32AM -0800, Anthony Molinaro wrote: > > Well, yes, I did see the tcp_closed message, my real question is I guess > > related to your statement > > > > "It may be that R15 is more accurate at reporting here than R14, so the > older version just accepts the result blindly and then have the no change, > whereas R15 actually reports the correct error." > > > > Can anyone from OTP clarify if any work was done with regards to error > > messages? I did not find anything in the release notes which is why > > I'm wondering. > > > > I suppose I could attempt to git bisect this or something, but I'm not > > sure how difficult that might be. > > > > Thanks, > > > > -Anthony > > > > On Wed, Dec 12, 2012 at 05:18:57PM +0100, Jesper Louis Andersen wrote: > > > > > > On Dec 6, 2012, at 8:38 PM, Anthony Molinaro < > anthonym@REDACTED> wrote: > > > > > > > 2012-12-01 00:01:33 =CRASH REPORT==== > > > > crasher: > > > > initial call: mochiweb_acceptor:init/3 > > > > pid: <0.2942.3599> > > > > registered_name: [] > > > > exception error: > {{badmatch,{error,einval}},[{mochiweb_http,new_request,3,[]},{mochiweb_http,handle_invalid_request,3,[]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} > > > > ancestors: [webmachine_mochiweb,web_serve_sup,<0.619.0>] > > > > messages: [{tcp_closed,#Port<0.222190389>}] > > > > links: [<0.873.0>] > > > > dictionary: [] > > > > trap_exit: false > > > > status: running > > > > heap_size: 987 > > > > stack_size: 24 > > > > reductions: 541 > > > > neighbours: > > > > > > If you read through the Mochi code, you will see this: > > > > > > > https://github.com/mochi/mochiweb/blob/master/src/mochiweb_http.erl#L125 > > > > > > ok = mochiweb_socket:setopts(Socket, [{packet, raw}]), > > > > > > So assuming that is true, the problem is a posix() message einval > (Invalid Argument), probably because the > > > socket is long dead and gone. It may be that R15 is more accurate at > reporting here than R14, so the older version just accepts the result > blindly and then have the no change, whereas R15 actually reports the > correct error. > > > > > > A thing supporting my hypothesis is that there is a tcp_closed message > in your mailbox :) > > > > > > > > > Jesper Louis Andersen > > > Erlang Solutions Ltd., Copenhagen > > > > > > > > > > > > > -- > > ------------------------------------------------------------------------ > > Anthony Molinaro > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- > ------------------------------------------------------------------------ > Anthony Molinaro > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Thu Dec 13 01:24:55 2012 From: anthonym@REDACTED (Anthony Molinaro) Date: Wed, 12 Dec 2012 16:24:55 -0800 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? In-Reply-To: <20121212234803.GA75710@alumni.caltech.edu> References: <20121206193809.GC27411@alumni.caltech.edu> <15C0D662-A77C-4858-AD93-3306E72317F6@erlang-solutions.com> <20121212182632.GB72594@alumni.caltech.edu> <20121212234803.GA75710@alumni.caltech.edu> Message-ID: <20121213002455.GA75861@alumni.caltech.edu> Okay, I found it. Looks like this is the cause OTP-9389 Honor option packet_size for http packet parsing by both TCP socket and erlang:decode_packet. This gives the ability to accept HTTP headers larger than the default setting, but also avoid DoS attacks by accepting lines only up to whatever length you wish to allow. For consistency, packet type line also honor option packet_size. (Thanks to Steve Vinoski) With R14B04 when you have a header which is too long, your process gets a message like {http,#Port<0.120387186>,{http_error,"X2qutbTBbXg7VHgDDhGrEvDbzuxiyDlI7VFloMyAJKVqY2fTEkMc70UchLPRG8Cjowzmib4KszbCRwA5IBbAd2MbRi5X_tK2nfRtheavXdhQv8XbinzmhCM1E9YCeuFAg_9TfqUS0sWUd52mgjkWGqNe4Z9S0IxFYnFtf5..."}} in R15B02 you get a message like {tcp_error,#Port<0.104208233>,emsgsize} The prior case is handled correctly by mochiweb, the latter is not. I'm not sure if the commits for OTP-9389 actually caused this change or if it was some other change. Also, not sure if it was meant to be a backward compatible change or not (the comment on the commit https://github.com/erlang/otp/commit/5984409d1264871cbe61bfec875de53e51713efb seems to suggest it was supposed to be backward compatible, but maybe this was a side effect? It seems like emsgsize is more correct. -Anthony On Wed, Dec 12, 2012 at 03:48:03PM -0800, Anthony Molinaro wrote: > Okay, here's some more information, seems like the real error is > > {tcp_error,#Port<0.98735301>,emsgsize} > > earlier. So maybe sometime between R14B04 and R15B02 the handling of > emsgsize changed? > > I think what happens with mochiweb is it has a receive which matches several > message types, then has a catch all. The catch all gets the emsgsize error > and the tcp_closed error is in the mailbox, then the code handling the catch > all attempts to setopts on the socket and gets the einval error because > the socket is closed. > > So I'm really curious what might have changed in the tcp driver that might > have changed this behavior. I'm trying to figure out how to reproduce so > I can patch mochiweb to work around this (most likely by just adding a > clause to the receive). > > -Anthony > > On Wed, Dec 12, 2012 at 10:26:32AM -0800, Anthony Molinaro wrote: > > Well, yes, I did see the tcp_closed message, my real question is I guess > > related to your statement > > > > "It may be that R15 is more accurate at reporting here than R14, so the older version just accepts the result blindly and then have the no change, whereas R15 actually reports the correct error." > > > > Can anyone from OTP clarify if any work was done with regards to error > > messages? I did not find anything in the release notes which is why > > I'm wondering. > > > > I suppose I could attempt to git bisect this or something, but I'm not > > sure how difficult that might be. > > > > Thanks, > > > > -Anthony > > > > On Wed, Dec 12, 2012 at 05:18:57PM +0100, Jesper Louis Andersen wrote: > > > > > > On Dec 6, 2012, at 8:38 PM, Anthony Molinaro wrote: > > > > > > > 2012-12-01 00:01:33 =CRASH REPORT==== > > > > crasher: > > > > initial call: mochiweb_acceptor:init/3 > > > > pid: <0.2942.3599> > > > > registered_name: [] > > > > exception error: {{badmatch,{error,einval}},[{mochiweb_http,new_request,3,[]},{mochiweb_http,handle_invalid_request,3,[]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} > > > > ancestors: [webmachine_mochiweb,web_serve_sup,<0.619.0>] > > > > messages: [{tcp_closed,#Port<0.222190389>}] > > > > links: [<0.873.0>] > > > > dictionary: [] > > > > trap_exit: false > > > > status: running > > > > heap_size: 987 > > > > stack_size: 24 > > > > reductions: 541 > > > > neighbours: > > > > > > If you read through the Mochi code, you will see this: > > > > > > https://github.com/mochi/mochiweb/blob/master/src/mochiweb_http.erl#L125 > > > > > > ok = mochiweb_socket:setopts(Socket, [{packet, raw}]), > > > > > > So assuming that is true, the problem is a posix() message einval (Invalid Argument), probably because the > > > socket is long dead and gone. It may be that R15 is more accurate at reporting here than R14, so the older version just accepts the result blindly and then have the no change, whereas R15 actually reports the correct error. > > > > > > A thing supporting my hypothesis is that there is a tcp_closed message in your mailbox :) > > > > > > > > > Jesper Louis Andersen > > > Erlang Solutions Ltd., Copenhagen > > > > > > > > > > > > > -- > > ------------------------------------------------------------------------ > > Anthony Molinaro > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- > ------------------------------------------------------------------------ > Anthony Molinaro -- ------------------------------------------------------------------------ Anthony Molinaro From ok@REDACTED Thu Dec 13 01:33:59 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 13 Dec 2012 13:33:59 +1300 Subject: [erlang-questions] noob namespace question In-Reply-To: References: Message-ID: On 13/12/2012, at 6:58 AM, Rustom Mody wrote: > In Erlang variables start with a capital letter, atoms start with a lowercase letter. > Functions also start with lowercase. > > How does Erlang distinguish atoms and functions? Especially in the context of higher order functions.? atom in 'fun /' => function reference atom followed by left parenthesis => function reference atom anywhere else => just an atom Similar rule for variables. f(G(h,I)) f( static call to f/1 G( dynamic call to whatever G is bound to (which must be arity 2) h plain data, used not called I value of I, used not called. If you want to pass the name of a function as a parameter to a higher-order function, you have to use 'fun /', not _entirely_ unlike Lisp's #'. From ok@REDACTED Thu Dec 13 01:37:59 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 13 Dec 2012 13:37:59 +1300 Subject: [erlang-questions] noob namespace question In-Reply-To: References: <50C8C639.6060006@ninenines.eu> Message-ID: On 13/12/2012, at 7:14 AM, Rustom Mody wrote: > I dont understand. Let me try and be more explicit about my concern. > > Let h be a higher-order function of one argument (h/1) > > Now in the call h(foo) > How does Erlang know whether foo is a function or an atom? For one thing, function names in Erlang are *NOT* atoms. They are *PAIRS* /. So if you just see an atom, it cannot possibly be a function (or the name of a function) because there simply is not enough information there. Actually, I told a lie. Function names are module:name/arity triples. If you are referring to a function in the same module, you can omit the module: prefix. If you wanted to call h passing it a reference to a local function named foo with 3 arguments, you would have to write h(fun foo/3). In Common Lisp, (h 'foo) the atom FOO (h foo) value of the variable FOO (h #'foo) the function FOO From vinoski@REDACTED Thu Dec 13 02:35:11 2012 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 12 Dec 2012 20:35:11 -0500 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? In-Reply-To: <20121213002455.GA75861@alumni.caltech.edu> References: <20121206193809.GC27411@alumni.caltech.edu> <15C0D662-A77C-4858-AD93-3306E72317F6@erlang-solutions.com> <20121212182632.GB72594@alumni.caltech.edu> <20121212234803.GA75710@alumni.caltech.edu> <20121213002455.GA75861@alumni.caltech.edu> Message-ID: On Wed, Dec 12, 2012 at 7:24 PM, Anthony Molinaro < anthonym@REDACTED> wrote: > Okay, I found it. Looks like this is the cause > > OTP-9389 Honor option packet_size for http packet parsing by both TCP > socket and erlang:decode_packet. This gives the ability to > accept HTTP headers larger than the default setting, but also > avoid DoS attacks by accepting lines only up to > whatever length you wish to allow. For consistency, > packet type line also honor option packet_size. (Thanks > to Steve Vinoski) > > With R14B04 when you have a header which is too long, your process > gets a message like > > > {http,#Port<0.120387186>,{http_error,"X2qutbTBbXg7VHgDDhGrEvDbzuxiyDlI7VFloMyAJKVqY2fTEkMc70UchLPRG8Cjowzmib4KszbCRwA5IBbAd2MbRi5X_tK2nfRtheavXdhQv8XbinzmhCM1E9YCeuFAg_9TfqUS0sWUd52mgjkWGqNe4Z9S0IxFYnFtf5..."}} > > in R15B02 you get a message like > > {tcp_error,#Port<0.104208233>,emsgsize} > > The prior case is handled correctly by mochiweb, the latter is not. > I just ran a simple test case against the commit prior to 5984409d and then against 5984409d, in both cases sending an HTTP header of various lengths, and in both cases always received http_error, and never tcp_error. The server simply did a receive with the socket in http_bin mode. How long is the header in your case? Do you have a test case you can send me? I'm not sure if the commits for OTP-9389 actually caused this change or if > it was some other change. Also, not sure if it was meant to be a backward > compatible change or not (the comment on the commit > > https://github.com/erlang/otp/commit/5984409d1264871cbe61bfec875de53e51713efb > seems to suggest it was supposed to be backward compatible, but maybe this > was a side effect? It seems like emsgsize is more correct. > I think the intent was to be backward compatible; I seem to remember that putting the patch together was very difficult and time-consuming due to trying to achieve that, with numerous iterations and reviews. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Thu Dec 13 03:15:00 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Wed, 12 Dec 2012 20:15:00 -0600 Subject: [erlang-questions] SSL record overflow In-Reply-To: References: Message-ID: Well. Confession time. This is what happens when you hit an service on port 80 that you should be hitting on 443... Tristan On Wed, Dec 12, 2012 at 6:05 PM, Tristan Sloughter < tristan.sloughter@REDACTED> wrote: > This issue appeared using couchbeam to connect to a Cloudant database. But > it appears to be related to the ssl app, not couchbeam or ibrowse. > > When trying to make a request to the server in R15B03 I get: > > ** exception error: no match of right hand side value > {error,{conn_failed,{error,"record overflow"}}} > > and in R14 its: > > SSL: hello: ./ssl_record.erl:364:Fatal error: record overflow > ** exception error: no match of right hand side value > {error,{conn_failed,{error,"record overflow"}}} > > Has anyone experienced a similar issue? Suggestions on what I should do to > debug this issue? > > Thanks, > Tristan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From naikvin@REDACTED Thu Dec 13 06:17:12 2012 From: naikvin@REDACTED (Vineet Naik) Date: Thu, 13 Dec 2012 10:47:12 +0530 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> Message-ID: Hi Roman, Thanks for replying. I have /log/sasl but it's empty. And there is no other file/dir in log dir. Do I need to include anything in code/config to make it log errors to a file? Regards, Vineet On Wed, Dec 12, 2012 at 9:27 PM, Roman Gafiyatullin wrote: > Vineet, most probably something went wrong during startup. > Can anything similar to sasl-error.log be found in > the /log/ directory? > > -- > RG > > On Wednesday, December 12, 2012 at 6:31 pm, Vineet Naik wrote: > > Hello, > > I am trying to package my app using rebar. `rebar generate` command runs > without any > problem (no errors at least). But when I try to start the console by > running the script generated > by rebar in bin directory, it fails with "Kernel pid terminated" > > A huge crash_dump is also created but I am not able to gather any info > from it. > > I checked the erlang docs[1], I found this paragraph about the error > > "Kernel pid terminated (Who) (Exit-reason)" - The kernel supervisor has > detected a failure, usually that the application_controller has shut down > (Who = application_controller, Why = shutdown). The application controller > may have shut down for a number of reasons, the most usual being that the > node name of the distributed Erlang node is already in use. A complete > supervisor tree "crash" (i.e., the top supervisors have exited) will give > about the same result. This message comes from the Erlang code and not from > the virtual machine itself. It is always due to some kind of failure in an > application, either within OTP or a "user-written" one. Looking at the > error log for your application is probably the first step to take. > > > If I run `console_clean` command, then the console start in the correct > node. Does > this rule out the "Node is already in use" case above? > > I also tried running the code from an erlang shell with the ebin dir paths > of all required > libs added manually to sys path and it works perfectly. > > How can I go about debugging this? > > [1] http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 > > Thanks, > Vineet > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -- Vineet Naik -------------- next part -------------- An HTML attachment was scrubbed... URL: From gopienko@REDACTED Thu Dec 13 07:07:11 2012 From: gopienko@REDACTED (Andrew Gopienko) Date: Thu, 13 Dec 2012 12:07:11 +0600 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> Message-ID: Make sure you have sasl dependency in your app file. {applications, [ kernel, stdlib, sasl ]}, 2012/12/13 Vineet Naik > Hi Roman, > > Thanks for replying. I have /log/sasl but it's empty. And > there is no > other file/dir in log dir. Do I need to include anything in code/config to > make > it log errors to a file? > > Regards, > Vineet > > On Wed, Dec 12, 2012 at 9:27 PM, Roman Gafiyatullin > wrote: > >> Vineet, most probably something went wrong during startup. >> Can anything similar to sasl-error.log be found in >> the /log/ directory? >> >> -- >> RG >> >> On Wednesday, December 12, 2012 at 6:31 pm, Vineet Naik wrote: >> >> Hello, >> >> I am trying to package my app using rebar. `rebar generate` command runs >> without any >> problem (no errors at least). But when I try to start the console by >> running the script generated >> by rebar in bin directory, it fails with "Kernel pid terminated" >> >> A huge crash_dump is also created but I am not able to gather any info >> from it. >> >> I checked the erlang docs[1], I found this paragraph about the error >> >> "Kernel pid terminated (Who) (Exit-reason)" - The kernel supervisor has >> detected a failure, usually that the application_controller has shut down >> (Who = application_controller, Why = shutdown). The application controller >> may have shut down for a number of reasons, the most usual being that the >> node name of the distributed Erlang node is already in use. A complete >> supervisor tree "crash" (i.e., the top supervisors have exited) will give >> about the same result. This message comes from the Erlang code and not from >> the virtual machine itself. It is always due to some kind of failure in an >> application, either within OTP or a "user-written" one. Looking at the >> error log for your application is probably the first step to take. >> >> >> If I run `console_clean` command, then the console start in the correct >> node. Does >> this rule out the "Node is already in use" case above? >> >> I also tried running the code from an erlang shell with the ebin dir >> paths of all required >> libs added manually to sys path and it works perfectly. >> >> How can I go about debugging this? >> >> [1] http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 >> >> Thanks, >> Vineet >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > > > -- > Vineet Naik > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gianfranco.alongi@REDACTED Thu Dec 13 07:25:27 2012 From: gianfranco.alongi@REDACTED (Gianfranco Alongi) Date: Thu, 13 Dec 2012 07:25:27 +0100 Subject: [erlang-questions] noob namespace question In-Reply-To: References: <50C8C639.6060006@ninenines.eu> Message-ID: By implication - what I will write has been said. But I will just spend a second reinforcing this and dispelling any misconceptions. They can - if you force it into an atom. -module(capsFuncs). -export(['Run'/1]). 'Run'(this) -> that. 1> c(capsFuncs). {ok,capsFuncs} 2> capsFuncs:'Run'(this). that On Thu, Dec 13, 2012 at 1:37 AM, Richard O'Keefe wrote: > > On 13/12/2012, at 7:14 AM, Rustom Mody wrote: > > I dont understand. Let me try and be more explicit about my concern. > > > > Let h be a higher-order function of one argument (h/1) > > > > Now in the call h(foo) > > How does Erlang know whether foo is a function or an atom? > > For one thing, function names in Erlang are *NOT* atoms. > They are *PAIRS* /. So if you just see an > atom, it cannot possibly be a function (or the name of a > function) because there simply is not enough information > there. > > Actually, I told a lie. Function names are > module:name/arity triples. If you are referring to a > function in the same module, you can omit the module: > prefix. > > If you wanted to call h passing it a reference to a local > function named foo with 3 arguments, you would have to write > h(fun foo/3). > > In Common Lisp, > (h 'foo) the atom FOO > (h foo) value of the variable FOO > (h #'foo) the function FOO > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From naikvin@REDACTED Thu Dec 13 08:28:58 2012 From: naikvin@REDACTED (Vineet Naik) Date: Thu, 13 Dec 2012 12:58:58 +0530 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> Message-ID: On Thu, Dec 13, 2012 at 11:37 AM, Andrew Gopienko wrote: > Make sure you have sasl dependency in your app file. > > {applications, [ > kernel, > stdlib, > sasl > ]}, Thanks Andrew, that was indeed the problem why no sasl error log was generated. After adding sasl in applications, the error logged in it is, exception exit: {undef,[{inets,start,[]}, {exmpp_component,init,1}, {gen_fsm,init_it,6}, {proc_lib,init_p_do_apply,3}]} in function gen_fsm:init_it/6 I had added `{app, inets, [{incl_cond, exclude}]},` in reltool.config myself earlier. I tried removing that option out now and I am getting the following error while starting up {"init terminating in do_boot",{'cannot load',ftp,get_file}} I was getting a series of such errors before and I could fix them by excluding non required libraries one by one. 'inets' was excluded in order to fix the error for 'ftp' module. But it seems exmpp code needs inets:start/0. I tried replacing it with - {app, ftp, [{incl_cond, exclude}]}, and later {app, inets, [{incl_cond, derived}]}, But both don't work. What is the correct way to configure this? BTW, isn't ftp in the stdlib? I can load ftp module from inside any erlang shell Thanks, Vineet > > > > > 2012/12/13 Vineet Naik > >> Hi Roman, >> >> Thanks for replying. I have /log/sasl but it's empty. And >> there is no >> other file/dir in log dir. Do I need to include anything in code/config >> to make >> it log errors to a file? >> >> Regards, >> Vineet >> >> On Wed, Dec 12, 2012 at 9:27 PM, Roman Gafiyatullin < >> r.gafiyatullin@REDACTED> wrote: >> >>> Vineet, most probably something went wrong during startup. >>> Can anything similar to sasl-error.log be found in >>> the /log/ directory? >>> >>> -- >>> RG >>> >>> On Wednesday, December 12, 2012 at 6:31 pm, Vineet Naik wrote: >>> >>> Hello, >>> >>> I am trying to package my app using rebar. `rebar generate` command runs >>> without any >>> problem (no errors at least). But when I try to start the console by >>> running the script generated >>> by rebar in bin directory, it fails with "Kernel pid terminated" >>> >>> A huge crash_dump is also created but I am not able to gather any info >>> from it. >>> >>> I checked the erlang docs[1], I found this paragraph about the error >>> >>> "Kernel pid terminated (Who) (Exit-reason)" - The kernel supervisor has >>> detected a failure, usually that the application_controller has shut down >>> (Who = application_controller, Why = shutdown). The application controller >>> may have shut down for a number of reasons, the most usual being that the >>> node name of the distributed Erlang node is already in use. A complete >>> supervisor tree "crash" (i.e., the top supervisors have exited) will give >>> about the same result. This message comes from the Erlang code and not from >>> the virtual machine itself. It is always due to some kind of failure in an >>> application, either within OTP or a "user-written" one. Looking at the >>> error log for your application is probably the first step to take. >>> >>> >>> If I run `console_clean` command, then the console start in the correct >>> node. Does >>> this rule out the "Node is already in use" case above? >>> >>> I also tried running the code from an erlang shell with the ebin dir >>> paths of all required >>> libs added manually to sys path and it works perfectly. >>> >>> How can I go about debugging this? >>> >>> [1] http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 >>> >>> Thanks, >>> Vineet >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >> >> >> -- >> Vineet Naik >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- Vineet Naik -------------- next part -------------- An HTML attachment was scrubbed... URL: From dm.klionsky@REDACTED Thu Dec 13 10:16:12 2012 From: dm.klionsky@REDACTED (Dmitry Klionsky) Date: Thu, 13 Dec 2012 12:16:12 +0300 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> Message-ID: <50C99CDC.8050401@gmail.com> Hi, >> BTW, isn't ftp in the stdlib? No, it's inside the inets application >> {"init terminating in do_boot",{'cannot load',ftp,get_file}} Maybe this will clarify things for you: http://stackoverflow.com/questions/10417469/could-not-start-release-with-rebar-generate-epgsql >> {app, ftp, [{incl_cond, exclude}]}, This won't work. The `ftp' isn't a separate application >> {app, inets, [{incl_cond, derived}]}, Add {app, inets, [{incl_cond, include}]}. But better include `inets' into a YOUR_APP.app.src file like this http://code.google.com/p/mochiweb/source/browse/trunk/src/mochiweb.app.src Best regards, Dmitry On 12/13/2012 10:28 AM, Vineet Naik wrote: > On Thu, Dec 13, 2012 at 11:37 AM, Andrew Gopienko > wrote: > > Make sure you have sasl dependency in your app file. > > {applications, [ > kernel, > stdlib, > sasl > ]}, > > > Thanks Andrew, that was indeed the problem why no sasl error log was > generated. > After adding sasl in applications, the error logged in it is, > > exception exit: {undef,[{inets,start,[]}, > {exmpp_component,init,1}, > {gen_fsm,init_it,6}, > {proc_lib,init_p_do_apply,3}]} > in function gen_fsm:init_it/6 > > I had added `{app, inets, [{incl_cond, exclude}]},` in reltool.config > myself earlier. > I tried removing that option out now and I am getting the following > error while > starting up > > {"init terminating in do_boot",{'cannot load',ftp,get_file}} > > I was getting a series of such errors before and I could fix them by > excluding > non required libraries one by one. 'inets' was excluded in order to > fix the error for > 'ftp' module. But it seems exmpp code needs inets:start/0. > > I tried replacing it with - > > {app, ftp, [{incl_cond, exclude}]}, > > and later > > {app, inets, [{incl_cond, derived}]}, > > But both don't work. What is the correct way to configure this? > BTW, isn't ftp in the stdlib? I can load ftp module from inside any > erlang shell > > Thanks, > Vineet > > > > > > 2012/12/13 Vineet Naik > > > Hi Roman, > > Thanks for replying. I have /log/sasl but it's > empty. And there is no > other file/dir in log dir. Do I need to include anything in > code/config to make > it log errors to a file? > > Regards, > Vineet > > On Wed, Dec 12, 2012 at 9:27 PM, Roman Gafiyatullin > > wrote: > > Vineet, most probably something went wrong during startup. > Can anything similar to sasl-error.log be found in > the /log/ directory? > > -- > RG > > On Wednesday, December 12, 2012 at 6:31 pm, Vineet Naik wrote: > >> Hello, >> >> I am trying to package my app using rebar. `rebar >> generate` command runs without any >> problem (no errors at least). But when I try to start the >> console by running the script generated >> by rebar in bin directory, it fails with "Kernel pid >> terminated" >> >> A huge crash_dump is also created but I am not able to >> gather any info from it. >> >> I checked the erlang docs[1], I found this paragraph >> about the error >> >>> "Kernel pid terminated (Who) (Exit-reason)" - The kernel >>> supervisor has detected a failure, usually that the >>> application_controller has shut down (Who = >>> application_controller, Why = shutdown). The application >>> controller may have shut down for a number of reasons, >>> the most usual being that the node name of the >>> distributed Erlang node is already in use. A complete >>> supervisor tree "crash" (i.e., the top supervisors have >>> exited) will give about the same result. This message >>> comes from the Erlang code and not from the virtual >>> machine itself. It is always due to some kind of failure >>> in an application, either within OTP or a "user-written" >>> one. Looking at the error log for your application is >>> probably the first step to take. >> >> If I run `console_clean` command, then the console start >> in the correct node. Does >> this rule out the "Node is already in use" case above? >> >> I also tried running the code from an erlang shell with >> the ebin dir paths of all required >> libs added manually to sys path and it works perfectly. >> >> How can I go about debugging this? >> >> [1] >> http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 >> >> Thanks, >> Vineet >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Vineet Naik > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > Vineet Naik > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Best regards, Dmitry Klionsky -------------- next part -------------- An HTML attachment was scrubbed... URL: From cao.xu@REDACTED Thu Dec 13 10:21:35 2012 From: cao.xu@REDACTED (cao xu) Date: Thu, 13 Dec 2012 17:21:35 +0800 Subject: [erlang-questions] Getting more info from crash dump file Message-ID: <7F74F70D-A77B-4842-9230-D21AB1458BDB@rytong.com> Hi, I got an erlang crash dump file on a 2G memory linux with following info: eheap_alloc: Cannot allocate 78362800 bytes of memory (of type "heap"). =memory total: 1510245164 processes: 1360838774 processes_used: 1360720438 The total memory consumed by the VM was about 1.5G and most of them were used by process. According to the process info in the dump file, the biggest Stack+Heap was 15672560 and the owner's state was garbing. I am not sure whether the process was the killer of the whole VM. Name Spawned as proc_lib:init_p/5 State Garbing (limited info) Last scheduled in for xmerl_scan:'-initial_state0/2-fun-1-'/2 Started Tue Nov 6 17:00:39 2012 Spawned by <0.48.0> Reductions 20600887 Stack+heap 15672560 OldHeap 8024355 Heap unused 202 OldHeap unused 8024355 Number of heap fragments 1 Heap fragment data 119 Program counter 0x02ea24dc (xmerl_scan:'-initial_state0/2-fun-1-'/2 + 4) Continuation pointer 0x02e8a254 (xmerl_scan:scan_char_data/5 + 2408) Arity Link list [#Port<0.16239279>, <0.48.0>] Msg queue length 0 Is there a way to find out which process or function call was the reason of the allocating action, and how can I get more info from the dump file to help locating the error? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gustav.simonsson@REDACTED Thu Dec 13 10:34:27 2012 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Thu, 13 Dec 2012 10:34:27 +0100 Subject: [erlang-questions] Getting more info from crash dump file In-Reply-To: <7F74F70D-A77B-4842-9230-D21AB1458BDB@rytong.com> References: <7F74F70D-A77B-4842-9230-D21AB1458BDB@rytong.com> Message-ID: Hi, In the crashdump viewer, you can click on the process that spawned this process (the "Spawned by" field) several times to go up the process hierarchy of your application to see where in the process tree it was spawned. Also, you can see it was executing a call to xmerl, and is linked to a Erlang port, which should help you narrow down which process it was in your application. Cheers, Gustav Simonsson On Thu, Dec 13, 2012 at 10:21 AM, cao xu wrote: > Hi, > > I got an erlang crash dump file on a 2G memory linux with following info: > > eheap_alloc: Cannot allocate 78362800 bytes of memory (of type "heap"). > > =memory > total: 1510245164 > processes: 1360838774 > processes_used: 1360720438 > > The total memory consumed by the VM was about 1.5G and most of them were > used by process. According to the process info in the dump file, the > biggest Stack+Heap was 15672560 and the owner's state was garbing. I am > not sure whether the process was the killer of the whole VM. > > *Name* *Spawned as*proc_lib:init_p/5*State*Garbing (limited info)*Last > scheduled in for*xmerl_scan:'-initial_state0/2-fun-1-'/2*Started*Tue Nov > 6 17:00:39 2012*Spawned by*<0.48.0> > *Reductions*20600887*Stack+heap*15672560*OldHeap*8024355*Heap unused*202*OldHeap > unused*8024355*Number of heap fragments*1*Heap fragment data*119*Program > counter*0x02ea24dc (xmerl_scan:'-initial_state0/2-fun-1-'/2 + 4)*Continuation > pointer*0x02e8a254 (xmerl_scan:scan_char_data/5 + 2408)*Arity* *Link list* > [#Port<0.16239279> > , <0.48.0> > ]*Msg queue length*0 > > > Is there a way to find out which process or function call was the reason > of the allocating action, and how can I get more info from the dump file to > help locating the error? > > Thanks. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From naikvin@REDACTED Thu Dec 13 10:49:59 2012 From: naikvin@REDACTED (Vineet Naik) Date: Thu, 13 Dec 2012 15:19:59 +0530 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: <50C99CDC.8050401@gmail.com> References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> <50C99CDC.8050401@gmail.com> Message-ID: Thanks Dmitry, After adding inets to the app config file it failed with - {"init terminating in do_boot",{'cannot load',mnesia_backup,get_file}} I then excluded mnesia in reltool.config as I am not using it (Is this the correct thing to do?) Now it shows - {"init terminating in do_boot",{'cannot load',erts_alloc_config,get_file}} I tried adding erts to app config but get the same error. I don't think erts can be excluded, right? I am not able to understand why is it required to either include in app config or exclude it in reltool config. What's going on here and who is behind the magic, reltools or rebar :-) Really appreciate all the help, it would have very difficult for me to reach this far otherwise. Thanks a lot. On Thu, Dec 13, 2012 at 2:46 PM, Dmitry Klionsky wrote: > Hi, > > > >> BTW, isn't ftp in the stdlib? > No, it's inside the inets application > > > >> {"init terminating in do_boot",{'cannot load',ftp,get_file}} > Maybe this will clarify things for you: > http://stackoverflow.com/questions/10417469/could-not-start-release-with-rebar-generate-epgsql > > > >> {app, ftp, [{incl_cond, exclude}]}, > This won't work. The `ftp' isn't a separate application > > > >> {app, inets, [{incl_cond, derived}]}, > Add {app, inets, [{incl_cond, include}]}. > But better include `inets' into a YOUR_APP.app.src file like this > http://code.google.com/p/mochiweb/source/browse/trunk/src/mochiweb.app.src > > > Best regards, > Dmitry > > > On 12/13/2012 10:28 AM, Vineet Naik wrote: > > On Thu, Dec 13, 2012 at 11:37 AM, Andrew Gopienko wrote: > >> Make sure you have sasl dependency in your app file. >> >> {applications, [ >> kernel, >> stdlib, >> sasl >> ]}, > > > Thanks Andrew, that was indeed the problem why no sasl error log was > generated. > After adding sasl in applications, the error logged in it is, > > exception exit: {undef,[{inets,start,[]}, > {exmpp_component,init,1}, > {gen_fsm,init_it,6}, > {proc_lib,init_p_do_apply,3}]} > in function gen_fsm:init_it/6 > > I had added `{app, inets, [{incl_cond, exclude}]},` in reltool.config > myself earlier. > I tried removing that option out now and I am getting the following error > while > starting up > > {"init terminating in do_boot",{'cannot load',ftp,get_file}} > > I was getting a series of such errors before and I could fix them by > excluding > non required libraries one by one. 'inets' was excluded in order to fix > the error for > 'ftp' module. But it seems exmpp code needs inets:start/0. > > I tried replacing it with - > > {app, ftp, [{incl_cond, exclude}]}, > > and later > > {app, inets, [{incl_cond, derived}]}, > > But both don't work. What is the correct way to configure this? > BTW, isn't ftp in the stdlib? I can load ftp module from inside any erlang > shell > > Thanks, > Vineet > > > >> >> >> >> >> 2012/12/13 Vineet Naik >> >>> Hi Roman, >>> >>> Thanks for replying. I have /log/sasl but it's empty. >>> And there is no >>> other file/dir in log dir. Do I need to include anything in code/config >>> to make >>> it log errors to a file? >>> >>> Regards, >>> Vineet >>> >>> On Wed, Dec 12, 2012 at 9:27 PM, Roman Gafiyatullin < >>> r.gafiyatullin@REDACTED> wrote: >>> >>>> Vineet, most probably something went wrong during startup. >>>> Can anything similar to sasl-error.log be found in >>>> the /log/ directory? >>>> >>>> -- >>>> RG >>>> >>>> On Wednesday, December 12, 2012 at 6:31 pm, Vineet Naik wrote: >>>> >>>> Hello, >>>> >>>> I am trying to package my app using rebar. `rebar generate` command >>>> runs without any >>>> problem (no errors at least). But when I try to start the console by >>>> running the script generated >>>> by rebar in bin directory, it fails with "Kernel pid terminated" >>>> >>>> A huge crash_dump is also created but I am not able to gather any >>>> info from it. >>>> >>>> I checked the erlang docs[1], I found this paragraph about the error >>>> >>>> "Kernel pid terminated (Who) (Exit-reason)" - The kernel supervisor >>>> has detected a failure, usually that the application_controller has shut >>>> down (Who = application_controller, Why = shutdown). The application >>>> controller may have shut down for a number of reasons, the most usual being >>>> that the node name of the distributed Erlang node is already in use. A >>>> complete supervisor tree "crash" (i.e., the top supervisors have exited) >>>> will give about the same result. This message comes from the Erlang code >>>> and not from the virtual machine itself. It is always due to some kind of >>>> failure in an application, either within OTP or a "user-written" one. >>>> Looking at the error log for your application is probably the first step to >>>> take. >>>> >>>> >>>> If I run `console_clean` command, then the console start in the >>>> correct node. Does >>>> this rule out the "Node is already in use" case above? >>>> >>>> I also tried running the code from an erlang shell with the ebin dir >>>> paths of all required >>>> libs added manually to sys path and it works perfectly. >>>> >>>> How can I go about debugging this? >>>> >>>> [1] http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 >>>> >>>> Thanks, >>>> Vineet >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> >>> >>> >>> -- >>> Vineet Naik >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > > -- > Vineet Naik > > > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Best regards, > Dmitry Klionsky > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Vineet Naik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Dec 13 10:50:31 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 13 Dec 2012 10:50:31 +0100 Subject: [erlang-questions] app for watching/monitoring other erlang nodes In-Reply-To: References: <50AB7A4E.70507@ninenines.eu> Message-ID: <02DB336D-4680-4EE2-A32F-C8D30A6C0EFC@erlang-solutions.com> On Nov 20, 2012, at 1:46 PM, Vlad Dumitrescu wrote: > Yes, you're right, snmp matches the description, but it feels a bit too heavy for me. I will read about it, maybe it's not true. I would like to have the monitored nodes without any specific application running, just answering to some RPCs once in a while. The SNMP agent parts are not that hard to use. Basically you define a MIB file which exposes the values your system supports. Then you define a mapping from the MIB file to Erlang functions, or to a mnesia table. The SNMP subsystem then responds on requests and calls the appropriate Erlang code, wraps things in ASN.1 and so on. What makes it "heavy" is that SNMP itself is rather heavy. And that there is quite some configuration needed in order to expose these values to the rest of the world. The advantage is, however, that once you have set it up properly, it is extremely easy to add more metrics and probes to your node. Erlang/OTP also provides a couple of things itself for monitoring. Including "what process uses the most memory and how much" etc. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From vladdu55@REDACTED Thu Dec 13 10:56:17 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 13 Dec 2012 10:56:17 +0100 Subject: [erlang-questions] app for watching/monitoring other erlang nodes In-Reply-To: <02DB336D-4680-4EE2-A32F-C8D30A6C0EFC@erlang-solutions.com> References: <50AB7A4E.70507@ninenines.eu> <02DB336D-4680-4EE2-A32F-C8D30A6C0EFC@erlang-solutions.com> Message-ID: Thanks Jesper, It was the configuration part that I feel is something that I don't want to learn unless it's absolutely necessary. For a one-time simple job, there are better alternatives, I think. In my case, it was pointed out to me that erlang:system_monitor/2 can make the runtime send messages when a process gets too large, which is 98% of what I needed to do. regards, Vlad On Thu, Dec 13, 2012 at 10:50 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Nov 20, 2012, at 1:46 PM, Vlad Dumitrescu wrote: > > > Yes, you're right, snmp matches the description, but it feels a bit too > heavy for me. I will read about it, maybe it's not true. I would like to > have the monitored nodes without any specific application running, just > answering to some RPCs once in a while. > > The SNMP agent parts are not that hard to use. Basically you define a MIB > file which exposes the values your system supports. Then you define a > mapping from the MIB file to Erlang functions, or to a mnesia table. The > SNMP subsystem then responds on requests and calls the appropriate Erlang > code, wraps things in ASN.1 and so on. > > What makes it "heavy" is that SNMP itself is rather heavy. And that there > is quite some configuration needed in order to expose these values to the > rest of the world. The advantage is, however, that once you have set it up > properly, it is extremely easy to add more metrics and probes to your node. > Erlang/OTP also provides a couple of things itself for monitoring. > Including "what process uses the most memory and how much" etc. > > Jesper Louis Andersen > Erlang Solutions Ltd., Copenhagen > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Dec 13 11:17:14 2012 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 13 Dec 2012 11:17:14 +0100 Subject: [erlang-questions] ANN: ezwebframe - an easy web framework Message-ID: ezwebframe ========== https://github.com/joearms/ezwebframe Pronounced "Easy web frame." About ===== Ezwebframe attempts to make web programming just a little bit easier. >From Erlang point of view the browser *is* an Erlang process. Assume we have a web page populated with divs. For example:
...
...
Erlang thinks the browser is a process. To fill div a with HTML an Erlang process evaluates the command: Browser ! [{cmd, fill_div}, {id, a}, {txt, B}] Where B is a binary containing HTML. In the browser controls can be programmed to send messages to Erlang, for example, when we click on a button in the browser the Erlang process controlling the window will be sent a message which can be received with the statement: receive {Browser, {struct, [{clicked, ButtonName}]}} -> ... end All this is achieved using a thin JSON layer over websockets and with cowboy managing the websockets. Cheers /Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From dm.klionsky@REDACTED Thu Dec 13 11:18:12 2012 From: dm.klionsky@REDACTED (Dmitry Klionsky) Date: Thu, 13 Dec 2012 13:18:12 +0300 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: <50C9A890.6040104@gmail.com> References: <50C9A890.6040104@gmail.com> Message-ID: <50C9AB64.4090307@gmail.com> Actually, it doesn't seem to be a good idea to manually exclude any application until you know precisely what you're doing. Reltool will exclude any unused applications itself. Just stop excluding the applications in reltool.config and see if it works. On 12/13/2012 12:49 PM, Vineet Naik wrote: > Thanks Dmitry, > > After adding inets to the app config file it failed with - > > {"init terminating in do_boot",{'cannot load',mnesia_backup,get_file}} > > I then excluded mnesia in reltool.config as I am not using it (Is this > the correct thing to do?) > > Now it shows - > > {"init terminating in do_boot",{'cannot load',erts_alloc_config,get_file}} > > I tried adding erts to app config but get the same error. > > I don't think erts can be excluded, right? > > I am not able to understand why is it required to either include in > app config or > exclude it in reltool config. What's going on here and who is behind > the magic, > reltools or rebar :-) > > Really appreciate all the help, it would have very difficult for me to > reach this far otherwise. Thanks a lot. > > > On Thu, Dec 13, 2012 at 2:46 PM, Dmitry Klionsky > > wrote: > > Hi, > > > >> BTW, isn't ftp in the stdlib? > No, it's inside the inets application > > > >> {"init terminating in do_boot",{'cannot load',ftp,get_file}} > Maybe this will clarify things for you: > http://stackoverflow.com/questions/10417469/could-not-start-release-with-rebar-generate-epgsql > > > > >> {app, ftp, [{incl_cond, exclude}]}, > This won't work. The `ftp' isn't a separate application > > > >> {app, inets, [{incl_cond, derived}]}, > Add {app, inets, [{incl_cond, include}]}. > But better include `inets' into a YOUR_APP.app.src file like this > http://code.google.com/p/mochiweb/source/browse/trunk/src/mochiweb.app.src > > > Best regards, > Dmitry > > > On 12/13/2012 10:28 AM, Vineet Naik wrote: >> On Thu, Dec 13, 2012 at 11:37 AM, Andrew Gopienko >> > wrote: >> >> Make sure you have sasl dependency in your app file. >> >> {applications, [ >> kernel, >> stdlib, >> sasl >> ]}, >> >> >> Thanks Andrew, that was indeed the problem why no sasl error log >> was generated. >> After adding sasl in applications, the error logged in it is, >> >> exception exit: {undef,[{inets,start,[]}, >> {exmpp_component,init,1}, >> {gen_fsm,init_it,6}, >> {proc_lib,init_p_do_apply,3}]} >> in function gen_fsm:init_it/6 >> >> I had added `{app, inets, [{incl_cond, exclude}]},` in >> reltool.config myself earlier. >> I tried removing that option out now and I am getting the >> following error while >> starting up >> >> {"init terminating in do_boot",{'cannot load',ftp,get_file}} >> >> I was getting a series of such errors before and I could fix them >> by excluding >> non required libraries one by one. 'inets' was excluded in order >> to fix the error for >> 'ftp' module. But it seems exmpp code needs inets:start/0. >> >> I tried replacing it with - >> >> {app, ftp, [{incl_cond, exclude}]}, >> >> and later >> >> {app, inets, [{incl_cond, derived}]}, >> >> But both don't work. What is the correct way to configure this? >> BTW, isn't ftp in the stdlib? I can load ftp module from inside >> any erlang shell >> >> Thanks, >> Vineet >> >> >> >> >> >> 2012/12/13 Vineet Naik > > >> >> Hi Roman, >> >> Thanks for replying. I have /log/sasl but >> it's empty. And there is no >> other file/dir in log dir. Do I need to include anything >> in code/config to make >> it log errors to a file? >> >> Regards, >> Vineet >> >> On Wed, Dec 12, 2012 at 9:27 PM, Roman Gafiyatullin >> > wrote: >> >> Vineet, most probably something went wrong during >> startup. >> Can anything similar to sasl-error.log be found in >> the /log/ directory? >> >> -- >> RG >> >> On Wednesday, December 12, 2012 at 6:31 pm, Vineet >> Naik wrote: >> >>> Hello, >>> >>> I am trying to package my app using rebar. `rebar >>> generate` command runs without any >>> problem (no errors at least). But when I try to >>> start the console by running the script generated >>> by rebar in bin directory, it fails with "Kernel pid >>> terminated" >>> >>> A huge crash_dump is also created but I am not able >>> to gather any info from it. >>> >>> I checked the erlang docs[1], I found this paragraph >>> about the error >>> >>>> "Kernel pid terminated (Who) (Exit-reason)" - The >>>> kernel supervisor has detected a failure, usually >>>> that the application_controller has shut down (Who >>>> = application_controller, Why = shutdown). The >>>> application controller may have shut down for a >>>> number of reasons, the most usual being that the >>>> node name of the distributed Erlang node is already >>>> in use. A complete supervisor tree "crash" (i.e., >>>> the top supervisors have exited) will give about >>>> the same result. This message comes from the Erlang >>>> code and not from the virtual machine itself. It is >>>> always due to some kind of failure in an >>>> application, either within OTP or a "user-written" >>>> one. Looking at the error log for your application >>>> is probably the first step to take. >>> >>> If I run `console_clean` command, then the console >>> start in the correct node. Does >>> this rule out the "Node is already in use" case above? >>> >>> I also tried running the code from an erlang shell >>> with the ebin dir paths of all required >>> libs added manually to sys path and it works perfectly. >>> >>> How can I go about debugging this? >>> >>> [1] >>> http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 >>> >>> Thanks, >>> Vineet >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> -- >> Vineet Naik >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> >> -- >> Vineet Naik >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > -- > Best regards, > Dmitry Klionsky > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Vineet Naik > > -- Best regards, Dmitry Klionsky -------------- next part -------------- An HTML attachment was scrubbed... URL: From cao.xu@REDACTED Thu Dec 13 11:26:06 2012 From: cao.xu@REDACTED (cao xu) Date: Thu, 13 Dec 2012 18:26:06 +0800 Subject: [erlang-questions] Getting more info from crash dump file In-Reply-To: References: <7F74F70D-A77B-4842-9230-D21AB1458BDB@rytong.com> Message-ID: The app was a web server based on yaws, so most processes were spawned by yaws_server(<0.48.0>), and they were doing the same kind of work: 1, parse the http request 2, request remote server 3, parse the returned xml ... When the VM crashed, there were about 1,000 this kind of processes but only this one seemed abnormal. Do you mean this process was definitely the one who caused the allocation of 78362800 bytes' memory. If it's true, it will help me add error locater in the app. ? 2012-12-13???5:34? Gustav Simonsson ??? > Hi, > > In the crashdump viewer, you can click on the process that spawned this process (the "Spawned by" field) several times to go up the process hierarchy of your application to see where in the process tree it was spawned. > Also, you can see it was executing a call to xmerl, and is linked to a Erlang port, which should help you narrow down which process it was in your application. > > Cheers, > Gustav Simonsson > > > On Thu, Dec 13, 2012 at 10:21 AM, cao xu wrote: > Hi, > > I got an erlang crash dump file on a 2G memory linux with following info: > > eheap_alloc: Cannot allocate 78362800 bytes of memory (of type "heap"). > > =memory > total: 1510245164 > processes: 1360838774 > processes_used: 1360720438 > > The total memory consumed by the VM was about 1.5G and most of them were used by process. According to the process info in the dump file, the biggest Stack+Heap was 15672560 and the owner's state was garbing. I am not sure whether the process was the killer of the whole VM. > > Name Spawned as proc_lib:init_p/5 > State Garbing (limited info) Last scheduled in for xmerl_scan:'-initial_state0/2-fun-1-'/2 > Started Tue Nov 6 17:00:39 2012 Spawned by <0.48.0> > Reductions 20600887 > Stack+heap 15672560 OldHeap 8024355 > Heap unused 202 OldHeap unused 8024355 > Number of heap fragments 1 Heap fragment data 119 > Program counter 0x02ea24dc (xmerl_scan:'-initial_state0/2-fun-1-'/2 + 4) > Continuation pointer 0x02e8a254 (xmerl_scan:scan_char_data/5 + 2408) > Arity > Link list [#Port<0.16239279>, <0.48.0>] > Msg queue length 0 > > > Is there a way to find out which process or function call was the reason of the allocating action, and how can I get more info from the dump file to help locating the error? > > Thanks. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Thu Dec 13 11:35:23 2012 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 13 Dec 2012 11:35:23 +0100 Subject: [erlang-questions] ANN: ezwebframe - an easy web framework In-Reply-To: References: Message-ID: On Thu, Dec 13, 2012 at 11:17 AM, Joe Armstrong wrote: > ezwebframe > ========== > > https://github.com/joearms/ezwebframe > > Pronounced "Easy web frame." > > About > ===== > > Ezwebframe attempts to make web programming just a little bit easier. > > From Erlang point of view the browser *is* an Erlang process. > > Assume we have a web page populated with divs. For example: > >
> ... >
> >
> ... >
> > Erlang thinks the browser is a process. To fill div a with HTML an > Erlang process evaluates the command: > > Browser ! [{cmd, fill_div}, {id, a}, {txt, B}] > > Where B is a binary containing HTML. > > In the browser controls can be programmed to send messages to Erlang, > for example, when we click on a button in the browser the Erlang > process controlling the window will be sent a message which can be > received with the statement: > > receive > {Browser, {struct, [{clicked, ButtonName}]}} -> > ... > end > > All this is achieved using a thin JSON layer over websockets and > with cowboy managing the websockets. > > Cheers > > /Joe > > > > That's ?ber cool. I really like the idea . I can see how you can distribute easily the rendering with that. - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Thu Dec 13 11:41:32 2012 From: bourinov@REDACTED (Max Bourinov) Date: Thu, 13 Dec 2012 14:41:32 +0400 Subject: [erlang-questions] ANN: ezwebframe - an easy web framework In-Reply-To: References: Message-ID: Really elegant and small! Best regards, Max On Thu, Dec 13, 2012 at 2:35 PM, Benoit Chesneau wrote: > > > > On Thu, Dec 13, 2012 at 11:17 AM, Joe Armstrong wrote: > >> ezwebframe >> ========== >> >> https://github.com/joearms/ezwebframe >> >> Pronounced "Easy web frame." >> >> About >> ===== >> >> Ezwebframe attempts to make web programming just a little bit easier. >> >> From Erlang point of view the browser *is* an Erlang process. >> >> Assume we have a web page populated with divs. For example: >> >>
>> ... >>
>> >>
>> ... >>
>> >> Erlang thinks the browser is a process. To fill div a with HTML an >> Erlang process evaluates the command: >> >> Browser ! [{cmd, fill_div}, {id, a}, {txt, B}] >> >> Where B is a binary containing HTML. >> >> In the browser controls can be programmed to send messages to Erlang, >> for example, when we click on a button in the browser the Erlang >> process controlling the window will be sent a message which can be >> received with the statement: >> >> receive >> {Browser, {struct, [{clicked, ButtonName}]}} -> >> ... >> end >> >> All this is achieved using a thin JSON layer over websockets and >> with cowboy managing the websockets. >> >> Cheers >> >> /Joe >> >> >> >> > That's ?ber cool. I really like the idea . I can see how you can > distribute easily the rendering with that. > - benoit > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Dec 13 11:45:27 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 13 Dec 2012 11:45:27 +0100 Subject: [erlang-questions] ANN: ezwebframe - an easy web framework In-Reply-To: References: Message-ID: <50C9B1C7.3060109@ninenines.eu> On 12/13/2012 11:17 AM, Joe Armstrong wrote: > ezwebframe > ========== > > https://github.com/joearms/ezwebframe > > Pronounced "Easy web frame." > > About > ===== > > Ezwebframe attempts to make web programming just a little bit easier. > > From Erlang point of view the browser *is* an Erlang process. > > Assume we have a web page populated with divs. For example: > >
> ... >
> >
> ... >
> > Erlang thinks the browser is a process. To fill div a with HTML an > Erlang process evaluates the command: > > Browser ! [{cmd, fill_div}, {id, a}, {txt, B}] > > Where B is a binary containing HTML. Why not implement this for the modifications instead, though? It's well defined and works very well: http://taconite.sourceforge.net/ Don't need to be XML of course, but the actions defined are a good start of what frontend people usually want. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From naikvin@REDACTED Thu Dec 13 11:56:11 2012 From: naikvin@REDACTED (Vineet Naik) Date: Thu, 13 Dec 2012 16:26:11 +0530 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: <50C9A890.6040104@gmail.com> References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> <50C99CDC.8050401@gmail.com> <50C9A890.6040104@gmail.com> Message-ID: On Thu, Dec 13, 2012 at 3:36 PM, Dmitry Klionsky wrote: > > Actually, it doesn't seem to be a good idea to manually exclude any > application until you know precisely what you're doing. > Reltool will exclude any unused applications itself. Just stop excluding > the applications in reltool.config and see if it works. > Not able to get this to work by removing the excludes. Getting all the previous errors of "init terminating in do_boot" again. > > > > > On 12/13/2012 12:49 PM, Vineet Naik wrote: > > Thanks Dmitry, > > After adding inets to the app config file it failed with - > > {"init terminating in do_boot",{'cannot load',mnesia_backup,get_file}} > > I then excluded mnesia in reltool.config as I am not using it (Is this > the correct thing to do?) > > Now it shows - > > {"init terminating in do_boot",{'cannot > load',erts_alloc_config,get_file}} > > I tried adding erts to app config but get the same error. > > I don't think erts can be excluded, right? > > I am not able to understand why is it required to either include in app > config or > exclude it in reltool config. What's going on here and who is behind the > magic, > reltools or rebar :-) > > Really appreciate all the help, it would have very difficult for me to > reach this far otherwise. Thanks a lot. > > > On Thu, Dec 13, 2012 at 2:46 PM, Dmitry Klionsky wrote: > >> Hi, >> >> >> >> BTW, isn't ftp in the stdlib? >> No, it's inside the inets application >> >> >> >> {"init terminating in do_boot",{'cannot load',ftp,get_file}} >> Maybe this will clarify things for you: >> http://stackoverflow.com/questions/10417469/could-not-start-release-with-rebar-generate-epgsql >> >> >> >> {app, ftp, [{incl_cond, exclude}]}, >> This won't work. The `ftp' isn't a separate application >> >> >> >> {app, inets, [{incl_cond, derived}]}, >> Add {app, inets, [{incl_cond, include}]}. >> But better include `inets' into a YOUR_APP.app.src file like this >> http://code.google.com/p/mochiweb/source/browse/trunk/src/mochiweb.app.src >> >> >> Best regards, >> Dmitry >> >> >> On 12/13/2012 10:28 AM, Vineet Naik wrote: >> >> On Thu, Dec 13, 2012 at 11:37 AM, Andrew Gopienko wrote: >> >>> Make sure you have sasl dependency in your app file. >>> >>> {applications, [ >>> kernel, >>> stdlib, >>> sasl >>> ]}, >> >> >> Thanks Andrew, that was indeed the problem why no sasl error log was >> generated. >> After adding sasl in applications, the error logged in it is, >> >> exception exit: {undef,[{inets,start,[]}, >> {exmpp_component,init,1}, >> {gen_fsm,init_it,6}, >> {proc_lib,init_p_do_apply,3}]} >> in function gen_fsm:init_it/6 >> >> I had added `{app, inets, [{incl_cond, exclude}]},` in reltool.config >> myself earlier. >> I tried removing that option out now and I am getting the following error >> while >> starting up >> >> {"init terminating in do_boot",{'cannot load',ftp,get_file}} >> >> I was getting a series of such errors before and I could fix them by >> excluding >> non required libraries one by one. 'inets' was excluded in order to fix >> the error for >> 'ftp' module. But it seems exmpp code needs inets:start/0. >> >> I tried replacing it with - >> >> {app, ftp, [{incl_cond, exclude}]}, >> >> and later >> >> {app, inets, [{incl_cond, derived}]}, >> >> But both don't work. What is the correct way to configure this? >> BTW, isn't ftp in the stdlib? I can load ftp module from inside any >> erlang shell >> >> Thanks, >> Vineet >> >> >> >>> >>> >>> >>> >>> 2012/12/13 Vineet Naik >>> >>>> Hi Roman, >>>> >>>> Thanks for replying. I have /log/sasl but it's empty. >>>> And there is no >>>> other file/dir in log dir. Do I need to include anything in code/config >>>> to make >>>> it log errors to a file? >>>> >>>> Regards, >>>> Vineet >>>> >>>> On Wed, Dec 12, 2012 at 9:27 PM, Roman Gafiyatullin < >>>> r.gafiyatullin@REDACTED> wrote: >>>> >>>>> Vineet, most probably something went wrong during startup. >>>>> Can anything similar to sasl-error.log be found in >>>>> the /log/ directory? >>>>> >>>>> -- >>>>> RG >>>>> >>>>> On Wednesday, December 12, 2012 at 6:31 pm, Vineet Naik wrote: >>>>> >>>>> Hello, >>>>> >>>>> I am trying to package my app using rebar. `rebar generate` command >>>>> runs without any >>>>> problem (no errors at least). But when I try to start the console by >>>>> running the script generated >>>>> by rebar in bin directory, it fails with "Kernel pid terminated" >>>>> >>>>> A huge crash_dump is also created but I am not able to gather any >>>>> info from it. >>>>> >>>>> I checked the erlang docs[1], I found this paragraph about the error >>>>> >>>>> "Kernel pid terminated (Who) (Exit-reason)" - The kernel supervisor >>>>> has detected a failure, usually that the application_controller has shut >>>>> down (Who = application_controller, Why = shutdown). The application >>>>> controller may have shut down for a number of reasons, the most usual being >>>>> that the node name of the distributed Erlang node is already in use. A >>>>> complete supervisor tree "crash" (i.e., the top supervisors have exited) >>>>> will give about the same result. This message comes from the Erlang code >>>>> and not from the virtual machine itself. It is always due to some kind of >>>>> failure in an application, either within OTP or a "user-written" one. >>>>> Looking at the error log for your application is probably the first step to >>>>> take. >>>>> >>>>> >>>>> If I run `console_clean` command, then the console start in the >>>>> correct node. Does >>>>> this rule out the "Node is already in use" case above? >>>>> >>>>> I also tried running the code from an erlang shell with the ebin dir >>>>> paths of all required >>>>> libs added manually to sys path and it works perfectly. >>>>> >>>>> How can I go about debugging this? >>>>> >>>>> [1] http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 >>>>> >>>>> Thanks, >>>>> Vineet >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Vineet Naik >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> >> >> -- >> Vineet Naik >> >> >> >> >> _______________________________________________ >> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> -- >> Best regards, >> Dmitry Klionsky >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Vineet Naik > > > > > -- > Best regards, > Dmitry Klionsky > > -- Vineet Naik -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Thu Dec 13 12:12:10 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Thu, 13 Dec 2012 12:12:10 +0100 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> <50C99CDC.8050401@gmail.com> <50C9A890.6040104@gmail.com> Message-ID: On Thu, Dec 13, 2012 at 11:56 AM, Vineet Naik wrote: > On Thu, Dec 13, 2012 at 3:36 PM, Dmitry Klionsky wrote: > > > > Actually, it doesn't seem to be a good idea to manually exclude any > > application until you know precisely what you're doing. Reltool > > will exclude any unused applications itself. Just stop excluding > > the applications in reltool.config and see if it works. > > Not able to get this to work by removing the excludes. Getting all > the previous errors of "init terminating in do_boot" again. Can you post the complete reltool.config or preferably a demo project so that we can help you fix the reltool spec first? What OTP version are you using? As you can see in https://github.com/rebar/rebar/blob/master/priv/templates/simplenode.reltool.config if you have complete (and correct) app resource files, reltool figures out all required apps and modules automatically without the need to specify detailed inclusion/exclusion rules. From erlang@REDACTED Thu Dec 13 12:13:42 2012 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 13 Dec 2012 12:13:42 +0100 Subject: [erlang-questions] ANN: ezwebframe - an easy web framework In-Reply-To: <50C9B1C7.3060109@ninenines.eu> References: <50C9B1C7.3060109@ninenines.eu> Message-ID: On Thu, Dec 13, 2012 at 11:45 AM, Lo?c Hoguin wrote: > On 12/13/2012 11:17 AM, Joe Armstrong wrote: > >> ezwebframe >> ========== >> >> https://github.com/joearms/**ezwebframe >> >> Pronounced "Easy web frame." >> >> About >> ===== >> >> Ezwebframe attempts to make web programming just a little bit easier. >> >> From Erlang point of view the browser *is* an Erlang process. >> >> Assume we have a web page populated with divs. For example: >> >>
>> ... >>
>> >>
>> ... >>
>> >> Erlang thinks the browser is a process. To fill div a with HTML an >> Erlang process evaluates the command: >> >> Browser ! [{cmd, fill_div}, {id, a}, {txt, B}] >> >> Where B is a binary containing HTML. >> > > Why not implement this for the modifications instead, though? It's well > defined and works very well: http://taconite.sourceforge.**net/ > I don't understand ... What actually happens is this ... Browser ! [{cmd, fill_div}, {id, X}, {txt, Y}] gets converted to the JS function call fill_div(o) where o = {cmd:'fill_div', id:X, txt:Y} and I have pre-defined fill_div as function fill_div(o){ $("#" + o.id).html(o.txt); } (assumes jQuery) If you want to extend the system you say Browser ! [{cmd,new_cmd},{x,...},{y,...}] and supply a new js function new_cmd(o) which will be called with o = {cmd:'new_cmd'},{x,...},...} This provides an extensible abstract interface between the browser and Erlang. Websockets + JSON + a tiny bit of glue = good stuff /Joe > > Don't need to be XML of course, but the actions defined are a good start > of what frontend people usually want. > So if you want an action kick_ball I'd write Browser ! [{cmd,kick_ball}, {velocity, 23}, ...] in Erlang, and provide some JS to do the kicking function kick_ball(o){ ... } which would be called with o = {cmd:'kick_ball', velocity:23, ...} /Joe > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From send2philip@REDACTED Thu Dec 13 12:36:52 2012 From: send2philip@REDACTED (Philip Clarke) Date: Thu, 13 Dec 2012 11:36:52 +0000 Subject: [erlang-questions] backing up mnesia ram_copies tables Message-ID: Hi, Should it be possible to make a backup of tables in mnesia if you just have ram_copies ? The documentation at http://www.erlang.org/doc/man/mnesia.html#backup-1 does not seem to imply that there is any such limitation. However when I examine the backup file created from mnesia:backup/1 it does not seem to have any table data. I use this simple test to check the backup. It will fail with {aborted,{no_exists,bup_rec,all}}. -record(bup_rec, {key, val}). test_cannot_restore_backup() -> Node = node(), mnesia:lkill(), ok = mnesia:delete_schema([Node]), mnesia:start(), mnesia:wait_for_tables([schema], infinity), OldBup = "old.BUP", file:delete(OldBup), CreateList = [{ram_copies, [Node]}, {attributes, record_info(fields, bup_rec)}], {atomic, ok} = mnesia:create_table(bup_rec, CreateList), mnesia:wait_for_tables([bup_rec], infinity), OldRecs = [#bup_rec{key = I, val = I * I} || I <- lists:seq(1, 10)], lists:foreach(fun(R) -> ok = mnesia:dirty_write(R) end, OldRecs), mnesia:backup(OldBup), mnesia:stop(), ok = mnesia:delete_schema([Node]), mnesia:start(), mnesia:wait_for_tables([schema], infinity), mnesia:restore(OldBup, [{default_op, recreate_tables}]), ok = mnesia:table_info(bup_rec, all). I would like to know if I am doing the backup wrong, or if this is something which is not supported. Thanks Philip -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Dec 13 12:55:28 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 13 Dec 2012 12:55:28 +0100 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> <50C99CDC.8050401@gmail.com> <50C9A890.6040104@gmail.com> Message-ID: <50C9C230.7040606@ninenines.eu> The problem is probably that rebar doesn't use the default reltool mod_cond value. So not all modules are included. Try changing it to this: {mod_cond, all} On 12/13/2012 11:56 AM, Vineet Naik wrote: > On Thu, Dec 13, 2012 at 3:36 PM, Dmitry Klionsky > wrote: > > > Actually, it doesn't seem to be a good idea to manually exclude any > application until you know precisely what you're doing. > Reltool will exclude any unused applications itself. Just stop > excluding the applications in reltool.config and see if it works. > > > Not able to get this to work by removing the excludes. Getting all the > previous errors of > "init terminating in do_boot" again. > > > > > > On 12/13/2012 12:49 PM, Vineet Naik wrote: >> Thanks Dmitry, >> >> After adding inets to the app config file it failed with - >> >> {"init terminating in do_boot",{'cannot load',mnesia_backup,get_file}} >> >> I then excluded mnesia in reltool.config as I am not using it (Is >> this the correct thing to do?) >> >> Now it shows - >> >> {"init terminating in do_boot",{'cannot >> load',erts_alloc_config,get_file}} >> >> I tried adding erts to app config but get the same error. >> >> I don't think erts can be excluded, right? >> >> I am not able to understand why is it required to either include >> in app config or >> exclude it in reltool config. What's going on here and who is >> behind the magic, >> reltools or rebar :-) >> >> Really appreciate all the help, it would have very difficult for >> me to >> reach this far otherwise. Thanks a lot. >> >> >> On Thu, Dec 13, 2012 at 2:46 PM, Dmitry Klionsky >> > wrote: >> >> Hi, >> >> >> >> BTW, isn't ftp in the stdlib? >> No, it's inside the inets application >> >> >> >> {"init terminating in do_boot",{'cannot load',ftp,get_file}} >> Maybe this will clarify things for you: >> http://stackoverflow.com/questions/10417469/could-not-start-release-with-rebar-generate-epgsql >> >> >> >> >> {app, ftp, [{incl_cond, exclude}]}, >> This won't work. The `ftp' isn't a separate application >> >> >> >> {app, inets, [{incl_cond, derived}]}, >> Add {app, inets, [{incl_cond, include}]}. >> But better include `inets' into a YOUR_APP.app.src file like >> this >> http://code.google.com/p/mochiweb/source/browse/trunk/src/mochiweb.app.src >> >> >> Best regards, >> Dmitry >> >> >> On 12/13/2012 10:28 AM, Vineet Naik wrote: >>> On Thu, Dec 13, 2012 at 11:37 AM, Andrew Gopienko >>> > wrote: >>> >>> Make sure you have sasl dependency in your app file. >>> >>> {applications, [ >>> kernel, >>> stdlib, >>> sasl >>> ]}, >>> >>> >>> Thanks Andrew, that was indeed the problem why no sasl error >>> log was generated. >>> After adding sasl in applications, the error logged in it is, >>> >>> exception exit: {undef,[{inets,start,[]}, >>> {exmpp_component,init,1}, >>> {gen_fsm,init_it,6}, >>> {proc_lib,init_p_do_apply,3}]} >>> in function gen_fsm:init_it/6 >>> >>> I had added `{app, inets, [{incl_cond, exclude}]},` in >>> reltool.config myself earlier. >>> I tried removing that option out now and I am getting the >>> following error while >>> starting up >>> >>> {"init terminating in do_boot",{'cannot load',ftp,get_file}} >>> >>> I was getting a series of such errors before and I could fix >>> them by excluding >>> non required libraries one by one. 'inets' was excluded in >>> order to fix the error for >>> 'ftp' module. But it seems exmpp code needs inets:start/0. >>> >>> I tried replacing it with - >>> >>> {app, ftp, [{incl_cond, exclude}]}, >>> >>> and later >>> >>> {app, inets, [{incl_cond, derived}]}, >>> >>> But both don't work. What is the correct way to configure this? >>> BTW, isn't ftp in the stdlib? I can load ftp module from >>> inside any erlang shell >>> >>> Thanks, >>> Vineet >>> >>> >>> >>> >>> >>> 2012/12/13 Vineet Naik >> > >>> >>> Hi Roman, >>> >>> Thanks for replying. I have /log/sasl >>> but it's empty. And there is no >>> other file/dir in log dir. Do I need to include >>> anything in code/config to make >>> it log errors to a file? >>> >>> Regards, >>> Vineet >>> >>> On Wed, Dec 12, 2012 at 9:27 PM, Roman Gafiyatullin >>> >> > wrote: >>> >>> Vineet, most probably something went wrong during >>> startup. >>> Can anything similar to sasl-error.log be found >>> in the /log/ directory? >>> >>> -- >>> RG >>> >>> On Wednesday, December 12, 2012 at 6:31 pm, >>> Vineet Naik wrote: >>> >>>> Hello, >>>> >>>> I am trying to package my app using rebar. >>>> `rebar generate` command runs without any >>>> problem (no errors at least). But when I try to >>>> start the console by running the script generated >>>> by rebar in bin directory, it fails with "Kernel >>>> pid terminated" >>>> >>>> A huge crash_dump is also created but I am not >>>> able to gather any info from it. >>>> >>>> I checked the erlang docs[1], I found this >>>> paragraph about the error >>>> >>>>> "Kernel pid terminated (Who) (Exit-reason)" - >>>>> The kernel supervisor has detected a failure, >>>>> usually that the application_controller has >>>>> shut down (Who = application_controller, Why = >>>>> shutdown). The application controller may have >>>>> shut down for a number of reasons, the most >>>>> usual being that the node name of the >>>>> distributed Erlang node is already in use. A >>>>> complete supervisor tree "crash" (i.e., the top >>>>> supervisors have exited) will give about the >>>>> same result. This message comes from the Erlang >>>>> code and not from the virtual machine itself. >>>>> It is always due to some kind of failure in an >>>>> application, either within OTP or a >>>>> "user-written" one. Looking at the error log >>>>> for your application is probably the first step >>>>> to take. >>>> >>>> If I run `console_clean` command, then the >>>> console start in the correct node. Does >>>> this rule out the "Node is already in use" case >>>> above? >>>> >>>> I also tried running the code from an erlang >>>> shell with the ebin dir paths of all required >>>> libs added manually to sys path and it works >>>> perfectly. >>>> >>>> How can I go about debugging this? >>>> >>>> [1] >>>> http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 >>>> >>>> Thanks, >>>> Vineet >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> >>> -- >>> Vineet Naik >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> >>> >>> -- >>> Vineet Naik >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> -- >> Best regards, >> Dmitry Klionsky >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> -- >> Vineet Naik >> >> > > > -- > Best regards, > Dmitry Klionsky > > > > > -- > Vineet Naik > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From naikvin@REDACTED Thu Dec 13 12:59:31 2012 From: naikvin@REDACTED (Vineet Naik) Date: Thu, 13 Dec 2012 17:29:31 +0530 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> <50C99CDC.8050401@gmail.com> <50C9A890.6040104@gmail.com> Message-ID: I could finally get it working by changing the profile option from embedded to development. Everything works fine now and I have also removed the excludes which I were added earlier. Besides the large size of the release (72 MB in this case) does development profile have any side effects on performance etc? Is development profile recommended for production? It's starting to make a lot more sense to me now although not entirely! Thanks for all the help. On Thu, Dec 13, 2012 at 4:42 PM, Tuncer Ayaz wrote: > On Thu, Dec 13, 2012 at 11:56 AM, Vineet Naik wrote: > > On Thu, Dec 13, 2012 at 3:36 PM, Dmitry Klionsky wrote: > > > > > > Actually, it doesn't seem to be a good idea to manually exclude any > > > application until you know precisely what you're doing. Reltool > > > will exclude any unused applications itself. Just stop excluding > > > the applications in reltool.config and see if it works. > > > > Not able to get this to work by removing the excludes. Getting all > > the previous errors of "init terminating in do_boot" again. > > Can you post the complete reltool.config or preferably a demo project > so that we can help you fix the reltool spec first? > > What OTP version are you using? > > As you can see in > > https://github.com/rebar/rebar/blob/master/priv/templates/simplenode.reltool.config > if you have complete (and correct) app resource files, reltool figures > out all required apps and modules automatically without the need to > specify detailed inclusion/exclusion rules. > -- Vineet Naik -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Thu Dec 13 13:10:33 2012 From: dangud@REDACTED (Dan Gudmundsson) Date: Thu, 13 Dec 2012 13:10:33 +0100 Subject: [erlang-questions] backing up mnesia ram_copies tables In-Reply-To: References: Message-ID: It is the way it is intended. If you create your own checkpoint you can specify what you want to be included in the backup. /Dan On Thu, Dec 13, 2012 at 12:36 PM, Philip Clarke wrote: > Hi, > > Should it be possible to make a backup of tables in mnesia if you just > have ram_copies ? > > The documentation at http://www.erlang.org/doc/man/mnesia.html#backup-1 does > not seem to imply that there is any such limitation. > > However when I examine the backup file created from mnesia:backup/1 it > does not seem to have any table data. > > I use this simple test to check the backup. It will fail > with {aborted,{no_exists,bup_rec,all}}. > > > -record(bup_rec, {key, val}). > > test_cannot_restore_backup() -> > Node = node(), > mnesia:lkill(), > ok = mnesia:delete_schema([Node]), > mnesia:start(), > mnesia:wait_for_tables([schema], infinity), > > OldBup = "old.BUP", > file:delete(OldBup), > > CreateList = [{ram_copies, [Node]}, > {attributes, record_info(fields, bup_rec)}], > {atomic, ok} = mnesia:create_table(bup_rec, CreateList), > mnesia:wait_for_tables([bup_rec], infinity), > OldRecs = [#bup_rec{key = I, val = I * I} || I <- lists:seq(1, 10)], > lists:foreach(fun(R) -> ok = mnesia:dirty_write(R) end, OldRecs), > > mnesia:backup(OldBup), > > mnesia:stop(), > ok = mnesia:delete_schema([Node]), > mnesia:start(), > mnesia:wait_for_tables([schema], infinity), > > mnesia:restore(OldBup, [{default_op, recreate_tables}]), > ok = mnesia:table_info(bup_rec, all). > > > I would like to know if I am doing the backup wrong, or if this is > something which is not supported. > > Thanks > Philip > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Thu Dec 13 13:14:58 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Thu, 13 Dec 2012 13:14:58 +0100 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: <50C9C230.7040606@ninenines.eu> References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> <50C99CDC.8050401@gmail.com> <50C9A890.6040104@gmail.com> <50C9C230.7040606@ninenines.eu> Message-ID: On Thu, Dec 13, 2012 at 12:55 PM, Loic Hoguin wrote: > The problem is probably that rebar doesn't use the default reltool > mod_cond value. So not all modules are included. Try changing it to > this: > > {mod_cond, all} For certain incorrect/incomplete app resource files you may have to use such a work around. Normally the right solution is to fix the app resource files. From ulf@REDACTED Thu Dec 13 13:16:27 2012 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 13 Dec 2012 13:16:27 +0100 Subject: [erlang-questions] ANN: ezwebframe - an easy web framework In-Reply-To: References: Message-ID: Nice! I sent a little pull request on the shell demo: https://github.com/joearms/ezwebframe/pull/2 "First thing I tried with the shell demo was to enter c:regs(), which resulted in output in the real erlang shell, but not much in the browser. This patch spawns a small group_leader process which collects IO generated by the evaluated command and sends it to the browser. The browser-side output is currently pretty ugly. I'll gladly leave that to some HTML/JS wiz to fix. :)" BR, Ulf W PS OTP team - I noticed in kernel/src/group.erl that the group_leader code has gotten a bit more complex with the addition of unicode. It would be nice to separate the group_leader logic from the interaction with the tty driver. It would make a nice behavior. On 13 Dec 2012, at 11:17, Joe Armstrong wrote: > ezwebframe > ========== > > https://github.com/joearms/ezwebframe > > Pronounced "Easy web frame." > > About > ===== > > Ezwebframe attempts to make web programming just a little bit easier. > > From Erlang point of view the browser *is* an Erlang process. > > Assume we have a web page populated with divs. For example: > >
> ... >
> >
> ... >
> > Erlang thinks the browser is a process. To fill div a with HTML an > Erlang process evaluates the command: > > Browser ! [{cmd, fill_div}, {id, a}, {txt, B}] > > Where B is a binary containing HTML. > > In the browser controls can be programmed to send messages to Erlang, > for example, when we click on a button in the browser the Erlang > process controlling the window will be sent a message which can be > received with the statement: > > receive > {Browser, {struct, [{clicked, ButtonName}]}} -> > ... > end > > All this is achieved using a thin JSON layer over websockets and > with cowboy managing the websockets. > > Cheers > > /Joe > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Thu Dec 13 13:24:40 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Thu, 13 Dec 2012 13:24:40 +0100 Subject: [erlang-questions] PropEr commercial training? Message-ID: Dear Erlangers, Do you know about any property-based testing training using PropEr? I'd be interested on hearing out about the types of training available, the target audiences and how they've worked in the past. Kind regards, Motiejus Jak?tys From tuncer.ayaz@REDACTED Thu Dec 13 13:28:02 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Thu, 13 Dec 2012 13:28:02 +0100 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> <50C99CDC.8050401@gmail.com> <50C9A890.6040104@gmail.com> Message-ID: On Thu, Dec 13, 2012 at 12:59 PM, Vineet Naik wrote: > I could finally get it working by changing the profile option from > embedded to development. Everything works fine now and I have also > removed the excludes which I were added earlier. > > Besides the large size of the release (72 MB in this case) does > development profile have any side effects on performance etc? Is > development profile recommended for production? Changing the profile affects the default filters. See http://www.erlang.org/doc/man/reltool.html. Since R15B02 reltool can create "slim" releases without erts and otp apps. https://github.com/rebar/rebar/issues/7 is almost ready to be submitted as a pull request to rebar/rebar thanks to Shunichi Shinohara, but some details remain to be fixed. > It's starting to make a lot more sense to me now although not > entirely! Thanks for all the help. From essen@REDACTED Thu Dec 13 13:28:48 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 13 Dec 2012 13:28:48 +0100 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> <50C99CDC.8050401@gmail.com> <50C9A890.6040104@gmail.com> <50C9C230.7040606@ninenines.eu> Message-ID: <50C9CA00.1010806@ninenines.eu> On 12/13/2012 01:14 PM, Tuncer Ayaz wrote: > On Thu, Dec 13, 2012 at 12:55 PM, Loic Hoguin wrote: >> The problem is probably that rebar doesn't use the default reltool >> mod_cond value. So not all modules are included. Try changing it to >> this: >> >> {mod_cond, all} > > For certain incorrect/incomplete app resource files you may have to > use such a work around. Normally the right solution is to fix the app > resource files. Why change the default reltool value in the first place, though? I don't get it. Also how would reltool know I need certain modules if they're only used dynamically? -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From tuncer.ayaz@REDACTED Thu Dec 13 13:42:51 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Thu, 13 Dec 2012 13:42:51 +0100 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: <50C9CA00.1010806@ninenines.eu> References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> <50C99CDC.8050401@gmail.com> <50C9A890.6040104@gmail.com> <50C9C230.7040606@ninenines.eu> <50C9CA00.1010806@ninenines.eu> Message-ID: On Thu, Dec 13, 2012 at 1:28 PM, Loic Hoguin wrote: > On 12/13/2012 01:14 PM, Tuncer Ayaz wrote: >> >> On Thu, Dec 13, 2012 at 12:55 PM, Loic Hoguin wrote: >>> >>> The problem is probably that rebar doesn't use the default reltool >>> mod_cond value. So not all modules are included. Try changing it >>> to this: >>> >>> {mod_cond, all} >> >> >> For certain incorrect/incomplete app resource files you may have to >> use such a work around. Normally the right solution is to fix the >> app resource files. > > > Why change the default reltool value in the first place, though? I > don't get it. Also how would reltool know I need certain modules if > they're only used dynamically? The default reltool spec template you get when using 'rebar create' is configured to create a release that contains only what's required. I don't think reltool can or should know modules you load from unreferenced applications at runtime. If it's always the same module from the same application, you should specify the application in your app's app resource file. If this can't be done for some reason, just change the reltool spec accordingly. Unsurprisingly the default template won't work for every project and it's not supposed to. From essen@REDACTED Thu Dec 13 13:54:31 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 13 Dec 2012 13:54:31 +0100 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> <50C99CDC.8050401@gmail.com> <50C9A890.6040104@gmail.com> <50C9C230.7040606@ninenines.eu> <50C9CA00.1010806@ninenines.eu> Message-ID: <50C9D007.8070902@ninenines.eu> On 12/13/2012 01:42 PM, Tuncer Ayaz wrote: > On Thu, Dec 13, 2012 at 1:28 PM, Loic Hoguin wrote: >> On 12/13/2012 01:14 PM, Tuncer Ayaz wrote: >>> >>> On Thu, Dec 13, 2012 at 12:55 PM, Loic Hoguin wrote: >>>> >>>> The problem is probably that rebar doesn't use the default reltool >>>> mod_cond value. So not all modules are included. Try changing it >>>> to this: >>>> >>>> {mod_cond, all} >>> >>> >>> For certain incorrect/incomplete app resource files you may have to >>> use such a work around. Normally the right solution is to fix the >>> app resource files. >> >> >> Why change the default reltool value in the first place, though? I >> don't get it. Also how would reltool know I need certain modules if >> they're only used dynamically? > > The default reltool spec template you get when using 'rebar create' is > configured to create a release that contains only what's required. I > don't think reltool can or should know modules you load from > unreferenced applications at runtime. If it's always the same module > from the same application, you should specify the application in your > app's app resource file. If this can't be done for some reason, just > change the reltool spec accordingly. Unsurprisingly the default > template won't work for every project and it's not supposed to. Sure but you're contradicting people's expectations. If you could use rebar's releases without having to know reltool this wouldn't be a problem, but the reltool documentation here http://www.erlang.org/doc/man/reltool.html says: mod_cond This parameter controls the module inclusion policy. It defaults to all which means that if an application is included (either explicitly or implicitly) all modules in that application will be included. Not setting this to all by default is not only against the existing documentation, but also has the problematic side effect that following this guide https://github.com/rebar/rebar/wiki/Release-handling will most likely *not* create a working release. It used to, but I suppose the default was changed in recent months, which means that a few extra steps and checks are needed. Also with dynamically used modules (e.g Module = ranch_tcp), having mod_cond not set to all means you need to specifically include it (or include everything from the application), which is yet another extra missing step. The default should be to make it work, not to make it pretty and broken. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From mjanczyk@REDACTED Thu Dec 13 14:06:09 2012 From: mjanczyk@REDACTED (Marcin Janczyk) Date: Thu, 13 Dec 2012 14:06:09 +0100 Subject: [erlang-questions] pm_webshere_mq release (was: Re: Erlang and Websphere MQ) Message-ID: <50C9D2C1.3010305@power.com.pl> Hi, pm_websphere_mq is a simple Erlang application bundled with a C port program. It allows reading from and writing to an IBM WMQ queue. pm_websphere_mq is published under 2-clause BSD license. You can find it at https://github.com/powermedia/pm_websphere_mq . Regards, Marcin > Please keep us informed. > On Friday, December 7, 2012 12:09:44 PM UTC-5, Wojtek Narczy?ski wrote: >> >> On 2012-12-07 10:46, Yash Ganthe wrote: >> > >> > What is the best way to read from and write to IBM Websphere MQ using >> > Erlang. Are there any modules available for doing that? >> We have a port program in C++. Will investigate its quality / >> releasability next week. From essen@REDACTED Thu Dec 13 14:51:23 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 13 Dec 2012 14:51:23 +0100 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: <50C9D007.8070902@ninenines.eu> References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> <50C99CDC.8050401@gmail.com> <50C9A890.6040104@gmail.com> <50C9C230.7040606@ninenines.eu> <50C9CA00.1010806@ninenines.eu> <50C9D007.8070902@ninenines.eu> Message-ID: <50C9DD5B.8020802@ninenines.eu> On 12/13/2012 01:54 PM, Lo?c Hoguin wrote: > On 12/13/2012 01:42 PM, Tuncer Ayaz wrote: >> On Thu, Dec 13, 2012 at 1:28 PM, Loic Hoguin wrote: >>> On 12/13/2012 01:14 PM, Tuncer Ayaz wrote: >>>> >>>> On Thu, Dec 13, 2012 at 12:55 PM, Loic Hoguin wrote: >>>>> >>>>> The problem is probably that rebar doesn't use the default reltool >>>>> mod_cond value. So not all modules are included. Try changing it >>>>> to this: >>>>> >>>>> {mod_cond, all} >>>> >>>> >>>> For certain incorrect/incomplete app resource files you may have to >>>> use such a work around. Normally the right solution is to fix the >>>> app resource files. >>> >>> >>> Why change the default reltool value in the first place, though? I >>> don't get it. Also how would reltool know I need certain modules if >>> they're only used dynamically? >> >> The default reltool spec template you get when using 'rebar create' is >> configured to create a release that contains only what's required. I >> don't think reltool can or should know modules you load from >> unreferenced applications at runtime. If it's always the same module >> from the same application, you should specify the application in your >> app's app resource file. If this can't be done for some reason, just >> change the reltool spec accordingly. Unsurprisingly the default >> template won't work for every project and it's not supposed to. > > Sure but you're contradicting people's expectations. If you could use > rebar's releases without having to know reltool this wouldn't be a > problem, but the reltool documentation here > http://www.erlang.org/doc/man/reltool.html says: > > mod_cond > > This parameter controls the module inclusion policy. It defaults to > all which means that if an application is included (either explicitly > or implicitly) all modules in that application will be included. > > Not setting this to all by default is not only against the existing > documentation, but also has the problematic side effect that following > this guide https://github.com/rebar/rebar/wiki/Release-handling will > most likely *not* create a working release. It used to, but I suppose > the default was changed in recent months, which means that a few extra > steps and checks are needed. > > Also with dynamically used modules (e.g Module = ranch_tcp), having > mod_cond not set to all means you need to specifically include it (or > include everything from the application), which is yet another extra > missing step. > > The default should be to make it work, not to make it pretty and broken. Taking this off-list, no need to pollute the ML more. :) -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From tuncer.ayaz@REDACTED Thu Dec 13 14:54:15 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Thu, 13 Dec 2012 14:54:15 +0100 Subject: [erlang-questions] Running rebar generated release failing with Kernel pid terminated" In-Reply-To: <50C9DD5B.8020802@ninenines.eu> References: <3F7D7EDABA3445E8A7CE21600862F5A1@me.com> <50C99CDC.8050401@gmail.com> <50C9A890.6040104@gmail.com> <50C9C230.7040606@ninenines.eu> <50C9CA00.1010806@ninenines.eu> <50C9D007.8070902@ninenines.eu> <50C9DD5B.8020802@ninenines.eu> Message-ID: On Thu, Dec 13, 2012 at 2:51 PM, Loic Hoguin wrote: > Taking this off-list, no need to pollute the ML more. :) ACK, let's see if we can come up with a patch for the template. From arif.ishaq@REDACTED Thu Dec 13 16:06:09 2012 From: arif.ishaq@REDACTED (Arif Ishaq) Date: Thu, 13 Dec 2012 15:06:09 +0000 Subject: [erlang-questions] simple_one_for_one supervisor, transient restart strategy In-Reply-To: References: Message-ID: <1CAB695D2C2A8F4BB0B242A5B44C75E901E6FF@ESESSMB301.ericsson.se> Hi, I have tried this with R15B and it works for me. In practice, I have the child implemented as a gen_server which handles messages stopnormal, stopshutdown and stopabort treated with {stop, normal, State}, {stop, shutdown, State} and {stop, abort, State} respectively. When I send the messages stopnormal or stopshutdown, the child dies without getting restarted. When I send a stopabort, it gets restarted with the arguments with which it was created. Can you please post your code to see what went wrong? Regards Arif -----Original Message----- From: Paul Peregud [mailto:paulperegud@REDACTED] Sent: mercoled? 12 dicembre 2012 16.20 To: erlang-questions@REDACTED Subject: [erlang-questions] simple_one_for_one supervisor, transient restart strategy I've searched mailing list for this question and found nothing. There is this one peculiar behavior when you use simple_one_for_one with transient strategy. You do supervisor:start_child(Ref, Args), the child executes and if it dies gracefully everything is fine, child is removed from supervisor tree. However, if it fails, it is restarted WITHOUT Args. This combination of type/strategy would be a great tool for fire-and-forget tasks if it was not for that small issue. Is it a bug or is it a feature? -- Best regards, Paul Peregud +48602112091 From desired.mta@REDACTED Thu Dec 13 17:34:49 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Thu, 13 Dec 2012 17:34:49 +0100 Subject: [erlang-questions] "official" pre-built rebar binaries Message-ID: Hi, this is how I used to start my Erlang project Makefiles: REBAR ?= ./rebar REBAR_URL ?= https://github.com/downloads/basho/rebar/rebar ./rebar: erl -noshell -s inets start -s ssl start \ -eval 'httpc:request(get, {"$(REBAR_URL)", []}, [], [{stream, "./rebar"}])' \ -s inets stop -s init stop chmod +x ./rebar It worked fine and such, but for a new project I need a newer rebar. Would it be possible to have a more recent pre-built "official" rebar in an "official" repository? Suggestion: create a rebar binary for every tag. Compile it on a lowest supported platform (R13B04?), and put the resulting binaries like this: https://github.com/downloads/rebar/rebar/rebar-2.0.0 https://github.com/downloads/rebar/rebar/rebar-2.1.0-pre -- Motiejus Jak?tys From essen@REDACTED Thu Dec 13 17:37:02 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Thu, 13 Dec 2012 17:37:02 +0100 Subject: [erlang-questions] "official" pre-built rebar binaries In-Reply-To: References: Message-ID: <50CA042E.3040903@ninenines.eu> On 12/13/2012 05:34 PM, Motiejus Jak?tys wrote: > Hi, > > this is how I used to start my Erlang project Makefiles: > > REBAR ?= ./rebar > REBAR_URL ?= https://github.com/downloads/basho/rebar/rebar > > ./rebar: > erl -noshell -s inets start -s ssl start \ > -eval 'httpc:request(get, {"$(REBAR_URL)", []}, [], [{stream, > "./rebar"}])' \ > -s inets stop -s init stop > chmod +x ./rebar > > It worked fine and such, but for a new project I need a newer rebar. > Would it be possible to have a more recent pre-built "official" rebar > in an "official" repository? > > Suggestion: create a rebar binary for every tag. Compile it on a > lowest supported platform (R13B04?), and put the resulting binaries > like this: > https://github.com/downloads/rebar/rebar/rebar-2.0.0 > https://github.com/downloads/rebar/rebar/rebar-2.1.0-pre Github just announced that they removed support for file upload though. https://github.com/blog/1302-goodbye-uploads -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From desired.mta@REDACTED Thu Dec 13 17:42:21 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Thu, 13 Dec 2012 17:42:21 +0100 Subject: [erlang-questions] "official" pre-built rebar binaries In-Reply-To: <50CA042E.3040903@ninenines.eu> References: <50CA042E.3040903@ninenines.eu> Message-ID: On Thu, Dec 13, 2012 at 5:37 PM, Lo?c Hoguin wrote: > On 12/13/2012 05:34 PM, Motiejus Jak?tys wrote: > > Github just announced that they removed support for file upload though. > > https://github.com/blog/1302-goodbye-uploads Oh what a bad news, pity. However, if community and basho guys agree with the concept, maybe it could be hosted somewhere in basho? Or still is it still seen as the best practice to bundle rebar in the repository or trust the user to have one and it's just me against having binary stuff in the repository? -- Motiejus Jak?tys From tuncer.ayaz@REDACTED Thu Dec 13 17:52:09 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Thu, 13 Dec 2012 17:52:09 +0100 Subject: [erlang-questions] "official" pre-built rebar binaries In-Reply-To: References: Message-ID: On Thu, Dec 13, 2012 at 5:34 PM, Motiejus Jak?tys wrote: > Hi, > > this is how I used to start my Erlang project Makefiles: > > REBAR ?= ./rebar > REBAR_URL ?= https://github.com/downloads/basho/rebar/rebar Dizzy usually puts a new binary per release in the wiki by calling 'make binary': REBAR_URL ?= https://raw.github.com/wiki/rebar/rebar/rebar > ./rebar: > erl -noshell -s inets start -s ssl start \ > -eval 'httpc:request(get, {"$(REBAR_URL)", []}, [], [{stream, > "./rebar"}])' \ > -s inets stop -s init stop > chmod +x ./rebar > > It worked fine and such, but for a new project I need a newer rebar. > Would it be possible to have a more recent pre-built "official" rebar > in an "official" repository? > > Suggestion: create a rebar binary for every tag. Compile it on a > lowest supported platform (R13B04?), and put the resulting binaries > like this: > https://github.com/downloads/rebar/rebar/rebar-2.0.0 > https://github.com/downloads/rebar/rebar/rebar-2.1.0-pre From freza@REDACTED Thu Dec 13 17:34:44 2012 From: freza@REDACTED (Jachym Holecek) Date: Thu, 13 Dec 2012 11:34:44 -0500 Subject: [erlang-questions] "official" pre-built rebar binaries In-Reply-To: References: <50CA042E.3040903@ninenines.eu> Message-ID: <20121213163444.GA12549@circlewave.net> # Motiejus Jak?tys 2012-12-13: > On Thu, Dec 13, 2012 at 5:37 PM, Lo?c Hoguin wrote: > > On 12/13/2012 05:34 PM, Motiejus Jak?tys wrote: > > > > Github just announced that they removed support for file upload though. > > > > https://github.com/blog/1302-goodbye-uploads > > Oh what a bad news, pity. However, if community and basho guys agree > with the concept, maybe it could be hosted somewhere in basho? > > Or still is it still seen as the best practice to bundle rebar in the > repository or trust the user to have one and it's just me against > having binary stuff in the repository? Many people (including myself) bundle pre-built rebar in the repository out of convenience, but it could hardly be considered best practice from technical point of view -- due to ERTS' compatibility contract on BEAM bytecode versions by including pre-built rebar we also implicitly dictate the maximum Erlang version that will be able to build our project. If we were to ever upgrade the pre-built rebar file we'd on the other hand dictate minimum supported Erlang version. Probably not what anybody really wants in the long run, I suppose? Perhaps best to require rebar as a build-time dependency if you want to play safe? BR, -- Jachym From desired.mta@REDACTED Thu Dec 13 17:59:21 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Thu, 13 Dec 2012 17:59:21 +0100 Subject: [erlang-questions] "official" pre-built rebar binaries In-Reply-To: References: Message-ID: On Thu, Dec 13, 2012 at 5:52 PM, Tuncer Ayaz wrote: > On Thu, Dec 13, 2012 at 5:34 PM, Motiejus Jak?tys wrote: >> Hi, >> >> this is how I used to start my Erlang project Makefiles: >> >> REBAR ?= ./rebar >> REBAR_URL ?= https://github.com/downloads/basho/rebar/rebar > > Dizzy usually puts a new binary per release in the wiki by > calling 'make binary': > REBAR_URL ?= https://raw.github.com/wiki/rebar/rebar/rebar This is better. But how does the file get there? I cannot find how to add a file to the github wiki. Any thoughts about binary versioning ("my project works with rebar 2.0, don't know about rebar 2.1"). -- Motiejus Jak?tys From tuncer.ayaz@REDACTED Thu Dec 13 18:22:58 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Thu, 13 Dec 2012 18:22:58 +0100 Subject: [erlang-questions] "official" pre-built rebar binaries In-Reply-To: References: Message-ID: On Thu, Dec 13, 2012 at 5:59 PM, Motiejus Jakstys wrote: > This is better. But how does the file get there? I cannot find how to > add a file to the github wiki. https://github.com/rebar/rebar/blob/71c717d8/Makefile#L30-L33 From anthonym@REDACTED Thu Dec 13 19:38:30 2012 From: anthonym@REDACTED (Anthony Molinaro) Date: Thu, 13 Dec 2012 10:38:30 -0800 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? In-Reply-To: References: <20121206193809.GC27411@alumni.caltech.edu> <15C0D662-A77C-4858-AD93-3306E72317F6@erlang-solutions.com> <20121212182632.GB72594@alumni.caltech.edu> <20121212234803.GA75710@alumni.caltech.edu> <20121213002455.GA75861@alumni.caltech.edu> Message-ID: <20121213183830.GA81730@alumni.caltech.edu> Hi Steve, It may not be directly caused by your patch, but that was the first patch that stuck out when I started looking through the readmes for the releases. I've not been able to untangle mochiweb enough to do a test without it, so attached is a test file which uses mochiweb. If you compile it, you should see this with R14B04 Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.5 (abort with ^G) 1> mochi_test:start(). {ok,<0.44.0>} 2> mochi_test:test(10000). {ok,{{"HTTP/1.1",400,"Bad Request"}, [{"date","Thu, 13 Dec 2012 18:36:06 GMT"}, {"server","MochiWeb/1.0 (Any of you quaids got a smint?)"}, {"content-length","0"}], []}} 3> and this with R15B02 Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:2:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.2 (abort with ^G) 1> mochi_test:start(). {ok,<0.44.0>} 2> mochi_test:test(10000). {error,socket_closed_remotely} 3> I used Mochiweb 2.3.1 and compiled it and the test code with R14B04, then ran the test with the 2 different versions. Discovering the exact errors consisted of adding some debug output to mochiweb_http:headers/5 and mochiweb_http:request/2 in the _Other clauses. After talking with the mochiweb devs on the mochiweb group, I'll probably submit a patch there to deal with the tcp_error, but it would be interesting to know what caused the behavior change. Looking forward to any results you might get, -Anthony On Wed, Dec 12, 2012 at 08:35:11PM -0500, Steve Vinoski wrote: > On Wed, Dec 12, 2012 at 7:24 PM, Anthony Molinaro < > anthonym@REDACTED> wrote: > > > Okay, I found it. Looks like this is the cause > > > > OTP-9389 Honor option packet_size for http packet parsing by both TCP > > socket and erlang:decode_packet. This gives the ability to > > accept HTTP headers larger than the default setting, but also > > avoid DoS attacks by accepting lines only up to > > whatever length you wish to allow. For consistency, > > packet type line also honor option packet_size. (Thanks > > to Steve Vinoski) > > > > With R14B04 when you have a header which is too long, your process > > gets a message like > > > > > > {http,#Port<0.120387186>,{http_error,"X2qutbTBbXg7VHgDDhGrEvDbzuxiyDlI7VFloMyAJKVqY2fTEkMc70UchLPRG8Cjowzmib4KszbCRwA5IBbAd2MbRi5X_tK2nfRtheavXdhQv8XbinzmhCM1E9YCeuFAg_9TfqUS0sWUd52mgjkWGqNe4Z9S0IxFYnFtf5..."}} > > > > in R15B02 you get a message like > > > > {tcp_error,#Port<0.104208233>,emsgsize} > > > > The prior case is handled correctly by mochiweb, the latter is not. > > > > > I just ran a simple test case against the commit prior to 5984409d and then > against 5984409d, in both cases sending an HTTP header of various lengths, > and in both cases always received http_error, and never tcp_error. The > server simply did a receive with the socket in http_bin mode. > > How long is the header in your case? Do you have a test case you can send > me? > > > I'm not sure if the commits for OTP-9389 actually caused this change or if > > it was some other change. Also, not sure if it was meant to be a backward > > compatible change or not (the comment on the commit > > > > https://github.com/erlang/otp/commit/5984409d1264871cbe61bfec875de53e51713efb > > seems to suggest it was supposed to be backward compatible, but maybe this > > was a side effect? It seems like emsgsize is more correct. > > > > > I think the intent was to be backward compatible; I seem to remember that > putting the patch together was very difficult and time-consuming due to > trying to achieve that, with numerous iterations and reviews. > > --steve > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ------------------------------------------------------------------------ Anthony Molinaro -------------- next part -------------- -module(mochi_test). -export([start/0, stop/0, handle_http/1, test/1]). start() -> application:start (inets), mochiweb_http:start([{port, 5678}, {loop, fun(Req) -> handle_http(Req) end}]). stop() -> mochiweb_http:stop(). handle_http(Req) -> Req:respond({ 200, [ {"Content-Type", "text/html"} ], [ "Hello" ] }). test (Len) -> Random = get_random_string (Len, "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"), httpc:request (get, {"http://127.0.0.1:5678/", [{"X-Random", Random}]}, [], []). % grabbed from % http://blog.teemu.im/2009/11/07/generating-random-strings-in-erlang/ get_random_string(Length, AllowedChars) -> lists:foldl(fun(_, Acc) -> [lists:nth(random:uniform(length(AllowedChars)), AllowedChars)] ++ Acc end, [], lists:seq(1, Length)). From desired.mta@REDACTED Thu Dec 13 19:51:05 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Thu, 13 Dec 2012 19:51:05 +0100 Subject: [erlang-questions] "official" pre-built rebar binaries In-Reply-To: References: Message-ID: On Thu, Dec 13, 2012 at 6:22 PM, Tuncer Ayaz wrote: > On Thu, Dec 13, 2012 at 5:59 PM, Motiejus Jakstys wrote: > >> This is better. But how does the file get there? I cannot find how to >> add a file to the github wiki. > > https://github.com/rebar/rebar/blob/71c717d8/Makefile#L30-L33 Oh, didn't realize that. Thanks. So we have a way, now need (?) to agree on the rules. Any more people believe it is necessary? -- Motiejus Jak?tys From robert.virding@REDACTED Thu Dec 13 23:04:48 2012 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 13 Dec 2012 22:04:48 +0000 (GMT) Subject: [erlang-questions] noob namespace question In-Reply-To: Message-ID: <1417867152.2405756.1355436288454.JavaMail.root@erlang-solutions.com> Yes, Erlang is definitely much closer to Lisp-2 than Lisp-1, especially as you can functions with the same name and different number of arguments (arity) and they are different functions. Don't stretch the analogy too far though. It is NOT that the functions are properties of the name (atom). To uniquely identify a function you need its module name, function name and arity, often written a M:F/A. Erlang doesn't have packages in the CL sense, there is only only one atom space containing all atoms. Robert ----- Original Message ----- > From: "Rustom Mody" > To: "Thomas Allen" > Cc: "Erlang Users' List" > Sent: Wednesday, 12 December, 2012 7:40:26 PM > Subject: Re: [erlang-questions] noob namespace question > On Thu, Dec 13, 2012 at 12:00 AM, Thomas Allen < thomas@REDACTED > > wrote: > > On Wed, Dec 12, 2012 at 11:44:43PM +0530, Rustom Mody wrote: > > > > I dont understand. Let me try and be more explicit about my > > > concern. > > > > > > > > Let h be a higher-order function of one argument (h/1) > > > > > > > > Now in the call h(foo) > > > > How does Erlang know whether foo is a function or an atom? > > > This is not valid, and you will get a "bad function" error if you > > try > > to > > > pass your function this way. You must create a "fun" like so: > > > h(fun foo/0) > > > Similarly, you could pass an anonymous fun (like fun(X) -> X end), > > or > > > a module-function combination (like fun lists:reverse/1). > > Thanks Thomas that answers succinctly. > In Lisp terminology Erlang is more like a Lisp-2 than a Lisp-1 > http://en.wikipedia.org/wiki/Common_Lisp#The_function_namespace > -- > http://www.the-magus.in > http://blog.languager.org > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Dec 14 03:16:59 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 14 Dec 2012 15:16:59 +1300 Subject: [erlang-questions] noob namespace question In-Reply-To: References: <50C8C639.6060006@ninenines.eu> Message-ID: On 13/12/2012, at 7:25 PM, Gianfranco Alongi wrote: > By implication - what I will write has been said. > But I will just spend a second reinforcing this and dispelling any misconceptions. > > They can - if you force it into an atom. I am confused by the pronoun references here. "They" refers to what things? "can" refers to what action? "it" refers to what thing? In the text of mine that was quoted, the claim was that an atom cannot be a function and it cannot be a function name all by itself either. This is true, and has nothing whatever to do with quotation marks. > > -module(capsFuncs). > -export(['Run'/1]). > 'Run'(this) -> > that. > > 1> c(capsFuncs). > {ok,capsFuncs} > 2> capsFuncs:'Run'(this). > that The only thing this illustrates is that a quoted atom can be used as part of a function name, and as far as I could tell, nobody ever saidotherwise. This example does not show an atom being a function or being a function name all by itself. The only function in sight is the one whose name is the triple capsFuncs:'Run'/1, which may be abbreviated to 'Run'/1 within the capsFuncs module. IIRC, the original poster said he knew something about Lisp, so the fact that (defun |Run| (arg) 'that) is legal Lisp would have prepared him to _expect_ quoted atoms to be usable as (parts of) function names. From 7stud@REDACTED Fri Dec 14 03:29:27 2012 From: 7stud@REDACTED (7stud) Date: Thu, 13 Dec 2012 21:29:27 -0500 Subject: [erlang-questions] bit syntax Message-ID: <20121213212927.15862@web008.roc2.bluetie.com> -----Original Message----- From: "Dmitry Klionsky" [dm.klionsky@REDACTED] Date: 11/30/2012 03:26 AM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] bit syntax Hi! All below comes from http://www.erlang.org/doc/programming_examples/bit_syntax.html ... Value:Size ... The Size part of the segment multiplied by the unit in the TypeSpecifierList (described below) gives the number of bits for the segment. In construction, Size is any expression that evaluates to an integer. In matching, Size must be a constant expression or a variable. ... ============= Hi, Thanks for the explanation. What is the reason for allowing a variable but not an expression containing a variable? ===END=== And (2*8) is a legal Size: 7> <> = X. <<2,1,0>> This works because (2*8) really evaluates at compile time and it equals to <> = X. 12> <> = X. * 1: illegal bit size The expression (Length*8) is supported when constructing a binary. This is definitely a pattern matching, so as stated above: the Size must be a constant expression or a variable. Not an expression to be evaluated at runtime. As a workaround you can do this > <> = X. > ValueLen = Length*8. > <> = Rest. 76> << [49, 50, 51, 52] >>. ** exception error: bad argument 77> X = "1234". "1234" 78> <>. ** exception error: bad argument The only way you can do this is list_to_binary([49, 50, 51, 52]). and list_to_binary("1234"). BR, Dmitry On 11/30/2012 09:20 AM, 7stud wrote: > Hi, > > I have a couple of questions about the bit syntax. > > 1) I can read the first byte of a binary to get the length of the next value: > > 1> X = <<16, 256:16>>. > <<16,1,0>> > > 2> <> = X. > <<16,1,0>> > > 3> Length. > 16 > > 4> Value. > 256 > > > > And (2*8) is a legal Size: > > 5> f(). > ok > > 6> X = <<2, 256:16>>. > <<2,1,0>> > > 7> <> = X. > <<2,1,0>> > > 8> Length. > 2 > > 9> Value. > 256 > > > So why does Size = Length*8 fail when Length=2? > > 10> f(Length). > ok > > 11> f(Value). > ok > > 12> <> = X. > * 1: illegal bit size > > > > > 2) What is going on here: > > > 73> f(). > ok > > 74> <<"1234">>. > <<"1234">> > > 75> "1234" == [49, 50, 51, 52]. > true > > 76> << [49, 50, 51, 52] >>. > ** exception error: bad argument > > 77> X = "1234". > "1234" > > 78> <>. > ** exception error: bad argument > > In line 74, I can use a string while constructing a binary, but in 76 I can't use the equivalent list. And in line 77, when I bind the string to a variable, I can't use the variable to construct a binary. > > Thanks. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Best regards, Dmitry Klionsky _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From 7stud@REDACTED Fri Dec 14 03:35:10 2012 From: 7stud@REDACTED (7stud) Date: Thu, 13 Dec 2012 21:35:10 -0500 Subject: [erlang-questions] bit syntax Message-ID: <20121213213510.6253@web002.roc2.bluetie.com> Hi, ========= -----Original Message----- From: "Erik S?e S?rensen" [eriksoe@REDACTED] Date: 11/30/2012 04:06 AM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] bit syntax As a workaround you can do this > <> = X. > ValueLen = Length*8. > <> = Rest. Or this: ?2> <> = X. <<2,1,0>> 3> {Length,Value}. {2,256} Multiplication of a length by a constant is a special case which is supported through the unit qualifier. ========== Thanks for the examples. =========== 76> << [49, 50, 51, 52] >>. ** exception error: bad argument 77> X = "1234". "1234" 78> <>. ** exception error: bad argument The only way you can do this is list_to_binary([49, 50, 51, 52]). and list_to_binary("1234"). ? The thing is that <<"ABC">> is *syntactic sugar* for <<65,66,67>>; this (literal strings) is a special case, and it is a *syntactic* special case ============== Okay, thanks. From gianfranco.alongi@REDACTED Fri Dec 14 08:20:54 2012 From: gianfranco.alongi@REDACTED (Gianfranco Alongi) Date: Fri, 14 Dec 2012 08:20:54 +0100 Subject: [erlang-questions] noob namespace question In-Reply-To: References: <50C8C639.6060006@ninenines.eu> Message-ID: I see your confusion - and I would as well be confused, for the email did not follow the tracks of your shoes. I replied to the last email in the discussion, but did not intend to quote you or follow the fashion. >From the previous emails in this thread, it seemed to be implied about function names, that they must only be in lowercase. But, as Max Lapshin ? (I think) - reminded us all, 'UpperCaseText' is an atom as well, thus it okay to StareAt:'TheWall'(). G On Fri, Dec 14, 2012 at 3:16 AM, Richard O'Keefe wrote: > > On 13/12/2012, at 7:25 PM, Gianfranco Alongi wrote: > > > By implication - what I will write has been said. > > But I will just spend a second reinforcing this and dispelling any > misconceptions. > > > > They can - if you force it into an atom. > > I am confused by the pronoun references here. > "They" refers to what things? > "can" refers to what action? > "it" refers to what thing? > > In the text of mine that was quoted, the claim was that > an atom cannot be a function and it cannot be a function name > all by itself either. > > This is true, and has nothing whatever to do with quotation marks. > > > > -module(capsFuncs). > > -export(['Run'/1]). > > 'Run'(this) -> > > that. > > > > 1> c(capsFuncs). > > {ok,capsFuncs} > > 2> capsFuncs:'Run'(this). > > that > > The only thing this illustrates is that a quoted atom can be used > as part of a function name, and as far as I could tell, nobody ever > saidotherwise. This example does not show an atom being a function > or being a function name all by itself. The only function in > sight is the one whose name is the triple capsFuncs:'Run'/1, > which may be abbreviated to 'Run'/1 within the capsFuncs module. > > IIRC, the original poster said he knew something about Lisp, > so the fact that > (defun |Run| (arg) 'that) > is legal Lisp would have prepared him to _expect_ quoted atoms > to be usable as (parts of) function names. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Fri Dec 14 08:23:19 2012 From: bourinov@REDACTED (Max Bourinov) Date: Fri, 14 Dec 2012 11:23:19 +0400 Subject: [erlang-questions] noob namespace question In-Reply-To: References: <50C8C639.6060006@ninenines.eu> Message-ID: That was Max Bourinov. We have at least 3 Maxs in Russia that do Erlang: Max Lapshin, Max Treskin and Max Bourinov. On Fri, Dec 14, 2012 at 11:20 AM, Gianfranco Alongi < gianfranco.alongi@REDACTED> wrote: > I see your confusion - and I would as well be confused, > for the email did not follow the tracks of your shoes. > > I replied to the last email in the discussion, > but did not intend to quote you or follow the fashion. > > From the previous emails in this thread, > it seemed to be implied about function names, > that they must only be in lowercase. > > But, as Max Lapshin ? (I think) - reminded us all, > 'UpperCaseText' is an atom as well, > thus it okay to StareAt:'TheWall'(). > > G > > > On Fri, Dec 14, 2012 at 3:16 AM, Richard O'Keefe wrote: > >> >> On 13/12/2012, at 7:25 PM, Gianfranco Alongi wrote: >> >> > By implication - what I will write has been said. >> > But I will just spend a second reinforcing this and dispelling any >> misconceptions. >> > >> > They can - if you force it into an atom. >> >> I am confused by the pronoun references here. >> "They" refers to what things? >> "can" refers to what action? >> "it" refers to what thing? >> >> In the text of mine that was quoted, the claim was that >> an atom cannot be a function and it cannot be a function name >> all by itself either. >> >> This is true, and has nothing whatever to do with quotation marks. >> > >> > -module(capsFuncs). >> > -export(['Run'/1]). >> > 'Run'(this) -> >> > that. >> > >> > 1> c(capsFuncs). >> > {ok,capsFuncs} >> > 2> capsFuncs:'Run'(this). >> > that >> >> The only thing this illustrates is that a quoted atom can be used >> as part of a function name, and as far as I could tell, nobody ever >> saidotherwise. This example does not show an atom being a function >> or being a function name all by itself. The only function in >> sight is the one whose name is the triple capsFuncs:'Run'/1, >> which may be abbreviated to 'Run'/1 within the capsFuncs module. >> >> IIRC, the original poster said he knew something about Lisp, >> so the fact that >> (defun |Run| (arg) 'that) >> is legal Lisp would have prepared him to _expect_ quoted atoms >> to be usable as (parts of) function names. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Fri Dec 14 08:51:19 2012 From: bourinov@REDACTED (Max Bourinov) Date: Fri, 14 Dec 2012 11:51:19 +0400 Subject: [erlang-questions] Erlang Process Notation? Message-ID: Hi Erlangers, I am wondering if there is any notation for Erlang processes? I want to use it to describe my systems visually. Something like UML class diagrams but for Erlang (I know that class diagram is not the best example). p.s.: I heavily use sequence diagrams and they help a lot. Best regards, Max -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Fri Dec 14 09:07:31 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Fri, 14 Dec 2012 10:07:31 +0200 Subject: [erlang-questions] Erlang Process Notation? In-Reply-To: References: Message-ID: Hello, Would pi-calculus suite you needs? another lighter approach is use UML sequence or collaboration diagrams. - Dmitry On Dec 14, 2012, at 9:51 AM, Max Bourinov wrote: > Hi Erlangers, > > I am wondering if there is any notation for Erlang processes? I want to use it to describe my systems visually. Something like UML class diagrams but for Erlang (I know that class diagram is not the best example). > > p.s.: I heavily use sequence diagrams and they help a lot. > > Best regards, > Max > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bourinov@REDACTED Fri Dec 14 09:20:15 2012 From: bourinov@REDACTED (Max Bourinov) Date: Fri, 14 Dec 2012 12:20:15 +0400 Subject: [erlang-questions] Erlang Process Notation? In-Reply-To: References: Message-ID: Hi Dmitry, Thank you for you reply. Could you please provide a link to proper pi-calculus description, because all I found in google is something that won't help me. Best regards, Max On Fri, Dec 14, 2012 at 12:07 PM, Dmitry Kolesnikov wrote: > pi-calculus -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri Dec 14 09:28:48 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 14 Dec 2012 09:28:48 +0100 Subject: [erlang-questions] Erlang Process Notation? In-Reply-To: References: Message-ID: <7974B8AF-B17F-464B-BE4C-59090B790169@feuerlabs.com> A notation that might serve you reasonably well is FMC http://fmc-modeling.org/ It seems to be reasonably aligned with mainstream and have decent traction. This paper might work as a a starting point (I have only skimmed it): "A System of Patterns for Concurrent Request Processing Servers" Bernhard Gr?ne, Peter Tabeling Hasso-Plattner-Institute for Software Systems Engineering http://www.fmc-modeling.org/download/publications/groene_tabeling_2003-server_patterns.pdf BR, Ulf W On 14 Dec 2012, at 08:51, Max Bourinov wrote: > Hi Erlangers, > > I am wondering if there is any notation for Erlang processes? I want to use it to describe my systems visually. Something like UML class diagrams but for Erlang (I know that class diagram is not the best example). > > p.s.: I heavily use sequence diagrams and they help a lot. > > Best regards, > Max > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Fri Dec 14 10:50:11 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 14 Dec 2012 10:50:11 +0100 Subject: [erlang-questions] Erlang Process Notation? In-Reply-To: References: Message-ID: <50CAF653.70700@gmail.com> Hi Max, You might want to have a look at Object Process Methodology: https://en.wikipedia.org/wiki/Object_Process_Methodology The thing that makes OPM stand out is that it makes both objects and processes (not the same meaning as in Erlang) first class citizens. This allows for a very natural description of how data is processed and where the functionality should be located. I use it as a way to depict the high level architecture and data flow more than anything else. It is sufficiently light weight to be done and maintained and expressive enough to have conversations about what is going on. If you want to model an Erlang process with OPM it would come out as an object (sounds weird, but stay with me) that exhibits a number of processes (where a process is an API function of the process). I have not tried the tool that has been developed for OPM - I made my own stencil for Dia. Other nice features of OPM: * there is a 1-to-1 mapping between the graphical representation and its textual one. * you can zoom in and out in your model - not seen with UML or anything else. But you would have to get the OPM tool or write some tool yourself to get the most out of these things. Cheers, ___ /orben On 2012-12-14 08:51, Max Bourinov wrote: > Hi Erlangers, > > I am wondering if there is any notation for Erlang processes? I want > to use it to describe my systems visually. Something like UML class > diagrams but for Erlang (I know that class diagram is not the best > example). > > p.s.: I heavily use sequence diagrams and they help a lot. > > Best regards, > Max > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Fri Dec 14 10:59:57 2012 From: bourinov@REDACTED (Max Bourinov) Date: Fri, 14 Dec 2012 13:59:57 +0400 Subject: [erlang-questions] Erlang Process Notation? In-Reply-To: <50CAF653.70700@gmail.com> References: <50CAF653.70700@gmail.com> Message-ID: Hi Torben, OPM looks promising. Thank you for sharing! Best regards, Max On Fri, Dec 14, 2012 at 1:50 PM, Torben Hoffmann wrote: > Hi Max, > > You might want to have a look at Object Process Methodology: > https://en.wikipedia.org/wiki/Object_Process_Methodology > > The thing that makes OPM stand out is that it makes both objects and > processes (not the same meaning as in Erlang) first class citizens. This > allows for a very natural description of how data is processed and where > the functionality should be located. > > I use it as a way to depict the high level architecture and data flow more > than anything else. It is sufficiently light weight to be done and > maintained and expressive enough to have conversations about what is going > on. > > If you want to model an Erlang process with OPM it would come out as an > object (sounds weird, but stay with me) that exhibits a number of processes > (where a process is an API function of the process). > > I have not tried the tool that has been developed for OPM - I made my own > stencil for Dia. > > Other nice features of OPM: > > - there is a 1-to-1 mapping between the graphical representation and > its textual one. > - you can zoom in and out in your model - not seen with UML or > anything else. > > But you would have to get the OPM tool or write some tool yourself to get > the most out of these things. > Cheers, > ___ > /orben > > > On 2012-12-14 08:51, Max Bourinov wrote: > > Hi Erlangers, > > I am wondering if there is any notation for Erlang processes? I want to > use it to describe my systems visually. Something like UML class diagrams > but for Erlang (I know that class diagram is not the best example). > > p.s.: I heavily use sequence diagrams and they help a lot. > > Best regards, > Max > > > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > -- http://www.linkedin.com/in/torbenhoffmann > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hobson42@REDACTED Fri Dec 14 11:26:30 2012 From: hobson42@REDACTED (Ian) Date: Fri, 14 Dec 2012 10:26:30 +0000 Subject: [erlang-questions] Finding erlang talent Message-ID: <50CAFED6.1020609@gmail.com> Hi all, I have a client with a project that is crying out to be written in erlang. However I'm a one man band, and the client wants to have a "way out" should I fall under the proverbial bus. If you wanted to find a UK based person to assist/take over a freelance project written in Erlang, how would you go about doing it? Regards Ian From naikvin@REDACTED Fri Dec 14 11:35:55 2012 From: naikvin@REDACTED (Vineet Naik) Date: Fri, 14 Dec 2012 16:05:55 +0530 Subject: [erlang-questions] Deploying on a server with different architecture and version of Erlang Message-ID: Hello, Now that I have successfully built a release of my app using rebar, I am stuck at deploying the packaged release because the server I want to deploy to has different version of erlang and different architecture from my development machine. Development Env ------------------------- i686 i686 i386 GNU/Linux Erlang R14B04 erts-5.8.5 Remote Server ---------------------- x86_64 GNU/Linux Erlang R13B03 erts-5.7.4 I tried compiling on the server but it's failing with a series of errors such as ERROR: Unable to generate spec: Mandatory application kernel is not included in [{app,crypto,false,undefined, "/usr/lib/erlang/lib/crypto-1.6.3", ["/usr/lib/erlang/lib/crypto-1.6.3"], "1.6.3","crypto-1.6.3", ..... I have two questions: 1. Is this due to the old version of erlang? And considering that I have ejabberd and rabbitmq-server running in production on the same server can I upgrade erlang to R14B04 on it without having to take them down for much time? 2. What is the recommended strategy for developing and deploying erlang code on differently configured servers? Thanks, Vineet -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.joseph.davis@REDACTED Fri Dec 14 12:26:58 2012 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Fri, 14 Dec 2012 05:26:58 -0600 Subject: [erlang-questions] bif_return_trap Message-ID: Preemptive apologies that this turned into a bit of a novel... I've been working on debugging a cluster of three Erlang nodes for a couple days and I've run up against some interesting data points that I can't quite figure out how to investigate much further. For background, I'm running R14B01 on three nodes with one of the three nodes in a remote data center that's about 40ms away from the other two which are <1ms apart. What I'm observing is that the remote node ends up accumulating processes stuck in erlang:bif_return_trap/1 which eventually accumulate to the point where the node exhausts RAM and the node reboots (if I let it go that long). Each process stuck in bif_return_trap is related to distributed message passing. My biggest question is what would be holding processes in this function. From the Googling and source browsing I've been doing it looks as though it has to do with reductions and scheduling but I can't quite cover the last bit of ground in connecting how that explains the behavior I'm seeing. The obvious theory is if this is an IO related issue over a long distance link then perhaps I'm just seeing saturation of the network. Though watching Boundary graphs I can see that the link is not at all saturated. I'm running around 200Mbps over a 1Gps link. There's spikes above that as well that suggest its not just a busy link. This email has gotten a bit long, so I'll just enumerate a number of questions and call it quits: 1. What exactly could bif_return_trap be waiting on? 2. How might reductions affect bif_return_trap? Also, links of note that I keep seeing in my Googling: http://www.snookles.com/slf-blog/2012/01/05/tcp-incast-what-is-it/ http://erlang.org/pipermail/erlang-bugs/2010-May/001806.html Neither of which have an obvious resolution. Thanks, Paul Davis From paulperegud@REDACTED Fri Dec 14 13:40:04 2012 From: paulperegud@REDACTED (Paul Peregud) Date: Fri, 14 Dec 2012 13:40:04 +0100 Subject: [erlang-questions] simple_one_for_one supervisor, transient restart strategy In-Reply-To: <1CAB695D2C2A8F4BB0B242A5B44C75E901E6FF@ESESSMB301.ericsson.se> References: <1CAB695D2C2A8F4BB0B242A5B44C75E901E6FF@ESESSMB301.ericsson.se> Message-ID: I've distilled the code to minimal form. And the error is no longer there. So it seems that I misdiagnosed the source of the problem. Anyway - here is the code: test case: http://pastebin.com/80JbEE0k supervisor: http://pastebin.com/kyy7pnjq worker: http://pastebin.com/q5QQ3WM3 On Thu, Dec 13, 2012 at 4:06 PM, Arif Ishaq wrote: > > Hi, > > I have tried this with R15B and it works for me. > > In practice, I have the child implemented as a gen_server which handles messages stopnormal, stopshutdown and stopabort treated with {stop, normal, State}, {stop, shutdown, State} and {stop, abort, State} respectively. When I send the messages stopnormal or stopshutdown, the child dies without getting restarted. When I send a stopabort, it gets restarted with the arguments with which it was created. > > Can you please post your code to see what went wrong? > > Regards > Arif > > > -----Original Message----- > From: Paul Peregud [mailto:paulperegud@REDACTED] > Sent: mercoled? 12 dicembre 2012 16.20 > To: erlang-questions@REDACTED > Subject: [erlang-questions] simple_one_for_one supervisor, transient restart strategy > > I've searched mailing list for this question and found nothing. > > There is this one peculiar behavior when you use simple_one_for_one with transient strategy. You do supervisor:start_child(Ref, Args), the child executes and if it dies gracefully everything is fine, child is removed from supervisor tree. However, if it fails, it is restarted WITHOUT Args. > > This combination of type/strategy would be a great tool for fire-and-forget tasks if it was not for that small issue. > > Is it a bug or is it a feature? > > -- > Best regards, > Paul Peregud > +48602112091 > -- Best regards, Paul Peregud +48602112091 From sarva.v@REDACTED Fri Dec 14 14:39:07 2012 From: sarva.v@REDACTED (Saravanan Vijayakumaran) Date: Fri, 14 Dec 2012 19:09:07 +0530 Subject: [erlang-questions] gen_server bottleneck Message-ID: Hi all, I have been trying to write a network simulator in Erlang modelled on ns-3. The code can be found at https://github.com/avras/nsime NSIME is a discrete event simulator where a registered gen_server process nsime_simulator holds all the events to be simulated in a gb_trees sorted by event time. This process has become a bottlneck when the simulation is large as multiple entities try to schedule events by doing a call on the single nsime_simulator process. I am still a newbie in Erlang profiling so this is more of a hunch arising from running the example scenarios. How can I remove this bottleneck? Thanks in advance sarva -------------- next part -------------- An HTML attachment was scrubbed... URL: From naikvin@REDACTED Fri Dec 14 14:41:55 2012 From: naikvin@REDACTED (Vineet Naik) Date: Fri, 14 Dec 2012 19:11:55 +0530 Subject: [erlang-questions] Deploying on a server with different architecture and version of Erlang In-Reply-To: References: Message-ID: Now I got past this error, it was a mistake from my side. I didn't create the node on the server before running generate but copied files from my dev machine. After creating node and with some tweaking to the reltool config, release is generated on the prod server but fails on startup with, {"init terminating in do_boot",{'cannot load',error_handler,get_file}} >From erlang docs[1] "Init terminating in do_boot ()" - The primitive Erlang boot sequence was terminated, most probably because the boot script has errors or cannot be read. This is usually a configuration error - the system may have been started with a faulty -boot parameter or with a boot script from the wrong version of OTP. How can I check whether boot script running from the wrong version of OTP is the problem? [1]: http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 Regards, Vineet On Fri, Dec 14, 2012 at 4:05 PM, Vineet Naik wrote: > Hello, > > Now that I have successfully built a release of my app using rebar, I am > stuck at > deploying the packaged release because the server I want to deploy to has > different > version of erlang and different architecture from my development machine. > > Development Env > ------------------------- > i686 i686 i386 GNU/Linux > Erlang R14B04 > erts-5.8.5 > > Remote Server > ---------------------- > x86_64 GNU/Linux > Erlang R13B03 > erts-5.7.4 > > I tried compiling on the server but it's failing with a series of errors > such as > > ERROR: Unable to generate spec: Mandatory application kernel is not > included in [{app,crypto,false,undefined, > > "/usr/lib/erlang/lib/crypto-1.6.3", > > ["/usr/lib/erlang/lib/crypto-1.6.3"], > "1.6.3","crypto-1.6.3", > ..... > > I have two questions: > > 1. Is this due to the old version of erlang? And considering that I have > ejabberd and rabbitmq-server running in production on the same server > can I upgrade erlang to R14B04 on it without having to take them down for > much > time? > > 2. What is the recommended strategy for developing and deploying erlang > code on > differently configured servers? > > Thanks, > Vineet > > > -- Vineet Naik -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel@REDACTED Fri Dec 14 15:30:51 2012 From: daniel@REDACTED (Daniel Luna) Date: Fri, 14 Dec 2012 09:30:51 -0500 Subject: [erlang-questions] Finding erlang talent In-Reply-To: <50CAFED6.1020609@gmail.com> References: <50CAFED6.1020609@gmail.com> Message-ID: On 14 December 2012 05:26, Ian wrote: > If you wanted to find a UK based person to assist/take over a freelance > project written in Erlang, how would you go about doing it? In order of preference from my point of view: Grow a personal network of Erlang people (takes time) The Erlang groups on LinkedIn Randomly searching for people with Erlang experience on LinkedIn Go to a couple of London Erlang User Group meetings and start asking around Advertise on job boards Reach out to individual developers on this list (google them to find out if they are UK based) Find a good recruiter I guess the obvious one is that you probably already have a message or two from people looking for a job right now. :-) Cheers, Daniel From g@REDACTED Fri Dec 14 15:47:00 2012 From: g@REDACTED (Garrett Smith) Date: Fri, 14 Dec 2012 08:47:00 -0600 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: Hi Saravanan, On Fri, Dec 14, 2012 at 7:39 AM, Saravanan Vijayakumaran wrote: > Hi all, > I have been trying to write a network simulator in Erlang modelled > on ns-3. The code can be found at https://github.com/avras/nsime > > NSIME is a discrete event simulator where a registered gen_server process > nsime_simulator holds all the events to be simulated in a gb_trees sorted by > event time. This process has become a bottlneck when the simulation is large > as multiple entities try to schedule events by doing a call on the single > nsime_simulator process. I am still a newbie in Erlang profiling so this is > more of a hunch arising from running the example scenarios. Single registered processes are a potential bottleneck, so your hunch is right. But measurement will both confirm this and help you understand Erlang application architecture. Erlang provides a dazzling array of diagnostic tools to help track down process related problems. But I'd start with etop: http://erlang.org/doc/man/etop.html The registered process in question should show up at the top and let you know what's spiking, growing, etc. You can run your various stress tests and watch etop to get some idea what's going on with that process (and others). > How can I remove this bottleneck? Once you understand the bottleneck, you can start to think about this. If you're bottlenecking on CPU (all your cores are fully utilized at peak load) then you need either a faster machine or you'll need to distribute your application to multiple machines. I don't know enough about Erlang's SMP support these days to give you any advice about core utilization. There are plenty of experts here. Distributing the processing to multiple processes (e.g. one per core) *might* give you a speedup but I don't know. If the bottleneck is that your single process is accumulating messages over time, you'll need to look at either slowing down senders (e.g. rate limiting) or speeding up message handling (e.g. spill to disk, drop messages, speed up processing). Or maybe this isn't a problem -- the queue may grow and then come back to zero eventually. Judicious use of ETS might give you some options for addressing problems. I'd start with measurement and go from there. Garrett From g@REDACTED Fri Dec 14 15:51:05 2012 From: g@REDACTED (Garrett Smith) Date: Fri, 14 Dec 2012 08:51:05 -0600 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: On Fri, Dec 14, 2012 at 8:47 AM, Garrett Smith wrote: > Hi Saravanan, > If you're bottlenecking on CPU (all your cores are fully utilized at > peak load) then you need either a faster machine or you'll need to > distribute your application to multiple machines. I should add there a number of ways you can improve efficiency, short of adding hardware resources. The big win, once you understand what to target, is C ports (or NIFs). Garrett From naikvin@REDACTED Fri Dec 14 15:54:01 2012 From: naikvin@REDACTED (Vineet Naik) Date: Fri, 14 Dec 2012 20:24:01 +0530 Subject: [erlang-questions] Deploying on a server with different architecture and version of Erlang In-Reply-To: References: Message-ID: IRC to the rescue! Changed mod_cond from derived to all and it solved the problem. Thanks a lot Vineet On Fri, Dec 14, 2012 at 7:11 PM, Vineet Naik wrote: > Now I got past this error, it was a mistake from my side. I didn't create > the node on the server before running generate but copied files from my > dev machine. After creating node and with some tweaking to the reltool > config, > release is generated on the prod server but fails on startup with, > > {"init terminating in do_boot",{'cannot load',error_handler,get_file}} > > From erlang docs[1] > > "Init terminating in do_boot ()" - The primitive Erlang boot sequence was > terminated, most probably because the boot script has errors or cannot be > read. This is usually a configuration error - the system may have been > started with a faulty -boot parameter or with a boot script from the wrong > version of OTP. > > How can I check whether boot script running from the wrong version of OTP > is > the problem? > > [1]: http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 > > Regards, > Vineet > > On Fri, Dec 14, 2012 at 4:05 PM, Vineet Naik wrote: > >> Hello, >> >> Now that I have successfully built a release of my app using rebar, I am >> stuck at >> deploying the packaged release because the server I want to deploy to has >> different >> version of erlang and different architecture from my development machine. >> >> Development Env >> ------------------------- >> i686 i686 i386 GNU/Linux >> Erlang R14B04 >> erts-5.8.5 >> >> Remote Server >> ---------------------- >> x86_64 GNU/Linux >> Erlang R13B03 >> erts-5.7.4 >> >> I tried compiling on the server but it's failing with a series of errors >> such as >> >> ERROR: Unable to generate spec: Mandatory application kernel is not >> included in [{app,crypto,false,undefined, >> >> "/usr/lib/erlang/lib/crypto-1.6.3", >> >> ["/usr/lib/erlang/lib/crypto-1.6.3"], >> "1.6.3","crypto-1.6.3", >> ..... >> >> I have two questions: >> >> 1. Is this due to the old version of erlang? And considering that I have >> ejabberd and rabbitmq-server running in production on the same server >> can I upgrade erlang to R14B04 on it without having to take them down for >> much >> time? >> >> 2. What is the recommended strategy for developing and deploying erlang >> code on >> differently configured servers? >> >> Thanks, >> Vineet >> >> >> > > > -- > Vineet Naik > > > -- Vineet Naik -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.majadas@REDACTED Fri Dec 14 16:57:04 2012 From: roberto.majadas@REDACTED (Roberto Majadas Lopez) Date: Fri, 14 Dec 2012 16:57:04 +0100 Subject: [erlang-questions] Redirecting stdout in erlang Message-ID: Hello, I want to redirect all io:format outputs to one gen_server or similar. How can i do it in erlang? Roberto -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel@REDACTED Fri Dec 14 17:05:01 2012 From: daniel@REDACTED (Daniel Luna) Date: Fri, 14 Dec 2012 11:05:01 -0500 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: On 14 December 2012 09:51, Garrett Smith wrote: > On Fri, Dec 14, 2012 at 8:47 AM, Garrett Smith wrote: >> Hi Saravanan, >> If you're bottlenecking on CPU (all your cores are fully utilized at >> peak load) then you need either a faster machine or you'll need to >> distribute your application to multiple machines. > > I should add there a number of ways you can improve efficiency, short > of adding hardware resources. The big win, once you understand what to > target, is C ports (or NIFs). But long before you start looking at either NIFs or new hardware, look into the complexity of the code itself. Does that expensive function even have to be called in all cases, etc. I guess this is my pet peeve when it comes to optimizing anything. First you optimize for readability (often gaining speed or at least finding issues) Then you measure Then you optimize the hotspots you've discovered by measuring *Then* you can start looking into hardware or non-Erlang solutions I've also seen situations where minor changes in the requirements have seen the possibility to speed up code by a factor 10 so that's also a possibility. Cheers, Daniel From daniel@REDACTED Fri Dec 14 17:09:14 2012 From: daniel@REDACTED (Daniel Luna) Date: Fri, 14 Dec 2012 11:09:14 -0500 Subject: [erlang-questions] Deploying on a server with different architecture and version of Erlang In-Reply-To: References: Message-ID: If this fixed your problem you have missing dependencies in your .app.src file. We always run {mod_cond, derived} here and it works like a charm. Until you miss to add a dependency in the .app.src file that is. Cheers, Daniel On 14 December 2012 09:54, Vineet Naik wrote: > IRC to the rescue! Changed mod_cond from derived to all > and it solved the problem. > > Thanks a lot > Vineet > > > On Fri, Dec 14, 2012 at 7:11 PM, Vineet Naik wrote: >> >> Now I got past this error, it was a mistake from my side. I didn't create >> the node on the server before running generate but copied files from my >> dev machine. After creating node and with some tweaking to the reltool >> config, >> release is generated on the prod server but fails on startup with, >> >> {"init terminating in do_boot",{'cannot load',error_handler,get_file}} >> >> From erlang docs[1] >> >> "Init terminating in do_boot ()" - The primitive Erlang boot sequence was >> terminated, most probably because the boot script has errors or cannot be >> read. This is usually a configuration error - the system may have been >> started with a faulty -boot parameter or with a boot script from the wrong >> version of OTP. >> >> How can I check whether boot script running from the wrong version of OTP >> is >> the problem? >> >> [1]: http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 >> >> Regards, >> Vineet >> >> On Fri, Dec 14, 2012 at 4:05 PM, Vineet Naik wrote: >>> >>> Hello, >>> >>> Now that I have successfully built a release of my app using rebar, I am >>> stuck at >>> deploying the packaged release because the server I want to deploy to has >>> different >>> version of erlang and different architecture from my development machine. >>> >>> Development Env >>> ------------------------- >>> i686 i686 i386 GNU/Linux >>> Erlang R14B04 >>> erts-5.8.5 >>> >>> Remote Server >>> ---------------------- >>> x86_64 GNU/Linux >>> Erlang R13B03 >>> erts-5.7.4 >>> >>> I tried compiling on the server but it's failing with a series of errors >>> such as >>> >>> ERROR: Unable to generate spec: Mandatory application kernel is not >>> included in [{app,crypto,false,undefined, >>> >>> "/usr/lib/erlang/lib/crypto-1.6.3", >>> >>> ["/usr/lib/erlang/lib/crypto-1.6.3"], >>> "1.6.3","crypto-1.6.3", >>> ..... >>> >>> I have two questions: >>> >>> 1. Is this due to the old version of erlang? And considering that I have >>> ejabberd and rabbitmq-server running in production on the same server >>> can I upgrade erlang to R14B04 on it without having to take them down for >>> much >>> time? >>> >>> 2. What is the recommended strategy for developing and deploying erlang >>> code on >>> differently configured servers? >>> >>> Thanks, >>> Vineet >>> >>> >> >> >> >> -- >> Vineet Naik >> >> > > > > -- > Vineet Naik > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From vinoski@REDACTED Fri Dec 14 17:32:11 2012 From: vinoski@REDACTED (Steve Vinoski) Date: Fri, 14 Dec 2012 11:32:11 -0500 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? In-Reply-To: <20121213183830.GA81730@alumni.caltech.edu> References: <20121206193809.GC27411@alumni.caltech.edu> <15C0D662-A77C-4858-AD93-3306E72317F6@erlang-solutions.com> <20121212182632.GB72594@alumni.caltech.edu> <20121212234803.GA75710@alumni.caltech.edu> <20121213002455.GA75861@alumni.caltech.edu> <20121213183830.GA81730@alumni.caltech.edu> Message-ID: On Thu, Dec 13, 2012 at 1:38 PM, Anthony Molinaro < anthonym@REDACTED> wrote: > Hi Steve, > > It may not be directly caused by your patch, but that was the first > patch that stuck out when I started looking through the readmes for the > releases. I've not been able to untangle mochiweb enough to do a test > without it, so attached is a test file which uses mochiweb. > > If you compile it, you should see this with R14B04 > > Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.5 (abort with ^G) > 1> mochi_test:start(). > {ok,<0.44.0>} > 2> mochi_test:test(10000). > {ok,{{"HTTP/1.1",400,"Bad Request"}, > [{"date","Thu, 13 Dec 2012 18:36:06 GMT"}, > {"server","MochiWeb/1.0 (Any of you quaids got a smint?)"}, > {"content-length","0"}], > []}} > 3> > > and this with R15B02 > > Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:2:2] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.9.2 (abort with ^G) > 1> mochi_test:start(). > {ok,<0.44.0>} > 2> mochi_test:test(10000). > {error,socket_closed_remotely} > 3> > > I used Mochiweb 2.3.1 and compiled it and the test code with R14B04, then > ran the test with the 2 different versions. Discovering the exact errors > consisted of adding some debug output to mochiweb_http:headers/5 and > mochiweb_http:request/2 in the _Other clauses. > > After talking with the mochiweb devs on the mochiweb group, I'll probably > submit a patch there to deal with the tcp_error, but it would be > interesting > to know what caused the behavior change. > > Looking forward to any results you might get, Hi Anthony, based on your example, it looks like a follow-up commit that modified some of the changes introduced by my patch introduced the issue you're seeing: commit dc5f7190. You might want to report it over on erlang-bugs. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.majadas@REDACTED Fri Dec 14 17:36:58 2012 From: roberto.majadas@REDACTED (Roberto Majadas Lopez) Date: Fri, 14 Dec 2012 17:36:58 +0100 Subject: [erlang-questions] Redirecting stdout in erlang In-Reply-To: References: Message-ID: No i don't need a logger system. I need a way to redirect all standard output strings written with io:format, including from other source code. 2012/12/14 Yu-ri Cho > Maybe lager is what you want > https://github.com/basho/lager > > On Fri, Dec 14, 2012 at 10:57 AM, Roberto Majadas Lopez < > roberto.majadas@REDACTED> wrote: > >> >> Hello, >> >> I want to redirect all io:format outputs to one gen_server or similar. >> How can i do it in erlang? >> >> Roberto >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Yu-ri Gordon > trueEX Group LLC > 646-786-8520 > > > *CONFIDENTIALITY NOTICE* > > THE INFORMATION CONTAINED IN THIS COMMUNICATION IS INTENDED FOR THE NAMED > RECEIVER ONLY. THE TRANSMISSION MAY CONTAIN PRIVILEGED AND CONFIDENTIAL > MATERIAL. IF YOU ARE NOT THE NAMED RECIPIENT, PLEASE BE ADVISED THAT ANY > USE, DISSEMINATION OR UNAUTHORIZED COPYING OF THE MATERIAL IS STRICTLY > PROHIBITED. IF YOU HAVE RECEIVED THIS TRANSMISSION IN ERROR, PLEASE NOTIFY > SUPPORT@REDACTED AND DESTROY THE RECEIVED COPY. THANK YOU. > -- Roberto Majadas OpenShine S.L email : roberto.majadas@REDACTED tlf : +34 663 273 501 www.openshine.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Fri Dec 14 17:48:33 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Fri, 14 Dec 2012 17:48:33 +0100 Subject: [erlang-questions] Redirecting stdout in erlang In-Reply-To: References: Message-ID: Hello! I think you have to implement an "io_server" process and set it with erlang:group_leader/2. I think this is the relevant documentation: http://erlang.org/doc/apps/stdlib/io_protocol.html 2012/12/14 Roberto Majadas Lopez : > No i don't need a logger system. I need a way to redirect all standard > output strings written with io:format, including from other source code. > > > > 2012/12/14 Yu-ri Cho >> >> Maybe lager is what you want >> https://github.com/basho/lager >> >> On Fri, Dec 14, 2012 at 10:57 AM, Roberto Majadas Lopez >> wrote: >>> >>> >>> Hello, >>> >>> I want to redirect all io:format outputs to one gen_server or similar. >>> How can i do it in erlang? >>> >>> Roberto >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> >> -- >> Yu-ri Gordon >> trueEX Group LLC >> 646-786-8520 >> >> >> CONFIDENTIALITY NOTICE >> >> THE INFORMATION CONTAINED IN THIS COMMUNICATION IS INTENDED FOR THE NAMED >> RECEIVER ONLY. THE TRANSMISSION MAY CONTAIN PRIVILEGED AND CONFIDENTIAL >> MATERIAL. IF YOU ARE NOT THE NAMED RECIPIENT, PLEASE BE ADVISED THAT ANY >> USE, DISSEMINATION OR UNAUTHORIZED COPYING OF THE MATERIAL IS STRICTLY >> PROHIBITED. IF YOU HAVE RECEIVED THIS TRANSMISSION IN ERROR, PLEASE NOTIFY >> SUPPORT@REDACTED AND DESTROY THE RECEIVED COPY. THANK YOU. > > > > > -- > Roberto Majadas > OpenShine S.L > > email : roberto.majadas@REDACTED > tlf : +34 663 273 501 > > www.openshine.com > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From bourinov@REDACTED Fri Dec 14 18:00:37 2012 From: bourinov@REDACTED (Max Bourinov) Date: Fri, 14 Dec 2012 21:00:37 +0400 Subject: [erlang-questions] Ho to keep-alive inactive RabbitMQ connections? In-Reply-To: References: Message-ID: Hi Den, RabbitMQ works very well. My code also included auto-recovery functionality and some other useful stuff. Please describe your problem and I will try to provide you with detailed solution. Best regards, Max On Fri, Dec 14, 2012 at 8:17 PM, ????? None wrote: > Hi Max, > > Have you overcome the "[erlang-questions] Ho to keep-alive inactive > RabbitMQ connections?" issue. If so, kindly advise me on this problem. > > Thank you in advance. > > Best Regards, > Denis > -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivier.boudeville@REDACTED Fri Dec 14 18:02:46 2012 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Fri, 14 Dec 2012 18:02:46 +0100 Subject: [erlang-questions] RE gen_server bottleneck In-Reply-To: Message-ID: Hello Sarva, I suppose it depends on how scalable you want to be; I think the first approach is to lean first on your simulation algorithm; is the management of your discrete time event-driven or time-stepped? In either case you have solutions to implement a parallel evaluation of model instances with Erlang (for example, in the first case, with a conservative or optimistic scheduling), knowing that maintaining a centralised event queue on which the engine would iterate sequentially will induce by design immediate bottlenecks. Once the parallel algorithm is implemented (most probably making use of SMP resources), to obtain further scalability gains it is probably useful to split it into, say, a tree of time managers where communications between time managers could be a lot terser than between a time manager and the local model instances it would manage; this would open the way to a still more parallel mode of operation - and would immediately offer the opportunity for distribution, which might be efficient, at least to some extent. Finally I would go for a raise in the priority of the time manager processes and other key simulation services, efficient data-structures for scheduling, load balancing and smart placement of model instances to minimize the number of messages sent over the network and the induced latency. Then only I would go for fancy hardware, detailed profiling, native compilation, C-nodes and the like! At least it is the path we are going through with Sim-Diasca (which is an open-source generic discrete-time simulation engine, maybe not too far from what you intend to do, cf http://www.sim-diasca.com). In this process, we are currently just before the profiling. Hope this helps! Best regards, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 erlang-questions-bounces@REDACTED a ?crit sur 14/12/2012 14:39:07 : > Hi all, > I have been trying to write a network simulator in Erlang > modelled on ns-3. The code can be found at https://github.com/avras/nsime > > NSIME is a discrete event simulator where a registered gen_server > process nsime_simulator holds all the events to be simulated in a > gb_trees sorted by event time. This process has become a bottlneck > when the simulation is large as multiple entities try to schedule > events by doing a call on the single nsime_simulator process. I am > still a newbie in Erlang profiling so this is more of a hunch > arising from running the example scenarios. > > How can I remove this bottleneck? > > Thanks in advance > sarva_______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Fri Dec 14 18:08:37 2012 From: bourinov@REDACTED (Max Bourinov) Date: Fri, 14 Dec 2012 21:08:37 +0400 Subject: [erlang-questions] Ho to keep-alive inactive RabbitMQ connections? In-Reply-To: References: Message-ID: Hi Den, Here is my anonymized code from production: https://gist.github.com/6cc6099af6c2255282e8 See line: 86 erlang:monitor for connection recovery I hope it will help Best regards, Max On Fri, Dec 14, 2012 at 9:00 PM, Max Bourinov wrote: > Hi Den, > > RabbitMQ works very well. My code also included auto-recovery > functionality and some other useful stuff. Please describe your problem and > I will try to provide you with detailed solution. > > Best regards, > Max > > > > > On Fri, Dec 14, 2012 at 8:17 PM, ????? None wrote: > >> Hi Max, >> >> Have you overcome the "[erlang-questions] Ho to keep-alive inactive >> RabbitMQ connections?" issue. If so, kindly advise me on this problem. >> >> Thank you in advance. >> >> Best Regards, >> Denis >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From naikvin@REDACTED Fri Dec 14 18:09:57 2012 From: naikvin@REDACTED (Vineet Naik) Date: Fri, 14 Dec 2012 22:39:57 +0530 Subject: [erlang-questions] Deploying on a server with different architecture and version of Erlang In-Reply-To: References: Message-ID: On Fri, Dec 14, 2012 at 9:39 PM, Daniel Luna wrote: > If this fixed your problem you have missing dependencies in your > .app.src file. We always run {mod_cond, derived} here and it works > like a charm. > > Until you miss to add a dependency in the .app.src file that is. > Hi Daniel, I am am not able to exactly understand which dependencies are to be added in app.src. My code directly depends upon just two - emysql and exmpp (Not the processone version but this one [1] which is rebar compatible). Besides these, kernel and stdlib are defined. While creating a release on my development machine I could fix some initial errors by adding inets, sasl and crypto. That solved the problems with mod_cond still being 'derived' Today while creating a release on the server (where the app will be deployed eventually), it again failed with `{"init terminating in do_boot",{'cannot load',X,get_file}}` kind of errors and adding 'X' to app.src didn't work. Changing mod_cond to 'all' worked. Now if I check the libraries in rel//lib directory, there are a lot of them but not all are added to app.src and it still works. How to decide which ones go in app.src and which don't? [1]: Rebar compatible fork of exmpp https://github.com/Zert/exmpp Thanks, Vineet > Cheers, > > Daniel > > On 14 December 2012 09:54, Vineet Naik wrote: > > IRC to the rescue! Changed mod_cond from derived to all > > and it solved the problem. > > > > Thanks a lot > > Vineet > > > > > > On Fri, Dec 14, 2012 at 7:11 PM, Vineet Naik wrote: > >> > >> Now I got past this error, it was a mistake from my side. I didn't > create > >> the node on the server before running generate but copied files from my > >> dev machine. After creating node and with some tweaking to the reltool > >> config, > >> release is generated on the prod server but fails on startup with, > >> > >> {"init terminating in do_boot",{'cannot load',error_handler,get_file}} > >> > >> From erlang docs[1] > >> > >> "Init terminating in do_boot ()" - The primitive Erlang boot sequence > was > >> terminated, most probably because the boot script has errors or cannot > be > >> read. This is usually a configuration error - the system may have been > >> started with a faulty -boot parameter or with a boot script from the > wrong > >> version of OTP. > >> > >> How can I check whether boot script running from the wrong version of > OTP > >> is > >> the problem? > >> > >> [1]: http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 > >> > >> Regards, > >> Vineet > >> > >> On Fri, Dec 14, 2012 at 4:05 PM, Vineet Naik wrote: > >>> > >>> Hello, > >>> > >>> Now that I have successfully built a release of my app using rebar, I > am > >>> stuck at > >>> deploying the packaged release because the server I want to deploy to > has > >>> different > >>> version of erlang and different architecture from my development > machine. > >>> > >>> Development Env > >>> ------------------------- > >>> i686 i686 i386 GNU/Linux > >>> Erlang R14B04 > >>> erts-5.8.5 > >>> > >>> Remote Server > >>> ---------------------- > >>> x86_64 GNU/Linux > >>> Erlang R13B03 > >>> erts-5.7.4 > >>> > >>> I tried compiling on the server but it's failing with a series of > errors > >>> such as > >>> > >>> ERROR: Unable to generate spec: Mandatory application kernel is not > >>> included in [{app,crypto,false,undefined, > >>> > >>> "/usr/lib/erlang/lib/crypto-1.6.3", > >>> > >>> ["/usr/lib/erlang/lib/crypto-1.6.3"], > >>> > "1.6.3","crypto-1.6.3", > >>> ..... > >>> > >>> I have two questions: > >>> > >>> 1. Is this due to the old version of erlang? And considering that I > have > >>> ejabberd and rabbitmq-server running in production on the same server > >>> can I upgrade erlang to R14B04 on it without having to take them down > for > >>> much > >>> time? > >>> > >>> 2. What is the recommended strategy for developing and deploying erlang > >>> code on > >>> differently configured servers? > >>> > >>> Thanks, > >>> Vineet > >>> > >>> > >> > >> > >> > >> -- > >> Vineet Naik > >> > >> > > > > > > > > -- > > Vineet Naik > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -- Vineet Naik -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Fri Dec 14 18:15:53 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 14 Dec 2012 17:15:53 +0000 Subject: [erlang-questions] Ho to keep-alive inactive RabbitMQ connections? In-Reply-To: References: Message-ID: <58E21B49-16FE-4DF8-822A-73AEC52FF997@gmail.com> Rabbit connections can be set to use AMQP heartbeats, which provide something similar to keep-alive but also let's the broker deal explicitly with disappearing clients efficiently. There is the issue of automatically reconnecting, which the native (erlang) client doesn't do. There are several libraries that add similar support, one of which is written by Max from the sounds of things. This can be quite involved if you're using exclusive/auto-delete objects so I'd suggest looking at some of the tools out there before rollin your own, even if it's just instructional to do so. Do you have code shared anywhere Max? If so we could add it to the tools page on rabbitmq.com, unless it's there already? let me know. Cheers Tim Staff Engineer RabbitMQ/VMWare On 14 Dec 2012, at 17:00, Max Bourinov wrote: > Hi Den, > > RabbitMQ works very well. My code also included auto-recovery functionality and some other useful stuff. Please describe your problem and I will try to provide you with detailed solution. > > Best regards, > Max > > > > > On Fri, Dec 14, 2012 at 8:17 PM, ????? None wrote: > Hi Max, > > Have you overcome the "[erlang-questions] Ho to keep-alive inactive RabbitMQ connections?" issue. If so, kindly advise me on this problem. > > Thank you in advance. > > Best Regards, > Denis > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Fri Dec 14 18:30:21 2012 From: bourinov@REDACTED (Max Bourinov) Date: Fri, 14 Dec 2012 21:30:21 +0400 Subject: [erlang-questions] Ho to keep-alive inactive RabbitMQ connections? In-Reply-To: <58E21B49-16FE-4DF8-822A-73AEC52FF997@gmail.com> References: <58E21B49-16FE-4DF8-822A-73AEC52FF997@gmail.com> Message-ID: Hi Tim, Here is another a little bit optimized example of RabbitMQ connection monitoring: https://gist.github.com/4287139 Line 69: add erlang:monitor Lines 43, 47: Here you get 'DOWN' from failed link Line 118: Here I restart the link. (of course it could be erlang:send_after/3, but since link is already down timer:sleep/1 also works.) I hope it will help somehow to the great team of RabbitMQ Best regards, Max On Fri, Dec 14, 2012 at 9:15 PM, Tim Watson wrote: > Rabbit connections can be set to use AMQP heartbeats, which provide > something similar to keep-alive but also let's the broker deal explicitly > with disappearing clients efficiently. > > There is the issue of automatically reconnecting, which the native > (erlang) client doesn't do. There are several libraries that add similar > support, one of which is written by Max from the sounds of things. This can > be quite involved if you're using exclusive/auto-delete objects so I'd > suggest looking at some of the tools out there before rollin your own, even > if it's just instructional to do so. > > Do you have code shared anywhere Max? If so we could add it to the tools > page on rabbitmq.com, unless it's there already? let me know. > > Cheers > > Tim > Staff Engineer > RabbitMQ/VMWare > > On 14 Dec 2012, at 17:00, Max Bourinov wrote: > > Hi Den, > > RabbitMQ works very well. My code also included auto-recovery > functionality and some other useful stuff. Please describe your problem and > I will try to provide you with detailed solution. > > Best regards, > Max > > > > > On Fri, Dec 14, 2012 at 8:17 PM, ????? None wrote: > >> Hi Max, >> >> Have you overcome the "[erlang-questions] Ho to keep-alive inactive >> RabbitMQ connections?" issue. If so, kindly advise me on this problem. >> >> Thank you in advance. >> >> Best Regards, >> Denis >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Fri Dec 14 20:20:47 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 14 Dec 2012 19:20:47 +0000 Subject: [erlang-questions] Ho to keep-alive inactive RabbitMQ connections? In-Reply-To: References: <58E21B49-16FE-4DF8-822A-73AEC52FF997@gmail.com> Message-ID: Hi max, thanks for sharing. Have you looked at bunny farm and projects like that? There's some good stuff out there. On 14 Dec 2012, at 17:30, Max Bourinov wrote: > Line 118: Here I restart the link. (of course it could be erlang:send_after/3, but since link is already down timer:sleep/1 also works.) > The stop return tuple causes the gen server to halt, so presumably you stick this under a supervisor? From sarva.v@REDACTED Fri Dec 14 20:21:02 2012 From: sarva.v@REDACTED (Saravanan Vijayakumaran) Date: Sat, 15 Dec 2012 00:51:02 +0530 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: Dear Garrett, Daniel and Olivier, Thanks for your responses. I will try your suggestions. @Olivier: Thanks for sending the Sim-Diasca sources and docs earlier. The ns-3 simulator which is the basis for NSIME is discrete event. Since Sim-Diasca was discrete time, I put it on the backburner and haven't got a chance to check it out. @All: Let me elaborate a little on the bottleneck I am seeing. The simulation scenario consists of 10,000 UDP echo client server pairs with a pairwise point to point connection between them. Each UDP echo client sends 10 packets to a unique UDP server with a inter-packet time of 1s. Each UDP server replies to every packet it receives. In terms of the simulation, the 10k UDP echo clients each schedule a packet send event using a gen_server:call on nsime_simulator. When the simulation is run, the send events are executed resulting in receive events being scheduled at the UDP echo servers. The receive events are executed resulting in reply events being scheduled. Every event being scheduled is a gen_server:call on the nsime_simulator process. The parallelism I am trying to exploit right now is in the form of simultaneous events (events having same time stamp). All the 10k send events are simultaneous, the 10k receives are simultaneous as the point to point connections between each pair has the same delay and data rate and the 10k reply events are simultaneous. The parallel execution of the simultaneous events is done using the following code fragment from src/nsime_simulator.erl. The gen_server:call returns a list of simultaneous events and Stephen Marsh's plists module ( https://github.com/rmies/plists) is used to execute each event (an MFA triple) in the list by dividing the list among 5 processes. This was for a quad core machine. The plists code suggests using number of cores + 1. Each time an event is executed it might result in another event being scheduled which itself is a gen_server:call to nsime_simulator. parallel_run() -> case gen_server:call(?MODULE, parallel_run) of {events, EventList} -> plists:foreach( fun(Event) -> erlang:apply( Event#nsime_event.module, Event#nsime_event.function, Event#nsime_event.arguments ) end, EventList, {processes, 5} ), ?MODULE:parallel_run(); none -> simulation_complete end. On a quad core machine, the ns-3 C++ code for this example runs in 59 seconds while occupying only one core. The NSIME code runs in 65 seconds while occupying about 300% out of a maximum of 400%. That doesn't sound too bad. Another non-trivial C++ vs Erlang benchmark, one can argue. But the NSIME code which does not execute the simultaneous events in parallel runs in 79 seconds while occupying only one core . So I am seeing sublinear speedups. The even more surprising thing is that on a 32 core machine the parallel_run runs in 68 seconds when the simultaneous event list is divided among 32 processes. Since the 32 core and quad core had different CPU speeds I ran the simulation on the 32 core machine with the simultaneous event list divided among 5 processes and that took 66 seconds. So I actually saw a slowdown when I use more cores. When I divided the event list among 32 processes, only 30-40% of each core was being utilized (as seen in htop). I hope my description makes some sense. My gut feeling is that the single nsime_simulator is the bottleneck. I will try to confirm it using the Erlang profiling tools. I was hoping pg or pg2 can provide a solution by distributing the message handling workload. I could schedule events in a randomly chosen process in the process group and then collect the earliest events from all the process group members. But to choose a random process I may have to use pg2:get_closest_pid which itself may cause a bottleneck. sarva On Fri, Dec 14, 2012 at 9:35 PM, Daniel Luna wrote: > On 14 December 2012 09:51, Garrett Smith wrote: > > On Fri, Dec 14, 2012 at 8:47 AM, Garrett Smith wrote: > >> Hi Saravanan, > >> If you're bottlenecking on CPU (all your cores are fully utilized at > >> peak load) then you need either a faster machine or you'll need to > >> distribute your application to multiple machines. > > > > I should add there a number of ways you can improve efficiency, short > > of adding hardware resources. The big win, once you understand what to > > target, is C ports (or NIFs). > > But long before you start looking at either NIFs or new hardware, look > into the complexity of the code itself. Does that expensive function > even have to be called in all cases, etc. > > I guess this is my pet peeve when it comes to optimizing anything. > > First you optimize for readability (often gaining speed or at least > finding issues) > Then you measure > Then you optimize the hotspots you've discovered by measuring > *Then* you can start looking into hardware or non-Erlang solutions > > I've also seen situations where minor changes in the requirements have > seen the possibility to speed up code by a factor 10 so that's also a > possibility. > > Cheers, > > Daniel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Fri Dec 14 21:12:26 2012 From: bourinov@REDACTED (Max Bourinov) Date: Sat, 15 Dec 2012 00:12:26 +0400 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: Hi guys, Saravanan, maybe I am missing the point, but why not to create 10.000 clients and servers processes? In the code you provided I don't see much process creation... Described load is relatively small and you should not see any performance issues with this amount of UDP traffic. p.s.: I am not familiar with discrete system simulation, so maybe my view is absolutely wrong)) Best regards, Max On Fri, Dec 14, 2012 at 11:21 PM, Saravanan Vijayakumaran wrote: > Dear Garrett, Daniel and Olivier, > Thanks for your responses. I > will try your suggestions. > > @Olivier: Thanks for sending the Sim-Diasca sources and docs earlier. The > ns-3 simulator which is the basis for NSIME is discrete event. Since > Sim-Diasca was discrete time, I put it on the backburner and haven't got a > chance to check it out. > > @All: Let me elaborate a little on the bottleneck I am seeing. > > The simulation scenario consists of 10,000 UDP echo client server pairs > with a pairwise point to point connection between them. Each UDP echo > client sends 10 packets to a unique UDP server with a inter-packet time of > 1s. Each UDP server replies to every packet it receives. In terms of the > simulation, the 10k UDP echo clients each schedule a packet send event > using a gen_server:call on nsime_simulator. When the simulation is run, the > send events are executed resulting in receive events being scheduled at the > UDP echo servers. The receive events are executed resulting in reply events > being scheduled. Every event being scheduled is a gen_server:call on the > nsime_simulator process. > > The parallelism I am trying to exploit right now is in the form of > simultaneous events (events having same time stamp). All the 10k send > events are simultaneous, the 10k receives are simultaneous as the point to > point connections between each pair has the same delay and data rate and > the 10k reply events are simultaneous. > > The parallel execution of the simultaneous events is done using the > following code fragment from src/nsime_simulator.erl. The gen_server:call > returns a list of simultaneous events and Stephen Marsh's plists module ( > https://github.com/rmies/plists) is used to execute each event (an MFA > triple) in the list by dividing the list among 5 processes. This was for a > quad core machine. The plists code suggests using number of cores + 1. > Each time an event is executed it might result in another event being > scheduled which itself is a gen_server:call to nsime_simulator. > > parallel_run() -> > case gen_server:call(?MODULE, parallel_run) of > {events, EventList} -> > plists:foreach( > fun(Event) -> > erlang:apply( > Event#nsime_event.module, > Event#nsime_event.function, > Event#nsime_event.arguments > ) > > end, > EventList, > {processes, 5} > > > ), > ?MODULE:parallel_run(); > none -> > simulation_complete > end. > > > On a quad core machine, the ns-3 C++ code for this example runs in 59 > seconds while occupying only one core. The NSIME code runs in 65 seconds > while occupying about 300% out of a maximum of 400%. That doesn't sound too > bad. Another non-trivial C++ vs Erlang benchmark, one can argue. But the > NSIME code which does not execute the simultaneous events in parallel runs > in 79 seconds while occupying only one core . So I am seeing sublinear > speedups. > > The even more surprising thing is that on a 32 core machine the > parallel_run runs in 68 seconds when the simultaneous event list is divided > among 32 processes. Since the 32 core and quad core had different CPU > speeds I ran the simulation on the 32 core machine with the simultaneous > event list divided among 5 processes and that took 66 seconds. So I > actually saw a slowdown when I use more cores. When I divided the event > list among 32 processes, only 30-40% of each core was being utilized (as > seen in htop). > > I hope my description makes some sense. My gut feeling is that the single > nsime_simulator is the bottleneck. I will try to confirm it using the > Erlang profiling tools. I was hoping pg or pg2 can provide a solution by > distributing the message handling workload. I could schedule events in a > randomly chosen process in the process group and then collect the earliest > events from all the process group members. But to choose a random process I > may have to use pg2:get_closest_pid which itself may cause a bottleneck. > > sarva > > > > > On Fri, Dec 14, 2012 at 9:35 PM, Daniel Luna wrote: > >> On 14 December 2012 09:51, Garrett Smith wrote: >> > On Fri, Dec 14, 2012 at 8:47 AM, Garrett Smith wrote: >> >> Hi Saravanan, >> >> If you're bottlenecking on CPU (all your cores are fully utilized at >> >> peak load) then you need either a faster machine or you'll need to >> >> distribute your application to multiple machines. >> > >> > I should add there a number of ways you can improve efficiency, short >> > of adding hardware resources. The big win, once you understand what to >> > target, is C ports (or NIFs). >> >> But long before you start looking at either NIFs or new hardware, look >> into the complexity of the code itself. Does that expensive function >> even have to be called in all cases, etc. >> >> I guess this is my pet peeve when it comes to optimizing anything. >> >> First you optimize for readability (often gaining speed or at least >> finding issues) >> Then you measure >> Then you optimize the hotspots you've discovered by measuring >> *Then* you can start looking into hardware or non-Erlang solutions >> >> I've also seen situations where minor changes in the requirements have >> seen the possibility to speed up code by a factor 10 so that's also a >> possibility. >> >> Cheers, >> >> Daniel >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Fri Dec 14 21:21:45 2012 From: bourinov@REDACTED (Max Bourinov) Date: Sat, 15 Dec 2012 00:21:45 +0400 Subject: [erlang-questions] Ho to keep-alive inactive RabbitMQ connections? In-Reply-To: References: <58E21B49-16FE-4DF8-822A-73AEC52FF997@gmail.com> Message-ID: Thank you telling about bunny_farm - I will give it a try! Best regards, Max On Fri, Dec 14, 2012 at 11:20 PM, Tim Watson wrote: > bunny farm -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Dec 14 22:01:44 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Fri, 14 Dec 2012 22:01:44 +0100 Subject: [erlang-questions] process manager, R16, and 64-bit Macs In-Reply-To: References: <50BE791E.1050202@simonstl.com> <50BF1EA5.3040201@erlang.org> <50BF30A6.3030608@erlang.org> <50BF33FC.5080608@ninenines.eu> Message-ID: <50CB93B8.1070203@ninenines.eu> On 12/05/2012 01:17 PM, Joan Valduvieco Llopart wrote: > I am using observer with mountain lion with wxgtk + xquartz without a > glitch. :) > I have a macports repo with my packages. > https://github.com/jvalduvieco/macports Any chance you could port this to homebrew? From what I hear most people who use Macs use homebrew so you doing this would be a tremendous help for the community and for the OTP team. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From anthonym@REDACTED Fri Dec 14 22:11:17 2012 From: anthonym@REDACTED (Anthony Molinaro) Date: Fri, 14 Dec 2012 13:11:17 -0800 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? In-Reply-To: References: <20121206193809.GC27411@alumni.caltech.edu> <15C0D662-A77C-4858-AD93-3306E72317F6@erlang-solutions.com> <20121212182632.GB72594@alumni.caltech.edu> <20121212234803.GA75710@alumni.caltech.edu> <20121213002455.GA75861@alumni.caltech.edu> <20121213183830.GA81730@alumni.caltech.edu> Message-ID: <20121214211117.GA86501@alumni.caltech.edu> On Fri, Dec 14, 2012 at 11:32:11AM -0500, Steve Vinoski wrote: > Hi Anthony, based on your example, it looks like a follow-up commit that > modified some of the changes introduced by my patch introduced the issue > you're seeing: commit dc5f7190. You might want to report it over on > erlang-bugs. Nice catch, I guess that it's actually not enough to return error for backward compatibility ;) I think the main thing I'm still not sure of is if the decode_packet changes are actually the cause. I was able to call decode_packet with long headers and not actually see an issue. This erlang:decode_packet ( httph, list_to_binary ([ "X-Random: ", [$a || _ <- lists:seq(1,10000)], "\r\n\r\n" ]), []). Returns the same thing in both R14B04 and R15B02. And as far as I am able to tell that is the only call made to decode_packet in mochiweb. I think the actual error is around recvbuf sizing (mochiweb uses 8192 as the default, so 10000 is definitely larger than that, however, the standard libraries must be doing something different as now it returns emsgsize where it did no in the past). I really wish there was a way to search commits on github as then I could search for recvbuf or something like that to see what might have changed. Before I go over to bugs I'll probably try to detangle mochiweb enough to create a small reproducible test case. I also have a pull request into mochiweb which prevents the crash but has the annoying behavior at the moment that it returns an error to the client about disconnecting version the 400 error it returned before. https://github.com/mochi/mochiweb/pull/91 But since the standard library seems to be the one closing the socket there's not much that can be done other than the fix I put in place. Thanks for your help Steve, -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From anders.nygren@REDACTED Fri Dec 14 22:33:31 2012 From: anders.nygren@REDACTED (Anders Nygren) Date: Fri, 14 Dec 2012 15:33:31 -0600 Subject: [erlang-questions] Deploying on a server with different architecture and version of Erlang In-Reply-To: References: Message-ID: On Fri, Dec 14, 2012 at 11:09 AM, Vineet Naik wrote: > On Fri, Dec 14, 2012 at 9:39 PM, Daniel Luna wrote: >> >> If this fixed your problem you have missing dependencies in your >> .app.src file. We always run {mod_cond, derived} here and it works >> like a charm. >> >> Until you miss to add a dependency in the .app.src file that is. > > > Hi Daniel, > > I am am not able to exactly understand which dependencies are to be added in > app.src. > > My code directly depends upon just two - emysql and exmpp (Not the > processone version > but this one [1] which is rebar compatible). Besides these, kernel and > stdlib are defined. > > While creating a release on my development machine I could fix some initial > errors by > adding inets, sasl and crypto. That solved the problems with mod_cond still > being 'derived' > > Today while creating a release on the server (where the app will be deployed > eventually), > it again failed with `{"init terminating in do_boot",{'cannot > load',X,get_file}}` kind of errors > and adding 'X' to app.src didn't work. Changing mod_cond to 'all' worked. > > Now if I check the libraries in rel//lib directory, there > are a lot of them > but not all are added to app.src and it still works. > > How to decide which ones go in app.src and which don't? > The ones to put in the .app.src are the applications that MUST be started* before Your app is started. This information is used when the .boot script is created, so that all the applications are started in the correct order. *, Yes, I know the documentation says that stdlib shall always be included even though stdlib doesn't have any processes that needs to be started. > [1]: Rebar compatible fork of exmpp https://github.com/Zert/exmpp > > Thanks, > Vineet > >> >> Cheers, >> >> Daniel >> >> On 14 December 2012 09:54, Vineet Naik wrote: >> > IRC to the rescue! Changed mod_cond from derived to all >> > and it solved the problem. >> > >> > Thanks a lot >> > Vineet >> > >> > >> > On Fri, Dec 14, 2012 at 7:11 PM, Vineet Naik wrote: >> >> >> >> Now I got past this error, it was a mistake from my side. I didn't >> >> create >> >> the node on the server before running generate but copied files from my >> >> dev machine. After creating node and with some tweaking to the reltool >> >> config, >> >> release is generated on the prod server but fails on startup with, >> >> >> >> {"init terminating in do_boot",{'cannot load',error_handler,get_file}} >> >> >> >> From erlang docs[1] >> >> >> >> "Init terminating in do_boot ()" - The primitive Erlang boot sequence >> >> was >> >> terminated, most probably because the boot script has errors or cannot >> >> be >> >> read. This is usually a configuration error - the system may have been >> >> started with a faulty -boot parameter or with a boot script from the >> >> wrong >> >> version of OTP. >> >> >> >> How can I check whether boot script running from the wrong version of >> >> OTP >> >> is >> >> the problem? >> >> >> >> [1]: http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 >> >> >> >> Regards, >> >> Vineet >> >> >> >> On Fri, Dec 14, 2012 at 4:05 PM, Vineet Naik wrote: >> >>> >> >>> Hello, >> >>> >> >>> Now that I have successfully built a release of my app using rebar, I >> >>> am >> >>> stuck at >> >>> deploying the packaged release because the server I want to deploy to >> >>> has >> >>> different >> >>> version of erlang and different architecture from my development >> >>> machine. >> >>> >> >>> Development Env >> >>> ------------------------- >> >>> i686 i686 i386 GNU/Linux >> >>> Erlang R14B04 >> >>> erts-5.8.5 >> >>> >> >>> Remote Server >> >>> ---------------------- >> >>> x86_64 GNU/Linux >> >>> Erlang R13B03 >> >>> erts-5.7.4 >> >>> >> >>> I tried compiling on the server but it's failing with a series of >> >>> errors >> >>> such as >> >>> >> >>> ERROR: Unable to generate spec: Mandatory application kernel is not >> >>> included in [{app,crypto,false,undefined, >> >>> >> >>> "/usr/lib/erlang/lib/crypto-1.6.3", >> >>> >> >>> ["/usr/lib/erlang/lib/crypto-1.6.3"], >> >>> >> >>> "1.6.3","crypto-1.6.3", >> >>> ..... >> >>> >> >>> I have two questions: >> >>> >> >>> 1. Is this due to the old version of erlang? And considering that I >> >>> have >> >>> ejabberd and rabbitmq-server running in production on the same server >> >>> can I upgrade erlang to R14B04 on it without having to take them down >> >>> for >> >>> much >> >>> time? >> >>> >> >>> 2. What is the recommended strategy for developing and deploying >> >>> erlang >> >>> code on >> >>> differently configured servers? >> >>> >> >>> Thanks, >> >>> Vineet >> >>> >> >>> >> >> >> >> >> >> >> >> -- >> >> Vineet Naik >> >> >> >> >> > >> > >> > >> > -- >> > Vineet Naik >> > >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > > > > > > -- > Vineet Naik > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From comptekki@REDACTED Fri Dec 14 23:16:47 2012 From: comptekki@REDACTED (Wes James) Date: Fri, 14 Dec 2012 15:16:47 -0700 Subject: [erlang-questions] erlsrv.html info Message-ID: Looking at this documentation to see where .erlang.cookie might be saved for a service (installed with erlsrv) so I don't have to do -cookie on the command line, the docs at: http://www.erlang.org/doc/man/erlsrv.html say that the default working directory is: %SystemDrive%%SystemPath% There is a %SystemDrive% in win 7 (type set in the cmd.exe window) but there is no %SystemPath%. There is a %windir% and %SystemRoot% which are c:\windows on my system. Anyway, I don't see .erlang.cookie in c:\windows after using erlsrv and starting the service. Where is .erlang.cookie for erlsrv services? Thanks, wes -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Sat Dec 15 01:30:16 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Sat, 15 Dec 2012 01:30:16 +0100 Subject: [erlang-questions] =?iso-8859-1?q?Identity_and_Access_Management_?= =?iso-8859-1?q?in_Erlang_=28=E0_la_Amazon_AWS=29?= Message-ID: <1A4A1D39-B7D9-4204-8D78-23B0EE51613C@gmail.com> Hi guys, I'm looking for an "Identity and Access Management" Erlang API similar to what Amazon offers around the AWS services. With AWS, you (only) need: AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY The idea is to offer our customers a secure access to a bunch of services (similar to AWS). Any Erlang project which already handle this? Hints on how implement such an API? Help much appreciated. Regards, /Z From kaiduanx@REDACTED Sat Dec 15 02:04:34 2012 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Fri, 14 Dec 2012 20:04:34 -0500 Subject: [erlang-questions] SSL decrypt error during SSL handshake Message-ID: Hi, I ran into a situation where server sends back SSL decrypt error to client during SSL handshake, both client and server are written in Erlang. The SSL handshake looks as below, 1) Client sends Client Hello to server 2) Server sends Server Hello back 3) Server sends Certificate, Server Key Exchange and Server Hello Done 4) Client sends Client Key Exchange 5) Client sends Change Cipher Spec, Encrypted Handshake Message 6) Server sends Alert (Level: Fatal, Description: Decrypt Error) The certificate is from godaddy, any idea why server sends Decrypt Error? The server is running R14B01 while the client is running R15B02. Thanks, /Kaiduan From sarva.v@REDACTED Sat Dec 15 03:22:12 2012 From: sarva.v@REDACTED (Saravanan Vijayakumaran) Date: Sat, 15 Dec 2012 07:52:12 +0530 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: Dear Max, There are in fact 10,000 client and server processes. Each of them is a gen_server [1][2]. These are just simulation entities which are connected by point-to-point channels [3] which are also a simulation entities. The purpose of network simulation is to guide decision making with respect to choice of network protocols and parameters in a particular scenario. The scenarios are typically too expensive to try out in real networks. Imagine a thousand node network placed in different cities. Regarding discrete event simulation, suppose we want to figure out how many runways to have in a new airport. The more the runways the more costly it is but it is also will result in less waiting time for arriving/departing flights. One way to characterize the runway count vs waiting time tradeoff is to use queueing theory. An alternative is to do simulate flight arrivals and departures. At any point of time, the simulated world will consists of flights in transit, flights waiting to depart, flights waiting to land, flights departing and flights landing. In discrete time simulation, the state of the simulated world is updated at a regular time step (say every minute). Note that this is simulated time and not wall-clock time. An alternative which is much faster to execute is discrete event simulation. Here there is a queue of simulation events with the earliest events at the head of the queue. The events are executed one by one by picking the event at the head of the queue. When a flight departure event is executed, a waiting to land event is inserted into the queue having a timestamp equal to current time + the transit time of the flight. When the waiting to land event is executed, if a runway is free a land event is inserted into the queue having timestamp equal to the current time + landing time. If a runway is not free, a circle the airport and check again event is inserted into the queue having timestamp equal to the current time + circling time. The purpose of the simulation queue is to provide causal ordering of the simulation events. In my case, I think it is the simulation queue which is becoming the bottleneck. I must add that network simulation falls under the category of number-crunching applications which Erlang is not supposed to be suited for. But people have been trying to accelerate it in C++-based simulators using pthreads. I thought it was worth a try in Erlang given the amount of work which has already gone into Erlang SMP. Regards sarva [1] https://github.com/avras/nsime/blob/master/src/nsime_udp_echo_client.erl [2] https://github.com/avras/nsime/blob/master/src/nsime_udp_echo_server.erl [3] https://github.com/avras/nsime/blob/master/src/nsime_ptp_channel.erl On Sat, Dec 15, 2012 at 1:42 AM, Max Bourinov wrote: > Hi guys, > > Saravanan, maybe I am missing the point, but why not to create 10.000 > clients and servers processes? In the code you provided I don't see much > process creation... > > Described load is relatively small and you should not see any performance > issues with this amount of UDP traffic. > > p.s.: I am not familiar with discrete system simulation, so maybe my view > is absolutely wrong)) > > Best regards, > Max > > > > > On Fri, Dec 14, 2012 at 11:21 PM, Saravanan Vijayakumaran < > sarva.v@REDACTED> wrote: > >> Dear Garrett, Daniel and Olivier, >> Thanks for your responses. >> I will try your suggestions. >> >> @Olivier: Thanks for sending the Sim-Diasca sources and docs earlier. The >> ns-3 simulator which is the basis for NSIME is discrete event. Since >> Sim-Diasca was discrete time, I put it on the backburner and haven't got a >> chance to check it out. >> >> @All: Let me elaborate a little on the bottleneck I am seeing. >> >> The simulation scenario consists of 10,000 UDP echo client server pairs >> with a pairwise point to point connection between them. Each UDP echo >> client sends 10 packets to a unique UDP server with a inter-packet time of >> 1s. Each UDP server replies to every packet it receives. In terms of the >> simulation, the 10k UDP echo clients each schedule a packet send event >> using a gen_server:call on nsime_simulator. When the simulation is run, the >> send events are executed resulting in receive events being scheduled at the >> UDP echo servers. The receive events are executed resulting in reply events >> being scheduled. Every event being scheduled is a gen_server:call on the >> nsime_simulator process. >> >> The parallelism I am trying to exploit right now is in the form of >> simultaneous events (events having same time stamp). All the 10k send >> events are simultaneous, the 10k receives are simultaneous as the point to >> point connections between each pair has the same delay and data rate and >> the 10k reply events are simultaneous. >> >> The parallel execution of the simultaneous events is done using the >> following code fragment from src/nsime_simulator.erl. The gen_server:call >> returns a list of simultaneous events and Stephen Marsh's plists module ( >> https://github.com/rmies/plists) is used to execute each event (an MFA >> triple) in the list by dividing the list among 5 processes. This was for a >> quad core machine. The plists code suggests using number of cores + 1. >> Each time an event is executed it might result in another event being >> scheduled which itself is a gen_server:call to nsime_simulator. >> >> parallel_run() -> >> case gen_server:call(?MODULE, parallel_run) of >> {events, EventList} -> >> plists:foreach( >> fun(Event) -> >> erlang:apply( >> Event#nsime_event.module, >> Event#nsime_event.function, >> Event#nsime_event.arguments >> ) >> >> end, >> EventList, >> {processes, 5} >> >> >> >> ), >> ?MODULE:parallel_run(); >> none -> >> simulation_complete >> end. >> >> >> On a quad core machine, the ns-3 C++ code for this example runs in 59 >> seconds while occupying only one core. The NSIME code runs in 65 seconds >> while occupying about 300% out of a maximum of 400%. That doesn't sound too >> bad. Another non-trivial C++ vs Erlang benchmark, one can argue. But the >> NSIME code which does not execute the simultaneous events in parallel runs >> in 79 seconds while occupying only one core . So I am seeing sublinear >> speedups. >> >> The even more surprising thing is that on a 32 core machine the >> parallel_run runs in 68 seconds when the simultaneous event list is divided >> among 32 processes. Since the 32 core and quad core had different CPU >> speeds I ran the simulation on the 32 core machine with the simultaneous >> event list divided among 5 processes and that took 66 seconds. So I >> actually saw a slowdown when I use more cores. When I divided the event >> list among 32 processes, only 30-40% of each core was being utilized (as >> seen in htop). >> >> I hope my description makes some sense. My gut feeling is that the single >> nsime_simulator is the bottleneck. I will try to confirm it using the >> Erlang profiling tools. I was hoping pg or pg2 can provide a solution by >> distributing the message handling workload. I could schedule events in a >> randomly chosen process in the process group and then collect the earliest >> events from all the process group members. But to choose a random process I >> may have to use pg2:get_closest_pid which itself may cause a bottleneck. >> >> sarva >> >> >> >> >> On Fri, Dec 14, 2012 at 9:35 PM, Daniel Luna wrote: >> >>> On 14 December 2012 09:51, Garrett Smith wrote: >>> > On Fri, Dec 14, 2012 at 8:47 AM, Garrett Smith wrote: >>> >> Hi Saravanan, >>> >> If you're bottlenecking on CPU (all your cores are fully utilized at >>> >> peak load) then you need either a faster machine or you'll need to >>> >> distribute your application to multiple machines. >>> > >>> > I should add there a number of ways you can improve efficiency, short >>> > of adding hardware resources. The big win, once you understand what to >>> > target, is C ports (or NIFs). >>> >>> But long before you start looking at either NIFs or new hardware, look >>> into the complexity of the code itself. Does that expensive function >>> even have to be called in all cases, etc. >>> >>> I guess this is my pet peeve when it comes to optimizing anything. >>> >>> First you optimize for readability (often gaining speed or at least >>> finding issues) >>> Then you measure >>> Then you optimize the hotspots you've discovered by measuring >>> *Then* you can start looking into hardware or non-Erlang solutions >>> >>> I've also seen situations where minor changes in the requirements have >>> seen the possibility to speed up code by a factor 10 so that's also a >>> possibility. >>> >>> Cheers, >>> >>> Daniel >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Sat Dec 15 07:30:57 2012 From: bourinov@REDACTED (Max Bourinov) Date: Sat, 15 Dec 2012 10:30:57 +0400 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: Hi Saravanan, Thank you for explanations. Please correct me if I am wrong: neglecting the fact that you have many processes you have to execute event one by one on one gen_server because ir requires by simulation approach. Is that correct? Best regards, Max On Sat, Dec 15, 2012 at 6:22 AM, Saravanan Vijayakumaran wrote: > Dear Max, > There are in fact 10,000 client and server processes. Each > of them is a gen_server [1][2]. These are just simulation entities which > are connected by point-to-point channels [3] which are also a simulation > entities. > > The purpose of network simulation is to guide decision making with respect > to choice of network protocols and parameters in a particular scenario. The > scenarios are typically too expensive to try out in real networks. Imagine > a thousand node network placed in different cities. > > Regarding discrete event simulation, suppose we want to figure out how > many runways to have in a new airport. The more the runways the more costly > it is but it is also will result in less waiting time for > arriving/departing flights. One way to characterize the runway count vs > waiting time tradeoff is to use queueing theory. An alternative is to do > simulate flight arrivals and departures. At any point of time, the > simulated world will consists of flights in transit, flights waiting to > depart, flights waiting to land, flights departing and flights landing. In > discrete time simulation, the state of the simulated world is updated at a > regular time step (say every minute). Note that this is simulated time and > not wall-clock time. An alternative which is much faster to execute is > discrete event simulation. Here there is a queue of simulation events with > the earliest events at the head of the queue. The events are executed one > by one by picking the event at the head of the queue. When a flight > departure event is executed, a waiting to land event is inserted into the > queue having a timestamp equal to current time + the transit time of the > flight. When the waiting to land event is executed, if a runway is free a > land event is inserted into the queue having timestamp equal to the current > time + landing time. If a runway is not free, a circle the airport and > check again event is inserted into the queue having timestamp equal to the > current time + circling time. The purpose of the simulation queue is to > provide causal ordering of the simulation events. > > In my case, I think it is the simulation queue which is becoming the > bottleneck. I must add that network simulation falls under the category of > number-crunching applications which Erlang is not supposed to be suited > for. But people have been trying to accelerate it in C++-based simulators > using pthreads. I thought it was worth a try in Erlang given the amount of > work which has already gone into Erlang SMP. > > Regards > sarva > > [1] > https://github.com/avras/nsime/blob/master/src/nsime_udp_echo_client.erl > [2] > https://github.com/avras/nsime/blob/master/src/nsime_udp_echo_server.erl > [3] https://github.com/avras/nsime/blob/master/src/nsime_ptp_channel.erl > > > > On Sat, Dec 15, 2012 at 1:42 AM, Max Bourinov wrote: > >> Hi guys, >> >> Saravanan, maybe I am missing the point, but why not to create 10.000 >> clients and servers processes? In the code you provided I don't see much >> process creation... >> >> Described load is relatively small and you should not see any performance >> issues with this amount of UDP traffic. >> >> p.s.: I am not familiar with discrete system simulation, so maybe my view >> is absolutely wrong)) >> >> Best regards, >> Max >> >> >> >> >> On Fri, Dec 14, 2012 at 11:21 PM, Saravanan Vijayakumaran < >> sarva.v@REDACTED> wrote: >> >>> Dear Garrett, Daniel and Olivier, >>> Thanks for your responses. >>> I will try your suggestions. >>> >>> @Olivier: Thanks for sending the Sim-Diasca sources and docs earlier. >>> The ns-3 simulator which is the basis for NSIME is discrete event. Since >>> Sim-Diasca was discrete time, I put it on the backburner and haven't got a >>> chance to check it out. >>> >>> @All: Let me elaborate a little on the bottleneck I am seeing. >>> >>> The simulation scenario consists of 10,000 UDP echo client server pairs >>> with a pairwise point to point connection between them. Each UDP echo >>> client sends 10 packets to a unique UDP server with a inter-packet time of >>> 1s. Each UDP server replies to every packet it receives. In terms of the >>> simulation, the 10k UDP echo clients each schedule a packet send event >>> using a gen_server:call on nsime_simulator. When the simulation is run, the >>> send events are executed resulting in receive events being scheduled at the >>> UDP echo servers. The receive events are executed resulting in reply events >>> being scheduled. Every event being scheduled is a gen_server:call on the >>> nsime_simulator process. >>> >>> The parallelism I am trying to exploit right now is in the form of >>> simultaneous events (events having same time stamp). All the 10k send >>> events are simultaneous, the 10k receives are simultaneous as the point to >>> point connections between each pair has the same delay and data rate and >>> the 10k reply events are simultaneous. >>> >>> The parallel execution of the simultaneous events is done using the >>> following code fragment from src/nsime_simulator.erl. The gen_server:call >>> returns a list of simultaneous events and Stephen Marsh's plists module ( >>> https://github.com/rmies/plists) is used to execute each event (an MFA >>> triple) in the list by dividing the list among 5 processes. This was for a >>> quad core machine. The plists code suggests using number of cores + 1. >>> Each time an event is executed it might result in another event being >>> scheduled which itself is a gen_server:call to nsime_simulator. >>> >>> parallel_run() -> >>> case gen_server:call(?MODULE, parallel_run) of >>> {events, EventList} -> >>> plists:foreach( >>> fun(Event) -> >>> erlang:apply( >>> Event#nsime_event.module, >>> Event#nsime_event.function, >>> Event#nsime_event.arguments >>> ) >>> >>> end, >>> EventList, >>> {processes, 5} >>> >>> >>> >>> >>> >>> ), >>> ?MODULE:parallel_run(); >>> none -> >>> simulation_complete >>> end. >>> >>> >>> On a quad core machine, the ns-3 C++ code for this example runs in 59 >>> seconds while occupying only one core. The NSIME code runs in 65 seconds >>> while occupying about 300% out of a maximum of 400%. That doesn't sound too >>> bad. Another non-trivial C++ vs Erlang benchmark, one can argue. But the >>> NSIME code which does not execute the simultaneous events in parallel runs >>> in 79 seconds while occupying only one core . So I am seeing sublinear >>> speedups. >>> >>> The even more surprising thing is that on a 32 core machine the >>> parallel_run runs in 68 seconds when the simultaneous event list is divided >>> among 32 processes. Since the 32 core and quad core had different CPU >>> speeds I ran the simulation on the 32 core machine with the simultaneous >>> event list divided among 5 processes and that took 66 seconds. So I >>> actually saw a slowdown when I use more cores. When I divided the event >>> list among 32 processes, only 30-40% of each core was being utilized (as >>> seen in htop). >>> >>> I hope my description makes some sense. My gut feeling is that the >>> single nsime_simulator is the bottleneck. I will try to confirm it using >>> the Erlang profiling tools. I was hoping pg or pg2 can provide a solution >>> by distributing the message handling workload. I could schedule events in a >>> randomly chosen process in the process group and then collect the earliest >>> events from all the process group members. But to choose a random process I >>> may have to use pg2:get_closest_pid which itself may cause a bottleneck. >>> >>> sarva >>> >>> >>> >>> >>> On Fri, Dec 14, 2012 at 9:35 PM, Daniel Luna wrote: >>> >>>> On 14 December 2012 09:51, Garrett Smith wrote: >>>> > On Fri, Dec 14, 2012 at 8:47 AM, Garrett Smith wrote: >>>> >> Hi Saravanan, >>>> >> If you're bottlenecking on CPU (all your cores are fully utilized at >>>> >> peak load) then you need either a faster machine or you'll need to >>>> >> distribute your application to multiple machines. >>>> > >>>> > I should add there a number of ways you can improve efficiency, short >>>> > of adding hardware resources. The big win, once you understand what to >>>> > target, is C ports (or NIFs). >>>> >>>> But long before you start looking at either NIFs or new hardware, look >>>> into the complexity of the code itself. Does that expensive function >>>> even have to be called in all cases, etc. >>>> >>>> I guess this is my pet peeve when it comes to optimizing anything. >>>> >>>> First you optimize for readability (often gaining speed or at least >>>> finding issues) >>>> Then you measure >>>> Then you optimize the hotspots you've discovered by measuring >>>> *Then* you can start looking into hardware or non-Erlang solutions >>>> >>>> I've also seen situations where minor changes in the requirements have >>>> seen the possibility to speed up code by a factor 10 so that's also a >>>> possibility. >>>> >>>> Cheers, >>>> >>>> Daniel >>>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Sat Dec 15 10:12:35 2012 From: bourinov@REDACTED (Max Bourinov) Date: Sat, 15 Dec 2012 13:12:35 +0400 Subject: [erlang-questions] appmon crash Message-ID: Hi Erlangers, Does anybody know what could it be? I happened when I started appmon to see how new processes are dealing to the supervisor. I suspect this is error in appmon code? 13:09:52.485 [error] gen_server <0.96.0> terminated with reason: bad arithmetic expression in appmon_a:fit_tree_to_win/2 line 1023 13:09:52.489 [error] CRASH REPORT Process <0.96.0> with 0 neighbours exited with reason: bad arithmetic expression in appmon_a:fit_tree_to_win/2 line 1023 in gen_server:terminate/6 line 737 Best regards, Max -------------- next part -------------- An HTML attachment was scrubbed... URL: From sarva.v@REDACTED Sat Dec 15 10:45:28 2012 From: sarva.v@REDACTED (Saravanan Vijayakumaran) Date: Sat, 15 Dec 2012 15:15:28 +0530 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: Yes. That is correct. The nsime_simulator gen_server is the central authority which decides the event ordering. On Sat, Dec 15, 2012 at 12:00 PM, Max Bourinov wrote: > Hi Saravanan, > > Thank you for explanations. Please correct me if I am wrong: neglecting > the fact that you have many processes you have to execute event one by one > on one gen_server because ir requires by simulation approach. Is that > correct? > > Best regards, > Max > > > On Sat, Dec 15, 2012 at 6:22 AM, Saravanan Vijayakumaran < > sarva.v@REDACTED> wrote: > >> Dear Max, >> There are in fact 10,000 client and server processes. >> Each of them is a gen_server [1][2]. These are just simulation entities >> which are connected by point-to-point channels [3] which are also a >> simulation entities. >> >> The purpose of network simulation is to guide decision making with >> respect to choice of network protocols and parameters in a particular >> scenario. The scenarios are typically too expensive to try out in real >> networks. Imagine a thousand node network placed in different cities. >> >> Regarding discrete event simulation, suppose we want to figure out how >> many runways to have in a new airport. The more the runways the more costly >> it is but it is also will result in less waiting time for >> arriving/departing flights. One way to characterize the runway count vs >> waiting time tradeoff is to use queueing theory. An alternative is to do >> simulate flight arrivals and departures. At any point of time, the >> simulated world will consists of flights in transit, flights waiting to >> depart, flights waiting to land, flights departing and flights landing. In >> discrete time simulation, the state of the simulated world is updated at a >> regular time step (say every minute). Note that this is simulated time and >> not wall-clock time. An alternative which is much faster to execute is >> discrete event simulation. Here there is a queue of simulation events with >> the earliest events at the head of the queue. The events are executed one >> by one by picking the event at the head of the queue. When a flight >> departure event is executed, a waiting to land event is inserted into the >> queue having a timestamp equal to current time + the transit time of the >> flight. When the waiting to land event is executed, if a runway is free a >> land event is inserted into the queue having timestamp equal to the current >> time + landing time. If a runway is not free, a circle the airport and >> check again event is inserted into the queue having timestamp equal to the >> current time + circling time. The purpose of the simulation queue is to >> provide causal ordering of the simulation events. >> >> In my case, I think it is the simulation queue which is becoming the >> bottleneck. I must add that network simulation falls under the category of >> number-crunching applications which Erlang is not supposed to be suited >> for. But people have been trying to accelerate it in C++-based simulators >> using pthreads. I thought it was worth a try in Erlang given the amount of >> work which has already gone into Erlang SMP. >> >> Regards >> sarva >> >> [1] >> https://github.com/avras/nsime/blob/master/src/nsime_udp_echo_client.erl >> [2] >> https://github.com/avras/nsime/blob/master/src/nsime_udp_echo_server.erl >> [3] https://github.com/avras/nsime/blob/master/src/nsime_ptp_channel.erl >> >> >> >> On Sat, Dec 15, 2012 at 1:42 AM, Max Bourinov wrote: >> >>> Hi guys, >>> >>> Saravanan, maybe I am missing the point, but why not to create 10.000 >>> clients and servers processes? In the code you provided I don't see much >>> process creation... >>> >>> Described load is relatively small and you should not see any >>> performance issues with this amount of UDP traffic. >>> >>> p.s.: I am not familiar with discrete system simulation, so maybe my >>> view is absolutely wrong)) >>> >>> Best regards, >>> Max >>> >>> >>> >>> >>> On Fri, Dec 14, 2012 at 11:21 PM, Saravanan Vijayakumaran < >>> sarva.v@REDACTED> wrote: >>> >>>> Dear Garrett, Daniel and Olivier, >>>> Thanks for your >>>> responses. I will try your suggestions. >>>> >>>> @Olivier: Thanks for sending the Sim-Diasca sources and docs earlier. >>>> The ns-3 simulator which is the basis for NSIME is discrete event. Since >>>> Sim-Diasca was discrete time, I put it on the backburner and haven't got a >>>> chance to check it out. >>>> >>>> @All: Let me elaborate a little on the bottleneck I am seeing. >>>> >>>> The simulation scenario consists of 10,000 UDP echo client server pairs >>>> with a pairwise point to point connection between them. Each UDP echo >>>> client sends 10 packets to a unique UDP server with a inter-packet time of >>>> 1s. Each UDP server replies to every packet it receives. In terms of the >>>> simulation, the 10k UDP echo clients each schedule a packet send event >>>> using a gen_server:call on nsime_simulator. When the simulation is run, the >>>> send events are executed resulting in receive events being scheduled at the >>>> UDP echo servers. The receive events are executed resulting in reply events >>>> being scheduled. Every event being scheduled is a gen_server:call on the >>>> nsime_simulator process. >>>> >>>> The parallelism I am trying to exploit right now is in the form of >>>> simultaneous events (events having same time stamp). All the 10k send >>>> events are simultaneous, the 10k receives are simultaneous as the point to >>>> point connections between each pair has the same delay and data rate and >>>> the 10k reply events are simultaneous. >>>> >>>> The parallel execution of the simultaneous events is done using the >>>> following code fragment from src/nsime_simulator.erl. The gen_server:call >>>> returns a list of simultaneous events and Stephen Marsh's plists module ( >>>> https://github.com/rmies/plists) is used to execute each event (an MFA >>>> triple) in the list by dividing the list among 5 processes. This was for a >>>> quad core machine. The plists code suggests using number of cores + 1. >>>> Each time an event is executed it might result in another event being >>>> scheduled which itself is a gen_server:call to nsime_simulator. >>>> >>>> parallel_run() -> >>>> case gen_server:call(?MODULE, parallel_run) of >>>> {events, EventList} -> >>>> plists:foreach( >>>> fun(Event) -> >>>> erlang:apply( >>>> Event#nsime_event.module, >>>> Event#nsime_event.function, >>>> Event#nsime_event.arguments >>>> ) >>>> >>>> end, >>>> EventList, >>>> {processes, 5} >>>> >>>> >>>> >>>> >>>> >>>> >>>> ), >>>> ?MODULE:parallel_run(); >>>> none -> >>>> simulation_complete >>>> end. >>>> >>>> >>>> On a quad core machine, the ns-3 C++ code for this example runs in 59 >>>> seconds while occupying only one core. The NSIME code runs in 65 seconds >>>> while occupying about 300% out of a maximum of 400%. That doesn't sound too >>>> bad. Another non-trivial C++ vs Erlang benchmark, one can argue. But the >>>> NSIME code which does not execute the simultaneous events in parallel runs >>>> in 79 seconds while occupying only one core . So I am seeing sublinear >>>> speedups. >>>> >>>> The even more surprising thing is that on a 32 core machine the >>>> parallel_run runs in 68 seconds when the simultaneous event list is divided >>>> among 32 processes. Since the 32 core and quad core had different CPU >>>> speeds I ran the simulation on the 32 core machine with the simultaneous >>>> event list divided among 5 processes and that took 66 seconds. So I >>>> actually saw a slowdown when I use more cores. When I divided the event >>>> list among 32 processes, only 30-40% of each core was being utilized (as >>>> seen in htop). >>>> >>>> I hope my description makes some sense. My gut feeling is that the >>>> single nsime_simulator is the bottleneck. I will try to confirm it using >>>> the Erlang profiling tools. I was hoping pg or pg2 can provide a solution >>>> by distributing the message handling workload. I could schedule events in a >>>> randomly chosen process in the process group and then collect the earliest >>>> events from all the process group members. But to choose a random process I >>>> may have to use pg2:get_closest_pid which itself may cause a >>>> bottleneck. >>>> >>>> sarva >>>> >>>> >>>> >>>> >>>> On Fri, Dec 14, 2012 at 9:35 PM, Daniel Luna wrote: >>>> >>>>> On 14 December 2012 09:51, Garrett Smith wrote: >>>>> > On Fri, Dec 14, 2012 at 8:47 AM, Garrett Smith wrote: >>>>> >> Hi Saravanan, >>>>> >> If you're bottlenecking on CPU (all your cores are fully utilized at >>>>> >> peak load) then you need either a faster machine or you'll need to >>>>> >> distribute your application to multiple machines. >>>>> > >>>>> > I should add there a number of ways you can improve efficiency, short >>>>> > of adding hardware resources. The big win, once you understand what >>>>> to >>>>> > target, is C ports (or NIFs). >>>>> >>>>> But long before you start looking at either NIFs or new hardware, look >>>>> into the complexity of the code itself. Does that expensive function >>>>> even have to be called in all cases, etc. >>>>> >>>>> I guess this is my pet peeve when it comes to optimizing anything. >>>>> >>>>> First you optimize for readability (often gaining speed or at least >>>>> finding issues) >>>>> Then you measure >>>>> Then you optimize the hotspots you've discovered by measuring >>>>> *Then* you can start looking into hardware or non-Erlang solutions >>>>> >>>>> I've also seen situations where minor changes in the requirements have >>>>> seen the possibility to speed up code by a factor 10 so that's also a >>>>> possibility. >>>>> >>>>> Cheers, >>>>> >>>>> Daniel >>>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bombadil@REDACTED Sat Dec 15 10:47:53 2012 From: bombadil@REDACTED (Manuel A. Rubio "Bombadil") Date: Sat, 15 Dec 2012 10:47:53 +0100 Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish Message-ID: <19414408d3bf707c451df9b4c3e23a89@bosqueviejo.net> Hi, is available now the first spanish book about Erlang/OTP in spanish: http://erlang.bosqueviejo.net Regards, Manuel Rubio. From bourinov@REDACTED Sat Dec 15 11:07:46 2012 From: bourinov@REDACTED (Max Bourinov) Date: Sat, 15 Dec 2012 14:07:46 +0400 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: Then I don't see advantage of Erlang here. Is it possible to distribute work that nsime_simulator does over several nodes? If not - I afraid there is no reason to use Erlang. If yes - load must be distributed. On Sat, Dec 15, 2012 at 1:45 PM, Saravanan Vijayakumaran wrote: > Yes. That is correct. The nsime_simulator gen_server is the central > authority which decides the event ordering. > > > On Sat, Dec 15, 2012 at 12:00 PM, Max Bourinov wrote: > >> Hi Saravanan, >> >> Thank you for explanations. Please correct me if I am wrong: neglecting >> the fact that you have many processes you have to execute event one by one >> on one gen_server because ir requires by simulation approach. Is that >> correct? >> >> Best regards, >> Max >> >> >> On Sat, Dec 15, 2012 at 6:22 AM, Saravanan Vijayakumaran < >> sarva.v@REDACTED> wrote: >> >>> Dear Max, >>> There are in fact 10,000 client and server processes. >>> Each of them is a gen_server [1][2]. These are just simulation entities >>> which are connected by point-to-point channels [3] which are also a >>> simulation entities. >>> >>> The purpose of network simulation is to guide decision making with >>> respect to choice of network protocols and parameters in a particular >>> scenario. The scenarios are typically too expensive to try out in real >>> networks. Imagine a thousand node network placed in different cities. >>> >>> Regarding discrete event simulation, suppose we want to figure out how >>> many runways to have in a new airport. The more the runways the more costly >>> it is but it is also will result in less waiting time for >>> arriving/departing flights. One way to characterize the runway count vs >>> waiting time tradeoff is to use queueing theory. An alternative is to do >>> simulate flight arrivals and departures. At any point of time, the >>> simulated world will consists of flights in transit, flights waiting to >>> depart, flights waiting to land, flights departing and flights landing. In >>> discrete time simulation, the state of the simulated world is updated at a >>> regular time step (say every minute). Note that this is simulated time and >>> not wall-clock time. An alternative which is much faster to execute is >>> discrete event simulation. Here there is a queue of simulation events with >>> the earliest events at the head of the queue. The events are executed one >>> by one by picking the event at the head of the queue. When a flight >>> departure event is executed, a waiting to land event is inserted into the >>> queue having a timestamp equal to current time + the transit time of the >>> flight. When the waiting to land event is executed, if a runway is free a >>> land event is inserted into the queue having timestamp equal to the current >>> time + landing time. If a runway is not free, a circle the airport and >>> check again event is inserted into the queue having timestamp equal to the >>> current time + circling time. The purpose of the simulation queue is to >>> provide causal ordering of the simulation events. >>> >>> In my case, I think it is the simulation queue which is becoming the >>> bottleneck. I must add that network simulation falls under the category of >>> number-crunching applications which Erlang is not supposed to be suited >>> for. But people have been trying to accelerate it in C++-based simulators >>> using pthreads. I thought it was worth a try in Erlang given the amount of >>> work which has already gone into Erlang SMP. >>> >>> Regards >>> sarva >>> >>> [1] >>> https://github.com/avras/nsime/blob/master/src/nsime_udp_echo_client.erl >>> [2] >>> https://github.com/avras/nsime/blob/master/src/nsime_udp_echo_server.erl >>> [3] https://github.com/avras/nsime/blob/master/src/nsime_ptp_channel.erl >>> >>> >>> >>> On Sat, Dec 15, 2012 at 1:42 AM, Max Bourinov wrote: >>> >>>> Hi guys, >>>> >>>> Saravanan, maybe I am missing the point, but why not to create 10.000 >>>> clients and servers processes? In the code you provided I don't see much >>>> process creation... >>>> >>>> Described load is relatively small and you should not see any >>>> performance issues with this amount of UDP traffic. >>>> >>>> p.s.: I am not familiar with discrete system simulation, so maybe my >>>> view is absolutely wrong)) >>>> >>>> Best regards, >>>> Max >>>> >>>> >>>> >>>> >>>> On Fri, Dec 14, 2012 at 11:21 PM, Saravanan Vijayakumaran < >>>> sarva.v@REDACTED> wrote: >>>> >>>>> Dear Garrett, Daniel and Olivier, >>>>> Thanks for your >>>>> responses. I will try your suggestions. >>>>> >>>>> @Olivier: Thanks for sending the Sim-Diasca sources and docs earlier. >>>>> The ns-3 simulator which is the basis for NSIME is discrete event. Since >>>>> Sim-Diasca was discrete time, I put it on the backburner and haven't got a >>>>> chance to check it out. >>>>> >>>>> @All: Let me elaborate a little on the bottleneck I am seeing. >>>>> >>>>> The simulation scenario consists of 10,000 UDP echo client server >>>>> pairs with a pairwise point to point connection between them. Each UDP echo >>>>> client sends 10 packets to a unique UDP server with a inter-packet time of >>>>> 1s. Each UDP server replies to every packet it receives. In terms of the >>>>> simulation, the 10k UDP echo clients each schedule a packet send event >>>>> using a gen_server:call on nsime_simulator. When the simulation is run, the >>>>> send events are executed resulting in receive events being scheduled at the >>>>> UDP echo servers. The receive events are executed resulting in reply events >>>>> being scheduled. Every event being scheduled is a gen_server:call on the >>>>> nsime_simulator process. >>>>> >>>>> The parallelism I am trying to exploit right now is in the form of >>>>> simultaneous events (events having same time stamp). All the 10k send >>>>> events are simultaneous, the 10k receives are simultaneous as the point to >>>>> point connections between each pair has the same delay and data rate and >>>>> the 10k reply events are simultaneous. >>>>> >>>>> The parallel execution of the simultaneous events is done using the >>>>> following code fragment from src/nsime_simulator.erl. The gen_server:call >>>>> returns a list of simultaneous events and Stephen Marsh's plists module ( >>>>> https://github.com/rmies/plists) is used to execute each event (an >>>>> MFA triple) in the list by dividing the list among 5 processes. This was >>>>> for a quad core machine. The plists code suggests using number of cores + >>>>> 1. Each time an event is executed it might result in another event being >>>>> scheduled which itself is a gen_server:call to nsime_simulator. >>>>> >>>>> parallel_run() -> >>>>> case gen_server:call(?MODULE, parallel_run) of >>>>> {events, EventList} -> >>>>> plists:foreach( >>>>> fun(Event) -> >>>>> erlang:apply( >>>>> Event#nsime_event.module, >>>>> Event#nsime_event.function, >>>>> Event#nsime_event.arguments >>>>> ) >>>>> >>>>> end, >>>>> EventList, >>>>> {processes, 5} >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> ), >>>>> ?MODULE:parallel_run(); >>>>> none -> >>>>> simulation_complete >>>>> end. >>>>> >>>>> >>>>> On a quad core machine, the ns-3 C++ code for this example runs in 59 >>>>> seconds while occupying only one core. The NSIME code runs in 65 seconds >>>>> while occupying about 300% out of a maximum of 400%. That doesn't sound too >>>>> bad. Another non-trivial C++ vs Erlang benchmark, one can argue. But the >>>>> NSIME code which does not execute the simultaneous events in parallel runs >>>>> in 79 seconds while occupying only one core . So I am seeing sublinear >>>>> speedups. >>>>> >>>>> The even more surprising thing is that on a 32 core machine the >>>>> parallel_run runs in 68 seconds when the simultaneous event list is divided >>>>> among 32 processes. Since the 32 core and quad core had different CPU >>>>> speeds I ran the simulation on the 32 core machine with the simultaneous >>>>> event list divided among 5 processes and that took 66 seconds. So I >>>>> actually saw a slowdown when I use more cores. When I divided the event >>>>> list among 32 processes, only 30-40% of each core was being utilized (as >>>>> seen in htop). >>>>> >>>>> I hope my description makes some sense. My gut feeling is that the >>>>> single nsime_simulator is the bottleneck. I will try to confirm it using >>>>> the Erlang profiling tools. I was hoping pg or pg2 can provide a solution >>>>> by distributing the message handling workload. I could schedule events in a >>>>> randomly chosen process in the process group and then collect the earliest >>>>> events from all the process group members. But to choose a random process I >>>>> may have to use pg2:get_closest_pid which itself may cause a >>>>> bottleneck. >>>>> >>>>> sarva >>>>> >>>>> >>>>> >>>>> >>>>> On Fri, Dec 14, 2012 at 9:35 PM, Daniel Luna wrote: >>>>> >>>>>> On 14 December 2012 09:51, Garrett Smith wrote: >>>>>> > On Fri, Dec 14, 2012 at 8:47 AM, Garrett Smith wrote: >>>>>> >> Hi Saravanan, >>>>>> >> If you're bottlenecking on CPU (all your cores are fully utilized >>>>>> at >>>>>> >> peak load) then you need either a faster machine or you'll need to >>>>>> >> distribute your application to multiple machines. >>>>>> > >>>>>> > I should add there a number of ways you can improve efficiency, >>>>>> short >>>>>> > of adding hardware resources. The big win, once you understand what >>>>>> to >>>>>> > target, is C ports (or NIFs). >>>>>> >>>>>> But long before you start looking at either NIFs or new hardware, look >>>>>> into the complexity of the code itself. Does that expensive function >>>>>> even have to be called in all cases, etc. >>>>>> >>>>>> I guess this is my pet peeve when it comes to optimizing anything. >>>>>> >>>>>> First you optimize for readability (often gaining speed or at least >>>>>> finding issues) >>>>>> Then you measure >>>>>> Then you optimize the hotspots you've discovered by measuring >>>>>> *Then* you can start looking into hardware or non-Erlang solutions >>>>>> >>>>>> I've also seen situations where minor changes in the requirements have >>>>>> seen the possibility to speed up code by a factor 10 so that's also a >>>>>> possibility. >>>>>> >>>>>> Cheers, >>>>>> >>>>>> Daniel >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Sat Dec 15 11:12:31 2012 From: bourinov@REDACTED (Max Bourinov) Date: Sat, 15 Dec 2012 14:12:31 +0400 Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish In-Reply-To: <19414408d3bf707c451df9b4c3e23a89@bosqueviejo.net> References: <19414408d3bf707c451df9b4c3e23a89@bosqueviejo.net> Message-ID: Great book! Do you plan to translate it to English? Best regards, Max On Sat, Dec 15, 2012 at 1:47 PM, Manuel A. Rubio "Bombadil" < bombadil@REDACTED> wrote: > Hi, > > is available now the first spanish book about Erlang/OTP in spanish: > > http://erlang.bosqueviejo.net > > Regards, > Manuel Rubio. > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco@REDACTED Sat Dec 15 11:58:56 2012 From: francesco@REDACTED (Francesco Cesarini) Date: Sat, 15 Dec 2012 10:58:56 +0000 (GMT) Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish In-Reply-To: <19414408d3bf707c451df9b4c3e23a89@bosqueviejo.net> Message-ID: <512488132.2438613.1355569136161.JavaMail.root@erlang-solutions.com> Woohoo! Well done. The space in that book shelf is now decreasing by the day. What next? Portuguese? Afrikaans? Italian? Esperanto? Erlang for Dummies? Looking forward to reading it. Francesco ----- Original Message ----- From: "Manuel A. Rubio \"Bombadil\"" To: erlang-questions@REDACTED Sent: Saturday, December 15, 2012 9:47:53 AM Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish Hi, is available now the first spanish book about Erlang/OTP in spanish: http://erlang.bosqueviejo.net Regards, Manuel Rubio. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From chandrashekhar.mullaparthi@REDACTED Sat Dec 15 12:08:57 2012 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Sat, 15 Dec 2012 11:08:57 +0000 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: Surely the expressive power of erlang is a justification if the performance is 'good enough'? One way to serialise inserts into the work queue is to use an ordered_set ets table with a unique index such as the return value of now(). Also look into ets:select_delete to take matching tasks out of the queue. I haven't looked at the code in great detail, but I suspect these might be useful to you Saravanan. cheers Chandru On 15 December 2012 10:07, Max Bourinov wrote: > Then I don't see advantage of Erlang here. Is it possible to distribute work > that nsime_simulator does over several nodes? > > If not - I afraid there is no reason to use Erlang. If yes - load must be > distributed. > > > On Sat, Dec 15, 2012 at 1:45 PM, Saravanan Vijayakumaran > wrote: >> >> Yes. That is correct. The nsime_simulator gen_server is the central >> authority which decides the event ordering. >> >> >> On Sat, Dec 15, 2012 at 12:00 PM, Max Bourinov wrote: >>> >>> Hi Saravanan, >>> >>> Thank you for explanations. Please correct me if I am wrong: neglecting >>> the fact that you have many processes you have to execute event one by one >>> on one gen_server because ir requires by simulation approach. Is that >>> correct? >>> >>> Best regards, >>> Max >>> >>> >>> On Sat, Dec 15, 2012 at 6:22 AM, Saravanan Vijayakumaran >>> wrote: >>>> >>>> Dear Max, >>>> There are in fact 10,000 client and server processes. >>>> Each of them is a gen_server [1][2]. These are just simulation entities >>>> which are connected by point-to-point channels [3] which are also a >>>> simulation entities. >>>> >>>> The purpose of network simulation is to guide decision making with >>>> respect to choice of network protocols and parameters in a particular >>>> scenario. The scenarios are typically too expensive to try out in real >>>> networks. Imagine a thousand node network placed in different cities. >>>> >>>> Regarding discrete event simulation, suppose we want to figure out how >>>> many runways to have in a new airport. The more the runways the more costly >>>> it is but it is also will result in less waiting time for arriving/departing >>>> flights. One way to characterize the runway count vs waiting time tradeoff >>>> is to use queueing theory. An alternative is to do simulate flight arrivals >>>> and departures. At any point of time, the simulated world will consists of >>>> flights in transit, flights waiting to depart, flights waiting to land, >>>> flights departing and flights landing. In discrete time simulation, the >>>> state of the simulated world is updated at a regular time step (say every >>>> minute). Note that this is simulated time and not wall-clock time. An >>>> alternative which is much faster to execute is discrete event simulation. >>>> Here there is a queue of simulation events with the earliest events at the >>>> head of the queue. The events are executed one by one by picking the event >>>> at the head of the queue. When a flight departure event is executed, a >>>> waiting to land event is inserted into the queue having a timestamp equal to >>>> current time + the transit time of the flight. When the waiting to land >>>> event is executed, if a runway is free a land event is inserted into the >>>> queue having timestamp equal to the current time + landing time. If a runway >>>> is not free, a circle the airport and check again event is inserted into the >>>> queue having timestamp equal to the current time + circling time. The >>>> purpose of the simulation queue is to provide causal ordering of the >>>> simulation events. >>>> >>>> In my case, I think it is the simulation queue which is becoming the >>>> bottleneck. I must add that network simulation falls under the category of >>>> number-crunching applications which Erlang is not supposed to be suited for. >>>> But people have been trying to accelerate it in C++-based simulators using >>>> pthreads. I thought it was worth a try in Erlang given the amount of work >>>> which has already gone into Erlang SMP. >>>> >>>> Regards >>>> sarva >>>> >>>> [1] >>>> https://github.com/avras/nsime/blob/master/src/nsime_udp_echo_client.erl >>>> [2] >>>> https://github.com/avras/nsime/blob/master/src/nsime_udp_echo_server.erl >>>> [3] https://github.com/avras/nsime/blob/master/src/nsime_ptp_channel.erl >>>> >>>> >>>> >>>> On Sat, Dec 15, 2012 at 1:42 AM, Max Bourinov >>>> wrote: >>>>> >>>>> Hi guys, >>>>> >>>>> Saravanan, maybe I am missing the point, but why not to create 10.000 >>>>> clients and servers processes? In the code you provided I don't see much >>>>> process creation... >>>>> >>>>> Described load is relatively small and you should not see any >>>>> performance issues with this amount of UDP traffic. >>>>> >>>>> p.s.: I am not familiar with discrete system simulation, so maybe my >>>>> view is absolutely wrong)) >>>>> >>>>> Best regards, >>>>> Max >>>>> >>>>> >>>>> >>>>> >>>>> On Fri, Dec 14, 2012 at 11:21 PM, Saravanan Vijayakumaran >>>>> wrote: >>>>>> >>>>>> Dear Garrett, Daniel and Olivier, >>>>>> Thanks for your >>>>>> responses. I will try your suggestions. >>>>>> >>>>>> @Olivier: Thanks for sending the Sim-Diasca sources and docs earlier. >>>>>> The ns-3 simulator which is the basis for NSIME is discrete event. Since >>>>>> Sim-Diasca was discrete time, I put it on the backburner and haven't got a >>>>>> chance to check it out. >>>>>> >>>>>> @All: Let me elaborate a little on the bottleneck I am seeing. >>>>>> >>>>>> The simulation scenario consists of 10,000 UDP echo client server >>>>>> pairs with a pairwise point to point connection between them. Each UDP echo >>>>>> client sends 10 packets to a unique UDP server with a inter-packet time of >>>>>> 1s. Each UDP server replies to every packet it receives. In terms of the >>>>>> simulation, the 10k UDP echo clients each schedule a packet send event using >>>>>> a gen_server:call on nsime_simulator. When the simulation is run, the send >>>>>> events are executed resulting in receive events being scheduled at the UDP >>>>>> echo servers. The receive events are executed resulting in reply events >>>>>> being scheduled. Every event being scheduled is a gen_server:call on the >>>>>> nsime_simulator process. >>>>>> >>>>>> The parallelism I am trying to exploit right now is in the form of >>>>>> simultaneous events (events having same time stamp). All the 10k send events >>>>>> are simultaneous, the 10k receives are simultaneous as the point to point >>>>>> connections between each pair has the same delay and data rate and the 10k >>>>>> reply events are simultaneous. >>>>>> >>>>>> The parallel execution of the simultaneous events is done using the >>>>>> following code fragment from src/nsime_simulator.erl. The gen_server:call >>>>>> returns a list of simultaneous events and Stephen Marsh's plists module >>>>>> (https://github.com/rmies/plists) is used to execute each event (an MFA >>>>>> triple) in the list by dividing the list among 5 processes. This was for a >>>>>> quad core machine. The plists code suggests using number of cores + 1. Each >>>>>> time an event is executed it might result in another event being scheduled >>>>>> which itself is a gen_server:call to nsime_simulator. >>>>>> >>>>>> parallel_run() -> >>>>>> case gen_server:call(?MODULE, parallel_run) of >>>>>> {events, EventList} -> >>>>>> plists:foreach( >>>>>> fun(Event) -> >>>>>> erlang:apply( >>>>>> Event#nsime_event.module, >>>>>> Event#nsime_event.function, >>>>>> Event#nsime_event.arguments >>>>>> ) >>>>>> end, >>>>>> EventList, >>>>>> {processes, 5} >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ), >>>>>> ?MODULE:parallel_run(); >>>>>> none -> >>>>>> simulation_complete >>>>>> end. >>>>>> >>>>>> >>>>>> On a quad core machine, the ns-3 C++ code for this example runs in 59 >>>>>> seconds while occupying only one core. The NSIME code runs in 65 seconds >>>>>> while occupying about 300% out of a maximum of 400%. That doesn't sound too >>>>>> bad. Another non-trivial C++ vs Erlang benchmark, one can argue. But the >>>>>> NSIME code which does not execute the simultaneous events in parallel runs >>>>>> in 79 seconds while occupying only one core . So I am seeing sublinear >>>>>> speedups. >>>>>> >>>>>> The even more surprising thing is that on a 32 core machine the >>>>>> parallel_run runs in 68 seconds when the simultaneous event list is divided >>>>>> among 32 processes. Since the 32 core and quad core had different CPU speeds >>>>>> I ran the simulation on the 32 core machine with the simultaneous event list >>>>>> divided among 5 processes and that took 66 seconds. So I actually saw a >>>>>> slowdown when I use more cores. When I divided the event list among 32 >>>>>> processes, only 30-40% of each core was being utilized (as seen in htop). >>>>>> >>>>>> I hope my description makes some sense. My gut feeling is that the >>>>>> single nsime_simulator is the bottleneck. I will try to confirm it using the >>>>>> Erlang profiling tools. I was hoping pg or pg2 can provide a solution by >>>>>> distributing the message handling workload. I could schedule events in a >>>>>> randomly chosen process in the process group and then collect the earliest >>>>>> events from all the process group members. But to choose a random process I >>>>>> may have to use pg2:get_closest_pid which itself may cause a bottleneck. >>>>>> >>>>>> sarva >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Dec 14, 2012 at 9:35 PM, Daniel Luna wrote: >>>>>>> >>>>>>> On 14 December 2012 09:51, Garrett Smith wrote: >>>>>>> > On Fri, Dec 14, 2012 at 8:47 AM, Garrett Smith wrote: >>>>>>> >> Hi Saravanan, >>>>>>> >> If you're bottlenecking on CPU (all your cores are fully utilized >>>>>>> >> at >>>>>>> >> peak load) then you need either a faster machine or you'll need to >>>>>>> >> distribute your application to multiple machines. >>>>>>> > >>>>>>> > I should add there a number of ways you can improve efficiency, >>>>>>> > short >>>>>>> > of adding hardware resources. The big win, once you understand what >>>>>>> > to >>>>>>> > target, is C ports (or NIFs). >>>>>>> >>>>>>> But long before you start looking at either NIFs or new hardware, >>>>>>> look >>>>>>> into the complexity of the code itself. Does that expensive function >>>>>>> even have to be called in all cases, etc. >>>>>>> >>>>>>> I guess this is my pet peeve when it comes to optimizing anything. >>>>>>> >>>>>>> First you optimize for readability (often gaining speed or at least >>>>>>> finding issues) >>>>>>> Then you measure >>>>>>> Then you optimize the hotspots you've discovered by measuring >>>>>>> *Then* you can start looking into hardware or non-Erlang solutions >>>>>>> >>>>>>> I've also seen situations where minor changes in the requirements >>>>>>> have >>>>>>> seen the possibility to speed up code by a factor 10 so that's also a >>>>>>> possibility. >>>>>>> >>>>>>> Cheers, >>>>>>> >>>>>>> Daniel >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>> >>>> >>> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From sarva.v@REDACTED Sat Dec 15 14:02:12 2012 From: sarva.v@REDACTED (Saravanan Vijayakumaran) Date: Sat, 15 Dec 2012 18:32:12 +0530 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: Max, you are right about the application not having enough decoupling typical of Erlang applications. But as Chandrasekhar has pointed out, the dynamic typing and higher order functions made it much easier to implement the functionality in Erlang which was buck ugly in C++ (the ns-3 simulator I was porting is in C++). For example, consider the following code fragment which defines callbacks in ns-3 [1]. In NSIME, a callback is simply an MFA triple where varying number of arguments is not an issue [2],[3]. If not for performance, Erlang wins in terms of readability. template Callback MakeCallback (R (*fnPtr)()) { return Callback (fnPtr, true, true); } template Callback MakeCallback (R (*fnPtr)(T1)) { return Callback (fnPtr, true, true); } template Callback MakeCallback (R (*fnPtr)(T1,T2)) { return Callback (fnPtr, true, true); } template Callback MakeCallback (R (*fnPtr)(T1,T2,T3)) { return Callback (fnPtr, true, true); } .... and so on till nine arguments are supported! @Chandrasekhar Thanks for the ets pointer. I will check it out. [1] http://www.nsnam.org/docs/release/3.15/doxygen/callback_8h_source.html [2] https://github.com/avras/nsime/blob/master/src/nsime_callback.erl [3] https://github.com/avras/nsime/blob/master/include/nsime_types.hrl#L40 On Sat, Dec 15, 2012 at 4:38 PM, Chandru < chandrashekhar.mullaparthi@REDACTED> wrote: > Surely the expressive power of erlang is a justification if the > performance is 'good enough'? > > One way to serialise inserts into the work queue is to use an > ordered_set ets table with a unique index such as the return value of > now(). Also look into ets:select_delete to take matching tasks out of > the queue. I haven't looked at the code in great detail, but I suspect > these might be useful to you Saravanan. > > cheers > Chandru > > On 15 December 2012 10:07, Max Bourinov wrote: > > Then I don't see advantage of Erlang here. Is it possible to distribute > work > > that nsime_simulator does over several nodes? > > > > If not - I afraid there is no reason to use Erlang. If yes - load must be > > distributed. > > > > > > On Sat, Dec 15, 2012 at 1:45 PM, Saravanan Vijayakumaran < > sarva.v@REDACTED> > > wrote: > >> > >> Yes. That is correct. The nsime_simulator gen_server is the central > >> authority which decides the event ordering. > >> > >> > >> On Sat, Dec 15, 2012 at 12:00 PM, Max Bourinov > wrote: > >>> > >>> Hi Saravanan, > >>> > >>> Thank you for explanations. Please correct me if I am wrong: neglecting > >>> the fact that you have many processes you have to execute event one by > one > >>> on one gen_server because ir requires by simulation approach. Is that > >>> correct? > >>> > >>> Best regards, > >>> Max > >>> > >>> > >>> On Sat, Dec 15, 2012 at 6:22 AM, Saravanan Vijayakumaran > >>> wrote: > >>>> > >>>> Dear Max, > >>>> There are in fact 10,000 client and server processes. > >>>> Each of them is a gen_server [1][2]. These are just simulation > entities > >>>> which are connected by point-to-point channels [3] which are also a > >>>> simulation entities. > >>>> > >>>> The purpose of network simulation is to guide decision making with > >>>> respect to choice of network protocols and parameters in a particular > >>>> scenario. The scenarios are typically too expensive to try out in real > >>>> networks. Imagine a thousand node network placed in different cities. > >>>> > >>>> Regarding discrete event simulation, suppose we want to figure out how > >>>> many runways to have in a new airport. The more the runways the more > costly > >>>> it is but it is also will result in less waiting time for > arriving/departing > >>>> flights. One way to characterize the runway count vs waiting time > tradeoff > >>>> is to use queueing theory. An alternative is to do simulate flight > arrivals > >>>> and departures. At any point of time, the simulated world will > consists of > >>>> flights in transit, flights waiting to depart, flights waiting to > land, > >>>> flights departing and flights landing. In discrete time simulation, > the > >>>> state of the simulated world is updated at a regular time step (say > every > >>>> minute). Note that this is simulated time and not wall-clock time. An > >>>> alternative which is much faster to execute is discrete event > simulation. > >>>> Here there is a queue of simulation events with the earliest events > at the > >>>> head of the queue. The events are executed one by one by picking the > event > >>>> at the head of the queue. When a flight departure event is executed, > a > >>>> waiting to land event is inserted into the queue having a timestamp > equal to > >>>> current time + the transit time of the flight. When the waiting to > land > >>>> event is executed, if a runway is free a land event is inserted into > the > >>>> queue having timestamp equal to the current time + landing time. If a > runway > >>>> is not free, a circle the airport and check again event is inserted > into the > >>>> queue having timestamp equal to the current time + circling time. The > >>>> purpose of the simulation queue is to provide causal ordering of the > >>>> simulation events. > >>>> > >>>> In my case, I think it is the simulation queue which is becoming the > >>>> bottleneck. I must add that network simulation falls under the > category of > >>>> number-crunching applications which Erlang is not supposed to be > suited for. > >>>> But people have been trying to accelerate it in C++-based simulators > using > >>>> pthreads. I thought it was worth a try in Erlang given the amount of > work > >>>> which has already gone into Erlang SMP. > >>>> > >>>> Regards > >>>> sarva > >>>> > >>>> [1] > >>>> > https://github.com/avras/nsime/blob/master/src/nsime_udp_echo_client.erl > >>>> [2] > >>>> > https://github.com/avras/nsime/blob/master/src/nsime_udp_echo_server.erl > >>>> [3] > https://github.com/avras/nsime/blob/master/src/nsime_ptp_channel.erl > >>>> > >>>> > >>>> > >>>> On Sat, Dec 15, 2012 at 1:42 AM, Max Bourinov > >>>> wrote: > >>>>> > >>>>> Hi guys, > >>>>> > >>>>> Saravanan, maybe I am missing the point, but why not to create 10.000 > >>>>> clients and servers processes? In the code you provided I don't see > much > >>>>> process creation... > >>>>> > >>>>> Described load is relatively small and you should not see any > >>>>> performance issues with this amount of UDP traffic. > >>>>> > >>>>> p.s.: I am not familiar with discrete system simulation, so maybe my > >>>>> view is absolutely wrong)) > >>>>> > >>>>> Best regards, > >>>>> Max > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On Fri, Dec 14, 2012 at 11:21 PM, Saravanan Vijayakumaran > >>>>> wrote: > >>>>>> > >>>>>> Dear Garrett, Daniel and Olivier, > >>>>>> Thanks for your > >>>>>> responses. I will try your suggestions. > >>>>>> > >>>>>> @Olivier: Thanks for sending the Sim-Diasca sources and docs > earlier. > >>>>>> The ns-3 simulator which is the basis for NSIME is discrete event. > Since > >>>>>> Sim-Diasca was discrete time, I put it on the backburner and > haven't got a > >>>>>> chance to check it out. > >>>>>> > >>>>>> @All: Let me elaborate a little on the bottleneck I am seeing. > >>>>>> > >>>>>> The simulation scenario consists of 10,000 UDP echo client server > >>>>>> pairs with a pairwise point to point connection between them. Each > UDP echo > >>>>>> client sends 10 packets to a unique UDP server with a inter-packet > time of > >>>>>> 1s. Each UDP server replies to every packet it receives. In terms > of the > >>>>>> simulation, the 10k UDP echo clients each schedule a packet send > event using > >>>>>> a gen_server:call on nsime_simulator. When the simulation is run, > the send > >>>>>> events are executed resulting in receive events being scheduled at > the UDP > >>>>>> echo servers. The receive events are executed resulting in reply > events > >>>>>> being scheduled. Every event being scheduled is a gen_server:call > on the > >>>>>> nsime_simulator process. > >>>>>> > >>>>>> The parallelism I am trying to exploit right now is in the form of > >>>>>> simultaneous events (events having same time stamp). All the 10k > send events > >>>>>> are simultaneous, the 10k receives are simultaneous as the point to > point > >>>>>> connections between each pair has the same delay and data rate and > the 10k > >>>>>> reply events are simultaneous. > >>>>>> > >>>>>> The parallel execution of the simultaneous events is done using the > >>>>>> following code fragment from src/nsime_simulator.erl. The > gen_server:call > >>>>>> returns a list of simultaneous events and Stephen Marsh's plists > module > >>>>>> (https://github.com/rmies/plists) is used to execute each event > (an MFA > >>>>>> triple) in the list by dividing the list among 5 processes. This > was for a > >>>>>> quad core machine. The plists code suggests using number of cores > + 1. Each > >>>>>> time an event is executed it might result in another event being > scheduled > >>>>>> which itself is a gen_server:call to nsime_simulator. > >>>>>> > >>>>>> parallel_run() -> > >>>>>> case gen_server:call(?MODULE, parallel_run) of > >>>>>> {events, EventList} -> > >>>>>> plists:foreach( > >>>>>> fun(Event) -> > >>>>>> erlang:apply( > >>>>>> Event#nsime_event.module, > >>>>>> Event#nsime_event.function, > >>>>>> Event#nsime_event.arguments > >>>>>> ) > >>>>>> end, > >>>>>> EventList, > >>>>>> {processes, 5} > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> ), > >>>>>> ?MODULE:parallel_run(); > >>>>>> none -> > >>>>>> simulation_complete > >>>>>> end. > >>>>>> > >>>>>> > >>>>>> On a quad core machine, the ns-3 C++ code for this example runs in > 59 > >>>>>> seconds while occupying only one core. The NSIME code runs in 65 > seconds > >>>>>> while occupying about 300% out of a maximum of 400%. That doesn't > sound too > >>>>>> bad. Another non-trivial C++ vs Erlang benchmark, one can argue. > But the > >>>>>> NSIME code which does not execute the simultaneous events in > parallel runs > >>>>>> in 79 seconds while occupying only one core . So I am seeing > sublinear > >>>>>> speedups. > >>>>>> > >>>>>> The even more surprising thing is that on a 32 core machine the > >>>>>> parallel_run runs in 68 seconds when the simultaneous event list is > divided > >>>>>> among 32 processes. Since the 32 core and quad core had different > CPU speeds > >>>>>> I ran the simulation on the 32 core machine with the simultaneous > event list > >>>>>> divided among 5 processes and that took 66 seconds. So I actually > saw a > >>>>>> slowdown when I use more cores. When I divided the event list among > 32 > >>>>>> processes, only 30-40% of each core was being utilized (as seen in > htop). > >>>>>> > >>>>>> I hope my description makes some sense. My gut feeling is that the > >>>>>> single nsime_simulator is the bottleneck. I will try to confirm it > using the > >>>>>> Erlang profiling tools. I was hoping pg or pg2 can provide a > solution by > >>>>>> distributing the message handling workload. I could schedule events > in a > >>>>>> randomly chosen process in the process group and then collect the > earliest > >>>>>> events from all the process group members. But to choose a random > process I > >>>>>> may have to use pg2:get_closest_pid which itself may cause a > bottleneck. > >>>>>> > >>>>>> sarva > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Fri, Dec 14, 2012 at 9:35 PM, Daniel Luna > wrote: > >>>>>>> > >>>>>>> On 14 December 2012 09:51, Garrett Smith wrote: > >>>>>>> > On Fri, Dec 14, 2012 at 8:47 AM, Garrett Smith wrote: > >>>>>>> >> Hi Saravanan, > >>>>>>> >> If you're bottlenecking on CPU (all your cores are fully > utilized > >>>>>>> >> at > >>>>>>> >> peak load) then you need either a faster machine or you'll need > to > >>>>>>> >> distribute your application to multiple machines. > >>>>>>> > > >>>>>>> > I should add there a number of ways you can improve efficiency, > >>>>>>> > short > >>>>>>> > of adding hardware resources. The big win, once you understand > what > >>>>>>> > to > >>>>>>> > target, is C ports (or NIFs). > >>>>>>> > >>>>>>> But long before you start looking at either NIFs or new hardware, > >>>>>>> look > >>>>>>> into the complexity of the code itself. Does that expensive > function > >>>>>>> even have to be called in all cases, etc. > >>>>>>> > >>>>>>> I guess this is my pet peeve when it comes to optimizing anything. > >>>>>>> > >>>>>>> First you optimize for readability (often gaining speed or at least > >>>>>>> finding issues) > >>>>>>> Then you measure > >>>>>>> Then you optimize the hotspots you've discovered by measuring > >>>>>>> *Then* you can start looking into hardware or non-Erlang solutions > >>>>>>> > >>>>>>> I've also seen situations where minor changes in the requirements > >>>>>>> have > >>>>>>> seen the possibility to speed up code by a factor 10 so that's > also a > >>>>>>> possibility. > >>>>>>> > >>>>>>> Cheers, > >>>>>>> > >>>>>>> Daniel > >>>>>> > >>>>>> > >>>>>> > >>>>>> _______________________________________________ > >>>>>> erlang-questions mailing list > >>>>>> erlang-questions@REDACTED > >>>>>> http://erlang.org/mailman/listinfo/erlang-questions > >>>>>> > >>>>> > >>>> > >>> > >> > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lovei.laszlo@REDACTED Sat Dec 15 15:54:55 2012 From: lovei.laszlo@REDACTED (Laszlo Lovei) Date: Sat, 15 Dec 2012 06:54:55 -0800 (PST) Subject: [erlang-questions] noob namespace question In-Reply-To: References: <50C8C639.6060006@ninenines.eu> <20121212183003.GB13660@members.linode.com> Message-ID: <10ccdc72-632a-459b-8d26-5563d4b64e6a@googlegroups.com> On Wednesday, December 12, 2012 7:40:26 PM UTC+1, rusi wrote: > > On Thu, Dec 13, 2012 at 12:00 AM, Thomas Allen > > wrote: > >> On Wed, Dec 12, 2012 at 11:44:43PM +0530, Rustom Mody wrote: >> > I dont understand. Let me try and be more explicit about my concern. >> > >> > Let h be a higher-order function of one argument (h/1) >> > >> > Now in the call h(foo) >> > How does Erlang know whether foo is a function or an atom? >> >> This is not valid, and you will get a "bad function" error if you try to >> pass your function this way. You must create a "fun" like so: >> >> h(fun foo/0) >> >> Similarly, you could pass an anonymous fun (like fun(X) -> X end), or >> a module-function combination (like fun lists:reverse/1). >> >> > Thanks Thomas that answers succinctly. > That is a succinct but incorrect answer :) The call `h(foo)' is completely valid, and might even refer to function `foo', although in a different way than `h(fun foo/0)'. I think the common source of misunderstanding here is that atoms are used as both run time data and compile time identifiers. The expression `foo' simply creates a data of type atom, without any additional meaning. This data can be be used to identify something (a module, a function, a record, a message - we don't know in advance) at run time. The expression `fun foo/0' however creates a run time data of type function reference to function foo, and the atom foo itself is only used to identify the function at compile time. The same thing happens in expression `foo()', which is compiled into a function call to function foo, without actually involving the atom foo at run time. The connection between the compile time identifiers of functions and run time atom data is that exported functions are identified by atom data at run time, instead of atom identifiers at compile time. So the expression `m:foo()' means "look up a function with identifier foo in the export table of module m, and call that function", where both atom m and atom foo are run time data. You can even use arbitrary expressions to compute the module or function name, so for example `h(F) -> m:F().' is valid code, and calling `h(foo)' means the same as `m:foo()'. Note that you can't do this with local functions: the code `h(F) -> F().' works only when F is a function reference, so it must be called as `h(fun foo/0)'. This is what Thomas meant. In `h(foo)', we don't know whether the atom foo stands for a function or something else - `h(M) -> M:bar().' is also valid code, and in this case `h(foo)' means the same as `foo:bar()', so foo turns out to be a module name. Or you can write `h(Tag) -> receive {Tag, Data} -> Data end.', in this case `h(foo)' means "wait for a message tagged with foo". And you can even choose between these meanings at run time. L. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bombadil@REDACTED Sat Dec 15 16:20:47 2012 From: bombadil@REDACTED (Manuel A. Rubio "Bombadil") Date: Sat, 15 Dec 2012 16:20:47 +0100 Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish In-Reply-To: References: <19414408d3bf707c451df9b4c3e23a89@bosqueviejo.net> Message-ID: Hi, El 2012-12-15 11:12, Max Bourinov escribi?: > Great book! Thanks! > Do you plan to translate it to English? is a good posibility... but my english is very poor, I would need reviewers! :D Thanks again. Manuel Rubio. From naikvin@REDACTED Sat Dec 15 16:36:24 2012 From: naikvin@REDACTED (Vineet Naik) Date: Sat, 15 Dec 2012 21:06:24 +0530 Subject: [erlang-questions] Deploying on a server with different architecture and version of Erlang In-Reply-To: References: Message-ID: On Sat, Dec 15, 2012 at 3:03 AM, Anders Nygren wrote: > On Fri, Dec 14, 2012 at 11:09 AM, Vineet Naik wrote: > > On Fri, Dec 14, 2012 at 9:39 PM, Daniel Luna wrote: > >> > >> If this fixed your problem you have missing dependencies in your > >> .app.src file. We always run {mod_cond, derived} here and it works > >> like a charm. > >> > >> Until you miss to add a dependency in the .app.src file that is. > > > > > > Hi Daniel, > > > > I am am not able to exactly understand which dependencies are to be > added in > > app.src. > > > > My code directly depends upon just two - emysql and exmpp (Not the > > processone version > > but this one [1] which is rebar compatible). Besides these, kernel and > > stdlib are defined. > > > > While creating a release on my development machine I could fix some > initial > > errors by > > adding inets, sasl and crypto. That solved the problems with mod_cond > still > > being 'derived' > > > > Today while creating a release on the server (where the app will be > deployed > > eventually), > > it again failed with `{"init terminating in do_boot",{'cannot > > load',X,get_file}}` kind of errors > > and adding 'X' to app.src didn't work. Changing mod_cond to 'all' worked. > > > > Now if I check the libraries in rel//lib directory, there > > are a lot of them > > but not all are added to app.src and it still works. > > > > How to decide which ones go in app.src and which don't? > > > > The ones to put in the .app.src are the applications that MUST be > started* before Your app is started. > This information is used when the .boot script is created, so that all > the applications are started in the correct order. > > *, Yes, I know the documentation says that stdlib shall always be > included even though stdlib doesn't have any processes that needs to > be started. > Ok, now I get it. In fact it's mentioned in LYSE which is one of the resources I am referring to. >From LYSE[1] {mod_cond, all | app | ebin | derived | none}This controls the module > inclusion policy. Picking none means no modules will be kept. That's not > very useful. The derived option means that Reltool will try to figure out > what modules are used by other modules which are already included and add > them in that case. Setting the option to app will mean that Reltool keeps > all the modules mentioned in the app file and those that were derived. > Setting it to ebin will keep those in the ebin/directory and the derived > ones. Using the option all will be a mix of using ebin and app. That's the > default value. In the process of desperately trying to get the release working I had forgotten of having read this. In free time, I will try creating the release from scratch just using reltool (without rebar). Probably that will make things more clear for me. Thanks [1] http://learnyousomeerlang.com/release-is-the-word#releases-with-reltool > > > > [1]: Rebar compatible fork of exmpp https://github.com/Zert/exmpp > > > > Thanks, > > Vineet > > > >> > >> Cheers, > >> > >> Daniel > >> > >> On 14 December 2012 09:54, Vineet Naik wrote: > >> > IRC to the rescue! Changed mod_cond from derived to all > >> > and it solved the problem. > >> > > >> > Thanks a lot > >> > Vineet > >> > > >> > > >> > On Fri, Dec 14, 2012 at 7:11 PM, Vineet Naik > wrote: > >> >> > >> >> Now I got past this error, it was a mistake from my side. I didn't > >> >> create > >> >> the node on the server before running generate but copied files from > my > >> >> dev machine. After creating node and with some tweaking to the > reltool > >> >> config, > >> >> release is generated on the prod server but fails on startup with, > >> >> > >> >> {"init terminating in do_boot",{'cannot > load',error_handler,get_file}} > >> >> > >> >> From erlang docs[1] > >> >> > >> >> "Init terminating in do_boot ()" - The primitive Erlang boot sequence > >> >> was > >> >> terminated, most probably because the boot script has errors or > cannot > >> >> be > >> >> read. This is usually a configuration error - the system may have > been > >> >> started with a faulty -boot parameter or with a boot script from the > >> >> wrong > >> >> version of OTP. > >> >> > >> >> How can I check whether boot script running from the wrong version of > >> >> OTP > >> >> is > >> >> the problem? > >> >> > >> >> [1]: http://www.erlang.org/doc/apps/erts/crash_dump.html#id71973 > >> >> > >> >> Regards, > >> >> Vineet > >> >> > >> >> On Fri, Dec 14, 2012 at 4:05 PM, Vineet Naik > wrote: > >> >>> > >> >>> Hello, > >> >>> > >> >>> Now that I have successfully built a release of my app using rebar, > I > >> >>> am > >> >>> stuck at > >> >>> deploying the packaged release because the server I want to deploy > to > >> >>> has > >> >>> different > >> >>> version of erlang and different architecture from my development > >> >>> machine. > >> >>> > >> >>> Development Env > >> >>> ------------------------- > >> >>> i686 i686 i386 GNU/Linux > >> >>> Erlang R14B04 > >> >>> erts-5.8.5 > >> >>> > >> >>> Remote Server > >> >>> ---------------------- > >> >>> x86_64 GNU/Linux > >> >>> Erlang R13B03 > >> >>> erts-5.7.4 > >> >>> > >> >>> I tried compiling on the server but it's failing with a series of > >> >>> errors > >> >>> such as > >> >>> > >> >>> ERROR: Unable to generate spec: Mandatory application kernel is not > >> >>> included in [{app,crypto,false,undefined, > >> >>> > >> >>> "/usr/lib/erlang/lib/crypto-1.6.3", > >> >>> > >> >>> ["/usr/lib/erlang/lib/crypto-1.6.3"], > >> >>> > >> >>> "1.6.3","crypto-1.6.3", > >> >>> ..... > >> >>> > >> >>> I have two questions: > >> >>> > >> >>> 1. Is this due to the old version of erlang? And considering that I > >> >>> have > >> >>> ejabberd and rabbitmq-server running in production on the same > server > >> >>> can I upgrade erlang to R14B04 on it without having to take them > down > >> >>> for > >> >>> much > >> >>> time? > >> >>> > >> >>> 2. What is the recommended strategy for developing and deploying > >> >>> erlang > >> >>> code on > >> >>> differently configured servers? > >> >>> > >> >>> Thanks, > >> >>> Vineet > >> >>> > >> >>> > >> >> > >> >> > >> >> > >> >> -- > >> >> Vineet Naik > >> >> > >> >> > >> > > >> > > >> > > >> > -- > >> > Vineet Naik > >> > > >> > > >> > > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > > > > > > > > -- > > Vineet Naik > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -- Vineet Naik -------------- next part -------------- An HTML attachment was scrubbed... URL: From arif@REDACTED Sat Dec 15 17:36:46 2012 From: arif@REDACTED (arif@REDACTED) Date: Sat, 15 Dec 2012 08:36:46 -0800 Subject: [erlang-questions] rpc call and mnesia transaction Message-ID: <3ff96840cd78d2847a065a37c3d518a7.squirrel@fulvetta.riseup.net> Hi, I'm having some trouble with an rpc call in R5B03-1. I have one node, mnesianode@REDACTED, with mnesia running and with a table mytable. If I use rpc to read a record from a shell running on another node, it works fine. (another@REDACTED)1> Fun = fun() -> mnesia:read(mytable, akey) end. #Fun (another@REDACTED)2> rpc:call('mnesianode@REDACTED', mnesia, transaction, [Fun]). {atomic,[{mytable,akey,}]} (another@REDACTED)3> However, if I put the code in a file: -module(anrpctest). -compile(export_all). do() -> F = fun() -> mnesia:read(mytable, akey) end, {atomic, Result} = rpc:call('mnesianode@REDACTED', mnesia, transaction, [F]), Result. and try to execute anrpctest:do(), it fails. (another@REDACTED)4> c(anrpctest). {ok,anrpctest} (another@REDACTED)5> anrpctest:do(). ** exception error: no match of right hand side value {aborted, {undef, [{#Fun,[],[]}, {mnesia_tm,apply_fun,3, [{file,"mnesia_tm.erl"},{line,829}]}, {mnesia_tm,execute_transaction,5, [{file,"mnesia_tm.erl"},{line,809}]}, {rpc,'-handle_call_call/6-fun-0-',5, [{file,"rpc.erl"},{line,203}]}]}} in function anrpctest:do/0 (anrpctest.erl, .... Any idea why this is happening? Thanks and regards Arif From dgud@REDACTED Sat Dec 15 19:27:25 2012 From: dgud@REDACTED (Dan Gudmundsson) Date: Sat, 15 Dec 2012 19:27:25 +0100 Subject: [erlang-questions] rpc call and mnesia transaction In-Reply-To: <3ff96840cd78d2847a065a37c3d518a7.squirrel@fulvetta.riseup.net> References: <3ff96840cd78d2847a065a37c3d518a7.squirrel@fulvetta.riseup.net> Message-ID: Your code is not available on the other node. If that code is not in the code path. But if want you can avoid the rpc if the node will be permanent, it will be slightly slower though. mnesia:start([{extra_db_nodes, [Node]}]), mnesia:transaction(fun() -> mnesia:read(...) end). On Sat, Dec 15, 2012 at 5:36 PM, wrote: > Hi, > > I'm having some trouble with an rpc call in R5B03-1. > > I have one node, mnesianode@REDACTED, with mnesia running and with a table > mytable. > > If I use rpc to read a record from a shell running on another node, it > works fine. > > (another@REDACTED)1> Fun = fun() -> mnesia:read(mytable, akey) end. > #Fun > (another@REDACTED)2> rpc:call('mnesianode@REDACTED', mnesia, transaction, > [Fun]). > {atomic,[{mytable,akey,}]} > (another@REDACTED)3> > > > However, if I put the code in a file: > > -module(anrpctest). > -compile(export_all). > > do() -> > F = fun() -> mnesia:read(mytable, akey) end, > {atomic, Result} = rpc:call('mnesianode@REDACTED', mnesia, transaction, > [F]), > Result. > > and try to execute anrpctest:do(), it fails. > > (another@REDACTED)4> c(anrpctest). > {ok,anrpctest} > (another@REDACTED)5> anrpctest:do(). > ** exception error: no match of right hand side value > {aborted, > {undef, > [{#Fun,[],[]}, > {mnesia_tm,apply_fun,3, > [{file,"mnesia_tm.erl"},{line,829}]}, > {mnesia_tm,execute_transaction,5, > [{file,"mnesia_tm.erl"},{line,809}]}, > {rpc,'-handle_call_call/6-fun-0-',5, > [{file,"rpc.erl"},{line,203}]}]}} > in function anrpctest:do/0 (anrpctest.erl, .... > > Any idea why this is happening? > > Thanks and regards > Arif > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sat Dec 15 21:19:26 2012 From: g@REDACTED (Garrett Smith) Date: Sat, 15 Dec 2012 14:19:26 -0600 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: On Fri, Dec 14, 2012 at 8:22 PM, Saravanan Vijayakumaran wrote: > Dear Max, > There are in fact 10,000 client and server processes. Each > of them is a gen_server [1][2]. These are just simulation entities which are > connected by point-to-point channels [3] which are also a simulation > entities. > > The purpose of network simulation is to guide decision making with respect > to choice of network protocols and parameters in a particular scenario. The > scenarios are typically too expensive to try out in real networks. Imagine a > thousand node network placed in different cities. > > Regarding discrete event simulation, suppose we want to figure out how many > runways to have in a new airport. The more the runways the more costly it is > but it is also will result in less waiting time for arriving/departing > flights. One way to characterize the runway count vs waiting time tradeoff > is to use queueing theory. An alternative is to do simulate flight arrivals > and departures. At any point of time, the simulated world will consists of > flights in transit, flights waiting to depart, flights waiting to land, > flights departing and flights landing. In discrete time simulation, the > state of the simulated world is updated at a regular time step (say every > minute). Note that this is simulated time and not wall-clock time. An > alternative which is much faster to execute is discrete event simulation. > Here there is a queue of simulation events with the earliest events at the > head of the queue. The events are executed one by one by picking the event > at the head of the queue. When a flight departure event is executed, a > waiting to land event is inserted into the queue having a timestamp equal to > current time + the transit time of the flight. When the waiting to land > event is executed, if a runway is free a land event is inserted into the > queue having timestamp equal to the current time + landing time. If a runway > is not free, a circle the airport and check again event is inserted into the > queue having timestamp equal to the current time + circling time. The > purpose of the simulation queue is to provide causal ordering of the > simulation events. > > In my case, I think it is the simulation queue which is becoming the > bottleneck. I must add that network simulation falls under the category of > number-crunching applications which Erlang is not supposed to be suited for. > But people have been trying to accelerate it in C++-based simulators using > pthreads. I thought it was worth a try in Erlang given the amount of work > which has already gone into Erlang SMP. I've lost track of this thread -- are you still guessing, or have you spent any time measuring? I looked back and couldn't see any discussion about e.g. specific processes with high reduction counts, growing queue sizes, etc. Garrett From sarva.v@REDACTED Sun Dec 16 03:23:41 2012 From: sarva.v@REDACTED (Saravanan Vijayakumaran) Date: Sun, 16 Dec 2012 07:53:41 +0530 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: I have been away from my dev box for the past week. I get back tonight. Will let you know in a few days. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Sun Dec 16 10:59:42 2012 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 16 Dec 2012 10:59:42 +0100 Subject: [erlang-questions] unhappy interaction between rebar and appl. controller Message-ID: <3416AF5A-EEF8-4EAD-A6C7-BEBBB7DD25F4@feuerlabs.com> Is 'included_applications' really an environment variable? I have spent some time trying to figure out why one of my eunit tests crashed at the end with the tasty error message: {error_logger,{{2012,12,16},{10,33,28}},"~s~n",["Error in process <0.551.0> with exit value: {terminated,[{io,format,[<0.22.0>,\"Internal error: ~P.\n\",[{error,terminated,[{io,format,[<0.22.0>,\"~s\n\",[undefined]],[]},{eunit_tty,handle_cancel,3,[{file,\"eunit_tty.erl\"},{line,159}]},{eunit_listener,call,3,[{file,\"eunit_listener.erl\"},{line... \n"]} {"Kernel pid terminated",application_controller,{{badmatch,undefined},[{application_controller,unload,2,[{file,"application_controller.erl"},{line,1287}]},{application_controller,handle_call,3,[{file,"application_controller.erl"},{line,650}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,588}]}]}} The crash happens because application_controller:unload/2 is a bit particular about requiring the presence of the 'included_applications' entry: https://github.com/erlang/otp/blob/master/lib/kernel/src/application_controller.erl#L1276 unload(AppName, S) -> {ok, IncApps} = get_env(AppName, included_applications), It took me some time to figure out who actually deleted this entry in the first place. After a bit of tracing (using the {message,caller} match body), I discovered that it was rebar, in rebar_eunit:reset_after_eunit/1. https://github.com/basho/rebar/blob/master/src/rebar_eunit.erl#L612 OldApps = [App || {App, _} <- OldAppEnvs], Apps = get_app_names(), _ = [begin _ = case lists:member(App, OldApps) of true -> ok; false -> application:stop(App) end, ok = application:unset_env(App, K) end || App <- Apps, App /= rebar, {K, _V} <- application:get_all_env(App)], After this, rebar rebuilds the application environment by re-reading the .app file from disk and trying to emulate application:load/1. I see two problems with this: - For historical reasons (originally my 'fault'), 'included_applications' is represented as an environment variable, although it no longer really is one. Rebar has to either exclude it from the unset_env operation or explicitly reconstruct it later. - Application controller needs to make up its mind about whether 'included_applications' is a critical element. If it is, perhaps it shouldn't so readily allow people to delete it? BTW, why doesn't rebar simply call application:unload(App)? Or, if it really needs to reconstruct an app env that wasn't there from the beginning, perhaps application:unload(App) followed by application:load(App)? The problem on my side was that a previous eunit test had done start and stop on an app, but forgot unload. A later test tried the unload and crashed. - I will add unload calls to my first eunit test. - Rebar should not try to call application:unset_env(App, included_applications). - Application_controller should perhaps not crash if included_applications is not present at unload, or better yet, not allow users to delete it through the official API in the first place. - I should find more fun things to do on a Sunday. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From lovei.laszlo@REDACTED Sun Dec 16 11:48:14 2012 From: lovei.laszlo@REDACTED (Laszlo Lovei) Date: Sun, 16 Dec 2012 02:48:14 -0800 (PST) Subject: [erlang-questions] rpc call and mnesia transaction In-Reply-To: <3ff96840cd78d2847a065a37c3d518a7.squirrel@fulvetta.riseup.net> References: <3ff96840cd78d2847a065a37c3d518a7.squirrel@fulvetta.riseup.net> Message-ID: <05b45ac0-8311-43ec-af71-c85850252693@googlegroups.com> Hi, Dan is right, the issue is your code not being available on the remote node, and the reason is the following. In this code: (another@REDACTED)1> Fun = fun() -> mnesia:read(mytable, akey) end. > #Fun > the code referred by `Fun' is in the module `erl_eval' (as you can see from #Fun). It is a symbolic evaluator which evaluates the code you typed in the shell; the code itself is bound to variables in `Fun' (stored in a kind of closure, which is sent over to the remote node during the rpc call). This works because `erl_eval' is also loaded on the remote node. However, in this code: -module(anrpctest). > > do() -> > F = fun() -> mnesia:read(mytable, akey) end, > the code of `F' is compiled into the module `anrpctest', you can also see this from the error message: {aborted, {undef, [{#Fun,[],[]}, > This will work only if you load the module `anrpctest' on the remote node too. (If you are just experimenting, use the l() command in a shell after each recompilation.) L. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Sun Dec 16 12:39:28 2012 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 16 Dec 2012 12:39:28 +0100 Subject: [erlang-questions] more flexible code generation Message-ID: I made a little addition to my parse_trans library, in parse_trans_codegen: The 'original' method for generating code for a function was e.g.: g(Name, V) -> codegen:gen_function( Name, fun(L) -> member({'$var',V}, L) end). Where the {'$var', V} notation is a way to insert a reference to a variable, rather than using the variable itself. To illustrate, let's first create a little pretty-printer fun: 1> PP = fun(F) -> io:fwrite("~s~n", [erl_pp:form(F)]) end. #Fun 2> PP(ex_codegen:g(foo,17)). foo(L) -> member(17, L). There are a few other, similar, functions, but in some cases, this is still too restrictive. A few days ago, I added support for specifying a more dynamic pattern: k(L) -> codegen:gen_function( lcf, [fun({'$var',X}) -> {'$var',Y} end || {X, Y} <- L]). The list comprehension results in a list of clauses based on data resolved at run-time. For example: 3> PP(ex_codegen:k([ {"a", {1,2,3}}, {"b", {4,5,6}} ])). lcf("a") -> {1,2,3}; lcf("b") -> {4,5,6}. https://github.com/uwiger/parse_trans/blob/master/doc/parse_trans_codegen.md Feedback is welcome. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From zabrane3@REDACTED Sun Dec 16 12:40:19 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Sun, 16 Dec 2012 12:40:19 +0100 Subject: [erlang-questions] =?iso-8859-1?q?Identity_and_Access_Management_?= =?iso-8859-1?q?in_Erlang_=28=E0_la_Amazon_AWS=29?= In-Reply-To: References: <1A4A1D39-B7D9-4204-8D78-23B0EE51613C@gmail.com> Message-ID: <2C1AAFAA-3F3C-41CE-8914-0B8864FFAD31@gmail.com> Hi Antoine, > Isn't this oauth ? Was suspecting OAuth, but not sure at all. > There are implementations on github search for erlang oauth. Found multiple ones: https://github.com/tim/erlang-oauth (Zotonic is using this, so it should be the one to pick). https://github.com/opscode/erlang-oauth https://github.com/kivra/oauth2 (<-- OAuth2) https://github.com/kivra/oauth2_client https://github.com/JLarky/erlang-oauth-2.0 Suggestions? > Hope this helps :-) Can someone more experienced in "security" confirm that AMAZON is using OAuth please? If yes, should I use OAuth or OAuth2? I'm bit lost. Regards, Zabrane > On Saturday, December 15, 2012, Zabrane Mickael wrote: > Hi guys, > > I'm looking for an "Identity and Access Management" Erlang API similar to what Amazon offers around the AWS services. > With AWS, you (only) need: > AWS_ACCESS_KEY_ID > AWS_SECRET_ACCESS_KEY > > The idea is to offer our customers a secure access to a bunch of services (similar to AWS). > > Any Erlang project which already handle this? Hints on how implement such an API? > Help much appreciated. > > Regards, > /Z > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sun Dec 16 13:02:53 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Sun, 16 Dec 2012 13:02:53 +0100 Subject: [erlang-questions] Deploying on a server with different architecture and version of Erlang In-Reply-To: References: Message-ID: <50CDB86D.3000402@ninenines.eu> On 12/14/2012 10:33 PM, Anders Nygren wrote: > On Fri, Dec 14, 2012 at 11:09 AM, Vineet Naik wrote: >> On Fri, Dec 14, 2012 at 9:39 PM, Daniel Luna wrote: >>> >>> If this fixed your problem you have missing dependencies in your >>> .app.src file. We always run {mod_cond, derived} here and it works >>> like a charm. >>> >>> Until you miss to add a dependency in the .app.src file that is. >> >> >> Hi Daniel, >> >> I am am not able to exactly understand which dependencies are to be added in >> app.src. >> >> My code directly depends upon just two - emysql and exmpp (Not the >> processone version >> but this one [1] which is rebar compatible). Besides these, kernel and >> stdlib are defined. >> >> While creating a release on my development machine I could fix some initial >> errors by >> adding inets, sasl and crypto. That solved the problems with mod_cond still >> being 'derived' >> >> Today while creating a release on the server (where the app will be deployed >> eventually), >> it again failed with `{"init terminating in do_boot",{'cannot >> load',X,get_file}}` kind of errors >> and adding 'X' to app.src didn't work. Changing mod_cond to 'all' worked. >> >> Now if I check the libraries in rel//lib directory, there >> are a lot of them >> but not all are added to app.src and it still works. >> >> How to decide which ones go in app.src and which don't? >> > > The ones to put in the .app.src are the applications that MUST be > started* before Your app is started. > This information is used when the .boot script is created, so that all > the applications are started in the correct order. > > *, Yes, I know the documentation says that stdlib shall always be > included even though stdlib doesn't have any processes that needs to > be started. Apparently does because stdlib is a library application and all library applications return ok for application:start(App). I suppose you should put all applications you depend on in there, not just applications that must be started. But even with that reltool might not find all the modules it needs if you just use {mod_cond, derived}, especially if some of the modules are only used dynamically. If you really need to use this option, you'll want to check it included everything you use and test it properly. The new option to not include erts in releases is a much better and easier option if you want a small release in my opinion. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From jesper.louis.andersen@REDACTED Sun Dec 16 15:16:57 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 16 Dec 2012 15:16:57 +0100 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: On Dec 15, 2012, at 9:19 PM, Garrett Smith wrote: > I've lost track of this thread -- are you still guessing, or have you > spent any time measuring? I looked back and couldn't see any > discussion about e.g. specific processes with high reduction counts, > growing queue sizes, etc. Sometimes, a well-placed eprof measurement does wonders to tell you what is slow. We had a system that was spending 40% of its time in a specific regular expression once, but this is hard to see unless you profile. I'd say you should look at processes with high reduction counts, or groups of those and then look into why they are reducing all the time. But I have a hunch that this has more to do with the fact that the simulation is serial more than parallel. If you want to utilize more than one core then, you need to figure out how the problem can be split up. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From zabrane3@REDACTED Sun Dec 16 17:11:55 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Sun, 16 Dec 2012 17:11:55 +0100 Subject: [erlang-questions] =?iso-8859-1?q?Identity_and_Access_Management_?= =?iso-8859-1?q?in_Erlang_=28=E0_la_Amazon_AWS=29?= In-Reply-To: <2C1AAFAA-3F3C-41CE-8914-0B8864FFAD31@gmail.com> References: <1A4A1D39-B7D9-4204-8D78-23B0EE51613C@gmail.com> <2C1AAFAA-3F3C-41CE-8914-0B8864FFAD31@gmail.com> Message-ID: <211E7BBA-94FE-4D06-8B59-C32E823E256D@gmail.com> Seems like OAuth2 is weak: http://www.infoq.com/news/2010/09/oauth2-bad-for-web Regards, Zabrane On Dec 16, 2012, at 12:40 PM, Zabrane Mickael wrote: > Hi Antoine, > >> Isn't this oauth ? > > Was suspecting OAuth, but not sure at all. > >> There are implementations on github search for erlang oauth. > > Found multiple ones: > https://github.com/tim/erlang-oauth (Zotonic is using this, so it should be the one to pick). > https://github.com/opscode/erlang-oauth > > https://github.com/kivra/oauth2 (<-- OAuth2) > https://github.com/kivra/oauth2_client > > https://github.com/JLarky/erlang-oauth-2.0 > > Suggestions? > >> Hope this helps :-) > > Can someone more experienced in "security" confirm that AMAZON is using OAuth please? > If yes, should I use OAuth or OAuth2? I'm bit lost. > > Regards, > Zabrane > >> On Saturday, December 15, 2012, Zabrane Mickael wrote: >> Hi guys, >> >> I'm looking for an "Identity and Access Management" Erlang API similar to what Amazon offers around the AWS services. >> With AWS, you (only) need: >> AWS_ACCESS_KEY_ID >> AWS_SECRET_ACCESS_KEY >> >> The idea is to offer our customers a secure access to a bunch of services (similar to AWS). >> >> Any Erlang project which already handle this? Hints on how implement such an API? >> Help much appreciated. >> >> Regards, >> /Z >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rustompmody@REDACTED Sun Dec 16 17:35:08 2012 From: rustompmody@REDACTED (Rustom Mody) Date: Sun, 16 Dec 2012 22:05:08 +0530 Subject: [erlang-questions] question about erlang doc Message-ID: In http://www.erlang.org/doc/man/random.html I see this line: seed(X1 :: {A1, A2, A3}) -> undefined | ran() Whats the '::' ? Also I cant figure out how to read what follows the '->' Please note I am not so much interested in the specifics of random numbers as in being able to read the manual! -------------- next part -------------- An HTML attachment was scrubbed... URL: From jrosenblum@REDACTED Sun Dec 16 17:51:08 2012 From: jrosenblum@REDACTED (Jim) Date: Sun, 16 Dec 2012 11:51:08 -0500 Subject: [erlang-questions] question about erlang doc In-Reply-To: References: Message-ID: <395B9871-9B38-4268-98DB-16B14F2CA5F1@prodigy.net> I believe this is a type specification indicating that the function seed/1 takes a parameter X1 of type (that's what the :: means) 3-tuple ({A1, A2, A3}) and returns (indicated by the ->) undefined or a random number (whatever ran/0 indicates). Notice that A1, A2, A3 are further specified to be integers in the lines below. Sent from my iPad On Dec 16, 2012, at 11:35 AM, Rustom Mody wrote: > In http://www.erlang.org/doc/man/random.html I see this line: > > seed(X1 :: {A1, A2, A3}) -> undefined | ran() > > Whats the '::' ? > Also I cant figure out how to read what follows the '->' > > Please note I am not so much interested in the specifics of random numbers as in being able to read the manual! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy@REDACTED Sun Dec 16 20:12:50 2012 From: jeremy@REDACTED (Jeremy Ong) Date: Sun, 16 Dec 2012 11:12:50 -0800 Subject: [erlang-questions] question about erlang doc In-Reply-To: References: Message-ID: See http://www.erlang.org/doc/reference_manual/typespec.html for more details. Think of the type specification as a sort of function prototype (coming from the C world). On Sun, Dec 16, 2012 at 8:35 AM, Rustom Mody wrote: > In http://www.erlang.org/doc/man/random.html I see this line: > > seed(X1 :: {A1, A2, A3}) -> undefined | ran() > > Whats the '::' ? > Also I cant figure out how to read what follows the '->' > > Please note I am not so much interested in the specifics of random numbers > as in being able to read the manual! > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sven.heyll@REDACTED Sun Dec 16 20:24:16 2012 From: sven.heyll@REDACTED (Sven Heyll) Date: Sun, 16 Dec 2012 20:24:16 +0100 Subject: [erlang-questions] [ANN] erlymock 4.0.0 Message-ID: Hi list, a new release of erlymock was uploaded to github, containing a single new function 'em:await(Mock, ExpectationRef)': http://sheyll.github.com/erlymock/em.html#await-2 It can be used 1. to block the test until some async invokation of a mocked function has happened and 2. to 'capture' the parameters and pid of an invokation. Internally we usually use a fun that send some message to the test process as parameter to em:strict/5 in combination with 'receive' to block the test process. With this release this should not be necessary anymore. Calling 'em:strict/4,5' will now return a reference to the expectation which can be passed to 'em:await/2' to wait for that expecation. I really think, as small as this change seems, it is useful enough to be announced. Also this is release has a new major version: 4.0.0. From now on semantic versioning 2.0.0-rc1 will be followed. Cheers, Sven -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Dec 17 00:52:51 2012 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 17 Dec 2012 12:52:51 +1300 Subject: [erlang-questions] Erlang Process Notation? In-Reply-To: References: Message-ID: http://www.erlang.org/faq/tools.html question 7.3 In particular, http://www.erlang.se/euc/99/Event.ps was _designed_ for Erlang. It should be possible to adapt dia or Violet to this easily enough. From anders.nygren@REDACTED Mon Dec 17 04:48:38 2012 From: anders.nygren@REDACTED (Anders Nygren) Date: Sun, 16 Dec 2012 21:48:38 -0600 Subject: [erlang-questions] Deploying on a server with different architecture and version of Erlang In-Reply-To: <50CDB86D.3000402@ninenines.eu> References: <50CDB86D.3000402@ninenines.eu> Message-ID: On Sun, Dec 16, 2012 at 6:02 AM, Lo?c Hoguin wrote: > On 12/14/2012 10:33 PM, Anders Nygren wrote: >> >> On Fri, Dec 14, 2012 at 11:09 AM, Vineet Naik wrote: >>> >>> On Fri, Dec 14, 2012 at 9:39 PM, Daniel Luna wrote: >>>> >>>> >>>> If this fixed your problem you have missing dependencies in your >>>> .app.src file. We always run {mod_cond, derived} here and it works >>>> like a charm. >>>> >>>> Until you miss to add a dependency in the .app.src file that is. >>> >>> >>> >>> Hi Daniel, >>> >>> I am am not able to exactly understand which dependencies are to be added >>> in >>> app.src. >>> >>> My code directly depends upon just two - emysql and exmpp (Not the >>> processone version >>> but this one [1] which is rebar compatible). Besides these, kernel and >>> stdlib are defined. >>> >>> While creating a release on my development machine I could fix some >>> initial >>> errors by >>> adding inets, sasl and crypto. That solved the problems with mod_cond >>> still >>> being 'derived' >>> >>> Today while creating a release on the server (where the app will be >>> deployed >>> eventually), >>> it again failed with `{"init terminating in do_boot",{'cannot >>> load',X,get_file}}` kind of errors >>> and adding 'X' to app.src didn't work. Changing mod_cond to 'all' worked. >>> >>> Now if I check the libraries in rel//lib directory, there >>> are a lot of them >>> but not all are added to app.src and it still works. >>> >>> How to decide which ones go in app.src and which don't? >>> >> >> The ones to put in the .app.src are the applications that MUST be >> started* before Your app is started. >> This information is used when the .boot script is created, so that all >> the applications are started in the correct order. >> >> *, Yes, I know the documentation says that stdlib shall always be >> included even though stdlib doesn't have any processes that needs to >> be started. > > > Apparently does because stdlib is a library application and all library > applications return ok for application:start(App). I suppose you should put > all applications you depend on in there, not just applications that must be > started. In the documentation it says, in http://www.erlang.org/doc/design_principles/applications.html#id73720 about the .app file "applications All applications which must be started before this application is started. systools uses this list to generate correct boot scripts. Defaults to [], but note that all applications have dependencies to at least kernel and stdlib." a few years ago I complained about an OTP application that did not list a library application that it depended on, but did not have to be started and was told that I was wrong. see, http://erlang.org/pipermail/erlang-bugs/2007-April/000333.html for more details /Anders > > But even with that reltool might not find all the modules it needs if you > just use {mod_cond, derived}, especially if some of the modules are only > used dynamically. If you really need to use this option, you'll want to > check it included everything you use and test it properly. The new option to > not include erts in releases is a much better and easier option if you want > a small release in my opinion. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu From rustompmody@REDACTED Mon Dec 17 05:40:02 2012 From: rustompmody@REDACTED (Rustom Mody) Date: Mon, 17 Dec 2012 10:10:02 +0530 Subject: [erlang-questions] question about erlang doc In-Reply-To: References: Message-ID: On Mon, Dec 17, 2012 at 12:42 AM, Jeremy Ong wrote: > See http://www.erlang.org/doc/reference_manual/typespec.html for more > details. Think of the type specification as a sort of function prototype > (coming from the C world). > Thanks Jeremy. I was looking for (something like) that. Also thank you Jim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Mon Dec 17 09:30:57 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 17 Dec 2012 09:30:57 +0100 Subject: [erlang-questions] init:stop() not working? Message-ID: Hi! I have encountered situations where the beam process keeps running after init:stop() has been called. I can't open a remote shell to that node, so it looks that the node is in a zombie state. When I changed the shutdown command to use erlang:halt() instead, then the vm process died as expected. I don't have an easy way to reproduce this, it happens for some backends started by erlide. Has anyone an explanation for this? Is it something worth exploring? best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Dec 17 10:05:56 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Mon, 17 Dec 2012 10:05:56 +0100 Subject: [erlang-questions] Dependencies on library applications (was: Deploying on a server with different architecture and version of Erlang) In-Reply-To: References: <50CDB86D.3000402@ninenines.eu> Message-ID: <50CEE074.4020908@ninenines.eu> On 12/17/2012 04:48 AM, Anders Nygren wrote: > On Sun, Dec 16, 2012 at 6:02 AM, Lo?c Hoguin wrote: >> On 12/14/2012 10:33 PM, Anders Nygren wrote: >>> >>> On Fri, Dec 14, 2012 at 11:09 AM, Vineet Naik wrote: >>>> >>>> On Fri, Dec 14, 2012 at 9:39 PM, Daniel Luna wrote: >>>>> >>>>> >>>>> If this fixed your problem you have missing dependencies in your >>>>> .app.src file. We always run {mod_cond, derived} here and it works >>>>> like a charm. >>>>> >>>>> Until you miss to add a dependency in the .app.src file that is. >>>> >>>> >>>> >>>> Hi Daniel, >>>> >>>> I am am not able to exactly understand which dependencies are to be added >>>> in >>>> app.src. >>>> >>>> My code directly depends upon just two - emysql and exmpp (Not the >>>> processone version >>>> but this one [1] which is rebar compatible). Besides these, kernel and >>>> stdlib are defined. >>>> >>>> While creating a release on my development machine I could fix some >>>> initial >>>> errors by >>>> adding inets, sasl and crypto. That solved the problems with mod_cond >>>> still >>>> being 'derived' >>>> >>>> Today while creating a release on the server (where the app will be >>>> deployed >>>> eventually), >>>> it again failed with `{"init terminating in do_boot",{'cannot >>>> load',X,get_file}}` kind of errors >>>> and adding 'X' to app.src didn't work. Changing mod_cond to 'all' worked. >>>> >>>> Now if I check the libraries in rel//lib directory, there >>>> are a lot of them >>>> but not all are added to app.src and it still works. >>>> >>>> How to decide which ones go in app.src and which don't? >>>> >>> >>> The ones to put in the .app.src are the applications that MUST be >>> started* before Your app is started. >>> This information is used when the .boot script is created, so that all >>> the applications are started in the correct order. >>> >>> *, Yes, I know the documentation says that stdlib shall always be >>> included even though stdlib doesn't have any processes that needs to >>> be started. >> >> >> Apparently does because stdlib is a library application and all library >> applications return ok for application:start(App). I suppose you should put >> all applications you depend on in there, not just applications that must be >> started. > > In the documentation it says, in > http://www.erlang.org/doc/design_principles/applications.html#id73720 > about the .app file > "applications > All applications which must be started before this application is > started. systools uses this list to generate correct boot scripts. > Defaults to [], but note that all applications have dependencies to at > least kernel and stdlib." > > a few years ago I complained about an OTP application that did not > list a library application that it depended on, but did not have to be > started and was told that I was wrong. > > see, http://erlang.org/pipermail/erlang-bugs/2007-April/000333.html > for more details Alright. But, directing my question to OTP guys now, wouldn't it be simpler if we'd list library applications there too? Building releases would be much easier if all dependencies were listed somewhere and it appears this has no negative impact to put them in applications. Correct me if I'm wrong. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From watson.timothy@REDACTED Mon Dec 17 10:38:26 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 17 Dec 2012 09:38:26 +0000 Subject: [erlang-questions] more flexible code generation In-Reply-To: References: Message-ID: <9EEE8568-D7B5-478E-9A29-C9ACEC0B2A9A@gmail.com> Ulf - some feedback: this is totally AWESOME! Damn I need to find time to go and refactor at least three projects to take advantage of this now.... :D On 16 Dec 2012, at 11:39, Ulf Wiger wrote: > > I made a little addition to my parse_trans library, in parse_trans_codegen: > > The 'original' method for generating code for a function was e.g.: > > g(Name, V) -> > codegen:gen_function( > Name, > fun(L) -> > member({'$var',V}, L) > end). > > Where the {'$var', V} notation is a way to insert a reference to a variable, rather than using the variable itself. > > To illustrate, let's first create a little pretty-printer fun: > > 1> PP = fun(F) -> io:fwrite("~s~n", [erl_pp:form(F)]) end. > #Fun > > > 2> PP(ex_codegen:g(foo,17)). > foo(L) -> > member(17, L). > > There are a few other, similar, functions, but in some cases, this is still too restrictive. A few days ago, I added support for specifying a more dynamic pattern: > > k(L) -> > codegen:gen_function( > lcf, > [fun({'$var',X}) -> > {'$var',Y} > end || {X, Y} <- L]). > > The list comprehension results in a list of clauses based on data resolved at run-time. For example: > > 3> PP(ex_codegen:k([ {"a", {1,2,3}}, {"b", {4,5,6}} ])). > lcf("a") -> > {1,2,3}; > lcf("b") -> > {4,5,6}. > > https://github.com/uwiger/parse_trans/blob/master/doc/parse_trans_codegen.md > > Feedback is welcome. > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From rustompmody@REDACTED Mon Dec 17 13:21:03 2012 From: rustompmody@REDACTED (Rustom Mody) Date: Mon, 17 Dec 2012 17:51:03 +0530 Subject: [erlang-questions] Erlang Process Notation? In-Reply-To: References: Message-ID: On Mon, Dec 17, 2012 at 5:22 AM, Richard O'Keefe wrote: > http://www.erlang.org/faq/tools.html > question 7.3 > > In particular, > http://www.erlang.se/euc/99/Event.ps > was _designed_ for Erlang. > > It should be possible to adapt dia or Violet to this easily enough. > Is there any attempt to map Petri nets to Erlang? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Mon Dec 17 14:09:36 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 17 Dec 2012 08:09:36 -0500 Subject: [erlang-questions] init:stop() not working? In-Reply-To: References: Message-ID: <20121217130935.GA44779@ferdmbp.local> Hi Vlad, Whenever I've had this problem in the past, it was due to someone (likely me) not putting all dependencies in the .app file of my application, most notably stdlib and kernel when using a release. The applications are started in a given order, and when calling init:stop/0-1 to shut down the VM, the application controller goes through the list in reverse and terminates and unload applications and libraries in that order. Thus if you have an app where you don't have your dependencies on stdlib and kernel set, there's a possibility one of your libraries gets loaded before them (and thus unloaded after them), if I recall correctly. That ends up being an application that is technically still running, but not actually able to run. The VM ends up looking like a zombie. That's the explanation I think I got by reading the source a bit more than a year ago. I do remember that putting both kernel and stdlib in your app dependencies and re-building the release would solve that issue. I'm not sure if it applies to your current case and have no idea what ErlIDE is doing though. Regards, Fred. On 12/17, Vlad Dumitrescu wrote: > Hi! > > I have encountered situations where the beam process keeps running after > init:stop() has been called. I can't open a remote shell to that node, so > it looks that the node is in a zombie state. When I changed the shutdown > command to use erlang:halt() instead, then the vm process died as expected. > > I don't have an easy way to reproduce this, it happens for some backends > started by erlide. > > Has anyone an explanation for this? Is it something worth exploring? > > best regards, > Vlad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vladdu55@REDACTED Mon Dec 17 15:09:57 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 17 Dec 2012 15:09:57 +0100 Subject: [erlang-questions] init:stop() not working? In-Reply-To: <20121217130935.GA44779@ferdmbp.local> References: <20121217130935.GA44779@ferdmbp.local> Message-ID: Hi Fred, Thanks for the input. We don't have an application structure, just loading modules freely. The thing is that only some of the vms get in this state, so my first reaction was that it's something nondeterministic, like hanging on some socket or something like that. But it happens deterministically enough (the same kind of instance gets zombified) to make that less probable. regards, Vlad On Mon, Dec 17, 2012 at 2:09 PM, Fred Hebert wrote: > Hi Vlad, > > Whenever I've had this problem in the past, it was due to someone > (likely me) not putting all dependencies in the .app file of my > application, most notably stdlib and kernel when using a release. > > The applications are started in a given order, and when calling > init:stop/0-1 to shut down the VM, the application controller goes > through the list in reverse and terminates and unload applications and > libraries in that order. > > Thus if you have an app where you don't have your dependencies on stdlib > and kernel set, there's a possibility one of your libraries gets loaded > before them (and thus unloaded after them), if I recall correctly. That > ends up being an application that is technically still running, but not > actually able to run. The VM ends up looking like a zombie. > > That's the explanation I think I got by reading the source a bit more > than a year ago. I do remember that putting both kernel and stdlib in > your app dependencies and re-building the release would solve that > issue. > > I'm not sure if it applies to your current case and have no idea what > ErlIDE is doing though. > > Regards, > Fred. > > On 12/17, Vlad Dumitrescu wrote: > > Hi! > > > > I have encountered situations where the beam process keeps running after > > init:stop() has been called. I can't open a remote shell to that node, so > > it looks that the node is in a zombie state. When I changed the shutdown > > command to use erlang:halt() instead, then the vm process died as > expected. > > > > I don't have an easy way to reproduce this, it happens for some backends > > started by erlide. > > > > Has anyone an explanation for this? Is it something worth exploring? > > > > best regards, > > Vlad > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jean-sebastien.pedron@REDACTED Mon Dec 17 15:13:15 2012 From: jean-sebastien.pedron@REDACTED (=?ISO-8859-1?Q?Jean-S=E9bastien_P=E9dron?=) Date: Mon, 17 Dec 2012 15:13:15 +0100 Subject: [erlang-questions] ANN: yamerl - YAML parser in pure Erlang Message-ID: <50CF287B.2020309@dumbbell.fr> Hi all, I'm pleased to release yamerl, our YAML parser, written in pure Erlang (no port driver or NIF). This parser supports YAML 1.1 and YAML 1.2, as well as JSON. The code, documentation and testsuite are available on GitHub: https://github.com/yakaz/yamerl Here are some of its features: o Support for stream parsing. o Support for YAML 1.2. This version isn't widely supported in other libraries/languages. Compared to 1.1, it mostly brings official support of JSON as a subset to YAML and several "bugfixes". o Erlang atom node type. It supports options to autodetect atoms and to use list_to_existing_atom/1. Autodetection differs from yamler[1]'s behavior: single-quoted scalar are not implicitly converted to atoms, because, from my understanding, it breaks the YAML specifications. o Erlang fun() node type. Compared to yamler[1], which uses a NIF above libyaml, yamerl has the common advantages and caveats we find in native-code-based vs. pure-erlang-based implementations: o yamerl is slower than yamler. o yamerl may scale better (though I never verified this). o yamerl won't take down the whole VM because of a crash. Everything is distributed under the terms of the 2-clause BSD license. [1] yamler: https://github.com/goertzenator/yamler -- Jean-S?bastien P?dron Yakaz - http://www.yakaz.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From g@REDACTED Mon Dec 17 15:20:51 2012 From: g@REDACTED (Garrett Smith) Date: Mon, 17 Dec 2012 08:20:51 -0600 Subject: [erlang-questions] Dependencies on library applications (was: Deploying on a server with different architecture and version of Erlang) In-Reply-To: <50CEE074.4020908@ninenines.eu> References: <50CDB86D.3000402@ninenines.eu> <50CEE074.4020908@ninenines.eu> Message-ID: On Mon, Dec 17, 2012 at 3:05 AM, Lo?c Hoguin wrote: -snip- > On 12/17/2012 04:48 AM, Anders Nygren wrote: >> In the documentation it says, in >> http://www.erlang.org/doc/design_principles/applications.html#id73720 >> about the .app file >> "applications >> All applications which must be started before this application is >> started. systools uses this list to generate correct boot scripts. >> Defaults to [], but note that all applications have dependencies to at >> least kernel and stdlib." >> >> a few years ago I complained about an OTP application that did not >> list a library application that it depended on, but did not have to be >> started and was told that I was wrong. >> >> see, http://erlang.org/pipermail/erlang-bugs/2007-April/000333.html >> for more details > > > Alright. > > But, directing my question to OTP guys now, wouldn't it be simpler if we'd > list library applications there too? Building releases would be much easier > if all dependencies were listed somewhere and it appears this has no > negative impact to put them in applications. Correct me if I'm wrong. Are you not talking about included applications? http://www.erlang.org/doc/design_principles/included_applications.html Garrett From essen@REDACTED Mon Dec 17 15:25:56 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 17 Dec 2012 15:25:56 +0100 Subject: [erlang-questions] Dependencies on library applications In-Reply-To: References: <50CDB86D.3000402@ninenines.eu> <50CEE074.4020908@ninenines.eu> Message-ID: <50CF2B74.1070503@ninenines.eu> On 12/17/2012 03:20 PM, Garrett Smith wrote: > On Mon, Dec 17, 2012 at 3:05 AM, Lo?c Hoguin wrote: > > -snip- > >> On 12/17/2012 04:48 AM, Anders Nygren wrote: >>> In the documentation it says, in >>> http://www.erlang.org/doc/design_principles/applications.html#id73720 >>> about the .app file >>> "applications >>> All applications which must be started before this application is >>> started. systools uses this list to generate correct boot scripts. >>> Defaults to [], but note that all applications have dependencies to at >>> least kernel and stdlib." >>> >>> a few years ago I complained about an OTP application that did not >>> list a library application that it depended on, but did not have to be >>> started and was told that I was wrong. >>> >>> see, http://erlang.org/pipermail/erlang-bugs/2007-April/000333.html >>> for more details >> >> >> Alright. >> >> But, directing my question to OTP guys now, wouldn't it be simpler if we'd >> list library applications there too? Building releases would be much easier >> if all dependencies were listed somewhere and it appears this has no >> negative impact to put them in applications. Correct me if I'm wrong. > > Are you not talking about included applications? > > http://www.erlang.org/doc/design_principles/included_applications.html No. Included applications are normal applications that are included into another application's supervision tree directly. Library applications are applications that do not need to be started (like stdlib). Problem is with the exception of stdlib it's recommended to not put library applications in the {applications, [...]} startup dependencies, which means there is no way to know all the dependencies of an application without some complex analysis of the source. However, if you go against current recommendations and put them in the startup dependencies (even though they don't need to be started, it still works), then everything is much clearer. But I do not know if there's more concerns than just starting and stopping. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From sverker.eriksson@REDACTED Mon Dec 17 16:10:55 2012 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Mon, 17 Dec 2012 16:10:55 +0100 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? In-Reply-To: <20121214211117.GA86501@alumni.caltech.edu> References: <20121206193809.GC27411@alumni.caltech.edu> <15C0D662-A77C-4858-AD93-3306E72317F6@erlang-solutions.com> <20121212182632.GB72594@alumni.caltech.edu> <20121212234803.GA75710@alumni.caltech.edu> <20121213002455.GA75861@alumni.caltech.edu> <20121213183830.GA81730@alumni.caltech.edu> <20121214211117.GA86501@alumni.caltech.edu> Message-ID: <50CF35FF.6020101@erix.ericsson.se> Have I understood it correctly so far if I say that my commit dc5f7190 seems to cause a {tcp_error,Port,emsgsize} while earlier versions caused a {http,Port,{http_error,"..."}} for the same scenario with some sort of very long http lines? /Sverker, Erlang/OTP Anthony Molinaro wrote: > On Fri, Dec 14, 2012 at 11:32:11AM -0500, Steve Vinoski wrote: > >> Hi Anthony, based on your example, it looks like a follow-up commit that >> modified some of the changes introduced by my patch introduced the issue >> you're seeing: commit dc5f7190. You might want to report it over on >> erlang-bugs. >> > > Nice catch, I guess that it's actually not enough to return error > for backward compatibility ;) > > I think the main thing I'm still not sure of is if the decode_packet changes > are actually the cause. I was able to call decode_packet with long headers > and not actually see an issue. > > This > > erlang:decode_packet ( httph, > list_to_binary ([ "X-Random: ", > [$a || _ <- lists:seq(1,10000)], > "\r\n\r\n" > ]), > []). > > Returns the same thing in both R14B04 and R15B02. And as far as I am able > to tell that is the only call made to decode_packet in mochiweb. > > I think the actual error is around recvbuf sizing (mochiweb uses 8192 as > the default, so 10000 is definitely larger than that, however, the standard > libraries must be doing something different as now it returns emsgsize > where it did no in the past). I really wish there was a way to search > commits on github as then I could search for recvbuf or something like > that to see what might have changed. > > Before I go over to bugs I'll probably try to detangle mochiweb enough > to create a small reproducible test case. > > I also have a pull request into mochiweb which prevents the crash but > has the annoying behavior at the moment that it returns an error to the > client about disconnecting version the 400 error it returned before. > > https://github.com/mochi/mochiweb/pull/91 > > But since the standard library seems to be the one closing the socket > there's not much that can be done other than the fix I put in place. > > Thanks for your help Steve, > > -Anthony > > From dmkolesnikov@REDACTED Mon Dec 17 16:20:28 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Mon, 17 Dec 2012 17:20:28 +0200 Subject: [erlang-questions] Dependencies on library applications In-Reply-To: <50CF2B74.1070503@ninenines.eu> References: <50CDB86D.3000402@ninenines.eu> <50CEE074.4020908@ninenines.eu> <50CF2B74.1070503@ninenines.eu> Message-ID: <1555005282570617532@unknownmsgid> Hello all, I am using exclude mode or reltool and manually managing dependencies of my releases, even including library application. There was not any issue so far. Best Regards, Dmitry >-|-|-*> On 17.12.2012, at 16.26, "Lo?c Hoguin" wrote: > On 12/17/2012 03:20 PM, Garrett Smith wrote: >> On Mon, Dec 17, 2012 at 3:05 AM, Lo?c Hoguin wrote: >> >> -snip- >> >>> On 12/17/2012 04:48 AM, Anders Nygren wrote: >>>> In the documentation it says, in >>>> http://www.erlang.org/doc/design_principles/applications.html#id73720 >>>> about the .app file >>>> "applications >>>> All applications which must be started before this application is >>>> started. systools uses this list to generate correct boot scripts. >>>> Defaults to [], but note that all applications have dependencies to at >>>> least kernel and stdlib." >>>> >>>> a few years ago I complained about an OTP application that did not >>>> list a library application that it depended on, but did not have to be >>>> started and was told that I was wrong. >>>> >>>> see, http://erlang.org/pipermail/erlang-bugs/2007-April/000333.html >>>> for more details >>> >>> >>> Alright. >>> >>> But, directing my question to OTP guys now, wouldn't it be simpler if we'd >>> list library applications there too? Building releases would be much easier >>> if all dependencies were listed somewhere and it appears this has no >>> negative impact to put them in applications. Correct me if I'm wrong. >> >> Are you not talking about included applications? >> >> http://www.erlang.org/doc/design_principles/included_applications.html > > No. > > Included applications are normal applications that are included into another application's supervision tree directly. > > Library applications are applications that do not need to be started (like stdlib). > > Problem is with the exception of stdlib it's recommended to not put library applications in the {applications, [...]} startup dependencies, which means there is no way to know all the dependencies of an application without some complex analysis of the source. However, if you go against current recommendations and put them in the startup dependencies (even though they don't need to be started, it still works), then everything is much clearer. > > But I do not know if there's more concerns than just starting and stopping. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Mon Dec 17 16:22:19 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 17 Dec 2012 16:22:19 +0100 Subject: [erlang-questions] Dependencies on library applications In-Reply-To: <1555005282570617532@unknownmsgid> References: <50CDB86D.3000402@ninenines.eu> <50CEE074.4020908@ninenines.eu> <50CF2B74.1070503@ninenines.eu> <1555005282570617532@unknownmsgid> Message-ID: <50CF38AB.1000503@ninenines.eu> It's not an issue, it's just tedious and has to be done for all releases instead of just once in the .app file. On 12/17/2012 04:20 PM, dmitry kolesnikov wrote: > Hello all, > > I am using exclude mode or reltool and manually managing dependencies > of my releases, even including library application. There was not any > issue so far. > > Best Regards, > Dmitry >-|-|-*> > > > On 17.12.2012, at 16.26, "Lo?c Hoguin" wrote: > >> On 12/17/2012 03:20 PM, Garrett Smith wrote: >>> On Mon, Dec 17, 2012 at 3:05 AM, Lo?c Hoguin wrote: >>> >>> -snip- >>> >>>> On 12/17/2012 04:48 AM, Anders Nygren wrote: >>>>> In the documentation it says, in >>>>> http://www.erlang.org/doc/design_principles/applications.html#id73720 >>>>> about the .app file >>>>> "applications >>>>> All applications which must be started before this application is >>>>> started. systools uses this list to generate correct boot scripts. >>>>> Defaults to [], but note that all applications have dependencies to at >>>>> least kernel and stdlib." >>>>> >>>>> a few years ago I complained about an OTP application that did not >>>>> list a library application that it depended on, but did not have to be >>>>> started and was told that I was wrong. >>>>> >>>>> see, http://erlang.org/pipermail/erlang-bugs/2007-April/000333.html >>>>> for more details >>>> >>>> >>>> Alright. >>>> >>>> But, directing my question to OTP guys now, wouldn't it be simpler if we'd >>>> list library applications there too? Building releases would be much easier >>>> if all dependencies were listed somewhere and it appears this has no >>>> negative impact to put them in applications. Correct me if I'm wrong. >>> >>> Are you not talking about included applications? >>> >>> http://www.erlang.org/doc/design_principles/included_applications.html >> >> No. >> >> Included applications are normal applications that are included into another application's supervision tree directly. >> >> Library applications are applications that do not need to be started (like stdlib). >> >> Problem is with the exception of stdlib it's recommended to not put library applications in the {applications, [...]} startup dependencies, which means there is no way to know all the dependencies of an application without some complex analysis of the source. However, if you go against current recommendations and put them in the startup dependencies (even though they don't need to be started, it still works), then everything is much clearer. >> >> But I do not know if there's more concerns than just starting and stopping. >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From daniel@REDACTED Mon Dec 17 16:34:15 2012 From: daniel@REDACTED (Daniel Luna) Date: Mon, 17 Dec 2012 10:34:15 -0500 Subject: [erlang-questions] Any tcp related changes in R15B* that might cause this? In-Reply-To: <20121214211117.GA86501@alumni.caltech.edu> References: <20121206193809.GC27411@alumni.caltech.edu> <15C0D662-A77C-4858-AD93-3306E72317F6@erlang-solutions.com> <20121212182632.GB72594@alumni.caltech.edu> <20121212234803.GA75710@alumni.caltech.edu> <20121213002455.GA75861@alumni.caltech.edu> <20121213183830.GA81730@alumni.caltech.edu> <20121214211117.GA86501@alumni.caltech.edu> Message-ID: On 14 December 2012 16:11, Anthony Molinaro wrote: > On Fri, Dec 14, 2012 at 11:32:11AM -0500, Steve Vinoski wrote: > I really wish there was a way to search > commits on github as then I could search for recvbuf or something like > that to see what might have changed. If you have a local clone you can use the git pickaxe. Basically adding -S and your string to search for. To just list the commit names you can use "git log -S"recvbuf" --pretty=oneline --abbrev-commit" I usually have 'tig' installed and use it thus: tig -S"recvbuf" which will give you every single commit in your history that was added or deleted (including changed obviously since that is a delete and an add), containing the word recvbuf. Cheers, Daniel From g@REDACTED Mon Dec 17 16:45:15 2012 From: g@REDACTED (Garrett Smith) Date: Mon, 17 Dec 2012 09:45:15 -0600 Subject: [erlang-questions] Erlang Process Notation? In-Reply-To: References: Message-ID: On Fri, Dec 14, 2012 at 1:51 AM, Max Bourinov wrote: > Hi Erlangers, > > I am wondering if there is any notation for Erlang processes? I want to use > it to describe my systems visually. Something like UML class diagrams but > for Erlang (I know that class diagram is not the best example). > > p.s.: I heavily use sequence diagrams and they help a lot. This was a question I had once a while back. After a few years working with Erlang, I haven't seen anything that I personally would consider all that useful. But I have seen complex Erlang systems -- that work! All without a single diagram. Does *not* having sophisticated design tools actually us build sophisticated systems? This sounds like the start of a Zen meditation :) But in my experience, there's a certain threshold of complexity that signals "stop and rethink". Then I work very hard to avoid that complexity -- even at the expense of functionality. I know that software that reaches that threshold is garbage, not because it's complex, but because *I* don't understand it. Will a diagram help my brain in that case? For me, no. Never. Only working software that I can see and understand will help me. Of course everyone's different. Between functions and processes (actors, agents, services, etc.) Erlang gives you the ability to create small, easily understandable components that, when assembled, create stunningly complex, hard to understand systems -- that work surprisingly well. But of course if diagrams help, use them! But do you need a specialized notation? Garrett From gleber.p@REDACTED Mon Dec 17 16:54:14 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 17 Dec 2012 16:54:14 +0100 Subject: [erlang-questions] ANN: yamerl - YAML parser in pure Erlang In-Reply-To: <50CF287B.2020309@dumbbell.fr> References: <50CF287B.2020309@dumbbell.fr> Message-ID: Great timing! I am currently extending my CI server with support of .travis.yml file and I was in need of a good Yaml parsing library :) Thank you! On Mon, Dec 17, 2012 at 3:13 PM, Jean-S?bastien P?dron wrote: > Hi all, > > I'm pleased to release yamerl, our YAML parser, written in pure Erlang > (no port driver or NIF). > > This parser supports YAML 1.1 and YAML 1.2, as well as JSON. The code, > documentation and testsuite are available on GitHub: > https://github.com/yakaz/yamerl > > Here are some of its features: > o Support for stream parsing. > o Support for YAML 1.2. This version isn't widely supported in > other libraries/languages. Compared to 1.1, it mostly brings > official support of JSON as a subset to YAML and several > "bugfixes". > o Erlang atom node type. It supports options to autodetect atoms > and to use list_to_existing_atom/1. Autodetection differs from > yamler[1]'s behavior: single-quoted scalar are not implicitly > converted to atoms, because, from my understanding, it breaks > the YAML specifications. > o Erlang fun() node type. > > Compared to yamler[1], which uses a NIF above libyaml, yamerl has the > common advantages and caveats we find in native-code-based vs. > pure-erlang-based implementations: > o yamerl is slower than yamler. > o yamerl may scale better (though I never verified this). > o yamerl won't take down the whole VM because of a crash. > > Everything is distributed under the terms of the 2-clause BSD license. > > [1] yamler: https://github.com/goertzenator/yamler > > -- > Jean-S?bastien P?dron > Yakaz - http://www.yakaz.com/ > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From g@REDACTED Mon Dec 17 16:54:14 2012 From: g@REDACTED (Garrett Smith) Date: Mon, 17 Dec 2012 09:54:14 -0600 Subject: [erlang-questions] Dependencies on library applications In-Reply-To: <50CF2B74.1070503@ninenines.eu> References: <50CDB86D.3000402@ninenines.eu> <50CEE074.4020908@ninenines.eu> <50CF2B74.1070503@ninenines.eu> Message-ID: On Mon, Dec 17, 2012 at 8:25 AM, Lo?c Hoguin wrote: > On 12/17/2012 03:20 PM, Garrett Smith wrote: >> >> On Mon, Dec 17, 2012 at 3:05 AM, Lo?c Hoguin wrote: >> >> -snip- >> >>> On 12/17/2012 04:48 AM, Anders Nygren wrote: >>>> >>>> In the documentation it says, in >>>> http://www.erlang.org/doc/design_principles/applications.html#id73720 >>>> about the .app file >>>> "applications >>>> All applications which must be started before this application is >>>> started. systools uses this list to generate correct boot scripts. >>>> Defaults to [], but note that all applications have dependencies to at >>>> least kernel and stdlib." >>>> >>>> a few years ago I complained about an OTP application that did not >>>> list a library application that it depended on, but did not have to be >>>> started and was told that I was wrong. >>>> >>>> see, http://erlang.org/pipermail/erlang-bugs/2007-April/000333.html >>>> for more details >>> >>> >>> >>> Alright. >>> >>> But, directing my question to OTP guys now, wouldn't it be simpler if >>> we'd >>> list library applications there too? Building releases would be much >>> easier >>> if all dependencies were listed somewhere and it appears this has no >>> negative impact to put them in applications. Correct me if I'm wrong. >> >> >> Are you not talking about included applications? >> >> http://www.erlang.org/doc/design_principles/included_applications.html > > No. > > Included applications are normal applications that are included into another > application's supervision tree directly. You're just quoting the docs :) They don't *have* to be included in anything -- but they are nonetheless listed as a dependency. The point is that they aren't started by init on behalf of your app. Does listing the app in the included_applications list present a particular problem for your case? Garrett From essen@REDACTED Mon Dec 17 17:00:43 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Mon, 17 Dec 2012 17:00:43 +0100 Subject: [erlang-questions] Dependencies on library applications In-Reply-To: References: <50CDB86D.3000402@ninenines.eu> <50CEE074.4020908@ninenines.eu> <50CF2B74.1070503@ninenines.eu> Message-ID: <50CF41AB.4080905@ninenines.eu> On 12/17/2012 04:54 PM, Garrett Smith wrote: > On Mon, Dec 17, 2012 at 8:25 AM, Lo?c Hoguin wrote: >> On 12/17/2012 03:20 PM, Garrett Smith wrote: >>> >>> On Mon, Dec 17, 2012 at 3:05 AM, Lo?c Hoguin wrote: >>> >>> -snip- >>> >>>> On 12/17/2012 04:48 AM, Anders Nygren wrote: >>>>> >>>>> In the documentation it says, in >>>>> http://www.erlang.org/doc/design_principles/applications.html#id73720 >>>>> about the .app file >>>>> "applications >>>>> All applications which must be started before this application is >>>>> started. systools uses this list to generate correct boot scripts. >>>>> Defaults to [], but note that all applications have dependencies to at >>>>> least kernel and stdlib." >>>>> >>>>> a few years ago I complained about an OTP application that did not >>>>> list a library application that it depended on, but did not have to be >>>>> started and was told that I was wrong. >>>>> >>>>> see, http://erlang.org/pipermail/erlang-bugs/2007-April/000333.html >>>>> for more details >>>> >>>> >>>> >>>> Alright. >>>> >>>> But, directing my question to OTP guys now, wouldn't it be simpler if >>>> we'd >>>> list library applications there too? Building releases would be much >>>> easier >>>> if all dependencies were listed somewhere and it appears this has no >>>> negative impact to put them in applications. Correct me if I'm wrong. >>> >>> >>> Are you not talking about included applications? >>> >>> http://www.erlang.org/doc/design_principles/included_applications.html >> >> No. >> >> Included applications are normal applications that are included into another >> application's supervision tree directly. > > You're just quoting the docs :) No I know about them, because... > They don't *have* to be included in anything -- but they are > nonetheless listed as a dependency. The point is that they aren't > started by init on behalf of your app. > > Does listing the app in the included_applications list present a > particular problem for your case? AFAIK, you can only have one application defined as an included_application anywhere. You can't have 2 applications depend on app_X with both listing it in included_applications. That's pretty much all I know about them, but maybe I'm wrong. :) -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From g@REDACTED Mon Dec 17 17:22:50 2012 From: g@REDACTED (Garrett Smith) Date: Mon, 17 Dec 2012 10:22:50 -0600 Subject: [erlang-questions] Dependencies on library applications In-Reply-To: <50CF41AB.4080905@ninenines.eu> References: <50CDB86D.3000402@ninenines.eu> <50CEE074.4020908@ninenines.eu> <50CF2B74.1070503@ninenines.eu> <50CF41AB.4080905@ninenines.eu> Message-ID: On Mon, Dec 17, 2012 at 10:00 AM, Lo?c Hoguin wrote: > On 12/17/2012 04:54 PM, Garrett Smith wrote: >> >> On Mon, Dec 17, 2012 at 8:25 AM, Lo?c Hoguin wrote: >>> >>> On 12/17/2012 03:20 PM, Garrett Smith wrote: >>>> >>>> >>>> On Mon, Dec 17, 2012 at 3:05 AM, Lo?c Hoguin wrote: >>>> >>>> -snip- >>>> >>>>> On 12/17/2012 04:48 AM, Anders Nygren wrote: >>>>>> >>>>>> >>>>>> In the documentation it says, in >>>>>> http://www.erlang.org/doc/design_principles/applications.html#id73720 >>>>>> about the .app file >>>>>> "applications >>>>>> All applications which must be started before this application is >>>>>> started. systools uses this list to generate correct boot scripts. >>>>>> Defaults to [], but note that all applications have dependencies to at >>>>>> least kernel and stdlib." >>>>>> >>>>>> a few years ago I complained about an OTP application that did not >>>>>> list a library application that it depended on, but did not have to be >>>>>> started and was told that I was wrong. >>>>>> >>>>>> see, http://erlang.org/pipermail/erlang-bugs/2007-April/000333.html >>>>>> for more details >>>>> >>>>> >>>>> >>>>> >>>>> Alright. >>>>> >>>>> But, directing my question to OTP guys now, wouldn't it be simpler if >>>>> we'd >>>>> list library applications there too? Building releases would be much >>>>> easier >>>>> if all dependencies were listed somewhere and it appears this has no >>>>> negative impact to put them in applications. Correct me if I'm wrong. >>>> >>>> >>>> >>>> Are you not talking about included applications? >>>> >>>> http://www.erlang.org/doc/design_principles/included_applications.html >>> >>> >>> No. >>> >>> Included applications are normal applications that are included into >>> another >>> application's supervision tree directly. >> >> >> You're just quoting the docs :) > > > No I know about them, because... > > >> They don't *have* to be included in anything -- but they are >> nonetheless listed as a dependency. The point is that they aren't >> started by init on behalf of your app. >> >> Does listing the app in the included_applications list present a >> particular problem for your case? > > > AFAIK, you can only have one application defined as an included_application > anywhere. You can't have 2 applications depend on app_X with both listing it > in included_applications. > > That's pretty much all I know about them, but maybe I'm wrong. :) You're right! I just tried this and it indeed does not work. I was hoping it was something in the docs that wasn't enforced by systools. So yeah, I totally agree, Erlang should have this :) Garrett From g@REDACTED Mon Dec 17 19:31:34 2012 From: g@REDACTED (Garrett Smith) Date: Mon, 17 Dec 2012 12:31:34 -0600 Subject: [erlang-questions] Chicago Erlang meetup tonight Message-ID: If you're in the Chicago area tonight and don't have plans, feel free to stop by Jefferson Tap for a beer centered Erlang extravaganza! Details here: http://www.meetup.com/ErlangChicago/events/94997122/ Garrett P.S. Soft drinks available! From yashgt@REDACTED Mon Dec 17 21:27:01 2012 From: yashgt@REDACTED (Yash Ganthe) Date: Mon, 17 Dec 2012 15:27:01 -0500 Subject: [erlang-questions] Custom performance counters in Erlang Message-ID: Hi, The perfmon utility on Windows allows an administrator to monitor system-level metrics such as CPU and memory consumption as well as application -level metrics such as number of records processed, number of errors, etc. What is the best way to achieve this in Erlang? How to get application-level metrics logged to the perfmon? Thanks, Yash -------------- next part -------------- An HTML attachment was scrubbed... URL: From jpuig@REDACTED Mon Dec 17 23:35:23 2012 From: jpuig@REDACTED (Juan Puig) Date: Mon, 17 Dec 2012 14:35:23 -0800 Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish In-Reply-To: References: <19414408d3bf707c451df9b4c3e23a89@bosqueviejo.net> Message-ID: Hi, I could help you out with that. /Juan -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Manuel A. Rubio "Bombadil" Sent: Saturday, December 15, 2012 7:21 AM To: Max Bourinov Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] [ANN] Erlang/OTP book in spanish Hi, El 2012-12-15 11:12, Max Bourinov escribi?: > Great book! Thanks! > Do you plan to translate it to English? is a good posibility... but my english is very poor, I would need reviewers! :D Thanks again. Manuel Rubio. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From dmercer@REDACTED Mon Dec 17 23:53:01 2012 From: dmercer@REDACTED (David Mercer) Date: Mon, 17 Dec 2012 16:53:01 -0600 Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish In-Reply-To: <19414408d3bf707c451df9b4c3e23a89@bosqueviejo.net> References: <19414408d3bf707c451df9b4c3e23a89@bosqueviejo.net> Message-ID: <002f01cddca9$44ce0ab0$ce6a2010$@gmail.com> Out of curiosity, did you name your variables and functions in English or Spanish? We had a very heated discussion about this a month or so back. I forget what the final resolution was, though. Cheers, DBM > -----Original Message----- > From: erlang-questions-bounces@REDACTED [mailto:erlang-questions- > bounces@REDACTED] On Behalf Of Manuel A. Rubio "Bombadil" > Sent: Saturday, December 15, 2012 03:48 > To: erlang-questions@REDACTED > Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish > > Hi, > > is available now the first spanish book about Erlang/OTP in spanish: > > http://erlang.bosqueviejo.net > > Regards, > Manuel Rubio. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Mon Dec 17 23:58:22 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 17 Dec 2012 23:58:22 +0100 Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish In-Reply-To: <002f01cddca9$44ce0ab0$ce6a2010$@gmail.com> References: <19414408d3bf707c451df9b4c3e23a89@bosqueviejo.net> <002f01cddca9$44ce0ab0$ce6a2010$@gmail.com> Message-ID: <50CFA38E.4030307@ninenines.eu> Some of the first snippets: area(Base, Altura) -> Base * Altura. [ martillea_clavo(X) || X <- Clavos, clavar(i) =:= 'si' ]. And so on. There was no final resolution though, some people still didn't believe other languages than English existed, and I'm sure they'll complain that this book has code in Spanish and totally miss the point. Grats on the book, I'll forward it to someone. :) On 12/17/2012 11:53 PM, David Mercer wrote: > Out of curiosity, did you name your variables and functions in English or Spanish? We had a very heated discussion about this a month or so back. I forget what the final resolution was, though. > > Cheers, > > DBM > >> -----Original Message----- >> From: erlang-questions-bounces@REDACTED [mailto:erlang-questions- >> bounces@REDACTED] On Behalf Of Manuel A. Rubio "Bombadil" >> Sent: Saturday, December 15, 2012 03:48 >> To: erlang-questions@REDACTED >> Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish >> >> Hi, >> >> is available now the first spanish book about Erlang/OTP in spanish: >> >> http://erlang.bosqueviejo.net >> >> Regards, >> Manuel Rubio. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From loris.fichera@REDACTED Tue Dec 18 00:02:01 2012 From: loris.fichera@REDACTED (Loris Fichera) Date: Tue, 18 Dec 2012 00:02:01 +0100 Subject: [erlang-questions] eBook version of the Handbook of Neuroevolution Through Erlang, is now available from Springer. In-Reply-To: References: Message-ID: <50CFA469.3060808@gmail.com> Hello Gene, Il 11/11/2012 06:02, Gene Sher ha scritto: > Hello Erlangers, > > The eBook version of my the Handbook of Neuroevolution Through Erlang, > is now in print: http://www.springer.com/computer/swe/book/978-1-4614-4462-6 > The Hardcover book will be available within the next 2-3 weeks from > Amazon, Barnes & Noble, and Springer directly. congratulations for this achievement! I'll purchase the book for sure, it definitely turned on my curiosity! [cut] > /Handbook of Neuroevolution Through Erlang/ explains > how to leverage Erlang?s features in the field of machine learning, and > the system?s real world applications, ranging from algorithmic financial > trading to artificial life and robotics. As a graduate student in machine learning and robotics, and as an Erlang enthusiast, I got a question for you: earlier, this year, I had to implement a self-made replica of Jeff Hawkins' HTM[1]. From an abstract perspective, HTM is all about neurons (which Hawkins calls *cells*) and connections among them. Erlang seemed to be a perfect fit for the job but... when it comes to developing practical learning algorithms for HTM, matricial representations of HTM are used[2] (to boost performance, of course). For this reason, at that time, I ended up preferring Python (+ numpy) over Erlang. Did you have to cope with the same issue in your research? If so, do you cover that in your book? --Loris. [1] http://en.wikipedia.org/wiki/Hierarchical_temporal_memory [2] http://en.wikipedia.org/wiki/Hierarchical_temporal_memory#Cortical_learning_algorithms From bombadil@REDACTED Tue Dec 18 00:36:46 2012 From: bombadil@REDACTED (Manuel A. Rubio "Bombadil") Date: Tue, 18 Dec 2012 00:36:46 +0100 Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish In-Reply-To: References: <19414408d3bf707c451df9b4c3e23a89@bosqueviejo.net> Message-ID: <1e76ba2b795ed2ab3d5dd529a42aeb8e@bosqueviejo.net> Ok, could be a good point to do :-) El 2012-12-17 23:35, Juan Puig escribi?: > Hi, > > I could help you out with that. > > /Juan > > -----Original Message----- > From: erlang-questions-bounces@REDACTED > [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Manuel A. > Rubio "Bombadil" > Sent: Saturday, December 15, 2012 7:21 AM > To: Max Bourinov > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] [ANN] Erlang/OTP book in spanish > > Hi, > > El 2012-12-15 11:12, Max Bourinov escribi?: >> Great book! > > Thanks! > >> Do you plan to translate it to English? > > is a good posibility... but my english is very poor, I would need > reviewers! :D > > Thanks again. > Manuel Rubio. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Manuel A. Rubio "Bombadil" Usuario de GNU/Linux #323628 acorde a http://counter.li.org/ T?cnico en Admin. Sistemas Inform?ticos From bombadil@REDACTED Tue Dec 18 00:46:40 2012 From: bombadil@REDACTED (Manuel A. Rubio "Bombadil") Date: Tue, 18 Dec 2012 00:46:40 +0100 Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish In-Reply-To: <002f01cddca9$44ce0ab0$ce6a2010$@gmail.com> References: <19414408d3bf707c451df9b4c3e23a89@bosqueviejo.net> <002f01cddca9$44ce0ab0$ce6a2010$@gmail.com> Message-ID: In spanish we have some symbols to emphatize the words... but are unneded to express in the code. By example: Espa?a ... can be writed as: espana, espagna, espanya, espania... no problem. I think the requirement to write code isn't the language. The requirement is ASCII, don't use nothing out of this table. In my book I use spanish vars because are a good point to identify what you can customize and what you should keep as is. Regards. El 2012-12-17 23:53, David Mercer escribi?: > Out of curiosity, did you name your variables and functions in > English or Spanish? We had a very heated discussion about this a > month or so back. I forget what the final resolution was, though. > > Cheers, > > DBM > >> -----Original Message----- >> From: erlang-questions-bounces@REDACTED [mailto:erlang-questions- >> bounces@REDACTED] On Behalf Of Manuel A. Rubio "Bombadil" >> Sent: Saturday, December 15, 2012 03:48 >> To: erlang-questions@REDACTED >> Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish >> >> Hi, >> >> is available now the first spanish book about Erlang/OTP in spanish: >> >> http://erlang.bosqueviejo.net >> >> Regards, >> Manuel Rubio. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -- Manuel A. Rubio "Bombadil" Usuario de GNU/Linux #323628 acorde a http://counter.li.org/ T?cnico en Admin. Sistemas Inform?ticos From bombadil@REDACTED Tue Dec 18 00:50:04 2012 From: bombadil@REDACTED (Manuel A. Rubio "Bombadil") Date: Tue, 18 Dec 2012 00:50:04 +0100 Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish In-Reply-To: <50CFA38E.4030307@ninenines.eu> References: <19414408d3bf707c451df9b4c3e23a89@bosqueviejo.net> <002f01cddca9$44ce0ab0$ce6a2010$@gmail.com> <50CFA38E.4030307@ninenines.eu> Message-ID: <69dc5e51f6257430b8cddc10441441db@bosqueviejo.net> The code is writed for humans... if you write code only for machines you should use assembler... or perl :D The vars should be understanded by the programmers, and they should be related to domain. Regards. El 2012-12-17 23:58, Lo?c Hoguin escribi?: > Some of the first snippets: > > area(Base, Altura) -> Base * Altura. > > [ martillea_clavo(X) || X <- Clavos, clavar(i) =:= 'si' ]. > > And so on. > > There was no final resolution though, some people still didn't > believe other languages than English existed, and I'm sure they'll > complain that this book has code in Spanish and totally miss the > point. > > Grats on the book, I'll forward it to someone. :) > > On 12/17/2012 11:53 PM, David Mercer wrote: >> Out of curiosity, did you name your variables and functions in >> English or Spanish? We had a very heated discussion about this a >> month or so back. I forget what the final resolution was, though. >> >> Cheers, >> >> DBM >> >>> -----Original Message----- >>> From: erlang-questions-bounces@REDACTED [mailto:erlang-questions- >>> bounces@REDACTED] On Behalf Of Manuel A. Rubio "Bombadil" >>> Sent: Saturday, December 15, 2012 03:48 >>> To: erlang-questions@REDACTED >>> Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish >>> >>> Hi, >>> >>> is available now the first spanish book about Erlang/OTP in >>> spanish: >>> >>> http://erlang.bosqueviejo.net >>> >>> Regards, >>> Manuel Rubio. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> -- Manuel A. Rubio "Bombadil" Usuario de GNU/Linux #323628 acorde a http://counter.li.org/ T?cnico en Admin. Sistemas Inform?ticos From bombadil@REDACTED Tue Dec 18 00:52:31 2012 From: bombadil@REDACTED (Manuel A. Rubio "Bombadil") Date: Tue, 18 Dec 2012 00:52:31 +0100 Subject: [erlang-questions] Custom performance counters in Erlang In-Reply-To: References: Message-ID: <338564e3a617f211edc5f4e4c7bcd3a6@bosqueviejo.net> Hi Yash, I use this lib to gather data from the system: https://github.com/boundary/folsom Another good tool to show status (in real-time) is observer: http://www.erlang.org/doc/apps/observer/observer_ug.html Regards. El 2012-12-17 21:27, Yash Ganthe escribi?: > Hi, > > ? > > The perfmon utility on Windows allows an administrator to monitor > system-level metrics such as CPU and memory consumption as well as > application -level metrics such as number of records processed, > number > of errors, etc. > > ? > > What is the best way to achieve this in Erlang? How to get > application-level metrics logged to the perfmon? > > ? > > Thanks, > > Yash -- Manuel A. Rubio "Bombadil" Usuario de GNU/Linux #323628 acorde a http://counter.li.org/ T?cnico en Admin. Sistemas Inform?ticos From ok@REDACTED Tue Dec 18 01:38:20 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 18 Dec 2012 13:38:20 +1300 Subject: [erlang-questions] [ANN] Erlang/OTP book in spanish In-Reply-To: References: <19414408d3bf707c451df9b4c3e23a89@bosqueviejo.net> <002f01cddca9$44ce0ab0$ce6a2010$@gmail.com> Message-ID: <2F2D27B2-CA37-43A6-B006-DF0EE3C4683C@cs.otago.ac.nz> On 18/12/2012, at 12:46 PM, Manuel A. Rubio Bombadil wrote: > In spanish we have some symbols to emphatize the words... but are unneded to express in the code. By example: > > Espa?a ... can be writed as: espana, espagna, espanya, espania... no problem. There is also no problem with using "Espa?a" as a variable or ?no as an atom without quotes. > > I think the requirement to write code isn't the language. The requirement is ASCII, don't use nothing out of this table. Erlang currently uses Latin-1; there is no requirement to live within the US 1960's character set. From yashgt@REDACTED Tue Dec 18 02:23:00 2012 From: yashgt@REDACTED (Yash Ganthe) Date: Mon, 17 Dec 2012 20:23:00 -0500 Subject: [erlang-questions] Custom performance counters in Erlang Message-ID: Hi, Windows Performance Monitor allows an administrator to monitor the CPU, memory utilization and several other perf characteristics of applications running on the server. Additionally, applications can design their own custom performance counters which can report application-specific perf information such as number of records processed, number of errors encountered in processing, etc. These perf counters can be displayed in a graph in the per monitor along-side the system-level perf measures. What is the mechanism in Erlang to define the perf counters and set values to them? Thanks, Yash -- Using Opera's revolutionary email client: http://www.opera.com/mail/ From yashgt@REDACTED Tue Dec 18 02:23:05 2012 From: yashgt@REDACTED (Yash Ganthe) Date: Mon, 17 Dec 2012 20:23:05 -0500 Subject: [erlang-questions] Custom performance counters in Erlang Message-ID: Hi, The perfmon utility on Windows allows an administrator to monitor system-level metrics such as CPU and memory consumption as well as application -level metrics such as number of records processed, number of errors, etc. What is the best way to achieve this in Erlang? How to get application-level metrics logged to the perfmon? Thanks, Yash -- Using Opera's revolutionary email client: http://www.opera.com/mail/ From jeremy@REDACTED Tue Dec 18 06:15:24 2012 From: jeremy@REDACTED (Jeremy Ong) Date: Mon, 17 Dec 2012 21:15:24 -0800 Subject: [erlang-questions] Custom performance counters in Erlang In-Reply-To: References: Message-ID: I think the thing you're looking for is http://www.erlang.org/doc/apps/os_mon/index.html See also: http://www.erlang.org/doc/man/etop.html http://www.erlang.org/doc/man/observer.html There is a third party library as well: https://github.com/mazenharake/entopalthough I cannot comment on its performance. To aggregate stats to a third party application, you could use something like https://github.com/boundary/folsom, use an AMPQ, or roll your own message queuing process. On Mon, Dec 17, 2012 at 5:23 PM, Yash Ganthe wrote: > Hi, > > The perfmon utility on Windows allows an administrator to monitor > system-level metrics such as CPU and memory consumption as well as > application -level metrics such as number of records processed, number of > errors, etc. > > What is the best way to achieve this in Erlang? How to get > application-level metrics logged to the perfmon? > > > Thanks, > Yash > > -- > Using Opera's revolutionary email client: http://www.opera.com/mail/ > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From esente@REDACTED Tue Dec 18 06:28:13 2012 From: esente@REDACTED (Son Tran-Nguyen) Date: Mon, 17 Dec 2012 23:28:13 -0600 Subject: [erlang-questions] rpc:call to a restricted shell Message-ID: Hello list, In one of my application, I start a restricted shell as a slave node, using a callback module as follow: The callback module, for example, returns {false, State} for gen_event:start/0 function to prevent that to be run. When tested in an actual restricted shell, it is prevented as expected: smba:eggs esente$ erl -pa apps/*/ebin -stdlib restricted_shell shell_callback Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [dtrace] Restricted Eshell V5.9.2 (abort with ^G) 1> gen_event:start(). ** exception exit: restricted shell does not allow gen_event:start() 2> However, when used in my app by making a call to the remote restricted shell using rpc:call/4, the call went through: Args = "-stdlib restricted_shell editor_shell", {ok, Node} = slave:start_link(smba, editor, Args), {ok, Entity} = rpc:call(Node, gen_event, start, []), It returns Entity as a pid(). I understand that the callback functions cannot be used to allow or disallow execution of functions called from compiled code, but only functions called from expressions entered at the shell prompt. I would like to confirm that rpc:call/4 does not enter the call into the remote shell, but doing some stuffs to run the call directly. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fritchie@REDACTED Tue Dec 18 07:02:49 2012 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 18 Dec 2012 00:02:49 -0600 Subject: [erlang-questions] bif_return_trap In-Reply-To: Message of "Fri, 14 Dec 2012 05:26:58 CST." Message-ID: <91140.1355810569@snookles.snookles.com> Paul Davis wrote: pd> For background, I'm running R14B01 on three nodes with one of the pd> three nodes in a remote data center that's about 40ms away from the pd> other two which are <1ms apart. Hrm, well, if the link truly is 1 Gbit (with ~8% fudge for TCP/IP overhead), 2x the bandwidth delay product is about 79 Mbits or 9647 Kbytes. IIRC, kernel TCP settings to allow sliding windows at least that big in order to utilize all that bandwidth by a single TCP connection. Ditto for the buffering inside the VM ... except that the +zdbbl flag to "erl" wasn't added until well after R14B01's release. I don't have the R14B01 release date handy, but R14B02 may have been released near March 2012? My patch for +zdbbl wasn't done for several more months: commit 8faf1746ece60fc5fa634e5fd16e98df1ef7f3ba Author: Scott Lystig Fritchie Date: Fri Oct 22 15:25:10 2010 -0500 Add flag-based setting for the distribution buffer busy limit pd> http://erlang.org/pipermail/erlang-bugs/2010-May/001806.html IIRC, if you hit that one (which was also fixed after R14B01?), all distributed Erlang communication freezes. Attempting a new connection via "erl [-name foo@REDACTED | -sname foo] -remsh frozen@REDACTED" won't work. pd> What I'm observing is that the remote node ends up accumulating pd> processes stuck in erlang:bif_return_trap/1 which eventually pd> accumulate to the point where the node exhausts RAM and the node pd> reboots (if I let it go that long). Each process stuck in pd> bif_return_trap is related to distributed message passing. Are you seeing busy_dist_port messages sent to the system monitor process defined by erlang:system_monitor/2? -Scott From jvalduvieco@REDACTED Tue Dec 18 10:36:40 2012 From: jvalduvieco@REDACTED (Joan Valduvieco Llopart) Date: Tue, 18 Dec 2012 10:36:40 +0100 Subject: [erlang-questions] process manager, R16, and 64-bit Macs In-Reply-To: <50CB93B8.1070203@ninenines.eu> References: <50BE791E.1050202@simonstl.com> <50BF1EA5.3040201@erlang.org> <50BF30A6.3030608@erlang.org> <50BF33FC.5080608@ninenines.eu> <50CB93B8.1070203@ninenines.eu> Message-ID: Sure! I'll try. Installing brew right now.. ;) Joan 2012/12/14 Lo?c Hoguin > On 12/05/2012 01:17 PM, Joan Valduvieco Llopart wrote: > >> I am using observer with mountain lion with wxgtk + xquartz without a >> glitch. :) >> I have a macports repo with my packages. >> https://github.com/**jvalduvieco/macports >> > > Any chance you could port this to homebrew? From what I hear most people > who use Macs use homebrew so you doing this would be a tremendous help for > the community and for the OTP team. > > -- > Lo?c Hoguin > > Erlang Cowboy > Nine Nines > http://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pan@REDACTED Tue Dec 18 11:41:04 2012 From: pan@REDACTED (Patrik Nyblom) Date: Tue, 18 Dec 2012 11:41:04 +0100 Subject: [erlang-questions] erlsrv.html info In-Reply-To: References: Message-ID: <50D04840.6080001@erlang.org> On 12/14/2012 11:16 PM, Wes James wrote: > Looking at this documentation to see where .erlang.cookie might be > saved for a service (installed with erlsrv) so I don't have to do > -cookie on the command line, the docs at: > > http://www.erlang.org/doc/man/erlsrv.html > > say that the default working directory is: > > %SystemDrive%%SystemPath% > > There is a %SystemDrive% in win 7 (type set in the cmd.exe window) but > there is no %SystemPath%. There is a %windir% and %SystemRoot% which > are c:\windows on my system. Anyway, I don't see .erlang.cookie in > c:\windows after using erlsrv and starting the service. Where is > .erlang.cookie for erlsrv services? Yes, the %SystemPath% is not present on modern Windows. The local administrator (which is the user that runs the service) has c:\Windows as home directory by default, but the service runs with c:\Windows\system32 as default directory. One way to make sure is to simply start a service with a debug console and add -emu_args as a parameter, click "Show message" or whatever your locale has to select in the dialogue that pops up (or does not pop up but starts to blink in the task bar) when you start such a service and then do a pwd(). In the Erlang shell that you will see the -home parameter. Example (replace mentally any paths with the appropriate ones for your system): C:\blabla> cd "c:\Program Files\erl5.9.3.1\erts-5.9.3.1\bin" C:\Program Files\erl5.9.3.1\erts-5.9.3.1\bin>erlsrv add testservice -debug console -arg -emu_args C:\Program Files\erl5.9.3.1\erts-5.9.3.1\bin>erlsrv start testservice ...click on the "show me the message" button in the service control manager dialogue, which will take you to a blank screen with only the Erlang shell of the service and a dialogue that can take you back to the desktop. In that shell, you will see the full command line to the VM including the '-home' parameter: --------------- Executing: C:\blablabla...\beam.smp.dll C:\blablabla...\beam.smp.dll -- -root C:\blablabla -progname erl -- -home c:\Windows etc etc etc 1> pwd(). c:/Windows/system32 2> --------------- ...click on the get me back button in the dialogue that hangs beside the shell. The directory after the -home in the "slogan" at the top of the shell should be the home directory. The result from pwd() is the default "current directory" of the service. So the .erlang.cookie should reside in what -home says, which corresponds to %SystemDrive%%SystemRoot%, while files created in current directory will go into whatever pwd() shows. Now stop and remove the service: C:\Program Files\erl5.9.3.1\erts-5.9.3.1\bin>erlsrv stop testservice ...will take some time because of the peculiarities of a debug console.... C:\Program Files\erl5.9.3.1\erts-5.9.3.1\bin>erlsrv remove service Hope that helps! Cheers, /Patrik > > Thanks, > > wes > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From pan@REDACTED Tue Dec 18 11:51:05 2012 From: pan@REDACTED (Patrik Nyblom) Date: Tue, 18 Dec 2012 11:51:05 +0100 Subject: [erlang-questions] rpc:call to a restricted shell In-Reply-To: References: Message-ID: <50D04A99.6070901@erlang.org> Hi! On 12/18/2012 06:28 AM, Son Tran-Nguyen wrote: > Hello list, > > In one of my application, I start a restricted shell as a slave node, > using a callback module as follow: > > The callback module, for example, returns {false, State} for > gen_event:start/0 function to prevent that to be run. When tested in > an actual restricted shell, it is prevented as expected: > > smba:eggs esente$ erl -pa apps/*/ebin -stdlib restricted_shell > shell_callback > Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:4:4] > [async-threads:0] [hipe] [kernel-poll:false] [dtrace] > > Restricted Eshell V5.9.2 (abort with ^G) > 1> gen_event:start(). > ** exception exit: restricted shell does not allow gen_event:start() > 2> > > However, when used in my app by making a call to the remote restricted > shell using rpc:call/4, the call went through: > > Args = "-stdlib restricted_shell editor_shell", > {ok, Node} = slave:start_link(smba, editor, Args), > {ok, Entity} = rpc:call(Node, gen_event, start, []), > > It returns Entity as a pid(). > > I understand that the callback functions cannot be used to allow or > disallow execution of functions called from compiled code, but only > functions called from expressions entered at the shell prompt. > > I would like to confirm that rpc:call/4 does not enter the call into > the remote shell, but doing some stuffs to run the call directly. You are correct, rpc:call has nothing to do with the shell. Rpc calls an already started server (rex) on the node and asks it to apply the module, function and parameters and send the result back (basiacally). That never passes any shell or interpreter. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Cheers, /Patrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.vb80@REDACTED Tue Dec 18 11:54:53 2012 From: pablo.vb80@REDACTED (Pablo Vieytes) Date: Tue, 18 Dec 2012 11:54:53 +0100 Subject: [erlang-questions] Bound(?) var in anonymous fun Message-ID: I want to delete an item from a list and I've written this fun. delete_key_from_list(Key, StatusList)-> lists:foldr( fun(Key, Acc) -> Acc; (Another, Acc) -> [Another|Acc] end, [], StatusList). src/file.erl:172: Warning: variable 'Key' is unused src/file.erl:174: Warning: variable 'Key' is unused src/file.erl:174: Warning: variable 'Key' shadowed in 'fun' src/file.erl:176: Warning: this clause cannot match because a previous clause at line 174 always matches I guessed Key was bound so I could use for pattern matching. It's very easy to fix it but I don't know why it doesn't work. Fixed version: delete_key_from_list(Key, StatusList)-> lists:foldr( fun(K, Acc) when K == Key -> Acc; (Another, Acc) -> [Another|Acc] end, [], StatusList). -------------- next part -------------- An HTML attachment was scrubbed... URL: From bombadil@REDACTED Tue Dec 18 12:08:33 2012 From: bombadil@REDACTED (Manuel A. Rubio "Bombadil") Date: Tue, 18 Dec 2012 12:08:33 +0100 Subject: [erlang-questions] =?utf-8?q?Bound=28=3F=29_var_in_anonymous_fun?= In-Reply-To: References: Message-ID: <3109f8ce963b3dc34511ec765dfa7953@bosqueviejo.net> Hi Pablo, El 2012-12-18 11:54, Pablo Vieytes escribi?: > I guessed Key was bound so I could use for pattern matching.?It's > very easy to fix it but I don't know why it doesn't work. This doesn't works well because the params are not inside of binding in closures. The params are the interface with the outside world and will be confuse know if a param is a binding or not. By default Erlang hasn't binding in function params. > Fixed version: > > delete_key_from_list(Key, StatusList)-> > ? ? lists:foldr( > ? ? ? fun(K, Acc) when K == Key -> > ? ? ?Acc; > (Another, Acc) -> > ? ? ?[Another|Acc] > ? ? ? end, > ? ? ? [], > ? ? ? StatusList). This code could be writted as: delete_key_from_list(Key, StatusList)-> lists:filter(fun(X) -> Key =/= X end, StatusList). Regards, Manuel Rubio. From bourinov@REDACTED Tue Dec 18 12:12:06 2012 From: bourinov@REDACTED (Max Bourinov) Date: Tue, 18 Dec 2012 15:12:06 +0400 Subject: [erlang-questions] Bound(?) var in anonymous fun In-Reply-To: <3109f8ce963b3dc34511ec765dfa7953@bosqueviejo.net> References: <3109f8ce963b3dc34511ec765dfa7953@bosqueviejo.net> Message-ID: Hi Pablo, Have a look at lists module [1]. Maybe it worth to use existing functions? --- 1. http://www.erlang.org/doc/man/lists.html Best regards, Max On Tue, Dec 18, 2012 at 3:08 PM, Manuel A. Rubio "Bombadil" < bombadil@REDACTED> wrote: > Hi Pablo, > > El 2012-12-18 11:54, Pablo Vieytes escribi?: > > I guessed Key was bound so I could use for pattern matching. It's >> very easy to fix it but I don't know why it doesn't work. >> > > This doesn't works well because the params are not inside of binding in > closures. The params are the interface with the outside world and will be > confuse know if a param is a binding or not. By default Erlang hasn't > binding in function params. > > > Fixed version: >> >> delete_key_from_list(Key, StatusList)-> >> lists:foldr( >> fun(K, Acc) when K == Key -> >> Acc; >> (Another, Acc) -> >> [Another|Acc] >> end, >> [], >> StatusList). >> > > This code could be writted as: > > delete_key_from_list(Key, StatusList)-> > lists:filter(fun(X) -> Key =/= X end, StatusList). > > Regards, > Manuel Rubio. > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mls@REDACTED Tue Dec 18 12:16:34 2012 From: mls@REDACTED (Mailingslists) Date: Tue, 18 Dec 2012 12:16:34 +0100 Subject: [erlang-questions] Too much memory consumption when generating magic squares in erlang - Need help for optimization Message-ID: <6EF644D0-FDB9-4BEA-8FAF-549E171287E3@psy-coding.com> Hey there, I've some problems with an exercise for university. I've already made some steps in the right direction but until now it is still not working as expected. The code and more detailed information are already on stackoverflow. Hope you guys can give me the kick in the right direction to solve the problem! http://stackoverflow.com/questions/13856286/too-much-memory-consumption-when-generating-magic-squares-in-erlang-need-help Best regards Felix From dm.klionsky@REDACTED Tue Dec 18 12:22:07 2012 From: dm.klionsky@REDACTED (Dmitry Klionsky) Date: Tue, 18 Dec 2012 14:22:07 +0300 Subject: [erlang-questions] Bound(?) var in anonymous fun In-Reply-To: <3109f8ce963b3dc34511ec765dfa7953@bosqueviejo.net> References: <3109f8ce963b3dc34511ec765dfa7953@bosqueviejo.net> Message-ID: <50D051DF.3030703@gmail.com> On 12/18/2012 02:08 PM, Manuel A. Rubio "Bombadil" wrote: > Hi Pablo, > > El 2012-12-18 11:54, Pablo Vieytes escribi?: >> I guessed Key was bound so I could use for pattern matching. It's >> very easy to fix it but I don't know why it doesn't work. > > This doesn't works well because the params are not inside of binding > in closures. The params are the interface with the outside world and > will be confuse know if a param is a binding or not. By default Erlang > hasn't binding in function params. > >> Fixed version: >> >> delete_key_from_list(Key, StatusList)-> >> lists:foldr( >> fun(K, Acc) when K == Key -> >> Acc; >> (Another, Acc) -> >> [Another|Acc] >> end, >> [], >> StatusList). > > This code could be writted as: > > delete_key_from_list(Key, StatusList)-> > lists:filter(fun(X) -> Key =/= X end, StatusList). > > Regards, > Manuel Rubio. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Or even more readable as: delete_key_from_list(Key, StatusList)-> [Status || Status <- StatusList, Status =/= Key]. -- Best regards, Dmitry Klionsky From bourinov@REDACTED Tue Dec 18 12:34:36 2012 From: bourinov@REDACTED (Max Bourinov) Date: Tue, 18 Dec 2012 15:34:36 +0400 Subject: [erlang-questions] Bound(?) var in anonymous fun In-Reply-To: <50D051DF.3030703@gmail.com> References: <3109f8ce963b3dc34511ec765dfa7953@bosqueviejo.net> <50D051DF.3030703@gmail.com> Message-ID: Or even more shooter: lists:delete(Key, StatusList). On Tue, Dec 18, 2012 at 3:22 PM, Dmitry Klionsky wrote: > On 12/18/2012 02:08 PM, Manuel A. Rubio "Bombadil" wrote: > >> Hi Pablo, >> >> El 2012-12-18 11:54, Pablo Vieytes escribi?: >> >>> I guessed Key was bound so I could use for pattern matching. It's >>> very easy to fix it but I don't know why it doesn't work. >>> >> >> This doesn't works well because the params are not inside of binding in >> closures. The params are the interface with the outside world and will be >> confuse know if a param is a binding or not. By default Erlang hasn't >> binding in function params. >> >> Fixed version: >>> >>> delete_key_from_list(Key, StatusList)-> >>> lists:foldr( >>> fun(K, Acc) when K == Key -> >>> Acc; >>> (Another, Acc) -> >>> [Another|Acc] >>> end, >>> [], >>> StatusList). >>> >> >> This code could be writted as: >> >> delete_key_from_list(Key, StatusList)-> >> lists:filter(fun(X) -> Key =/= X end, StatusList). >> >> Regards, >> Manuel Rubio. >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > Or even more readable as: > > delete_key_from_list(Key, StatusList)-> > [Status || Status <- StatusList, Status =/= Key]. > > -- > Best regards, > Dmitry Klionsky > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Dec 18 12:36:25 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 18 Dec 2012 12:36:25 +0100 Subject: [erlang-questions] Bound(?) var in anonymous fun In-Reply-To: References: <3109f8ce963b3dc34511ec765dfa7953@bosqueviejo.net> <50D051DF.3030703@gmail.com> Message-ID: <50D05539.9050507@ninenines.eu> This one only removes the first element. :) On 12/18/2012 12:34 PM, Max Bourinov wrote: > Or even more shooter: > > lists:delete(Key, StatusList). > > > On Tue, Dec 18, 2012 at 3:22 PM, Dmitry Klionsky > wrote: > > On 12/18/2012 02:08 PM, Manuel A. Rubio "Bombadil" wrote: > > Hi Pablo, > > El 2012-12-18 11:54, Pablo Vieytes escribi?: > > I guessed Key was bound so I could use for pattern matching. > It's > very easy to fix it but I don't know why it doesn't work. > > > This doesn't works well because the params are not inside of > binding in closures. The params are the interface with the > outside world and will be confuse know if a param is a binding > or not. By default Erlang hasn't binding in function params. > > Fixed version: > > delete_key_from_list(Key, StatusList)-> > lists:foldr( > fun(K, Acc) when K == Key -> > Acc; > (Another, Acc) -> > [Another|Acc] > end, > [], > StatusList). > > > This code could be writted as: > > delete_key_from_list(Key, StatusList)-> > lists:filter(fun(X) -> Key =/= X end, StatusList). > > Regards, > Manuel Rubio. > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/__listinfo/erlang-questions > > > > Or even more readable as: > > delete_key_from_list(Key, StatusList)-> > [Status || Status <- StatusList, Status =/= Key]. > > -- > Best regards, > Dmitry Klionsky > > > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/__listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From pablo.vb80@REDACTED Tue Dec 18 13:01:27 2012 From: pablo.vb80@REDACTED (Pablo Vieytes) Date: Tue, 18 Dec 2012 13:01:27 +0100 Subject: [erlang-questions] Bound(?) var in anonymous fun In-Reply-To: <50D05539.9050507@ninenines.eu> References: <3109f8ce963b3dc34511ec765dfa7953@bosqueviejo.net> <50D051DF.3030703@gmail.com> <50D05539.9050507@ninenines.eu> Message-ID: I have to get used to use list comprehension. I know how it works but I never remember use it. 2012/12/18 Lo?c Hoguin > This one only removes the first element. :) > > > On 12/18/2012 12:34 PM, Max Bourinov wrote: > >> Or even more shooter: >> >> lists:delete(Key, StatusList). >> >> >> On Tue, Dec 18, 2012 at 3:22 PM, Dmitry Klionsky > **> wrote: >> >> On 12/18/2012 02:08 PM, Manuel A. Rubio "Bombadil" wrote: >> >> Hi Pablo, >> >> El 2012-12-18 11:54, Pablo Vieytes escribi?: >> >> I guessed Key was bound so I could use for pattern matching. >> It's >> very easy to fix it but I don't know why it doesn't work. >> >> >> This doesn't works well because the params are not inside of >> binding in closures. The params are the interface with the >> outside world and will be confuse know if a param is a binding >> or not. By default Erlang hasn't binding in function params. >> >> Fixed version: >> >> delete_key_from_list(Key, StatusList)-> >> lists:foldr( >> fun(K, Acc) when K == Key -> >> Acc; >> (Another, Acc) -> >> [Another|Acc] >> end, >> [], >> StatusList). >> >> >> This code could be writted as: >> >> delete_key_from_list(Key, StatusList)-> >> lists:filter(fun(X) -> Key =/= X end, StatusList). >> >> Regards, >> Manuel Rubio. >> ______________________________**___________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > >> http://erlang.org/mailman/__**listinfo/erlang-questions >> >> >> > >> >> >> Or even more readable as: >> >> delete_key_from_list(Key, StatusList)-> >> [Status || Status <- StatusList, Status =/= Key]. >> >> -- >> Best regards, >> Dmitry Klionsky >> >> >> ______________________________**___________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > >> http://erlang.org/mailman/__**listinfo/erlang-questions >> >> > >> >> >> >> >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> >> > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Tue Dec 18 13:25:20 2012 From: bourinov@REDACTED (Max Bourinov) Date: Tue, 18 Dec 2012 16:25:20 +0400 Subject: [erlang-questions] Bound(?) var in anonymous fun In-Reply-To: <50D05539.9050507@ninenines.eu> References: <3109f8ce963b3dc34511ec765dfa7953@bosqueviejo.net> <50D051DF.3030703@gmail.com> <50D05539.9050507@ninenines.eu> Message-ID: You are right!!! But it should work according to the specification: Pablo wrote: "I want to delete *an item* from a list and I've written this fun." So, it must be only one item :-) Best regards, Max On Tue, Dec 18, 2012 at 3:36 PM, Lo?c Hoguin wrote: > This one only removes the first element. :) > > > On 12/18/2012 12:34 PM, Max Bourinov wrote: > >> Or even more shooter: >> >> lists:delete(Key, StatusList). >> >> >> On Tue, Dec 18, 2012 at 3:22 PM, Dmitry Klionsky > **> wrote: >> >> On 12/18/2012 02:08 PM, Manuel A. Rubio "Bombadil" wrote: >> >> Hi Pablo, >> >> El 2012-12-18 11:54, Pablo Vieytes escribi?: >> >> I guessed Key was bound so I could use for pattern matching. >> It's >> very easy to fix it but I don't know why it doesn't work. >> >> >> This doesn't works well because the params are not inside of >> binding in closures. The params are the interface with the >> outside world and will be confuse know if a param is a binding >> or not. By default Erlang hasn't binding in function params. >> >> Fixed version: >> >> delete_key_from_list(Key, StatusList)-> >> lists:foldr( >> fun(K, Acc) when K == Key -> >> Acc; >> (Another, Acc) -> >> [Another|Acc] >> end, >> [], >> StatusList). >> >> >> This code could be writted as: >> >> delete_key_from_list(Key, StatusList)-> >> lists:filter(fun(X) -> Key =/= X end, StatusList). >> >> Regards, >> Manuel Rubio. >> ______________________________**___________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > >> http://erlang.org/mailman/__**listinfo/erlang-questions >> >> >> > >> >> >> Or even more readable as: >> >> delete_key_from_list(Key, StatusList)-> >> [Status || Status <- StatusList, Status =/= Key]. >> >> -- >> Best regards, >> Dmitry Klionsky >> >> >> ______________________________**___________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > >> http://erlang.org/mailman/__**listinfo/erlang-questions >> >> > >> >> >> >> >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> >> > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc@REDACTED Tue Dec 18 13:59:11 2012 From: marc@REDACTED (Marc Worrell) Date: Tue, 18 Dec 2012 13:59:11 +0100 Subject: [erlang-questions] ANN: Zotonic 0.9.0 Message-ID: <8D04382E-6364-445F-8160-39873255F330@worrell.nl> Dear Erlang users and Zotonic enthusiasts, Zotonic is the Erlang CMS and web framework. After months of hard work, the team is proud to present the next version of Zotonic, release 0.9.0! The highlights of these release are the adoption of "mobile first" adaptive web development with automatic template selection based on the users's browser; the use of Twitter Bootstrap CSS framework for the base templates; a completely revamped admin interface, and, most importantly, documentation! Zotonic's core concepts and most modules have been fully documented. For the full release notes, please read http://zotonic.com/docs/latest/dev/releasenotes/rel_0.9.0.html As always, the release can be downloaded from http://zotonic.com/download Kind regards, on behalf of the Zotonic team, Arjan Scherpenisse & Marc Worrell -------------- next part -------------- An HTML attachment was scrubbed... URL: From barcojie@REDACTED Tue Dec 18 14:42:13 2012 From: barcojie@REDACTED (Barco You) Date: Tue, 18 Dec 2012 21:42:13 +0800 Subject: [erlang-questions] ANN: Zotonic 0.9.0 In-Reply-To: <8D04382E-6364-445F-8160-39873255F330@worrell.nl> References: <8D04382E-6364-445F-8160-39873255F330@worrell.nl> Message-ID: Very good to hear of it. Congratulations! Best regards, Barco On Tuesday, December 18, 2012, Marc Worrell wrote: > Dear Erlang users and Zotonic enthusiasts, > > Zotonic is the Erlang CMS and web framework. > > After months of hard work, the team is proud to present the next version > of Zotonic, release 0.9.0! > > The highlights of these release are the adoption of "mobile first" > adaptive web development with automatic template selection based on the > users's browser; the use of Twitter Bootstrap CSS framework for the base > templates; a completely revamped admin interface, and, most importantly, > documentation! Zotonic's core concepts and most modules have been fully > documented. > > For the full release notes, please read > http://zotonic.com/docs/latest/dev/releasenotes/rel_0.9.0.html > > As always, the release can be downloaded from > http://zotonic.com/download > > Kind regards, > > on behalf of the Zotonic team, > > Arjan Scherpenisse & Marc Worrell > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jerome@REDACTED Tue Dec 18 14:49:03 2012 From: jerome@REDACTED (=?UTF-8?B?SsOpcsO0bWUgRGVzcXVpbGJldA==?=) Date: Tue, 18 Dec 2012 14:49:03 +0100 Subject: [erlang-questions] =?utf-8?b?W8OAIFBhcmlzXSAoZW4gZnJhbsOnYWlzKSBF?= =?utf-8?q?rlang_Dojo+Erlounge_ce_soir?= Message-ID: <50D0744F.8070206@desquilbet.org> Bonjour ! ? Paris, nous essayons (avec succ?s pour le moment) de maintenir le rythme d'un Dojo+Erlounge le troisi?me mardi de chaque mois. C'est donc ce soir pour d?cembre. HEURE: Dojo: 19h Erlounge: 21h LIEU: Dojo: chez /UT7, 10 rue d'Uz?s, 75002 Paris, 4?me ?tage, gauche. Erlounge: chez Papa, 153 rue Montmartre, 75002 Paris. J?r?me. From jerome@REDACTED Tue Dec 18 15:32:34 2012 From: jerome@REDACTED (=?UTF-8?B?SsOpcsO0bWUgRGVzcXVpbGJldA==?=) Date: Tue, 18 Dec 2012 15:32:34 +0100 Subject: [erlang-questions] [in Paris] Erlang Dojo+Erlounge tonight Message-ID: <50D07E82.9040805@desquilbet.org> Hello! Any erlanger visiting Paris will be welcome tonight (Tuesday 18th of december) to attend our monthly Dojo+Erlounge. When: Dojo: 19h Erlounge: 21h Where: Dojo: at /UT7, 10 rue d'Uz?s, 75002 Paris, 4th floor, left. Erlounge: chez Papa, 153 rue Montmartre, 75002 Paris. J?r?me. From ulf@REDACTED Tue Dec 18 16:06:11 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 18 Dec 2012 16:06:11 +0100 Subject: [erlang-questions] more flexible code generation In-Reply-To: <9EEE8568-D7B5-478E-9A29-C9ACEC0B2A9A@gmail.com> References: <9EEE8568-D7B5-478E-9A29-C9ACEC0B2A9A@gmail.com> Message-ID: <6D875D6A-741E-47A2-A0BF-CB2727BDA80B@feuerlabs.com> On 17 Dec 2012, at 10:38, Tim Watson wrote: > Ulf - some feedback: this is totally AWESOME! > > Damn I need to find time to go and refactor at least three projects to take advantage of this now.... :D Thanks for the encouraging feedback. :) I've just pushed some additions for improving error reporting, which I would also like feedback on. Handling errors in parse transforms has been a bit tricky. My latest approach is to use a new function, parse_trans:return(Forms, Context), which checks for the presence of {error, Info} or {warning, Info} tuples in the abstract tree. It will then extract them and produce a return that the compiler is comfortable with. I have also added a function, parse_trans:format_exception(Class, Error), which can be used to produce a less user-hostile exception trace. To illustrate: I've started working on a module for code generation from YANG specs: https://github.com/tonyrog/yang/blob/master/src/yang_codegen.erl I got some horrible exception dumps, and was motivated to address the problem. The parse_transform/2 function in parse_trans_codegen.erl now looks like this: parse_transform(Forms, Options) -> Context = parse_trans:initial_context(Forms, Options), {NewForms, _} = parse_trans:do_depth_first( fun xform_fun/4, _Acc = Forms, Forms, Context), parse_trans:return(parse_trans:revert(NewForms), Context). where the parse_trans:return(Forms, Context) is new. A crash in the code generation now looks like: ==> yang (compile) src/yang_codegen.erl:71: exception error: no case clause matching match_expr in function parse_trans_codegen:gen_function_/4 (src/parse_trans_codegen.erl, line 189) in call from parse_trans_codegen:gen_function/5 (src/parse_trans_codegen.erl, line 181) in call from parse_trans_codegen:xform_fun/4 (src/parse_trans_codegen.erl, line 156) ERROR: compile failed while processing /Users/uwiger/FL/git/yang: rebar_abort make: *** [compile] Error 1 ?which, by parse_transform standards, is pretty decent. Note that we get the line info in the original source code. The error-trapping code in parse_trans_codegen.erl looks like this: https://github.com/uwiger/parse_trans/blob/master/src/parse_trans_codegen.erl#L180 gen_function(NameF, FunF, L0, L, Acc) -> try gen_function_(NameF, FunF, L, Acc) catch error:E -> ErrStr = parse_trans:format_exception(error, E), {error, {L0, ?MODULE, ErrStr}} end. Now, I *could* rework the existing functions in parse_trans to produce error info in this way. It won't happen immediately, since I have other, more urgent, things on my plate, but - would anyone have objections to this? It's not entirely trivial. If I have the existing functions add {error, I} and {warning, I} tuples in the form tree, and the parse_transform() functions don't end with a call to parse_trans:return/2, the linter will crash, which is neither pretty, nor informative. If the old functions call parse_trans:return/2 automatically, this will mess up code that makes successive calls to them. BW compatibility sucks, as Robert Virding once put it. I am open to suggestions. BR, Ulf W > > On 16 Dec 2012, at 11:39, Ulf Wiger wrote: > >> >> I made a little addition to my parse_trans library, in parse_trans_codegen: >> >> The 'original' method for generating code for a function was e.g.: >> >> g(Name, V) -> >> codegen:gen_function( >> Name, >> fun(L) -> >> member({'$var',V}, L) >> end). >> >> Where the {'$var', V} notation is a way to insert a reference to a variable, rather than using the variable itself. >> >> To illustrate, let's first create a little pretty-printer fun: >> >> 1> PP = fun(F) -> io:fwrite("~s~n", [erl_pp:form(F)]) end. >> #Fun >> >> >> 2> PP(ex_codegen:g(foo,17)). >> foo(L) -> >> member(17, L). >> >> There are a few other, similar, functions, but in some cases, this is still too restrictive. A few days ago, I added support for specifying a more dynamic pattern: >> >> k(L) -> >> codegen:gen_function( >> lcf, >> [fun({'$var',X}) -> >> {'$var',Y} >> end || {X, Y} <- L]). >> >> The list comprehension results in a list of clauses based on data resolved at run-time. For example: >> >> 3> PP(ex_codegen:k([ {"a", {1,2,3}}, {"b", {4,5,6}} ])). >> lcf("a") -> >> {1,2,3}; >> lcf("b") -> >> {4,5,6}. >> >> https://github.com/uwiger/parse_trans/blob/master/doc/parse_trans_codegen.md >> >> Feedback is welcome. >> >> BR, >> Ulf W >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From comptekki@REDACTED Tue Dec 18 16:33:53 2012 From: comptekki@REDACTED (Wes James) Date: Tue, 18 Dec 2012 08:33:53 -0700 Subject: [erlang-questions] erlsrv.html info In-Reply-To: <50D04840.6080001@erlang.org> References: <50D04840.6080001@erlang.org> Message-ID: Thanks - I'll do some testing. wes On Tue, Dec 18, 2012 at 3:41 AM, Patrik Nyblom wrote: > On 12/14/2012 11:16 PM, Wes James wrote: > > Looking at this documentation to see where .erlang.cookie might be saved > for a service (installed with erlsrv) so I don't have to do -cookie on the > command line, the docs at: > > http://www.erlang.org/doc/man/erlsrv.html > > say that the default working directory is: > > %SystemDrive%%SystemPath% > > There is a %SystemDrive% in win 7 (type set in the cmd.exe window) but > there is no %SystemPath%. There is a %windir% and %SystemRoot% which are > c:\windows on my system. Anyway, I don't see .erlang.cookie in c:\windows > after using erlsrv and starting the service. Where is .erlang.cookie for > erlsrv services? > > Yes, the %SystemPath% is not present on modern Windows. The local > administrator (which is the user that runs the service) has c:\Windows as > home directory by default, but the service runs with c:\Windows\system32 as > default directory. > > One way to make sure is to simply start a service with a debug console and > add -emu_args as a parameter, click "Show message" or whatever your locale > has to select in the dialogue that pops up (or does not pop up but starts > to blink in the task bar) when you start such a service and then do a > pwd(). In the Erlang shell that you will see the -home parameter. > > Example (replace mentally any paths with the appropriate ones for your > system): > > C:\blabla> cd "c:\Program Files\erl5.9.3.1\erts-5.9.3.1\bin" > C:\Program Files\erl5.9.3.1\erts-5.9.3.1\bin>erlsrv add testservice -debug > console -arg -emu_args > C:\Program Files\erl5.9.3.1\erts-5.9.3.1\bin>erlsrv start testservice > > ...click on the "show me the message" button in the service control > manager dialogue, which will take you to a blank screen with only the > Erlang shell of the service and a dialogue that can take you back to the > desktop. In that shell, you will see the full command line to the VM > including the '-home' parameter: > --------------- > Executing: C:\blablabla...\beam.smp.dll C:\blablabla...\beam.smp.dll -- > -root C:\blablabla -progname erl -- -home c:\Windows etc etc etc > 1> pwd(). > c:/Windows/system32 > 2> > --------------- > ...click on the get me back button in the dialogue that hangs beside the > shell. The directory after the -home in the "slogan" at the top of the > shell should be the home directory. The result from pwd() is the default > "current directory" of the service. > > So the .erlang.cookie should reside in what -home says, which corresponds > to %SystemDrive%%SystemRoot%, while files created in current directory will > go into whatever pwd() shows. > > Now stop and remove the service: > C:\Program Files\erl5.9.3.1\erts-5.9.3.1\bin>erlsrv stop testservice > ...will take some time because of the peculiarities of a debug console.... > C:\Program Files\erl5.9.3.1\erts-5.9.3.1\bin>erlsrv remove service > > Hope that helps! > > Cheers, > /Patrik > > > Thanks, > > wes > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasily@REDACTED Tue Dec 18 20:32:13 2012 From: vasily@REDACTED (Vasily Sulatskov) Date: Tue, 18 Dec 2012 20:32:13 +0100 Subject: [erlang-questions] Several gen_leader questions Message-ID: Hi, I am using gen_leader from https://github.com/abecciu/gen_leader_revivaland I want to use it with a fixed list of candidate nodes - as far as I understand that's the easiest way. I tried several variations of starting gen_leader but none was satisfactory. I start it as part of a supervision tree, so all examples are taken from gen_supervisor modules init/1 functions: Here's what I tried so far: init(_Args) -> %% The same value is used on all machines in the cluster Leader_nodes = ['foobar@REDACTED', 'foobar@REDACTED', 'foobar@REDACTED'], Home = os:getenv("HOME"), Gen_leader_config = {gen_leader_module, {gen_leader_module, start_link, [Leader_nodes, [{vardir, Home}]]}, permanent, 2000, worker, [gen_leader_module]}, {ok, {{one_for_one, 10000, 1}, [Gen_leader_config]}}. If I do this, then nodes specified in Leader_nodes work just fine, they all participate in elections, leaders are elected properly, they are able to do gen_leader:leader_call() to the actual leader etc. The problem is that on all other nodes (which are not specified in Leader_nodes) gen_leader is not started at all. Gen_leader checks if the node it's running on is one of "candidate nodes" or "worker nodes" and if that's not the case - it simply doesn't start. All further attempts at gen_leader:leader_call from that node fail. I tried to run every node in the cluster except for "candidate nodes" as a "worker node", so I changed supervisor to something like: init(_Args) -> %% The same value is used on all machines in the cluster Leader_nodes = ['foobar@REDACTED', 'foobar@REDACTED', 'foobar@REDACTED'], Workers = case lists:member(node(), Leader_nodes) of true -> []; false -> [node()] end, Home = os:getenv("HOME"), {ok, {{one_for_one, 10000, 1}, [{scheduler, {scheduler, start_link, [Leader_nodes, [{vardir, Home}, {workers, Workers}]]}, permanent, 2000, worker, [scheduler_leader]}]}}. As far as I understand, when gen_leader runs in a worker configuration, it doesn't participate in elections, but still keeps track of where an actual leader is running, so gen_leader:leader_call is still possible. This setup kind of works, but it seems that gen_leader process on "worker" nodes constantly grows in memory usage, past several Gb at least, eventually crashing the whole VM. Am I running gen_leader correctly? What is the correct way of running gen_leader with a fixed set of "candidate" nodes and that every other node is aware of where a leader is running, so that gen_leader:leader_call() is possible? Which version of gen_leader is recommended to use? This one https://github.com/abecciu/gen_leader_revival? Or maybe the version from gproc? By the way can someone explain what's the difference between them? And I have another, most likely unrelated, issue with gen_leader. On one deployment, sometimes I find a cluster in a state with two leaders - most of the nodes think that the leader is one node, but some other node thinks that the leader is on the other node. I am not sure if the other leader is the node that diverges from the consensus - I don't have a cluster in this state right now to check. It seems to happen after a gen_leader process crashes somehow (some internal work, not related to gen_leader magic). The other thing that I think might be important here, is that gen_leader process in that setup can get stuck in handle_leader_call for quite some long time. Can it cause problems with leader elections? Should gen_leader processes not block in handle_whatever functions and always be able to handle election callback? Thanks in advance. -- Best regards, Vasily Sulatskov -------------- next part -------------- An HTML attachment was scrubbed... URL: From lovei.laszlo@REDACTED Tue Dec 18 21:28:15 2012 From: lovei.laszlo@REDACTED (Laszlo Lovei) Date: Tue, 18 Dec 2012 12:28:15 -0800 (PST) Subject: [erlang-questions] rpc:call to a restricted shell In-Reply-To: References: Message-ID: <82b1d73e-d83e-4d89-90b1-fbbf8e9c7661@googlegroups.com> It's quite important to understand that Erlang is not a scripting language where your source code is interpreted and you can use the same interpreter as a shell. It's exactly the opposite: the shell is an interpreter (written in Erlang and compiled into binary code understood by the Erlang VM) that evaluates Erlang expressions entered at the prompt and formatting their results. This has a number of consequences regarding performance and semantics, expressions evaluated by the shell do not have the same meaning as code compiled into modules. To answer your question too: the rpc server runs independently of the shell, and the latter may impose artificial restrictions on what it accepts without affecting any other part of the system. L. On Tuesday, December 18, 2012 6:28:13 AM UTC+1, Son Tran-Nguyen wrote: > > Hello list, > > In one of my application, I start a restricted shell as a slave node, > using a callback module as follow: > > The callback module, for example, returns {false, State} for > gen_event:start/0 function to prevent that to be run. When tested in an > actual restricted shell, it is prevented as expected: > > smba:eggs esente$ erl -pa apps/*/ebin -stdlib restricted_shell > shell_callback > Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:4:4] [async-threads:0] > [hipe] [kernel-poll:false] [dtrace] > > Restricted Eshell V5.9.2 (abort with ^G) > 1> gen_event:start(). > ** exception exit: restricted shell does not allow gen_event:start() > 2> > > However, when used in my app by making a call to the remote restricted > shell using rpc:call/4, the call went through: > > Args = "-stdlib restricted_shell editor_shell", > {ok, Node} = slave:start_link(smba, editor, Args), > {ok, Entity} = rpc:call(Node, gen_event, start, []), > > It returns Entity as a pid(). > > I understand that the callback functions cannot be used to allow or > disallow execution of functions called from compiled code, but only > functions called from expressions entered at the shell prompt. > > I would like to confirm that rpc:call/4 does not enter the call into the > remote shell, but doing some stuffs to run the call directly. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From esente@REDACTED Wed Dec 19 01:32:41 2012 From: esente@REDACTED (Son Tran-Nguyen) Date: Tue, 18 Dec 2012 18:32:41 -0600 Subject: [erlang-questions] rpc:call to a restricted shell In-Reply-To: <82b1d73e-d83e-4d89-90b1-fbbf8e9c7661@googlegroups.com> References: <82b1d73e-d83e-4d89-90b1-fbbf8e9c7661@googlegroups.com> Message-ID: Thanks all. I guess I can't rely on that then. Sincerely, Son Tran-Nguyen On Tue, Dec 18, 2012 at 2:28 PM, Laszlo Lovei wrote: > It's quite important to understand that Erlang is not a scripting language > where your source code is interpreted and you can use the same interpreter > as a shell. It's exactly the opposite: the shell is an interpreter (written > in Erlang and compiled into binary code understood by the Erlang VM) that > evaluates Erlang expressions entered at the prompt and formatting their > results. This has a number of consequences regarding performance and > semantics, expressions evaluated by the shell do not have the same meaning > as code compiled into modules. > > To answer your question too: the rpc server runs independently of the > shell, and the latter may impose artificial restrictions on what it accepts > without affecting any other part of the system. > > > L. > > > On Tuesday, December 18, 2012 6:28:13 AM UTC+1, Son Tran-Nguyen wrote: >> >> Hello list, >> >> In one of my application, I start a restricted shell as a slave node, >> using a callback module as follow: >> >> The callback module, for example, returns {false, State} for >> gen_event:start/0 function to prevent that to be run. When tested in an >> actual restricted shell, it is prevented as expected: >> >> smba:eggs esente$ erl -pa apps/*/ebin -stdlib restricted_shell >> shell_callback >> Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:4:4] [async-threads:0] >> [hipe] [kernel-poll:false] [dtrace] >> >> Restricted Eshell V5.9.2 (abort with ^G) >> 1> gen_event:start(). >> ** exception exit: restricted shell does not allow gen_event:start() >> 2> >> >> However, when used in my app by making a call to the remote restricted >> shell using rpc:call/4, the call went through: >> >> Args = "-stdlib restricted_shell editor_shell", >> {ok, Node} = slave:start_link(smba, editor, Args), >> {ok, Entity} = rpc:call(Node, gen_event, start, []), >> >> It returns Entity as a pid(). >> >> I understand that the callback functions cannot be used to allow or >> disallow execution of functions called from compiled code, but only >> functions called from expressions entered at the shell prompt. >> >> I would like to confirm that rpc:call/4 does not enter the call into the >> remote shell, but doing some stuffs to run the call directly. >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Wed Dec 19 08:37:23 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 19 Dec 2012 08:37:23 +0100 Subject: [erlang-questions] clarify: run_erl and ssh Message-ID: <1355902643.4805.22.camel@sekic1152.release> Greetings, How do I get the prompt when pressing "ENTER" when I use "ssh" to start Erlang with "run_erl"? I have this script: #! /bin/sh run_erl -daemon /tmp/eleberg/ /tmp/eleberg/ "exec erl" It can be run like this: sekic1152 [8:24am] [/home/eleberg] -> ./afile And I get a nice interaction: sekic1152 [8:26am] [/home/eleberg] -> to_erl /tmp/eleberg/ Attaching to /tmp/eleberg/erlang.pipe.1 (^D to exit) 1> q(). ok 2> [End] For the second example I use "ssh" and the nice interaction gets worse: sekic1152 [8:26am] [/home/eleberg] -> ssh sekic1152 ./afile These computer resources, specifically Internet access and E-mail, are ... IF YOU ARE NOT AN AUTHORIZED USER, PLEASE EXIT IMMEDIATELY. sekic1152 [8:26am] [/home/eleberg] -> sekic1152 [8:26am] [/home/eleberg] -> to_erl /tmp/eleberg/ Attaching to /tmp/eleberg/erlang.pipe.1 (^D to exit) ^R q(). ok 2> [End] How can I persuade the prompt (1>) to show up upon "CR" in the second example? This is not important, as you can see it is possible to give command in the Erlang shell. I can even see them, which is nice. But pressing "CR" does not print the prompt and I would like that. bengt From barcojie@REDACTED Wed Dec 19 09:12:33 2012 From: barcojie@REDACTED (Barco You) Date: Wed, 19 Dec 2012 16:12:33 +0800 Subject: [erlang-questions] ANN: ezwebframe - an easy web framework In-Reply-To: References: Message-ID: Very nice idea. But how the browser understand the commands you send from erlang process? for example here: {cmd, fill_div} Thants to say, you have to define all the commands and relevant parsing fucntions in a Javascript library? Thanks, Barco On Thu, Dec 13, 2012 at 6:17 PM, Joe Armstrong wrote: > ezwebframe > ========== > > https://github.com/joearms/ezwebframe > > Pronounced "Easy web frame." > > About > ===== > > Ezwebframe attempts to make web programming just a little bit easier. > > From Erlang point of view the browser *is* an Erlang process. > > Assume we have a web page populated with divs. For example: > >
> ... >
> >
> ... >
> > Erlang thinks the browser is a process. To fill div a with HTML an > Erlang process evaluates the command: > > Browser ! [{cmd, fill_div}, {id, a}, {txt, B}] > > Where B is a binary containing HTML. > > In the browser controls can be programmed to send messages to Erlang, > for example, when we click on a button in the browser the Erlang > process controlling the window will be sent a message which can be > received with the statement: > > receive > {Browser, {struct, [{clicked, ButtonName}]}} -> > ... > end > > All this is achieved using a thin JSON layer over websockets and > with cowboy managing the websockets. > > Cheers > > /Joe > > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Wed Dec 19 09:13:05 2012 From: bourinov@REDACTED (Max Bourinov) Date: Wed, 19 Dec 2012 12:13:05 +0400 Subject: [erlang-questions] Parsing JSON Message-ID: Hi Erlangers, I want to know what do you use to parse JSON messages? I need to make calls to a REST API that returns JSON and then I need to parse it and extract some values from it. One of my candidates is mochijson. Do you use anything else? Best regards, Max -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy@REDACTED Wed Dec 19 09:14:47 2012 From: jeremy@REDACTED (Jeremy Ong) Date: Wed, 19 Dec 2012 00:14:47 -0800 Subject: [erlang-questions] Parsing JSON In-Reply-To: References: Message-ID: I like jsx https://github.com/talentdeficit/jsx On Wed, Dec 19, 2012 at 12:13 AM, Max Bourinov wrote: > Hi Erlangers, > > I want to know what do you use to parse JSON messages? > > I need to make calls to a REST API that returns JSON and then I need to > parse it and extract some values from it. One of my candidates is > mochijson. Do you use anything else? > > Best regards, > Max > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From barcojie@REDACTED Wed Dec 19 09:17:44 2012 From: barcojie@REDACTED (Barco You) Date: Wed, 19 Dec 2012 16:17:44 +0800 Subject: [erlang-questions] Parsing JSON In-Reply-To: References: Message-ID: I use jsx too, but never used mochijson. Hope to know someone have comparison between them. On Wed, Dec 19, 2012 at 4:14 PM, Jeremy Ong wrote: > I like jsx > > https://github.com/talentdeficit/jsx > > > On Wed, Dec 19, 2012 at 12:13 AM, Max Bourinov wrote: > >> Hi Erlangers, >> >> I want to know what do you use to parse JSON messages? >> >> I need to make calls to a REST API that returns JSON and then I need to >> parse it and extract some values from it. One of my candidates is >> mochijson. Do you use anything else? >> >> Best regards, >> Max >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dm.klionsky@REDACTED Wed Dec 19 09:20:55 2012 From: dm.klionsky@REDACTED (Dmitry Klionsky) Date: Wed, 19 Dec 2012 11:20:55 +0300 Subject: [erlang-questions] Parsing JSON In-Reply-To: References: Message-ID: <50D178E7.5030404@gmail.com> +1 On 12/19/2012 11:14 AM, Jeremy Ong wrote: > I like jsx > > https://github.com/talentdeficit/jsx > > > On Wed, Dec 19, 2012 at 12:13 AM, Max Bourinov > wrote: > > Hi Erlangers, > > I want to know what do you use to parse JSON messages? > > I need to make calls to a REST API that returns JSON and then I > need to parse it and extract some values from it. One of my > candidates is mochijson. Do you use anything else? > > Best regards, > Max > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Best regards, Dmitry Klionsky -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Dec 19 09:27:43 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 19 Dec 2012 11:27:43 +0300 Subject: [erlang-questions] Parsing JSON In-Reply-To: <50D178E7.5030404@gmail.com> References: <50D178E7.5030404@gmail.com> Message-ID: jiffy is really, really fast. By "really fast" I mean 10-40 times faster than native erlang parsing on moderate size JSON documents. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Wed Dec 19 09:43:56 2012 From: bourinov@REDACTED (Max Bourinov) Date: Wed, 19 Dec 2012 12:43:56 +0400 Subject: [erlang-questions] Parsing JSON In-Reply-To: References: <50D178E7.5030404@gmail.com> Message-ID: Hi Max, Thank you for reminding me that jiffy can not only encode, but also decode ))) I forgot about it. I will go with jiffy because we use it in many places. Thank you everybody for answering! On Wed, Dec 19, 2012 at 12:27 PM, Max Lapshin wrote: > jiffy is really, really fast. > > By "really fast" I mean 10-40 times faster than native erlang parsing on > moderate size JSON documents. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Wed Dec 19 12:32:52 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 19 Dec 2012 12:32:52 +0100 Subject: [erlang-questions] PropEr commercial training? In-Reply-To: References: Message-ID: <50D1A5E4.9080801@cs.ntua.gr> On 12/13/2012 01:24 PM, Motiejus Jak?tys wrote: > Dear Erlangers, > > Do you know about any property-based testing training using > PropEr? I'd be interested on hearing out about the types of > training available, the target audiences and how they've worked > in the past. Hi Motiejus, I am not sure what you consider as "commercial" training. Last year we offered a 3-hour PropEr tutorial in conjunction with the Erlang Factory in San Francisco. (*) It only covered "basic" proper functionality and the integration with the language of types and specs. There were about fifteen people attending it, all very familiar with Erlang programming, and it went well, I think. Besides that I know that there has been a handful of local Erlang user groups that have held PropEr tutorials or training sessions in various places around the world, typically in conjunction with Erlounges. But I am not aware of anyone who organizes training in PropEr on a commercial basis. (There is of course a company, Quviq, that provides courses in property-based testing but it does this for a different tool, which is not open source.) Hope this answers your question, Kostis (*) You can freely access the material of that tutorial here: http://proper.softlab.ntua.gr/Tutorials/Basic_PropEr_Tutorial.html From erlang@REDACTED Wed Dec 19 12:58:10 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 19 Dec 2012 12:58:10 +0100 Subject: [erlang-questions] ANN: ezwebframe - an easy web framework In-Reply-To: References: Message-ID: > On Wed, Dec 19, 2012 at 9:12 AM, Barco You wrote: > Very nice idea. > But how the browser understand the commands you send from erlang process? for > example here: {cmd, fill_div} This is easy, here's an overview: - Convert the Erlang command to JSON - send the JSON on the websocket uing Erlang - receive the JSON in Javascript - parse the JSON - build a Javascript command - execute the Javascript command Here are the details 0) assume there is a div xyz on the HTML page
...
1) We send a message to the browser: Browser ! [{cmd,'fill_div'},{div,xyz},{txt,<<"

Hello

">>}] This will fill the div with some HTML 2) The message [cmd,'fill_div'} ,,...] is converted to JSON and sent to the websocket: This is done in lines 227-229 of https://github.com/joearms/ezwebframe/blob/master/src/ezwebframe.erl websocket_info([{cmd,_}|_]=L, Req, Pid) -> B = list_to_binary(encode([{struct,L}])), {reply, {text, B}, Req, Pid, hibernate}; Here's an example of encode in the Erlang shell: > Cmd = [{cmd,fill_div},{id,xzy},{txt,<<"

Hello

">>}]. [{cmd,fill_div},{id,xyz},{txt,<<"

Hello

">>}] > list_to_binary(ezwebframe_mochijson2:encode([{struct,Cmd}])). <<"[{\"cmd\":\"fill_div\",\"id\":\"xyz\",\"txt\":\"

Hello

\"}]">> This is a bit unreadable because the quotes are escaped: The JSON term without escaping the quotes is: [{"cmd":"fill_div","id":"xyz","txt":"

Hello

"}] 3) The client code has an event handler that is triggered when the websocket receives data: Line 37 of websock.js is websocket.onmessage = onMessage; 4) onMessage is defined like this: function onMessage(evt) { var json = JSON.parse(evt.data); do_cmds(json); } function do_cmds(objs){ // console.log('do_cmds', objs); for(var i = 0; i < objs.length; i++){ var o = objs[i]; // as a safety measure we only evaluate js that is loaded if(eval("typeof("+o.cmd+")") == "function"){ eval(o.cmd + "(o)"); } else { // console.log('bad_cmd', o); alert("bad_command:"+o.cmd); }; }; } The JSON command on the websocket is of the form [{cmd:Cmd, ...}] After parsing with JSON.parse(evt.data) I can iterate over the commands which are stored in an array objs[...] The command I just send becomes the Javascript object o = {cmd:fill_div, id:'xyz', txt:'

Hello

'} I check that there really is a function called fill_div with this if(eval("typeof("+o.cmd+")") == "function"){ and if there is I do this: eval(o.cmd + "(o)"); now o.cmd = 'fill_div' so this is equivalent to eval('fill_div(o)') and so fill_div(o) is called where o = {cmd:fill_div, id:'xyz', txt:'

Hello

'} 5) Finally fill_div is defined like this (line 110 of https://github.com/joearms/ezwebframe/blob/master/priv/websock.js) function fill_div(o){ $('#'+o.id).html(o.txt); } This method is easily extended. Suppose Erlang sends a new command called color_page to change the background color of the page Browser ! [{cmd,color_page},{color,<<"green">>}] All you have to do is supply a javascript function called color_page: function color_page(o){ document.body.style.backgroundColor=o.color; } and make sure this gets loaded Cheers /Joe Thants to say, you have to define all the commands and relevant parsing > fucntions in a Javascript library? > > Thanks, > Barco > > On Thu, Dec 13, 2012 at 6:17 PM, Joe Armstrong wrote: > >> ezwebframe >> ========== >> >> https://github.com/joearms/ezwebframe >> >> Pronounced "Easy web frame." >> >> About >> ===== >> >> Ezwebframe attempts to make web programming just a little bit easier. >> >> From Erlang point of view the browser *is* an Erlang process. >> >> Assume we have a web page populated with divs. For example: >> >>
>> ... >>
>> >>
>> ... >>
>> >> Erlang thinks the browser is a process. To fill div a with HTML an >> Erlang process evaluates the command: >> >> Browser ! [{cmd, fill_div}, {id, a}, {txt, B}] >> >> Where B is a binary containing HTML. >> >> In the browser controls can be programmed to send messages to Erlang, >> for example, when we click on a button in the browser the Erlang >> process controlling the window will be sent a message which can be >> received with the statement: >> >> receive >> {Browser, {struct, [{clicked, ButtonName}]}} -> >> ... >> end >> >> All this is achieved using a thin JSON layer over websockets and >> with cowboy managing the websockets. >> >> Cheers >> >> /Joe >> >> >> >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Wed Dec 19 13:07:27 2012 From: bourinov@REDACTED (Max Bourinov) Date: Wed, 19 Dec 2012 16:07:27 +0400 Subject: [erlang-questions] ANN: ezwebframe - an easy web framework In-Reply-To: References: Message-ID: Joe, thank you for great explanation! Max On Wed, Dec 19, 2012 at 3:58 PM, Joe Armstrong wrote: > > On Wed, Dec 19, 2012 at 9:12 AM, Barco You wrote: > > Very nice idea. > > > But how the browser understand the commands you send from erlang > process? for > > example here: {cmd, fill_div} > > This is easy, here's an overview: > > - Convert the Erlang command to JSON > - send the JSON on the websocket uing Erlang > - receive the JSON in Javascript > - parse the JSON > - build a Javascript command > - execute the Javascript command > > Here are the details > > 0) assume there is a div xyz on the HTML page > >
...
> > 1) We send a message to the browser: > > Browser ! [{cmd,'fill_div'},{div,xyz},{txt,<<"

Hello

">>}] > > This will fill the div with some HTML > > 2) The message [cmd,'fill_div'} ,,...] is converted to JSON > and sent to the websocket: > > This is done in lines 227-229 of > https://github.com/joearms/ezwebframe/blob/master/src/ezwebframe.erl > > websocket_info([{cmd,_}|_]=L, Req, Pid) -> > B = list_to_binary(encode([{struct,L}])), > {reply, {text, B}, Req, Pid, hibernate}; > > > Here's an example of encode in the Erlang shell: > > > Cmd = [{cmd,fill_div},{id,xzy},{txt,<<"

Hello

">>}]. > [{cmd,fill_div},{id,xyz},{txt,<<"

Hello

">>}] > > list_to_binary(ezwebframe_mochijson2:encode([{struct,Cmd}])). > <<"[{\"cmd\":\"fill_div\",\"id\":\"xyz\",\"txt\":\"

Hello

\"}]">> > > This is a bit unreadable because the quotes are escaped: The JSON term > without escaping the quotes is: > > [{"cmd":"fill_div","id":"xyz","txt":"

Hello

"}] > > 3) The client code has an event handler that is triggered when the > websocket receives data: > > Line 37 of websock.js is > > websocket.onmessage = onMessage; > > 4) onMessage is defined like this: > > function onMessage(evt) { > var json = JSON.parse(evt.data); > do_cmds(json); > } > > function do_cmds(objs){ > // console.log('do_cmds', objs); > for(var i = 0; i < objs.length; i++){ > var o = objs[i]; > // as a safety measure we only evaluate js that is loaded > if(eval("typeof("+o.cmd+")") == "function"){ > eval(o.cmd + "(o)"); > } else { > // console.log('bad_cmd', o); > alert("bad_command:"+o.cmd); > }; > }; > } > > The JSON command on the websocket is of the form > > [{cmd:Cmd, ...}] > > After parsing with JSON.parse(evt.data) I can iterate over the commands > which are stored in an array objs[...] > > The command I just send becomes the Javascript object > > o = {cmd:fill_div, id:'xyz', txt:'

Hello

'} > > I check that there really is a function called fill_div > > with this > > if(eval("typeof("+o.cmd+")") == "function"){ > > and if there is I do this: > > eval(o.cmd + "(o)"); > > now o.cmd = 'fill_div' so this is equivalent to eval('fill_div(o)') > > and so > > fill_div(o) is called where > > o = {cmd:fill_div, id:'xyz', txt:'

Hello

'} > > 5) Finally fill_div is defined like this > > (line 110 of > https://github.com/joearms/ezwebframe/blob/master/priv/websock.js) > > function fill_div(o){ > $('#'+o.id).html(o.txt); > } > > This method is easily extended. > > Suppose Erlang sends a new command called color_page to change the > background color of the page > > Browser ! [{cmd,color_page},{color,<<"green">>}] > > All you have to do is supply a javascript function called color_page: > > function color_page(o){ > document.body.style.backgroundColor=o.color; > } > > > and make sure this gets loaded > > Cheers > > /Joe > > > > > Thants to say, you have to define all the commands and relevant parsing >> fucntions in a Javascript library? >> >> Thanks, >> Barco >> >> On Thu, Dec 13, 2012 at 6:17 PM, Joe Armstrong wrote: >> >>> ezwebframe >>> ========== >>> >>> https://github.com/joearms/ezwebframe >>> >>> Pronounced "Easy web frame." >>> >>> About >>> ===== >>> >>> Ezwebframe attempts to make web programming just a little bit easier. >>> >>> From Erlang point of view the browser *is* an Erlang process. >>> >>> Assume we have a web page populated with divs. For example: >>> >>>
>>> ... >>>
>>> >>>
>>> ... >>>
>>> >>> Erlang thinks the browser is a process. To fill div a with HTML an >>> Erlang process evaluates the command: >>> >>> Browser ! [{cmd, fill_div}, {id, a}, {txt, B}] >>> >>> Where B is a binary containing HTML. >>> >>> In the browser controls can be programmed to send messages to Erlang, >>> for example, when we click on a button in the browser the Erlang >>> process controlling the window will be sent a message which can be >>> received with the statement: >>> >>> receive >>> {Browser, {struct, [{clicked, ButtonName}]}} -> >>> ... >>> end >>> >>> All this is achieved using a thin JSON layer over websockets and >>> with cowboy managing the websockets. >>> >>> Cheers >>> >>> /Joe >>> >>> >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Wed Dec 19 13:50:02 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 19 Dec 2012 06:50:02 -0600 Subject: [erlang-questions] Protocol Mapping Message-ID: <056D3C2E-1E31-4791-8E47-882C600F1720@gmail.com> Hi, I'm frequently facing situations with binary protocol translations/transforms where I need to map binary codes to e.g. atoms and back. This is of course "easy" in Erlang, but suffers some inconveniences. I've attached a code snippet distilled down to the simplest case to explain my issue. I'm sure the first codec pattern below is familiar, and for sure is efficient. However, there are multiple places where updating is required to add new message types. The second pattern is much less usual, but handy as one line achieves the addition of a new message type. It helps particularly when there is more than one pairing involved (e.g. {atom, code, status_message}). For sure this cannot be something nobody has seen/thought about. I'm wondering if anyone has comment on this, and maybe suggestions for approaches that I haven't thought of. /s ------- -module(mapping). -compile(export_all). %% traditional -define(HELLO, 100). -define(HELP, 101). -define(DONE, 102). encode(hello) -> ?HELLO; encode(help) -> ?HELP; encode(done) -> ?DONE. decode(?HELLO) -> hello; decode(?HELP) -> help; decode(?DONE) -> done. %% alternative -define(MAP, [ {hello, 100}, {help, 101}, {done, 102} ]). encode0(X) -> {X, Y} = lists:keyfind(X, 1, ?MAP), Y. decode0(X) -> {Y, X} = lists:keyfind(X, 2, ?MAP), Y. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Wed Dec 19 13:59:51 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 19 Dec 2012 04:59:51 -0800 (PST) Subject: [erlang-questions] ANN: ezwebframe - an easy web framework In-Reply-To: References: Message-ID: Hi Joe, It occurs to me that the DOM selection model of D3 (http://d3js.org) may be a better fit than JQuery for this approach? regs, /s On Thursday, December 13, 2012 4:17:14 AM UTC-6, Joe Armstrong wrote: > > ezwebframe > ========== > > https://github.com/joearms/ezwebframe > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Wed Dec 19 14:05:53 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 19 Dec 2012 14:05:53 +0100 Subject: [erlang-questions] Protocol Mapping In-Reply-To: <056D3C2E-1E31-4791-8E47-882C600F1720@gmail.com> References: <056D3C2E-1E31-4791-8E47-882C600F1720@gmail.com> Message-ID: Hello! You could try to generate the code that implements the first pattern. 2012/12/19 Steve Davis : > Hi, > > I'm frequently facing situations with binary protocol > translations/transforms where I need to map binary codes to e.g. atoms and > back. > > This is of course "easy" in Erlang, but suffers some inconveniences. I've > attached a code snippet distilled down to the simplest case to explain my > issue. > > I'm sure the first codec pattern below is familiar, and for sure is > efficient. > However, there are multiple places where updating is required to add new > message types. > > The second pattern is much less usual, but handy as one line achieves the > addition of a new message type. > It helps particularly when there is more than one pairing involved (e.g. > {atom, code, status_message}). > > For sure this cannot be something nobody has seen/thought about. I'm > wondering if anyone has comment on this, and maybe suggestions for > approaches that I haven't thought of. > > /s > ------- > -module(mapping). > > -compile(export_all). > > %% traditional > > -define(HELLO, 100). > -define(HELP, 101). > -define(DONE, 102). > > encode(hello) -> ?HELLO; > encode(help) -> ?HELP; > encode(done) -> ?DONE. > > decode(?HELLO) -> hello; > decode(?HELP) -> help; > decode(?DONE) -> done. > > %% alternative > > -define(MAP, [ > {hello, 100}, > {help, 101}, > {done, 102} > ]). > > encode0(X) -> > {X, Y} = lists:keyfind(X, 1, ?MAP), > Y. > > decode0(X) -> > {Y, X} = lists:keyfind(X, 2, ?MAP), > Y. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From freza@REDACTED Wed Dec 19 13:54:10 2012 From: freza@REDACTED (Jachym Holecek) Date: Wed, 19 Dec 2012 07:54:10 -0500 Subject: [erlang-questions] Protocol Mapping In-Reply-To: <056D3C2E-1E31-4791-8E47-882C600F1720@gmail.com> References: <056D3C2E-1E31-4791-8E47-882C600F1720@gmail.com> Message-ID: <20121219125410.GA21090@circlewave.net> # Steve Davis 2012-12-19: > I'm frequently facing situations with binary protocol translations/transforms where I need to map binary codes to e.g. atoms and back. > > This is of course "easy" in Erlang, but suffers some inconveniences. I've attached a code snippet distilled down to the simplest case to explain my issue. > > I'm sure the first codec pattern below is familiar, and for sure is efficient. > However, there are multiple places where updating is required to add new message types. > > The second pattern is much less usual, but handy as one line achieves the addition of a new message type. > It helps particularly when there is more than one pairing involved (e.g. {atom, code, status_message}). > > For sure this cannot be something nobody has seen/thought about. I'm wondering if anyone has comment on this, and maybe suggestions for approaches that I haven't thought of. > > /s > ------- > -module(mapping). > > -compile(export_all). > > %% traditional > > -define(HELLO, 100). > -define(HELP, 101). > -define(DONE, 102). > > encode(hello) -> ?HELLO; > encode(help) -> ?HELP; > encode(done) -> ?DONE. > > decode(?HELLO) -> hello; > decode(?HELP) -> help; > decode(?DONE) -> done. > > %% alternative > > -define(MAP, [ > {hello, 100}, > {help, 101}, > {done, 102} > ]). > > encode0(X) -> > {X, Y} = lists:keyfind(X, 1, ?MAP), > Y. > > decode0(X) -> > {Y, X} = lists:keyfind(X, 2, ?MAP), > Y. Option 3: You already have macros in option one, can just use them everywhere. Ugly but the fastest you can get (no translation = no cost). Decoded messages will be very non-user-friendly though. Option 4: -record(foo_map, {key, val}). %% Installation procedure, only run once. install() -> %% XXX create the table with appropriate options if nonexistent. [install_one(K, C) || {K, C} <- fields()], ok. install_one(Key, Code) -> mnesia:dirty_write(#foo_map{key = {enc, Key}, val = Code}), mnesia:dirty_write(#foo_map{key = {dec, Code}, val = Key}). fields() -> [{hello, 1}, {help, 2}, {done, 3}]. %% Runtime interface. encode0(Key) -> ets:lookup_element(foo_map, {enc, Key}, #foo_map.val). decode0(Code) -> ets:lookup_element(foo_map, {dec, Code}, #foo_map.val). Option 5: Like option (1), only you write fun code that writes the boring code for you when needed (these tables tend to change infrequently). All definitions would be in your codegen module in a function similar to fields() above, there would be a few functions converting that to AST and using erl_pp to write out resulting source file. You than keep this under source control and remember to run codegen when you change master table (or teach it to some Makefile). HTH, -- Jachym From barcojie@REDACTED Wed Dec 19 15:47:17 2012 From: barcojie@REDACTED (Barco You) Date: Wed, 19 Dec 2012 22:47:17 +0800 Subject: [erlang-questions] ANN: ezwebframe - an easy web framework In-Reply-To: References: Message-ID: Hi Joe, Thank you very much for making a detailed illumination. That is data structure determines algorithm -- what kind of data you put in the Json object determines what kind of action you make at client side. Very good. Regards, Barco On Wednesday, December 19, 2012, Max Bourinov wrote: > Joe, thank you for great explanation! > > Max > > > > > On Wed, Dec 19, 2012 at 3:58 PM, Joe Armstrong wrote: > > > On Wed, Dec 19, 2012 at 9:12 AM, Barco You wrote: > > Very nice idea. > > > But how the browser understand the commands you send from erlang > process? for > > example here: {cmd, fill_div} > > This is easy, here's an overview: > > - Convert the Erlang command to JSON > - send the JSON on the websocket uing Erlang > - receive the JSON in Javascript > - parse the JSON > - build a Javascript command > - execute the Javascript command > > Here are the details > > 0) assume there is a div xyz on the HTML page > >
...
> > 1) We send a message to the browser: > > Browser ! [{cmd,'fill_div'},{div,xyz},{txt,<<"

Hello

">>}] > > This will fill the div with some HTML > > 2) The message [cmd,'fill_div'} ,,...] is converted to JSON > and sent to the websocket: > > This is done in lines 227-229 of > https://github.com/joearms/ezwebframe/blob/master/src/ezwebframe.erl > > websocket_info([{cmd,_}|_]=L, Req, Pid) -> > B = list_to_binary(encode([{struct,L}])), > {reply, {text, B}, Req, Pid, hibernate}; > > > Here's an example of encode in the Erlang shell: > > > Cmd = [{cmd,fill_div},{id,xzy},{txt,<<"

Hello

">>}]. > [{cmd,fill_div},{id,xyz},{txt,<<"

Hello

">>}] > > list_to_binary(ezwebframe_mochijson2:encode([{struct,Cmd}])). > <<"[{\"cmd\":\"fill_div\",\"id\":\"xyz\",\"txt\":\"

Hello

\"}]">> > > This is a bit unreadable because the quotes are escaped: The JSON term > without escaping the quotes is: > > [{"cmd":"fill_div","id":"xyz","txt":"

Hello

"}] > > 3) The client code has an event handler that is triggered when the > websocket receives data: > > Line 37 of websock.js is > > websocket.onmessage = onMessage; > > 4) onMessage is defined like this: > > function onMessage(evt) { > var json = JSON.parse(evt.data); > do_cmds(json); > } > > function do_cmds(objs){ > // console.log('do_cmds', objs); > for(var i = 0; i < objs.length; i++){ > var o = objs[i]; > // as a safety measure we only evaluate js that is loaded > if(eval("typeof("+o.cmd+")") == "function"){ > eval(o.cmd + "(o)"); > } else { > // console.log('bad_cmd', o); > alert("bad_command:"+o.cmd); > }; > }; > } > > The JSON command on the websocket is of the form > > [{cmd:Cmd, ...}] > > After parsing with JSON.parse(evt.data) I can iterate over the commands > which are stored in an array objs[...] > > The command I just send becomes the Javascript object > > o = {cmd:fill_div, id:'xyz', txt:'

Hello

'} > > I check that there really is a function called fill_div > > with this > > if(eval("typeof("+o.cmd+")") == "function"){ > > and if there is I do this: > > eval(o.cmd + "(o)"); > > now o.cmd = 'fill_div' so this is equivalent to eval('fill_div(o)') > > and so > > fill_div(o) is called where > > o = {cmd:fill_div, id:'xyz', txt:'

Hello

'} > > 5) Finally fill_div is defined like this > > (line 110 of > https://github.com/joearms/ezwebframe/blob/master/priv/websock.js) > > function fill_div(o){ > $('#'+o.id).html(o.txt); > } > > This method is easily extended. > > Suppose Erlang sends a new command called color_page to change the > background color of the page > > Browser ! [{cmd,color_page},{color,<<"green">>}] > > All you have to do is supply a javascript function called color_page: > > function color_page(o){ > document.body.style.backgroundColor=o.color; > } > > > and make sure this gets loaded > > Cheers > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From acidbriggs@REDACTED Wed Dec 19 16:34:41 2012 From: acidbriggs@REDACTED (acidbriggs@REDACTED) Date: Wed, 19 Dec 2012 10:34:41 -0500 Subject: [erlang-questions] ANN: ezwebframe - an easy web framework In-Reply-To: References: Message-ID: <18502BC4-4816-46BC-B0FC-36272B932079@gmail.com> IMHO, I would say that JQuery is a great choice for this as it makes it very easy to integrate this into current applications using JQuery. Since JQuery is probably the most used javascript library out there, it's a an awesome fit. That being said... The design of the ezwebframe API is very simple to understand an I would say that it would probably be very easy to do the same thing for D3. I love the idea of this. I reminds me a lot of the java framework called Ice Faces. But that is a different animal and it failed miserably in our testing. Given the history of Erlang, I bet this scales much better. Thanks Joe! --Briggs On Dec 19, 2012, at 7:59 AM, Steve Davis wrote: > Hi Joe, > > It occurs to me that the DOM selection model of D3 (http://d3js.org) may be a better fit than JQuery for this approach? > > regs, > /s > > On Thursday, December 13, 2012 4:17:14 AM UTC-6, Joe Armstrong wrote: > ezwebframe > ========== > > https://github.com/joearms/ezwebframe > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Wed Dec 19 16:47:47 2012 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 19 Dec 2012 09:47:47 -0600 Subject: [erlang-questions] Protocol Mapping In-Reply-To: <056D3C2E-1E31-4791-8E47-882C600F1720@gmail.com> References: <056D3C2E-1E31-4791-8E47-882C600F1720@gmail.com> Message-ID: This worked for me: -define(TRANSCODE(Atom,Int), transcode(Atom) -> Int; transcode(Int) -> Atom). ?TRANSCODE(hello, 100); ?TRANSCODE(help, 101); ?TRANSCODE(done, 102). Inspection of the expanded output with compile:file(File, ['P']). shows... transcode(hello) -> 100; transcode(100) -> hello; transcode(help) -> 101; transcode(101) -> help; transcode(done) -> 102; transcode(102) -> done. Dan. On Wed, Dec 19, 2012 at 6:50 AM, Steve Davis wrote: > Hi, > > I'm frequently facing situations with binary protocol > translations/transforms where I need to map binary codes to e.g. atoms and > back. > > This is of course "easy" in Erlang, but suffers some inconveniences. I've > attached a code snippet distilled down to the simplest case to explain my > issue. > > I'm sure the first codec pattern below is familiar, and for sure is > efficient. > However, there are multiple places where updating is required to add new > message types. > > The second pattern is much less usual, but handy as one line achieves the > addition of a new message type. > It helps particularly when there is more than one pairing involved (e.g. > {atom, code, status_message}). > > For sure this cannot be something nobody has seen/thought about. I'm > wondering if anyone has comment on this, and maybe suggestions for > approaches that I haven't thought of. > > /s > ------- > -module(mapping). > > -compile(export_all). > > %% traditional > > -define(HELLO, 100). > -define(HELP, 101). > -define(DONE, 102). > > encode(hello) -> ?HELLO; > encode(help) -> ?HELP; > encode(done) -> ?DONE. > > decode(?HELLO) -> hello; > decode(?HELP) -> help; > decode(?DONE) -> done. > > %% alternative > > -define(MAP, [ > {hello, 100}, > {help, 101}, > {done, 102} > ]). > > encode0(X) -> > {X, Y} = lists:keyfind(X, 1, ?MAP), > Y. > > decode0(X) -> > {Y, X} = lists:keyfind(X, 2, ?MAP), > Y. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Dec 19 16:50:16 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 19 Dec 2012 18:50:16 +0300 Subject: [erlang-questions] Protocol Mapping In-Reply-To: References: <056D3C2E-1E31-4791-8E47-882C600F1720@gmail.com> Message-ID: it is a very bad practice to have one function for atom() -> int() and int() -> atom() that will detect whether it is encoding or decoding. You will meet some unknown commands in protocol and decide to bypass them as an integer. Everything will break. On Wed, Dec 19, 2012 at 7:47 PM, Daniel Goertzen wrote: > This worked for me: > > -define(TRANSCODE(Atom,Int), transcode(Atom) -> Int; transcode(Int) -> > Atom). > > ?TRANSCODE(hello, 100); > ?TRANSCODE(help, 101); > ?TRANSCODE(done, 102). > > > > Inspection of the expanded output with compile:file(File, ['P']). shows... > > transcode(hello) -> > 100; > transcode(100) -> > hello; > transcode(help) -> > 101; > transcode(101) -> > help; > transcode(done) -> > 102; > transcode(102) -> > done. > > > Dan. > > On Wed, Dec 19, 2012 at 6:50 AM, Steve Davis < > steven.charles.davis@REDACTED> wrote: > >> Hi, >> >> I'm frequently facing situations with binary protocol >> translations/transforms where I need to map binary codes to e.g. atoms and >> back. >> >> This is of course "easy" in Erlang, but suffers some inconveniences. I've >> attached a code snippet distilled down to the simplest case to explain my >> issue. >> >> I'm sure the first codec pattern below is familiar, and for sure is >> efficient. >> However, there are multiple places where updating is required to add new >> message types. >> >> The second pattern is much less usual, but handy as one line achieves the >> addition of a new message type. >> It helps particularly when there is more than one pairing involved (e.g. >> {atom, code, status_message}). >> >> For sure this cannot be something nobody has seen/thought about. I'm >> wondering if anyone has comment on this, and maybe suggestions for >> approaches that I haven't thought of. >> >> /s >> ------- >> -module(mapping). >> >> -compile(export_all). >> >> %% traditional >> >> -define(HELLO, 100). >> -define(HELP, 101). >> -define(DONE, 102). >> >> encode(hello) -> ?HELLO; >> encode(help) -> ?HELP; >> encode(done) -> ?DONE. >> >> decode(?HELLO) -> hello; >> decode(?HELP) -> help; >> decode(?DONE) -> done. >> >> %% alternative >> >> -define(MAP, [ >> {hello, 100}, >> {help, 101}, >> {done, 102} >> ]). >> >> encode0(X) -> >> {X, Y} = lists:keyfind(X, 1, ?MAP), >> Y. >> >> decode0(X) -> >> {Y, X} = lists:keyfind(X, 2, ?MAP), >> Y. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmeiklejohn@REDACTED Wed Dec 19 16:53:16 2012 From: cmeiklejohn@REDACTED (Christopher Meiklejohn) Date: Wed, 19 Dec 2012 10:53:16 -0500 Subject: [erlang-questions] ANN: ezwebframe - an easy web framework In-Reply-To: References: Message-ID: On Wednesday, December 19, 2012 at 6:58 AM, Joe Armstrong wrote: > > But how the browser understand the commands you send from erlang process? for > > example here: {cmd, fill_div} > > > This is easy, here's an overview: > > - Convert the Erlang command to JSON > - send the JSON on the websocket uing Erlang > - receive the JSON in Javascript > - parse the JSON > - build a Javascript command > - execute the Javascript command Hi Joe, Thank you for putting this out there, I had a great time digging through the source code learning how the examples worked. I found it so inspiring that I went off and started playing around with some alternative approaches to handling the messages on the JavaScript side. See here: [1] I've been exploring having the messages, when received on the JavaScript side, distributed using a pub/sub approach so the Erlang code doesn't have to know about JavaScript function names or argument lists. I felt this made it feel a bit less RPC-ish and made it feel a bit more idiomatic. See below for examples. One thing that I couldn't get a hold of when playing with your code was if you intended to write *most* of the client side application in Erlang and just write a thin-layer in JavaScript for event handling and DOM-manipulation or if you were leaving that up to the developer. Again, thanks so much! -- For example (partially taken from the README): Given the following Erlang: BrowserPid ! {client_count, 1}. The following message is sent on the wire: { type: "client_count", message: "1" } Followed by the following message published via AmplifyJS: amplify.publish("client_count", 1); This allows multiple consumers in your JavaScript application to subscribe to these messages and respond to them accordingly. A subscriber could then call a method to update the DOM with that new value (via jQuery or whatever, this is no longer specific). [1] https://github.com/cmeiklejohn/piper -- Christopher Meiklejohn Software Engineer Basho Technologies, Inc. From daniel.goertzen@REDACTED Wed Dec 19 16:57:35 2012 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 19 Dec 2012 09:57:35 -0600 Subject: [erlang-questions] Protocol Mapping In-Reply-To: References: <056D3C2E-1E31-4791-8E47-882C600F1720@gmail.com> Message-ID: That does weaken things, but you can fix it with a wrapper: encode(Val) when is_atom(Val) -> transcode(Val). decode(Val) when is_integer(Val) -> transcode(Val). Or add a direction parameter to transcode(): -define(TRANSCODE(Atom,Int), transcode(decode,Atom) -> Int; transcode(encode,Int) -> Atom). ?TRANSCODE(hello, 100); yields... transcode(encode, hello) -> 100; transcode(decode, 100) -> hello; Dan. On Wed, Dec 19, 2012 at 9:50 AM, Max Lapshin wrote: > it is a very bad practice to have one function for atom() -> int() and > int() -> atom() that will detect whether it is encoding or decoding. > > You will meet some unknown commands in protocol and decide to bypass them > as an integer. Everything will break. > > > > On Wed, Dec 19, 2012 at 7:47 PM, Daniel Goertzen < > daniel.goertzen@REDACTED> wrote: > >> This worked for me: >> >> -define(TRANSCODE(Atom,Int), transcode(Atom) -> Int; transcode(Int) -> >> Atom). >> >> ?TRANSCODE(hello, 100); >> ?TRANSCODE(help, 101); >> ?TRANSCODE(done, 102). >> >> >> >> Inspection of the expanded output with compile:file(File, ['P']). >> shows... >> >> transcode(hello) -> >> 100; >> transcode(100) -> >> hello; >> transcode(help) -> >> 101; >> transcode(101) -> >> help; >> transcode(done) -> >> 102; >> transcode(102) -> >> done. >> >> >> Dan. >> >> On Wed, Dec 19, 2012 at 6:50 AM, Steve Davis < >> steven.charles.davis@REDACTED> wrote: >> >>> Hi, >>> >>> I'm frequently facing situations with binary protocol >>> translations/transforms where I need to map binary codes to e.g. atoms and >>> back. >>> >>> This is of course "easy" in Erlang, but suffers some >>> inconveniences. I've attached a code snippet distilled down to the simplest >>> case to explain my issue. >>> >>> I'm sure the first codec pattern below is familiar, and for sure is >>> efficient. >>> However, there are multiple places where updating is required to add new >>> message types. >>> >>> The second pattern is much less usual, but handy as one line achieves >>> the addition of a new message type. >>> It helps particularly when there is more than one pairing involved (e.g. >>> {atom, code, status_message}). >>> >>> For sure this cannot be something nobody has seen/thought about. I'm >>> wondering if anyone has comment on this, and maybe suggestions for >>> approaches that I haven't thought of. >>> >>> /s >>> ------- >>> -module(mapping). >>> >>> -compile(export_all). >>> >>> %% traditional >>> >>> -define(HELLO, 100). >>> -define(HELP, 101). >>> -define(DONE, 102). >>> >>> encode(hello) -> ?HELLO; >>> encode(help) -> ?HELP; >>> encode(done) -> ?DONE. >>> >>> decode(?HELLO) -> hello; >>> decode(?HELP) -> help; >>> decode(?DONE) -> done. >>> >>> %% alternative >>> >>> -define(MAP, [ >>> {hello, 100}, >>> {help, 101}, >>> {done, 102} >>> ]). >>> >>> encode0(X) -> >>> {X, Y} = lists:keyfind(X, 1, ?MAP), >>> Y. >>> >>> decode0(X) -> >>> {Y, X} = lists:keyfind(X, 2, ?MAP), >>> Y. >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Dec 19 17:00:35 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 19 Dec 2012 19:00:35 +0300 Subject: [erlang-questions] Protocol Mapping In-Reply-To: References: <056D3C2E-1E31-4791-8E47-882C600F1720@gmail.com> Message-ID: I've just described why you will fail with idea that atom() is an internal and int() is an external representation. -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Wed Dec 19 17:07:43 2012 From: g@REDACTED (Garrett Smith) Date: Wed, 19 Dec 2012 10:07:43 -0600 Subject: [erlang-questions] Parsing JSON In-Reply-To: References: <50D178E7.5030404@gmail.com> Message-ID: On Wed, Dec 19, 2012 at 2:27 AM, Max Lapshin wrote: > jiffy is really, really fast. > > By "really fast" I mean 10-40 times faster than native erlang parsing on > moderate size JSON documents. We use jiffy, without problems. Garrett From ulf@REDACTED Wed Dec 19 17:36:27 2012 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 19 Dec 2012 17:36:27 +0100 Subject: [erlang-questions] Protocol Mapping In-Reply-To: References: <056D3C2E-1E31-4791-8E47-882C600F1720@gmail.com> Message-ID: For fun, I made a version with parse_trans which generates a module: -module(mapping). -compile(export_all). -include_lib("parse_trans/include/codegen.hrl"). -define(MAP, [ {hello, 100}, {help, 101}, {done, 102} ]). codegen(Mod) -> codegen:gen_module( {'$var',Mod}, [{encode,1}, {decode,1}], [{encode, [fun({'$var',X}) -> {'$var', Y} end || {X, Y} <- ?MAP]}, {decode, [fun({'$var', Y}) -> {'$var', X} end || {X, Y} <- ?MAP]}]). Eshell V5.9.2 (abort with ^G) 1> c(mapping). {ok,mapping} 2> mapping:codegen(x). [{attribute,1,module,x}, {attribute,37,export,[{encode,1},{decode,1}]}, {function,39,encode,1, [{clause,39,[{atom,39,hello}],[],[{integer,40,100}]}, {clause,39,[{atom,39,help}],[],[{integer,40,101}]}, {clause,39,[{atom,39,done}],[],[{integer,40,102}]}]}, {function,42,decode,1, [{clause,42,[{integer,42,100}],[],[{atom,43,hello}]}, {clause,42,[{integer,42,101}],[],[{atom,43,help}]}, {clause,42,[{integer,42,102}],[],[{atom,43,done}]}]}] 3> compile:forms(v(2),[]). {ok,x, <<70,79,82,49,0,0,2,48,66,69,65,77,65,116,111,109,0,0,0, 71,0,0,0,9,1,120,...>>} 4> code:load_binary(x,"/tmp/x.beam",element(3,v(3))). {module,x} 5> x:encode(hello). 100 6> x:decode(100). hello The codegen:gen_module/3 pseudo-function is documented here: https://github.com/esl/parse_trans/blob/master/doc/parse_trans_codegen.md#gen_module3 BR, Ulf W On 19 Dec 2012, at 14:05, Attila Rajmund Nohl wrote: > Hello! > > You could try to generate the code that implements the first pattern. > > 2012/12/19 Steve Davis : >> Hi, >> >> I'm frequently facing situations with binary protocol >> translations/transforms where I need to map binary codes to e.g. atoms and >> back. >> >> This is of course "easy" in Erlang, but suffers some inconveniences. I've >> attached a code snippet distilled down to the simplest case to explain my >> issue. >> >> I'm sure the first codec pattern below is familiar, and for sure is >> efficient. >> However, there are multiple places where updating is required to add new >> message types. >> >> The second pattern is much less usual, but handy as one line achieves the >> addition of a new message type. >> It helps particularly when there is more than one pairing involved (e.g. >> {atom, code, status_message}). >> >> For sure this cannot be something nobody has seen/thought about. I'm >> wondering if anyone has comment on this, and maybe suggestions for >> approaches that I haven't thought of. >> >> /s >> ------- >> -module(mapping). >> >> -compile(export_all). >> >> %% traditional >> >> -define(HELLO, 100). >> -define(HELP, 101). >> -define(DONE, 102). >> >> encode(hello) -> ?HELLO; >> encode(help) -> ?HELP; >> encode(done) -> ?DONE. >> >> decode(?HELLO) -> hello; >> decode(?HELP) -> help; >> decode(?DONE) -> done. >> >> %% alternative >> >> -define(MAP, [ >> {hello, 100}, >> {help, 101}, >> {done, 102} >> ]). >> >> encode0(X) -> >> {X, Y} = lists:keyfind(X, 1, ?MAP), >> Y. >> >> decode0(X) -> >> {Y, X} = lists:keyfind(X, 2, ?MAP), >> Y. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From bourinov@REDACTED Wed Dec 19 17:44:32 2012 From: bourinov@REDACTED (Max Bourinov) Date: Wed, 19 Dec 2012 20:44:32 +0400 Subject: [erlang-questions] Parsing JSON In-Reply-To: References: <50D178E7.5030404@gmail.com> Message-ID: We also use jiffy )) Works just fine. Best regards, Max On Wed, Dec 19, 2012 at 8:07 PM, Garrett Smith wrote: > On Wed, Dec 19, 2012 at 2:27 AM, Max Lapshin > wrote: > > jiffy is really, really fast. > > > > By "really fast" I mean 10-40 times faster than native erlang parsing on > > moderate size JSON documents. > > We use jiffy, without problems. > > Garrett > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Dec 19 18:17:09 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 19 Dec 2012 18:17:09 +0100 Subject: [erlang-questions] Protocol Mapping In-Reply-To: References: <056D3C2E-1E31-4791-8E47-882C600F1720@gmail.com> Message-ID: On Wed, Dec 19, 2012 at 4:50 PM, Max Lapshin wrote: > it is a very bad practice to have one function for atom() -> int() and > int() -> atom() that will detect whether it is encoding or decoding. > > You will meet some unknown commands in protocol and decide to bypass them > as an integer. Everything will break. > I've been doing this for years :-) You shouldn't have unknown commands in the protocol, and if you do the only thing you can do is crash with a decent error message so you can debug your program later. Having crashed some supervision layer should try and repair the damage. But if you do only have atoms and integers and you stick to your conventions it will be fine. You should also add a final exit clause ... ?transcode(done, 102); transcode(Last) -> exit({transcode,bug,Last}). The important thing is to crash if you get an unknown code. Produce two errors - a polite error for the user, and a long and informative error in the log for the developer Cheers /Joe > > > > On Wed, Dec 19, 2012 at 7:47 PM, Daniel Goertzen < > daniel.goertzen@REDACTED> wrote: > >> This worked for me: >> >> -define(TRANSCODE(Atom,Int), transcode(Atom) -> Int; transcode(Int) -> >> Atom). >> >> ?TRANSCODE(hello, 100); >> ?TRANSCODE(help, 101); >> ?TRANSCODE(done, 102). >> >> >> >> Inspection of the expanded output with compile:file(File, ['P']). >> shows... >> >> transcode(hello) -> >> 100; >> transcode(100) -> >> hello; >> transcode(help) -> >> 101; >> transcode(101) -> >> help; >> transcode(done) -> >> 102; >> transcode(102) -> >> done. >> >> >> Dan. >> >> On Wed, Dec 19, 2012 at 6:50 AM, Steve Davis < >> steven.charles.davis@REDACTED> wrote: >> >>> Hi, >>> >>> I'm frequently facing situations with binary protocol >>> translations/transforms where I need to map binary codes to e.g. atoms and >>> back. >>> >>> This is of course "easy" in Erlang, but suffers some >>> inconveniences. I've attached a code snippet distilled down to the simplest >>> case to explain my issue. >>> >>> I'm sure the first codec pattern below is familiar, and for sure is >>> efficient. >>> However, there are multiple places where updating is required to add new >>> message types. >>> >>> The second pattern is much less usual, but handy as one line achieves >>> the addition of a new message type. >>> It helps particularly when there is more than one pairing involved (e.g. >>> {atom, code, status_message}). >>> >>> For sure this cannot be something nobody has seen/thought about. I'm >>> wondering if anyone has comment on this, and maybe suggestions for >>> approaches that I haven't thought of. >>> >>> /s >>> ------- >>> -module(mapping). >>> >>> -compile(export_all). >>> >>> %% traditional >>> >>> -define(HELLO, 100). >>> -define(HELP, 101). >>> -define(DONE, 102). >>> >>> encode(hello) -> ?HELLO; >>> encode(help) -> ?HELP; >>> encode(done) -> ?DONE. >>> >>> decode(?HELLO) -> hello; >>> decode(?HELP) -> help; >>> decode(?DONE) -> done. >>> >>> %% alternative >>> >>> -define(MAP, [ >>> {hello, 100}, >>> {help, 101}, >>> {done, 102} >>> ]). >>> >>> encode0(X) -> >>> {X, Y} = lists:keyfind(X, 1, ?MAP), >>> Y. >>> >>> decode0(X) -> >>> {Y, X} = lists:keyfind(X, 2, ?MAP), >>> Y. >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Dec 19 18:18:37 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 19 Dec 2012 18:18:37 +0100 Subject: [erlang-questions] ANN: ezwebframe - an easy web framework In-Reply-To: <18502BC4-4816-46BC-B0FC-36272B932079@gmail.com> References: <18502BC4-4816-46BC-B0FC-36272B932079@gmail.com> Message-ID: On Wed, Dec 19, 2012 at 4:34 PM, wrote: > IMHO, I would say that JQuery is a great choice for this as it makes it > very easy to integrate this into current applications using JQuery. Since > JQuery is probably the most used javascript library out there, it's a an > awesome fit. > > I'm pretty familiar with jQuery so it is rather easy > That being said... The design of the ezwebframe API is very simple to > understand an I would say that it would probably be very easy to do the > same thing for D3. > > I love the idea of this. I reminds me a lot of the java framework called > Ice Faces. But that is a different animal and it failed miserably in our > testing. Given the history of Erlang, I bet this scales much better. > > You really should try the IRC example in https://github.com/joearms/ezwebframe/blob/master/demos/chat/chat2.html This is 145 lines of code in total - this is *everything* the neat thing is that the program that manages the interface is also the IRC server there is only one process involved. >From a local point of view localhost:1456 *is* a web server but from a remote point of view it is an IRC server in a web page :-) I don't know if you've tried this: On machine one - note your IP adress (suppose it's 223.45.12.34) start ezwebframe and go to localhost:1456 and open the chat2.html page Open your firewall (if you have one) to allow remote access to port 1456 Go to any other machine on the planet and go to http://223.45.12.34:1456 And it all works. I've tried this on several machine and it works fine. When I get time I'll open up my firewall and we can try this (not today, I'm busy) I'd recommend anybody who has downloaded the code to try this to get the wow feeling. I was amazed that it worked so easily and was so short. I wrote this to update the chapter on IRC in my book - and found that an entire chapter vanished down to one page after I made the interface to web sockets. I think the key point is uniform messaging - the IRC server AND the web server are the same process and all they see are uniform messages (in this case JSON) - to get the browser to do things you send it JSON messages. When things happen in the browser you get sent JSON messages. The point is the bits fit together nicely - conceptual integrity rules. Cheers /Joe Thanks Joe! > > --Briggs > > > On Dec 19, 2012, at 7:59 AM, Steve Davis > wrote: > > Hi Joe, > > It occurs to me that the DOM selection model of D3 (http://d3js.org) may > be a better fit than JQuery for this approach? > > regs, > /s > > On Thursday, December 13, 2012 4:17:14 AM UTC-6, Joe Armstrong wrote: >> >> ezwebframe >> ========== >> >> https://github.com/joearms/**ezwebframe >> >> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Dec 19 18:28:05 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 19 Dec 2012 18:28:05 +0100 Subject: [erlang-questions] ANN: ezwebframe - an easy web framework In-Reply-To: References: Message-ID: On Wed, Dec 19, 2012 at 4:53 PM, Christopher Meiklejohn < cmeiklejohn@REDACTED> wrote: > On Wednesday, December 19, 2012 at 6:58 AM, Joe Armstrong wrote: > > > But how the browser understand the commands you send from erlang > process? for > > > example here: {cmd, fill_div} > > > > > > This is easy, here's an overview: > > > > - Convert the Erlang command to JSON > > - send the JSON on the websocket uing Erlang > > - receive the JSON in Javascript > > - parse the JSON > > - build a Javascript command > > - execute the Javascript command > > > Hi Joe, > > Thank you for putting this out there, I had a great time digging through > the source code learning how the examples worked. > > I found it so inspiring that I went off and started playing around with > some alternative approaches to handling the messages on the JavaScript > side. See here: [1] > > I've been exploring having the messages, when received on the JavaScript > side, distributed using a pub/sub approach so the Erlang code doesn't have > to know about JavaScript function names or argument lists. I felt this > made it feel a bit less RPC-ish and made it feel a bit more idiomatic. See > below for examples. > > One thing that I couldn't get a hold of when playing with your code was if > you intended to write *most* of the client side application in Erlang and > just write a thin-layer in JavaScript for event handling and > DOM-manipulation or if you were leaving that up to the developer. > It depends on the application - if you look at the viserl1.html it's all JavaScript and no Erlang - I was just playing around - I think for latency reasons you have to handle events in the browser (like drag and drop). For a complex application I'd view the bowser as a black box that did things then invent my own protocol to get it to do things. In another experiment I just made a page with one function on it that just does eval to anything you send it - this is just *too* general - but then again you'd only ever need one web page for *everything* /Joe > > Again, thanks so much! > > -- > > For example (partially taken from the README): > > Given the following Erlang: > > BrowserPid ! {client_count, 1}. > > The following message is sent on the wire: > { type: "client_count", message: "1" } > > Followed by the following message published via AmplifyJS: > amplify.publish("client_count", 1); > > This allows multiple consumers in your JavaScript application to subscribe > to these messages and respond to them accordingly. A subscriber could then > call a method to update the DOM with that new value (via jQuery or > whatever, this is no longer specific). > > [1] https://github.com/cmeiklejohn/piper > > -- > > Christopher Meiklejohn > Software Engineer > Basho Technologies, Inc. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.hughes@REDACTED Thu Dec 20 00:17:58 2012 From: john.hughes@REDACTED (John Hughes) Date: Thu, 20 Dec 2012 00:17:58 +0100 Subject: [erlang-questions] PropEr commercial training? In-Reply-To: <50D1A5E4.9080801@cs.ntua.gr> References: <50D1A5E4.9080801@cs.ntua.gr> Message-ID: <87C0BC1762AF4A11A49B943F3E50724A@HALL> Indeed, Motiejus, Quviq offers property-based testing courses of 2-3 days, on site at the customer. It works best to combine a course with another few days of consultancy to get people started on making specifications of their own code. There's a big difference between solving problems on a course, which have to be relatively small to be accessible in a short time, and tackling a real piece of software, and a few days of help testing your own code makes a big contribution to bridging that gap. We usually sell this service in conjunction with QuickCheck licences, but that's not a requirement. Of course, we use QuickCheck as the property-based testing tool, not PropEr. We supply licences for course participants for a limited time. We measure feedback on our courses, and they've been highly appreciated. If you're interested I can supply references. John Hughes -----Ursprungligt meddelande----- From: Kostis Sagonas Sent: Wednesday, December 19, 2012 12:32 PM To: Motiejus Jak?tys Cc: erlang-questions@REDACTED ; proper@REDACTED Subject: Re: [erlang-questions] PropEr commercial training? On 12/13/2012 01:24 PM, Motiejus Jak?tys wrote: > Dear Erlangers, > > Do you know about any property-based testing training using > PropEr? I'd be interested on hearing out about the types of > training available, the target audiences and how they've worked > in the past. Hi Motiejus, I am not sure what you consider as "commercial" training. Last year we offered a 3-hour PropEr tutorial in conjunction with the Erlang Factory in San Francisco. (*) It only covered "basic" proper functionality and the integration with the language of types and specs. There were about fifteen people attending it, all very familiar with Erlang programming, and it went well, I think. Besides that I know that there has been a handful of local Erlang user groups that have held PropEr tutorials or training sessions in various places around the world, typically in conjunction with Erlounges. But I am not aware of anyone who organizes training in PropEr on a commercial basis. (There is of course a company, Quviq, that provides courses in property-based testing but it does this for a different tool, which is not open source.) Hope this answers your question, Kostis (*) You can freely access the material of that tutorial here: http://proper.softlab.ntua.gr/Tutorials/Basic_PropEr_Tutorial.html _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From steven.charles.davis@REDACTED Thu Dec 20 01:15:13 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 19 Dec 2012 18:15:13 -0600 Subject: [erlang-questions] Protocol Mapping In-Reply-To: References: <056D3C2E-1E31-4791-8E47-882C600F1720@gmail.com> Message-ID: <99962F45-9194-4ACA-A23B-42DA19AC4D36@gmail.com> Wow - an embarrassment of response riches. I love the macro approach simplicity; I hear Max; and I wonder if the extra step involved with the transform may be worth it for the flexibility I need for n-tuple mappings. Will take me a bit of thinking time/prototyping to digest all this properly. Thank you all, as ever, for your help, expertise, time and responses. regs, /s From ingela.andin@REDACTED Thu Dec 20 09:33:42 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 20 Dec 2012 09:33:42 +0100 Subject: [erlang-questions] SSL decrypt error during SSL handshake In-Reply-To: References: Message-ID: Hi! Decryption error happens if the inputdata is somehow corrupted. I can not think of an obvious reason that this could happen with you setup, but your server is using a fairly old version so upgrading might be a good idea regardless. Regards Ingela Erlang/OTP team - Ericsson AB 2012/12/15, Kaiduan Xie : > Hi, > > I ran into a situation where server sends back SSL decrypt error to > client during SSL handshake, both client and server are written in > Erlang. The SSL handshake looks as below, > > 1) Client sends Client Hello to server > 2) Server sends Server Hello back > 3) Server sends Certificate, Server Key Exchange and Server Hello Done > 4) Client sends Client Key Exchange > 5) Client sends Change Cipher Spec, Encrypted Handshake Message > 6) Server sends Alert (Level: Fatal, Description: Decrypt Error) > > The certificate is from godaddy, any idea why server sends Decrypt Error? > > The server is running R14B01 while the client is running R15B02. > > Thanks, > > /Kaiduan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From pan@REDACTED Thu Dec 20 10:49:15 2012 From: pan@REDACTED (Patrik Nyblom) Date: Thu, 20 Dec 2012 10:49:15 +0100 Subject: [erlang-questions] clarify: run_erl and ssh In-Reply-To: <1355902643.4805.22.camel@sekic1152.release> References: <1355902643.4805.22.camel@sekic1152.release> Message-ID: <50D2DF1B.3000906@erlang.org> On 12/19/2012 08:37 AM, Bengt Kleberg wrote: > Greetings, > > How do I get the prompt when pressing "ENTER" when I use "ssh" to start > Erlang with "run_erl"? > > I have this script: > > #! /bin/sh > run_erl -daemon /tmp/eleberg/ /tmp/eleberg/ "exec erl" > > It can be run like this: > sekic1152 [8:24am] [/home/eleberg] -> ./afile > > And I get a nice interaction: > > sekic1152 [8:26am] [/home/eleberg] -> to_erl /tmp/eleberg/ > Attaching to /tmp/eleberg/erlang.pipe.1 (^D to exit) > > > 1> q(). > ok > 2> [End] > > For the second example I use "ssh" and the nice interaction gets worse: > > sekic1152 [8:26am] [/home/eleberg] -> ssh sekic1152 ./afile > These computer resources, specifically Internet access and E-mail, are > ... > IF YOU ARE NOT AN AUTHORIZED USER, PLEASE EXIT IMMEDIATELY. > sekic1152 [8:26am] [/home/eleberg] -> > > > sekic1152 [8:26am] [/home/eleberg] -> to_erl /tmp/eleberg/ > Attaching to /tmp/eleberg/erlang.pipe.1 (^D to exit) > > ^R > > > > > q(). > ok > 2> [End] > > > How can I persuade the prompt (1>) to show up upon "CR" in the second > example? I guess the PTY does not get initialized correctly when you run via ssh. I'd try something like setting TERM=vt100 before the run_erl to inform the program that you have a terminal capable of running "newshell", i.e. the interactive shell we're all used to. > > This is not important, as you can see it is possible to give command in > the Erlang shell. I can even see them, which is nice. But pressing "CR" > does not print the prompt and I would like that. > > > bengt > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions CHeers, /Patrik From sarva.v@REDACTED Thu Dec 20 11:34:25 2012 From: sarva.v@REDACTED (Saravanan Vijayakumaran) Date: Thu, 20 Dec 2012 16:04:25 +0530 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: I tried running etop to profile the simulation. The commands I used and a snapshot of the etop data are below. The nsime_simulator process is undergoing a lot of reductions. But the message queue is zero. I was expecting to see a lot of messages there because 10,000 event schedule calls each result in a gen_server:call to nsime_simulator. The nsime_simulator contains the event list gb_trees and hence it occupies a lot of memory. So that is not surprising. The plists module is used to run the simultaneous events in parallel. It is occupying the maximum memory. I don't understand how it can occupy more memory than the nsime_simulator itself. I will have to look at the code again to figure this out. I will keep you posted. Thanks for all your suggestions. Simulation command erl -sname try -noshell +P 1000000 -pa ebin/ -pa examples/ebin/ -run udp_cs_pairs start 10000 -run init stop etop command: /usr/lib/erlang/lib/observer-0.9.10/priv/bin/getop -node try@REDACTED off ======================================================================================== 'try@REDACTED' 10:26:35 Load: cpu 177 Memory: total 784544 binary 105148 procs 130041 processes 664962 code 1888 runq 4 atom 143 ets 144 Pid Name or Initial Func Time Reds Memory MsgQ Current Function ---------------------------------------------------------------------------------------- <4696.2.0> erlang:apply/2 '-' 70800 13799852 0 plists:cluster_runmany <4696.35.0> nsime_simulator '-' 2315977 10471532 0 gen_server:loop/6 <4696.31801.3> erlang:apply/2 '-' 70568 556908 1 gen:do_call/4 <4696.31804.3> erlang:apply/2 '-' 77260 344376 1 io:wait_io_mon_reply <4696.31800.3> erlang:apply/2 '-' 72686 344360 0 io:wait_io_mon_reply <4696.31802.3> erlang:apply/2 '-' 75552 344360 0 io:wait_io_mon_reply <4696.31803.3> erlang:apply/2 '-' 74720 344360 0 io:wait_io_mon_reply <4696.37.0> nsime_node_list '-' 0 185928 0 gen_server:loop/6 <4696.25.0> code_server '-' 0 142164 0 code_server:loop/1 <4696.38.0> nsime_channel_list '-' 0 142144 0 gen_server:loop/6 ======================================================================================== I also get a bunch of messages in the etop window if I don't set "-tracing off". Erlang top dropped data 151 Erlang top got garbage {trace_ts,<4695.18979.3>,out, {gen_server,loop,6}, {1355,998744,703693}} Erlang top got garbage {trace_ts,<4695.8399.1>,out, {gen,do_call,4}, {1355,998744,703731}} On Sun, Dec 16, 2012 at 7:46 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Dec 15, 2012, at 9:19 PM, Garrett Smith wrote: > > > I've lost track of this thread -- are you still guessing, or have you > > spent any time measuring? I looked back and couldn't see any > > discussion about e.g. specific processes with high reduction counts, > > growing queue sizes, etc. > > Sometimes, a well-placed eprof measurement does wonders to tell you what > is slow. We had a system that was spending 40% of its time in a specific > regular expression once, but this is hard to see unless you profile. I'd > say you should look at processes with high reduction counts, or groups of > those and then look into why they are reducing all the time. > > But I have a hunch that this has more to do with the fact that the > simulation is serial more than parallel. If you want to utilize more than > one core then, you need to figure out how the problem can be split up. > > Jesper Louis Andersen > Erlang Solutions Ltd., Copenhagen > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From redvasily@REDACTED Thu Dec 20 13:56:03 2012 From: redvasily@REDACTED (Vasily Sulatskov) Date: Thu, 20 Dec 2012 04:56:03 -0800 (PST) Subject: [erlang-questions] Several gen_leader questions In-Reply-To: References: Message-ID: <8f0e6d10-2987-45f8-919f-e322ed54c2c0@googlegroups.com> Bumping the thread in hope that someone will answer. On Tuesday, December 18, 2012 8:32:13 PM UTC+1, Vasily Sulatskov wrote: > > Hi, > > I am using gen_leader from https://github.com/abecciu/gen_leader_revivaland I > want to use it with a fixed list of candidate nodes - as far as I > understand > that's the easiest way. > > I tried several variations of starting gen_leader but none was > satisfactory. > > I start it as part of a supervision tree, so all examples are taken from > gen_supervisor modules init/1 functions: > > Here's what I tried so far: > > init(_Args) -> > %% The same value is used on all machines in the cluster > Leader_nodes = ['foobar@REDACTED', 'foobar@REDACTED', 'foobar@REDACTED'], > > Home = os:getenv("HOME"), > > Gen_leader_config = {gen_leader_module, {gen_leader_module, > start_link, > [Leader_nodes, > [{vardir, Home}]]}, > permanent, 2000, worker, [gen_leader_module]}, > > {ok, {{one_for_one, 10000, 1}, > [Gen_leader_config]}}. > > If I do this, then nodes specified in Leader_nodes work just fine, they > all > participate in elections, leaders are elected properly, they are able to > do > gen_leader:leader_call() to the actual leader etc. > > The problem is that on all other nodes (which are not specified in > Leader_nodes) gen_leader is not started at all. Gen_leader checks if the > node > it's running on is one of "candidate nodes" or "worker nodes" and if > that's not > the case - it simply doesn't start. All further attempts at > gen_leader:leader_call from that node fail. > > I tried to run every node in the cluster except for "candidate nodes" as a > "worker node", so I changed supervisor to something like: > > init(_Args) -> > %% The same value is used on all machines in the cluster > Leader_nodes = ['foobar@REDACTED', 'foobar@REDACTED', 'foobar@REDACTED'], > > Workers = > case lists:member(node(), Leader_nodes) of > true -> > []; > false -> > [node()] > end, > > Home = os:getenv("HOME"), > > {ok, {{one_for_one, 10000, 1}, > [{scheduler, {scheduler, start_link, > [Leader_nodes, > [{vardir, Home}, > {workers, Workers}]]}, > permanent, 2000, worker, [scheduler_leader]}]}}. > > As far as I understand, when gen_leader runs in a worker configuration, it > doesn't participate in elections, but still keeps track of where an actual > leader is running, so gen_leader:leader_call is still possible. > > This setup kind of works, but it seems that gen_leader process on "worker" > nodes > constantly grows in memory usage, past several Gb at least, eventually > crashing > the whole VM. > > Am I running gen_leader correctly? > > What is the correct way of running gen_leader with a fixed set of > "candidate" > nodes and that every other node is aware of where a leader is running, so > that > gen_leader:leader_call() is possible? > > Which version of gen_leader is recommended to use? This one > https://github.com/abecciu/gen_leader_revival? Or maybe the version from > gproc? By the way can someone explain what's the difference between them? > > > And I have another, most likely unrelated, issue with gen_leader. On one > deployment, sometimes I find a cluster in a state with two leaders - most > of > the nodes think that the leader is one node, but some other node thinks > that the leader is on the other node. I am not sure if the other leader is > the > node that diverges from the consensus - I don't have a cluster in this > state > right now to check. > > It seems to happen after a gen_leader process crashes somehow (some > internal > work, not related to gen_leader magic). > > The other thing that I think might be important here, is that gen_leader > process in that setup can get stuck in handle_leader_call for quite some > long > time. Can it cause problems with leader elections? Should gen_leader > processes > not block in handle_whatever functions and always be able to handle > election > callback? > > > Thanks in advance. > > -- > Best regards, > Vasily Sulatskov > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Thu Dec 20 15:29:20 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 20 Dec 2012 09:29:20 -0500 Subject: [erlang-questions] =?utf-8?q?Meetup_in_Montreal_tonight_/_=C3=A0_?= =?utf-8?q?Montr=C3=A9al_ce_soir?= Message-ID: <20121220142919.GA41927@ferdmbp.local> Hi everyone in Montreal or around, I'm sending this e-mail to let you know we're organizing a user group meeting tonight for Erlang programmers of the area. It should start at around 18:30, and it should take place in our offices at AdGear Technologies, Inc., located at 481 Avenue Viger West, suite 300, in Montreal (you can get a map at http://adgear.com/contact/) No idea how many of us will be there in the end, but it would be nice meeting other Erlangers in the area. Regards, Fred. From per@REDACTED Thu Dec 20 15:56:05 2012 From: per@REDACTED (Per Hedeland) Date: Thu, 20 Dec 2012 15:56:05 +0100 (CET) Subject: [erlang-questions] clarify: run_erl and ssh In-Reply-To: <50D2DF1B.3000906@erlang.org> Message-ID: <201212201456.qBKEu5Qf020673@pluto.hedeland.org> Patrik Nyblom wrote: > >On 12/19/2012 08:37 AM, Bengt Kleberg wrote: [snip] >> sekic1152 [8:26am] [/home/eleberg] -> ssh sekic1152 ./afile [snip] >I guess the PTY does not get initialized correctly when you run via ssh. Or rather, in the above command, there is no pty allocated *at all* on 'sekic1152'. Try 'ssh -t sekic1152 ./afile' instead. 'man ssh'. --Per Hedeland From daniel@REDACTED Thu Dec 20 16:39:27 2012 From: daniel@REDACTED (Daniel Luna) Date: Thu, 20 Dec 2012 10:39:27 -0500 Subject: [erlang-questions] SSL decrypt error during SSL handshake In-Reply-To: References: Message-ID: Most likely this is the same bug that bit us a while ago. With certificates from GoDaddy. Here http://erlang.org/pipermail/erlang-bugs/2012-August/002996.html plus related emails. This was fixed in R15B02. From the release notes: OTP-10222 Workaround for handling certificates that wrongly encode X509countryname in utf-8 when the actual value is a valid ASCCI value of length 2. Such certificates are accepted by many browsers such as Chrome and Fierfox so for interoperability reasons we will too. Cheers, Daniel On 20 December 2012 03:33, Ingela Andin wrote: > Hi! > > Decryption error happens if the inputdata is somehow corrupted. > I can not think of an obvious reason that this could happen with you setup, but > your server is using a fairly old version so upgrading might be a good > idea regardless. > > Regards Ingela Erlang/OTP team - Ericsson AB > > > 2012/12/15, Kaiduan Xie : >> Hi, >> >> I ran into a situation where server sends back SSL decrypt error to >> client during SSL handshake, both client and server are written in >> Erlang. The SSL handshake looks as below, >> >> 1) Client sends Client Hello to server >> 2) Server sends Server Hello back >> 3) Server sends Certificate, Server Key Exchange and Server Hello Done >> 4) Client sends Client Key Exchange >> 5) Client sends Change Cipher Spec, Encrypted Handshake Message >> 6) Server sends Alert (Level: Fatal, Description: Decrypt Error) >> >> The certificate is from godaddy, any idea why server sends Decrypt Error? >> >> The server is running R14B01 while the client is running R15B02. >> >> Thanks, >> >> /Kaiduan >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Thu Dec 20 16:42:12 2012 From: g@REDACTED (Garrett Smith) Date: Thu, 20 Dec 2012 09:42:12 -0600 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: Hi Saravanan, While you're running these scenarios, keep an eye on your Erlang process as well -- e.g. with top, pidstat, etc. CPU and memory the two usual suspects to watch. High reductions is just a relative indicator until you start to see your CPU bottlenecking. I.e. "high" might not matter in the least if the applicable processes are keeping up with demand. If you're not seeing an actual bottleneck at typical load, you might drop this as a problem (I probably would :) -- or work to increase load until you start to see something fall over, at which point you now have the tools to look more closely. Garrett On Thu, Dec 20, 2012 at 4:34 AM, Saravanan Vijayakumaran wrote: > I tried running etop to profile the simulation. The commands I used and a > snapshot of the etop data are below. The nsime_simulator process is > undergoing a lot of reductions. But the message queue is zero. I was > expecting to see a lot of messages there because 10,000 event schedule calls > each result in a gen_server:call to nsime_simulator. The nsime_simulator > contains the event list gb_trees and hence it occupies a lot of memory. So > that is not surprising. The plists module is used to run the simultaneous > events in parallel. It is occupying the maximum memory. I don't understand > how it can occupy more memory than the nsime_simulator itself. I will have > to look at the code again to figure this out. I will keep you posted. Thanks > for all your suggestions. > > Simulation command > erl -sname try -noshell +P 1000000 -pa ebin/ -pa examples/ebin/ -run > udp_cs_pairs start 10000 -run init stop > > etop command: > /usr/lib/erlang/lib/observer-0.9.10/priv/bin/getop -node try@REDACTED > -tracing off > > > ======================================================================================== > 'try@REDACTED' > 10:26:35 > Load: cpu 177 Memory: total 784544 binary > 105148 > procs 130041 processes 664962 code > 1888 > runq 4 atom 143 ets > 144 > > Pid Name or Initial Func Time Reds Memory MsgQ Current > Function > ---------------------------------------------------------------------------------------- > <4696.2.0> erlang:apply/2 '-' 70800 13799852 0 > plists:cluster_runmany > <4696.35.0> nsime_simulator '-' 2315977 10471532 0 > gen_server:loop/6 > <4696.31801.3> erlang:apply/2 '-' 70568 556908 1 > gen:do_call/4 > <4696.31804.3> erlang:apply/2 '-' 77260 344376 1 > io:wait_io_mon_reply > <4696.31800.3> erlang:apply/2 '-' 72686 344360 0 > io:wait_io_mon_reply > <4696.31802.3> erlang:apply/2 '-' 75552 344360 0 > io:wait_io_mon_reply > <4696.31803.3> erlang:apply/2 '-' 74720 344360 0 > io:wait_io_mon_reply > <4696.37.0> nsime_node_list '-' 0 185928 0 > gen_server:loop/6 > <4696.25.0> code_server '-' 0 142164 0 > code_server:loop/1 > <4696.38.0> nsime_channel_list '-' 0 142144 0 > gen_server:loop/6 > ======================================================================================== > > I also get a bunch of messages in the etop window if I don't set "-tracing > off". > > Erlang top dropped data 151 > Erlang top got garbage {trace_ts,<4695.18979.3>,out, > {gen_server,loop,6}, > {1355,998744,703693}} > Erlang top got garbage {trace_ts,<4695.8399.1>,out, > {gen,do_call,4}, > {1355,998744,703731}} > > > > On Sun, Dec 16, 2012 at 7:46 PM, Jesper Louis Andersen > wrote: >> >> >> On Dec 15, 2012, at 9:19 PM, Garrett Smith wrote: >> >> > I've lost track of this thread -- are you still guessing, or have you >> > spent any time measuring? I looked back and couldn't see any >> > discussion about e.g. specific processes with high reduction counts, >> > growing queue sizes, etc. >> >> Sometimes, a well-placed eprof measurement does wonders to tell you what >> is slow. We had a system that was spending 40% of its time in a specific >> regular expression once, but this is hard to see unless you profile. I'd say >> you should look at processes with high reduction counts, or groups of those >> and then look into why they are reducing all the time. >> >> But I have a hunch that this has more to do with the fact that the >> simulation is serial more than parallel. If you want to utilize more than >> one core then, you need to figure out how the problem can be split up. >> >> Jesper Louis Andersen >> Erlang Solutions Ltd., Copenhagen >> >> >> > From jesper.louis.andersen@REDACTED Thu Dec 20 17:38:45 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 20 Dec 2012 17:38:45 +0100 Subject: [erlang-questions] gen_server bottleneck In-Reply-To: References: Message-ID: <13A12C61-F879-4E9E-987B-A83FC47ED65C@erlang-solutions.com> A couple of quick comments: Perhaps you don't really have much parallel execution if everyone is waiting on a go on the main simulator process. This could explain the small msg queue. A call is synchronous, so it incurs lots and lots of waiting. io:wait_mon_reply suggests you are outputting debug information. Don't. It is slowing you down. Use tracing instead if you need to dig into code. If the storage in gb_trees are very small values, consider storing them outside the process heap in ETS and use ets:lookup_element/3 to pick out the value. This avoids GC'ing and allocation in the simulator and it is often faster. do try to run eprof on the simulator to verify it uses all its time in gb_trees functions. The plists could have been passed a very large data structure it then keeps in memory. eprof it, or ask about its current state. Also consider not passing around data if it can mostly be stored in ETS or you can keep a process alive with the data in it. Passing around large structures is rather expensive when copying is involved, so it is often better to have a process play the role of the data. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen On Dec 20, 2012, at 11:34 AM, Saravanan Vijayakumaran wrote: > I tried running etop to profile the simulation. The commands I used and a snapshot of the etop data are below. The nsime_simulator process is undergoing a lot of reductions. But the message queue is zero. I was expecting to see a lot of messages there because 10,000 event schedule calls each result in a gen_server:call to nsime_simulator. The nsime_simulator contains the event list gb_trees and hence it occupies a lot of memory. So that is not surprising. The plists module is used to run the simultaneous events in parallel. It is occupying the maximum memory. I don't understand how it can occupy more memory than the nsime_simulator itself. I will have to look at the code again to figure this out. I will keep you posted. Thanks for all your suggestions. > > Simulation command > erl -sname try -noshell +P 1000000 -pa ebin/ -pa examples/ebin/ -run udp_cs_pairs start 10000 -run init stop > > etop command: > /usr/lib/erlang/lib/observer-0.9.10/priv/bin/getop -node try@REDACTED -tracing off > > > ======================================================================================== > 'try@REDACTED' 10:26:35 > Load: cpu 177 Memory: total 784544 binary 105148 > procs 130041 processes 664962 code 1888 > runq 4 atom 143 ets 144 > > Pid Name or Initial Func Time Reds Memory MsgQ Current Function > ---------------------------------------------------------------------------------------- > <4696.2.0> erlang:apply/2 '-' 70800 13799852 0 plists:cluster_runmany > <4696.35.0> nsime_simulator '-' 2315977 10471532 0 gen_server:loop/6 > <4696.31801.3> erlang:apply/2 '-' 70568 556908 1 gen:do_call/4 > <4696.31804.3> erlang:apply/2 '-' 77260 344376 1 io:wait_io_mon_reply > <4696.31800.3> erlang:apply/2 '-' 72686 344360 0 io:wait_io_mon_reply > <4696.31802.3> erlang:apply/2 '-' 75552 344360 0 io:wait_io_mon_reply > <4696.31803.3> erlang:apply/2 '-' 74720 344360 0 io:wait_io_mon_reply > <4696.37.0> nsime_node_list '-' 0 185928 0 gen_server:loop/6 > <4696.25.0> code_server '-' 0 142164 0 code_server:loop/1 > <4696.38.0> nsime_channel_list '-' 0 142144 0 gen_server:loop/6 > ======================================================================================== > > I also get a bunch of messages in the etop window if I don't set "-tracing off". > > Erlang top dropped data 151 > Erlang top got garbage {trace_ts,<4695.18979.3>,out, > {gen_server,loop,6}, > {1355,998744,703693}} > Erlang top got garbage {trace_ts,<4695.8399.1>,out, > {gen,do_call,4}, > {1355,998744,703731}} > > > > On Sun, Dec 16, 2012 at 7:46 PM, Jesper Louis Andersen wrote: > > On Dec 15, 2012, at 9:19 PM, Garrett Smith wrote: > > > I've lost track of this thread -- are you still guessing, or have you > > spent any time measuring? I looked back and couldn't see any > > discussion about e.g. specific processes with high reduction counts, > > growing queue sizes, etc. > > Sometimes, a well-placed eprof measurement does wonders to tell you what is slow. We had a system that was spending 40% of its time in a specific regular expression once, but this is hard to see unless you profile. I'd say you should look at processes with high reduction counts, or groups of those and then look into why they are reducing all the time. > > But I have a hunch that this has more to do with the fact that the simulation is serial more than parallel. If you want to utilize more than one core then, you need to figure out how the problem can be split up. > > Jesper Louis Andersen > Erlang Solutions Ltd., Copenhagen > > > > From kaiduanx@REDACTED Thu Dec 20 17:58:20 2012 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Thu, 20 Dec 2012 11:58:20 -0500 Subject: [erlang-questions] SSL decrypt error during SSL handshake In-Reply-To: References: Message-ID: Ingela and Daniel, Thank you very much for the response. I upgraded the server to R15B03, the problem is still there. You can refer the attached pcap file on the server. Best regards, /Kaiduan On Thu, Dec 20, 2012 at 10:39 AM, Daniel Luna wrote: > Most likely this is the same bug that bit us a while ago. With > certificates from GoDaddy. > > Here http://erlang.org/pipermail/erlang-bugs/2012-August/002996.html > plus related emails. > > This was fixed in R15B02. From the release notes: > > OTP-10222 Workaround for handling certificates that wrongly encode > X509countryname in utf-8 when the actual value is a valid > ASCCI value of length 2. Such certificates are accepted by > many browsers such as Chrome and Fierfox so for > interoperability reasons we will too. > > Cheers, > > Daniel > > On 20 December 2012 03:33, Ingela Andin wrote: >> Hi! >> >> Decryption error happens if the inputdata is somehow corrupted. >> I can not think of an obvious reason that this could happen with you setup, but >> your server is using a fairly old version so upgrading might be a good >> idea regardless. >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> >> 2012/12/15, Kaiduan Xie : >>> Hi, >>> >>> I ran into a situation where server sends back SSL decrypt error to >>> client during SSL handshake, both client and server are written in >>> Erlang. The SSL handshake looks as below, >>> >>> 1) Client sends Client Hello to server >>> 2) Server sends Server Hello back >>> 3) Server sends Certificate, Server Key Exchange and Server Hello Done >>> 4) Client sends Client Key Exchange >>> 5) Client sends Change Cipher Spec, Encrypted Handshake Message >>> 6) Server sends Alert (Level: Fatal, Description: Decrypt Error) >>> >>> The certificate is from godaddy, any idea why server sends Decrypt Error? >>> >>> The server is running R14B01 while the client is running R15B02. >>> >>> Thanks, >>> >>> /Kaiduan >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: ssl-alert-error.pcap Type: application/octet-stream Size: 6595 bytes Desc: not available URL: From mvm_8@REDACTED Thu Dec 20 19:59:02 2012 From: mvm_8@REDACTED (Venu Middela) Date: Thu, 20 Dec 2012 12:59:02 -0600 Subject: [erlang-questions] one run-queue per scheduler?? Message-ID: Hi, Does the Version OTP-R15B* or any other versions implement one run-queue per scheduler? Please refer to section 5.2 of this Technical Paper. http://www.erlang.se/euc/08/euc_smp.pdf I'm interested to know if this has been implemented in any of the versions? Thanks, Venu -------------- next part -------------- An HTML attachment was scrubbed... URL: From k.petrauskas@REDACTED Fri Dec 21 02:24:29 2012 From: k.petrauskas@REDACTED (Karolis Petrauskas) Date: Fri, 21 Dec 2012 03:24:29 +0200 Subject: [erlang-questions] Accessing sibling processes in a supervisor. Message-ID: Hi, I have a question regarding an example [1] in LYSE. The example proposes a supervision scheme for a server-worker like processes. I have used this scheme a lot (I learnt Erlang from this book mainly), but now I'm in doubt. Is the proposed way of accessing a sibling process (access worker_sup from ppool_serv, see [1]) is the correct one? How will the ppool_serv get a PID of the worker_sup after a crash and restart? As I understand, if one of the processes will crash, both processes will be restarted and the server should get an error while starting the worker_sup again (the corresponding child already exists): handle_info({start_worker_supervisor, Sup, MFA}, S = #state{}) -> {ok, Pid} = supervisor:start_child(Sup, ?SPEC(MFA)), % Will this work after restart? Or maybe I missed the point? I am aware of some other ways of getting processes to know each other [2], but I would like to get your comments on this example. Other schemes for implementing communication of anonymous sibling processes (in the supervision tree) would be interesting also. [1] http://learnyousomeerlang.com/building-applications-with-otp#implementing-the-supervisors [2] http://erlang.2086793.n4.nabble.com/supervisor-children-s-pid-td3530959.html#a3531973 Best regards, Karolis Petrauskas From norton@REDACTED Fri Dec 21 02:49:49 2012 From: norton@REDACTED (=?utf-8?B?44OO44O844OI44OzIOOCuOODp+ODvOOCu+ODlSDjgqbjgqfjgqQg?= =?utf-8?B?44Oz?=) Date: Fri, 21 Dec 2012 10:49:49 +0900 Subject: [erlang-questions] Accessing sibling processes in a supervisor. In-Reply-To: References: Message-ID: Karolis - I'd recommend to study gproc (https://github.com/uwiger/gproc) and in particular it's process registry features. Worker processes and/or other processes can query the set of active processes using gproc:first/1, gproc:next/2, gproc:select/{1,2}, etc. For example, a worker process could register themselves with a key such as {?MODULE, self()}. If worker processes have a unique identifier, a worker process could register themselves with a key such as {?MODULE, Id::term()} where term() is an unique identifier. Hope this helps. Joe N. On Dec 21, 2012, at 10:24 , Karolis Petrauskas wrote: > Hi, > > I have a question regarding an example [1] in LYSE. The example > proposes a supervision scheme for a server-worker like processes. I > have used this scheme a lot (I learnt Erlang from this book mainly), > but now I'm in doubt. Is the proposed way of accessing a sibling > process (access worker_sup from ppool_serv, see [1]) is the correct > one? How will the ppool_serv get a PID of the worker_sup after a crash > and restart? As I understand, if one of the processes will crash, both > processes will be restarted and the server should get an error while > starting the worker_sup again (the corresponding child already > exists): > > handle_info({start_worker_supervisor, Sup, MFA}, S = #state{}) -> > {ok, Pid} = supervisor:start_child(Sup, ?SPEC(MFA)), % > Will this work after restart? > > Or maybe I missed the point? I am aware of some other ways of getting > processes to know each other [2], but I would like to get your > comments on this example. Other schemes for implementing communication > of anonymous sibling processes (in the supervision tree) would be > interesting also. > > [1] http://learnyousomeerlang.com/building-applications-with-otp#implementing-the-supervisors > [2] http://erlang.2086793.n4.nabble.com/supervisor-children-s-pid-td3530959.html#a3531973 > > Best regards, > Karolis Petrauskas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ababkin@REDACTED Fri Dec 21 03:03:10 2012 From: ababkin@REDACTED (Alex Babkin) Date: Thu, 20 Dec 2012 21:03:10 -0500 Subject: [erlang-questions] suggestions for implementing sftp server? Message-ID: Hi all (long time lurker, first time poster here) I am trying to implement a system that acts like an 'sftp proxy', basically i'd like it to provide an sftp server front so clients could upload files, then the system would crunch/check those files and, upon completion, sftp them onto a remote (3rd party non-erlang independent) sftp server (guess i'll use erlang's ssh ftp client for this part). >From the initial research, there is very little blog/example material on this and it's not very clear how to implement a sftp server from the official erlang docs. So my questions are - what modules and in what way should i use them to come up with a sftp server? - once a file is uploaded onto it, what hooks am i to use to grab the file and process it / transmit it further Thanks in advance Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Fri Dec 21 03:19:43 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 20 Dec 2012 21:19:43 -0500 Subject: [erlang-questions] Accessing sibling processes in a supervisor. In-Reply-To: References: Message-ID: <20121221021941.GC48322@ferdmbp.local> Hi, Author of LYSE here. The strategy I mention will work because the direct supervisor of both processes uses a 'one_for_all' restart strategy, meaning that if the gen_server crashes, or the supervisor it relies on crashes, both are killed and then restarted. The names are unregistered automatically upon their death, and things should back up error free. This is a decision made because the gen_server strongly relies on the supervisor to handle its children, and if it crashes, there's no easy way for a new server to pick up from where the other left regarding messages, references, tasks to do, etc. If the supervisor dies, then it means the children crashed a lot and you had some kind of problem there anyway. In both cases, it is *a lot* simpler (at least, I think it is) to crash and restart everything from a fresh state rather than have a new server register under a new name and try to figure out what the hell was going on before it came into existence. I think a lot of people want to limit what crashes in their systems in a way to eliminate errors as much as possible, but in this case I believe crashing more stuff makes the case much simpler, and more likely to avoid weird heisenbugs that take weeks to fix down the line when you wonder why you seem to be missing data or have rogue workers hanging around when they shouldn't be. There's a very direct dependency between the two processes, and they don't necessarily make sense without the other being there. They spawned together, and they should die together. Given this design decision, it becomes somewhat useless to register the names for the sake of it, and just passing the pid directly is entirely fine. Regards, Fred. On 12/21, Karolis Petrauskas wrote: > Hi, > > I have a question regarding an example [1] in LYSE. The example > proposes a supervision scheme for a server-worker like processes. I > have used this scheme a lot (I learnt Erlang from this book mainly), > but now I'm in doubt. Is the proposed way of accessing a sibling > process (access worker_sup from ppool_serv, see [1]) is the correct > one? How will the ppool_serv get a PID of the worker_sup after a crash > and restart? As I understand, if one of the processes will crash, both > processes will be restarted and the server should get an error while > starting the worker_sup again (the corresponding child already > exists): > > handle_info({start_worker_supervisor, Sup, MFA}, S = #state{}) -> > {ok, Pid} = supervisor:start_child(Sup, ?SPEC(MFA)), % > Will this work after restart? > > Or maybe I missed the point? I am aware of some other ways of getting > processes to know each other [2], but I would like to get your > comments on this example. Other schemes for implementing communication > of anonymous sibling processes (in the supervision tree) would be > interesting also. > > [1] http://learnyousomeerlang.com/building-applications-with-otp#implementing-the-supervisors > [2] http://erlang.2086793.n4.nabble.com/supervisor-children-s-pid-td3530959.html#a3531973 > > Best regards, > Karolis Petrauskas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From humeafo@REDACTED Fri Dec 21 04:22:33 2012 From: humeafo@REDACTED (krad) Date: Thu, 20 Dec 2012 19:22:33 -0800 (PST) Subject: [erlang-questions] inspect process mailbox size In-Reply-To: References: <49849562.4050202@moonpolysoft.com> Message-ID: <446f9f38-4ee6-4d09-bd82-f77fdf6bfd5c@googlegroups.com> a further question is, is there anyway to restrict the mailbox size? On Sunday, February 1, 2009 3:11:14 AM UTC+8, Geoff Cant wrote: > > Cliff Moon > writes: > > > is there any way to inspect mailbox size at runtime? Under load testing > > I'm seeing conditions pop up where a node will go out of control due to > > mailbox growth and it's rather hard to debug the issue post mortem from > > crash dumps. > > erlang:process_info/2 is very helpful for this: > erlang:process_info(Pid, messages_queue_len) gives you the length of the > queue and erlang:process_info(Pid, messages) gives you the queue itself. > > A common cause of mailbox blowout is that the process is blocked waiting > for a reply from something - in this case erlang:process_info(Pid, > backtrace) and erlang:process_info(Pid, current_function) can help you > figure out what it might be waiting for. > > Cheers, > -- > Geoff Cant > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwertymaniac@REDACTED Fri Dec 21 05:10:48 2012 From: qwertymaniac@REDACTED (Harsh J) Date: Fri, 21 Dec 2012 09:40:48 +0530 Subject: [erlang-questions] one run-queue per scheduler?? In-Reply-To: References: Message-ID: Yes the R15B with SMP support now uses multiple, scheduler specific run queues, instead of one globally shared run queue. This is highlighted at http://www.erlang.org/documentation/doc-5.7.4/doc/highlights.html On Fri, Dec 21, 2012 at 12:29 AM, Venu Middela wrote: > Hi, > > Does the Version OTP-R15B* or any other versions implement one run-queue > per scheduler? > > Please refer to section 5.2 of this Technical Paper. > > http://www.erlang.se/euc/08/euc_smp.pdf > > I'm interested to know if this has been implemented in any of the versions? > > > Thanks, > Venu > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Harsh J http://harshj.com From qwertymaniac@REDACTED Fri Dec 21 05:16:45 2012 From: qwertymaniac@REDACTED (Harsh J) Date: Fri, 21 Dec 2012 09:46:45 +0530 Subject: [erlang-questions] inspect process mailbox size In-Reply-To: <446f9f38-4ee6-4d09-bd82-f77fdf6bfd5c@googlegroups.com> References: <49849562.4050202@moonpolysoft.com> <446f9f38-4ee6-4d09-bd82-f77fdf6bfd5c@googlegroups.com> Message-ID: I'm not aware of any way to do that (but please let me know if there is one). However, this past thread may be helpful to you: http://erlang.org/pipermail/erlang-questions/2011-June/059355.html On Fri, Dec 21, 2012 at 8:52 AM, krad wrote: > a further question is, is there anyway to restrict the mailbox size? > > On Sunday, February 1, 2009 3:11:14 AM UTC+8, Geoff Cant wrote: >> >> Cliff Moon writes: >> >> > is there any way to inspect mailbox size at runtime? Under load testing >> > I'm seeing conditions pop up where a node will go out of control due to >> > mailbox growth and it's rather hard to debug the issue post mortem from >> > crash dumps. >> >> erlang:process_info/2 is very helpful for this: >> erlang:process_info(Pid, messages_queue_len) gives you the length of the >> queue and erlang:process_info(Pid, messages) gives you the queue itself. >> >> A common cause of mailbox blowout is that the process is blocked waiting >> for a reply from something - in this case erlang:process_info(Pid, >> backtrace) and erlang:process_info(Pid, current_function) can help you >> figure out what it might be waiting for. >> >> Cheers, >> -- >> Geoff Cant >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Harsh J http://harshj.com From a.zhuravlev@REDACTED Fri Dec 21 06:28:19 2012 From: a.zhuravlev@REDACTED (Alexander Zhuravlev) Date: Fri, 21 Dec 2012 09:28:19 +0400 Subject: [erlang-questions] suggestions for implementing sftp server? In-Reply-To: References: Message-ID: <20121221052819.GA54096@zmac> On Thu, Dec 20, 2012 at 09:03:10PM -0500, Alex Babkin wrote: > Hi all (long time lurker, first time poster here) > > I am trying to implement a system that acts like an 'sftp proxy', basically > i'd like it to provide an sftp server front so clients could upload files, > then the system would crunch/check those files and, upon completion, sftp > them onto a remote (3rd party non-erlang independent) sftp server (guess > i'll use erlang's ssh ftp client for this part). > > From the initial research, there is very little blog/example material on > this and it's not very clear how to implement a sftp server from the > official erlang docs. > > So my questions are > - what modules and in what way should i use them to come up with a sftp > server? I would recommend to take a look into https://github.com/erlang/otp/blob/maint/lib/ssh/test/ssh_sftpd_SUITE.erl first. > - once a file is uploaded onto it, what hooks am i to use to grab the file > and process it / transmit it further ssh_sftpd provides possibility to pass a custom file handler module. You can enhance the default one with your custom processing logic. -- Alexander Zhuravlev From k.petrauskas@REDACTED Fri Dec 21 11:00:00 2012 From: k.petrauskas@REDACTED (Karolis Petrauskas) Date: Fri, 21 Dec 2012 12:00:00 +0200 Subject: [erlang-questions] Accessing sibling processes in a supervisor. In-Reply-To: <20121221021941.GC48322@ferdmbp.local> References: <20121221021941.GC48322@ferdmbp.local> Message-ID: Thank you, Fred, for a great book! I changed your example (ppool-1.0/src/ppool_serv.erl) a bit to illustrate my concern. I just added a function die/1, that does nothing apart from stopping gen_server with reason != normal. Bellow is a diff showing my changes: learn-you-some-erlang$ git diff diff --git a/ppool-1.0/src/ppool_serv.erl b/ppool-1.0/src/ppool_serv.erl index bf901dc..ff11dfb 100644 --- a/ppool-1.0/src/ppool_serv.erl +++ b/ppool-1.0/src/ppool_serv.erl @@ -1,6 +1,6 @@ -module(ppool_serv). -behaviour(gen_server). --export([start/4, start_link/4, run/2, sync_queue/2, async_queue/2, stop/1]). +-export([start/4, start_link/4, run/2, sync_queue/2, async_queue/2, stop/1, die/1]). -export([init/1, handle_call/3, handle_cast/2, handle_info/2, code_change/3, terminate/2]). @@ -36,6 +36,9 @@ async_queue(Name, Args) -> stop(Name) -> gen_server:call(Name, stop). +die(Name) -> + gen_server:call(Name, die). + %% Gen server init({Limit, MFA, Sup}) -> %% We need to find the Pid of the worker supervisor from here, @@ -59,6 +62,8 @@ handle_call({sync, Args}, From, S = #state{queue=Q}) -> handle_call(stop, _From, State) -> {stop, normal, ok, State}; +handle_call(die, _From, State) -> + {stop, error, dying, State}; handle_call(_Msg, _From, State) -> {noreply, State}. I have compiled it and then called the following functions: application:start(ppool). ppool:start_pool(nagger, 2, {ppool_nagger, start_link, []}). ppool_serv:die(nagger). and got the following errors: =ERROR REPORT==== 21-Dec-2012::11:39:44 === ** Generic server nagger terminating ** Last message in was die ** When Server state == {state,2,<0.43.0>,{0,nil},{[],[]}} ** Reason for termination == ** error =ERROR REPORT==== 21-Dec-2012::11:39:44 === ** Generic server nagger terminating ** Last message in was {start_worker_supervisor,<0.41.0>, {ppool_nagger,start_link,[]}} ** When Server state == {state,2,undefined,{0,nil},{[],[]}} ** Reason for termination == ** {{badmatch,{error,{already_started,<0.46.0>}}}, [{ppool_serv,handle_info,2,[{file,"src/ppool_serv.erl"},{line,90}]}, {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,607}]}, {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} The second error is the one I was talking about. On the other hand, the entire application has not crashed, but the ppool_sup was terminated due to reached_max_restart_intensity and then restarted. Was that the intended behaviour? Best regards, Karolis Petrauskas On Fri, Dec 21, 2012 at 4:19 AM, Fred Hebert wrote: > Hi, Author of LYSE here. > > The strategy I mention will work because the direct supervisor of both > processes uses a 'one_for_all' restart strategy, meaning that if the > gen_server crashes, or the supervisor it relies on crashes, both are > killed and then restarted. The names are unregistered automatically upon > their death, and things should back up error free. > > This is a decision made because the gen_server strongly relies on the > supervisor to handle its children, and if it crashes, there's no easy > way for a new server to pick up from where the other left regarding > messages, references, tasks to do, etc. If the supervisor dies, then it > means the children crashed a lot and you had some kind of problem there > anyway. > > In both cases, it is *a lot* simpler (at least, I think it is) to crash > and restart everything from a fresh state rather than have a new server > register under a new name and try to figure out what the hell was going > on before it came into existence. > > I think a lot of people want to limit what crashes in their systems in a > way to eliminate errors as much as possible, but in this case I believe > crashing more stuff makes the case much simpler, and more likely to > avoid weird heisenbugs that take weeks to fix down the line when you > wonder why you seem to be missing data or have rogue workers hanging > around when they shouldn't be. There's a very direct dependency between > the two processes, and they don't necessarily make sense without the > other being there. They spawned together, and they should die together. > > Given this design decision, it becomes somewhat useless to register the > names for the sake of it, and just passing the pid directly is entirely > fine. > > Regards, > Fred. > > On 12/21, Karolis Petrauskas wrote: >> Hi, >> >> I have a question regarding an example [1] in LYSE. The example >> proposes a supervision scheme for a server-worker like processes. I >> have used this scheme a lot (I learnt Erlang from this book mainly), >> but now I'm in doubt. Is the proposed way of accessing a sibling >> process (access worker_sup from ppool_serv, see [1]) is the correct >> one? How will the ppool_serv get a PID of the worker_sup after a crash >> and restart? As I understand, if one of the processes will crash, both >> processes will be restarted and the server should get an error while >> starting the worker_sup again (the corresponding child already >> exists): >> >> handle_info({start_worker_supervisor, Sup, MFA}, S = #state{}) -> >> {ok, Pid} = supervisor:start_child(Sup, ?SPEC(MFA)), % >> Will this work after restart? >> >> Or maybe I missed the point? I am aware of some other ways of getting >> processes to know each other [2], but I would like to get your >> comments on this example. Other schemes for implementing communication >> of anonymous sibling processes (in the supervision tree) would be >> interesting also. >> >> [1] http://learnyousomeerlang.com/building-applications-with-otp#implementing-the-supervisors >> [2] http://erlang.2086793.n4.nabble.com/supervisor-children-s-pid-td3530959.html#a3531973 >> >> Best regards, >> Karolis Petrauskas >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From robert.virding@REDACTED Fri Dec 21 11:15:24 2012 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 21 Dec 2012 10:15:24 +0000 (GMT) Subject: [erlang-questions] one run-queue per scheduler?? In-Reply-To: Message-ID: <2087089229.2565493.1356084924014.JavaMail.root@erlang-solutions.com> Just to be more explicit: all versions from R13 and onwards have scheduler specific run-queues and no common run-queue. This was an important factor in making the BEAM scale better over many cores/schedulers. There is some global work done in trying to keep a reasonable balance of work between the schedulers. Robert ----- Original Message ----- > From: "Harsh J" > To: "Venu Middela" > Cc: erlang-questions@REDACTED > Sent: Friday, 21 December, 2012 5:10:48 AM > Subject: Re: [erlang-questions] one run-queue per scheduler?? > > Yes the R15B with SMP support now uses multiple, scheduler specific > run queues, instead of one globally shared run queue. This is > highlighted at > http://www.erlang.org/documentation/doc-5.7.4/doc/highlights.html > > On Fri, Dec 21, 2012 at 12:29 AM, Venu Middela > wrote: > > Hi, > > > > Does the Version OTP-R15B* or any other versions implement one > > run-queue > > per scheduler? > > > > Please refer to section 5.2 of this Technical Paper. > > > > http://www.erlang.se/euc/08/euc_smp.pdf > > > > I'm interested to know if this has been implemented in any of the > > versions? > > > > > > Thanks, > > Venu > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > Harsh J > http://harshj.com > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From pan@REDACTED Fri Dec 21 11:59:40 2012 From: pan@REDACTED (Patrik Nyblom) Date: Fri, 21 Dec 2012 11:59:40 +0100 Subject: [erlang-questions] clarify: run_erl and ssh In-Reply-To: <201212201456.qBKEu5Qf020673@pluto.hedeland.org> References: <201212201456.qBKEu5Qf020673@pluto.hedeland.org> Message-ID: <50D4411C.7000309@erlang.org> Hi Per! On 12/20/2012 03:56 PM, Per Hedeland wrote: > Patrik Nyblom wrote: >> On 12/19/2012 08:37 AM, Bengt Kleberg wrote: > [snip] >>> sekic1152 [8:26am] [/home/eleberg] -> ssh sekic1152 ./afile > [snip] >> I guess the PTY does not get initialized correctly when you run via ssh. > Or rather, in the above command, there is no pty allocated *at all* on > 'sekic1152'. Try 'ssh -t sekic1152 ./afile' instead. 'man ssh'. Well, I was unclear or downright misleading. I did not mean the ssh pty, but the pty allocated by run_erl, which can be allocated and setup regardless of the stdin/stdout/stderr situation when starting run_erl. Or, regardless of everything *except* for the fact that it needs to get a valid curses definition, which in turn relies on the terminal setting (or TERM environment variable) of the shell that runs run_erl (which is expected to match the terminal doing to_erl)... So ssh does not really need to have allocated a pty, we just need to tell run_erl that it should act as if we are going to connect via a VT100 (or xterm or something else that has a curses definition). The actual pty of run_erl get's initialized correctly regardless, we just does not get newshell if we do not have a valid TERM variable so we can find a valid curses definition. That's why TERM=vt100 run_erl -daemon /tmp/eleberg/ /tmp/eleberg/ "exec erl" works, but the aforementioned script doesn't (or, really it works, you just get the "oldshell"). ssh -t also works, as you then get the TERM variable set properly - it is inherited from the remote host in the terminal setup negotiation. So, saying that the pty did not get properly initialized was wrong, I take that back. Cheers, /Patrik > > --Per Hedeland From bengt.kleberg@REDACTED Fri Dec 21 13:08:54 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 21 Dec 2012 13:08:54 +0100 Subject: [erlang-questions] clarify: run_erl and ssh In-Reply-To: <50D4411C.7000309@erlang.org> References: <201212201456.qBKEu5Qf020673@pluto.hedeland.org> <50D4411C.7000309@erlang.org> Message-ID: <1356091734.4840.60.camel@sekic1152.release> FWIW, I did not get "-t" to work. The problem persisted. Probably a mistake on my part. Adding TERM=vt100 in front of my real start command ("afile" is just an example) was worse. Erlang did not start at all. Again, probably a mistake on my part. I will add a comment in the code about both possibilities so that it can be looked into when time permits. bengt On Fri, 2012-12-21 at 11:59 +0100, Patrik Nyblom wrote: > Hi Per! > > On 12/20/2012 03:56 PM, Per Hedeland wrote: > > Patrik Nyblom wrote: > >> On 12/19/2012 08:37 AM, Bengt Kleberg wrote: > > [snip] > >>> sekic1152 [8:26am] [/home/eleberg] -> ssh sekic1152 ./afile > > [snip] > >> I guess the PTY does not get initialized correctly when you run via ssh. > > Or rather, in the above command, there is no pty allocated *at all* on > > 'sekic1152'. Try 'ssh -t sekic1152 ./afile' instead. 'man ssh'. > Well, I was unclear or downright misleading. I did not mean the ssh > pty, but the pty allocated by run_erl, which can be allocated and setup > regardless of the stdin/stdout/stderr situation when starting run_erl. > Or, regardless of everything *except* for the fact that it needs to get > a valid curses definition, which in turn relies on the terminal setting > (or TERM environment variable) of the shell that runs run_erl (which is > expected to match the terminal doing to_erl)... > > So ssh does not really need to have allocated a pty, we just need to > tell run_erl that it should act as if we are going to connect via a > VT100 (or xterm or something else that has a curses definition). The > actual pty of run_erl get's initialized correctly regardless, we just > does not get newshell if we do not have a valid TERM variable so we can > find a valid curses definition. > > That's why > > TERM=vt100 run_erl -daemon /tmp/eleberg/ /tmp/eleberg/ "exec erl" > > works, but the aforementioned script doesn't (or, really it works, you > just get the "oldshell"). > ssh -t also works, as you then get the TERM variable set properly - it > is inherited from the remote host in the terminal setup negotiation. > > So, saying that the pty did not get properly initialized was wrong, I > take that back. > > Cheers, > /Patrik > > > > --Per Hedeland > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mononcqc@REDACTED Fri Dec 21 15:04:46 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 21 Dec 2012 09:04:46 -0500 Subject: [erlang-questions] Accessing sibling processes in a supervisor. In-Reply-To: References: Message-ID: <20121221140445.GA48662@ferdmbp.local> Ah, I understand the error. It's a very simple mistake there. The supervision structure is a bit as follows: supersup | pool_sup | \ serv worker_sup | workers Here, 'serv' is started as a permanent process, and it starts 'worker_sup' as another permanent process. Whenever one of them dies, both get killed, and the supervisor then restarts *both* of them. Then, the next time around, when the gen_server tries to start the child, it can't access it because it was already started. A very minor fix can handle the issue. Instead of just doing {ok, Pid}?= supervisor:start_child(...), the following can be done: {ok, Pid}?= case supervisor:start_child(...) of {ok, NewPid} -> {ok, NewPid}; {error, {already_started, OldPidPid}} -> {ok, OldPid} end ... Note that the error that currently exists is not that much of a big deal since as soon as the restart intensity is reached, things get cleaned up and the next restart is done on a blank slate. An attempt could have been done at making the worker supervisor a temporary process. At this point, it would not be restarted ever and we'd get the behavior we want, but we'd need to link it to the server so that they both die together and always provoke the server to die. Such a fix would instead be to change the worker_sup strategy to 'temporary' and use something as follows: {ok, Pid} = case supervisor:start_child(...) of link(Pid), .... I'll take a better look at it later today, I've got to run for now. Regards, Fred. On 12/21, Karolis Petrauskas wrote: > Thank you, Fred, for a great book! > > I changed your example (ppool-1.0/src/ppool_serv.erl) a bit to > illustrate my concern. I just added a function die/1, that does > nothing apart from stopping gen_server with reason != normal. Bellow > is a diff showing my changes: > > learn-you-some-erlang$ git diff > diff --git a/ppool-1.0/src/ppool_serv.erl b/ppool-1.0/src/ppool_serv.erl > index bf901dc..ff11dfb 100644 > --- a/ppool-1.0/src/ppool_serv.erl > +++ b/ppool-1.0/src/ppool_serv.erl > @@ -1,6 +1,6 @@ > -module(ppool_serv). > -behaviour(gen_server). > --export([start/4, start_link/4, run/2, sync_queue/2, async_queue/2, stop/1]). > +-export([start/4, start_link/4, run/2, sync_queue/2, async_queue/2, > stop/1, die/1]). > -export([init/1, handle_call/3, handle_cast/2, handle_info/2, > code_change/3, terminate/2]). > > @@ -36,6 +36,9 @@ async_queue(Name, Args) -> > stop(Name) -> > gen_server:call(Name, stop). > > +die(Name) -> > + gen_server:call(Name, die). > + > %% Gen server > init({Limit, MFA, Sup}) -> > %% We need to find the Pid of the worker supervisor from here, > @@ -59,6 +62,8 @@ handle_call({sync, Args}, From, S = #state{queue=Q}) -> > > handle_call(stop, _From, State) -> > {stop, normal, ok, State}; > +handle_call(die, _From, State) -> > + {stop, error, dying, State}; > handle_call(_Msg, _From, State) -> > {noreply, State}. > > I have compiled it and then called the following functions: > > application:start(ppool). > ppool:start_pool(nagger, 2, {ppool_nagger, start_link, []}). > ppool_serv:die(nagger). > > > and got the following errors: > > =ERROR REPORT==== 21-Dec-2012::11:39:44 === > ** Generic server nagger terminating > ** Last message in was die > ** When Server state == {state,2,<0.43.0>,{0,nil},{[],[]}} > ** Reason for termination == > ** error > > =ERROR REPORT==== 21-Dec-2012::11:39:44 === > ** Generic server nagger terminating > ** Last message in was {start_worker_supervisor,<0.41.0>, > {ppool_nagger,start_link,[]}} > ** When Server state == {state,2,undefined,{0,nil},{[],[]}} > ** Reason for termination == > ** {{badmatch,{error,{already_started,<0.46.0>}}}, > [{ppool_serv,handle_info,2,[{file,"src/ppool_serv.erl"},{line,90}]}, > {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,607}]}, > {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} > > The second error is the one I was talking about. On the other hand, > the entire application has not crashed, but the ppool_sup was > terminated due to reached_max_restart_intensity and then restarted. > Was that the intended behaviour? > > Best regards, > Karolis Petrauskas > > On Fri, Dec 21, 2012 at 4:19 AM, Fred Hebert wrote: > > Hi, Author of LYSE here. > > > > The strategy I mention will work because the direct supervisor of both > > processes uses a 'one_for_all' restart strategy, meaning that if the > > gen_server crashes, or the supervisor it relies on crashes, both are > > killed and then restarted. The names are unregistered automatically upon > > their death, and things should back up error free. > > > > This is a decision made because the gen_server strongly relies on the > > supervisor to handle its children, and if it crashes, there's no easy > > way for a new server to pick up from where the other left regarding > > messages, references, tasks to do, etc. If the supervisor dies, then it > > means the children crashed a lot and you had some kind of problem there > > anyway. > > > > In both cases, it is *a lot* simpler (at least, I think it is) to crash > > and restart everything from a fresh state rather than have a new server > > register under a new name and try to figure out what the hell was going > > on before it came into existence. > > > > I think a lot of people want to limit what crashes in their systems in a > > way to eliminate errors as much as possible, but in this case I believe > > crashing more stuff makes the case much simpler, and more likely to > > avoid weird heisenbugs that take weeks to fix down the line when you > > wonder why you seem to be missing data or have rogue workers hanging > > around when they shouldn't be. There's a very direct dependency between > > the two processes, and they don't necessarily make sense without the > > other being there. They spawned together, and they should die together. > > > > Given this design decision, it becomes somewhat useless to register the > > names for the sake of it, and just passing the pid directly is entirely > > fine. > > > > Regards, > > Fred. > > > > On 12/21, Karolis Petrauskas wrote: > >> Hi, > >> > >> I have a question regarding an example [1] in LYSE. The example > >> proposes a supervision scheme for a server-worker like processes. I > >> have used this scheme a lot (I learnt Erlang from this book mainly), > >> but now I'm in doubt. Is the proposed way of accessing a sibling > >> process (access worker_sup from ppool_serv, see [1]) is the correct > >> one? How will the ppool_serv get a PID of the worker_sup after a crash > >> and restart? As I understand, if one of the processes will crash, both > >> processes will be restarted and the server should get an error while > >> starting the worker_sup again (the corresponding child already > >> exists): > >> > >> handle_info({start_worker_supervisor, Sup, MFA}, S = #state{}) -> > >> {ok, Pid} = supervisor:start_child(Sup, ?SPEC(MFA)), % > >> Will this work after restart? > >> > >> Or maybe I missed the point? I am aware of some other ways of getting > >> processes to know each other [2], but I would like to get your > >> comments on this example. Other schemes for implementing communication > >> of anonymous sibling processes (in the supervision tree) would be > >> interesting also. > >> > >> [1] http://learnyousomeerlang.com/building-applications-with-otp#implementing-the-supervisors > >> [2] http://erlang.2086793.n4.nabble.com/supervisor-children-s-pid-td3530959.html#a3531973 > >> > >> Best regards, > >> Karolis Petrauskas > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions From per@REDACTED Fri Dec 21 17:24:52 2012 From: per@REDACTED (Per Hedeland) Date: Fri, 21 Dec 2012 17:24:52 +0100 (CET) Subject: [erlang-questions] clarify: run_erl and ssh In-Reply-To: <50D4411C.7000309@erlang.org> Message-ID: <201212211624.qBLGOqKn048709@pluto.hedeland.org> Patrik Nyblom wrote: > >On 12/20/2012 03:56 PM, Per Hedeland wrote: >> Patrik Nyblom wrote: >>> On 12/19/2012 08:37 AM, Bengt Kleberg wrote: >> [snip] >>>> sekic1152 [8:26am] [/home/eleberg] -> ssh sekic1152 ./afile >> [snip] >>> I guess the PTY does not get initialized correctly when you run via ssh. >> Or rather, in the above command, there is no pty allocated *at all* on >> 'sekic1152'. Try 'ssh -t sekic1152 ./afile' instead. 'man ssh'. >Well, I was unclear or downright misleading. No, I think it was just me forgetting about run_erl's pty - 'ssh -t' should definitely not be needed then. --Per From ababkin@REDACTED Fri Dec 21 18:03:14 2012 From: ababkin@REDACTED (Alex Babkin) Date: Fri, 21 Dec 2012 12:03:14 -0500 Subject: [erlang-questions] suggestions for implementing sftp server? In-Reply-To: <20121221052819.GA54096@zmac> References: <20121221052819.GA54096@zmac> Message-ID: Thanks Alexander, this was very helpful On Fri, Dec 21, 2012 at 12:28 AM, Alexander Zhuravlev wrote: > ssh_sftpd -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattevans123@REDACTED Fri Dec 21 18:45:24 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Fri, 21 Dec 2012 12:45:24 -0500 Subject: [erlang-questions] Exprecs question Message-ID: Hi, Is there a simple way to use exprecs to create a match specification I can use in ets:match_delete/2? I can do something like: ets:match_delete(my_table, erlang:make_tuple(custom_record_handler:info_p2_data(size), '_', [{custom_record_handler:pos_p2_data(special_type),123}])). I was wondering if there was a "cleaner" or better way of doing this? Thanks Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From mvm_8@REDACTED Fri Dec 21 18:51:17 2012 From: mvm_8@REDACTED (Venu Middela) Date: Fri, 21 Dec 2012 11:51:17 -0600 Subject: [erlang-questions] one run-queue per scheduler?? In-Reply-To: <2087089229.2565493.1356084924014.JavaMail.root@erlang-solutions.com> References: , <2087089229.2565493.1356084924014.JavaMail.root@erlang-solutions.com> Message-ID: Thanks Robert and Harsha. Venu > Date: Fri, 21 Dec 2012 10:15:24 +0000 > From: robert.virding@REDACTED > To: qwertymaniac@REDACTED > CC: erlang-questions@REDACTED; mvm_8@REDACTED > Subject: Re: [erlang-questions] one run-queue per scheduler?? > > Just to be more explicit: all versions from R13 and onwards have scheduler specific run-queues and no common run-queue. This was an important factor in making the BEAM scale better over many cores/schedulers. There is some global work done in trying to keep a reasonable balance of work between the schedulers. > > Robert > > ----- Original Message ----- > > From: "Harsh J" > > To: "Venu Middela" > > Cc: erlang-questions@REDACTED > > Sent: Friday, 21 December, 2012 5:10:48 AM > > Subject: Re: [erlang-questions] one run-queue per scheduler?? > > > > Yes the R15B with SMP support now uses multiple, scheduler specific > > run queues, instead of one globally shared run queue. This is > > highlighted at > > http://www.erlang.org/documentation/doc-5.7.4/doc/highlights.html > > > > On Fri, Dec 21, 2012 at 12:29 AM, Venu Middela > > wrote: > > > Hi, > > > > > > Does the Version OTP-R15B* or any other versions implement one > > > run-queue > > > per scheduler? > > > > > > Please refer to section 5.2 of this Technical Paper. > > > > > > http://www.erlang.se/euc/08/euc_smp.pdf > > > > > > I'm interested to know if this has been implemented in any of the > > > versions? > > > > > > > > > Thanks, > > > Venu > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > > -- > > Harsh J > > http://harshj.com > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri Dec 21 19:02:25 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 21 Dec 2012 19:02:25 +0100 Subject: [erlang-questions] Exprecs question In-Reply-To: References: Message-ID: For some definition of 'cleaner' and 'better', perhaps? Given e.g. https://github.com/esl/parse_trans/blob/master/examples/test_exprecs.erl 1> Wild = fun(Mod, R, Vars) -> Mod:'#set-'(Vars, Mod:'#set-'([{F,'_'} || F <- Mod:'#info-'(R, fields)], Mod:'#new-'(R))) end. #Fun 2> Wild(test_exprecs, r, [{b,foo}]). {r,'_',foo,'_'} BR, Ulf W ;-) On 21 Dec 2012, at 18:45, Matthew Evans wrote: > Hi, > > Is there a simple way to use exprecs to create a match specification I can use in ets:match_delete/2? > > I can do something like: > > ets:match_delete(my_table, erlang:make_tuple(custom_record_handler:info_p2_data(size), '_', [{custom_record_handler:pos_p2_data(special_type),123}])). > > I was wondering if there was a "cleaner" or better way of doing this? > > Thanks > > Matt > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco@REDACTED Fri Dec 21 20:52:51 2012 From: francesco@REDACTED (Francesco Cesarini) Date: Fri, 21 Dec 2012 19:52:51 +0000 Subject: [erlang-questions] Evening School of Erlang in San Francisco, San Jose & the rest of the US Message-ID: <50D4BE13.3040703@erlang-solutions.com> Recruiting or looking for a job with Erlang? I spent a good part of this year visiting US based companies interesting in adopting Erlang this year. One of the biggest hurdle to adoption is lack of programmers. To jump start the Erlang job ecosystem in the US, we decided to try out the evening school of Erlang. Two nights a week, for four weeks, local trainers will giving Erlang classes to developers looking for a career change. A few weeks after the course, those attending will be offered to take the certification exam. These classes will be heavily subsidized by corporate sponsors looking at recruiting/placing Erlang developers and will cost 199 USD (including the material and the certification). In exchange, your contact details will be shared with these sponsors. For those not interested in changing jobs but interested in learning Erlang, there is a corporate rate available which reflects the actual cost of the class. In this case, the details of the attendee will not be shared. We will place the evening school in locations easy to access. We currently have classes scheduled in San Francisco (Starting in early January), San Jose, Mountain View, Pasadena and Santa Monica. The goal for 2013 is to train 400 developers while expanding the network to cover Chicago, Seattle, Boston, New York, DC, and any other city where companies recruiting are willing to sponsor. For more information, visit our evening school of Erlang page: https://www.erlang-solutions.com/conferences/courses/evening-school-erlang In the new year, we are starting San Francisco Jan 8th, close to Union Square, followed by San Jose, Mountain View, Santa Monica and Pasadena. For this and other classes, we are looking for * Attendees * Help in spreading the word to friends, colleagues and communities you believe might be interested * Sponsors interested in recruiting Erlang developers willing to help out cover the costs of the trainer * Trainers and sponsors with facilities all across the US We had a similar story in Krakow, where to recruit, we started with the evening School The first time, 7 people registered. The second time, the number went up to 20, and recently, we had 70 applicants. Word got around. We are hoping the same will happen in the US! Many thanks for your help, and Merry Xmas! Francesco & the events team at ESL -- Erlang Solutions Ltd. http://www.erlang-solutions.com From jbothma@REDACTED Fri Dec 21 21:51:18 2012 From: jbothma@REDACTED (JD Bothma) Date: Fri, 21 Dec 2012 21:51:18 +0100 Subject: [erlang-questions] suggestions for implementing sftp server? In-Reply-To: References: <20121221052819.GA54096@zmac> Message-ID: You probably also already found these, but they're handy for reference https://github.com/erlang/otp/blob/maint/lib/ssh/src/ssh_sftpd_file.erl https://github.com/erlang/otp/blob/maint/lib/ssh/src/ssh_sftpd_file_api.erl On 21 December 2012 18:03, Alex Babkin wrote: > Thanks Alexander, this was very helpful > > > On Fri, Dec 21, 2012 at 12:28 AM, Alexander Zhuravlev < > a.zhuravlev@REDACTED> wrote: > >> ssh_sftpd > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Sat Dec 22 06:20:08 2012 From: bourinov@REDACTED (Max Bourinov) Date: Sat, 22 Dec 2012 09:20:08 +0400 Subject: [erlang-questions] one run-queue per scheduler?? In-Reply-To: References: <2087089229.2565493.1356084924014.JavaMail.root@erlang-solutions.com> Message-ID: Hi Erlangers, I would like to know more about Erlang VM and how it works. Could you please point me to papers that would be interesting to read please? Something like Venu mentioned would be great to read. Best regards, Max On Fri, Dec 21, 2012 at 9:51 PM, Venu Middela wrote: > Thanks Robert and Harsha. > > Venu > > > Date: Fri, 21 Dec 2012 10:15:24 +0000 > > From: robert.virding@REDACTED > > To: qwertymaniac@REDACTED > > CC: erlang-questions@REDACTED; mvm_8@REDACTED > > > Subject: Re: [erlang-questions] one run-queue per scheduler?? > > > > Just to be more explicit: all versions from R13 and onwards have > scheduler specific run-queues and no common run-queue. This was an > important factor in making the BEAM scale better over many > cores/schedulers. There is some global work done in trying to keep a > reasonable balance of work between the schedulers. > > > > Robert > > > > ----- Original Message ----- > > > From: "Harsh J" > > > To: "Venu Middela" > > > Cc: erlang-questions@REDACTED > > > Sent: Friday, 21 December, 2012 5:10:48 AM > > > Subject: Re: [erlang-questions] one run-queue per scheduler?? > > > > > > Yes the R15B with SMP support now uses multiple, scheduler specific > > > run queues, instead of one globally shared run queue. This is > > > highlighted at > > > http://www.erlang.org/documentation/doc-5.7.4/doc/highlights.html > > > > > > On Fri, Dec 21, 2012 at 12:29 AM, Venu Middela > > > wrote: > > > > Hi, > > > > > > > > Does the Version OTP-R15B* or any other versions implement one > > > > run-queue > > > > per scheduler? > > > > > > > > Please refer to section 5.2 of this Technical Paper. > > > > > > > > http://www.erlang.se/euc/08/euc_smp.pdf > > > > > > > > I'm interested to know if this has been implemented in any of the > > > > versions? > > > > > > > > > > > > Thanks, > > > > Venu > > > > > > > > _______________________________________________ > > > > erlang-questions mailing list > > > > erlang-questions@REDACTED > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > > > > > > > -- > > > Harsh J > > > http://harshj.com > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Sat Dec 22 17:45:14 2012 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Sat, 22 Dec 2012 17:45:14 +0100 Subject: [erlang-questions] one run-queue per scheduler?? In-Reply-To: References: <2087089229.2565493.1356084924014.JavaMail.root@erlang-solutions.com> Message-ID: Hi Max, Wouldn't we all want that info handy? =) Best bet for info about the runtime system and VM are the HiPE papers, See Kostis papers: http://user.it.uu.se/~kostis/Papers/ I couldn't find Erik Stenmans (Happi) papers http://user.it.uu.se/~happi/ Some of them had a really good overview if I remember correctly. Also Jianrong Zhangs paper "Characterizing the Scalability of Erlang VM on Many-Core Processors" has a good overview of some of the internals. (You have to google for the link to the pdf. KTH paper) Jianrong Zhangs paper is newer then the HiPE papers and probably has more current information. Some information are hidden in the *draft* of the erlang spec, see http://www.erlang.org/doc.html I recommend checking out Rickard Greens and Patrik Nyblom talks at various EUC, Factories and SICS Multicore Days to find the bleeding edge info =) Happy hunting, Bj?rn-Egil 2012/12/22 Max Bourinov > Hi Erlangers, > > I would like to know more about Erlang VM and how it works. Could you > please point me to papers that would be interesting to read please? > Something like Venu mentioned would be great to read. > > Best regards, > Max > > > > On Fri, Dec 21, 2012 at 9:51 PM, Venu Middela wrote: > >> Thanks Robert and Harsha. >> >> Venu >> >> > Date: Fri, 21 Dec 2012 10:15:24 +0000 >> > From: robert.virding@REDACTED >> > To: qwertymaniac@REDACTED >> > CC: erlang-questions@REDACTED; mvm_8@REDACTED >> >> > Subject: Re: [erlang-questions] one run-queue per scheduler?? >> > >> > Just to be more explicit: all versions from R13 and onwards have >> scheduler specific run-queues and no common run-queue. This was an >> important factor in making the BEAM scale better over many >> cores/schedulers. There is some global work done in trying to keep a >> reasonable balance of work between the schedulers. >> > >> > Robert >> > >> > ----- Original Message ----- >> > > From: "Harsh J" >> > > To: "Venu Middela" >> > > Cc: erlang-questions@REDACTED >> > > Sent: Friday, 21 December, 2012 5:10:48 AM >> > > Subject: Re: [erlang-questions] one run-queue per scheduler?? >> > > >> > > Yes the R15B with SMP support now uses multiple, scheduler specific >> > > run queues, instead of one globally shared run queue. This is >> > > highlighted at >> > > http://www.erlang.org/documentation/doc-5.7.4/doc/highlights.html >> > > >> > > On Fri, Dec 21, 2012 at 12:29 AM, Venu Middela >> > > wrote: >> > > > Hi, >> > > > >> > > > Does the Version OTP-R15B* or any other versions implement one >> > > > run-queue >> > > > per scheduler? >> > > > >> > > > Please refer to section 5.2 of this Technical Paper. >> > > > >> > > > http://www.erlang.se/euc/08/euc_smp.pdf >> > > > >> > > > I'm interested to know if this has been implemented in any of the >> > > > versions? >> > > > >> > > > >> > > > Thanks, >> > > > Venu >> > > > >> > > > _______________________________________________ >> > > > erlang-questions mailing list >> > > > erlang-questions@REDACTED >> > > > http://erlang.org/mailman/listinfo/erlang-questions >> > > > >> > > >> > > >> > > >> > > -- >> > > Harsh J >> > > http://harshj.com >> > > _______________________________________________ >> > > erlang-questions mailing list >> > > erlang-questions@REDACTED >> > > http://erlang.org/mailman/listinfo/erlang-questions >> > > >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Sun Dec 23 10:10:40 2012 From: bourinov@REDACTED (Max Bourinov) Date: Sun, 23 Dec 2012 13:10:40 +0400 Subject: [erlang-questions] one run-queue per scheduler?? In-Reply-To: References: <2087089229.2565493.1356084924014.JavaMail.root@erlang-solutions.com> Message-ID: Thank you Bjorn! Now I have good starting points )) Best regards, Max On Sat, Dec 22, 2012 at 8:45 PM, Bj?rn-Egil Dahlberg < wallentin.dahlberg@REDACTED> wrote: > Hi Max, > > Wouldn't we all want that info handy? =) > > Best bet for info about the runtime system and VM are the HiPE papers, > > See Kostis papers: http://user.it.uu.se/~kostis/Papers/ > > I couldn't find Erik Stenmans (Happi) papers http://user.it.uu.se/~happi/ > Some of them had a really good overview if I remember correctly. > > Also Jianrong Zhangs paper "Characterizing the Scalability of Erlang VM on > Many-Core Processors" has a good overview of some of the internals. (You > have to google for the link to the pdf. KTH paper) > > Jianrong Zhangs paper is newer then the HiPE papers and probably has more > current information. > > Some information are hidden in the *draft* of the erlang spec, > see http://www.erlang.org/doc.html > > I recommend checking out Rickard Greens and Patrik Nyblom talks at various > EUC, Factories and SICS Multicore Days to find the bleeding edge info =) > > Happy hunting, > Bj?rn-Egil > > > 2012/12/22 Max Bourinov > >> Hi Erlangers, >> >> I would like to know more about Erlang VM and how it works. Could you >> please point me to papers that would be interesting to read please? >> Something like Venu mentioned would be great to read. >> >> Best regards, >> Max >> >> >> >> On Fri, Dec 21, 2012 at 9:51 PM, Venu Middela wrote: >> >>> Thanks Robert and Harsha. >>> >>> Venu >>> >>> > Date: Fri, 21 Dec 2012 10:15:24 +0000 >>> > From: robert.virding@REDACTED >>> > To: qwertymaniac@REDACTED >>> > CC: erlang-questions@REDACTED; mvm_8@REDACTED >>> >>> > Subject: Re: [erlang-questions] one run-queue per scheduler?? >>> > >>> > Just to be more explicit: all versions from R13 and onwards have >>> scheduler specific run-queues and no common run-queue. This was an >>> important factor in making the BEAM scale better over many >>> cores/schedulers. There is some global work done in trying to keep a >>> reasonable balance of work between the schedulers. >>> > >>> > Robert >>> > >>> > ----- Original Message ----- >>> > > From: "Harsh J" >>> > > To: "Venu Middela" >>> > > Cc: erlang-questions@REDACTED >>> > > Sent: Friday, 21 December, 2012 5:10:48 AM >>> > > Subject: Re: [erlang-questions] one run-queue per scheduler?? >>> > > >>> > > Yes the R15B with SMP support now uses multiple, scheduler specific >>> > > run queues, instead of one globally shared run queue. This is >>> > > highlighted at >>> > > http://www.erlang.org/documentation/doc-5.7.4/doc/highlights.html >>> > > >>> > > On Fri, Dec 21, 2012 at 12:29 AM, Venu Middela >>> > > wrote: >>> > > > Hi, >>> > > > >>> > > > Does the Version OTP-R15B* or any other versions implement one >>> > > > run-queue >>> > > > per scheduler? >>> > > > >>> > > > Please refer to section 5.2 of this Technical Paper. >>> > > > >>> > > > http://www.erlang.se/euc/08/euc_smp.pdf >>> > > > >>> > > > I'm interested to know if this has been implemented in any of the >>> > > > versions? >>> > > > >>> > > > >>> > > > Thanks, >>> > > > Venu >>> > > > >>> > > > _______________________________________________ >>> > > > erlang-questions mailing list >>> > > > erlang-questions@REDACTED >>> > > > http://erlang.org/mailman/listinfo/erlang-questions >>> > > > >>> > > >>> > > >>> > > >>> > > -- >>> > > Harsh J >>> > > http://harshj.com >>> > > _______________________________________________ >>> > > erlang-questions mailing list >>> > > erlang-questions@REDACTED >>> > > http://erlang.org/mailman/listinfo/erlang-questions >>> > > >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Sun Dec 23 10:56:35 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sun, 23 Dec 2012 11:56:35 +0200 Subject: [erlang-questions] silent death of node Message-ID: Hello, I got an interesting issue of silent death erlang node. I am in the process of investigation what is going on but may be some of you met similar issue. So, I am using R15B01 and R15B03-1 on EC2 small instance. The application is trivial it opens tcp/ip socket and periodically dumps a file to it, supervisor is used to manage processes, etc. I am using rebar to package the app andotp runtime. The application is started as expected but after some time it hands up. Both lager and sasl logs are empty, no error, crashes are reported. Any node attach attempts are failed with error 'Node is not running'... Any hints on the issue? Thank you in advanced and Best Regards, Dmitry From tony@REDACTED Sun Dec 23 12:03:21 2012 From: tony@REDACTED (Tony Rogvall) Date: Sun, 23 Dec 2012 12:03:21 +0100 Subject: [erlang-questions] silent death of node In-Reply-To: References: Message-ID: <2E3B77FC-E8C5-4D3C-9CD6-D3D4BEF78EA7@rogvall.se> Hi! I have seen the same in a very similar setup. I think this may have something to do with a bug in inet_drv.c writev returns 0 for some unknown reason causing the code looping forever. I think the forever loop was put there to fill the kernel buffer as much as possible. There exist a patch at erlang-bugs archive somewhere. Check if this patch works for you. http://erlang.org/pipermail/erlang-bugs/2012-November/003217.html /Tony On 23 dec 2012, at 10:56, Dmitry Kolesnikov wrote: > Hello, > > I got an interesting issue of silent death erlang node. I am in the process of investigation what is going on but may be some of you met similar issue. > > So, I am using R15B01 and R15B03-1 on EC2 small instance. The application is trivial it opens tcp/ip socket and periodically dumps a file to it, supervisor is used to manage processes, etc. I am using rebar to package the app andotp runtime. The application is started as expected but after some time it hands up. Both lager and sasl logs are empty, no error, crashes are reported. Any node attach attempts are failed with error 'Node is not running'... > > Any hints on the issue? > > Thank you in advanced and Best Regards, > Dmitry > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Sun Dec 23 21:44:21 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sun, 23 Dec 2012 22:44:21 +0200 Subject: [erlang-questions] silent death of node In-Reply-To: <2E3B77FC-E8C5-4D3C-9CD6-D3D4BEF78EA7@rogvall.se> References: <2E3B77FC-E8C5-4D3C-9CD6-D3D4BEF78EA7@rogvall.se> Message-ID: <5AC19B66-5379-42C5-9C76-5ED1B5ECA9CF@gmail.com> Hello, Thanks for suggestion! I've tried the patch. Unfortunately, it do not solve my issue. I am still observing same effect. - Dmitry On Dec 23, 2012, at 1:03 PM, Tony Rogvall wrote: > Hi! > I have seen the same in a very similar setup. > I think this may have something to do with a bug in inet_drv.c writev returns 0 for some unknown reason > causing the code looping forever. I think the forever loop was put there to fill the kernel buffer as much > as possible. There exist a patch at erlang-bugs archive somewhere. > > Check if this patch works for you. > > http://erlang.org/pipermail/erlang-bugs/2012-November/003217.html > > /Tony > > > On 23 dec 2012, at 10:56, Dmitry Kolesnikov wrote: > >> Hello, >> >> I got an interesting issue of silent death erlang node. I am in the process of investigation what is going on but may be some of you met similar issue. >> >> So, I am using R15B01 and R15B03-1 on EC2 small instance. The application is trivial it opens tcp/ip socket and periodically dumps a file to it, supervisor is used to manage processes, etc. I am using rebar to package the app andotp runtime. The application is started as expected but after some time it hands up. Both lager and sasl logs are empty, no error, crashes are reported. Any node attach attempts are failed with error 'Node is not running'... >> >> Any hints on the issue? >> >> Thank you in advanced and Best Regards, >> Dmitry >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rgmatthews@REDACTED Sun Dec 23 23:08:49 2012 From: rgmatthews@REDACTED (Bob Matthews) Date: Mon, 24 Dec 2012 11:08:49 +1300 Subject: [erlang-questions] Beginner questions Message-ID: <50D780F1.2090401@orcon.net.nz> Hi I have just started to learn Erlang............ I am using Windows 7 on a PC *Question 1*) when I get an error message, what is the easiest way for me to find the exact line number? I use ms Word as an editor. *Question 2*) the code I am looking at was run on Unix, on a server. Part of the code talks about server call-backs etc. It is unclear how I translate that to a windows PC Many thanks Bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From matti.oinas@REDACTED Sun Dec 23 23:58:05 2012 From: matti.oinas@REDACTED (Matti Oinas) Date: Mon, 24 Dec 2012 00:58:05 +0200 Subject: [erlang-questions] Beginner questions In-Reply-To: <50D780F1.2090401@orcon.net.nz> References: <50D780F1.2090401@orcon.net.nz> Message-ID: <50D78C7D.4060303@gmail.com> Hi, My first proposal is to change the editor to something more usable like emacs or if you prefer something different then notepad++, sublime text or eclipse + erlIDE. Everyone of these are able to show you the line numbers and offer syntax highlight features. It would help a lot if you tell us which code you are looking at. Server callbacks probably mean gen_server callbacks so it doesn't matter if you are using windows or linux. Matti On 12/24/2012 12:08 AM, Bob Matthews wrote: > Hi > > I have just started to learn Erlang............ > I am using Windows 7 on a PC > > *Question 1*) when I get an error message, what is the easiest way for > me to find the exact line number? > I use ms Word as an editor. > > *Question 2*) the code I am looking at was run on Unix, on a server. > Part of the code talks about server call-backs etc. > It is unclear how I translate that to a windows PC > > Many thanks > > Bob > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From oj@REDACTED Mon Dec 24 00:06:02 2012 From: oj@REDACTED (OJ Reeves) Date: Mon, 24 Dec 2012 09:06:02 +1000 Subject: [erlang-questions] Beginner questions In-Reply-To: <50D78C7D.4060303@gmail.com> References: <50D780F1.2090401@orcon.net.nz> <50D78C7D.4060303@gmail.com> Message-ID: Word is definitely a choice of editor that you might regret later. So +1 on the change of editor, but -1 for Emacs or anything too complicated. If you're looking to learn Erlang rather than a new editor then avoid an editor which might be considered a learning curve by itself. Notepad++ or Sublime are good options, but anything simple with syntax highlighting and line numbers would be fine. The errors that Erlang spits out might be a little hard to grok to start with. Make sure you're running the latest version of Erlang on your machine (ie. something >= 15) so that you get line numbers in the error output. HTH OJ On Mon, Dec 24, 2012 at 8:58 AM, Matti Oinas wrote: > Hi, > > My first proposal is to change the editor to something more usable like > emacs or if you prefer something different then notepad++, sublime text or > eclipse + erlIDE. Everyone of these are able to show you the line numbers > and offer syntax highlight features. > > It would help a lot if you tell us which code you are looking at. Server > callbacks probably mean gen_server callbacks so it doesn't matter if you > are using windows or linux. > > Matti > > > On 12/24/2012 12:08 AM, Bob Matthews wrote: > > Hi > > I have just started to learn Erlang............ > I am using Windows 7 on a PC > > *Question 1*) when I get an error message, what is the easiest way for > me to find the exact line number? > I use ms Word as an editor. > > *Question 2*) the code I am looking at was run on Unix, on a server. > Part of the code talks about server call-backs etc. > It is unclear how I translate that to a windows PC > > Many thanks > > Bob > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- OJ Reeves +61 431 952 586 http://buffered.io/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From matti.oinas@REDACTED Mon Dec 24 00:10:21 2012 From: matti.oinas@REDACTED (Matti Oinas) Date: Mon, 24 Dec 2012 01:10:21 +0200 Subject: [erlang-questions] Beginner questions In-Reply-To: References: <50D780F1.2090401@orcon.net.nz> <50D78C7D.4060303@gmail.com> Message-ID: <50D78F5D.6050801@gmail.com> Good point about emacs. I have been using it and vi too long to remember how hard those were to learn. On 12/24/2012 01:06 AM, OJ Reeves wrote: > Word is definitely a choice of editor that you might regret later. So > +1 on the change of editor, but -1 for Emacs or anything too > complicated. If you're looking to learn Erlang rather than a new > editor then avoid an editor which might be considered a learning curve > by itself. Notepad++ or Sublime are good options, but anything simple > with syntax highlighting and line numbers would be fine. > > The errors that Erlang spits out might be a little hard to grok to > start with. Make sure you're running the latest version of Erlang on > your machine (ie. something >= 15) so that you get line numbers in the > error output. > > HTH > OJ > > On Mon, Dec 24, 2012 at 8:58 AM, Matti Oinas > wrote: > > Hi, > > My first proposal is to change the editor to something more usable > like emacs or if you prefer something different then notepad++, > sublime text or eclipse + erlIDE. Everyone of these are able to > show you the line numbers and offer syntax highlight features. > > It would help a lot if you tell us which code you are looking at. > Server callbacks probably mean gen_server callbacks so it doesn't > matter if you are using windows or linux. > > Matti > > > On 12/24/2012 12:08 AM, Bob Matthews wrote: >> Hi >> >> I have just started to learn Erlang............ >> I am using Windows 7 on a PC >> >> *Question 1*) when I get an error message, what is the easiest >> way for me to find the exact line number? >> I use ms Word as an editor. >> >> *Question 2*) the code I am looking at was run on Unix, on a server. >> Part of the code talks about server call-backs etc. >> It is unclear how I translate that to a windows PC >> >> Many thanks >> >> Bob >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > > OJ Reeves > +61 431 952 586 > http://buffered.io/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Mon Dec 24 00:16:00 2012 From: tony@REDACTED (Tony Rogvall) Date: Mon, 24 Dec 2012 00:16:00 +0100 Subject: [erlang-questions] silent death of node In-Reply-To: <5AC19B66-5379-42C5-9C76-5ED1B5ECA9CF@gmail.com> References: <2E3B77FC-E8C5-4D3C-9CD6-D3D4BEF78EA7@rogvall.se> <5AC19B66-5379-42C5-9C76-5ED1B5ECA9CF@gmail.com> Message-ID: That was unfortunate. Any information about the EC2 setup? How long time has the node been running? how much memory is consumed? I am eager to hear what you find out, please keep us posted. /Tony On 23 dec 2012, at 21:44, Dmitry Kolesnikov wrote: > Hello, > > Thanks for suggestion! I've tried the patch. > Unfortunately, it do not solve my issue. > I am still observing same effect. > > - Dmitry > > On Dec 23, 2012, at 1:03 PM, Tony Rogvall wrote: > >> Hi! >> I have seen the same in a very similar setup. >> I think this may have something to do with a bug in inet_drv.c writev returns 0 for some unknown reason >> causing the code looping forever. I think the forever loop was put there to fill the kernel buffer as much >> as possible. There exist a patch at erlang-bugs archive somewhere. >> >> Check if this patch works for you. >> >> http://erlang.org/pipermail/erlang-bugs/2012-November/003217.html >> >> /Tony >> >> >> On 23 dec 2012, at 10:56, Dmitry Kolesnikov wrote: >> >>> Hello, >>> >>> I got an interesting issue of silent death erlang node. I am in the process of investigation what is going on but may be some of you met similar issue. >>> >>> So, I am using R15B01 and R15B03-1 on EC2 small instance. The application is trivial it opens tcp/ip socket and periodically dumps a file to it, supervisor is used to manage processes, etc. I am using rebar to package the app andotp runtime. The application is started as expected but after some time it hands up. Both lager and sasl logs are empty, no error, crashes are reported. Any node attach attempts are failed with error 'Node is not running'... >>> >>> Any hints on the issue? >>> >>> Thank you in advanced and Best Regards, >>> Dmitry >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" >> >> >> > "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From surferjeff@REDACTED Mon Dec 24 01:07:02 2012 From: surferjeff@REDACTED (Jeffrey Rennie) Date: Sun, 23 Dec 2012 16:07:02 -0800 Subject: [erlang-questions] Working around the Windows firewall Message-ID: <00d001cde16a$9a795bc0$cf6c1340$@com> Before I draft a formal EEP, Windows firewall is enabled by default, and blocks sockets used by erlang. Yes, exceptions can be added to the firewall, however, this is not possible in some situations. For example: 1. A corporate network with strict policies that prohibit opening ports used by erlang. 2. An application written in erlang that installs without administrator privileges. 3. (A generalization of the above) Whenever the user who wants to run erlang is not an administrator. In these environments, an alternative to sockets which get entangled in the firewall, is Win32 named pipes which do not get entangled in the firewall. Therefore, I propose a change that will allow erlang nodes to communicate over named pipes in addition to sockets. Of course, it would be controlled by command-line parameters. I've spent a day browsing the erlang source code and it looks pretty doable. Comments and questions please. P.S. It took me about 5 hours to successfully build on Windows; I had most of the tools like Visual Studio and cygwin installed. I got tripped up by git switching LFs to CRLFs, which chokes bash. Also, it's much faster to just run vcvars32.bat and then run bash than to write your own script that sets the vc environment variables. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rgmatthews@REDACTED Mon Dec 24 02:32:02 2012 From: rgmatthews@REDACTED (Bob Matthews) Date: Mon, 24 Dec 2012 14:32:02 +1300 Subject: [erlang-questions] Beginner questions In-Reply-To: <50D78C7D.4060303@gmail.com> References: <50D780F1.2090401@orcon.net.nz> <50D78C7D.4060303@gmail.com> Message-ID: <50D7B092.20009@orcon.net.nz> The code that I am looking at is DXNN2 copyrighted to Gene Sher. The error message I got which I am having difficulty interpreting is as follows.................... 4> population_monitor:start(). {error,{{badrecord,state}, [{population_monitor,init,1, [{file,"population_monitor.erl"},{line,82}]}, {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, {proc_lib,init_p_do_apply,3, [{file,"proc_lib.erl"},{line,227}]}]}} I am using the latest versionR15B03 Thank you for you help Bob Dunedin New Zealand On 24/12/2012 11:58 a.m., Matti Oinas wrote: > Hi, > > My first proposal is to change the editor to something more usable > like emacs or if you prefer something different then notepad++, > sublime text or eclipse + erlIDE. Everyone of these are able to show > you the line numbers and offer syntax highlight features. > > It would help a lot if you tell us which code you are looking at. > Server callbacks probably mean gen_server callbacks so it doesn't > matter if you are using windows or linux. > > Matti > > On 12/24/2012 12:08 AM, Bob Matthews wrote: >> Hi >> >> I have just started to learn Erlang............ >> I am using Windows 7 on a PC >> >> *Question 1*) when I get an error message, what is the easiest way >> for me to find the exact line number? >> I use ms Word as an editor. >> >> *Question 2*) the code I am looking at was run on Unix, on a server. >> Part of the code talks about server call-backs etc. >> It is unclear how I translate that to a windows PC >> >> Many thanks >> >> Bob >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: 12/11/12 > Internal Virus Database is out of date. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From oj@REDACTED Mon Dec 24 03:19:23 2012 From: oj@REDACTED (OJ Reeves) Date: Mon, 24 Dec 2012 12:19:23 +1000 Subject: [erlang-questions] Beginner questions In-Reply-To: <50D7B092.20009@orcon.net.nz> References: <50D780F1.2090401@orcon.net.nz> <50D78C7D.4060303@gmail.com> <50D7B092.20009@orcon.net.nz> Message-ID: Bob, I just went through the instructions that were listed in the README.txt: STARTING POINT -------------- >From inside Erlang. 1. %%%%Compilation%%%% make:all(). 2. %%%%Initialize All Databases%%%% First create a folder called ?benchmarks?, the system expects it to exist, and writes files to it when performing benchmarks. polis:create(). % This creates the database 3. %%%%Start The Polis Databases%%%% polis:start(). % This starts the polis process, the whole thing, the infrastructure (it runs the scapes...) 4. At this point you can summon NN based agents or populations of agents, construct Sensors and Actuators and provide them to the NNs... This section will be expanded in future additions. I followed those instructions and the application works perfectly for me. Did you do the same? I'm on Windows 8 x64. OJ On Mon, Dec 24, 2012 at 11:32 AM, Bob Matthews wrote: > The code that I am looking at is DXNN2 copyrighted to Gene Sher. > > The error message I got which I am having difficulty interpreting is as > follows.................... > > 4> population_monitor:start(). > {error,{{badrecord,state}, > [{population_monitor,init,1, > [{file,"population_monitor.erl"},{line,82}]}, > {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, > {proc_lib,init_p_do_apply,3, > [{file,"proc_lib.erl"},{line,227}]}]}} > > I am using the latest versionR15B03 > > Thank you for you help > > Bob > Dunedin New Zealand > > > On 24/12/2012 11:58 a.m., Matti Oinas wrote: > > Hi, > > My first proposal is to change the editor to something more usable like > emacs or if you prefer something different then notepad++, sublime text or > eclipse + erlIDE. Everyone of these are able to show you the line numbers > and offer syntax highlight features. > > It would help a lot if you tell us which code you are looking at. Server > callbacks probably mean gen_server callbacks so it doesn't matter if you > are using windows or linux. > > Matti > > On 12/24/2012 12:08 AM, Bob Matthews wrote: > > Hi > > I have just started to learn Erlang............ > I am using Windows 7 on a PC > > *Question 1*) when I get an error message, what is the easiest way for > me to find the exact line number? > I use ms Word as an editor. > > *Question 2*) the code I am looking at was run on Unix, on a server. > Part of the code talks about server call-backs etc. > It is unclear how I translate that to a windows PC > > Many thanks > > Bob > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: 12/11/12 > Internal Virus Database is out of date. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- OJ Reeves +61 431 952 586 http://buffered.io/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rgmatthews@REDACTED Mon Dec 24 03:54:05 2012 From: rgmatthews@REDACTED (Bob Matthews) Date: Mon, 24 Dec 2012 15:54:05 +1300 Subject: [erlang-questions] Beginner questions In-Reply-To: References: <50D780F1.2090401@orcon.net.nz> <50D78C7D.4060303@gmail.com> <50D7B092.20009@orcon.net.nz> Message-ID: <50D7C3CD.2040805@orcon.net.nz> yes I followed the instructions.................... Erlang R15B03 (erts-5.9.3.1) [smp:2:2] [async-threads:0] Eshell V5.9.3.1 (abort with ^G) 1> cd("e:/DXNN2_live"). e:/DXNN2_live ok 2> make:all(). up_to_date 3> polis:create(). {aborted,{already_exists,experiment}} 4> polis:sync(). up_to_date 5> polis:start(). Parameters:{[],[]} ******** Polis: ##MATHEMA## is now online. {ok,<0.252.0>} 6> fx:start(). true ******** FX Tables:[{'EURUSD60','EURUSD60'}, {'EURUSD30','EURUSD30'}, {'EURUSD15','EURUSD15'}, {metadata,metadata}] started 7> population_monitor:start(). {error,{{badrecord,state}, [{population_monitor,init,1, [{file,"population_monitor.erl"},{line,82}]}, {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, {proc_lib,init_p_do_apply,3, [{file,"proc_lib.erl"},{line,227}]}]}} 8> Bob On 24/12/2012 3:19 p.m., OJ Reeves wrote: > Bob, > > I just went through the instructions that were listed in the README.txt: > > STARTING POINT > -------------- > >From inside Erlang. > 1. %%%%Compilation%%%% > make:all(). > 2. %%%%Initialize All Databases%%%% > First create a folder called ?benchmarks?, the system expects it to exist, and writes files to it when performing benchmarks. > polis:create(). % This creates the database > 3. %%%%Start The Polis Databases%%%% > polis:start(). % This starts the polis process, the whole thing, the infrastructure (it runs the scapes...) > 4. At this point you can summon NN based agents or populations of agents, construct Sensors and Actuators and provide them to the NNs... This section will be expanded in future additions. > > > I followed those instructions and the application works perfectly for > me. Did you do the same? I'm on Windows 8 x64. > > OJ > > > On Mon, Dec 24, 2012 at 11:32 AM, Bob Matthews > > wrote: > > The code that I am looking at is DXNN2 copyrighted to Gene Sher. > > The error message I got which I am having difficulty interpreting > is as follows.................... > > 4> population_monitor:start(). > {error,{{badrecord,state}, > [{population_monitor,init,1, > [{file,"population_monitor.erl"},{line,82}]}, > {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, > {proc_lib,init_p_do_apply,3, > [{file,"proc_lib.erl"},{line,227}]}]}} > > I am using the latest versionR15B03 > > Thank you for you help > > Bob > Dunedin New Zealand > > > On 24/12/2012 11:58 a.m., Matti Oinas wrote: >> Hi, >> >> My first proposal is to change the editor to something more >> usable like emacs or if you prefer something different then >> notepad++, sublime text or eclipse + erlIDE. Everyone of these >> are able to show you the line numbers and offer syntax highlight >> features. >> >> It would help a lot if you tell us which code you are looking at. >> Server callbacks probably mean gen_server callbacks so it doesn't >> matter if you are using windows or linux. >> >> Matti >> >> On 12/24/2012 12:08 AM, Bob Matthews wrote: >>> Hi >>> >>> I have just started to learn Erlang............ >>> I am using Windows 7 on a PC >>> >>> *Question 1*) when I get an error message, what is the easiest >>> way for me to find the exact line number? >>> I use ms Word as an editor. >>> >>> *Question 2*) the code I am looking at was run on Unix, on a >>> server. >>> Part of the code talks about server call-backs etc. >>> It is unclear how I translate that to a windows PC >>> >>> Many thanks >>> >>> Bob >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> No virus found in this message. >> Checked by AVG - www.avg.com >> Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: >> 12/11/12 >> Internal Virus Database is out of date. >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > > OJ Reeves > +61 431 952 586 > http://buffered.io/ > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: 12/11/12 > Internal Virus Database is out of date. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Mon Dec 24 08:14:54 2012 From: ivan@REDACTED (Ivan Uemlianin) Date: Mon, 24 Dec 2012 07:14:54 +0000 Subject: [erlang-questions] Beginner questions In-Reply-To: <50D7B092.20009@orcon.net.nz> References: <50D780F1.2090401@orcon.net.nz> <50D78C7D.4060303@gmail.com> <50D7B092.20009@orcon.net.nz> Message-ID: I dont have my computer with me but the error is at line 82 of the file population_monitor.erl. Best wishes Ivan -- festina lente On 24 Dec 2012, at 01:32, Bob Matthews wrote: > The code that I am looking at is DXNN2 copyrighted to Gene Sher. > > The error message I got which I am having difficulty interpreting is as follows.................... > > 4> population_monitor:start(). > {error,{{badrecord,state}, > [{population_monitor,init,1, > [{file,"population_monitor.erl"},{line,82}]}, > {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, > {proc_lib,init_p_do_apply,3, > [{file,"proc_lib.erl"},{line,227}]}]}} > > I am using the latest versionR15B03 > > Thank you for you help > > Bob > Dunedin New Zealand > > On 24/12/2012 11:58 a.m., Matti Oinas wrote: >> Hi, >> >> My first proposal is to change the editor to something more usable like emacs or if you prefer something different then notepad++, sublime text or eclipse + erlIDE. Everyone of these are able to show you the line numbers and offer syntax highlight features. >> >> It would help a lot if you tell us which code you are looking at. Server callbacks probably mean gen_server callbacks so it doesn't matter if you are using windows or linux. >> >> Matti >> >> On 12/24/2012 12:08 AM, Bob Matthews wrote: >>> Hi >>> >>> I have just started to learn Erlang............ >>> I am using Windows 7 on a PC >>> >>> Question 1) when I get an error message, what is the easiest way for me to find the exact line number? >>> I use ms Word as an editor. >>> >>> Question 2) the code I am looking at was run on Unix, on a server. >>> Part of the code talks about server call-backs etc. >>> It is unclear how I translate that to a windows PC >>> >>> Many thanks >>> >>> Bob >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> No virus found in this message. >> Checked by AVG - www.avg.com >> Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: 12/11/12 >> Internal Virus Database is out of date. >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rgmatthews@REDACTED Mon Dec 24 08:24:22 2012 From: rgmatthews@REDACTED (Bob Matthews) Date: Mon, 24 Dec 2012 20:24:22 +1300 Subject: [erlang-questions] Beginner questions In-Reply-To: References: <50D780F1.2090401@orcon.net.nz> <50D78C7D.4060303@gmail.com> <50D7B092.20009@orcon.net.nz> Message-ID: <50D80326.4030202@orcon.net.nz> Thank you Ivan line 82: Population_Id = S#state.population_id, Merry Xmas to you too :) Bob Dunedin New Zealand On 24/12/2012 8:14 p.m., Ivan Uemlianin wrote: > I dont have my computer with me but the error is at line 82 of the > file population_monitor.erl. > > Best wishes > > Ivan > > -- > festina lente > > > On 24 Dec 2012, at 01:32, Bob Matthews > wrote: > >> The code that I am looking at is DXNN2 copyrighted to Gene Sher. >> >> The error message I got which I am having difficulty interpreting is >> as follows.................... >> >> 4> population_monitor:start(). >> {error,{{badrecord,state}, >> [{population_monitor,init,1, >> [{file,"population_monitor.erl"},{line,82}]}, >> {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, >> {proc_lib,init_p_do_apply,3, >> [{file,"proc_lib.erl"},{line,227}]}]}} >> >> I am using the latest versionR15B03 >> >> Thank you for you help >> >> Bob >> Dunedin New Zealand >> >> On 24/12/2012 11:58 a.m., Matti Oinas wrote: >>> Hi, >>> >>> My first proposal is to change the editor to something more usable >>> like emacs or if you prefer something different then notepad++, >>> sublime text or eclipse + erlIDE. Everyone of these are able to show >>> you the line numbers and offer syntax highlight features. >>> >>> It would help a lot if you tell us which code you are looking at. >>> Server callbacks probably mean gen_server callbacks so it doesn't >>> matter if you are using windows or linux. >>> >>> Matti >>> >>> On 12/24/2012 12:08 AM, Bob Matthews wrote: >>>> Hi >>>> >>>> I have just started to learn Erlang............ >>>> I am using Windows 7 on a PC >>>> >>>> *Question 1*) when I get an error message, what is the easiest way >>>> for me to find the exact line number? >>>> I use ms Word as an editor. >>>> >>>> *Question 2*) the code I am looking at was run on Unix, on a server. >>>> Part of the code talks about server call-backs etc. >>>> It is unclear how I translate that to a windows PC >>>> >>>> Many thanks >>>> >>>> Bob >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> No virus found in this message. >>> Checked by AVG - www.avg.com >>> Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: >>> 12/11/12 >>> Internal Virus Database is out of date. >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: 12/11/12 > Internal Virus Database is out of date. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Mon Dec 24 08:49:01 2012 From: ivan@REDACTED (Ivan Uemlianin) Date: Mon, 24 Dec 2012 07:49:01 +0000 Subject: [erlang-questions] Beginner questions In-Reply-To: <50D80326.4030202@orcon.net.nz> References: <50D780F1.2090401@orcon.net.nz> <50D78C7D.4060303@gmail.com> <50D7B092.20009@orcon.net.nz> <50D80326.4030202@orcon.net.nz> Message-ID: <5CD52A56-6662-4DB2-B2FC-B9650514E572@llaisdy.com> Dear Bob This line should be part of the function population_monitor:init/1 (that's file population_monitor, function init, which takes one argument). The error it complains of is {badrecord,state}, so perhaps the record S has been badly set up. Are you learning erlang through neuroevolution :) ? Do you have other reference resources? Nadolig Llawen Ivan -- festina lente On 24 Dec 2012, at 07:24, Bob Matthews wrote: > Thank you Ivan > > line 82: Population_Id = S#state.population_id, > > Merry Xmas to you too :) > > Bob > Dunedin > New Zealand > > On 24/12/2012 8:14 p.m., Ivan Uemlianin wrote: >> I dont have my computer with me but the error is at line 82 of the file population_monitor.erl. >> >> Best wishes >> >> Ivan >> >> -- >> festina lente >> >> >> On 24 Dec 2012, at 01:32, Bob Matthews wrote: >> >>> The code that I am looking at is DXNN2 copyrighted to Gene Sher. >>> >>> The error message I got which I am having difficulty interpreting is as follows.................... >>> >>> 4> population_monitor:start(). >>> {error,{{badrecord,state}, >>> [{population_monitor,init,1, >>> [{file,"population_monitor.erl"},{line,82}]}, >>> {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, >>> {proc_lib,init_p_do_apply,3, >>> [{file,"proc_lib.erl"},{line,227}]}]}} >>> >>> I am using the latest versionR15B03 >>> >>> Thank you for you help >>> >>> Bob >>> Dunedin New Zealand >>> >>> On 24/12/2012 11:58 a.m., Matti Oinas wrote: >>>> Hi, >>>> >>>> My first proposal is to change the editor to something more usable like emacs or if you prefer something different then notepad++, sublime text or eclipse + erlIDE. Everyone of these are able to show you the line numbers and offer syntax highlight features. >>>> >>>> It would help a lot if you tell us which code you are looking at. Server callbacks probably mean gen_server callbacks so it doesn't matter if you are using windows or linux. >>>> >>>> Matti >>>> >>>> On 12/24/2012 12:08 AM, Bob Matthews wrote: >>>>> Hi >>>>> >>>>> I have just started to learn Erlang............ >>>>> I am using Windows 7 on a PC >>>>> >>>>> Question 1) when I get an error message, what is the easiest way for me to find the exact line number? >>>>> I use ms Word as an editor. >>>>> >>>>> Question 2) the code I am looking at was run on Unix, on a server. >>>>> Part of the code talks about server call-backs etc. >>>>> It is unclear how I translate that to a windows PC >>>>> >>>>> Many thanks >>>>> >>>>> Bob >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> No virus found in this message. >>>> Checked by AVG - www.avg.com >>>> Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: 12/11/12 >>>> Internal Virus Database is out of date. >>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> No virus found in this message. >> Checked by AVG - www.avg.com >> Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: 12/11/12 >> Internal Virus Database is out of date. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rgmatthews@REDACTED Mon Dec 24 10:32:23 2012 From: rgmatthews@REDACTED (Bob Matthews) Date: Mon, 24 Dec 2012 22:32:23 +1300 Subject: [erlang-questions] Beginner questions In-Reply-To: <5CD52A56-6662-4DB2-B2FC-B9650514E572@llaisdy.com> References: <50D780F1.2090401@orcon.net.nz> <50D78C7D.4060303@gmail.com> <50D7B092.20009@orcon.net.nz> <50D80326.4030202@orcon.net.nz> <5CD52A56-6662-4DB2-B2FC-B9650514E572@llaisdy.com> Message-ID: <50D82127.4060303@orcon.net.nz> Hi Ivan Thank you for your advice....I will try and sort this out now! My interest is the forex market and neural networks.................... I know a little about the former but am learning about the latter, and of course, I am also trying to learn erlang Thank's for helping Bob New Zealand On 24/12/2012 8:49 p.m., Ivan Uemlianin wrote: > Dear Bob > > This line should be part of the function population_monitor:init/1 > (that's file population_monitor, function init, which takes one > argument). > > The error it complains of is {badrecord,state}, so perhaps the record > S has been badly set up. > > Are you learning erlang through neuroevolution :) ? Do you have other > reference resources? > > Nadolig Llawen > > Ivan > > > -- > festina lente > > > On 24 Dec 2012, at 07:24, Bob Matthews > wrote: > >> Thank you Ivan >> >> line 82: Population_Id = S#state.population_id, >> >> Merry Xmas to you too :) >> >> Bob >> Dunedin >> New Zealand >> >> On 24/12/2012 8:14 p.m., Ivan Uemlianin wrote: >>> I dont have my computer with me but the error is at line 82 of the >>> file population_monitor.erl. >>> >>> Best wishes >>> >>> Ivan >>> >>> -- >>> festina lente >>> >>> >>> On 24 Dec 2012, at 01:32, Bob Matthews >> > wrote: >>> >>>> The code that I am looking at is DXNN2 copyrighted to Gene Sher. >>>> >>>> The error message I got which I am having difficulty interpreting >>>> is as follows.................... >>>> >>>> 4> population_monitor:start(). >>>> {error,{{badrecord,state}, >>>> [{population_monitor,init,1, >>>> [{file,"population_monitor.erl"},{line,82}]}, >>>> {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, >>>> {proc_lib,init_p_do_apply,3, >>>> [{file,"proc_lib.erl"},{line,227}]}]}} >>>> >>>> I am using the latest versionR15B03 >>>> >>>> Thank you for you help >>>> >>>> Bob >>>> Dunedin New Zealand >>>> >>>> On 24/12/2012 11:58 a.m., Matti Oinas wrote: >>>>> Hi, >>>>> >>>>> My first proposal is to change the editor to something more usable >>>>> like emacs or if you prefer something different then notepad++, >>>>> sublime text or eclipse + erlIDE. Everyone of these are able to >>>>> show you the line numbers and offer syntax highlight features. >>>>> >>>>> It would help a lot if you tell us which code you are looking at. >>>>> Server callbacks probably mean gen_server callbacks so it doesn't >>>>> matter if you are using windows or linux. >>>>> >>>>> Matti >>>>> >>>>> On 12/24/2012 12:08 AM, Bob Matthews wrote: >>>>>> Hi >>>>>> >>>>>> I have just started to learn Erlang............ >>>>>> I am using Windows 7 on a PC >>>>>> >>>>>> *Question 1*) when I get an error message, what is the easiest >>>>>> way for me to find the exact line number? >>>>>> I use ms Word as an editor. >>>>>> >>>>>> *Question 2*) the code I am looking at was run on Unix, on a server. >>>>>> Part of the code talks about server call-backs etc. >>>>>> It is unclear how I translate that to a windows PC >>>>>> >>>>>> Many thanks >>>>>> >>>>>> Bob >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> No virus found in this message. >>>>> Checked by AVG - www.avg.com >>>>> Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: >>>>> 12/11/12 >>>>> Internal Virus Database is out of date. >>>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> No virus found in this message. >>> Checked by AVG - www.avg.com >>> Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: >>> 12/11/12 >>> Internal Virus Database is out of date. >>> >> > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: 12/11/12 > Internal Virus Database is out of date. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From oj@REDACTED Mon Dec 24 10:35:36 2012 From: oj@REDACTED (OJ Reeves) Date: Mon, 24 Dec 2012 19:35:36 +1000 Subject: [erlang-questions] Beginner questions In-Reply-To: <5CD52A56-6662-4DB2-B2FC-B9650514E572@llaisdy.com> References: <50D780F1.2090401@orcon.net.nz> <50D78C7D.4060303@gmail.com> <50D7B092.20009@orcon.net.nz> <50D80326.4030202@orcon.net.nz> <5CD52A56-6662-4DB2-B2FC-B9650514E572@llaisdy.com> Message-ID: All, The first issue is that the init() function isn't correct for this gen_server. On line 83, the init function starts like this: init(S) -> When it should (apparently) be like this: init([S]) -> This doesn't make the server run though. The next error states: 6> population_monitor:start(). ******** Population monitor started with parameters:{state, [gt], test,[],[],undefined, undefined,undefined,[],0, 0,0,0,0,undefined, undefined,undefined, undefined,undefined, undefined,0.5,10,10, mathema,100,100000,inf, undefined,false} {error,{{badrecord,population}, [{population_monitor,extract_AgentIds,2, [{file,"population_monitor.erl"},{line,372}]}, {population_monitor,init,1, [{file,"population_monitor.erl"},{line,89}]}, {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, {proc_lib,init_p_do_apply,3, [{file,"proc_lib.erl"},{line,227}]}]}} I get the feeling that there's going to be more stuff requiring fixes as you go. Perhaps the best thing its to get in touch with the author directly and find out about the state of the source base before continuing. Cheers OJ On Mon, Dec 24, 2012 at 5:49 PM, Ivan Uemlianin wrote: > Dear Bob > > This line should be part of the function population_monitor:init/1 (that's > file population_monitor, function init, which takes one argument). > > The error it complains of is {badrecord,state}, so perhaps the record S > has been badly set up. > > Are you learning erlang through neuroevolution :) ? Do you have other > reference resources? > > Nadolig Llawen > > Ivan > > > -- > festina lente > > > On 24 Dec 2012, at 07:24, Bob Matthews wrote: > > Thank you Ivan > > line 82: Population_Id = S#state.population_id, > > Merry Xmas to you too :) > > Bob > Dunedin > New Zealand > > On 24/12/2012 8:14 p.m., Ivan Uemlianin wrote: > > I dont have my computer with me but the error is at line 82 of the file > population_monitor.erl. > > Best wishes > > Ivan > > -- > festina lente > > > On 24 Dec 2012, at 01:32, Bob Matthews wrote: > > The code that I am looking at is DXNN2 copyrighted to Gene Sher. > > The error message I got which I am having difficulty interpreting is as > follows.................... > > 4> population_monitor:start(). > {error,{{badrecord,state}, > [{population_monitor,init,1, > [{file,"population_monitor.erl"},{line,82}]}, > {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, > {proc_lib,init_p_do_apply,3, > [{file,"proc_lib.erl"},{line,227}]}]}} > > I am using the latest versionR15B03 > > Thank you for you help > > Bob > Dunedin New Zealand > > On 24/12/2012 11:58 a.m., Matti Oinas wrote: > > Hi, > > My first proposal is to change the editor to something more usable like > emacs or if you prefer something different then notepad++, sublime text or > eclipse + erlIDE. Everyone of these are able to show you the line numbers > and offer syntax highlight features. > > It would help a lot if you tell us which code you are looking at. Server > callbacks probably mean gen_server callbacks so it doesn't matter if you > are using windows or linux. > > Matti > > On 12/24/2012 12:08 AM, Bob Matthews wrote: > > Hi > > I have just started to learn Erlang............ > I am using Windows 7 on a PC > > *Question 1*) when I get an error message, what is the easiest way for > me to find the exact line number? > I use ms Word as an editor. > > *Question 2*) the code I am looking at was run on Unix, on a server. > Part of the code talks about server call-backs etc. > It is unclear how I translate that to a windows PC > > Many thanks > > Bob > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: 12/11/12 > Internal Virus Database is out of date. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: 12/11/12 > Internal Virus Database is out of date. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- OJ Reeves +61 431 952 586 http://buffered.io/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From evrenweb@REDACTED Mon Dec 24 15:28:09 2012 From: evrenweb@REDACTED (Evren Bayraktar) Date: Mon, 24 Dec 2012 16:28:09 +0200 Subject: [erlang-questions] STRCMP or something like that Message-ID: Hi everyone, I'm having trouble with Erlang I hope someone can help me. I'm really newbie in Erlang :) I need to something like strcmp in php. For example; I have a two different string, Alicia and Alex. I need to steady order for them. By parsing each letter I guess I can do that. $A. returns 65 so I can't compare with first letters then $l. returns 108 but they are same too. Now, $e. returns 101 and $i. returns 105. By using this numbers I can give correct order, first Alex then Alica. I mean, imagine a function which compare two different variable, by using their ascii codes it will return correct order. Function("alicia", "alex") ---> will return alex+alicia I hope someone will help me Best Regards Evren -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Dec 24 15:34:32 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 24 Dec 2012 15:34:32 +0100 Subject: [erlang-questions] STRCMP or something like that In-Reply-To: References: Message-ID: <50D867F8.9030607@ninenines.eu> 1> lists:sort(["alicia", "alex"]). ["alex","alicia"] :) On 12/24/2012 03:28 PM, Evren Bayraktar wrote: > Hi everyone, > > I'm having trouble with Erlang I hope someone can help me. I'm really > newbie in Erlang :) > > I need to something like strcmp in php. > > For example; > > I have a two different string, Alicia and Alex. I need to steady order > for them. By parsing each letter I guess I can do that. > > $A. returns 65 so I can't compare with first letters then $l. returns > 108 but they are same too. > Now, $e. returns 101 and $i. returns 105. By using this numbers I can > give correct order, first Alex then Alica. > > I mean, imagine a function which compare two different variable, by > using their ascii codes it will return > correct order. > > Function("alicia", "alex") ---> will return alex+alicia > > I hope someone will help me > > Best Regards > Evren > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From dmkolesnikov@REDACTED Mon Dec 24 15:55:32 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Mon, 24 Dec 2012 16:55:32 +0200 Subject: [erlang-questions] silent death of node In-Reply-To: References: <2E3B77FC-E8C5-4D3C-9CD6-D3D4BEF78EA7@rogvall.se> <5AC19B66-5379-42C5-9C76-5ED1B5ECA9CF@gmail.com> Message-ID: <2FF34B8A-80DE-46A8-8EEC-BFED7DE3ED42@gmail.com> Hello, Finally, I've discovered the issue. This was a bug in my code :-( I was leaking a file descriptors. Node got out of them, I was not able to attach to it anymore... - Dmitry On Dec 24, 2012, at 1:16 AM, Tony Rogvall wrote: > That was unfortunate. > Any information about the EC2 setup? How long time has the node been running? > how much memory is consumed? > I am eager to hear what you find out, please keep us posted. > > /Tony > > On 23 dec 2012, at 21:44, Dmitry Kolesnikov wrote: > >> Hello, >> >> Thanks for suggestion! I've tried the patch. >> Unfortunately, it do not solve my issue. >> I am still observing same effect. >> >> - Dmitry >> >> On Dec 23, 2012, at 1:03 PM, Tony Rogvall wrote: >> >>> Hi! >>> I have seen the same in a very similar setup. >>> I think this may have something to do with a bug in inet_drv.c writev returns 0 for some unknown reason >>> causing the code looping forever. I think the forever loop was put there to fill the kernel buffer as much >>> as possible. There exist a patch at erlang-bugs archive somewhere. >>> >>> Check if this patch works for you. >>> >>> http://erlang.org/pipermail/erlang-bugs/2012-November/003217.html >>> >>> /Tony >>> >>> >>> On 23 dec 2012, at 10:56, Dmitry Kolesnikov wrote: >>> >>>> Hello, >>>> >>>> I got an interesting issue of silent death erlang node. I am in the process of investigation what is going on but may be some of you met similar issue. >>>> >>>> So, I am using R15B01 and R15B03-1 on EC2 small instance. The application is trivial it opens tcp/ip socket and periodically dumps a file to it, supervisor is used to manage processes, etc. I am using rebar to package the app andotp runtime. The application is started as expected but after some time it hands up. Both lager and sasl logs are empty, no error, crashes are reported. Any node attach attempts are failed with error 'Node is not running'... >>>> >>>> Any hints on the issue? >>>> >>>> Thank you in advanced and Best Regards, >>>> Dmitry >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" >>> >>> >>> >> > > "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonstl@REDACTED Mon Dec 24 16:22:52 2012 From: simonstl@REDACTED (Simon St.Laurent) Date: Mon, 24 Dec 2012 10:22:52 -0500 Subject: [erlang-questions] STRCMP or something like that In-Reply-To: References: Message-ID: <50D8734C.7030600@simonstl.com> On 12/24/12 9:28 AM, Evren Bayraktar wrote: > Hi everyone, > > I'm having trouble with Erlang I hope someone can help me. I'm really > newbie in Erlang :) > > I need to something like strcmp in php. > > For example; > > I have a two different string, Alicia and Alex. I need to steady order > for them. By parsing each letter I guess I can do that. > > $A. returns 65 so I can't compare with first letters then $l. returns > 108 but they are same too. > Now, $e. returns 101 and $i. returns 105. By using this numbers I can > give correct order, first Alex then Alica. lists:sort will order them, as was noted, but if you just need comparison, you don't need to go character by character. < and > are fine with strings - 1> "Alicia" < "Alex". false 2> "Alicia" > "Alex". true I'm contemplating writing a simple wrapper library for the string and list functions that looks more familiar to programmers coming from other languages. Thanks, Simon St.Laurent Introducing Erlang > I mean, imagine a function which compare two different variable, by > using their ascii codes it will return > correct order. > > Function("alicia", "alex") ---> will return alex+alicia > > I hope someone will help me > > Best Regards > Evren From corticalcomputer@REDACTED Mon Dec 24 18:16:51 2012 From: corticalcomputer@REDACTED (Gene Sher) Date: Mon, 24 Dec 2012 12:16:51 -0500 Subject: [erlang-questions] Beginner questions In-Reply-To: References: <50D780F1.2090401@orcon.net.nz> <50D78C7D.4060303@gmail.com> <50D7B092.20009@orcon.net.nz> <50D80326.4030202@orcon.net.nz> <5CD52A56-6662-4DB2-B2FC-B9650514E572@llaisdy.com> Message-ID: Hello all, The problem is not with the code, but that population_monitor should not be started with start(). Population_monitor should be started with the properly set up constraints. To use the default constraints you start with population_monitor:test(). test()-> init_population(#state{op_mode = [gt,benchmark]},?INIT_CONSTRAINTS). %The test/0 function starts the population monitor through init_population/1 with a set of default parameters specified by the macros of this module. prep_PopState(PMP,Specie_Constraints)-> S=#state{ op_mode=PMP#pmp.op_mode, population_id = PMP#pmp.population_id, survival_percentage=PMP#pmp.survival_percentage, specie_size_limit=PMP#pmp.specie_size_limit, init_specie_size=PMP#pmp.init_specie_size, polis_id=PMP#pmp.polis_id, generation_limit=PMP#pmp.generation_limit, evaluations_limit=PMP#pmp.evaluations_limit, fitness_goal=PMP#pmp.fitness_goal, benchmarker_pid=PMP#pmp.benchmarker_pid }, init_population(S,Specie_Constraints). init_population(Init_State,Specie_Constraints)-> random:seed(now()), Population_Id = Init_State#state.population_id, %OpMode = Init_State#state.op_mode, F = fun()-> case genotype:read({population,Population_Id}) of undefined -> create_Population(Population_Id,Init_State#state.init_specie_size,Specie_Constraints); _ -> delete_population(Population_Id), create_Population(Population_Id,Init_State#state.init_specie_size,Specie_Constraints) end end, Result = mnesia:transaction(F), case Result of {atomic,_} -> population_monitor:start(Init_State); Error -> io:format("******** ERROR in PopulationMonitor:~p~n",[Error]) end. %The function init_population/1 creates a new population with the id Population_Id, composed of length(Specie_Constraints) species, where each specie uses the particular specie constraint specified within the Specie_Constraints list. The function first checks if a population with the noted Population_Id already exists, if a population does exist, then the function first delets it, and then creates a fresh one. Since the ids are usually generated with the genotype:create_UniqueId/0, the only way an already existing Population_Id is dropped into the function as a parameter is if it is intended, usually when runing tests, with the Population_Id = test. You will note that population_monitor;start(Init_state) is the only way to properly start the population monitor. I did not wish to remove the function population_monitor:start() from the module, for the sake of future extensions. But there are no problems with the code. Best regards, -Gene On Mon, Dec 24, 2012 at 4:35 AM, OJ Reeves wrote: > All, > > The first issue is that the init() function isn't correct for this > gen_server. > > On line 83, the init function starts like this: > init(S) -> > When it should (apparently) be like this: > init([S]) -> > > This doesn't make the server run though. The next error states: > > 6> population_monitor:start(). > ******** Population monitor started with parameters:{state, > [gt], > test,[],[],undefined, > > undefined,undefined,[],0, > 0,0,0,0,undefined, > undefined,undefined, > undefined,undefined, > undefined,0.5,10,10, > > mathema,100,100000,inf, > undefined,false} > {error,{{badrecord,population}, > [{population_monitor,extract_AgentIds,2, > [{file,"population_monitor.erl"},{line,372}]}, > {population_monitor,init,1, > [{file,"population_monitor.erl"},{line,89}]}, > {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, > {proc_lib,init_p_do_apply,3, > [{file,"proc_lib.erl"},{line,227}]}]}} > > I get the feeling that there's going to be more stuff requiring fixes as > you go. Perhaps the best thing its to get in touch with the author directly > and find out about the state of the source base before continuing. > > Cheers > OJ > > > On Mon, Dec 24, 2012 at 5:49 PM, Ivan Uemlianin wrote: > >> Dear Bob >> >> This line should be part of the function population_monitor:init/1 >> (that's file population_monitor, function init, which takes one argument). >> >> The error it complains of is {badrecord,state}, so perhaps the record S >> has been badly set up. >> >> Are you learning erlang through neuroevolution :) ? Do you have other >> reference resources? >> >> Nadolig Llawen >> >> Ivan >> >> >> -- >> festina lente >> >> >> On 24 Dec 2012, at 07:24, Bob Matthews wrote: >> >> Thank you Ivan >> >> line 82: Population_Id = S#state.population_id, >> >> Merry Xmas to you too :) >> >> Bob >> Dunedin >> New Zealand >> >> On 24/12/2012 8:14 p.m., Ivan Uemlianin wrote: >> >> I dont have my computer with me but the error is at line 82 of the file >> population_monitor.erl. >> >> Best wishes >> >> Ivan >> >> -- >> festina lente >> >> >> On 24 Dec 2012, at 01:32, Bob Matthews wrote: >> >> The code that I am looking at is DXNN2 copyrighted to Gene Sher. >> >> The error message I got which I am having difficulty interpreting is as >> follows.................... >> >> 4> population_monitor:start(). >> {error,{{badrecord,state}, >> [{population_monitor,init,1, >> [{file,"population_monitor.erl"},{line,82}]}, >> {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, >> {proc_lib,init_p_do_apply,3, >> [{file,"proc_lib.erl"},{line,227}]}]}} >> >> I am using the latest versionR15B03 >> >> Thank you for you help >> >> Bob >> Dunedin New Zealand >> >> On 24/12/2012 11:58 a.m., Matti Oinas wrote: >> >> Hi, >> >> My first proposal is to change the editor to something more usable like >> emacs or if you prefer something different then notepad++, sublime text or >> eclipse + erlIDE. Everyone of these are able to show you the line numbers >> and offer syntax highlight features. >> >> It would help a lot if you tell us which code you are looking at. Server >> callbacks probably mean gen_server callbacks so it doesn't matter if you >> are using windows or linux. >> >> Matti >> >> On 12/24/2012 12:08 AM, Bob Matthews wrote: >> >> Hi >> >> I have just started to learn Erlang............ >> I am using Windows 7 on a PC >> >> *Question 1*) when I get an error message, what is the easiest way for >> me to find the exact line number? >> I use ms Word as an editor. >> >> *Question 2*) the code I am looking at was run on Unix, on a server. >> Part of the code talks about server call-backs etc. >> It is unclear how I translate that to a windows PC >> >> Many thanks >> >> Bob >> >> >> _______________________________________________ >> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >> >> >> No virus found in this message. >> Checked by AVG - www.avg.com >> Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: 12/11/12 >> Internal Virus Database is out of date. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> No virus found in this message. >> Checked by AVG - www.avg.com >> Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: 12/11/12 >> Internal Virus Database is out of date. >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > > OJ Reeves > +61 431 952 586 > http://buffered.io/ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From corticalcomputer@REDACTED Mon Dec 24 18:43:56 2012 From: corticalcomputer@REDACTED (Gene Sher) Date: Mon, 24 Dec 2012 12:43:56 -0500 Subject: [erlang-questions] Beginner questions In-Reply-To: References: <50D780F1.2090401@orcon.net.nz> <50D78C7D.4060303@gmail.com> <50D7B092.20009@orcon.net.nz> <50D80326.4030202@orcon.net.nz> <5CD52A56-6662-4DB2-B2FC-B9650514E572@llaisdy.com> Message-ID: Hello All, To answer Reeves' comments: The code is in a fully functional state, I left the start/0 function in there though, for later use. To start the programs without errors it's important to start them with proper parameters. I'll extend the readme file to include these notes. Although a larger update is planned in a while, that will add Hall of fame style selection function, and other modifications improving the system's performance. Same updates will be added to DXNN V1. Some other notes applicable to DXNN1 and 2: There are two ways to go about applying single and multi specie populations to a problem: population_monitor:start(Init_State), executed with default constraints (in which you can define the constraints of the species and population) from the population_monitor:test(). and with: population_monitor:start({OpMode,Population_Id,Selection_Algorithm}). which is executed from: continue(OpMode,Selection_Algorithm)-> Population_Id = test, population_monitor:start({OpMode,Population_Id,Selection_Algorithm}). continue(OpMode,Selection_Algorithm,Population_Id)-> population_monitor:start({OpMode,Population_Id,Selection_Algorithm}). %The function continue/2 and continue/3 are used to summon an already existing population with Population_Id, and continue with the experiment using the Selection_Algorithm. This allows you to continue with the evolution of an existing population of NN based agents, based on the id of the existing population, and some initial parameters you wish to use to continue with the evolutionary run. Granted, you can use population_monitor:start/1 directly, but you will then have to specify all the parameters from the console, that's why the population_monitor:test() was provided (perhaps test was a poor choice for the function's name). For complete experiments, which are usually composed of many evolutionary runs, you would run the benchmark module's functions. These execute the population_monitor:start/1 with appropriate parameters, and also keep track of the resulting evolutionary statistics, and enter these stats into the experiment table, which can be processed to produce GNUPlot files, to graph fitness vs evaluations, diversity vs evaluations and other things. As with a single evolutionary run and the "continue" function, an experiment too can be recovered from a crash, and continue running new evolutionary runs, accumulating the data into the experiment entry with the specified Id. Best regards, -Gene On Mon, Dec 24, 2012 at 12:16 PM, Gene Sher wrote: > Hello all, > > The problem is not with the code, but that population_monitor should not > be started with start(). Population_monitor should be started with the > properly set up constraints. To use the default constraints you start with > population_monitor:test(). > test()-> > init_population(#state{op_mode = [gt,benchmark]},?INIT_CONSTRAINTS). > %The test/0 function starts the population monitor through > init_population/1 with a set of default parameters specified by the macros > of this module. > > prep_PopState(PMP,Specie_Constraints)-> > S=#state{ > op_mode=PMP#pmp.op_mode, > population_id = PMP#pmp.population_id, > survival_percentage=PMP#pmp.survival_percentage, > specie_size_limit=PMP#pmp.specie_size_limit, > init_specie_size=PMP#pmp.init_specie_size, > polis_id=PMP#pmp.polis_id, > generation_limit=PMP#pmp.generation_limit, > evaluations_limit=PMP#pmp.evaluations_limit, > fitness_goal=PMP#pmp.fitness_goal, > benchmarker_pid=PMP#pmp.benchmarker_pid > }, > init_population(S,Specie_Constraints). > > init_population(Init_State,Specie_Constraints)-> > random:seed(now()), > Population_Id = Init_State#state.population_id, > %OpMode = Init_State#state.op_mode, > F = fun()-> > case genotype:read({population,Population_Id}) of > undefined -> > > create_Population(Population_Id,Init_State#state.init_specie_size,Specie_Constraints); > _ -> > delete_population(Population_Id), > > create_Population(Population_Id,Init_State#state.init_specie_size,Specie_Constraints) > end > end, > Result = mnesia:transaction(F), > case Result of > {atomic,_} -> > population_monitor:start(Init_State); > Error -> > io:format("******** ERROR in PopulationMonitor:~p~n",[Error]) > end. > %The function init_population/1 creates a new population with the id > Population_Id, composed of length(Specie_Constraints) species, where each > specie uses the particular specie constraint specified within the > Specie_Constraints list. The function first checks if a population with the > noted Population_Id already exists, if a population does exist, then the > function first delets it, and then creates a fresh one. Since the ids are > usually generated with the genotype:create_UniqueId/0, the only way an > already existing Population_Id is dropped into the function as a parameter > is if it is intended, usually when runing tests, with the Population_Id = > test. > > You will note that population_monitor;start(Init_state) is the only way to > properly start the population monitor. I did not wish to remove the > function population_monitor:start() from the module, for the sake of future > extensions. But there are no problems with the code. > > Best regards, > -Gene > > > On Mon, Dec 24, 2012 at 4:35 AM, OJ Reeves wrote: > >> All, >> >> The first issue is that the init() function isn't correct for this >> gen_server. >> >> On line 83, the init function starts like this: >> init(S) -> >> When it should (apparently) be like this: >> init([S]) -> >> >> This doesn't make the server run though. The next error states: >> >> 6> population_monitor:start(). >> ******** Population monitor started with parameters:{state, >> [gt], >> test,[],[],undefined, >> >> undefined,undefined,[],0, >> 0,0,0,0,undefined, >> undefined,undefined, >> undefined,undefined, >> undefined,0.5,10,10, >> >> mathema,100,100000,inf, >> undefined,false} >> {error,{{badrecord,population}, >> [{population_monitor,extract_AgentIds,2, >> >> [{file,"population_monitor.erl"},{line,372}]}, >> {population_monitor,init,1, >> [{file,"population_monitor.erl"},{line,89}]}, >> {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, >> {proc_lib,init_p_do_apply,3, >> [{file,"proc_lib.erl"},{line,227}]}]}} >> >> I get the feeling that there's going to be more stuff requiring fixes as >> you go. Perhaps the best thing its to get in touch with the author directly >> and find out about the state of the source base before continuing. >> >> Cheers >> OJ >> >> >> On Mon, Dec 24, 2012 at 5:49 PM, Ivan Uemlianin wrote: >> >>> Dear Bob >>> >>> This line should be part of the function population_monitor:init/1 >>> (that's file population_monitor, function init, which takes one argument). >>> >>> The error it complains of is {badrecord,state}, so perhaps the record S >>> has been badly set up. >>> >>> Are you learning erlang through neuroevolution :) ? Do you have other >>> reference resources? >>> >>> Nadolig Llawen >>> >>> Ivan >>> >>> >>> -- >>> festina lente >>> >>> >>> On 24 Dec 2012, at 07:24, Bob Matthews wrote: >>> >>> Thank you Ivan >>> >>> line 82: Population_Id = S#state.population_id, >>> >>> Merry Xmas to you too :) >>> >>> Bob >>> Dunedin >>> New Zealand >>> >>> On 24/12/2012 8:14 p.m., Ivan Uemlianin wrote: >>> >>> I dont have my computer with me but the error is at line 82 of the file >>> population_monitor.erl. >>> >>> Best wishes >>> >>> Ivan >>> >>> -- >>> festina lente >>> >>> >>> On 24 Dec 2012, at 01:32, Bob Matthews wrote: >>> >>> The code that I am looking at is DXNN2 copyrighted to Gene Sher. >>> >>> The error message I got which I am having difficulty interpreting is as >>> follows.................... >>> >>> 4> population_monitor:start(). >>> {error,{{badrecord,state}, >>> [{population_monitor,init,1, >>> >>> [{file,"population_monitor.erl"},{line,82}]}, >>> {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, >>> {proc_lib,init_p_do_apply,3, >>> [{file,"proc_lib.erl"},{line,227}]}]}} >>> >>> I am using the latest versionR15B03 >>> >>> Thank you for you help >>> >>> Bob >>> Dunedin New Zealand >>> >>> On 24/12/2012 11:58 a.m., Matti Oinas wrote: >>> >>> Hi, >>> >>> My first proposal is to change the editor to something more usable like >>> emacs or if you prefer something different then notepad++, sublime text or >>> eclipse + erlIDE. Everyone of these are able to show you the line numbers >>> and offer syntax highlight features. >>> >>> It would help a lot if you tell us which code you are looking at. Server >>> callbacks probably mean gen_server callbacks so it doesn't matter if you >>> are using windows or linux. >>> >>> Matti >>> >>> On 12/24/2012 12:08 AM, Bob Matthews wrote: >>> >>> Hi >>> >>> I have just started to learn Erlang............ >>> I am using Windows 7 on a PC >>> >>> *Question 1*) when I get an error message, what is the easiest way for >>> me to find the exact line number? >>> I use ms Word as an editor. >>> >>> *Question 2*) the code I am looking at was run on Unix, on a server. >>> Part of the code talks about server call-backs etc. >>> It is unclear how I translate that to a windows PC >>> >>> Many thanks >>> >>> Bob >>> >>> >>> _______________________________________________ >>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> No virus found in this message. >>> Checked by AVG - www.avg.com >>> Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: 12/11/12 >>> Internal Virus Database is out of date. >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> No virus found in this message. >>> Checked by AVG - www.avg.com >>> Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: 12/11/12 >>> Internal Virus Database is out of date. >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> -- >> >> OJ Reeves >> +61 431 952 586 >> http://buffered.io/ >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Dec 24 22:09:50 2012 From: ok@REDACTED (ok@REDACTED) Date: Tue, 25 Dec 2012 10:09:50 +1300 Subject: [erlang-questions] STRCMP or something like that In-Reply-To: References: Message-ID: <0cf19089974a3af559fe3dce0a39eac7.squirrel@chasm.otago.ac.nz> String operations are explained in every Erlang book I've seen. It's worth investing in one. The built in comparison operators work _exactly_ like strcmp() -- except for not treating 0 specially. Thus "Haigh" < "Hi", "Hi" < "High", and so on. > Function("alicia", "alex") ---> will return alex+alicia Strings are just lists. List concatenation is ++. Therefore "alicia" ++ "alex" --> "aliciaalex". From oj@REDACTED Mon Dec 24 22:17:08 2012 From: oj@REDACTED (OJ Reeves) Date: Tue, 25 Dec 2012 07:17:08 +1000 Subject: [erlang-questions] Beginner questions In-Reply-To: References: <50D780F1.2090401@orcon.net.nz> <50D78C7D.4060303@gmail.com> <50D7B092.20009@orcon.net.nz> <50D80326.4030202@orcon.net.nz> <5CD52A56-6662-4DB2-B2FC-B9650514E572@llaisdy.com> Message-ID: Gene, Apologies. I was just going off what Bob was attempting to do. I know almost nothing about your work and was just following blindly. I knew having you intervene and put him on the right path was a much better thing to have happen than me speculating. Apologies if I was misleading in any way. All the best and merry Christmas. OJ On 25 Dec, 2012 3:43 AM, "Gene Sher" wrote: > Hello All, > > To answer Reeves' comments: The code is in a fully functional state, I > left the start/0 function in there though, for later use. To start the > programs without errors it's important to start them with proper > parameters. I'll extend the readme file to include these notes. Although a > larger update is planned in a while, that will add Hall of fame style > selection function, and other modifications improving the system's > performance. Same updates will be added to DXNN V1. > > Some other notes applicable to DXNN1 and 2: > > There are two ways to go about applying single and multi specie > populations to a problem: > > population_monitor:start(Init_State), executed with default constraints > (in which you can define the constraints of the species and population) > from the population_monitor:test(). and with: > population_monitor:start({OpMode,Population_Id,Selection_Algorithm}). which > is executed from: > > continue(OpMode,Selection_Algorithm)-> > Population_Id = test, > population_monitor:start({OpMode,Population_Id,Selection_Algorithm}). > continue(OpMode,Selection_Algorithm,Population_Id)-> > population_monitor:start({OpMode,Population_Id,Selection_Algorithm}). > %The function continue/2 and continue/3 are used to summon an already > existing population with Population_Id, and continue with the experiment > using the Selection_Algorithm. > > This allows you to continue with the evolution of an existing population > of NN based agents, based on the id of the existing population, and some > initial parameters you wish to use to continue with the evolutionary run. > > Granted, you can use population_monitor:start/1 directly, but you will > then have to specify all the parameters from the console, that's why the > population_monitor:test() was provided (perhaps test was a poor choice for > the function's name). For complete experiments, which are usually composed > of many evolutionary runs, you would run the benchmark module's functions. > These execute the population_monitor:start/1 with appropriate parameters, > and also keep track of the resulting evolutionary statistics, and enter > these stats into the experiment table, which can be processed to produce > GNUPlot files, to graph fitness vs evaluations, diversity vs evaluations > and other things. As with a single evolutionary run and the "continue" > function, an experiment too can be recovered from a crash, and continue > running new evolutionary runs, accumulating the data into the experiment > entry with the specified Id. > > Best regards, > -Gene > > > On Mon, Dec 24, 2012 at 12:16 PM, Gene Sher wrote: > >> Hello all, >> >> The problem is not with the code, but that population_monitor should not >> be started with start(). Population_monitor should be started with the >> properly set up constraints. To use the default constraints you start with >> population_monitor:test(). >> test()-> >> init_population(#state{op_mode = [gt,benchmark]},?INIT_CONSTRAINTS). >> %The test/0 function starts the population monitor through >> init_population/1 with a set of default parameters specified by the macros >> of this module. >> >> prep_PopState(PMP,Specie_Constraints)-> >> S=#state{ >> op_mode=PMP#pmp.op_mode, >> population_id = PMP#pmp.population_id, >> survival_percentage=PMP#pmp.survival_percentage, >> specie_size_limit=PMP#pmp.specie_size_limit, >> init_specie_size=PMP#pmp.init_specie_size, >> polis_id=PMP#pmp.polis_id, >> generation_limit=PMP#pmp.generation_limit, >> evaluations_limit=PMP#pmp.evaluations_limit, >> fitness_goal=PMP#pmp.fitness_goal, >> benchmarker_pid=PMP#pmp.benchmarker_pid >> }, >> init_population(S,Specie_Constraints). >> >> init_population(Init_State,Specie_Constraints)-> >> random:seed(now()), >> Population_Id = Init_State#state.population_id, >> %OpMode = Init_State#state.op_mode, >> F = fun()-> >> case genotype:read({population,Population_Id}) of >> undefined -> >> >> create_Population(Population_Id,Init_State#state.init_specie_size,Specie_Constraints); >> _ -> >> delete_population(Population_Id), >> >> create_Population(Population_Id,Init_State#state.init_specie_size,Specie_Constraints) >> end >> end, >> Result = mnesia:transaction(F), >> case Result of >> {atomic,_} -> >> population_monitor:start(Init_State); >> Error -> >> io:format("******** ERROR in PopulationMonitor:~p~n",[Error]) >> end. >> %The function init_population/1 creates a new population with the id >> Population_Id, composed of length(Specie_Constraints) species, where each >> specie uses the particular specie constraint specified within the >> Specie_Constraints list. The function first checks if a population with the >> noted Population_Id already exists, if a population does exist, then the >> function first delets it, and then creates a fresh one. Since the ids are >> usually generated with the genotype:create_UniqueId/0, the only way an >> already existing Population_Id is dropped into the function as a parameter >> is if it is intended, usually when runing tests, with the Population_Id = >> test. >> >> You will note that population_monitor;start(Init_state) is the only way >> to properly start the population monitor. I did not wish to remove the >> function population_monitor:start() from the module, for the sake of future >> extensions. But there are no problems with the code. >> >> Best regards, >> -Gene >> >> >> On Mon, Dec 24, 2012 at 4:35 AM, OJ Reeves wrote: >> >>> All, >>> >>> The first issue is that the init() function isn't correct for this >>> gen_server. >>> >>> On line 83, the init function starts like this: >>> init(S) -> >>> When it should (apparently) be like this: >>> init([S]) -> >>> >>> This doesn't make the server run though. The next error states: >>> >>> 6> population_monitor:start(). >>> ******** Population monitor started with parameters:{state, >>> [gt], >>> >>> test,[],[],undefined, >>> >>> undefined,undefined,[],0, >>> 0,0,0,0,undefined, >>> undefined,undefined, >>> undefined,undefined, >>> undefined,0.5,10,10, >>> >>> mathema,100,100000,inf, >>> undefined,false} >>> {error,{{badrecord,population}, >>> [{population_monitor,extract_AgentIds,2, >>> >>> [{file,"population_monitor.erl"},{line,372}]}, >>> {population_monitor,init,1, >>> >>> [{file,"population_monitor.erl"},{line,89}]}, >>> {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, >>> {proc_lib,init_p_do_apply,3, >>> [{file,"proc_lib.erl"},{line,227}]}]}} >>> >>> I get the feeling that there's going to be more stuff requiring fixes as >>> you go. Perhaps the best thing its to get in touch with the author directly >>> and find out about the state of the source base before continuing. >>> >>> Cheers >>> OJ >>> >>> >>> On Mon, Dec 24, 2012 at 5:49 PM, Ivan Uemlianin wrote: >>> >>>> Dear Bob >>>> >>>> This line should be part of the function population_monitor:init/1 >>>> (that's file population_monitor, function init, which takes one argument). >>>> >>>> The error it complains of is {badrecord,state}, so perhaps the record S >>>> has been badly set up. >>>> >>>> Are you learning erlang through neuroevolution :) ? Do you have other >>>> reference resources? >>>> >>>> Nadolig Llawen >>>> >>>> Ivan >>>> >>>> >>>> -- >>>> festina lente >>>> >>>> >>>> On 24 Dec 2012, at 07:24, Bob Matthews wrote: >>>> >>>> Thank you Ivan >>>> >>>> line 82: Population_Id = S#state.population_id, >>>> >>>> Merry Xmas to you too :) >>>> >>>> Bob >>>> Dunedin >>>> New Zealand >>>> >>>> On 24/12/2012 8:14 p.m., Ivan Uemlianin wrote: >>>> >>>> I dont have my computer with me but the error is at line 82 of the file >>>> population_monitor.erl. >>>> >>>> Best wishes >>>> >>>> Ivan >>>> >>>> -- >>>> festina lente >>>> >>>> >>>> On 24 Dec 2012, at 01:32, Bob Matthews wrote: >>>> >>>> The code that I am looking at is DXNN2 copyrighted to Gene Sher. >>>> >>>> The error message I got which I am having difficulty interpreting is as >>>> follows.................... >>>> >>>> 4> population_monitor:start(). >>>> {error,{{badrecord,state}, >>>> [{population_monitor,init,1, >>>> >>>> [{file,"population_monitor.erl"},{line,82}]}, >>>> {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}, >>>> {proc_lib,init_p_do_apply,3, >>>> [{file,"proc_lib.erl"},{line,227}]}]}} >>>> >>>> I am using the latest versionR15B03 >>>> >>>> Thank you for you help >>>> >>>> Bob >>>> Dunedin New Zealand >>>> >>>> On 24/12/2012 11:58 a.m., Matti Oinas wrote: >>>> >>>> Hi, >>>> >>>> My first proposal is to change the editor to something more usable like >>>> emacs or if you prefer something different then notepad++, sublime text or >>>> eclipse + erlIDE. Everyone of these are able to show you the line numbers >>>> and offer syntax highlight features. >>>> >>>> It would help a lot if you tell us which code you are looking at. >>>> Server callbacks probably mean gen_server callbacks so it doesn't matter if >>>> you are using windows or linux. >>>> >>>> Matti >>>> >>>> On 12/24/2012 12:08 AM, Bob Matthews wrote: >>>> >>>> Hi >>>> >>>> I have just started to learn Erlang............ >>>> I am using Windows 7 on a PC >>>> >>>> *Question 1*) when I get an error message, what is the easiest way >>>> for me to find the exact line number? >>>> I use ms Word as an editor. >>>> >>>> *Question 2*) the code I am looking at was run on Unix, on a server. >>>> Part of the code talks about server call-backs etc. >>>> It is unclear how I translate that to a windows PC >>>> >>>> Many thanks >>>> >>>> Bob >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> No virus found in this message. >>>> Checked by AVG - www.avg.com >>>> Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: >>>> 12/11/12 >>>> Internal Virus Database is out of date. >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> No virus found in this message. >>>> Checked by AVG - www.avg.com >>>> Version: 2013.0.2805 / Virus Database: 2634/5951 - Release Date: >>>> 12/11/12 >>>> Internal Virus Database is out of date. >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >>> >>> -- >>> >>> OJ Reeves >>> +61 431 952 586 >>> http://buffered.io/ >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Dec 24 23:42:43 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 24 Dec 2012 23:42:43 +0100 Subject: [erlang-questions] [ANN] Ranch 0.6.0 Xmas Edition Released Message-ID: <50D8DA63.2060600@ninenines.eu> Ho ho ho! I have just tagged version 0.6.0 of the Ranch project! Ranch is a socket acceptor pool for TCP protocols. https://github.com/extend/ranch Ranch is used by the next version of Cowboy, 0.8.0, set to be released early February, but also in Basho's Riak multi-data center replication amongst others. All tickets have been resolved. A significant contribution was made by Andrew Majorov to improve the fault tolerance capabilities of the application, making sure it always restarts properly when things go wrong. This has been made possible thanks to the amazing project from Daniel Luna, chaos_monkey (https://github.com/dluna/chaos_monkey). The guide has also been improved and completed. http://ninenines.eu/docs/en/ranch/HEAD/guide/introduction If the guide isn't enough, drop by our new IRC channel dedicated to Cowboy, Ranch and all our other projects! #ninenines on Freenode. Following is the list of change since last time: * Improve fault tolerance thanks to chaos_monkey testing * Add 'nodelay' option to transports * Add 'verify' option to ranch_ssl transport * Add 'socket' option to pass an already open socket to the listener * Add Transport:sendfile/2 function (uses a fallback if unavailable) * Allow IP tuples in Transport:connect/3 * Add ranch:set_max_connections/2 to update the value live * Add ranch:get_max_connections/1 to retrieve it We are always looking for feedback, especially now that there is no ticket left open on this project. If you are using Ranch and have questions or needs that it doesn't cover, please send them to us. Commercial support will be available starting from January, ping me if you are interested. Details will be announced at a later time on the ninenines.eu mailing list. I want to thank all contributors for helping this project by opening tickets, sending patches and offering feedback. I am as always very grateful for any and all contributions. I wouldn't have made it this far without the tremendous help I receive everyday. Thanks to all and have a nice holiday! -- Lo?c Hoguin Erlang Santa Nine Nines http://ninenines.eu From evrenweb@REDACTED Tue Dec 25 07:24:32 2012 From: evrenweb@REDACTED (Evren Bayraktar) Date: Tue, 25 Dec 2012 08:24:32 +0200 Subject: [erlang-questions] STRCMP or something like that In-Reply-To: References: <50D867F8.9030607@ninenines.eu> Message-ID: <50D946A0.6000407@gmail.com> Te?ekk?rler G?khan abi, daha syntax'a bile al??mam??ken verdi?in bilgiler ?ok yard?mc? olacak. E?er yeni ba?layanlar i?in ?nerebilece?in kaynaklar var ise seve seve incelemek isterim. Linkedin ?zerinde payla?t???n slide'? g?rd?m, erlang beni daha da heyecanland?rd?. Thank you Lo?c , I didn't imagine how erlang works so easy and wonderfull :) I guess, I will fall in love with this language :) On 24-12-2012 16:42, Kunthar wrote: > Evren Pasa > > - Yoluna devam etmeden once Erlang library'lerini iyie bir incelemeni > tavsiye ederim. > > Ne tur seceneklerin oldugunu hizlica anlamak icin, konsolda > lists: > yazarak TAB'e basarsan lists modulu ile tanimlanmis tum fonksiyonlari > gorebilirsin. > > - eger hic bir sey yazmadan TAB'e basarsan, derlenmis tum modulleri gorursun. > > kolay gelsin. > > > $ kunthar : erl > Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:4:4] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.9.2 (abort with ^G) > 1> lists: > all/2 any/2 append/1 append/2 concat/1 > delete/2 dropwhile/2 duplicate/2 filter/2 flatlength/1 > flatmap/2 flatten/1 flatten/2 foldl/3 foldr/3 > foreach/2 keydelete/3 keyfind/3 keymap/3 keymember/3 > keymerge/3 keyreplace/4 keysearch/3 keysort/2 keystore/4 > keytake/3 last/1 map/2 mapfoldl/3 mapfoldr/3 > max/1 member/2 merge/1 merge/2 merge/3 > merge3/3 min/1 module_info/0 module_info/1 nth/2 > nthtail/2 partition/2 prefix/2 reverse/1 reverse/2 > rkeymerge/3 rmerge/2 rmerge/3 rmerge3/3 rukeymerge/3 > rumerge/2 rumerge/3 rumerge3/3 seq/2 seq/3 > sort/1 sort/2 split/2 splitwith/2 sublist/2 > sublist/3 subtract/2 suffix/2 sum/1 takewhile/2 > ukeymerge/3 ukeysort/2 umerge/1 umerge/2 umerge/3 > umerge3/3 unzip/1 unzip3/1 usort/1 usort/2 > zf/2 zip/2 zip3/3 zipwith/3 zipwith3/4 > > 1> lists: > > > > On Mon, Dec 24, 2012 at 4:34 PM, Lo?c Hoguin wrote: >> 1> lists:sort(["alicia", "alex"]). >> ["alex","alicia"] >> >> :) >> >> >> On 12/24/2012 03:28 PM, Evren Bayraktar wrote: >>> Hi everyone, >>> >>> I'm having trouble with Erlang I hope someone can help me. I'm really >>> newbie in Erlang :) >>> >>> I need to something like strcmp in php. >>> >>> For example; >>> >>> I have a two different string, Alicia and Alex. I need to steady order >>> for them. By parsing each letter I guess I can do that. >>> >>> $A. returns 65 so I can't compare with first letters then $l. returns >>> 108 but they are same too. >>> Now, $e. returns 101 and $i. returns 105. By using this numbers I can >>> give correct order, first Alex then Alica. >>> >>> I mean, imagine a function which compare two different variable, by >>> using their ascii codes it will return >>> correct order. >>> >>> Function("alicia", "alex") ---> will return alex+alicia >>> >>> I hope someone will help me >>> >>> Best Regards >>> Evren >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From norton@REDACTED Tue Dec 25 12:14:35 2012 From: norton@REDACTED (=?utf-8?B?44OO44O844OI44OzIOOCuOODp+ODvOOCu+ODlSDjgqbjgqfjgqQg?= =?utf-8?B?44Oz?=) Date: Tue, 25 Dec 2012 20:14:35 +0900 Subject: [erlang-questions] parsetools 2.0.7 esyntax - Call to missing or unexported function erl_parse:*/* Message-ID: <56626C55-019E-4E8E-9A7D-6314A95CA366@lovely.email.ne.jp> Hello. I stumbled upon some missing functions inside the syntax.yrl file of the parsetools application. These can be found by dialyzer. $ git log src/esyntax.yrl commit 8bc17e7dde4413e3f24f533b867e34ba951ebcff Author: Erlang/OTP Date: Fri Nov 20 14:54:40 2009 +0000 The R13B03 release. esyntax.yrl:164: Call to missing or unexported function erl_parse:erlang_bif/2 esyntax.yrl:202: Call to missing or unexported function erl_parse:is_term/1 esyntax.yrl:271: Call to missing or unexported function erl_parse:erlang_guard_bif/2 esyntax.yrl:283: Call to missing or unexported function erl_parse:erlang_guard_test/2 What is the purpose of this file? Should these errors be fixed? ignored? etc? thanks, Joe N. p.s. FYI. I fixed a few dialyzer warnings for leex and yecc. I'll post a patch for these separately after further testing and review. https://github.com/norton/parsetools/commit/23d333797fef023b6cee04a757cb07d46136d0c8 From jvalduvieco@REDACTED Wed Dec 26 01:19:56 2012 From: jvalduvieco@REDACTED (Joan Valduvieco Llopart) Date: Wed, 26 Dec 2012 01:19:56 +0100 Subject: [erlang-questions] process manager, R16, and 64-bit Macs In-Reply-To: References: <50BE791E.1050202@simonstl.com> <50BF1EA5.3040201@erlang.org> <50BF30A6.3030608@erlang.org> <50BF33FC.5080608@ninenines.eu> <50CB93B8.1070203@ninenines.eu> Message-ID: Hi all, I managed to add wxgtk Formula to Homebrew and to make some modifications to erlang Formula allowing Erlang GUI tools to work on Mountain Lion. You can find my changes here: https://github.com/jvalduvieco/homebrew/compare/master...erlang-wxgtk gists with complete files: https://gist.github.com/4376548 https://gist.github.com/4376555 If you want to use these formula just replace original Homebrew formula (in /usr/local/Library/Formula/) with mine and run: brew uninstall erlang brew install -vd --with-wxgtk erlang I'll submit a pull request to mxcl as soon as I get some positive feedback. Now it just works for me. Thanks, Joan 2012/12/18 Joan Valduvieco Llopart > Sure! > I'll try. Installing brew right now.. ;) > > Joan > > > 2012/12/14 Lo?c Hoguin > >> On 12/05/2012 01:17 PM, Joan Valduvieco Llopart wrote: >> >>> I am using observer with mountain lion with wxgtk + xquartz without a >>> glitch. :) >>> I have a macports repo with my packages. >>> https://github.com/**jvalduvieco/macports >>> >> >> Any chance you could port this to homebrew? From what I hear most people >> who use Macs use homebrew so you doing this would be a tremendous help for >> the community and for the OTP team. >> >> -- >> Lo?c Hoguin >> >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ababkin@REDACTED Wed Dec 26 07:18:38 2012 From: ababkin@REDACTED (Alex Babkin) Date: Wed, 26 Dec 2012 01:18:38 -0500 Subject: [erlang-questions] suggestions for implementing sftp server? In-Reply-To: References: <20121221052819.GA54096@zmac> Message-ID: Hi all Made some progress towards using sftpd and sftp but hitting this problem: I can read and write (get and put) files using erlang sftp library, but can only read (get) files from the erlang sftpd daemon if i do this with some other sftp software (i tried filezilla and the standard sftp cli that comes on mac osx). If i try to put file, i get this: sftp> put /tmp/erlang/test3.txt Uploading /tmp/erlang/test3.txt to /test3.txt remote open("/test3.txt"): Failure directory permissions should be fine since i can still upload files using erlang sftp lib the code is shared here for anyone who would like to take a look: https://github.com/ababkin/hl7_pipeline Cheers On Fri, Dec 21, 2012 at 3:51 PM, JD Bothma wrote: > You probably also already found these, but they're handy for reference > > https://github.com/erlang/otp/blob/maint/lib/ssh/src/ssh_sftpd_file.erl > https://github.com/erlang/otp/blob/maint/lib/ssh/src/ssh_sftpd_file_api.erl > > > On 21 December 2012 18:03, Alex Babkin wrote: > >> Thanks Alexander, this was very helpful >> >> >> On Fri, Dec 21, 2012 at 12:28 AM, Alexander Zhuravlev < >> a.zhuravlev@REDACTED> wrote: >> >>> ssh_sftpd >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Dec 26 16:23:08 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 26 Dec 2012 07:23:08 -0800 (PST) Subject: [erlang-questions] =?windows-1252?q?Cowboy=3A_init_and_handle=A0?= =?windows-1252?q?=97_how_to_use_them=3F?= Message-ID: Loic, would you kindly explain what are the exact rules to split code between init/3 and handle/2? Frankly speaking, I never use terminate, because I also doesn't understand what is it for, but most interesting is the question about init and handle. Where should I put database select, for example? From essen@REDACTED Wed Dec 26 16:48:20 2012 From: essen@REDACTED (=?windows-1252?Q?Lo=EFc_Hoguin?=) Date: Wed, 26 Dec 2012 16:48:20 +0100 Subject: [erlang-questions] =?windows-1252?q?Cowboy=3A_init_and_handle=A0?= =?windows-1252?q?=97_how_to_use_them=3F?= In-Reply-To: References: Message-ID: <50DB1C44.2090508@ninenines.eu> On 12/26/2012 04:23 PM, Max Lapshin wrote: > Loic, would you kindly explain what are the exact rules to split code > between init/3 and handle/2? > > Frankly speaking, I never use terminate, because I also doesn't > understand what is it for, but most interesting is the question about > init and handle. > > Where should I put database select, for example? In handle. It works like a gen_server with few differences. In init you would initialize the state if needed (plain HTTP requests may not need it, other kinds probably do), check that the request looks good and inform Cowboy what kind of handler this is. In handle you do whatever you need to do with the request. In terminate you cleanup. Like with gen_server, terminate will rarely do anything because most of the time when you need to clean things up you just have monitors reporting this process is 'DOWN' and do the cleanup there instead. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From max.lapshin@REDACTED Wed Dec 26 17:03:09 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 26 Dec 2012 19:03:09 +0300 Subject: [erlang-questions] =?utf-8?q?Cowboy=3A_init_and_handle_=E2=80=94_?= =?utf-8?q?how_to_use_them=3F?= In-Reply-To: <50DB1C44.2090508@ninenines.eu> References: <50DB1C44.2090508@ninenines.eu> Message-ID: I'm looking at https://github.com/extend/cowboy/blob/master/examples/rest_hello_world/src/toppage_handler.erland it looks like all logic should be done in rest_init with rendering in specific handlers. On Wed, Dec 26, 2012 at 7:48 PM, Lo?c Hoguin wrote: > On 12/26/2012 04:23 PM, Max Lapshin wrote: > >> Loic, would you kindly explain what are the exact rules to split code >> between init/3 and handle/2? >> >> Frankly speaking, I never use terminate, because I also doesn't >> understand what is it for, but most interesting is the question about >> init and handle. >> >> Where should I put database select, for example? >> > > In handle. > > It works like a gen_server with few differences. In init you would > initialize the state if needed (plain HTTP requests may not need it, other > kinds probably do), check that the request looks good and inform Cowboy > what kind of handler this is. In handle you do whatever you need to do with > the request. In terminate you cleanup. Like with gen_server, terminate will > rarely do anything because most of the time when you need to clean things > up you just have monitors reporting this process is 'DOWN' and do the > cleanup there instead. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Dec 26 17:06:46 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 26 Dec 2012 17:06:46 +0100 Subject: [erlang-questions] =?utf-8?q?Cowboy=3A_init_and_handle_=E2=80=94_?= =?utf-8?q?how_to_use_them=3F?= In-Reply-To: References: <50DB1C44.2090508@ninenines.eu> Message-ID: <50DB2096.7030106@ninenines.eu> REST handlers are different, these work like Webmachine, and the logic is spread over many different functions. On 12/26/2012 05:03 PM, Max Lapshin wrote: > I'm looking at > https://github.com/extend/cowboy/blob/master/examples/rest_hello_world/src/toppage_handler.erl > and it looks like all logic should be done in rest_init with rendering > in specific handlers. > > > > > On Wed, Dec 26, 2012 at 7:48 PM, Lo?c Hoguin > wrote: > > On 12/26/2012 04:23 PM, Max Lapshin wrote: > > Loic, would you kindly explain what are the exact rules to split > code > between init/3 and handle/2? > > Frankly speaking, I never use terminate, because I also doesn't > understand what is it for, but most interesting is the question > about > init and handle. > > Where should I put database select, for example? > > > In handle. > > It works like a gen_server with few differences. In init you would > initialize the state if needed (plain HTTP requests may not need it, > other kinds probably do), check that the request looks good and > inform Cowboy what kind of handler this is. In handle you do > whatever you need to do with the request. In terminate you cleanup. > Like with gen_server, terminate will rarely do anything because most > of the time when you need to clean things up you just have monitors > reporting this process is 'DOWN' and do the cleanup there instead. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From max.lapshin@REDACTED Wed Dec 26 17:21:06 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 26 Dec 2012 19:21:06 +0300 Subject: [erlang-questions] Why system_flag(scheduler_bind_type) is deprecated? Message-ID: Why system_flag(scheduler_bind_type, How) is deprecated in favor of +sbt flag? This sbt flag is different when I have to launch via escript or via erl and this is why it is less convenient than using system_flag call. Also system_flag call can be called according to some system configuration file and sbt needs full restart. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Dec 26 17:46:57 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 26 Dec 2012 08:46:57 -0800 (PST) Subject: [erlang-questions] Why system_flag(scheduler_bind_type) is deprecated? In-Reply-To: References: Message-ID: <6072f3ef-54a4-4e83-bebe-62186d18cbc2@17g2000vba.googlegroups.com> Mac OS X: $ erl +sbt s setting scheduler bind type 's' failed: not supported Usage: beam.smp [flags] [ -- [init_args] ] ... $ erl -sbt s Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:4:4] [async-threads: 0] [hipe] [kernel-poll:false] [dtrace] 2> erlang:system_info(scheduler_bindings). {unbound,unbound,unbound,unbound} Linux: $ erl +sbt s Erlang R15B (erts-5.9) [source] [64-bit] [smp:8:8] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9 (abort with ^G) 1> erlang:system_info(scheduler_bindings). {0,1,2,3,4,5,6,7} Setting this flag +sbt doesn't allow me to launch code under mac os. How should I change system_flag/2 call to this flag? From ababkin@REDACTED Wed Dec 26 20:39:43 2012 From: ababkin@REDACTED (Alex Babkin) Date: Wed, 26 Dec 2012 14:39:43 -0500 Subject: [erlang-questions] suggestions for implementing sftp server? In-Reply-To: References: <20121221052819.GA54096@zmac> Message-ID: nevermind! tried running daemon on another system and can write files now (must be something with create file permissions, or somesuch) cheers On Wed, Dec 26, 2012 at 1:18 AM, Alex Babkin wrote: > Hi all > > Made some progress towards using sftpd and sftp but hitting this problem: > I can read and write (get and put) files using erlang sftp library, but can > only read (get) files from the erlang sftpd daemon if i do this with some > other sftp software (i tried filezilla and the standard sftp cli that comes > on mac osx). If i try to put file, i get this: > > sftp> put /tmp/erlang/test3.txt > Uploading /tmp/erlang/test3.txt to /test3.txt > remote open("/test3.txt"): Failure > > directory permissions should be fine since i can still upload files using > erlang sftp lib > > the code is shared here for anyone who would like to take a look: > https://github.com/ababkin/hl7_pipeline > > > Cheers > > > > On Fri, Dec 21, 2012 at 3:51 PM, JD Bothma wrote: > >> You probably also already found these, but they're handy for reference >> >> https://github.com/erlang/otp/blob/maint/lib/ssh/src/ssh_sftpd_file.erl >> >> https://github.com/erlang/otp/blob/maint/lib/ssh/src/ssh_sftpd_file_api.erl >> >> >> On 21 December 2012 18:03, Alex Babkin wrote: >> >>> Thanks Alexander, this was very helpful >>> >>> >>> On Fri, Dec 21, 2012 at 12:28 AM, Alexander Zhuravlev < >>> a.zhuravlev@REDACTED> wrote: >>> >>>> ssh_sftpd >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Wed Dec 26 21:37:19 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 26 Dec 2012 21:37:19 +0100 Subject: [erlang-questions] Running multiple commands like os:cmd/1 Message-ID: Happy holidays, everyone! I want to run multiple commands in a single shell from my code through some sort of open_port({spawn, ...}). I have four requirements: 1) I want it to work asynchronously (i.e. I start a command and I get it's results streamed to a process as it happens); 2) I want to know exactly when a command finishes; 3) Binary output should be handled properly; 4) Changes in env variables should be preserved between commands. A naive sequence of os:cmd/1 or open_port({spawn, ...}) doesn't satisfy requirement 4. os:cmd/1 has a nice backend code in there, but it doesn't satisfy requirement 3. It has a bug. Try running length(os:cmd("head -c 1000 /dev/urandom")). open_port({spawn, "/bin/sh -s -"}, [line]) with port_command/2 doesn't satisfy requirement 2. Any ideas how to implement it? Is there any library which does it? Cheers, Gleb From ababkin@REDACTED Thu Dec 27 00:05:17 2012 From: ababkin@REDACTED (Alex Babkin) Date: Wed, 26 Dec 2012 18:05:17 -0500 Subject: [erlang-questions] strangeness with sftpd opening files for write Message-ID: Hi I'm experimenting with using ssh:daemon using the ssh_sftpd subsystem (other words the standard erlang/otp sftp daemon) and i'm seeing some strange and inconsistent behavior: (all code is here: https://github.com/ababkin/hl7_pipeline) when i run the sftpd daemon and try to connect to it with an external sftp client (standard sftp cli on osx lion) to 'put' a file, it seems to try to open a new file for write with these modes/flags: [binary,write,creat,trunc], which results in error. I intercept these in my custom file_handler for sftpd (essentially just log the arguments in the open callback) Doing manually in erl shell: > Result = file:open("/tmp/da.txt", [binary, write, creat, trunc]). results in: {error,badarg} so it looks like file:open doesn't like 'creat' or 'trunc' flags that sftpd tries to pass to it this is from the default file_handler sftpd uses: open(Path, Flags, State) -> {file:open(Path, Flags), State}. .. i.e just passes the flags as modes to file:open these results are further compounded with inconsistent behavior while running on different systems i've tried running sftpd on a mac (lion) and ubuntu, seeing the above results, but trying to run the sftpd on a DS211j (that uses marvell ARM cpu and a version of busybox distro and R15B01 probably somehow cross-compiled specifically for the ARM) is see a different behavior: on the DS211j, if i try to perform same steps as on above systems, the flags i intercept are [raw,binary,write], which writes the file fine and doesn't throw an error like in above cases Tried to run the otp testsuite, but seem to be failing on testsuite build step having to do with hipe Any insights would be welcome. For now i just overwrite the collection of modes/flags with [raw,binary,write] whenever there is 'write' present in the list, but this seems hacky -------------- next part -------------- An HTML attachment was scrubbed... URL: From alkondratenko@REDACTED Thu Dec 27 00:28:21 2012 From: alkondratenko@REDACTED (Aliaksey Kandratsenka) Date: Wed, 26 Dec 2012 15:28:21 -0800 Subject: [erlang-questions] Dialyzer and improper lists Message-ID: Hi. It appears dialyzer complains loudly about any code that builds improper lists. And there appears to be no local way to silence that warning. It's true that sometimes it's a sign of bug. But on the other hand dialyzer docs saying there's no reason to use improper lists (even in ALL CAPS) is clearly wrong. According to my understanding improper list of size 2 needs just 2 words (single cons pair and no headers), but binary tuple needs one extra word for header. And erlang's stdlib has a bunch of places where cons pair is used as a binary tuple. E.g. dict. While docs should not necessarily point folks to low-level tricks like that, they IMHO should be fixed to reflect reality. But IMHO most importantly, there should be some way to say to dialyzer "hey I really mean improper list here" without disabling improper list check completely. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rickard@REDACTED Thu Dec 27 00:45:05 2012 From: rickard@REDACTED (Rickard Green) Date: Thu, 27 Dec 2012 00:45:05 +0100 Subject: [erlang-questions] Why system_flag(scheduler_bind_type) is deprecated? In-Reply-To: References: Message-ID: <20A107A4-ADA5-46F0-A473-FF9418DC211A@erlang.org> On Dec 26, 2012, at 5:21 PM, Max Lapshin wrote: > Why system_flag(scheduler_bind_type, How) is deprecated in favor of +sbt flag? > > This sbt flag is different when I have to launch via escript or via erl and this is why it is less convenient than using system_flag call. > > Also system_flag call can be called according to some system configuration file and sbt needs full restart. > Both the 'cpu_topology' and the 'scheduler_bind_type' arguments of the system_flag/2 BIF are deprecated and will be removed since we do not want to change this configuration in runtime. With the support for this we got today the runtime configuration change isn't that problematic, but it prevents future planned improvements from being implemented. Regards, Rickard Green, Erlang/OTP, Ericsson AB From rickard@REDACTED Thu Dec 27 00:45:54 2012 From: rickard@REDACTED (Rickard Green) Date: Thu, 27 Dec 2012 00:45:54 +0100 Subject: [erlang-questions] Why system_flag(scheduler_bind_type) is deprecated? In-Reply-To: <6072f3ef-54a4-4e83-bebe-62186d18cbc2@17g2000vba.googlegroups.com> References: <6072f3ef-54a4-4e83-bebe-62186d18cbc2@17g2000vba.googlegroups.com> Message-ID: <373FD3CE-B816-45AF-A97C-F1F0E3C56586@erlang.org> On Dec 26, 2012, at 5:46 PM, Max Lapshin wrote: > Mac OS X: > > $ erl +sbt s > setting scheduler bind type 's' failed: not supported > Usage: beam.smp [flags] [ -- [init_args] ] > ... > $ erl -sbt s > Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:4:4] [async-threads: > 0] [hipe] [kernel-poll:false] [dtrace] > 2> erlang:system_info(scheduler_bindings). > {unbound,unbound,unbound,unbound} > > > > Linux: > > $ erl +sbt s > Erlang R15B (erts-5.9) [source] [64-bit] [smp:8:8] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.9 (abort with ^G) > 1> erlang:system_info(scheduler_bindings). > {0,1,2,3,4,5,6,7} > > > Setting this flag +sbt doesn't allow me to launch code under mac os. > How should I change system_flag/2 call to this flag? Neither the +sbt flag nor the 'scheduler_bind_type' argument of the system_flag/2 BIF have ever been supported on MacOSX. This since the last time I tried to find an api for binding threads to processors on MacOSX I couldn't find one. This might however have changed since then. It was a long time ago I checked. If you can find an api making it possible to implement this on MacOSX, a patch implementing this is welcome. Regards, Rickard Green, Erlang/OTP, Ericsson AB From ababkin@REDACTED Thu Dec 27 00:56:22 2012 From: ababkin@REDACTED (Alex Babkin) Date: Wed, 26 Dec 2012 18:56:22 -0500 Subject: [erlang-questions] strangeness with sftpd opening files for write In-Reply-To: References: Message-ID: seems like my issue has to do with this bit of (unfinished? see the comment) code. Seems that the code is 3 years old. I have a hunch that the difference in behavior i see is due to the Vsn difference. Any way to alter what's read as Vsn anywhere? (i presume it's a version of some sort) open(Vsn, ReqId, Data, State) when Vsn =< 3 -> <> = Data, Path = binary_to_list(BPath), Flags = ssh_xfer:decode_open_flags(Vsn, PFlags), do_open(ReqId, State, Path, Flags); open(Vsn, ReqId, Data, State) when Vsn >= 4 -> <> = Data, Path = binary_to_list(BPath), FlagBits = ssh_xfer:decode_open_flags(Vsn, PFlags), AcessBits = ssh_xfer:decode_ace_mask(Access), %% TODO: This is to make sure the Access flags are not ignored %% but this should be thought through better. This solution should %% be considered a hack in order to buy some time. At least %% it works better than when the Access flags where totally ignored. %% A better solution may need some code refactoring that we do %% not have time for right now. AcessFlags = decode_4_acess(AcessBits), Flags = lists:append(lists:umerge( [[decode_4_flags(FlagBits)] | AcessFlags])), do_open(ReqId, State, Path, Flags). On Wed, Dec 26, 2012 at 6:05 PM, Alex Babkin wrote: > Hi > > I'm experimenting with using ssh:daemon using the ssh_sftpd subsystem > (other words the standard erlang/otp sftp daemon) and i'm seeing some > strange and inconsistent behavior: > > (all code is here: https://github.com/ababkin/hl7_pipeline) > > when i run the sftpd daemon and try to connect to it with an external sftp > client (standard sftp cli on osx lion) to 'put' a file, it seems to try to > open a new file for write with these modes/flags: > [binary,write,creat,trunc], which results in error. > > I intercept these in my custom file_handler for sftpd (essentially just > log the arguments in the open callback) > > Doing manually in erl shell: > > Result = file:open("/tmp/da.txt", [binary, write, creat, trunc]). > results in: > {error,badarg} > > so it looks like file:open doesn't like 'creat' or 'trunc' flags that > sftpd tries to pass to it > > this is from the default file_handler sftpd uses: > > open(Path, Flags, State) -> > {file:open(Path, Flags), State}. > > > .. i.e just passes the flags as modes to file:open > > these results are further compounded with inconsistent behavior while > running on different systems > > i've tried running sftpd on a mac (lion) and ubuntu, seeing the above > results, but trying to run the sftpd on a DS211j (that uses marvell ARM cpu > and a version of busybox distro and R15B01 probably somehow cross-compiled > specifically for the ARM) is see a different behavior: > > on the DS211j, if i try to perform same steps as on above systems, the > flags i intercept are [raw,binary,write], which writes the file fine and > doesn't throw an error like in above cases > > Tried to run the otp testsuite, but seem to be failing on testsuite build > step having to do with hipe > > Any insights would be welcome. > For now i just overwrite the collection of modes/flags > with [raw,binary,write] whenever there is 'write' present in the list, but > this seems hacky > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Thu Dec 27 01:25:48 2012 From: siraaj@REDACTED (Siraaj Khandkar) Date: Wed, 26 Dec 2012 19:25:48 -0500 Subject: [erlang-questions] Running multiple commands like os:cmd/1 In-Reply-To: References: Message-ID: <5E7C8E8E-CE40-46AB-8D93-01EAC2A66759@khandkar.net> On Dec 26, 2012, at 3:37 PM, Gleb Peregud wrote: > Happy holidays, everyone! > > I want to run multiple commands in a single shell from my code through > some sort of open_port({spawn, ...}). I have four requirements: Why does it have to be a single shell? What is wrong with multiple ports? > 1) I want it to work asynchronously (i.e. I start a command and I get > it's results streamed to a process as it happens); If it really has to be a single shell, you can background each command and you can tell when each will finish, but I don't know how would you distinguish which output came from where. > 2) I want to know exactly when a command finishes; > 3) Binary output should be handled properly; > 4) Changes in env variables should be preserved between commands. > > A naive sequence of os:cmd/1 or open_port({spawn, ...}) doesn't > satisfy requirement 4. > > os:cmd/1 has a nice backend code in there, but it doesn't satisfy > requirement 3. It has a bug. Try running length(os:cmd("head -c 1000 > /dev/urandom")). > > open_port({spawn, "/bin/sh -s -"}, [line]) with port_command/2 doesn't > satisfy requirement 2. How about open_port with 'exit_status' and call erlang:now/0 when receiving {Port, {exit_status, ExitCode}} messages? -- Siraaj Khandkar .o. ..o ooo From max.lapshin@REDACTED Thu Dec 27 08:45:22 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 27 Dec 2012 10:45:22 +0300 Subject: [erlang-questions] Why system_flag(scheduler_bind_type) is deprecated? In-Reply-To: <373FD3CE-B816-45AF-A97C-F1F0E3C56586@erlang.org> References: <6072f3ef-54a4-4e83-bebe-62186d18cbc2@17g2000vba.googlegroups.com> <373FD3CE-B816-45AF-A97C-F1F0E3C56586@erlang.org> Message-ID: So, I need to change shell script that launches application that will check if it is Linux or not and add or not add +sbt flag? -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Thu Dec 27 12:08:40 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 27 Dec 2012 12:08:40 +0100 Subject: [erlang-questions] Running multiple commands like os:cmd/1 In-Reply-To: <5E7C8E8E-CE40-46AB-8D93-01EAC2A66759@khandkar.net> References: <5E7C8E8E-CE40-46AB-8D93-01EAC2A66759@khandkar.net> Message-ID: On Thu, Dec 27, 2012 at 1:25 AM, Siraaj Khandkar wrote: > Why does it have to be a single shell? What is wrong with multiple ports? It's because I want to preserve changes in env variables between calls (see requirement 4), since I want to be able to run commands like ". /opt/erlang/r15b/activate" to prepare environment for further commands. Think of a sequence of commands run by CI server. >> 1) I want it to work asynchronously (i.e. I start a command and I get >> it's results streamed to a process as it happens); > > If it really has to be a single shell, you can background each command > and you can tell when each will finish, but I don't know how would you > distinguish which output came from where. It's exactly the problem I am facing. >> open_port({spawn, "/bin/sh -s -"}, [line]) with port_command/2 doesn't >> satisfy requirement 2. > > How about open_port with 'exit_status' and call erlang:now/0 when receiving > {Port, {exit_status, ExitCode}} messages? One open_port can produce just one exit_status message. Not sure what you mean here. From gleber.p@REDACTED Thu Dec 27 12:30:46 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 27 Dec 2012 12:30:46 +0100 Subject: [erlang-questions] Running multiple commands like os:cmd/1 In-Reply-To: <684C6906-7017-4796-9817-30FA3E6932F8@licenser.net> References: <5E7C8E8E-CE40-46AB-8D93-01EAC2A66759@khandkar.net> <684C6906-7017-4796-9817-30FA3E6932F8@licenser.net> Message-ID: This is a reasonable solution (similar to one I have implemented yesterday here [1]), although there is still a very-very-very low risk that when running "cat /dev/urandom; echo " given UUID will be written twice and things will go haywire. I wonder if it is at all possible to make it 100% robust. 1: https://github.com/gleber/sh/blob/master/src/runner.erl#L66 On Thu, Dec 27, 2012 at 12:21 PM, Heinz Nikolaus Gies wrote: > open a shell via open_port and run commands with something like > ; echo ; > then you know the command ended when the UUID is outputted? > > Just a crazy idea :) > On Dec 27, 2012, at 12:08, Gleb Peregud wrote: > >> On Thu, Dec 27, 2012 at 1:25 AM, Siraaj Khandkar wrote: >>> Why does it have to be a single shell? What is wrong with multiple ports? >> >> It's because I want to preserve changes in env variables between calls >> (see requirement 4), since I want to be able to run commands like ". >> /opt/erlang/r15b/activate" to prepare environment for further >> commands. Think of a sequence of commands run by CI server. >> >>>> 1) I want it to work asynchronously (i.e. I start a command and I get >>>> it's results streamed to a process as it happens); >>> >>> If it really has to be a single shell, you can background each command >>> and you can tell when each will finish, but I don't know how would you >>> distinguish which output came from where. >> >> It's exactly the problem I am facing. >> >>>> open_port({spawn, "/bin/sh -s -"}, [line]) with port_command/2 doesn't >>>> satisfy requirement 2. >>> >>> How about open_port with 'exit_status' and call erlang:now/0 when receiving >>> {Port, {exit_status, ExitCode}} messages? >> >> One open_port can produce just one exit_status message. Not sure what >> you mean here. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From attila.r.nohl@REDACTED Thu Dec 27 13:22:42 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 27 Dec 2012 13:22:42 +0100 Subject: [erlang-questions] strangeness with sftpd opening files for write In-Reply-To: References: Message-ID: Hello! This is the code segment we use: open(Path0, Flags0, State) -> Flags = Flags0 -- [creat, excl, trunc], [...] Result = file:open(Path, Flags), I don't remember the details, but this might have something to do with the SFTP versions. As far as I know, SFTP was never standardized in an RFC, but there are I think 6 protocol versions out there documented in at least 12 drafts. Different clients use different versions, that could explain why you get different results from different clients. We actually stick to version 3 on the server side, we had problems with never versions. 2012/12/27 Alex Babkin : > Hi > > I'm experimenting with using ssh:daemon using the ssh_sftpd subsystem (other > words the standard erlang/otp sftp daemon) and i'm seeing some strange and > inconsistent behavior: > > (all code is here: https://github.com/ababkin/hl7_pipeline) > > when i run the sftpd daemon and try to connect to it with an external sftp > client (standard sftp cli on osx lion) to 'put' a file, it seems to try to > open a new file for write with these modes/flags: > [binary,write,creat,trunc], which results in error. > > I intercept these in my custom file_handler for sftpd (essentially just log > the arguments in the open callback) > > Doing manually in erl shell: >> Result = file:open("/tmp/da.txt", [binary, write, creat, trunc]). > results in: > {error,badarg} > > so it looks like file:open doesn't like 'creat' or 'trunc' flags that sftpd > tries to pass to it > > this is from the default file_handler sftpd uses: > > open(Path, Flags, State) -> > {file:open(Path, Flags), State}. > > > .. i.e just passes the flags as modes to file:open > > these results are further compounded with inconsistent behavior while > running on different systems > > i've tried running sftpd on a mac (lion) and ubuntu, seeing the above > results, but trying to run the sftpd on a DS211j (that uses marvell ARM cpu > and a version of busybox distro and R15B01 probably somehow cross-compiled > specifically for the ARM) is see a different behavior: > > on the DS211j, if i try to perform same steps as on above systems, the flags > i intercept are [raw,binary,write], which writes the file fine and doesn't > throw an error like in above cases > > Tried to run the otp testsuite, but seem to be failing on testsuite build > step having to do with hipe > > Any insights would be welcome. > For now i just overwrite the collection of modes/flags with > [raw,binary,write] whenever there is 'write' present in the list, but this > seems hacky > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From siraaj@REDACTED Thu Dec 27 13:41:50 2012 From: siraaj@REDACTED (Siraaj Khandkar) Date: Thu, 27 Dec 2012 07:41:50 -0500 Subject: [erlang-questions] Running multiple commands like os:cmd/1 In-Reply-To: References: <5E7C8E8E-CE40-46AB-8D93-01EAC2A66759@khandkar.net> Message-ID: <562A90D5-FD22-4962-9AC2-53E73277CDFD@khandkar.net> On Dec 27, 2012, at 6:08 AM, Gleb Peregud wrote: > On Thu, Dec 27, 2012 at 1:25 AM, Siraaj Khandkar wrote: >> Why does it have to be a single shell? What is wrong with multiple ports? > > It's because I want to preserve changes in env variables between calls > (see requirement 4), since I want to be able to run commands like ". > /opt/erlang/r15b/activate" to prepare environment for further > commands. Think of a sequence of commands run by CI server. OK, this is getting hacky now, but how about a script that reads lines on stdin and outputs them with a prefix, then execute your wrapped commands as backgrounded subshells. Something like this: ((command_1; echo $?) | wrapper --prefix "$ID1 => ") & ((command_2; echo $?) | wrapper --prefix "$ID2 => ") & -- Siraaj Khandkar .o. ..o ooo From gleber.p@REDACTED Thu Dec 27 14:03:15 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 27 Dec 2012 14:03:15 +0100 Subject: [erlang-questions] Running multiple commands like os:cmd/1 In-Reply-To: <562A90D5-FD22-4962-9AC2-53E73277CDFD@khandkar.net> References: <5E7C8E8E-CE40-46AB-8D93-01EAC2A66759@khandkar.net> <562A90D5-FD22-4962-9AC2-53E73277CDFD@khandkar.net> Message-ID: This solution won't work due to the fact that env variables changes in subshells are not propagated to main shell. At the moment solution 99.99% solution in the previous email works for me. I have a feeling that a proper solution would have to reimplement large part of the /bin/sh to make it really robust. On Thu, Dec 27, 2012 at 1:41 PM, Siraaj Khandkar wrote: > On Dec 27, 2012, at 6:08 AM, Gleb Peregud wrote: > >> On Thu, Dec 27, 2012 at 1:25 AM, Siraaj Khandkar wrote: >>> Why does it have to be a single shell? What is wrong with multiple ports? >> >> It's because I want to preserve changes in env variables between calls >> (see requirement 4), since I want to be able to run commands like ". >> /opt/erlang/r15b/activate" to prepare environment for further >> commands. Think of a sequence of commands run by CI server. > > OK, this is getting hacky now, but how about a script that reads lines on stdin > and outputs them with a prefix, then execute your wrapped commands as > backgrounded subshells. Something like this: > > ((command_1; echo $?) | wrapper --prefix "$ID1 => ") & > ((command_2; echo $?) | wrapper --prefix "$ID2 => ") & > > > -- > Siraaj Khandkar > .o. > ..o > ooo > From montuori@REDACTED Thu Dec 27 20:14:46 2012 From: montuori@REDACTED (Kevin Montuori) Date: Thu, 27 Dec 2012 14:14:46 -0500 Subject: [erlang-questions] Running multiple commands like os:cmd/1 In-Reply-To: References: <5E7C8E8E-CE40-46AB-8D93-01EAC2A66759@khandkar.net> <562A90D5-FD22-4962-9AC2-53E73277CDFD@khandkar.net> Message-ID: <1356635686.27005.140661170719873.73AA20B1@webmail.messagingengine.com> On Thu, Dec 27, 2012, at 8:03, Gleb Peregud wrote: > At the moment solution 99.99% solution in the previous email works for > me. I have a feeling that a proper solution would have to reimplement > large part of the /bin/sh to make it really robust. Although it doesn't answer the question you asked exactly, I'd suggest that when you get to this point you're fighting a little against the grain of the language. Sometimes it's easier to whomp up a shell (or Perl, Python, &c.) script and just call that instead of trying to figure out how to robustly string together individual shell commands. It often ends up being easier to test and debug because you've separated the tasks of calling the shell script from the actions the shell script performs. Plus you can maintain shell script changes separately from the Erlang wrapper. I don't presume to understand the issue you're trying to solve and it seems like you have things well in hand, so please take my observation with a grain of salt. k. -- Kevin Montuori montuori@REDACTED From dan353hehe@REDACTED Thu Dec 27 22:11:19 2012 From: dan353hehe@REDACTED (Daniel Barney) Date: Thu, 27 Dec 2012 14:11:19 -0700 Subject: [erlang-questions] supervisor using 500+ MB of ram to track 6.7 million dead workers? Message-ID: Hey Guys, This is what I am running into, I have a supervisor which manages a bunch of processes that are each in charge of a gen_tcp or ssl socket. After upgrading to R15B02 i started noticing that memory usage was growing slowly and over the course of three days I found that the machines were starting to run out of memory. I do not remember which version we upgraded from, but it was from last year. I can check if it is really needed. We upgraded to R15B02, wrote a small patch and then we cherry-picked it onto the stable version. The patch shouldn't be affecting what I am seeing as I only changed two files in the SSL application, relating to parsing certs that do not obey the standard. So assuming that i had a memory leak somewhere i first checked to see if any processes were using large amounts of memory, and this is what I found: <0.785.0> supervisor:cowboy_requests_sup/1 8024355 15270185 0 gen_server:loop/6 9 This processes was a supervisor so i checked how many children that it had, and this is what i got back: supervisor:count_children(Pid). [{specs,1},{active,80},{supervisors,0},{workers,6782028}] And then general process information: erlang:process_info(Pid). [{current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}, {status,waiting}, {message_queue_len,0}, {messages,[]}, {links,[<0.783.0>]}, {dictionary,[{'$ancestors',[<0.783.0>,cowboy_sup,<0.544.0>]}, {'$initial_call',{supervisor,cowboy_requests_sup,1}}]}, {trap_exit,true}, {error_handler,error_handler}, {priority,normal}, {group_leader,<0.543.0>}, {total_heap_size,67810415}, {heap_size,8024355}, {stack_size,9}, {reductions,1551847476}, {garbage_collection,[{min_bin_vheap_size,46368}, {min_heap_size,233}, {fullsweep_after,65535}, {minor_gcs,31155}]}, {suspending,[]}] erlang:process_info(Pid,memory). {memory,542484248} and this is how many processes i have on the machine: length(processes()). 588 so all of the workers for this supervisor are dead. but it doesn't think so for some reason? Is there any reason why a supervisor would hold onto 6.7 million workers that are already dead? Any help would be appreciated, Daniel From kostis@REDACTED Fri Dec 28 01:20:05 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 28 Dec 2012 02:20:05 +0200 Subject: [erlang-questions] Dialyzer and improper lists In-Reply-To: References: Message-ID: <50DCE5B5.8020109@cs.ntua.gr> On 12/27/2012 01:28 AM, Aliaksey Kandratsenka wrote: > Hi. > > It appears dialyzer complains loudly about any code that builds improper > lists. And there appears to be no local way to silence that warning. Dialyzer is an optional tool (nobody forces you to use it if you do not find it useful) and identifies code discrepancies (Dialyzer stands for `Discrepancy Analyzer for Erlang') for which it issues warnings. Nobody ever claimed that all these are discrepancies are bugs; instead, they are code points where a programmer action is arguably needed. Whether issuing a warning classifies as "complaining loudly" is a matter of definition. Your second sentence is true. Dialyzer has no local ways of silencing warnings but it does come with options (-Wno_improper_lists in this case) to silence specific categories of warnings that are on by default and also enable other categories of warnings (e.g. -Wunmatched_returns) which are off by default. > It's true that sometimes it's a sign of bug. But on the other hand > dialyzer docs saying there's no reason to use improper lists (even in > ALL CAPS) is clearly wrong. According to my understanding improper list > of size 2 needs just 2 words (single cons pair and no headers), but > binary tuple needs one extra word for header. Right. And do you have an application where you have experienced that this makes any considerable difference? If so, I would like to see it. More importantly, if you do have such an application, are you _really_ sure that Erlang is the most appropriate language for programming it? > And erlang's stdlib has a bunch of places where cons pair is used as a > binary tuple. E.g. dict. > > While docs should not necessarily point folks to low-level tricks like > that, they IMHO should be fixed to reflect reality. > > But IMHO most importantly, there should be some way to say to dialyzer > "hey I really mean improper list here" without disabling improper list > check completely. If you find such a feature useful, feel free to submit an appropriate patch and we may consider including it in dialyzer's code base. Please make sure to make this `local silencing of warnings' feature pretty robust under code changes. It's obviously not rocket science to design and implement such a scheme, but I am warning you that it's also not a trivial task. Kostis From g@REDACTED Fri Dec 28 06:44:56 2012 From: g@REDACTED (Garrett Smith) Date: Thu, 27 Dec 2012 23:44:56 -0600 Subject: [erlang-questions] Running multiple commands like os:cmd/1 In-Reply-To: References: Message-ID: Have you experimented with calling the shell as a single command, passing along the string to execute? E.g. bash -c "" Garrett On Dec 26, 2012 2:37 PM, "Gleb Peregud" wrote: > Happy holidays, everyone! > > I want to run multiple commands in a single shell from my code through > some sort of open_port({spawn, ...}). I have four requirements: > 1) I want it to work asynchronously (i.e. I start a command and I get > it's results streamed to a process as it happens); > 2) I want to know exactly when a command finishes; > 3) Binary output should be handled properly; > 4) Changes in env variables should be preserved between commands. > > A naive sequence of os:cmd/1 or open_port({spawn, ...}) doesn't > satisfy requirement 4. > > os:cmd/1 has a nice backend code in there, but it doesn't satisfy > requirement 3. It has a bug. Try running length(os:cmd("head -c 1000 > /dev/urandom")). > > open_port({spawn, "/bin/sh -s -"}, [line]) with port_command/2 doesn't > satisfy requirement 2. > > Any ideas how to implement it? Is there any library which does it? > > Cheers, > Gleb > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From baliulia@REDACTED Fri Dec 28 08:47:04 2012 From: baliulia@REDACTED (=?UTF-8?B?SWduYXMgVnnFoW5pYXVza2Fz?=) Date: Fri, 28 Dec 2012 09:47:04 +0200 Subject: [erlang-questions] relups and file handlers Message-ID: <50DD4E78.9030908@gmail.com> Hi Erlangers, I am about to do something nasty, so I wanted to ask you "how nasty you think this is". Due to technical reasons I want to do a relup in this way: 1. I have a release of `foobar` running from $SOMEPATH/foobar/ 2. I deploy new release binaries by completely removing $SOMEPATH/foobar/ and overrwriting them with the new ones 3. In addition, I include some old_vsn_to_new_vsn_relup.tar.gz which contains upgrade code. 4. Now I want to do the usual rebar-style relup using ./foobar/bin/foobar upgrade old_vsn_to_new_vsn_relup It all works fine, except for the fact that since $SOMEPATH/foobar/ gets deleted, the Erlang file_server has an invalid file handler for the CWD and thus file:get_cwd() fails and therefore `erl_untar.erl` fails trying to extract the .tar.gz relup archive. I managed to easily fix this by doing file:set_cwd/1. My quesion: is this safe? Perhaps other applications still have some invalid file handlers and I will start bumping into mysterious issues later? All hints/ideas are very appreciated. Ignas P.S. Note however that I don't care about any other issues: downgrades/keeping a release history/whatever, in this particular case it's of no interest and not a problem. From erlangsiri@REDACTED Fri Dec 28 10:50:37 2012 From: erlangsiri@REDACTED (Siri Hansen) Date: Fri, 28 Dec 2012 10:50:37 +0100 Subject: [erlang-questions] supervisor using 500+ MB of ram to track 6.7 million dead workers? In-Reply-To: References: Message-ID: Hi Daniel - It seems like you are using a different supervisor than the one provided in stdlib. Or maybe a patched version of it - if so, do you know which version it is based on? There has been problems with memory consumption in simple_one_for_one supervisors with *many* children earlier, but that has been fixed already a few releases ago. Regards /siri@REDACTED 2012/12/27 Daniel Barney > Hey Guys, > > This is what I am running into, I have a supervisor which manages a > bunch of processes that are each in charge of a gen_tcp or ssl socket. > After upgrading to R15B02 i started noticing that memory usage was > growing slowly and over the course of three days I found that the > machines were starting to run out of memory. I do not remember which > version we upgraded from, but it was from last year. I can check if it > is really needed. > > We upgraded to R15B02, wrote a small patch and then we cherry-picked > it onto the stable version. The patch shouldn't be affecting what I am > seeing as I only changed two files in the SSL application, relating to > parsing certs that do not obey the standard. > > So assuming that i had a memory leak somewhere i first checked to see > if any processes were using large amounts of memory, and this is what > I found: > > <0.785.0> supervisor:cowboy_requests_sup/1 8024355 15270185 > 0 > gen_server:loop/6 9 > > This processes was a supervisor so i checked how many children that it > had, and this is what i got back: > > supervisor:count_children(Pid). > [{specs,1},{active,80},{supervisors,0},{workers,6782028}] > > > And then general process information: > > erlang:process_info(Pid). > [{current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}, > {status,waiting}, > {message_queue_len,0}, > {messages,[]}, > {links,[<0.783.0>]}, > {dictionary,[{'$ancestors',[<0.783.0>,cowboy_sup,<0.544.0>]}, > {'$initial_call',{supervisor,cowboy_requests_sup,1}}]}, > {trap_exit,true}, > {error_handler,error_handler}, > {priority,normal}, > {group_leader,<0.543.0>}, > {total_heap_size,67810415}, > {heap_size,8024355}, > {stack_size,9}, > {reductions,1551847476}, > {garbage_collection,[{min_bin_vheap_size,46368}, > {min_heap_size,233}, > {fullsweep_after,65535}, > {minor_gcs,31155}]}, > {suspending,[]}] > > erlang:process_info(Pid,memory). > {memory,542484248} > > and this is how many processes i have on the machine: > length(processes()). > 588 > > so all of the workers for this supervisor are dead. but it doesn't > think so for some reason? > > Is there any reason why a supervisor would hold onto 6.7 million > workers that are already dead? > > Any help would be appreciated, > Daniel > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sserre.bx@REDACTED Fri Dec 28 11:21:25 2012 From: sserre.bx@REDACTED (=?ISO-8859-1?Q?S=E9bastien_Serre?=) Date: Fri, 28 Dec 2012 11:21:25 +0100 Subject: [erlang-questions] ASN.1 XER Message-ID: <50DD72A5.1080108@gmail.com> Hello, I have found this message from May 2007. Is there some news? Regards -- ssbx Sanjaya Vitharana wrote: > Hi > > Are there any plan to support XER ? > > Thanks in advance, > > Sanjaya Vitharana > Yes, we have plans to implement that, though it has been postponed into some future release. /Bertil From hyperboreean@REDACTED Sat Dec 29 08:35:35 2012 From: hyperboreean@REDACTED (hyperboreean) Date: Sat, 29 Dec 2012 09:35:35 +0200 Subject: [erlang-questions] Erlang - supervised gen_server to drain off a queue Message-ID: <20121229073535.GA18325@sagitarius.getae.net> Hi guys, I'm just trying to bring Erlang into my company - it's a hard road, but we might eventually use it for at least some of our projects. I want to get started by changing a small piece of code that reads an AMQP queue and appends the contents of each message to a file. To be more specific, there are multiple producers for that queue (which produce error messages, if any) and there's this consumer which gets the error messages from the queue and updates a file with them. Now, this piece of software has always been problematic for us, for one reason or another - it's not reliable and when it fails it might bring the AMQP broker to its knees because of the number of messages that are not consumed. I'd like to change this code with some Erlang code - maybe with a supervision tree so that if/when the server fails, it gets automatically restarted. I was thinking of trying a gen_server, but then I figured out that there's no client of the server, the code just has to constantly pull data out of that queue. Any good hints on how to get started on this ? What's the OTP way of doing this kind of things? Thank you! From steven.charles.davis@REDACTED Sat Dec 29 13:39:51 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sat, 29 Dec 2012 06:39:51 -0600 Subject: [erlang-questions] Frames proposal Message-ID: <3A1E3ECD-6E4B-42AB-94AA-13BC32CE090E@gmail.com> Looking at the age of the last entry in this thread, it seems that the discussion about frames has rather tailed off again. A pity. I'd like to add support for both the introduction of frames (and for the EEP 20 on "splitting atoms"). They appear to me to solve fairly pressing issues. Personally, I _strongly_ prefer the proposed "erlson form" of notation for frames above the suggested <{}> and ~. I do have a question (re: section 8.2 "What we've lost...")... I frequently use pattern matching to constrain arguments to type, e.g.: foo(R = #record{}) -> etc In frames, are we limited to: foo(F) is_frame(F) -> etc ? regs, /s On Thursday, May 3, 2012 8:54:46 PM UTC-5, Richard O'Keefe wrote: On 3/05/2012, at 10:55 PM, Max Lapshin wrote: >> - They have no default value >> - Only atoms may be keys > > These two poins are very interesting to discuss. > > > Problem is with handling external data. Response 1: Frames are very explicitly a replacement for RECORDS. You cannot currently use records in external data. Response 2: EEP 20 (http://www.erlang.org/eeps/eep-0020.html) has now been around for nearly four years. It offers a permanent fix to the limited size of the atom table, a fix which has been used before in another concurrent programming language. That's not the only way to fix the atom table problem. SWI Prolog offers true multicore concurrency AND a garbage collected atom table. That's been around for a few years now too. By any engineering standard I can think of, fixing this vulnerability in Erlang should have very high priority. The frames proposal presupposed that this flaw has been fixed somehow. We *cannot* let this flaw warp Erlang programming for the rest of our lives. > > We will not be able to use frames for handling user data if only atoms > can be keys, but if we allow to store atoms, binaries and strings as > keys, we will get a mess with different ways to introduce keys. We *WILL* be able to use frames for handling user data WHEN THE ATOM TABLE BUG IS FIXED. And it is LONG past time for that to happen. _______________________________________________ erlang-questions mailing list erlang-q...@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Sat Dec 29 13:42:05 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sat, 29 Dec 2012 06:42:05 -0600 Subject: [erlang-questions] Frames proposal In-Reply-To: <3A1E3ECD-6E4B-42AB-94AA-13BC32CE090E@gmail.com> References: <3A1E3ECD-6E4B-42AB-94AA-13BC32CE090E@gmail.com> Message-ID: <688DF805-5FCC-4EB4-B7C5-E3AC5F97E4D6@gmail.com> fixed typo of missed "when" to avoid the distraction from my real question! On Dec 29, 2012, at 6:39 AM, Steve Davis wrote: > Looking at the age of the last entry in this thread, it seems that the discussion about frames has rather tailed off again. A pity. > > I'd like to add support for both the introduction of frames (and for the EEP 20 on "splitting atoms"). They appear to me to solve fairly pressing issues. > > Personally, I _strongly_ prefer the proposed "erlson form" of notation for frames above the suggested <{}> and ~. > > I do have a question (re: section 8.2 "What we've lost...")... I frequently use pattern matching to constrain arguments to type, e.g.: > > foo(R = #record{}) -> etc > > In frames, are we limited to: > foo(F) when is_frame(F) -> etc > ? > > regs, > /s From erlang@REDACTED Sat Dec 29 14:51:31 2012 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 29 Dec 2012 14:51:31 +0100 Subject: [erlang-questions] Frames proposal In-Reply-To: <688DF805-5FCC-4EB4-B7C5-E3AC5F97E4D6@gmail.com> References: <3A1E3ECD-6E4B-42AB-94AA-13BC32CE090E@gmail.com> <688DF805-5FCC-4EB4-B7C5-E3AC5F97E4D6@gmail.com> Message-ID: On Sat, Dec 29, 2012 at 1:42 PM, Steve Davis wrote: > fixed typo of missed "when" to avoid the distraction from my real question! > > On Dec 29, 2012, at 6:39 AM, Steve Davis > wrote: > > > Looking at the age of the last entry in this thread, it seems that the > discussion about frames has rather tailed off again. A pity. > > > > I'd like to add support for both the introduction of frames (and for the > EEP 20 on "splitting atoms"). They appear to me to solve fairly pressing > issues. > > > > Personally, I _strongly_ prefer the proposed "erlson form" of notation > for frames above the suggested <{}> and ~. > > > > I do have a question (re: section 8.2 "What we've lost...")... I > frequently use pattern matching to constrain arguments to type, e.g.: > > > > foo(R = #record{}) -> etc > > > > In frames, are we limited to: > > foo(F) when is_frame(F) -> etc > > ? > I think they might be called maps in the future... maps can have any ground term as a key. If you want the equivalent of the above you'd have to reserve a specific key to emulate the record name. So you'd have something like this: foo(R = #record{}) -> ... foo(R = #another_record{}) -> ... would be (with maps) (using the key 'type' to name the record) foo(#{type=record}=R) -> ... foo(#{type=another_record}=R) -> ... /Joe > > > > regs, > > /s > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Sat Dec 29 15:05:22 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sat, 29 Dec 2012 08:05:22 -0600 Subject: [erlang-questions] Frames proposal In-Reply-To: References: <3A1E3ECD-6E4B-42AB-94AA-13BC32CE090E@gmail.com> <688DF805-5FCC-4EB4-B7C5-E3AC5F97E4D6@gmail.com> Message-ID: Ah yes... it also leaves the flexibility in place for frame/map processing without *having* to specify it being of a particular named type. That would work great. ....R16 anyone? best, /s On Dec 29, 2012, at 7:51 AM, Joe Armstrong wrote: > > would be (with maps) (using the key 'type' to name the record) > > foo(#{type=record}=R) -> ... > foo(#{type=another_record}=R) -> ... From steven.charles.davis@REDACTED Sat Dec 29 15:08:56 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sat, 29 Dec 2012 08:08:56 -0600 Subject: [erlang-questions] Strings and Text Processing Message-ID: Disclaimer :-) All the below is prefixed by a big IMHO Erlang has been correctly criticized for the difficulty of handling "strings". There are two reasons for this (fundamental decisions that were taken way-back-when): 1) "strings" are "just lists of integers" 2) "strings" are by default latin-1 representations This introduces major inconveniences, some of which are not resolvable: When faced with any list during pattern matching, it is not at all easy to determine whether that list is a "string". Further, since strings are "only" a subset of the set of lists of integers, it can be impossible to determine programmatically whether the list is a list of integers or is meant to represent a string. Determining whether a particular list even qualifies as a string in a program requires non-trivial processing of the entire list. It's rather unfortunate that Erlang has earned this reputation, since the truth is that Erlang is truly excellent at text processing. However, to benefit from this excellence, you need to do two things: 1) Represent and process text as binaries. 2) Assume that the text binary is UTF-8 encoded, unless otherwise stated (meaning, e.g. #text{encoding = cstring, value = <<116,101,120,116,0>>}). Suddenly, thanks to binary syntax and pattern matching, processing text in your programs becomes deterministic and easy. (Note that part of the reason for this is that binaries are "expected" to be opaque, whereas general list processing is fundamental to writing any program in Erlang). There's a couple of minor drawbacks, both of which are the result of the initial decisions about "strings": 1) The code is littered with additional angle brackets <<"string">> (annoying, but definitely worth the inconvenience) 2) The standard Erlang/OTP library functions require textual arguments as lists (requiring overuse of binary_to_list) And there are further benefits: 1) Parsing/transcoding different charset encodings is far more straightforward 2) Internationalization/localization is far more straightfoward I wonder if, had the current binary pattern matching/comprehensions been available "way-back-when", whether the decision about "string" representation in Erlang may have been different. (i.e. <<116,101,120,116>> = "text"). Finally, here's my two questions: 1) Is there any benefit at all to the "list representation" of strings above binary text? 2) If not, I wonder if there's any way to change our minds about "strings" as we enter 2013? regs, /s From dmkolesnikov@REDACTED Sat Dec 29 15:20:44 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sat, 29 Dec 2012 16:20:44 +0200 Subject: [erlang-questions] Strings and Text Processing In-Reply-To: References: Message-ID: <65EC42E5-54A3-4556-A3DA-9E949255C1D1@gmail.com> Hello Steve, You have raised a good point here. One more reason for binary is memory consumption and IPC overhead. On another hand list allows to represent a code point per element. iolists are also very handy to dynamically compose a complex strings. I am afraid that this is an application specific questions? However, I tend to use binary for strings... - Dmitry On Dec 29, 2012, at 4:08 PM, Steve Davis wrote: > Disclaimer :-) All the below is prefixed by a big IMHO > > Erlang has been correctly criticized for the difficulty of handling "strings". > > There are two reasons for this (fundamental decisions that were taken way-back-when): > 1) "strings" are "just lists of integers" > 2) "strings" are by default latin-1 representations > > This introduces major inconveniences, some of which are not resolvable: > When faced with any list during pattern matching, it is not at all easy to determine whether that list is a "string". > Further, since strings are "only" a subset of the set of lists of integers, it can be impossible to determine programmatically whether the list is a list of integers or is meant to represent a string. Determining whether a particular list even qualifies as a string in a program requires non-trivial processing of the entire list. > > It's rather unfortunate that Erlang has earned this reputation, since the truth is that Erlang is truly excellent at text processing. However, to benefit from this excellence, you need to do two things: > 1) Represent and process text as binaries. > 2) Assume that the text binary is UTF-8 encoded, unless otherwise stated (meaning, e.g. #text{encoding = cstring, value = <<116,101,120,116,0>>}). > > Suddenly, thanks to binary syntax and pattern matching, processing text in your programs becomes deterministic and easy. (Note that part of the reason for this is that binaries are "expected" to be opaque, whereas general list processing is fundamental to writing any program in Erlang). > > There's a couple of minor drawbacks, both of which are the result of the initial decisions about "strings": > 1) The code is littered with additional angle brackets <<"string">> (annoying, but definitely worth the inconvenience) > 2) The standard Erlang/OTP library functions require textual arguments as lists (requiring overuse of binary_to_list) > > And there are further benefits: > 1) Parsing/transcoding different charset encodings is far more straightforward > 2) Internationalization/localization is far more straightfoward > > I wonder if, had the current binary pattern matching/comprehensions been available "way-back-when", whether the decision about "string" representation in Erlang may have been different. (i.e. <<116,101,120,116>> = "text"). > > Finally, here's my two questions: > 1) Is there any benefit at all to the "list representation" of strings above binary text? > 2) If not, I wonder if there's any way to change our minds about "strings" as we enter 2013? > > regs, > /s > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From masklinn@REDACTED Sat Dec 29 15:34:18 2012 From: masklinn@REDACTED (Masklinn) Date: Sat, 29 Dec 2012 15:34:18 +0100 Subject: [erlang-questions] Strings and Text Processing In-Reply-To: References: Message-ID: <6E1DA248-66E9-43C0-B0CE-9D95A3079C8E@masklinn.net> On 2012-12-29, at 15:08 , Steve Davis wrote: > Erlang is truly excellent at text processing. However, to benefit from this excellence, you need to do two things: > 1) Represent and process text as binaries. > 2) Assume that the text binary is UTF-8 encoded, unless otherwise stated (meaning, e.g. #text{encoding = cstring, value = <<116,101,120,116,0>>}). I don't think the former and the latter match. Erlang/OTP can be nice at string processing where "string" is understood as "sequence of bytes", but it remains rather ungood at *text* processing: *as far as I know*, aside from encoding and decoding UTFs it has very limited support for it[0]: no support (note: by "support" I mean "support built into the core distribution", it's always possible to call into ICU) for UnicodeData queries (codepoint meta-information), unicode case folding, grapheme cluster handling, the important text-processing annexes (UAX 14 "line breaking algorithm", UAX 15 "normalization forms", UAX 29 "text segmentation") or standards (UTS 10 "collation algorithm" and UTS 18 "regular expressions" as well ? for other parts of the system but also part of unicode itself ? UTS 35 "LDML" and the its data-formatting and data-parsing components), ? In fact Dmitry's email demonstrates it rather well when he notes that > On another hand list allows to represent a code point per element. this can provide interesting properties (or not) but it's pretty much irrelevant when it comes to text processing, it's just an implementation detail. [0] http://www.erlang.org/doc/apps/stdlib/unicode_usage.html From erlang@REDACTED Sat Dec 29 16:30:37 2012 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 29 Dec 2012 16:30:37 +0100 Subject: [erlang-questions] Strings and Text Processing In-Reply-To: <65EC42E5-54A3-4556-A3DA-9E949255C1D1@gmail.com> References: <65EC42E5-54A3-4556-A3DA-9E949255C1D1@gmail.com> Message-ID: On Sat, Dec 29, 2012 at 3:20 PM, Dmitry Kolesnikov wrote: > Hello Steve, > > You have raised a good point here. > One more reason for binary is memory consumption and IPC overhead. > The point about memory consumption is raised *many* times - on a modern machine this is not a problem. Example: I am working on a text file of 84KB - in a 32 bit Erlang we use 8 bytes/character - so I use 0.6 MB - I have 4GB memory - so I use 0.015% of memory - ie no problem. My strategy is to keep large strings as binaries when I'm not working on them, turn them into lists in order to work on them, and turn them back into binaries when I'm done. Just because a string starts off in a binary does not mean that it has to stay as a binary as you work on it. Imagine I have a lot of text files, say each of 50KB, I can store 20 per/MB or 20,000 files per GB. Assume I have a quad core. I can only work on four things at the same time - so having (say) 20,000 files (at 50K) and work on four of them (unpacked) at a time is another 1.6 Meg. Gigabyte memories mean (among other things) what saving the odd byte here are there is hardly relevant. > On another hand list allows to represent a code point per element. > yes - the convenience of having one character per list element far outweighs the space saving of storing strings in binaries > iolists are also very handy to dynamically compose a complex strings. > > I am afraid that this is an application specific questions? However, I > tend to use binary for strings... > > My strings change form depending on what I'm doing. Sometimes they are binaries, sometimes lists, sometimes trees, ... Cheers /Joe > - Dmitry > > > On Dec 29, 2012, at 4:08 PM, Steve Davis > wrote: > > > Disclaimer :-) All the below is prefixed by a big IMHO > > > > Erlang has been correctly criticized for the difficulty of handling > "strings". > > > > There are two reasons for this (fundamental decisions that were taken > way-back-when): > > 1) "strings" are "just lists of integers" > > 2) "strings" are by default latin-1 representations > > > > This introduces major inconveniences, some of which are not resolvable: > > When faced with any list during pattern matching, it is not at all easy > to determine whether that list is a "string". > > Further, since strings are "only" a subset of the set of lists of > integers, it can be impossible to determine programmatically whether the > list is a list of integers or is meant to represent a string. Determining > whether a particular list even qualifies as a string in a program requires > non-trivial processing of the entire list. > > > > It's rather unfortunate that Erlang has earned this reputation, since > the truth is that Erlang is truly excellent at text processing. However, to > benefit from this excellence, you need to do two things: > > 1) Represent and process text as binaries. > > 2) Assume that the text binary is UTF-8 encoded, unless otherwise stated > (meaning, e.g. #text{encoding = cstring, value = <<116,101,120,116,0>>}). > > > > Suddenly, thanks to binary syntax and pattern matching, processing text > in your programs becomes deterministic and easy. (Note that part of the > reason for this is that binaries are "expected" to be opaque, whereas > general list processing is fundamental to writing any program in Erlang). > > > > There's a couple of minor drawbacks, both of which are the result of the > initial decisions about "strings": > > 1) The code is littered with additional angle brackets <<"string">> > (annoying, but definitely worth the inconvenience) > > 2) The standard Erlang/OTP library functions require textual arguments > as lists (requiring overuse of binary_to_list) > > > > And there are further benefits: > > 1) Parsing/transcoding different charset encodings is far more > straightforward > > 2) Internationalization/localization is far more straightfoward > > > > I wonder if, had the current binary pattern matching/comprehensions been > available "way-back-when", whether the decision about "string" > representation in Erlang may have been different. (i.e. <<116,101,120,116>> > = "text"). > > > > Finally, here's my two questions: > > 1) Is there any benefit at all to the "list representation" of strings > above binary text? > > 2) If not, I wonder if there's any way to change our minds about > "strings" as we enter 2013? > > > > regs, > > /s > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomasl_erlang@REDACTED Sat Dec 29 17:15:28 2012 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Sat, 29 Dec 2012 08:15:28 -0800 (PST) Subject: [erlang-questions] Strings and Text Processing In-Reply-To: References: Message-ID: <1356797728.23951.YahooMailNeo@web120703.mail.ne1.yahoo.com> >________________________________ > From: Steve Davis ... >Finally, here's my two questions: >1) Is there any benefit at all to the "list representation" of strings above binary text? >2) If not, I wonder if there's any way to change our minds about "strings" as we enter 2013? 1) Don't forget the good old I/O-list, which some might consider the "real" string representation. 2) The groundswell seems to be to use binaries instead of lists, even if the syntax is a bit of a pain. So keep building and backporting support for that, IMO.? Alternatively, it might be worth considering a higher-level datastructure that takes encoding and such into account too. Common Lisp took the route of making characters a separate, opaque datatype if memory serves. Strings as builtin CL-style "compact arrays of characters" (suitably updated to handle unicode!) could perhaps replace the use of binaries. ? Best, Thomas From dmkolesnikov@REDACTED Sat Dec 29 17:21:56 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sat, 29 Dec 2012 18:21:56 +0200 Subject: [erlang-questions] Strings and Text Processing In-Reply-To: References: <65EC42E5-54A3-4556-A3DA-9E949255C1D1@gmail.com> Message-ID: <963A0976-E5A4-4896-A6FF-1730447DC5DD@gmail.com> Hello Joe, I do re-call you strategy "keep large strings as binaries when I'm not working on them". I was trying to reference it in my previous email :-) but did not find a link to it. One thing, what I am not fully agree with you is a statement about memory on modern machines? Let me give you a simple example about EC2 cloud: * M1 Small Instance (Default) 1.7 GiB of memory * M1 Medium Instance 3.75 GiB of memory * M1 Large Instance 7.5 GiB of memory So I am using EC2 to run my service for consumers, I have to keep 4KB of json objects per registered user. Using lists, "Each character consumes 8 bytes of memory on a 32 bit machine?". 4KB data blows up 8x times. Taking into assumption that active users are kept in memory, each node handles * M1 Small Instance (Default) 1.7 GiB of memory --> 53K active users * M1 Medium Instance 3.75 GiB of memory --> 117K active users * M1 Large Instance 7.5 GiB of memory --> 234K active users Using binaries same metrics could be improved 8x times. Note: the following calculation is artificial. It just demonstrates that memory metrics is really depends on use-case. BTW, memory utilisation was a reason why I moved from mochijson to jsx, it parses json as binary. The drop in performance was acceptable but memory consumption was improved 2-3 times... All in-all, I do agree that string are changes during processing, use-case, etc. - Dmitry On Dec 29, 2012, at 5:30 PM, Joe Armstrong wrote: > > > On Sat, Dec 29, 2012 at 3:20 PM, Dmitry Kolesnikov wrote: > Hello Steve, > > You have raised a good point here. > One more reason for binary is memory consumption and IPC overhead. > > The point about memory consumption is raised *many* times - on a modern > machine this is not a problem. > > Example: I am working on a text file of 84KB - in a 32 bit Erlang we use > 8 bytes/character - so I use 0.6 MB - I have 4GB memory - so I use 0.015% of > memory - ie no problem. > > My strategy is to keep large strings as binaries when I'm not working on them, > turn them into lists in order to work on them, and turn them back into binaries > when I'm done. Just because a string starts off in a binary does not mean > that it has to stay as a binary as you work on it. > > > Imagine I have a lot of text files, say each of 50KB, I can store 20 per/MB or > 20,000 files per GB. Assume I have a quad core. I can only work on four things > at the same time - so having (say) 20,000 files (at 50K) and work on four of them > (unpacked) at a time is another 1.6 Meg. > > Gigabyte memories mean (among other things) what saving the odd byte here are there is hardly relevant. > > > On another hand list allows to represent a code point per element. > > yes - the convenience of having one character per list element far outweighs > the space saving of storing strings in binaries > > > iolists are also very handy to dynamically compose a complex strings. > > I am afraid that this is an application specific questions? However, I tend to use binary for strings... > > > My strings change form depending on what I'm doing. Sometimes they are > binaries, sometimes lists, sometimes trees, ... > > Cheers > > /Joe > > > - Dmitry > > > On Dec 29, 2012, at 4:08 PM, Steve Davis wrote: > > > Disclaimer :-) All the below is prefixed by a big IMHO > > > > Erlang has been correctly criticized for the difficulty of handling "strings". > > > > There are two reasons for this (fundamental decisions that were taken way-back-when): > > 1) "strings" are "just lists of integers" > > 2) "strings" are by default latin-1 representations > > > > This introduces major inconveniences, some of which are not resolvable: > > When faced with any list during pattern matching, it is not at all easy to determine whether that list is a "string". > > Further, since strings are "only" a subset of the set of lists of integers, it can be impossible to determine programmatically whether the list is a list of integers or is meant to represent a string. Determining whether a particular list even qualifies as a string in a program requires non-trivial processing of the entire list. > > > > It's rather unfortunate that Erlang has earned this reputation, since the truth is that Erlang is truly excellent at text processing. However, to benefit from this excellence, you need to do two things: > > 1) Represent and process text as binaries. > > 2) Assume that the text binary is UTF-8 encoded, unless otherwise stated (meaning, e.g. #text{encoding = cstring, value = <<116,101,120,116,0>>}). > > > > Suddenly, thanks to binary syntax and pattern matching, processing text in your programs becomes deterministic and easy. (Note that part of the reason for this is that binaries are "expected" to be opaque, whereas general list processing is fundamental to writing any program in Erlang). > > > > There's a couple of minor drawbacks, both of which are the result of the initial decisions about "strings": > > 1) The code is littered with additional angle brackets <<"string">> (annoying, but definitely worth the inconvenience) > > 2) The standard Erlang/OTP library functions require textual arguments as lists (requiring overuse of binary_to_list) > > > > And there are further benefits: > > 1) Parsing/transcoding different charset encodings is far more straightforward > > 2) Internationalization/localization is far more straightfoward > > > > I wonder if, had the current binary pattern matching/comprehensions been available "way-back-when", whether the decision about "string" representation in Erlang may have been different. (i.e. <<116,101,120,116>> = "text"). > > > > Finally, here's my two questions: > > 1) Is there any benefit at all to the "list representation" of strings above binary text? > > 2) If not, I wonder if there's any way to change our minds about "strings" as we enter 2013? > > > > regs, > > /s > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From masklinn@REDACTED Sat Dec 29 17:26:59 2012 From: masklinn@REDACTED (Masklinn) Date: Sat, 29 Dec 2012 17:26:59 +0100 Subject: [erlang-questions] Strings and Text Processing In-Reply-To: <1356797728.23951.YahooMailNeo@web120703.mail.ne1.yahoo.com> References: <1356797728.23951.YahooMailNeo@web120703.mail.ne1.yahoo.com> Message-ID: On 2012-12-29, at 17:15 , Thomas Lindgren wrote: > Alternatively, it might be worth considering a higher-level datastructure that takes encoding and such into account too. Common Lisp took the route of making characters a separate, opaque datatype if memory serves. Strings as builtin CL-style "compact arrays of characters" (suitably updated to handle unicode!) could perhaps replace the use of binaries. > My experience is that a "character" datatype has very limited use for text processing, and tends to drive users (of the API) towards the wrong patterns, especially when trying to build a good unicode-based text-processing API: whatever you pick for a "character" (usually a byte, a code unit or a unicode codepoint) will be the wrong thing more often than not at a higher level. On the other hand, defining erlang strings as iolists (or an opaque datatype implemented through iolists) could work nicely. And it'd be backwards-compatible: existing strings are already valid iolists (although raw binaries are not and have to be wrapped in a list). It would also go a long way towards fixing issues such as http://prog21.dadgum.com/70.html: > Ideally filenames would be IO lists, but for compatibility reasons > there's still the need to support atoms in filenames. That brings up an > interesting idea: why not allow atoms as part of the general IO list > specification? > [?] > I find I'm often calling atom_to_list before sending data to external > ports, and that would no longer be necessary. From g@REDACTED Sat Dec 29 17:44:29 2012 From: g@REDACTED (Garrett Smith) Date: Sat, 29 Dec 2012 10:44:29 -0600 Subject: [erlang-questions] Erlang - supervised gen_server to drain off a queue In-Reply-To: <20121229073535.GA18325@sagitarius.getae.net> References: <20121229073535.GA18325@sagitarius.getae.net> Message-ID: Hello! On Sat, Dec 29, 2012 at 1:35 AM, hyperboreean wrote: > Hi guys, > > I'm just trying to bring Erlang into my company - it's a hard road, but > we might eventually use it for at least some of our projects. I want to > get started by changing a small piece of code that reads an AMQP queue > and appends the contents of each message to a file. To be more specific, > there are multiple producers for that queue (which produce error > messages, if any) and there's this consumer which gets the error > messages from the queue and updates a file with them. Now, this piece of > software has always been problematic for us, for one reason or another - > it's not reliable and when it fails it might bring the AMQP broker to > its knees because of the number of messages that are not consumed. While it doesn't help you get Erlang into your org, the "next obvious step" might be to place your current queue reader under OS process supervision like runit, systemd, launchd, etc. For this to work, you'll need to make sure that the program crashes (process death) whenever it gets into a flaky state. Depending on the program, that may be practically impossible. You could alternatively write a small program that monitors the AMQP queue size and restarts the reader process when the queue reaches a certain size -- assuming there's a correlation there. But at that point you're writing new code, so Erlang is back in play. > I'd like to change this code with some Erlang code - maybe with a > supervision tree so that if/when the server fails, it gets automatically > restarted. You should always run Erlang apps under a supervisory tree. If indeed you're using OTP apps, this is automatic. However, the benefit of Erlang process supervision may be different from what you're thinking. Supervision is the secret that lets you skip error handling in the default case. That leads to hugely simplified code that has a much better chance of working correctly. You can easily write software that "keeps running" by using defensive measures. E.g. a top level try/catch block will keep your program running forever. But God forbid you ever hit that block -- your program will be in some unknown state without any chance of recovery! Unless you're absolutely certain about how an error should be handled (e.g. unexpected input from a user -> return an error message), the best approach is to let a process crash catastrophically (process death) and let someone else restart it to a known state. Erlang is like getting paid to eat a free lunch -- you skip the error handling *and* have more reliable software! > I was thinking of trying a gen_server, but then I figured out > that there's no client of the server, the code just has to constantly > pull data out of that queue. You want a gen_server that starts itself. There's a trick... in the gen_server init/1 callback, return a 0 for the timeout value: -behavior(gen_server). ... init(Args) -> {ok, init_state(Args), 0}. This will send a timeout message to the process, which will be delivered before any other messages: handle_info(timeout, State) -> pull_messages(State), {noreply, State, 0}. This turns a "server" into a self starting process. I personally dislike these semantics, so I use e2 [1] "tasks". In e2, the equivalent looks like this: -behavior(e2_task). ... init(Args) -> {ok, init_state(Args)}. handle_task(State) -> pull_messages(State), {repeat, State}. It uses the same underlying gen_server behavior, so you're accomplishing *exactly* the same thing (standard OTP) without the odd timeout semantics. Even if you don't mind the timeout trick, consider others in your organization who might want to understand the code at some point! If you get your Erlang based queue reader going, you'll have a fantastic platform for adding more features... it's just a matter plugging new tasks, services into your supervisor tree. E.g. you might want a separate monitoring task that routinely checks the AMQP queue size and takes some action if it gets too high (send an email, pull messages from the queue and discard, delete the queue, etc.) That's just another "task" that repeats: init(Args) -> {ok, init_state(Args), {?INITIAL_DELAY, ?REPEAT_INTERVAL}} handle_task(State) -> check_queue_size(State), {repeat, State}. This task will repeat at REPEAT_INTERVAL milliseconds. Good luck with your project. I think you'll see some success: a) Erlang is perfect for what you're trying to do and b) you're tackling a small enough problem that your chances of getting something useful is quite high! Garrett [1] http://e2project.org From arthurbeall@REDACTED Sat Dec 29 17:47:39 2012 From: arthurbeall@REDACTED (Art Beall) Date: Sat, 29 Dec 2012 08:47:39 -0800 (PST) Subject: [erlang-questions] Erlang - supervised gen_server to drain off a queue In-Reply-To: References: Message-ID: <1356799659.56346.YahooMailNeo@web39306.mail.mud.yahoo.com> One way is to use the timeout in the return tuple to gen_server. After the timeout period, gen_server will call your handle_info function with the atom timeout.? You can handle?the queue then. ? http://www.erlang.org/doc/man/gen_server.html#Module:handle_info-2 ? ? Art > >Hi guys, > >I'm just trying to bring Erlang into my company - it's a hard road, but >we might eventually use it for at least some of our projects. I want to >get started by changing a small piece of code that reads an AMQP queue >and appends the contents of each message to a file. To be more specific, >there are multiple producers for that queue (which produce error >messages, if any) and there's this consumer which gets the error >messages from the queue and updates a file with them. Now, this piece of >software has always been problematic for us, for one reason or another - >it's not reliable and when it fails it might bring the AMQP broker to >its knees because of the number of messages that are not consumed. > >I'd like to change this code with some Erlang code - maybe with a >supervision tree so that if/when the server fails, it gets automatically >restarted. I was thinking of trying a gen_server, but then I figured out >that there's no client of the server, the code just has to constantly >pull data out of that queue. > >Any good hints on how to get started on this ? What's the OTP way of >doing this kind of things? > >Thank you! > >? -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomasl_erlang@REDACTED Sat Dec 29 17:52:20 2012 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Sat, 29 Dec 2012 08:52:20 -0800 (PST) Subject: [erlang-questions] Frames proposal In-Reply-To: <3A1E3ECD-6E4B-42AB-94AA-13BC32CE090E@gmail.com> References: <3A1E3ECD-6E4B-42AB-94AA-13BC32CE090E@gmail.com> Message-ID: <1356799940.34463.YahooMailNeo@web120704.mail.ne1.yahoo.com> >________________________________ > From: Steve Davis ... >I'd like to add support for both the introduction of frames (and for the EEP 20 on "splitting atoms"). They appear to me to solve fairly pressing issues. Regarding atom garbage collection, I'd also recommend OTP to check if there's anything useful in my old paper, inventively named "Atom Garbage Collection", Erlang Workshop 2005. Give me a call if it's too obscure (as I recall, it didn't really set the world on fire).? Two comments on EEP 20?(I looked at this one: http://www.erlang.org/eeps/eep-0020.html ): 1. EEP 20 doesn't mention handling of message passing or similar scenarios as far as I can see. Perhaps there should be an implementation study where someone could figure out what's the best policy. E.g., should local atoms be globalized when sent? Or should there be translation at sender and/or receiver? Or something else?? 2. Towards the end, leaking global atoms is mentioned. This can in principle happen not only maliciously or by error, but also by upgrading your code sufficiently many times or in other situations where valid external data containing atoms arrives. So while using local atoms reduces the problem of running out of atom table, some way of collecting the global atoms would still be useful. The paper above is one proposal for this. Best, Thomas From thomasl_erlang@REDACTED Sat Dec 29 17:59:08 2012 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Sat, 29 Dec 2012 08:59:08 -0800 (PST) Subject: [erlang-questions] Strings and Text Processing In-Reply-To: <6E1DA248-66E9-43C0-B0CE-9D95A3079C8E@masklinn.net> References: <6E1DA248-66E9-43C0-B0CE-9D95A3079C8E@masklinn.net> Message-ID: <1356800348.72926.YahooMailNeo@web120705.mail.ne1.yahoo.com> ----- Original Message ----- > From: Masklinn ... > > I don't think the former and the latter match. Erlang/OTP can be nice at > string processing where "string" is understood as "sequence of > bytes", > but it remains rather ungood at *text* processing: *as far as I know*, > aside from encoding and decoding UTFs it has very limited support for > it[0]: no support (note: by "support" I mean "support built into > the > core distribution", it's always possible to call into ICU) for > UnicodeData queries (codepoint meta-information), unicode case folding, > grapheme cluster handling, the important text-processing annexes (UAX 14 > "line breaking algorithm", UAX 15 "normalization forms", UAX > 29 "text > segmentation") or standards (UTS 10 "collation algorithm" and UTS > 18 > "regular expressions" as well ? for other parts of the system but also > part of unicode itself ? UTS 35 "LDML" and the its data-formatting and > data-parsing components), ? Good point. A strong erlang unicode library implementing the above would be very nice. (I'm not a great fan of drivers myself.) Best regards to Arcturus, Thomas? From max.lapshin@REDACTED Sat Dec 29 18:30:26 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 29 Dec 2012 20:30:26 +0300 Subject: [erlang-questions] Frames proposal In-Reply-To: <1356799940.34463.YahooMailNeo@web120704.mail.ne1.yahoo.com> References: <3A1E3ECD-6E4B-42AB-94AA-13BC32CE090E@gmail.com> <1356799940.34463.YahooMailNeo@web120704.mail.ne1.yahoo.com> Message-ID: What is the idea to make atoms collectable? They are designed as a limited number of domain specific words. You cannot deal with endless amount of domain language, it is always finite. So atoms are also finite. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Sat Dec 29 23:36:57 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 29 Dec 2012 17:36:57 -0500 Subject: [erlang-questions] process manager, R16, and 64-bit Macs In-Reply-To: References: <50BE791E.1050202@simonstl.com> <50BF1EA5.3040201@erlang.org> <50BF30A6.3030608@erlang.org> <50BF33FC.5080608@ninenines.eu> <50CB93B8.1070203@ninenines.eu> Message-ID: <20121229223655.GA3540@ferdmbp.local> I just tried it on OSX (10.7.5). It worked fine, though I had messed up some of the paths with former installs done without brew. After working around things, I was able to get simlinks in place, boot R13B01-1 and get observer working. Thanks for this, much appreciated! On 12/26, Joan Valduvieco Llopart wrote: > Hi all, > I managed to add wxgtk Formula to Homebrew and to make some modifications > to erlang Formula allowing Erlang GUI tools to work on Mountain Lion. > You can find my changes here: > > https://github.com/jvalduvieco/homebrew/compare/master...erlang-wxgtk > > gists with complete files: > > https://gist.github.com/4376548 > https://gist.github.com/4376555 > > If you want to use these formula just replace original Homebrew formula (in > /usr/local/Library/Formula/) with mine and run: > > brew uninstall erlang > brew install -vd --with-wxgtk erlang > > I'll submit a pull request to mxcl as soon as I get some positive feedback. > Now it just works for me. > > Thanks, > Joan > > > 2012/12/18 Joan Valduvieco Llopart > > > Sure! > > I'll try. Installing brew right now.. ;) > > > > Joan > > > > > > 2012/12/14 Lo?c Hoguin > > > >> On 12/05/2012 01:17 PM, Joan Valduvieco Llopart wrote: > >> > >>> I am using observer with mountain lion with wxgtk + xquartz without a > >>> glitch. :) > >>> I have a macports repo with my packages. > >>> https://github.com/**jvalduvieco/macports > >>> > >> > >> Any chance you could port this to homebrew? From what I hear most people > >> who use Macs use homebrew so you doing this would be a tremendous help for > >> the community and for the OTP team. > >> > >> -- > >> Lo?c Hoguin > >> > >> Erlang Cowboy > >> Nine Nines > >> http://ninenines.eu > >> > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From masklinn@REDACTED Sat Dec 29 23:41:06 2012 From: masklinn@REDACTED (Masklinn) Date: Sat, 29 Dec 2012 23:41:06 +0100 Subject: [erlang-questions] Strings and Text Processing In-Reply-To: <1356800348.72926.YahooMailNeo@web120705.mail.ne1.yahoo.com> References: <6E1DA248-66E9-43C0-B0CE-9D95A3079C8E@masklinn.net> <1356800348.72926.YahooMailNeo@web120705.mail.ne1.yahoo.com> Message-ID: On 2012-12-29, at 17:59 , Thomas Lindgren wrote: > Good point. A strong erlang unicode library implementing the above would be very nice. If anybody wants to give it a shot (beware, it's not for the faint of heart) I would strongly recommend reading Daniel Ehrenberg's unicode-tagged posts[0], especially his 6 "Unicode implementer's guide" (but the rest as well, the whole tag is a goldmine) before diving into the Unicode Standard itself. [0] http://useless-factor.blogspot.fr/search/label/unicode From essen@REDACTED Sun Dec 30 04:24:54 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Sun, 30 Dec 2012 04:24:54 +0100 Subject: [erlang-questions] Frames proposal In-Reply-To: <1356799940.34463.YahooMailNeo@web120704.mail.ne1.yahoo.com> References: <3A1E3ECD-6E4B-42AB-94AA-13BC32CE090E@gmail.com> <1356799940.34463.YahooMailNeo@web120704.mail.ne1.yahoo.com> Message-ID: <50DFB406.5000405@ninenines.eu> On 12/29/2012 05:52 PM, Thomas Lindgren wrote: > > >> ________________________________ >> From: Steve Davis > ... >> I'd like to add support for both the introduction of frames (and for the EEP 20 on "splitting atoms"). They appear to me to solve fairly pressing issues. > > > Regarding atom garbage collection, I'd also recommend OTP to check if there's anything useful in my old paper, inventively named "Atom Garbage Collection", Erlang Workshop 2005. Give me a call if it's too obscure (as I recall, it didn't really set the world on fire). > > Two comments on EEP 20 (I looked at this one: http://www.erlang.org/eeps/eep-0020.html ): > > 1. EEP 20 doesn't mention handling of message passing or similar scenarios as far as I can see. Perhaps there should be an implementation study where someone could figure out what's the best policy. E.g., should local atoms be globalized when sent? Or should there be translation at sender and/or receiver? Or something else? > > 2. Towards the end, leaking global atoms is mentioned. This can in principle happen not only maliciously or by error, but also by upgrading your code sufficiently many times or in other situations where valid external data containing atoms arrives. So while using local atoms reduces the problem of running out of atom table, some way of collecting the global atoms would still be useful. The paper above is one proposal for this. Anthony Ramine has done some work on local atoms which would be garbage collected. AFAIK it needs a few other patches to get in before that can happen though, and these weren't accepted for R16. :( https://github.com/nox/otp/tree/eep20 -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From essen@REDACTED Sun Dec 30 04:28:20 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Sun, 30 Dec 2012 04:28:20 +0100 Subject: [erlang-questions] Erlang - supervised gen_server to drain off a queue In-Reply-To: <1356799659.56346.YahooMailNeo@web39306.mail.mud.yahoo.com> References: <1356799659.56346.YahooMailNeo@web39306.mail.mud.yahoo.com> Message-ID: <50DFB4D4.6020108@ninenines.eu> This timeout only triggers if no message/cast/call was performed during the timeout period. So it's only useful for servers where more or less nothing happen. On 12/29/2012 05:47 PM, Art Beall wrote: > One way is to use the timeout in the return tuple to gen_server. After > the timeout period, gen_server will call your handle_info function with > the atom timeout. You can handle the queue then. > http://www.erlang.org/doc/man/gen_server.html#Module:handle_info-2 > > Art > > Hi guys, > > I'm just trying to bring Erlang into my company - it's a hard road, but > we might eventually use it for at least some of our projects. I want to > get started by changing a small piece of code that reads an AMQP queue > and appends the contents of each message to a file. To be more specific, > there are multiple producers for that queue (which produce error > messages, if any) and there's this consumer which gets the error > messages from the queue and updates a file with them. Now, this piece of > software has always been problematic for us, for one reason or another - > it's not reliable and when it fails it might bring the AMQP broker to > its knees because of the number of messages that are not consumed. > > I'd like to change this code with some Erlang code - maybe with a > supervision tree so that if/when the server fails, it gets automatically > restarted. I was thinking of trying a gen_server, but then I figured out > that there's no client of the server, the code just has to constantly > pull data out of that queue. > > Any good hints on how to get started on this ? What's the OTP way of > doing this kind of things? > > Thank you! > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From g.a.c.rijnders@REDACTED Sun Dec 30 09:28:45 2012 From: g.a.c.rijnders@REDACTED (Michel Rijnders) Date: Sun, 30 Dec 2012 09:28:45 +0100 Subject: [erlang-questions] process manager, R16, and 64-bit Macs In-Reply-To: References: <50BE791E.1050202@simonstl.com> <50BF1EA5.3040201@erlang.org> <50BF30A6.3030608@erlang.org> <50BF33FC.5080608@ninenines.eu> <50CB93B8.1070203@ninenines.eu> Message-ID: Hi, I had to change line 66 of the first file from: ENV.append 'LDFLAGS', '-framework Appkit' if build.include? 'with-wxgtk' to (note the uppercase K in "AppKit"): ENV.append 'LDFLAGS', '-framework AppKit' if build.include? 'with-wxgtk' To get it to work for me (OS X 10.8.2, XCode 4.5.2). Cheers, Michel On Wed, Dec 26, 2012 at 1:19 AM, Joan Valduvieco Llopart < jvalduvieco@REDACTED> wrote: > Hi all, > I managed to add wxgtk Formula to Homebrew and to make some modifications > to erlang Formula allowing Erlang GUI tools to work on Mountain Lion. > You can find my changes here: > > https://github.com/jvalduvieco/homebrew/compare/master...erlang-wxgtk > > gists with complete files: > > https://gist.github.com/4376548 > https://gist.github.com/4376555 > > If you want to use these formula just replace original Homebrew formula > (in /usr/local/Library/Formula/) with mine and run: > > brew uninstall erlang > brew install -vd --with-wxgtk erlang > > I'll submit a pull request to mxcl as soon as I get some positive > feedback. Now it just works for me. > > Thanks, > Joan > > > 2012/12/18 Joan Valduvieco Llopart > >> Sure! >> I'll try. Installing brew right now.. ;) >> >> Joan >> >> >> 2012/12/14 Lo?c Hoguin >> >>> On 12/05/2012 01:17 PM, Joan Valduvieco Llopart wrote: >>> >>>> I am using observer with mountain lion with wxgtk + xquartz without a >>>> glitch. :) >>>> I have a macports repo with my packages. >>>> https://github.com/**jvalduvieco/macports >>>> >>> >>> Any chance you could port this to homebrew? From what I hear most people >>> who use Macs use homebrew so you doing this would be a tremendous help for >>> the community and for the OTP team. >>> >>> -- >>> Lo?c Hoguin >>> >>> Erlang Cowboy >>> Nine Nines >>> http://ninenines.eu >>> >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- My other car is a cdr. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyron.zerafa@REDACTED Sun Dec 30 12:16:57 2012 From: tyron.zerafa@REDACTED (Tyron Zerafa) Date: Sun, 30 Dec 2012 12:16:57 +0100 Subject: [erlang-questions] Variables and side-effects Message-ID: I have been reading about Erlang and I am finding a problem grasping the meaning of some terms related to variables. What do global, free and unbound variables stand for in Erlang? Do they mean the same thing? Is there a notion of global/free/unbound variables in Erlang? For instance, can I somehow implement this lambda: ?y ->* *x+y+1 ? If Erlang does not support global/ free/unbound variables, can we say that a function does not have side-effects? Thanks :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From thebullno1@REDACTED Sun Dec 30 13:15:26 2012 From: thebullno1@REDACTED (Bach Le) Date: Sun, 30 Dec 2012 20:15:26 +0800 Subject: [erlang-questions] How to distribute where_to_read node? Message-ID: Hi, I have an Mnesia ram_copies table distributed over two nodes: A and B. Node C and D joins the cluster without storing copies. mnesia:table_info(table, where_to_read) returns A and B respectively on nodes C and D. When node A is restarted, where_to_read on both nodes C and D points to node B. Is there anyway to force the reader nodes to "load-balance" their reads across nodes A and B again (preferably without restarting them)? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Sun Dec 30 14:11:18 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sun, 30 Dec 2012 22:11:18 +0900 Subject: [erlang-questions] Variables and side-effects In-Reply-To: References: Message-ID: On Sun, Dec 30, 2012 at 8:16 PM, Tyron Zerafa wrote: > I have been reading about Erlang and I am finding a problem grasping the > meaning of some terms related to variables. > > What do global, free and unbound variables stand for in Erlang? Do they mean > the same thing? > > Is there a notion of global/free/unbound variables in Erlang? For instance, > can I somehow implement this lambda: ?y -> x+y+1 ? If you store a value in the process dictionary, you have a kind of process-specific "global" x. http://www.erlang.org/course/advanced.html#dict So yes, you can side effects. Regards, Michael Turner Project Persephone 1-25-33 Takadanobaba Shinjuku-ku Tokyo 169-0075 (+81) 90-5203-8682 turner@REDACTED http://www.projectpersephone.org/ "Love does not consist in gazing at each other, but in looking outward together in the same direction." -- Antoine de Saint-Exup?ry From max.lapshin@REDACTED Sun Dec 30 15:53:29 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 30 Dec 2012 17:53:29 +0300 Subject: [erlang-questions] Why system_flag(scheduler_bind_type) is deprecated? In-Reply-To: References: <6072f3ef-54a4-4e83-bebe-62186d18cbc2@17g2000vba.googlegroups.com> <373FD3CE-B816-45AF-A97C-F1F0E3C56586@erlang.org> Message-ID: There are some builds for linux that also come without this +sbt flag. How can I understand that erlang supports this flag from shell script? If I pass this +sbt flag to an erlang that doesn't support it, it just fails. And I mention that using erlang:system_flag/2 call doesn't bring any of this problems? -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomasl_erlang@REDACTED Sun Dec 30 16:40:18 2012 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Sun, 30 Dec 2012 07:40:18 -0800 (PST) Subject: [erlang-questions] Frames proposal In-Reply-To: References: <3A1E3ECD-6E4B-42AB-94AA-13BC32CE090E@gmail.com> <1356799940.34463.YahooMailNeo@web120704.mail.ne1.yahoo.com> Message-ID: <1356882018.40327.YahooMailNeo@web120706.mail.ne1.yahoo.com> >________________________________ > From: Max Lapshin > >What is the idea to make atoms collectable? > >They are designed as a limited number of domain specific words. You cannot deal with endless amount of domain language, it is always finite. So atoms are also finite. Hi Max, I'm not sure what you mean? As far as I know, atoms can be used for many purposes (and have been, in Erlang as well as its ancestors Prolog and Lisp). However, due to the implementation limitations in Erlang/OTP some of these uses are considered bad practice, particularly for long-running servers.? For example, Yaws used to represent HTTP header names as atoms, which is nice since comparison is fast and O(1) and atoms are represented once, unlike binaries, which saves memory. This approach was replaced by using binaries for header names once it was realized incoming HTTP requests could cause ERTS to run out of atoms, by accident or on purpose. It would have been simple, easy, fast and compact to use atoms, except that atoms leaked due to not having a GC. With that said, it is of course possible to work around this. Use binaries, use list_to_existing_atom/1, use coding standards, etc. Best regards, Thomas From ulf@REDACTED Sun Dec 30 16:49:01 2012 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 30 Dec 2012 16:49:01 +0100 Subject: [erlang-questions] Variables and side-effects In-Reply-To: References: Message-ID: <43512B8A-D273-497E-A944-2A549ECC517A@feuerlabs.com> On 30 Dec 2012, at 12:16, Tyron Zerafa wrote: > I have been reading about Erlang and I am finding a problem grasping the meaning of some terms related to variables. > > What do global, free and unbound variables stand for in Erlang? Do they mean the same thing? Global variables is what Erlang doesn't have. ;-) You can simulate global variables using processes or (within the scope of a process) using the process dictionary, as Michael Turner pointed out. > Is there a notion of global/free/unbound variables in Erlang? For instance, can I somehow implement this lambda: ?y -> x+y+1 ? You can do that, by passing X to the function that creates the lambda: lambda(X) -> fun(Y) -> X + Y + 1 end. The closure will include X in its environment. Of course X will be immutable. > If Erlang does not support global/ free/unbound variables, can we say that a function does not have side-effects? You can't say that generally, since you can call side-effecting functions from any function. There is currently no way to guarantee that a function is pure, either through static or dynamic analysis. OTOH, the number of side-effecting operations in Erlang are relatively few, so it's not too hard to determine that a function is pure in most cases. Note that any function that calls other modules will have to be treated as potentially side-effecting, since the semantics of the remote call is determined at the time of the call, and dynamic code loading can completely alter the type signature and semantics of an exported function. BTW, variables are free until bound, but you cannot reference an unbound variable unless that expression also binds a value to it. In other words, an unbound variable cannot appear in the RHS of an expression. Thus, you cannot get a null reference exception in Erlang, which even Sir Anthony Hoare will agree is a Very Good Thing. [1] BR, Ulf W [1] http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From surferjeff@REDACTED Sun Dec 30 18:38:39 2012 From: surferjeff@REDACTED (Jeffrey Rennie) Date: Sun, 30 Dec 2012 09:38:39 -0800 Subject: [erlang-questions] How can a gen_server receive messages from a port? Message-ID: <04a301cde6b4$81b92a30$852b7e90$@com> I'm building a gen_server that communicates with an external process via an erlang port. Sending messages to the port is straight-forward. However, I haven't yet figured out how to get my gen_server to Receive messages from the port. I've tried the following, but to no avail: handle_cast({Port, {data, Binary}}, State) when is_port(Port); is_binary(Binary) -> Term = binary_to_term(Binary), io:format("Received data on port: ~w~n", [Term]), handle_port(Term, State); How can a gen_server receive messages from a port? Sincerely, Jeffrey Rennie From tyron.zerafa@REDACTED Sun Dec 30 18:56:03 2012 From: tyron.zerafa@REDACTED (Tyron Zerafa) Date: Sun, 30 Dec 2012 18:56:03 +0100 Subject: [erlang-questions] Variables and side-effects In-Reply-To: <43512B8A-D273-497E-A944-2A549ECC517A@feuerlabs.com> References: <43512B8A-D273-497E-A944-2A549ECC517A@feuerlabs.com> Message-ID: My idea is to serialize a function, send it over to a remote node, decode it there and execute it. I am trying to find the possible things that may go wrong in this approach. Since there is no global variables, then I only need to care for the process dictionary and ets tables. However, I am finding a LOT of operations with side effects such as mnesia, messages and dets which somehow need to be reflected on the original node. Any idea how I can go about this differently or whether I am missing anything? On Sun, Dec 30, 2012 at 4:49 PM, Ulf Wiger wrote: > > On 30 Dec 2012, at 12:16, Tyron Zerafa wrote: > > I have been reading about Erlang and I am finding a problem grasping the > meaning of some terms related to variables. > > What do global, free and unbound variables stand for in Erlang? Do they > mean the same thing? > > > Global variables is what Erlang doesn't have. ;-) > > You can simulate global variables using processes or (within the scope of > a process) using the process dictionary, as Michael Turner pointed out. > > > Is there a notion of global/free/unbound variables in Erlang? For > instance, can I somehow implement this lambda: ?y ->* *x+y+1 ? > > > > You can do that, by passing X to the function that creates the lambda: > > lambda(X) -> > fun(Y) -> > X + Y + 1 > end. > > The closure will include X in its environment. Of course X will be > immutable. > > If Erlang does not support global/ free/unbound variables, can we say that > a function does not have side-effects? > > > You can't say that generally, since you can call side-effecting functions > from any function. There is currently no way to guarantee that a function > is pure, either through static or dynamic analysis. > > OTOH, the number of side-effecting operations in Erlang are relatively > few, so it's not too hard to determine that a function is pure in most > cases. Note that any function that calls other modules will have to be > treated as potentially side-effecting, since the semantics of the remote > call is determined at the time of the call, and dynamic code loading can > completely alter the type signature and semantics of an exported function. > > BTW, variables are free until bound, but you cannot reference an unbound > variable unless that expression also binds a value to it. In other words, > an unbound variable cannot appear in the RHS of an expression. Thus, you > cannot get a null reference exception in Erlang, which even Sir Anthony > Hoare will agree is a Very Good Thing. [1] > > BR, > Ulf W > > [1] > http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare > > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > -- Best Regards, Tyron Zerafa -------------- next part -------------- An HTML attachment was scrubbed... URL: From surferjeff@REDACTED Sun Dec 30 18:57:33 2012 From: surferjeff@REDACTED (Jeffrey Rennie) Date: Sun, 30 Dec 2012 09:57:33 -0800 Subject: [erlang-questions] How can a gen_server receive messages from a port? Message-ID: <04ab01cde6b7$25904d30$70b0e790$@com> Naturally, I find the answer 20 minutes after sending the email: use handle_info(). > -----Original Message----- > From: Jeffrey Rennie [mailto:surferjeff@REDACTED] > Sent: Sunday, December 30, 2012 9:39 AM > To: 'erlang-questions@REDACTED' > Subject: How can a gen_server receive messages from a port? > > I'm building a gen_server that communicates with an external process > via an erlang port. > > Sending messages to the port is straight-forward. > > However, I haven't yet figured out how to get my gen_server to Receive > messages from the port. I've tried the following, but to no avail: > > handle_cast({Port, {data, Binary}}, State) > when is_port(Port); is_binary(Binary) -> > Term = binary_to_term(Binary), > io:format("Received data on port: ~w~n", [Term]), > handle_port(Term, State); > > How can a gen_server receive messages from a port? > > Sincerely, > Jeffrey Rennie From jeremy@REDACTED Sun Dec 30 20:48:23 2012 From: jeremy@REDACTED (Jeremy Ong) Date: Sun, 30 Dec 2012 14:48:23 -0500 Subject: [erlang-questions] Variables and side-effects In-Reply-To: References: <43512B8A-D273-497E-A944-2A549ECC517A@feuerlabs.com> Message-ID: Other possibilities for handling global variables: Using app configs (either with the config file or application:set_env/2 application:get_env/3) Defining a macro in a header file that you include in every module that needs to access the constant Returning the value in a function exported by some module (this seems strange to me). If the value is truly constant and is meaningful to a specific application, I would strongly recommend the first approach. The second approach is a bit harder to reason about and requires more resource juggling for the developer. On Sun, Dec 30, 2012 at 12:56 PM, Tyron Zerafa wrote: > My idea is to serialize a function, send it over to a remote node, decode > it there and execute it. I am trying to find the possible things that may > go wrong in this approach. > > Since there is no global variables, then I only need to care for the > process dictionary and ets tables. > However, I am finding a LOT of operations with side effects such as > mnesia, messages and dets which somehow need to be reflected on the > original node. > > Any idea how I can go about this differently or whether I am missing > anything? > > > On Sun, Dec 30, 2012 at 4:49 PM, Ulf Wiger wrote: > >> >> On 30 Dec 2012, at 12:16, Tyron Zerafa wrote: >> >> I have been reading about Erlang and I am finding a problem grasping the >> meaning of some terms related to variables. >> >> What do global, free and unbound variables stand for in Erlang? Do they >> mean the same thing? >> >> >> Global variables is what Erlang doesn't have. ;-) >> >> You can simulate global variables using processes or (within the scope of >> a process) using the process dictionary, as Michael Turner pointed out. >> >> >> Is there a notion of global/free/unbound variables in Erlang? For >> instance, can I somehow implement this lambda: ?y ->* *x+y+1 ? >> >> >> >> You can do that, by passing X to the function that creates the lambda: >> >> lambda(X) -> >> fun(Y) -> >> X + Y + 1 >> end. >> >> The closure will include X in its environment. Of course X will be >> immutable. >> >> If Erlang does not support global/ free/unbound variables, can we say >> that a function does not have side-effects? >> >> >> You can't say that generally, since you can call side-effecting functions >> from any function. There is currently no way to guarantee that a function >> is pure, either through static or dynamic analysis. >> >> OTOH, the number of side-effecting operations in Erlang are relatively >> few, so it's not too hard to determine that a function is pure in most >> cases. Note that any function that calls other modules will have to be >> treated as potentially side-effecting, since the semantics of the remote >> call is determined at the time of the call, and dynamic code loading can >> completely alter the type signature and semantics of an exported function. >> >> BTW, variables are free until bound, but you cannot reference an unbound >> variable unless that expression also binds a value to it. In other words, >> an unbound variable cannot appear in the RHS of an expression. Thus, you >> cannot get a null reference exception in Erlang, which even Sir Anthony >> Hoare will agree is a Very Good Thing. [1] >> >> BR, >> Ulf W >> >> [1] >> http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare >> >> >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> > > > -- > Best Regards, > Tyron Zerafa > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikhail.sobolev@REDACTED Mon Dec 31 03:42:15 2012 From: mikhail.sobolev@REDACTED (Mikhail Sobolev) Date: Mon, 31 Dec 2012 04:42:15 +0200 Subject: [erlang-questions] internal error in lint_module Message-ID: Hi, Today I encountered a never seen error before :) I spent about an hour to understand what's wrong and the problem disappeared after fxing a typo. A minimal example I managed to come out with is: --- cut here --- -module(p2). -export([ manifest/2 ]). manifest(Module, Name) -> fun Module:Nine/1. --- cut here --- While compiling with erlc, the following error message is produced: --- cut here --- p2.erl:none: internal error in lint_module; crash reason: {{case_clause,{location,8}}, [{erl_lint,loc,1,[{file,"erl_lint.erl"},{line,598}]}, {erl_lint,add_error,3,[{file,"erl_lint.erl"},{line,588}]}, {erl_lint,expr_var,4,[{file,"erl_lint.erl"},{line,3155}]}, {erl_lint,'-expr_list/3-fun-0-',3, [{file,"erl_lint.erl"},{line,2289}]}, {lists,foldl,3,[{file,"lists.erl"},{line,1197}]}, {erl_lint,expr,3,[{file,"erl_lint.erl"},{line,2151}]}, {erl_lint,exprs,3,[{file,"erl_lint.erl"},{line,2047}]}, {erl_lint,clause,3,[{file,"erl_lint.erl"},{line,1399}]}]} --- cut here --- If the typo -- Nine -- is fixed, everything compiles just fine. A brief look at the error message and erl_lint:598 suggest that somehow the 'file' information anticipated in the case clauses is missing. Information about the erlang used: Erlang R15B02 (erts-5.9.2) [source] [smp:2:2] [async-threads:0] [hipe] [kernel-poll:false] I'm running an Ubuntu 12.10, and the package is taken from Debian/experimental (and rebuilt for my environment) :)) I wonder if it's a bug or something else. -- Misha From gleber.p@REDACTED Mon Dec 31 10:55:05 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 31 Dec 2012 10:55:05 +0100 Subject: [erlang-questions] Running multiple commands like os:cmd/1 In-Reply-To: References: Message-ID: This would work, but since sometimes sequence of commands can change depending on results of other commands in the sequence it does not fit my case. On Fri, Dec 28, 2012 at 6:44 AM, Garrett Smith wrote: > Have you experimented with calling the shell as a single command, passing > along the string to execute? E.g. > > bash -c "" > > Garrett > > On Dec 26, 2012 2:37 PM, "Gleb Peregud" wrote: >> >> Happy holidays, everyone! >> >> I want to run multiple commands in a single shell from my code through >> some sort of open_port({spawn, ...}). I have four requirements: >> 1) I want it to work asynchronously (i.e. I start a command and I get >> it's results streamed to a process as it happens); >> 2) I want to know exactly when a command finishes; >> 3) Binary output should be handled properly; >> 4) Changes in env variables should be preserved between commands. >> >> A naive sequence of os:cmd/1 or open_port({spawn, ...}) doesn't >> satisfy requirement 4. >> >> os:cmd/1 has a nice backend code in there, but it doesn't satisfy >> requirement 3. It has a bug. Try running length(os:cmd("head -c 1000 >> /dev/urandom")). >> >> open_port({spawn, "/bin/sh -s -"}, [line]) with port_command/2 doesn't >> satisfy requirement 2. >> >> Any ideas how to implement it? Is there any library which does it? >> >> Cheers, >> Gleb >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Mon Dec 31 12:04:29 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 31 Dec 2012 14:04:29 +0300 Subject: [erlang-questions] Why system_flag(scheduler_bind_type) is deprecated? In-Reply-To: References: <6072f3ef-54a4-4e83-bebe-62186d18cbc2@17g2000vba.googlegroups.com> <373FD3CE-B816-45AF-A97C-F1F0E3C56586@erlang.org> Message-ID: Damn, this removal is really a problem. I can't use erlang:system_flag/2 call anymore, I can't use +sbt option, because it is impossible to understand if this flag is supported with current erlang build, I can't work without this flag, because by default scheduler is trying to group processes on one core and udp packets are lost. Is it possible to ignore +sbt flag, not to fail the runtime? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbothma@REDACTED Mon Dec 31 14:38:23 2012 From: jbothma@REDACTED (JD Bothma) Date: Mon, 31 Dec 2012 14:38:23 +0100 Subject: [erlang-questions] Erlang - supervised gen_server to drain off a queue In-Reply-To: <20121229073535.GA18325@sagitarius.getae.net> References: <20121229073535.GA18325@sagitarius.getae.net> Message-ID: I'd suggest subscribing to the queue with a 1-message buffer using the rabbitmq erlang client. I haven't had time to look into how to properly close the channel if your gen server crashes but I think a good enough approach is to link your gen_server to the connection pid and the channel pid - if anything crashes the gen_server can reconnect when it gets restarted by its supervisor. no polling for messages, you deal with them as they come in. So you your application top supervisor can supervise one worker - your gen_server. Simple as that. There are very few situations leading to the Erlang VM crashing, especially with such a simple system. One thing to watch out for is running out of memory. I think a common way of getting into that situation is having messages building up in the process message queue, e.g. if you receive rabbitMQ messages without requiring an ack before getting the next message, and if your gen_server can't keep up. It would really take many many messages for this to get out of hand, but do some napkin maths and be aware of it. More about rabbitmq connection cleanup that I havent had time to look into properly... http://grokbase.com/t/rabbitmq/rabbitmq-discuss/116e7des0q/linking-to-a-channel-process-erlang-client RabbitMQ erlang client including subscribing to a queue with one message buffered http://www.rabbitmq.com/erlang-client-user-guide.html On 29 December 2012 08:35, hyperboreean wrote: > Hi guys, > > I'm just trying to bring Erlang into my company - it's a hard road, but > we might eventually use it for at least some of our projects. I want to > get started by changing a small piece of code that reads an AMQP queue > and appends the contents of each message to a file. To be more specific, > there are multiple producers for that queue (which produce error > messages, if any) and there's this consumer which gets the error > messages from the queue and updates a file with them. Now, this piece of > software has always been problematic for us, for one reason or another - > it's not reliable and when it fails it might bring the AMQP broker to > its knees because of the number of messages that are not consumed. > > I'd like to change this code with some Erlang code - maybe with a > supervision tree so that if/when the server fails, it gets automatically > restarted. I was thinking of trying a gen_server, but then I figured out > that there's no client of the server, the code just has to constantly > pull data out of that queue. > > Any good hints on how to get started on this ? What's the OTP way of > doing this kind of things? > > Thank you! > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Mon Dec 31 15:32:23 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 31 Dec 2012 15:32:23 +0100 Subject: [erlang-questions] Running multiple commands like os:cmd/1 In-Reply-To: References: Message-ID: In this case you might want to put this "sequence of command" into a shell script (because they are essentially that) and execute the script. Use if, while, for, etc. in the script for controlling the execution. 2012/12/31 Gleb Peregud : > This would work, but since sometimes sequence of commands can change > depending on results of other commands in the sequence it does not fit > my case. > > On Fri, Dec 28, 2012 at 6:44 AM, Garrett Smith wrote: >> Have you experimented with calling the shell as a single command, passing >> along the string to execute? E.g. >> >> bash -c "" >> >> Garrett >> >> On Dec 26, 2012 2:37 PM, "Gleb Peregud" wrote: >>> >>> Happy holidays, everyone! >>> >>> I want to run multiple commands in a single shell from my code through >>> some sort of open_port({spawn, ...}). I have four requirements: >>> 1) I want it to work asynchronously (i.e. I start a command and I get >>> it's results streamed to a process as it happens); >>> 2) I want to know exactly when a command finishes; >>> 3) Binary output should be handled properly; >>> 4) Changes in env variables should be preserved between commands. >>> >>> A naive sequence of os:cmd/1 or open_port({spawn, ...}) doesn't >>> satisfy requirement 4. >>> >>> os:cmd/1 has a nice backend code in there, but it doesn't satisfy >>> requirement 3. It has a bug. Try running length(os:cmd("head -c 1000 >>> /dev/urandom")). >>> >>> open_port({spawn, "/bin/sh -s -"}, [line]) with port_command/2 doesn't >>> satisfy requirement 2. >>> >>> Any ideas how to implement it? Is there any library which does it? >>> >>> Cheers, >>> Gleb >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Mon Dec 31 17:23:51 2012 From: g@REDACTED (Garrett Smith) Date: Mon, 31 Dec 2012 10:23:51 -0600 Subject: [erlang-questions] Running multiple commands like os:cmd/1 In-Reply-To: References: Message-ID: +1 If you do a lot of system scripting, you might want to make bash a tool-of-choice and use Erlang for control and supervisory functions. On Mon, Dec 31, 2012 at 8:32 AM, Attila Rajmund Nohl wrote: > In this case you might want to put this "sequence of command" into a > shell script (because they are essentially that) and execute the > script. Use if, while, for, etc. in the script for controlling the > execution. > > 2012/12/31 Gleb Peregud : >> This would work, but since sometimes sequence of commands can change >> depending on results of other commands in the sequence it does not fit >> my case. >> >> On Fri, Dec 28, 2012 at 6:44 AM, Garrett Smith wrote: >>> Have you experimented with calling the shell as a single command, passing >>> along the string to execute? E.g. >>> >>> bash -c "" >>> >>> Garrett >>> >>> On Dec 26, 2012 2:37 PM, "Gleb Peregud" wrote: >>>> >>>> Happy holidays, everyone! >>>> >>>> I want to run multiple commands in a single shell from my code through >>>> some sort of open_port({spawn, ...}). I have four requirements: >>>> 1) I want it to work asynchronously (i.e. I start a command and I get >>>> it's results streamed to a process as it happens); >>>> 2) I want to know exactly when a command finishes; >>>> 3) Binary output should be handled properly; >>>> 4) Changes in env variables should be preserved between commands. >>>> >>>> A naive sequence of os:cmd/1 or open_port({spawn, ...}) doesn't >>>> satisfy requirement 4. >>>> >>>> os:cmd/1 has a nice backend code in there, but it doesn't satisfy >>>> requirement 3. It has a bug. Try running length(os:cmd("head -c 1000 >>>> /dev/urandom")). >>>> >>>> open_port({spawn, "/bin/sh -s -"}, [line]) with port_command/2 doesn't >>>> satisfy requirement 2. >>>> >>>> Any ideas how to implement it? Is there any library which does it? >>>> >>>> Cheers, >>>> Gleb >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mattevans123@REDACTED Mon Dec 31 21:11:01 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Mon, 31 Dec 2012 15:11:01 -0500 Subject: [erlang-questions] Strings and Text Processing In-Reply-To: References: , <6E1DA248-66E9-43C0-B0CE-9D95A3079C8E@masklinn.net>, <1356800348.72926.YahooMailNeo@web120705.mail.ne1.yahoo.com>, Message-ID: Really there are advantages for using both lists or binaries, but I think one of the "issues" with string handling is the need for consistent library support between the two. What would be nice is to have consistent support with the Erlang libraries to handle both types (e.g. string, re, regexp etc.). Just my 2c... > From: masklinn@REDACTED > Date: Sat, 29 Dec 2012 23:41:06 +0100 > To: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Strings and Text Processing > > On 2012-12-29, at 17:59 , Thomas Lindgren wrote: > > Good point. A strong erlang unicode library implementing the above would be very nice. > > If anybody wants to give it a shot (beware, it's not for the faint of > heart) I would strongly recommend reading Daniel Ehrenberg's > unicode-tagged posts[0], especially his 6 "Unicode implementer's guide" > (but the rest as well, the whole tag is a goldmine) before diving into > the Unicode Standard itself. > > [0] http://useless-factor.blogspot.fr/search/label/unicode > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: