From behdad.forghani@REDACTED Sun Aug 1 02:08:09 2010 From: behdad.forghani@REDACTED (Behdad Forghani) Date: Sat, 31 Jul 2010 17:08:09 -0700 Subject: gen_server Question Message-ID: <4c54baef.0626730a.603e.05d7@mx.google.com> Hello, I am writing a module that mostly monitors a directory at certain intervals and processes the files that are dropped into that directory and sends messages to another process. I want this processes to be supervised. Two options come into my mind: 1 - Write a server myself, similar to server4.erl in Joe Armstrong's "Programming Erlang" book. In this case, I will use an "after Time ->" clause. 2 - Use gen_server behavior and use timer module to send messages to myself, since, I do not see any other way to get regular timeouts. My question is, shall I try to use gen_server? Do I get anything extra by using gen_server? I do not need multicall and I can write my own terminate routine. Is there anything else that I will be missing? Thanks in advance. Behdad From nem@REDACTED Sun Aug 1 03:51:50 2010 From: nem@REDACTED (Geoff Cant) Date: Sat, 31 Jul 2010 18:51:50 -0700 Subject: [erlang-questions] gen_server Question In-Reply-To: <4c54baef.0626730a.603e.05d7@mx.google.com> (Behdad Forghani's message of "Sat, 31 Jul 2010 17:08:09 -0700") References: <4c54baef.0626730a.603e.05d7@mx.google.com> Message-ID: gen_server has exactly the functionality you need. When returning from a callback, you can specify an extra timeout duration -- instead of {noreply, State}, use {noreply, State, timer:seconds(Timeout)}. After Timeout seconds of inactivity (this timer is cancelled every time you enter a callback), you'll enter the handle_info callback as: handle_info('timeout', State). The only thing you need to keep in mind with this approach is that all your callback return tuples should include this timeout (init/call/cast/info). This approach works well if you want to ensure you wait at least Timeout seconds between each timeout. If you take a long time to process a timeout, the next one will occur that much later as the clock starts from when you return from the callback. The other main approach is timer:send_interval/2 - this requires no modification of your gen_server return tuples, and will be sent every Interval, no matter how long it takes you to process one. It's useful if you need to do something every Interval and doing a few in a row is fine if the system slows down for some reason. One problem I have with it is I do development on my laptop, and when it wakes from sleep you get a barrage of timeouts. Being something of an OTP zealot, I'd advise you to use the gen_server behaviour unless you've got a good reason not to. Cheers, -Geoff "Behdad Forghani" writes: > Hello, > > > > I am writing a module that mostly monitors a directory at certain intervals > and processes the files that are dropped into that directory and sends > messages to another process. I want this processes to be supervised. Two > options come into my mind: > > > > 1 - Write a server myself, similar to server4.erl in Joe Armstrong's > "Programming Erlang" book. In this case, I will use an "after Time ->" > clause. > > 2 - Use gen_server behavior and use timer module to send messages to myself, > since, I do not see any other way to get regular timeouts. > > > > My question is, shall I try to use gen_server? Do I get anything extra by > using gen_server? I do not need multicall and I can write my own terminate > routine. Is there anything else that I will be missing? > > > > Thanks in advance. > > Behdad > > > -- Geoff Cant From mjtruog@REDACTED Sun Aug 1 04:42:36 2010 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 31 Jul 2010 19:42:36 -0700 Subject: [erlang-questions] gen_server Question In-Reply-To: References: <4c54baef.0626730a.603e.05d7@mx.google.com> Message-ID: <4C54DF1C.1030702@gmail.com> I agree a gen_server can be very useful for this task. There also is an inotify port that might help you here: http://www.trapexit.org/forum/viewtopic.php?p=44414 On 07/31/2010 06:51 PM, Geoff Cant wrote: > gen_server has exactly the functionality you need. When returning from a > callback, you can specify an extra timeout duration -- instead of > {noreply, State}, use {noreply, State, > timer:seconds(Timeout)}. After Timeout seconds of inactivity (this > timer is cancelled every time you enter a callback), you'll enter the > handle_info callback as: handle_info('timeout', State). > > The only thing you need to keep in mind with this approach is that all > your callback return tuples should include this timeout > (init/call/cast/info). This approach works well if you want to ensure > you wait at least Timeout seconds between each timeout. If you take a > long time to process a timeout, the next one will occur that much later > as the clock starts from when you return from the callback. > > The other main approach is timer:send_interval/2 - this requires no > modification of your gen_server return tuples, and will be sent every > Interval, no matter how long it takes you to process one. It's useful if > you need to do something every Interval and doing a few in a row is fine > if the system slows down for some reason. One problem I have with it is > I do development on my laptop, and when it wakes from sleep you get a > barrage of timeouts. > > Being something of an OTP zealot, I'd advise you to use the gen_server > behaviour unless you've got a good reason not to. > > Cheers, > -Geoff > > "Behdad Forghani" writes: > > >> Hello, >> >> >> >> I am writing a module that mostly monitors a directory at certain intervals >> and processes the files that are dropped into that directory and sends >> messages to another process. I want this processes to be supervised. Two >> options come into my mind: >> >> >> >> 1 - Write a server myself, similar to server4.erl in Joe Armstrong's >> "Programming Erlang" book. In this case, I will use an "after Time ->" >> clause. >> >> 2 - Use gen_server behavior and use timer module to send messages to myself, >> since, I do not see any other way to get regular timeouts. >> >> >> >> My question is, shall I try to use gen_server? Do I get anything extra by >> using gen_server? I do not need multicall and I can write my own terminate >> routine. Is there anything else that I will be missing? >> >> >> >> Thanks in advance. >> >> Behdad >> >> >> >> > From andrew@REDACTED Sun Aug 1 08:43:23 2010 From: andrew@REDACTED (Andrew Thompson) Date: Sun, 1 Aug 2010 02:43:23 -0400 Subject: Thoughts on when to use 'infinity' timeouts for gen_server:call and friends Message-ID: <20100801064323.GA16275@hijacked.us> So, in trying to scale my application, I run into some cases where the default 5 second timeout imposed on all gen_*:call behaviours isn't enough and I get cascading timeout failures. I've noticed that most of the applications in the OTP distribution use 'infinity' as the timeout for all their :calls(). Basically my question is: does anyone have any good rules of thumb for when its OK to use a non-standard timeout? Most of my timeouts are coming from an external source (waiting on another program to do something). Should I just bulletproof certain core modules and say "these won't infinite loop, so they can take as long as they need"? Should I just do like OTP does and just stick 'infinity' in most everywhere? Thanks, Andrew From mazen.harake@REDACTED Sun Aug 1 10:14:43 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Sun, 01 Aug 2010 11:14:43 +0300 Subject: [erlang-questions] Thoughts on when to use 'infinity' timeouts for gen_server:call and friends In-Reply-To: <20100801064323.GA16275@hijacked.us> References: <20100801064323.GA16275@hijacked.us> Message-ID: <4C552CF3.3000506@erlang-solutions.com> A timeout is an assertion. If you allow timeouts then you should prepare to handle them, if you don't then put infinity. Raising the timeout is not logical because you can not be sure if it is enough... My rule of thumb is: Don't have cascading timeouts; push the "timeout" boundary as far out to the edge as possible. If it times out there then you are safe because you will crash all the way back and you can then handle that. It is far harder to handle timeouts in the "middle" in a chain of calls between processes. E.g, consider this: P1 -> P2 -> P3 -> P4 The only place where you should put a timeout (if you need one) is between P3 and P4 [in general] and you should [in general] avoid putting timeouts between P1, P2 and P3 (unless TimeoutProcesss1 > TP2 > TP3, which pretty much defeats the purpose of timeouts anyway). I think [but am not sure] that this is the reason otp does it; because they don't want random timeouts occuring everywhere in between all the interprocess communication. Hope this makes sense :D /Mazen On 01/08/2010 09:43, Andrew Thompson wrote: > So, in trying to scale my application, I run into some cases where the > default 5 second timeout imposed on all gen_*:call behaviours isn't > enough and I get cascading timeout failures. I've noticed that most of > the applications in the OTP distribution use 'infinity' as the timeout > for all their :calls(). > > Basically my question is: does anyone have any good rules of thumb for > when its OK to use a non-standard timeout? Most of my timeouts are > coming from an external source (waiting on another program to do > something). Should I just bulletproof certain core modules and say > "these won't infinite loop, so they can take as long as they need"? > Should I just do like OTP does and just stick 'infinity' in most > everywhere? > > Thanks, > > Andrew > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From vincenzo.di.somma@REDACTED Sun Aug 1 12:14:34 2010 From: vincenzo.di.somma@REDACTED (Vincenzo Di Somma) Date: Sun, 01 Aug 2010 12:14:34 +0200 Subject: Download from web. Message-ID: <1280657674.8157.0.camel@blackrain> Hi all, sorry for the newby question: given a url, is there a way to download a big file from the web and stream it directly to the filesystem? Thanks, vds -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From olopierpa@REDACTED Sun Aug 1 15:58:14 2010 From: olopierpa@REDACTED (Pierpaolo Bernardi) Date: Sun, 1 Aug 2010 15:58:14 +0200 Subject: [erlang-questions] Download from web. In-Reply-To: <1280657674.8157.0.camel@blackrain> References: <1280657674.8157.0.camel@blackrain> Message-ID: Yes. wget is one possibility. 2010/8/1, Vincenzo Di Somma : > Hi all, > sorry for the newby question: given a url, is there a way to download a > big file from the web and stream it directly to the filesystem? > Thanks, > vds > -- Inviato dal mio dispositivo mobile From ngreco@REDACTED Sun Aug 1 17:23:34 2010 From: ngreco@REDACTED (Nahuel Greco) Date: Sun, 1 Aug 2010 12:23:34 -0300 Subject: [erlang-questions] Download from web. In-Reply-To: References: <1280657674.8157.0.camel@blackrain> Message-ID: See http://www.erlang.org/doc/man/httpc.html Saludos, Nahuel Greco. On Sun, Aug 1, 2010 at 10:58 AM, Pierpaolo Bernardi wrote: > Yes. ?wget is one possibility. > > > > 2010/8/1, Vincenzo Di Somma : >> Hi all, >> sorry for the newby question: given a url, is there a way to download a >> big file from the web and stream it directly to the filesystem? >> Thanks, >> ? ? ? ? ? ? ? vds >> > > -- > Inviato dal mio dispositivo mobile > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From lucevers@REDACTED Sun Aug 1 18:58:39 2010 From: lucevers@REDACTED (Luc Evers) Date: Sun, 1 Aug 2010 18:58:39 +0200 Subject: ct_telnet example Message-ID: Hi, I'm looking for an example using the ct_telnet module. Thanks! Luc. From elliot.murphy@REDACTED Sun Aug 1 19:30:40 2010 From: elliot.murphy@REDACTED (Elliot Murphy) Date: Sun, 1 Aug 2010 13:30:40 -0400 Subject: [erlang-questions] Download from web. In-Reply-To: References: <1280657674.8157.0.camel@blackrain> Message-ID: <9107EFAF-2FB8-44F2-B689-91A68693FA46@gmail.com> ibrowse is also a popular http client library, that is what couchdb uses internally. | Elliot Murphy | https://launchpad.net/~statik/ | On Aug 1, 2010, at 11:23 AM, Nahuel Greco wrote: > See http://www.erlang.org/doc/man/httpc.html > > Saludos, > Nahuel Greco. > > > > On Sun, Aug 1, 2010 at 10:58 AM, Pierpaolo Bernardi wrote: >> Yes. wget is one possibility. >> >> >> >> 2010/8/1, Vincenzo Di Somma : >>> Hi all, >>> sorry for the newby question: given a url, is there a way to download a >>> big file from the web and stream it directly to the filesystem? >>> Thanks, >>> vds >>> >> >> -- >> Inviato dal mio dispositivo mobile >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From luismarianoguerra@REDACTED Mon Aug 2 06:13:21 2010 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Mon, 2 Aug 2010 01:13:21 -0300 Subject: [erlang-questions] efene for windows? In-Reply-To: References: Message-ID: On Tue, Jul 27, 2010 at 12:04 PM, Roman Pelvetsky wrote: > Hello, > I was very interested by release of efene - new syntax for erlang, > especially the indented one. > ( http://marianoguerra.com.ar/efene/#home ). > But is it supposed to run on Windows? > Are there some instructions how to compile and run it on windows box? > fnc.exe and build instructions: http://marianoguerra.com.ar/efene/docs/build/windows.html http://efene.tumblr.com/post/890218342/new-compiler-frontend-and-documentation http://efene.tumblr.com/post/891503620/efene-builds-on-windows-documentation tell me if you have some problem From luismarianoguerra@REDACTED Mon Aug 2 06:19:31 2010 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Mon, 2 Aug 2010 01:19:31 -0300 Subject: [erlang-questions] erlang and web applications: which framework ? In-Reply-To: References: Message-ID: On Sat, Jul 31, 2010 at 6:11 PM, Xavier Maillard wrote: > Hi, > > I am planning on developing a web application and I want it to be > written in erlang. > > So, what is the set of possibilities on the framework side ? Which one > is recommended ? > > Thank you very much I never used any of them (other than plain mochiweb) so someone else could give us some advices here, but I can list some of them here in no particular order: * Chicago Boss: http://www.chicagoboss.org/ * ErlyWeb: http://erlyweb.org/ * Nitrogen: http://nitrogenproject.com/ is there a community wiki? would be nice to make a page about web frameworks comparison so when this question is made we can point them to a complete page From andre@REDACTED Mon Aug 2 05:25:45 2010 From: andre@REDACTED (Andre Nathan) Date: Mon, 02 Aug 2010 00:25:45 -0300 Subject: [erlang-questions] gen_tcp under gen_server design In-Reply-To: References: Message-ID: <1280719545.1663.3.camel@homesick> Hi Martin On Fri, 2010-07-30 at 11:57 -0500, Logan, Martin wrote: > Andrew, BTW ? you should not have to use controlling process. Have the > listen sock spawned in your application behaviour implementation > module, then pass it through to each acceptor which blocks on accept > in the handle_info(timeout clause which is prompted by using a timeout > of 0 out of init/1. Once you accept call into the simple_one_for_one > supervisor that is responsible for spawning off your acceptors and > have it spawn another one to wait for the next connection while the > first chugs along processing on a socket it already owns by virtue of > the fact that it accepted it. There is no worry here between accepting > one connection and spawning another to accept the next because the OS > queues up pending connections of which the call to accept/1 only pulls > off the first one. > > Hope that helps. Thanks, I wouldn't have thought of doing the accept call in handle_info. Is there any advantage of spawning a new acceptor process every time a new connection arrives and do the request processing in the original process, as opposed to spawning a new worker process on each request and using a single acceptor? Thanks again, Andre From kiszl@REDACTED Mon Aug 2 09:12:44 2010 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Mon, 2 Aug 2010 09:12:44 +0200 (CEST) Subject: [erlang-questions] erlang and web applications: which framework ? In-Reply-To: References: Message-ID: <18735.194.88.55.211.1280733164.squirrel@localhost> > On Sat, Jul 31, 2010 at 6:11 PM, Xavier Maillard wrote: >> Hi, >> >> I am planning on developing a web application and I want it to be >> written in erlang. >> >> So, what is the set of possibilities on the framework side ? Which one >> is recommended ? >> >> Thank you very much > > I never used any of them (other than plain mochiweb) so someone else > could give us some advices here, but I can list some of them here in > no particular order: > > * Chicago Boss: http://www.chicagoboss.org/ > * ErlyWeb: http://erlyweb.org/ > * Nitrogen: http://nitrogenproject.com/ > > is there a community wiki? > > would be nice to make a page about web frameworks comparison so when > this question is made we can point them to a complete page > There is a feature comparison on the Chicago Boss page: http://www.chicagoboss.org/compare.html From pablo.platt@REDACTED Mon Aug 2 10:21:20 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 2 Aug 2010 01:21:20 -0700 (PDT) Subject: [erlang-questions] Which Erlang JSON parser? In-Reply-To: References: <201597.67877.qm@web63605.mail.re1.yahoo.com> Message-ID: <938303.52632.qm@web112612.mail.gq1.yahoo.com> How does it avoid the need of tagging? The readme says: >>Note: proptuple is result of passing proplist to list_to_atom bif ;) >>This is concept of mine to nicely model javascript object without a need of >>tagging structures on erlang side. But I don't understand what a proptuple is. ________________________________ From: Max Lapshin To: "Logan, Martin" Cc: alisdair sullivan ; "erlang-questions@REDACTED" Sent: Fri, July 30, 2010 9:32:50 AM Subject: Re: [erlang-questions] Which Erlang JSON parser? I've just tested http://github.com/lambder/jsonerl It has cool macros ?record_to_json ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From krab@REDACTED Mon Aug 2 10:51:03 2010 From: krab@REDACTED (Kresten Krab Thorup) Date: Mon, 2 Aug 2010 10:51:03 +0200 Subject: [erlang-questions] gen_server Question In-Reply-To: <4c54baef.0626730a.603e.05d7@mx.google.com> References: <4c54baef.0626730a.603e.05d7@mx.google.com> Message-ID: I agree with the commenters. Use generic behavior whenever there is no compelling reason to not do it. An important reason for this is that it makes the resulting code more easy to read for other erlangers, and because it provides leverage for future enhancements of the platform. One such example is the new instructions in R14 that improve performance of rpc done from inside a gen server - you don't have to do anything, but the system behaves better under stress. Likewise, the level of indirection provided by using a generic behavior could be the foundation for other future optimizations, or operational pleasantries. Kresten On Aug 1, 2010, at 2:08 , Behdad Forghani wrote: > Hello, > > > > I am writing a module that mostly monitors a directory at certain intervals > and processes the files that are dropped into that directory and sends > messages to another process. I want this processes to be supervised. Two > options come into my mind: > > > > 1 - Write a server myself, similar to server4.erl in Joe Armstrong's > "Programming Erlang" book. In this case, I will use an "after Time ->" > clause. > > 2 - Use gen_server behavior and use timer module to send messages to myself, > since, I do not see any other way to get regular timeouts. > > > > My question is, shall I try to use gen_server? Do I get anything extra by > using gen_server? I do not need multicall and I can write my own terminate > routine. Is there anything else that I will be missing? > > > > Thanks in advance. > > Behdad > > > Kresten Krab Thorup, CTO, Trifork From zabrane3@REDACTED Mon Aug 2 11:45:12 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Mon, 2 Aug 2010 11:45:12 +0200 Subject: My linked-in driver gets unloaded after a crash Message-ID: Hi guys, I've implemented a toy linked-in driver called "foo_drv". Everything works fine as soon as there's no error in the driver side nor Erlang side. But when a crash happens, its gets unloaded by the VM. $ erl Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:128] [kernel-poll:true] Eshell V5.7.5 (abort with ^G) 1> foo_drv:start(). driver successfully loaded 2> erl_ddll:loaded_drivers(). {ok,["efile","tcp_inet","udp_inet","zlib_drv", "ram_file_drv","tty_sl", "foo_drv"]} 3> 1 / a. %% simulate a crash ** exception error: bad argument in an arithmetic expression in operator '/'/2 called as 1 / a 4> erl_ddll:loaded_drivers(). {ok,["efile","tcp_inet","udp_inet","zlib_drv", "ram_file_drv","tty_sl"]} 5> q(). Could someone tell me why my driver gets unloaded after any crash while other drivers (eg. efile, zlib_drv ...) are still loaded? -- Regards Zabrane From xramtsov@REDACTED Mon Aug 2 12:31:46 2010 From: xramtsov@REDACTED (Evgeniy Khramtsov) Date: Mon, 02 Aug 2010 20:31:46 +1000 Subject: [erlang-questions] My linked-in driver gets unloaded after a crash In-Reply-To: References: Message-ID: <4C569E92.9000404@gmail.com> 02.08.2010 19:45, zabrane Mikael wrote: > Could someone tell me why my driver gets unloaded after any crash while > other drivers (eg. efile, zlib_drv ...) are still loaded? > Because a driver gets unloaded when its owner gets dead. -- Regards, Evgeniy Khramtsov, ProcessOne. xmpp:xram@REDACTED From zabrane3@REDACTED Mon Aug 2 13:24:44 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Mon, 2 Aug 2010 13:24:44 +0200 Subject: [erlang-questions] My linked-in driver gets unloaded after a crash In-Reply-To: <4C569E92.9000404@gmail.com> References: <4C569E92.9000404@gmail.com> Message-ID: Thousand thanks Evgeniy. Because a driver gets unloaded when its owner gets dead. > -- Regards Zabrane From zvi.avraham@REDACTED Mon Aug 2 14:57:13 2010 From: zvi.avraham@REDACTED (Zvi) Date: Mon, 2 Aug 2010 05:57:13 -0700 (PDT) Subject: Binary encoding for small objects: BSON or erlang:term_to_binary ? Message-ID: <7b0e79cc-8f52-4371-b561-ce2ab1596541@m17g2000prl.googlegroups.com> Hi, I need to encode/decode, small Erlang data structures to and from binaries for Soft Realtime application. This binaries will be written to files and often will be read by non- Erlang applications, written in C/C++, Java, Python, etc. Which format would you choose: BSON or erlang:term_to_binary ? ProtoBuffers ruled out because it requires schema. Thanks in Advance, Zvi From max.lapshin@REDACTED Mon Aug 2 14:59:40 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 2 Aug 2010 16:59:40 +0400 Subject: [erlang-questions] Binary encoding for small objects: BSON or erlang:term_to_binary ? In-Reply-To: <7b0e79cc-8f52-4371-b561-ce2ab1596541@m17g2000prl.googlegroups.com> References: <7b0e79cc-8f52-4371-b561-ce2ab1596541@m17g2000prl.googlegroups.com> Message-ID: On Mon, Aug 2, 2010 at 4:57 PM, Zvi wrote: > Which format would you choose: BSON or erlang:term_to_binary ? > ProtoBuffers ruled out because it requires schema. > BSON will require less modifications to external source code, but erlang:term_to_binary is much more convenient. From zvi.avraham@REDACTED Mon Aug 2 15:08:18 2010 From: zvi.avraham@REDACTED (Zvi) Date: Mon, 2 Aug 2010 06:08:18 -0700 (PDT) Subject: Binary encoding for small objects: BSON or erlang:term_to_binary ? In-Reply-To: References: <7b0e79cc-8f52-4371-b561-ce2ab1596541@m17g2000prl.googlegroups.com> Message-ID: <4dd8b0fb-d9fb-4e82-83b6-bc3d507e6844@u38g2000prh.googlegroups.com> And in term of performance on the Erlang side? I guess term_to_binary is much faster? Also BSON has 2 Erlang implementations, anyone can recommend one of them? On Aug 2, 2:59?pm, Max Lapshin wrote: > On Mon, Aug 2, 2010 at 4:57 PM, Zvi wrote: > > Which format would you choose: BSON or erlang:term_to_binary ? > > ProtoBuffers ruled out because it requires schema. > > BSON will require less modifications to external source code, but > erlang:term_to_binary is much more convenient. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED From bob@REDACTED Mon Aug 2 15:12:51 2010 From: bob@REDACTED (Bob Ippolito) Date: Mon, 2 Aug 2010 21:12:51 +0800 Subject: [erlang-questions] Binary encoding for small objects: BSON or erlang:term_to_binary ? In-Reply-To: References: <7b0e79cc-8f52-4371-b561-ce2ab1596541@m17g2000prl.googlegroups.com> Message-ID: On Monday, August 2, 2010, Max Lapshin wrote: > On Mon, Aug 2, 2010 at 4:57 PM, Zvi wrote: >> Which format would you choose: BSON or erlang:term_to_binary ? >> ProtoBuffers ruled out because it requires schema. >> > > BSON will require less modifications to external source code, but > erlang:term_to_binary is much more convenient. There are plenty of implementations of external term format in other programming languages, it's probably not any more exotic than BSON and likely to be lots faster at least on the erlang side. -bob From fdmanana@REDACTED Mon Aug 2 15:44:38 2010 From: fdmanana@REDACTED (Filipe David Manana) Date: Mon, 2 Aug 2010 14:44:38 +0100 Subject: [erlang-questions] Which Erlang JSON parser? In-Reply-To: <87eiemmzkr.fsf@benington.loc> References: <87eiemmzkr.fsf@benington.loc> Message-ID: You also have this event oriented (optionally) parser: http://github.com/Damienkatz/json_stream_parse The Erlang structures it returns after parsing are 100% equivalent to those of Mochiweb's JSON parser. It's fully written in Erlang. On Thu, Jul 29, 2010 at 10:01 AM, Alexander Kotelnikov wrote: > Hello. > > It is a terrible story. I needed a JSON parcer to deal with JSON data in > my Erlang programm. > > At first I picked json_eep > (http://github.com/jchris/erlang-json-eep-parser.git) which worked quite > fine, but later I found out that it is not able to parse (some!) escaped > unicode characters: > 28> json_eep:json_to_term("\"\\u0433\\u043e\\u0440\\u043e\\u0434\""). > ** exception error: bad argument > ? ? in function ?list_to_binary/1 > ? ? ? ?called as list_to_binary([1075,1086,1088,1086,1076]) > ? ? in call from json_grammar:yeccpars2_9/7 > ? ? in call from json_grammar:yeccpars0/2 > ? ? in call from json_eep:json_to_term/1 > > My guess is that just a little change near list_to_binary should fix the > problem. > > > Then I start investigation of other parsers. I found around 7. Most of > them not eep0018 parsers. So I tried > http://github.com/davisp/eep0018.git > and > http://github.com/dizzyd/eep0018.git (both are based on yajl). > > The former did not build for me because of some rebar issues. The latter > did after some changes to Makefiles. A little problem with it is that I > do not understand, how it decodes unicode: > 1> eep0018:json_to_term("\"\\u0433\\u043e\\u0440\\u043e\\u0434\""). > <<208,179,208,190,209,128,208,190,208,180>> > > Probably authors of these modules read this list and will clarify and/or > fix if there is something to fix. > > Thanks, A. > > PS And, just in case if anyone cares, none of these parsers implements > json_to_term/2. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Filipe David Manana, fdmanana@REDACTED "Reasonable men adapt themselves to the world. ?Unreasonable men adapt the world to themselves. ?That's why all progress depends on unreasonable men." From olivier.boudeville@REDACTED Mon Aug 2 18:11:57 2010 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Mon, 2 Aug 2010 18:11:57 +0200 Subject: rpc documentation: suggested minor improvements Message-ID: Hi, In http://www.erlang.org/doc/man/rpc.html#multicall-5 one can see as an example: """ {ResL, _} = rpc:multicall(code, load_binary, [Mod, Bin, File,]) """ whereas I suppose it should be: """ {ResL, _} = rpc:multicall(code, load_binary, [Mod, File,Bin]) """ And in the entry http://www.erlang.org/doc/man/rpc.html#eval_everywhere-4 maybe it should be clearly stated that the calls are non-blocking and done in parallel, even if it is somewhat implicit? Best regards, Olivier Boudeville. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. From tsuraan@REDACTED Mon Aug 2 20:40:47 2010 From: tsuraan@REDACTED (tsuraan) Date: Mon, 2 Aug 2010 13:40:47 -0500 Subject: epmd: error in accept Message-ID: I have a system where we have multiple erlang processes running different programs. It looks like somehow a race is happening where two epmds are started at about the same time; one of them becomes a happy and functional epmd, while the other one loops eternally printing "epmd: error in accept" to syslog, which rapidly fills up the file system where /var/log lives. Looking at the epmd_srv.c file, it doesn't look like an error on the listensock can ever cause epmd to abort, which seems like a bit of a bug. If it helps for debugging, on the system where I currently am seeing this, the functional epmd has pid 5202, while the one constants looping on its error printout has pid 5203. They must have been started at almost the same time, which may be be a rare event in general. If I can provide any more useful information, I'd be happy to. Erlang R13B4, amd64, not sure what else is useful to know. From tsuraan@REDACTED Mon Aug 2 21:43:55 2010 From: tsuraan@REDACTED (tsuraan) Date: Mon, 2 Aug 2010 14:43:55 -0500 Subject: epmd: error in accept In-Reply-To: References: Message-ID: > I have a system where we have multiple erlang processes running > different programs. ?It looks like somehow a race is happening where > two epmds are started at about the same time; one of them becomes a > happy and functional epmd, while the other one loops eternally > printing "epmd: error in accept" to syslog, which rapidly fills up the > file system where /var/log lives. ?Looking at the epmd_srv.c file, it > doesn't look like an error on the listensock can ever cause epmd to > abort, which seems like a bit of a bug. ?If it helps for debugging, on > the system where I currently am seeing this, the functional epmd has > pid 5202, while the one constants looping on its error printout has > pid 5203. ?They must have been started at almost the same time, which > may be be a rare event in general. > > If I can provide any more useful information, I'd be happy to. Looking through my messages file, I see towards the top the following pair of messages: Aug 2 09:38:08 localhost epmd: epmd: epmd running - daemon = 1 Aug 2 09:38:08 localhost epmd: epmd: epmd running - daemon = 1 After that, I got lines like the following until I ran out of drive space: Aug 2 09:38:08 localhost epmd: epmd: error in accept From vinoski@REDACTED Mon Aug 2 22:43:34 2010 From: vinoski@REDACTED (Steve Vinoski) Date: Mon, 2 Aug 2010 16:43:34 -0400 Subject: [erlang-questions] epmd: error in accept In-Reply-To: References: Message-ID: On Mon, Aug 2, 2010 at 2:40 PM, tsuraan wrote: > I have a system where we have multiple erlang processes running > different programs. ?It looks like somehow a race is happening where > two epmds are started at about the same time; one of them becomes a > happy and functional epmd, while the other one loops eternally > printing "epmd: error in accept" to syslog, which rapidly fills up the > file system where /var/log lives. ?Looking at the epmd_srv.c file, it > doesn't look like an error on the listensock can ever cause epmd to > abort, which seems like a bit of a bug. ?If it helps for debugging, on > the system where I currently am seeing this, the functional epmd has > pid 5202, while the one constants looping on its error printout has > pid 5203. ?They must have been started at almost the same time, which > may be be a rare event in general. > > If I can provide any more useful information, I'd be happy to. I recently submitted this patch that might help you: http://github.com/erlang/otp/commit/c5c8baa686b2cd5c43ecbf40fbb47d04f200b552 It looks like it's in the otp proposed updates branch but not yet on the dev branch, but either way you should be able to apply this patch to R13B04 without issue. --steve From tsuraan@REDACTED Mon Aug 2 23:11:12 2010 From: tsuraan@REDACTED (tsuraan) Date: Mon, 2 Aug 2010 16:11:12 -0500 Subject: [erlang-questions] epmd: error in accept In-Reply-To: References: Message-ID: > I recently submitted this patch that might help you: > > http://github.com/erlang/otp/commit/c5c8baa686b2cd5c43ecbf40fbb47d04f200b552 > > It looks like it's in the otp proposed updates branch but not yet on > the dev branch, but either way you should be able to apply this patch > to R13B04 without issue. That does look pretty sane. I'm building a patched R13B4, and I'll see if it fixes things. Thanks! From ok@REDACTED Tue Aug 3 01:13:36 2010 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 3 Aug 2010 11:13:36 +1200 Subject: [erlang-questions] idea: function meta data In-Reply-To: <7348e9970711160029p213783b6oe82b801e49d6b79e@mail.gmail.com> References: <837721.47969.qm@web31507.mail.mud.yahoo.com> <7348e9970711160029p213783b6oe82b801e49d6b79e@mail.gmail.com> Message-ID: <9353296C-B172-453B-8797-A66518F5593C@cs.otago.ac.nz> > > On Nov 16, 2007 4:49 AM, Vat Raghavan wrote: >> i REALLY REALLY like the idea of meta doc strings. >> one possibility for a syntax for it is like pythons', which is something like this -> >> >> def func( bar ) >> """ This is the docstring for this function >> """ >> and then as someone else said, in the string you can do : >> >> help(Module, func). and the shell emits -> >> "This is the docstring for this function" Lisp-style docstrings do not make sense for a language like ML or CAML or Clean or Haskell or Mercury or Erlang where a function (A) may have multiple clauses -- so there is no obvious unique place to *put* the docstring and (B) normally use pattern matching -- so the arguments often fail to have *names* that the docstring can talk about. We're left with the Quintus Prolog style of documentation comment, or with EDoc. Since we already *have* tools for EDoc, let's stick with that. >> >> >> then in the shell >> help(Module, func). That should be something like help(Module, Name, Arity) and there's not the least reason why that couldn't be driven off EDoc. From nem@REDACTED Tue Aug 3 05:42:26 2010 From: nem@REDACTED (Geoff Cant) Date: Mon, 02 Aug 2010 20:42:26 -0700 Subject: [erlang-questions] idea: function meta data In-Reply-To: <9353296C-B172-453B-8797-A66518F5593C@cs.otago.ac.nz> (Richard O'Keefe's message of "Tue, 3 Aug 2010 11:13:36 +1200") References: <837721.47969.qm@web31507.mail.mud.yahoo.com> <7348e9970711160029p213783b6oe82b801e49d6b79e@mail.gmail.com> <9353296C-B172-453B-8797-A66518F5593C@cs.otago.ac.nz> Message-ID: "Richard O'Keefe" writes: >>> >>> then in the shell >>> help(Module, func). > > That should be something like help(Module, Name, Arity) > and there's not the least reason why that couldn't be > driven off EDoc. It would be wonderful to get some kind of function documentation available from the erlang shell -- the information from edoc or possibly even better, type specs would be a good way to start. If the data for type specs is retained in the debug_info for a module somewhere, then it's probably not a huge effort to print out the one for the function in question if the shell is trying to tab complete an already complete function name. Cheers, -- Geoff Cant From bob@REDACTED Tue Aug 3 06:32:58 2010 From: bob@REDACTED (Bob Ippolito) Date: Tue, 3 Aug 2010 12:32:58 +0800 Subject: [erlang-questions] idea: function meta data In-Reply-To: <9353296C-B172-453B-8797-A66518F5593C@cs.otago.ac.nz> References: <837721.47969.qm@web31507.mail.mud.yahoo.com> <7348e9970711160029p213783b6oe82b801e49d6b79e@mail.gmail.com> <9353296C-B172-453B-8797-A66518F5593C@cs.otago.ac.nz> Message-ID: On Tuesday, August 3, 2010, Richard O'Keefe wrote: >> >> On Nov 16, 2007 4:49 AM, Vat Raghavan wrote: >>> i REALLY REALLY like the idea of meta doc strings. >>> one possibility for a syntax for it is like pythons', which is something like this -> >>> >>> def func( bar ) >>> """ ?This is the docstring for this function >>> """ >>> and then as someone else said, in the string you can do : >>> >>> help(Module, func). and the shell emits ?-> >>> "This is the docstring for this function" > > Lisp-style docstrings do not make sense for a language like > ML or CAML or Clean or Haskell or Mercury or Erlang where a > function (A) may have multiple clauses -- so there is no > obvious unique place to *put* the docstring and (B) normally > use pattern matching -- so the arguments often fail to have > *names* that the docstring can talk about. > > We're left with the Quintus Prolog style of documentation > comment, or with EDoc. ?Since we already *have* tools for > EDoc, let's stick with that. >>> >>> >>> then in the shell >>> help(Module, func). > > That should be something like help(Module, Name, Arity) > and there's not the least reason why that couldn't be > driven off EDoc. It'd be easier if edoc was in the BEAM which would imply keeping the comments or using an alternative syntax. Compiler directives could work but string quoting would be a chore. From icfp.publicity@REDACTED Tue Aug 3 07:18:31 2010 From: icfp.publicity@REDACTED (Wouter Swierstra) Date: Tue, 3 Aug 2010 07:18:31 +0200 Subject: ICFP 2010: Call for participation Message-ID: ===================================================================== Call for Participation The 15th ACM SIGPLAN International Conference on Functional Programming (ICFP 2010) http://www.icfpconference.org/icfp2010/ Baltimore, Maryland September 25 ? October 2 ===================================================================== ICFP 2010 provides a forum for researchers and developers to hear about the latest work on the design, implementations, principles, and uses of functional programming. The conference covers the entire spectrum of work, from practice to theory, including its peripheries. * Program: http://www.icfpconference.org/icfp2010/program.html * Invited speakers: - Mike Gordon ML: Metalanguage or Object Language? - Matthias Felleisen TeachScheme!: A Checkpoint - Guy Blelloch Functional Parallel Algorithms Schedule including related events: * September 25: Workshop on Mechanizing Metatheory (WMM) Workshop on High-Level Parallel Programming and Applications (HLPP) * September 26: Workshop on ML Workshop on Generic Programming (WGP) * September 27-29: ICFP 2010 * September 30: Haskell Symposium Erlang Workshop * October 1: Commercial Users of Functional Programming ? Day 1 (CUFP Tutorials) Haskell Implementors' Workshop * October 2: Commercial Users of Functional Programming ? Day 2 (CUFP Talks) Registration information: * Registration link: https://regmaster3.com/2010conf/ICFP10/register.php Local arrangements (including travel and accommodation): * http://www.icfpconference.org/icfp2010/local.html * Conference reservation/rate deadline: September 1st Conference organizers: * General Chair: Paul Hudak, Yale University * Program Chair: Stephanie Weirich, University of Pennsylvania * Local Arrangements Chair: Michael Hicks, University of Maryland * Workshop Co-Chairs: Derek Dreyer, Max Planck Institute for Software Systems Christopher Stone, Harvey Mudd College * Programming Contest Chair: Johannes Waldmann, Hochschule f?r Technik, Wirtschaft und Kultur, Leipzig * Video Chair: Scott Smith, Johns Hopkins University * Publicity Chair: Wouter Swierstra, Vector Fabrics ===================================================================== From noel@REDACTED Tue Aug 3 13:53:23 2010 From: noel@REDACTED (Noel Bush) Date: Tue, 3 Aug 2010 13:53:23 +0200 Subject: [erlang-questions] Question about edoc and type specifications Message-ID: On Oct 21 2008, 4:30?pm, Richard Carlsson wrote: > Taavi Talvik wrote: > > Hello! > > > Has anyone done work on edoc to use type specifications (EEP-008). > > > If not - any hints where to start. Any pointer to -spec abstract syntax? > > > Compiler /spec them and they are nicely inserted into syntax tree. And > > it would be nice to see real type specifications in doc. > > It would be nice, yes. I have planned to make edoc parse the -spec > declarations, but have not had time to work on it yet. It should be > fixed for R13 next year, though. > > ? ? /Richard I'm trying to get this to work, but it doesn't seem to. I converted all my "@spec"s to "-spec"s and ran edoc, but all the function specifications disappeared from my documentation. Likewise I can't get edoc to make anything out of "-type"s. Is there an option I need to use when invoking edoc to get this to work, or is it just still in the pipeline? Noel From noel@REDACTED Tue Aug 3 13:57:01 2010 From: noel@REDACTED (Noel Bush) Date: Tue, 3 Aug 2010 13:57:01 +0200 Subject: [erlang-questions] Question about edoc and type specifications Message-ID: On Oct 21 2008, 4:30 pm, Richard Carlsson wrote: > Taavi Talvik wrote: > > Hello! > > > Has anyone done work on edoc to use type specifications (EEP-008). > > > If not - any hints where to start. Any pointer to -spec abstract syntax? > > > Compiler /spec them and they are nicely inserted into syntax tree. And > > it would be nice to see real type specifications in doc. > > It would be nice, yes. I have planned to make edoc parse the -spec > declarations, but have not had time to work on it yet. It should be > fixed for R13 next year, though. > > /Richard I'm trying to get this to work, but it doesn't seem to. I converted all my "@spec"s to "-spec"s and ran edoc, but all the function specifications disappeared from my documentation. Likewise I can't get edoc to make anything out of "-type"s. Is there an option I need to use when invoking edoc to get this to work, or is it just still in the pipeline? Noel [Sorry if this appears twice; I sent it from the wrong address at first.] From kenneth.lundin@REDACTED Tue Aug 3 14:31:08 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 3 Aug 2010 14:31:08 +0200 Subject: [erlang-questions] Question about edoc and type specifications In-Reply-To: References: Message-ID: Hi, Support for -spec and -type in edoc is under development but is not ready for release yet. Will probably be released in R14B01 or 02. /Kenneth Erlang/OTP Ericsson On Tue, Aug 3, 2010 at 1:53 PM, Noel Bush wrote: > On Oct 21 2008, 4:30?pm, Richard Carlsson wrote: >> Taavi Talvik wrote: >> > Hello! >> >> > Has anyone done work on edoc to use type specifications (EEP-008). >> >> > If not - any hints where to start. Any pointer to -spec abstract syntax? >> >> > Compiler /spec them and they are nicely inserted into syntax tree. And >> > it would be nice to see real type specifications in doc. >> >> It would be nice, yes. I have planned to make edoc parse the -spec >> declarations, but have not had time to work on it yet. It should be >> fixed for R13 next year, though. >> >> ? ? /Richard > > I'm trying to get this to work, but it doesn't seem to. I converted > all my "@spec"s to "-spec"s and ran edoc, but all the function > specifications disappeared from my documentation. Likewise I can't get > edoc to make anything out of "-type"s. Is there an option I need to > use when invoking edoc to get this to work, or is it just still in the > pipeline? > > Noel > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From peppe@REDACTED Tue Aug 3 15:09:21 2010 From: peppe@REDACTED (Peter Andersson) Date: Tue, 03 Aug 2010 15:09:21 +0200 Subject: [erlang-questions] Sequence does't apply to sub-groups in common tests? In-Reply-To: References: Message-ID: <4C581501.40208@erix.ericsson.se> Hi, This should work just like you describe it, so we must have an error in Common Test. I'll look into it pronto! Thanks for reporting it! /Peter Ericsson AB, Erlang/OTP Dawid Figiel wrote: > Hi, > > I need some feedback, why sequence doesn't apply to sub-groups in common > tests? > > example: > > This works: > > groups() -> [{group1, [sequence], [test1, > test2]}]. > > If test1 fail then test2 will be skipped. > > And this does not work: > > groups() -> [{group1, [sequence], [{subgroup1, [], [test1]}, > {subgroup2, [], [test2]}]}] > > end_per_group(Group, Config) -> > Status = ?config(tc_group_result, Config), > case proplists:get_value(failed, Status) of > [] -> {return_group_result, ok}; > _Failed -> {return_group_result, failed} > end. > > Test1 has failed, so subgroup1 is handled as a failed too, but subgroup2 is > not skipped although we have set up 'sequence'...why ? > > According to documentation: > > "A failed sub-group (Status == failed) will cause the execution of a > sequence to fail in the same way a test case does." > > > Any clues ? > > From peppe@REDACTED Tue Aug 3 16:04:46 2010 From: peppe@REDACTED (Peter Andersson) Date: Tue, 03 Aug 2010 16:04:46 +0200 Subject: [erlang-questions] ct_telnet example In-Reply-To: References: Message-ID: <4C5821FE.8000200@erix.ericsson.se> Hi Luc, I could prepare an example for you, but before I do, did you already read these parts of the documentation? http://www.erlang.org/doc/apps/common_test/config_file_chapter.html#id2270029 http://www.erlang.org/doc/man/unix_telnet.html http://www.erlang.org/doc/man/ct_telnet.html The ftp example in the config file chapter shows how you set up an ftp connection, and you would do it in a very similar way with telnet. The only "trick" is that with ct_telnet:open/3/4 you may specify the name of a callback module that helps ct_telnet with login and detecting prompts. The default callback module is unix_telnet, which is provided with Common Test. If you need to connect to a telnet host on a different system than unix/linux, you can modify the regexp-patterns in unix_telnet and pass the name of your modified module to ct_telnet:open for opening the connection. This is really the only thing that is handled differently compared to e.g. ct_ftp and ct_ssh. Tip: To test if/how it works, you can start Common Test in interactive mode (run_test -shell) and call each function from the shell prompt. /Peter Ericsson AB, Erlang/OTP Luc Evers wrote: > Hi, > > I'm looking for an example using the ct_telnet module. > > > Thanks! > > Luc. > > From zvi.avraham@REDACTED Tue Aug 3 16:59:10 2010 From: zvi.avraham@REDACTED (Zvi) Date: Tue, 3 Aug 2010 07:59:10 -0700 (PDT) Subject: Reading JPEG images in Erlang Message-ID: <7df337ad-2e05-4d16-b062-a23e6bb1e0ac@q35g2000yqn.googlegroups.com> Hi, I trying to parse JPEG file in Erlang. I tried jungerl erl_img library, it seems to be working for other image formats, but not for JPEG. I prefer native Erlang code, but if not available, I would use binding to some C library. Any pointers? Thanks in advance, Zvi From noel@REDACTED Tue Aug 3 17:04:16 2010 From: noel@REDACTED (Noel Bush) Date: Tue, 3 Aug 2010 17:04:16 +0200 Subject: [erlang-questions] Question about edoc and type specifications In-Reply-To: References: Message-ID: On Tue, Aug 3, 2010 at 2:31 PM, Kenneth Lundin wrote: > Hi, > > Support for -spec and -type in edoc is under development but is not > ready for release yet. > Will probably be released in R14B01 or 02. > > /Kenneth Erlang/OTP Ericsson Hi Kenneth, Is there a github branch I could follow to try things out? Or is it still too early? Noel From peppe@REDACTED Tue Aug 3 17:31:48 2010 From: peppe@REDACTED (Peter Andersson) Date: Tue, 03 Aug 2010 17:31:48 +0200 Subject: [erlang-questions] ct_telnet example In-Reply-To: References: Message-ID: <4C583664.5020103@erix.ericsson.se> Hi Luc, Here's an example suite for you: suite() -> [{require, host}, {require, telnet_conn, host}, {default_config, host, [{telnet,"myhost"}, {username,"tester"}, {password,"letmein"}]}]. all() -> [global, multiple, simple]. %% open a connection with a registered name (telnet_conn) init_per_testcase(global, Config) -> {ok,_} = ct_telnet:open(telnet_conn, telnet, unix_telnet), Config; %% open multiple connections (no connection name specified) init_per_testcase(multiple, Config) -> {ok,H1} = ct_telnet:open(host, telnet, unix_telnet), {ok,H2} = ct_telnet:open(host, telnet, unix_telnet), [{handles,[H1,H2]}|Config]; init_per_testcase(_, Config) -> Config. end_per_testcase(global, _Config) -> ok = ct_telnet:close(telnet_conn); end_per_testcase(multiple, Config) -> [ok = ct_telnet:close(H) || H <- proplists:get_value(handles, Config)], ok; end_per_testcase(_, _) -> ok. global(_) -> {ok,_Result} = ct_telnet:cmd(telnet_conn, "ls"), ok. multiple(Config) -> [{ok,_Result} = ct_telnet:cmd(H, "ls") || H <- proplists:get_value(handles, Config)], ok. %% with alias name == callback module, a registered connection can %% be opened with ct_telnet:open/1 simple() -> [{require, unix_telnet, unix}, {default_config, unix, [{telnet,"myhost"}, {username,"tester"}, {password,"letmein"}]}]. simple(_) -> {ok,_} = ct_telnet:open(unix_telnet), {ok,_Result} = ct_telnet:cmd(unix_telnet, "ls"), ct_telnet:close(unix_telnet), ok. /Peter Ericsson AB, Erlang/OTP Luc Evers wrote: > Hi, > > I'm looking for an example using the ct_telnet module. > > > Thanks! > > Luc. > > From bob@REDACTED Tue Aug 3 17:38:17 2010 From: bob@REDACTED (Bob Ippolito) Date: Tue, 3 Aug 2010 23:38:17 +0800 Subject: [erlang-questions] Reading JPEG images in Erlang In-Reply-To: <7df337ad-2e05-4d16-b062-a23e6bb1e0ac@q35g2000yqn.googlegroups.com> References: <7df337ad-2e05-4d16-b062-a23e6bb1e0ac@q35g2000yqn.googlegroups.com> Message-ID: On Tue, Aug 3, 2010 at 10:59 PM, Zvi wrote: > Hi, > > I trying to parse JPEG file in Erlang. I tried jungerl erl_img > library, it seems to be working for other image formats, but not for > JPEG. > I prefer native Erlang code, but if not available, I would use binding > to some C library. > Any pointers? Try using a different one, some of the versions of erl_img out there e just broken. I think this is the branch we use internally: http://github.com/emad/erl_img -bob From zvi.avraham@REDACTED Tue Aug 3 18:03:36 2010 From: zvi.avraham@REDACTED (Zvi) Date: Tue, 3 Aug 2010 09:03:36 -0700 (PDT) Subject: Reading JPEG images in Erlang In-Reply-To: References: <7df337ad-2e05-4d16-b062-a23e6bb1e0ac@q35g2000yqn.googlegroups.com> Message-ID: <8dc8abd4-d9df-42d9-9da8-0c0233c2e1cc@o19g2000yqb.googlegroups.com> On Aug 3, 5:38?pm, Bob Ippolito wrote: > On Tue, Aug 3, 2010 at 10:59 PM, Zvi wrote: > > Hi, > > > I trying to parse JPEG file in Erlang. I tried jungerl erl_img > > library, it seems to be working for other image formats, but not for > > JPEG. > > I prefer native Erlang code, but if not available, I would use binding > > to some C library. > > Any pointers? > > Try using a different one, some of the versions of erl_img out there e > just broken. I think this is the branch we use internally:http://github.com/emad/erl_img > > -bob This is the one I using. The problem is, unlike for other file formats, like PNG or GIF, it doesn't load pixels. For example: 1> erl_img:load("/home/me/some.jpg"). {ok,{erl_image,image_jpeg,undefined, "/home/me/some.jpg",undefined,undefined, undefined,undefined,[],undefined,200,200,8,3,1,[], upper_left,undefined,[]}} Compare it to PNG: 2> erl_img:load("/home/me/some.png"). {ok,{erl_image,image_png,undefined, "/home/me/some.png",undefined,undefined, undefined,undefined,[],r8g8b8a8,200,200,8,3,1, [{'Physical',{5906,5906,meter}}, {'ColorType',6}, {'Compression',0}, {'Filter',0}, {'Interlace',0}], left_to_right,undefined, [{erl_pixmap,0,0,200,200,undefined,r8g8b8a8,...}]}} From lucevers@REDACTED Tue Aug 3 18:28:50 2010 From: lucevers@REDACTED (Luc Evers) Date: Tue, 3 Aug 2010 18:28:50 +0200 Subject: [erlang-questions] ct_telnet example In-Reply-To: <4C583664.5020103@erix.ericsson.se> References: <4C583664.5020103@erix.ericsson.se> Message-ID: Peter, Thanks a lot for your help! Luc. On Tue, Aug 3, 2010 at 5:31 PM, Peter Andersson wrote: > Hi Luc, > > Here's an example suite for you: > > suite() -> > [{require, host}, > {require, telnet_conn, host}, > {default_config, host, [{telnet,"myhost"}, > {username,"tester"}, > {password,"letmein"}]}]. > > all() -> [global, multiple, simple]. > > %% open a connection with a registered name (telnet_conn) > init_per_testcase(global, Config) -> > {ok,_} = ct_telnet:open(telnet_conn, telnet, unix_telnet), > Config; > > %% open multiple connections (no connection name specified) > init_per_testcase(multiple, Config) -> > {ok,H1} = ct_telnet:open(host, telnet, unix_telnet), > {ok,H2} = ct_telnet:open(host, telnet, unix_telnet), > [{handles,[H1,H2]}|Config]; > > init_per_testcase(_, Config) -> > Config. > > end_per_testcase(global, _Config) -> > ok = ct_telnet:close(telnet_conn); > > end_per_testcase(multiple, Config) -> > [ok = ct_telnet:close(H) || > H <- proplists:get_value(handles, Config)], > ok; > > end_per_testcase(_, _) -> > ok. > > global(_) -> > {ok,_Result} = ct_telnet:cmd(telnet_conn, "ls"), > ok. > > multiple(Config) -> > [{ok,_Result} = ct_telnet:cmd(H, "ls") || > H <- proplists:get_value(handles, Config)], > ok. > > %% with alias name == callback module, a registered connection can > %% be opened with ct_telnet:open/1 > > simple() -> > [{require, unix_telnet, unix}, > {default_config, unix, [{telnet,"myhost"}, > {username,"tester"}, > {password,"letmein"}]}]. > > simple(_) -> > {ok,_} = ct_telnet:open(unix_telnet), > {ok,_Result} = ct_telnet:cmd(unix_telnet, "ls"), > ct_telnet:close(unix_telnet), > ok. > > > /Peter > > Ericsson AB, Erlang/OTP > > > Luc Evers wrote: > > Hi, > > > > I'm looking for an example using the ct_telnet module. > > > > > > Thanks! > > > > Luc. > > > > > > From tony@REDACTED Tue Aug 3 18:28:42 2010 From: tony@REDACTED (Tony Rogvall) Date: Tue, 3 Aug 2010 18:28:42 +0200 Subject: [erlang-questions] Re: Reading JPEG images in Erlang In-Reply-To: <8dc8abd4-d9df-42d9-9da8-0c0233c2e1cc@o19g2000yqb.googlegroups.com> References: <7df337ad-2e05-4d16-b062-a23e6bb1e0ac@q35g2000yqn.googlegroups.com> <8dc8abd4-d9df-42d9-9da8-0c0233c2e1cc@o19g2000yqb.googlegroups.com> Message-ID: Hi! I have written a JPEG reader in Erlang (for erl_img), but as you noticed it is not yet commited. It currently only support the basic format but I will see if I can polish it a bit to handle more types of JPEG's. It's not very fast but I have seen DCT implemented in openCL that can be handy if one uses the openCL binding (git@REDACTED:tonyrog/cl.git ;-) /Tony On 3 aug 2010, at 18.03, Zvi wrote: > > > On Aug 3, 5:38 pm, Bob Ippolito wrote: >> On Tue, Aug 3, 2010 at 10:59 PM, Zvi wrote: >>> Hi, >> >>> I trying to parse JPEG file in Erlang. I tried jungerl erl_img >>> library, it seems to be working for other image formats, but not for >>> JPEG. >>> I prefer native Erlang code, but if not available, I would use binding >>> to some C library. >>> Any pointers? >> >> Try using a different one, some of the versions of erl_img out there e >> just broken. I think this is the branch we use internally:http://github.com/emad/erl_img >> >> -bob > > This is the one I using. > The problem is, unlike for other file formats, like PNG or GIF, it > doesn't load pixels. > For example: > > 1> erl_img:load("/home/me/some.jpg"). > {ok,{erl_image,image_jpeg,undefined, > "/home/me/some.jpg",undefined,undefined, > undefined,undefined,[],undefined,200,200,8,3,1,[], > upper_left,undefined,[]}} > > Compare it to PNG: > > 2> erl_img:load("/home/me/some.png"). > {ok,{erl_image,image_png,undefined, > "/home/me/some.png",undefined,undefined, > undefined,undefined,[],r8g8b8a8,200,200,8,3,1, > [{'Physical',{5906,5906,meter}}, > {'ColorType',6}, > {'Compression',0}, > {'Filter',0}, > {'Interlace',0}], > left_to_right,undefined, > [{erl_pixmap,0,0,200,200,undefined,r8g8b8a8,...}]}} > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From kenneth.lundin@REDACTED Tue Aug 3 20:35:48 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 3 Aug 2010 20:35:48 +0200 Subject: [erlang-questions] Thoughts on when to use 'infinity' timeouts for gen_server:call and friends In-Reply-To: <20100801064323.GA16275@hijacked.us> References: <20100801064323.GA16275@hijacked.us> Message-ID: On Sun, Aug 1, 2010 at 8:43 AM, Andrew Thompson wrote: > So, in trying to scale my application, I run into some cases where the > default 5 second timeout imposed on all gen_*:call behaviours isn't > enough and I get cascading timeout failures. I've noticed that most of > the applications in the OTP distribution use 'infinity' as the timeout > for all their :calls(). > > Basically my question is: does anyone have any good rules of thumb for > when its OK to use a non-standard timeout? Most of my timeouts are > coming from an external source (waiting on another program to do > something). Should I just bulletproof certain core modules and say > "these won't infinite loop, so they can take as long as they need"? > Should I just do like OTP does and just stick 'infinity' in most > everywhere? > > Thanks, > > Andrew > The default timeout of 5 seconds is unfortunate but was motivated at the time when gen_server was implemented and when the erlang:monitor BIF was not available. The 5 sec timeout was a compromise in order to save the caller if the server process did not exist or died in the middle of handling the call. It was however impossible to distinguish between the case where the handling of the call took longer than the timeout and when the server crashed during the handling of the call. Now when we have erlang:monitor it is possible to return to the caller immediately if the server does not exist or crashes during handling of the call. The caller can also see the difference between a timeout and a crash. Because of backward compatibility reasons we have not been able to change the default timeout for gen_server:call/2 to infinity (it would most probably break old OTP code as well). We are however seriously considering to deprecate gen_server:call/2 and recommend all users to use gen_server:call/3 with infinity timeout for all calls except some rare occasions where it really is motivated to have a timeout and the user is prepared to handle all the potential problems and raises with that. If we can find a suitable name we could also introduce new functions with arity 2 and 3 replacing gen_server:call/2 and 3. Some suggestions on names are: call2 call_ex (following the style in the Windows API) call_nt/2 (nt standing for no timeout, but does that make sense for a call_nt/3 where the third argument is the timout in seconds or infinity) Suggestions on names are welcome. /Kenneth Erlang/OTP, Ericsson > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From andre@REDACTED Wed Aug 4 00:09:41 2010 From: andre@REDACTED (Andre Nathan) Date: Tue, 03 Aug 2010 19:09:41 -0300 Subject: gen_fsm to parse binary protocol frames Message-ID: <1280873381.1639.18.camel@homesick> Hello I'm dealing with a simple protocol whose frames consist of a fixed-size header, which contains a length field which specifies the size of the message following it. I implemented a gen_fsm to handle this, using two states, assemble_header and assemble_message. Each state responds to a 'data' event whenever new data arrives from a socket (this data is sent from a separate gen_server process). The assemble_header state responds to the header_complete event switching to assemble_message, and assemble_message responds to a message_complete event by starting message processing. My problem is the following. Since data comes from a socket, I don't have control of how much data is read. It's possible, then, that a whole frame (or more) has been already buffered when it's sent to the FSM. This would cause the FSM to parse the header and switch to the assemble_message state, but there will be no more messages to trigger the 'data' event, since the whole frame has already been sent. To solve this, I'm doing calling assemble_message({data, Rest}, State) directly from the assemble_header state when this situation is detected. That is, I'm forcing a state switch and triggering an event so that the data I've already received can be processed. This feels kinda ugly... So my question is, is the gen_fsm the preferred way to do this kind of thing? If so, is there a cleaner solution to the problem which wouldn't require the gen_server which feeds the data to the gen_fsm to know about the protocol message format? >From a little discussion on IRC, it seems that doing this processing in the gen_server process would be a simpler solution. Is this what people generally do? Thanks in advance, Andre From sgolovan@REDACTED Wed Aug 4 09:54:26 2010 From: sgolovan@REDACTED (Sergei Golovan) Date: Wed, 4 Aug 2010 11:54:26 +0400 Subject: Where to report a security bug? Message-ID: Hi! Max Vozeler from Debian has audited epmd code and found a few bugs which may have a security impact. Is there an address where we could report these issues without premature public disclosure? Cheers! -- Sergei Golovan From bgustavsson@REDACTED Wed Aug 4 10:53:10 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 4 Aug 2010 10:53:10 +0200 Subject: [erlang-questions] Where to report a security bug? In-Reply-To: References: Message-ID: On Wed, Aug 4, 2010 at 9:54 AM, Sergei Golovan wrote: > Hi! > > Max Vozeler from Debian has audited epmd code and found a few bugs > which may have a security impact. Is there an address where we could > report these issues without premature public disclosure? > Send them to bjorn@REDACTED -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From sgolovan@REDACTED Wed Aug 4 10:55:59 2010 From: sgolovan@REDACTED (Sergei Golovan) Date: Wed, 4 Aug 2010 12:55:59 +0400 Subject: [erlang-questions] Where to report a security bug? In-Reply-To: References: Message-ID: 2010/8/4 Bj?rn Gustavsson : > On Wed, Aug 4, 2010 at 9:54 AM, Sergei Golovan wrote: >> Hi! >> >> Max Vozeler from Debian has audited epmd code and found a few bugs >> which may have a security impact. Is there an address where we could >> report these issues without premature public disclosure? >> > > Send them to bjorn@REDACTED I've already sent them to Joe Armstrong and Kenneth Lundin. Cheers! -- Sergei Golovan From ulf.wiger@REDACTED Wed Aug 4 12:57:22 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 04 Aug 2010 12:57:22 +0200 Subject: [erlang-questions] Thoughts on when to use 'infinity' timeouts for gen_server:call and friends In-Reply-To: References: <20100801064323.GA16275@hijacked.us> Message-ID: <4C594792.1050300@erlang-solutions.com> On 08/03/2010 08:35 PM, Kenneth Lundin wrote: > > If we can find a suitable name we could also introduce new functions > with arity 2 and 3 > replacing gen_server:call/2 and 3. > > Some suggestions on names are: > call2 > call_ex (following the style in the Windows API) > call_nt/2 (nt standing for no timeout, but does that make sense for a > call_nt/3 where the third > argument is the timout in seconds or infinity) > > Suggestions on names are welcome. gen_server:rpc/2? (It might confuse some into thinking that it is only used for distributed communication, but this should pass quickly. :) gen_server:req/2 (as in 'request')? or... gen_server:synch/2 gen_server:asynch/2 (corresponding to cast/2) BR, Ulf W From alex.arnon@REDACTED Wed Aug 4 14:04:18 2010 From: alex.arnon@REDACTED (Alex Arnon) Date: Wed, 4 Aug 2010 15:04:18 +0300 Subject: [erlang-questions] gen_fsm to parse binary protocol frames In-Reply-To: <1280873381.1639.18.camel@homesick> References: <1280873381.1639.18.camel@homesick> Message-ID: I think that indeed for handling protocol endpoints, using gen_server or a handcrafted process is common. On Wed, Aug 4, 2010 at 1:09 AM, Andre Nathan wrote: > Hello > > I'm dealing with a simple protocol whose frames consist of a fixed-size > header, which contains a length field which specifies the size of the > message following it. > > I implemented a gen_fsm to handle this, using two states, > assemble_header and assemble_message. Each state responds to a 'data' > event whenever new data arrives from a socket (this data is sent from a > separate gen_server process). The assemble_header state responds to the > header_complete event switching to assemble_message, and > assemble_message responds to a message_complete event by starting > message processing. > > My problem is the following. Since data comes from a socket, I don't > have control of how much data is read. It's possible, then, that a whole > frame (or more) has been already buffered when it's sent to the FSM. > This would cause the FSM to parse the header and switch to the > assemble_message state, but there will be no more messages to trigger > the 'data' event, since the whole frame has already been sent. > > To solve this, I'm doing calling assemble_message({data, Rest}, State) > directly from the assemble_header state when this situation is detected. > That is, I'm forcing a state switch and triggering an event so that the > data I've already received can be processed. This feels kinda ugly... > > So my question is, is the gen_fsm the preferred way to do this kind of > thing? If so, is there a cleaner solution to the problem which wouldn't > require the gen_server which feeds the data to the gen_fsm to know about > the protocol message format? > > >From a little discussion on IRC, it seems that doing this processing in > the gen_server process would be a simpler solution. Is this what people > generally do? > > Thanks in advance, > Andre > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From bile@REDACTED Wed Aug 4 15:18:20 2010 From: bile@REDACTED (bile@REDACTED) Date: Wed, 4 Aug 2010 09:18:20 -0400 Subject: [erlang-questions] Thoughts on when to use 'infinity' timeouts for gen_server:call and friends In-Reply-To: <4C594792.1050300@erlang-solutions.com> References: <20100801064323.GA16275@hijacked.us> <4C594792.1050300@erlang-solutions.com> Message-ID: <20100804091820.6800548f@otis> I understand it's a drastic change but couldn't it be shouted from the rooftops that it's being made and keep the functions the same? Or provide a global switch to revert back from infinity to the old default? If it's a better way of doing things providing a new API will probably take years to take hold. All those that explicitly use infinity have no incentive to change it and all the tutorials/books that exist won't be using it. If it was a global flag then at most you have to do when moving to R15 or whatever is add commandline arg or such. For those outdated tutorials which refer to the default timeout so long as it's made clear it changed and why in the docs for the next few releases I'd think people would be OK. On Wed, 04 Aug 2010 12:57:22 +0200 Ulf Wiger wrote: > On 08/03/2010 08:35 PM, Kenneth Lundin wrote: > > > > If we can find a suitable name we could also introduce new functions > > with arity 2 and 3 > > replacing gen_server:call/2 and 3. > > > > Some suggestions on names are: > > call2 > > call_ex (following the style in the Windows API) > > call_nt/2 (nt standing for no timeout, but does that make sense for > > a call_nt/3 where the third > > argument is the timout in seconds or infinity) > > > > Suggestions on names are welcome. > > gen_server:rpc/2? > > (It might confuse some into thinking that it is only used for > distributed communication, but this should pass quickly. :) > > gen_server:req/2 (as in 'request')? > > or... > > gen_server:synch/2 > gen_server:asynch/2 (corresponding to cast/2) > > BR, > Ulf W > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From vladdu55@REDACTED Wed Aug 4 18:34:40 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 4 Aug 2010 18:34:40 +0200 Subject: Custom module attributes Message-ID: Hi! Does anyone know why are custom module attributes only allowed before all function definitions? I would like to use some attributes to annotate functions, just like -spec does. best regards, Vlad From pguyot@REDACTED Wed Aug 4 18:59:35 2010 From: pguyot@REDACTED (Paul Guyot) Date: Wed, 4 Aug 2010 18:59:35 +0200 Subject: Hipe and memory alignment (Yikes! erts_alloc() returned misaligned address) Message-ID: <33495D81-33AC-4AC8-9394-6377C01760CD@kallisys.net> Hello, When running erl with native (hipe) code on some platforms (FreeBSD amd64), one quickly encounters messages such as: Yikes! erts_alloc() returned misaligned address 0x8016a512c This message seems to come from hipe_bifs:alloc_data/2. The allocator is the system allocator, which on unix platform is malloc, and malloc does not necessarily return aligned blocks (on MacOS X, it does, but not on FreeBSD). Does it really matter? (hipe_bifs:alloc_data/2 is only called from hipe_unified_loader:create_data_segment/3). If it doesn't, can we remove the log line? If it does, can we replace the allocator with another one that always return aligned blocks? Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From noel@REDACTED Wed Aug 4 19:05:15 2010 From: noel@REDACTED (Noel Bush) Date: Wed, 4 Aug 2010 19:05:15 +0200 Subject: record type as variable Message-ID: Is there any way to specify the type of a record using a variable? I want to be able to do this sort of thing: create(Type, Name) -> #Type{name = Name}. (Obviously, I'm responsible for making sure that Type corresponds to a record type, that any record type has a name field, etc.) It doesn't seem like that much of a pain to write a few record-handling functions that would allow me to do this, but it would be great to find out that someone's already done the work. Thanks, Noel From brady.mccary@REDACTED Wed Aug 4 19:31:53 2010 From: brady.mccary@REDACTED (Brady McCary) Date: Wed, 4 Aug 2010 12:31:53 -0500 Subject: [erlang-questions] record type as variable In-Reply-To: References: Message-ID: Noel, The record syntax is syntactic sugar on top of the tuple syntax. When erlang source is read, the record syntax is mechanically transformed into a tuple. Because records are implemented in this way at compile time, there is no way for the compiler to know the value of Type. So the answer is no. Brady On Wed, Aug 4, 2010 at 12:05 PM, Noel Bush wrote: > Is there any way to specify the type of a record using a variable? I > want to be able to do this sort of thing: > > create(Type, Name) -> > ? ?#Type{name = Name}. > > (Obviously, I'm responsible for making sure that Type corresponds to a > record type, that any record type has a name field, etc.) It doesn't > seem like that much of a pain to write a few record-handling functions > that would allow me to do this, but it would be great to find out that > someone's already done the work. > > Thanks, > Noel > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From andre@REDACTED Wed Aug 4 21:38:27 2010 From: andre@REDACTED (Andre Nathan) Date: Wed, 04 Aug 2010 16:38:27 -0300 Subject: [erlang-questions] gen_tcp under gen_server design In-Reply-To: References: Message-ID: <1280950707.629.23.camel@andre.mz.digirati.com.br> On Fri, 2010-07-30 at 11:57 -0500, Logan, Martin wrote: > Once you accept call into the simple_one_for_one supervisor that is > responsible for spawning off your acceptors and have it spawn another > one to wait for the next connection while the first chugs along > processing on a socket it already owns by virtue of the fact that it > accepted it. There is no worry here between accepting one connection > and spawning another to accept the next because the OS queues up > pending connections of which the call to accept/1 only pulls off the > first one. How do you handle the case where this newly spawned acceptor dies? One would want to assert that there's always one acceptor blocked in gen_tcp:accept and that it is restarted if it dies, so that new connections are not refused. The processes that return from accept, though, shouldn't be restarted because if they die their connection will be closed, so it doesn't make sense to restart them. Since they're all under the same simple_one_for_one supervisor, I'm not sure how to specify that only the currently blocked acceptor must be restarted. I guess that if I need this level of control, I should keep a 'permanent' acceptor under one supervisor and spawn workers as needed from a simple_one_for_one supervisor, right? This would be the opposite of the "spawn a new acceptor" idiom that you describe in your book. Thanks, Andre From noel@REDACTED Wed Aug 4 22:27:18 2010 From: noel@REDACTED (Noel Bush) Date: Wed, 4 Aug 2010 22:27:18 +0200 Subject: [erlang-questions] record type as variable In-Reply-To: References: Message-ID: Tino Breddin pointed me to parse_trans, which creates record access functions and looks like it will be very useful for me. I know that the record syntax is syntactic sugar, so I guess my imaginary example was formed incorrectly. But it looks like parse_trans will let me do stuff like '#new-'(r) to make a new record of type r, and so forth. Pretty cool. parse_trans is here: http://github.com/esl/parse_trans Noel On Wed, Aug 4, 2010 at 7:31 PM, Brady McCary wrote: > Noel, > > The record syntax is syntactic sugar on top of the tuple syntax. When > erlang source is read, the record syntax is mechanically transformed > into a tuple. Because records are implemented in this way at compile > time, there is no way for the compiler to know the value of Type. > > So the answer is no. > > Brady > > On Wed, Aug 4, 2010 at 12:05 PM, Noel Bush wrote: >> Is there any way to specify the type of a record using a variable? I >> want to be able to do this sort of thing: >> >> create(Type, Name) -> >> ? ?#Type{name = Name}. >> >> (Obviously, I'm responsible for making sure that Type corresponds to a >> record type, that any record type has a name field, etc.) It doesn't >> seem like that much of a pain to write a few record-handling functions >> that would allow me to do this, but it would be great to find out that >> someone's already done the work. >> >> Thanks, >> Noel >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From mikpe@REDACTED Wed Aug 4 23:19:31 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Wed, 4 Aug 2010 23:19:31 +0200 Subject: [erlang-questions] Hipe and memory alignment (Yikes! erts_alloc() returned misaligned address) In-Reply-To: <33495D81-33AC-4AC8-9394-6377C01760CD@kallisys.net> References: <33495D81-33AC-4AC8-9394-6377C01760CD@kallisys.net> Message-ID: <19545.55651.737243.453894@pilspetsen.it.uu.se> Paul Guyot writes: > Hello, > > When running erl with native (hipe) code on some platforms (FreeBSD amd64), one quickly encounters messages such as: > > Yikes! erts_alloc() returned misaligned address 0x8016a512c > > This message seems to come from hipe_bifs:alloc_data/2. The allocator is the system allocator, which on unix platform is malloc, and malloc does not necessarily return aligned blocks (on MacOS X, it does, but not on FreeBSD). > > Does it really matter? (hipe_bifs:alloc_data/2 is only called from hipe_unified_loader:create_data_segment/3). > If it doesn't, can we remove the log line? > If it does, can we replace the allocator with another one that always return aligned blocks? Of course alignment matters, otherwise you'd only be able to store char arrays in malloc:d areas without risking alignment faults or poor performance. That's true even on x86/amd64. Upper levels (hipe_unified_loader and initially the hipe compiler) describe the required alignment and size of the data segment, and the alignment is a function of the type of data stored there. Now, it is possible that the alignment is wrong, esp. if the data segment for whatever reason turns out to be tiny. Please rebuild the erlang VM with the following patch and do what you normally do to trigger the Yikes! message. It should tell us a little bit more about the allocation parameters. FWIW, I've never heard of anyone being able to trigger this warning before. /Mikael --- otp_src_R14A/erts/emulator/hipe/hipe_bif0.c.~1~ 2010-02-19 19:04:06.000000000 +0100 +++ otp_src_R14A/erts/emulator/hipe/hipe_bif0.c 2010-08-04 23:02:40.000000000 +0200 @@ -442,7 +442,8 @@ BIF_RETTYPE hipe_bifs_alloc_data_2(BIF_A nrbytes = unsigned_val(BIF_ARG_2); block = erts_alloc(ERTS_ALC_T_HIPE, nrbytes); if ((unsigned long)block & (align-1)) - fprintf(stderr, "Yikes! erts_alloc() returned misaligned address %p\r\n", block); + fprintf(stderr, "Yikes! erts_alloc() returned %p for align %lu nrbytes %lu\r\n", + block, (unsigned long)align, (unsigned long)nrbytes); BIF_RET(address_to_term(block, BIF_P)); } From pguyot@REDACTED Wed Aug 4 23:50:53 2010 From: pguyot@REDACTED (Paul Guyot) Date: Wed, 4 Aug 2010 23:50:53 +0200 Subject: [erlang-questions] Hipe and memory alignment (Yikes! erts_alloc() returned misaligned address) In-Reply-To: <19545.55651.737243.453894@pilspetsen.it.uu.se> References: <33495D81-33AC-4AC8-9394-6377C01760CD@kallisys.net> <19545.55651.737243.453894@pilspetsen.it.uu.se> Message-ID: <8E2689F9-91F4-43AB-9D6B-7F29B9584AB0@kallisys.net> Le 4 ao?t 2010 ? 23:19, Mikael Pettersson a ?crit : > Now, it is possible that the alignment is wrong, esp. if the data > segment for whatever reason turns out to be tiny. Please rebuild > the erlang VM with the following patch and do what you normally do > to trigger the Yikes! message. It should tell us a little bit more > about the allocation parameters. > > FWIW, I've never heard of anyone being able to trigger this warning > before. Mikael, Thank you for your quick reply. I just invoke erl on FreeBSD 8.0 GENERIC running on amd64 with erlang configured with --enable-native-libs. Apprently, it's quite common with FreeBSD: http://www.erlang.org/cgi-bin/ezmlm-cgi/4/41055 http://www.erlang.org/cgi-bin/ezmlm-cgi/4/41237 Your patch reveals that the hipe loader tries to allocate 0 byte, which FreeBSD optimizes by giving only 2 bytes (where MacOS X 10.6 gives 16). > ./bin/erl Yikes! erts_alloc() returned 0x80169917e for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169917c for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169917a for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699176 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699174 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699172 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169916e for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169916c for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169916a for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699166 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699164 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699162 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169915e for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169915c for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169915a for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699156 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699154 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699152 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169914e for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169914c for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169914a for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699146 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699144 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699142 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169913e for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169913c for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169913a for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699136 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699134 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699132 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169912e for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169912c for align 8 nrbytes 0 Erlang R14A (erts-5.8) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [hipe] [kernel-poll:false] Yikes! erts_alloc() returned 0x80169912a for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699126 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699124 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x801699122 for align 8 nrbytes 0 Yikes! erts_alloc() returned 0x80169911e for align 8 nrbytes 0 Eshell V5.8 (abort with ^G) Yikes! erts_alloc() returned 0x80169911c for align 8 nrbytes 0 1> Avoiding the allocation when data size is 0 (which means there are only terms in the constants map, I guess), just removes the warning. Paul diff --git a/lib/kernel/src/hipe_unified_loader.erl b/lib/kernel/src/hipe_unified_loader.erl index f289b81..d7747f6 100644 --- a/lib/kernel/src/hipe_unified_loader.erl +++ b/lib/kernel/src/hipe_unified_loader.erl @@ -654,7 +654,10 @@ bif_address(Name) when is_atom(Name) -> %% create_data_segment(DataAlign, DataSize, DataList) -> %%io:format("create_data_segment: \nDataAlign: ~p\nDataSize: ~p\nDataList: ~p\n",[DataAlign,DataSize,DataList]), - DataAddress = hipe_bifs:alloc_data(DataAlign, DataSize), + DataAddress = case DataSize of + 0 -> 0; + _ -> hipe_bifs:alloc_data(DataAlign, DataSize) + end, enter_data(DataList, [], DataAddress, DataSize). enter_data(List, ConstMap2, DataAddress, DataSize) -> From allanwegan@REDACTED Wed Aug 4 23:54:12 2010 From: allanwegan@REDACTED (Allan Wegan) Date: Wed, 04 Aug 2010 23:54:12 +0200 Subject: [erlang-questions] Thoughts on when to use 'infinity' timeouts for gen_server:call and friends In-Reply-To: References: <20100801064323.GA16275@hijacked.us> Message-ID: <4C59E184.10704@allanwegan.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2010-08-03 20:35, Kenneth Lundin wrote: > We are however seriously considering to deprecate gen_server:call/2 and recommend all users to use gen_server:call/3 with infinity timeout for all calls except some rare occasions where it really is motivated to have a timeout and the user is prepared to handle all the potential problems and raises with that. I think, that would be the best solution. Just deprecate server:call/2. And update the documentation for server:call/3 to mention the timeout problem and that 'infinity' ist most often the right choice. Then any one could just look into the doc and see the red bordered warning about wrong timeout choices and possible undesired behavior under heavy load... - -- Allan Wegan Jabber: allanwegan@REDACTED ICQ: 209459114 Phone: +49 40 6732962 Sch?neberger Strasse 60, 22149 Hamburg -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (MingW32) iQEcBAEBAgAGBQJMWeGEAAoJENm5axHh7Acxf+wH+wWAHnN8LLFYIimqGpCk2P6J f8KOypZhesb9yLonHXAPcbjNhyDVbN8TF8SSv3p6bZCOtj/oZ7zBegfwEe8SjMoW bq7vqMFUEOEWoyeRQO3akOgllnEZd6sTWHmVH2uQeWmBRwc8rrKLMDmWsKhDBheg uBO6vsSCxnlcxMqUqbh1ag8ljLs9BdfaTSxtpwtQQQwuNgqhUKPZAzDC8TK0i+de fmSdNFM5YtVdhEKQW+gqOGIwHzD75e+AjP1DUI5snO4nDpsdkxxEn64WZkNj6GRy smtuwri4hlPKdG+DygtAka2Vq24aq6DQagq60Z0jTV+JC15r2e6vy/ikUYGIdzw= =1MPa -----END PGP SIGNATURE----- From ok@REDACTED Thu Aug 5 04:44:58 2010 From: ok@REDACTED (ok@REDACTED) Date: Thu, 5 Aug 2010 14:44:58 +1200 (NZST) Subject: [erlang-questions] record type as variable In-Reply-To: References: Message-ID: <6c38617c4974496c097c0ddcddd4e8ae.squirrel@chasm.otago.ac.nz> > On Wed, Aug 4, 2010 at 12:05 PM, Noel Bush wrote: >> Is there any way to specify the type of a record using a variable? I >> want to be able to do this sort of thing: >> >> create(Type, Name) -> >> ? ?#Type{name = Name}. I'm trying to think of a use case for this. When does it make sense to say "I want to make a record; I'm trusting someone else to tell me what kind, and they are trusting me to provide all the field values it needs and all of them with the right types, and I am trusting them to only sak for 'name' as a required field and Name as the right type, and we are all and I'm trusting them to give me something that only ne is a record name, but I don't know which one, and it's all very vague really but I'm sure it's going to work..." You can't do this in any conventional language with records that I know. You *can* do (someClass newNamed: aName) in Smalltalk, but it then #newNamed: is active code that can do all the validation it wants, not a totally passive record constructor. Since there can be a limited number of records in scope in any one file, what stops you writing make_named(r1, N) -> #r1{name = N}; ... make_named(rk, N) -> #rk{name = N}. Now you have something you can attach documentation to, can type check, can add extra validation code to, and can limit much more precisely than "any visible record type with a name field and no other required fields" You have to write one more line of code for each -record declaration, but considering the size of the typical -record declaration this is negligible. >> record type, that any record type has a name field, etc.) It doesn't >> seem like that much of a pain to write a few record-handling functions >> that would allow me to do this, It's not so much that "it's not much of a pain" to write suitable functions, it's that it's a whole LOT of a GAIN to do so. From arun.suresh@REDACTED Thu Aug 5 07:37:22 2010 From: arun.suresh@REDACTED (Arun Suresh) Date: Wed, 4 Aug 2010 22:37:22 -0700 Subject: [erlang-questions] erlang woes Message-ID: Hello folks.. Ive been using erlang for a while now.. and ive got a production system up and running from scratch.. but there are some really annoying aspects of the platform.. the most fundamental of which is the fact that when a node crashes it is very hard to figure out exactly why.. Almost ALL the time what i see in the crash dump is something akin to : =erl_crash_dump:0.1 Wed Aug 4 21:50:01 2010 Slogan: eheap_alloc: Cannot allocate 1140328500 bytes of memory (of type "heap"). System version: Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Compiled: Tue May 11 12:37:38 2010 Taints: at which point I start to comb the sasl logs... and 9 out of 10 times... it is because some critical process has died and the supervisor is busy restarting it.. for example, the other day.. my node crashed and from the sasl logs.. i see that the http manager for a profile I had created had crashed like so : =CRASH REPORT==== 4-Aug-2010::21:47:09 === crasher: initial call: httpc_manager:init/1 pid: <0.185.0> registered_name: httpc_manager_store exception exit: {{case_clause, [{handler_info,#Ref<0.0.17.61372>,<0.17225.36>, undefined,<0.15665.36>,initiating}]}, [{httpc_manager,handle_connect_and_send,5}, {httpc_manager,handle_info,2}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} in function gen_server:terminate/6 ancestors: [httpc_profile_sup,httpc_sup,inets_sup,<0.46.0>] messages: [{'EXIT',<0.16755.36>,normal}, {connect_and_send,<0.16752.36>,#Ref<0.0.17.61366>, and subsequent messages were related to the supervisor trying to restart the profile manager... and failing.. Now my point is... why did the node have to crash.. just because the manager has to be restarted ? and why does the crash.dump always keep telling me im out of memory.. The problem is.. I thought erlang was built to be fault tolerant.. the choice of me using erlang had a LOT to do with doing away with the having to code defensively.. "let it crash" and all that .. just make sure u have a supervisor that restarts ur process and everything will just work fine... but my experience is that most of the time.. simple process restarts bring the whole node crashing down... Would deeply appreciate it someone could tell me if there is something fundamentally wrong with the way im doing things.. or if anyones been in my situation and have had some enlightenments. thanks in advance -Arun From kenji.rikitake@REDACTED Thu Aug 5 08:46:47 2010 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Thu, 5 Aug 2010 15:46:47 +0900 Subject: [erlang-questions] mmap file to binary In-Reply-To: <4C3D87B6.5070407@erix.ericsson.se> References: <7A08415A-8730-4B61-9D6B-0394A1EC4F55@rogvall.se> <4D758E50-91CC-4E0C-873E-4018CF0F4AC6@rogvall.se> <4BBF5898.4020205@erix.ericsson.se> <19391.30636.410791.181135@pilspetsen.it.uu.se> <4C3D87B6.5070407@erix.ericsson.se> Message-ID: <20100805064647.GA97228@k2r.org> Is there any semantic difference between the enif_alloc()/enif_free() pair and the enif_alloc_resource/enig_release_resource() pair? I've only used the enif_alloc()/enif_free() pair so far, within a function and the allocated objects are not referred from outside the function. Regards, Kenji Rikitake In the message <4C3D87B6.5070407@REDACTED> dated Wed, Jul 14, 2010 at 11:47:10AM +0200, Sverker Eriksson writes: > Max Lapshin wrote: > >I have to do so: > > > > > > Mmap *mm; > > mm = (Mmap *)enif_alloc_resource(mmap_resource, sizeof(Mmap)); > > > > mm->ptr = m; > > mm->size = (size_t)file_stat.st_size; > > > > > > m_bin = enif_make_resource_binary(env, mm, mm->ptr, mm->size); > > enif_release_resource(mm); % <--------- > > return enif_make_tuple2(env, enif_make_atom(env, "ok"), m_bin); > > > > > >If I don't add enif_release_resource, it will never be garbage > >collected. Why I have to do so > That was a design choice. Ownership of resources are never transfered. > > enif_allloc_resource gives you ("the NIF") ownership of the resource. > enif_make_resource and enif_make_resource_binary adds a new ownership > from the created Erlang term. > > So, to really free a resource, both "the NIF" ownership has to be > released by calling enif_release_resource and all referring terms has to > be garbage collected. > > > /Sverker, Erlang/OTP From erlang@REDACTED Thu Aug 5 09:08:34 2010 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 5 Aug 2010 09:08:34 +0200 Subject: [erlang-questions] idea: function meta data In-Reply-To: References: <837721.47969.qm@web31507.mail.mud.yahoo.com> <7348e9970711160029p213783b6oe82b801e49d6b79e@mail.gmail.com> <9353296C-B172-453B-8797-A66518F5593C@cs.otago.ac.nz> Message-ID: 2010/8/3 Bob Ippolito : > On Tuesday, August 3, 2010, Richard O'Keefe wrote: >>> >>> On Nov 16, 2007 4:49 AM, Vat Raghavan wrote: >>>> i REALLY REALLY like the idea of meta doc strings. >>>> one possibility for a syntax for it is like pythons', which is something like this -> >>>> >>>> def func( bar ) >>>> """ ?This is the docstring for this function >>>> """ >>>> and then as someone else said, in the string you can do : >>>> >>>> help(Module, func). and the shell emits ?-> >>>> "This is the docstring for this function" >> >> Lisp-style docstrings do not make sense for a language like >> ML or CAML or Clean or Haskell or Mercury or Erlang where a >> function (A) may have multiple clauses -- so there is no >> obvious unique place to *put* the docstring and (B) normally >> use pattern matching -- so the arguments often fail to have >> *names* that the docstring can talk about. >> >> We're left with the Quintus Prolog style of documentation >> comment, or with EDoc. ?Since we already *have* tools for >> EDoc, let's stick with that. >>>> >>>> >>>> then in the shell >>>> help(Module, func). >> >> That should be something like help(Module, Name, Arity) >> and there's not the least reason why that couldn't be >> driven off EDoc. > > It'd be easier if edoc was in the BEAM which would imply keeping the > comments or using an alternative syntax. Compiler directives could > work but string quoting would be a chore. There are a couple of problems here - firstly the compiler ignores all things inside comments so edoc annotations are not noticed by the compiler. Things in annotations (ie lines starting '-') are parsed by the compiler and can be added to the compiled code (usually retreivable using module_info:... calls. Your problem could be solved in two ways - either pre-process a module and turn the parse tree of the edoc information into a function that could be compiled togther with the other functions in the module. Thus after compilation a function Mod:edoc_info() would appear by magic (similar to Mod:module_info() ...) Alternatively one could do something about quotes and annotations - for a long time I'd though it would be nice to have some kind of mechanism to turn off or change the meaning of of the characters inside a quoted string - something like: python raw strings r"......" perl multiple quoting qq^hello joe^ There is scope for excessive creativity here - so a guess a few years contemplation is in order here (unless somebody has already pondered this for a few years and knows the answer) The next change would to allow annotation anywhere in the source, at the moment they are only allowed before you start defining functions. There was no deep thinking about this. With a change to the quoting conventions and relaxing the restrictions on the placement of annotations then the default would be to compile all annotations into the module_info function. /joe > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tech@REDACTED Thu Aug 5 09:12:15 2010 From: tech@REDACTED (tech@REDACTED) Date: Thu, 5 Aug 2010 09:12:15 +0200 Subject: [erlang-questions] erlang woes In-Reply-To: References: Message-ID: <201008050912.15657.tech@erlfinsys.net> Arun, I think there are a few things from the mail that one can tackle and use to look at solving it, but first some more information may be useful in order to address your concerns. 1. Are you using OTP principles and using a supervision structure for your applaction? 2. It seems that it's an unmatched case entry that causes the crash {{case_clause, [{handler_info,#Ref<0.0.17.61372>,<0.17225.36>, undefined,<0.15665.36>,initiating}]}, [{httpc_manager,handle_connect_and_send,5}, {httpc_manager,handle_info,2}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} I have always found that in such a situation it's good to start looking from the top to the bottom. In this case it may seem that the problem case could be in httpc_manager handle_connect_and _send and that function doesn't match the specific inputs reflected in line one. Hope this helps for a start On Thursday, August 05, 2010 07:37:22 Arun Suresh wrote: > Hello folks.. > > Ive been using erlang for a while now.. and ive got a production system up > and running from scratch.. but there are some really annoying aspects of > the platform.. the most fundamental of which is the fact that when a node > crashes it is very hard to figure out exactly why.. Almost ALL the time > what i see in the crash dump is something akin to : > > =erl_crash_dump:0.1 > Wed Aug 4 21:50:01 2010 > Slogan: eheap_alloc: Cannot allocate 1140328500 bytes of memory (of type > "heap"). > System version: Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > Compiled: Tue May 11 12:37:38 2010 > Taints: > > > at which point I start to comb the sasl logs... and 9 out of 10 times... it > is because some critical process has died and the supervisor is busy > restarting it.. for example, the other day.. my node crashed and from the > sasl logs.. i see that the http manager for a profile I had created had > crashed like so : > > =CRASH REPORT==== 4-Aug-2010::21:47:09 === > crasher: > initial call: httpc_manager:init/1 > pid: <0.185.0> > registered_name: httpc_manager_store > exception exit: {{case_clause, > [{handler_info,#Ref<0.0.17.61372>,<0.17225.36>, > undefined,<0.15665.36>,initiating}]}, > [{httpc_manager,handle_connect_and_send,5}, > {httpc_manager,handle_info,2}, > {gen_server,handle_msg,5}, > {proc_lib,init_p_do_apply,3}]} > in function gen_server:terminate/6 > ancestors: [httpc_profile_sup,httpc_sup,inets_sup,<0.46.0>] > messages: [{'EXIT',<0.16755.36>,normal}, > {connect_and_send,<0.16752.36>,#Ref<0.0.17.61366>, > > > and subsequent messages were related to the supervisor trying to restart > the profile manager... and failing.. > > Now my point is... why did the node have to crash.. just because the > manager has to be restarted ? > and why does the crash.dump always keep telling me im out of memory.. > > The problem is.. I thought erlang was built to be fault tolerant.. the > choice of me using erlang had a LOT to do with doing away with the having > to code defensively.. "let it crash" and all that .. just make sure u have > a supervisor that restarts ur process and everything will just work > fine... but my experience is that most of the time.. simple process > restarts bring the whole node crashing down... > > Would deeply appreciate it someone could tell me if there is something > fundamentally wrong with the way im doing things.. or if anyones been in my > situation and have had some enlightenments. > > thanks in advance > -Arun From kenji.rikitake@REDACTED Thu Aug 5 09:37:51 2010 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Thu, 5 Aug 2010 16:37:51 +0900 Subject: [erlang-questions] Hipe and memory alignment (Yikes! erts_alloc() returned misaligned address) In-Reply-To: <33495D81-33AC-4AC8-9394-6377C01760CD@kallisys.net> References: <33495D81-33AC-4AC8-9394-6377C01760CD@kallisys.net> Message-ID: <20100805073751.GB97228@k2r.org> I also often see mostly the same message which only the address value is different on FreeBSD i386 7.3-RELEASE, with the stock gcc (gcc (GCC) 4.2.1 20070719 [FreeBSD]) Regards, Kenji Rikitake In the message <33495D81-33AC-4AC8-9394-6377C01760CD@REDACTED> dated Wed, Aug 04, 2010 at 06:59:11PM +0200, Paul Guyot writes: > Hello, > > When running erl with native (hipe) code on some platforms (FreeBSD amd64), one quickly encounters messages such as: > > Yikes! erts_alloc() returned misaligned address 0x8016a512c > > This message seems to come from hipe_bifs:alloc_data/2. The allocator is the system allocator, which on unix platform is malloc, and malloc does not necessarily return aligned blocks (on MacOS X, it does, but not on FreeBSD). > > Does it really matter? (hipe_bifs:alloc_data/2 is only called from hipe_unified_loader:create_data_segment/3). > If it doesn't, can we remove the log line? > If it does, can we replace the allocator with another one that always return aligned blocks? > > Paul > -- > Semiocast http://semiocast.com/ > +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From bgustavsson@REDACTED Thu Aug 5 09:58:38 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 5 Aug 2010 09:58:38 +0200 Subject: [erlang-questions] Thoughts on when to use 'infinity' timeouts for gen_server:call and friends In-Reply-To: <4C594792.1050300@erlang-solutions.com> References: <20100801064323.GA16275@hijacked.us> <4C594792.1050300@erlang-solutions.com> Message-ID: On Wed, Aug 4, 2010 at 12:57 PM, Ulf Wiger wrote: > gen_server:req/2 (as in 'request')? I like this name. It is short, and I have seen a lot of code that defines a local req() function as a wrapper to call gen_server:call(). -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Thu Aug 5 10:03:46 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 5 Aug 2010 10:03:46 +0200 Subject: [erlang-questions] Thoughts on when to use 'infinity' timeouts for gen_server:call and friends In-Reply-To: <20100804091820.6800548f@otis> References: <20100801064323.GA16275@hijacked.us> <4C594792.1050300@erlang-solutions.com> <20100804091820.6800548f@otis> Message-ID: On Wed, Aug 4, 2010 at 3:18 PM, wrote: > I understand it's a drastic change but couldn't it be shouted from the > rooftops that it's being made and keep the functions the same? Or > provide a global switch to revert back from infinity to the old default? We have tried do those things in the past. It was not a success (and that is probably understating it). -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From mikpe@REDACTED Thu Aug 5 10:07:32 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Thu, 5 Aug 2010 10:07:32 +0200 Subject: [erlang-questions] Hipe and memory alignment (Yikes! erts_alloc() returned misaligned address) In-Reply-To: <8E2689F9-91F4-43AB-9D6B-7F29B9584AB0@kallisys.net> References: <33495D81-33AC-4AC8-9394-6377C01760CD@kallisys.net> <19545.55651.737243.453894@pilspetsen.it.uu.se> <8E2689F9-91F4-43AB-9D6B-7F29B9584AB0@kallisys.net> Message-ID: <19546.28996.401994.216678@pilspetsen.it.uu.se> Paul Guyot writes: > > Le 4 ao?t 2010 ? 23:19, Mikael Pettersson a ?crit : > > > Now, it is possible that the alignment is wrong, esp. if the data > > segment for whatever reason turns out to be tiny. Please rebuild > > the erlang VM with the following patch and do what you normally do > > to trigger the Yikes! message. It should tell us a little bit more > > about the allocation parameters. > > > > FWIW, I've never heard of anyone being able to trigger this warning > > before. > > Mikael, > > Thank you for your quick reply. > I just invoke erl on FreeBSD 8.0 GENERIC running on amd64 with erlang configured with --enable-native-libs. > > Apprently, it's quite common with FreeBSD: > http://www.erlang.org/cgi-bin/ezmlm-cgi/4/41055 > http://www.erlang.org/cgi-bin/ezmlm-cgi/4/41237 Still, that's only two messages, with $Subject not mentioning HiPE, thus easily missed. > Your patch reveals that the hipe loader tries to allocate 0 byte, which FreeBSD optimizes by giving only 2 bytes (where MacOS X 10.6 gives 16). > > > ./bin/erl > Yikes! erts_alloc() returned 0x80169917e for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169917c for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169917a for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699176 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699174 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699172 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169916e for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169916c for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169916a for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699166 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699164 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699162 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169915e for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169915c for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169915a for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699156 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699154 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699152 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169914e for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169914c for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169914a for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699146 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699144 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699142 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169913e for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169913c for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169913a for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699136 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699134 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699132 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169912e for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169912c for align 8 nrbytes 0 > Erlang R14A (erts-5.8) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [hipe] [kernel-poll:false] > > Yikes! erts_alloc() returned 0x80169912a for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699126 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699124 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x801699122 for align 8 nrbytes 0 > Yikes! erts_alloc() returned 0x80169911e for align 8 nrbytes 0 > Eshell V5.8 (abort with ^G) > Yikes! erts_alloc() returned 0x80169911c for align 8 nrbytes 0 > 1> > > > Avoiding the allocation when data size is 0 (which means there are only terms in the constants map, I guess), just removes the warning. > > Paul > > diff --git a/lib/kernel/src/hipe_unified_loader.erl b/lib/kernel/src/hipe_unified_loader.erl > index f289b81..d7747f6 100644 > --- a/lib/kernel/src/hipe_unified_loader.erl > +++ b/lib/kernel/src/hipe_unified_loader.erl > @@ -654,7 +654,10 @@ bif_address(Name) when is_atom(Name) -> > %% > create_data_segment(DataAlign, DataSize, DataList) -> > %%io:format("create_data_segment: \nDataAlign: ~p\nDataSize: ~p\nDataList: ~p\n",[DataAlign,DataSize,DataList]), > - DataAddress = hipe_bifs:alloc_data(DataAlign, DataSize), > + DataAddress = case DataSize of > + 0 -> 0; > + _ -> hipe_bifs:alloc_data(DataAlign, DataSize) > + end, > enter_data(DataList, [], DataAddress, DataSize). > > enter_data(List, ConstMap2, DataAddress, DataSize) -> > > The logic change is Ok but could be done either here or in the VM. Doing it in the VM is safer. Part of the problem is the poor warning message from the VM which fails to clearly identify the context where it occurs. Concretely, the message should include the string "hipe". Therefore I'd rather do the logic change in the VM and at the same time prefix the warning with __FUNCTION__ not "Yikes!". Try this. I'll send it to erlang-patches if it works for you. /Mikael --- otp_src_R14A/erts/emulator/hipe/hipe_bif0.c.~1~ 2010-02-19 19:04:06.000000000 +0100 +++ otp_src_R14A/erts/emulator/hipe/hipe_bif0.c 2010-08-05 09:57:48.000000000 +0200 @@ -440,9 +440,12 @@ BIF_RETTYPE hipe_bifs_alloc_data_2(BIF_A align != sizeof(long) && align != sizeof(double))) BIF_ERROR(BIF_P, BADARG); nrbytes = unsigned_val(BIF_ARG_2); + if (nrbytes == 0) + BIF_RET(make_small(0)); block = erts_alloc(ERTS_ALC_T_HIPE, nrbytes); if ((unsigned long)block & (align-1)) - fprintf(stderr, "Yikes! erts_alloc() returned misaligned address %p\r\n", block); + fprintf(stderr, "%s: erts_alloc(%lu) returned %p which is not %lu-byte aligned\r\n", + __FUNCTION__, (unsigned long)nrbytes, block, (unsigned long)align); BIF_RET(address_to_term(block, BIF_P)); } From krab@REDACTED Thu Aug 5 10:07:44 2010 From: krab@REDACTED (Kresten Krab Thorup) Date: Thu, 5 Aug 2010 10:07:44 +0200 Subject: [erlang-questions] record type as variable In-Reply-To: References: Message-ID: <360FCD15-ABF3-4280-91AB-DB82192EB451@trifork.com> A simple macro will also do ... --------------------------- -module(foo). -record(x,{name,zz}). -record(y,{yy,name}). -export([test1/0,test2/0]). -define(create(Type,Name),#Type{name = Name}). test1() -> ?create(x,"Noel"). test2() -> ?create(y,"Noel"). --------------------------- Eshell V5.8 (abort with ^G) 1> c(foo). {ok,foo} 2> foo:test1(). {x,"Noel",undefined} 3> foo:test2(). {y,undefined,"Noel"} On Aug 4, 2010, at 19:05 , Noel Bush wrote: > Is there any way to specify the type of a record using a variable? I > want to be able to do this sort of thing: > > create(Type, Name) -> > #Type{name = Name}. > > (Obviously, I'm responsible for making sure that Type corresponds to a > record type, that any record type has a name field, etc.) It doesn't > seem like that much of a pain to write a few record-handling functions > that would allow me to do this, but it would be great to find out that > someone's already done the work. > > Thanks, > Noel > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Kresten Krab Thorup, CTO, Trifork From pguyot@REDACTED Thu Aug 5 11:16:10 2010 From: pguyot@REDACTED (Paul Guyot) Date: Thu, 5 Aug 2010 11:16:10 +0200 Subject: [erlang-questions] Hipe and memory alignment (Yikes! erts_alloc() returned misaligned address) In-Reply-To: <19546.28996.401994.216678@pilspetsen.it.uu.se> References: <33495D81-33AC-4AC8-9394-6377C01760CD@kallisys.net> <19545.55651.737243.453894@pilspetsen.it.uu.se> <8E2689F9-91F4-43AB-9D6B-7F29B9584AB0@kallisys.net> <19546.28996.401994.216678@pilspetsen.it.uu.se> Message-ID: Le 5 ao?t 2010 ? 10:07, Mikael Pettersson a ?crit : > Try this. I'll send it to erlang-patches if it works for you. Mikael, Thank you for your quick reply and patch. I successfully applied it and I confirm that the messages are gone. Thanks again. Regards, Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From attila.r.nohl@REDACTED Thu Aug 5 11:43:05 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 5 Aug 2010 11:43:05 +0200 Subject: [erlang-questions] erlang woes In-Reply-To: References: Message-ID: 2010/8/5, Arun Suresh : [...] > Now my point is... why did the node have to crash.. just because the manager > has to be restarted ? > and why does the crash.dump always keep telling me im out of memory.. Because you actually are out of memory? There was a memory leak in the httpc code, but I've seen fixes for it on this list. From erlang@REDACTED Thu Aug 5 12:27:37 2010 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 5 Aug 2010 12:27:37 +0200 Subject: [erlang-questions] erlang woes In-Reply-To: References: Message-ID: On Thu, Aug 5, 2010 at 7:37 AM, Arun Suresh wrote: > Hello folks.. > > Ive been using erlang for a while now.. and ive got a production system up > and running from scratch.. but there are some really annoying aspects of the > platform.. the most fundamental of which is the fact that when a node > crashes it is very hard to figure out exactly why.. Almost ALL the time what > i see in the crash dump is something akin to : > > =erl_crash_dump:0.1 > Wed Aug ?4 21:50:01 2010 > Slogan: eheap_alloc: Cannot allocate 1140328500 bytes of memory (of type > "heap"). > System version: Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > Compiled: Tue May 11 12:37:38 2010 > Taints: > > > at which point I start to comb the sasl logs... and 9 out of 10 times... it > is because some critical process has died and the supervisor is busy > restarting it.. for example, the other day.. my node crashed and from the > sasl logs.. i see that the http manager for a profile I had created had > crashed like so : > > =CRASH REPORT==== 4-Aug-2010::21:47:09 === > ?crasher: > ? ?initial call: httpc_manager:init/1 > ? ?pid: <0.185.0> > ? ?registered_name: httpc_manager_store > ? ?exception exit: {{case_clause, > ? ? ? ? ? ? ? ? ? ? ? ? [{handler_info,#Ref<0.0.17.61372>,<0.17225.36>, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?undefined,<0.15665.36>,initiating}]}, > ? ? ? ? ? ? ? ? ? ? [{httpc_manager,handle_connect_and_send,5}, > ? ? ? ? ? ? ? ? ? ? ?{httpc_manager,handle_info,2}, > ? ? ? ? ? ? ? ? ? ? ?{gen_server,handle_msg,5}, > ? ? ? ? ? ? ? ? ? ? ?{proc_lib,init_p_do_apply,3}]} > ? ? ?in function ?gen_server:terminate/6 > ? ?ancestors: [httpc_profile_sup,httpc_sup,inets_sup,<0.46.0>] > ? ?messages: [{'EXIT',<0.16755.36>,normal}, > ? ? ? ? ? ? ? ? ?{connect_and_send,<0.16752.36>,#Ref<0.0.17.61366>, > > > and subsequent messages were related to the supervisor trying to restart the > profile manager... and failing.. > > Now my point is... why did the node have to crash.. just because the manager > has to be restarted ? > and why does the crash.dump always keep telling me im out of memory.. > > The problem is.. I thought erlang was built to be fault tolerant.. the > choice of me using erlang had a LOT to do with doing away with the having to > code defensively.. "let it crash" and all that .. just make sure u have a > supervisor that restarts ur process and everything will just work fine... > but my experience is that most of the time.. simple process restarts bring > the whole node crashing down... Erlang was designed to be fault-tolerant. The point is that if you crash then "somebody else" has to diagnose the error - "you" can't diagnose the error because you are dead. Now if a process crashes, some other process can detect and fix the error. If you use otp behaviors this mechanism is hidden from you in the supervisor hierarchy. But if the entire node crashes then some other node must fix the error. In raw erlang (ie not using the OTP libraries) their is a spawn_link/4 primitive precisely to propagate errors over node boundaries. So to make an entire node fault-tolerant you need more than one node - the other node(s) should fix the error. The problem with error is that some you can fix at run-time others you can't. Running out of memory is something that you typically can't easily fix. If you run out of memory you have to kill something to reclaim memory - but what should you kill? - this depends upon your application. In such circumstances Erlang decides it can't do anything sensible and the whole node dies - with hopefully a helpful error message. When things crash you're supposed to leave enough information behind so you can figure out why things went wrong - the default behavior of the system is to try and leave some helpful clues as to what went wrong (so you don't have to code this yourself). Mostly you don't have to mess with this. For example, if you try to open a non existent file you might generate an 'enoent' exception, if your application only opens a single file then this information will be enough for you to discover and fix your error. But if your application opens many different files you'll have to add an explicit exception exit({missingFile, File}) at appropriate places in your code. No matter how good any exception and error recovery strategy is there will always be corner cases where automatic strategies fail, and here you're back to manual debugging. You have a pretty good clue - you've got a memory leak - some process is consuming too much memory - so now you have to start using various tools to figure out which process run wild. I suspect this is an uncommon error - if it were common then you'd find more tools for detecting rogue processes in the standard libraries - their absence indicates that either this is not a common problem, or that it is a common problem, but resolving it it easy. I know that none of what I've said helps you find your specific problem - but you have to understand the limitations of the system - a dead man cannot determine the reason why they themselves died - somebody else has to do this ... Good luck finding your error .. Cheers /Joe > > Would deeply appreciate it someone could tell me if there is something > fundamentally wrong with the way im doing things.. or if anyones been in my > situation and have had some enlightenments. > > thanks in advance > -Arun > From attila.r.nohl@REDACTED Thu Aug 5 12:42:30 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 5 Aug 2010 12:42:30 +0200 Subject: [erlang-questions] erlang woes In-Reply-To: References: Message-ID: 2010/8/5, Joe Armstrong : [...] > I suspect this is an uncommon error - if it were common then you'd > find more tools > for detecting rogue processes in the standard libraries - their > absence indicates that either this > is not a common problem, or that it is a common problem, but resolving > it it easy. I've seen two tools that can be used to detect rogue processes. I also have some one-liners on the local project Wikipage. So I'd go for the second option: it's a common problem, but not that hard to resolve (or at least to find the offending process - it's rather more complicated to find out why it leaks and I guess it's too hard to write an automated tool). By the way, the one-liners (in case some of you find it useful): Which process uses the most memory: lists:reverse(lists:keysort(2,[{P, erlang:process_info(P, heap_size)} || P <- erlang:processes()])). Which ets table uses the most memory: lists:reverse(lists:keysort(2,[{T, ets:info(T, memory)} || T <- ets:all()])). From ali.yakout@REDACTED Thu Aug 5 14:56:34 2010 From: ali.yakout@REDACTED (Ali Yakout) Date: Thu, 5 Aug 2010 14:56:34 +0200 Subject: Corba Help Message-ID: <591FA96352AC9946A403D860A60166520105941A@ESESSCMS0357.eemea.ericsson.se> Hi, I'm new to Corba and I can't find my way through... I want to do the same operation done with the following Java code: //----------------------------------------- org.omg.CORBA.Any any = _orb.create_any(); any.insert_wstring("1.1"); com.lhs.ccb.soi.types.NvElementI namedValue = new com.lhs.ccb.soi.types.NvElementI("CS_CODE", any, 0, (short)0); NvElementI[] input = new NvElementI[] {namedValue}; NvElementI[] result = execute("CUSTOMERS.SEARCH", input); //----------------------------------------- My problem is that I don't know how to create an object of NvElement with the above values. I have the following types: %%------------------------------------------ 254> com_lhs_ccb_soi_types_NvElementI:tc(). {tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", "NvElementI", [{"name",{tk_string,0}}, {"value",tk_any}, {"flags",tk_long}, {"length",tk_short}]} 255> com_lhs_ccb_soi_types_NvElementListI:tc(). {tk_sequence,{tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", "NvElementI", [{"name",{tk_string,0}}, {"value",tk_any}, {"flags",tk_long}, {"length",tk_short}]}, 0} %%------------------------------------------ I tried the following but I got 'MARSHAL' exception. 244> AnyStr=any:create(orber_tc:wstring(3),"1.1"). {any,{tk_wstring,3},"1.1"} 245> NV=any:create(com_lhs_ccb_soi_types_NvElementI:tc(),["CS_CODE",AnyStr,0,0]). {any,{tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", "NvElementI", [{"name",{tk_string,0}}, {"value",tk_any}, {"flags",tk_long}, {"length",tk_short}]}, ["CS_CODE",{any,{tk_wstring,3},"1.1"},0,0]} 246> com_lhs_ccb_soi_ServiceObjectI:executeI(CMIObj,"CUSTOMERS.SEARCH",[NV]). ** exception throw: {'EXCEPTION',{'MARSHAL',[],1163001856,'COMPLETED_NO'}} in function corba:raise/1 in call from orber_iiop:request/8 So my question is, how to create Corba types in Erlang? Thanks, Ali From nick@REDACTED Thu Aug 5 15:31:26 2010 From: nick@REDACTED (Niclas Eklund) Date: Thu, 5 Aug 2010 15:31:26 +0200 (CEST) Subject: [erlang-questions] Corba Help In-Reply-To: <591FA96352AC9946A403D860A60166520105941A@ESESSCMS0357.eemea.ericsson.se> References: <591FA96352AC9946A403D860A60166520105941A@ESESSCMS0357.eemea.ericsson.se> Message-ID: Hello! Check out the chapter "6.8 Typecode, Identity and Name Access Functions" in Orber's User's Guide: http://www.erlang.org/doc/apps/orber/ch_idl_to_erlang_mapping.html#id2277452 The record is defined in a generated hrl-file after you've compiled the IDL-file. Remember to include that file. In fact, you should read all in "6 OMG IDL to Erlang Mapping", since it describe step by step how one should do. The debugging chapter should also be helpfull - http://www.erlang.org/doc/apps/orber/ch_debugging.html You will probably find the IC backend erl_template rather usefull (will generate a complete call-back module). Note, it will overwrite any existing version of the *impl.erl file. Read more about it in chapter 6. If possible you should use Light IFR when compiling the IDL files and starting Orber. Best regards, Niclas @ Erlang/OTP On Thu, 5 Aug 2010, Ali Yakout wrote: > Hi, > > I'm new to Corba and I can't find my way through... > I want to do the same operation done with the following Java code: > > //----------------------------------------- > org.omg.CORBA.Any any = _orb.create_any(); > any.insert_wstring("1.1"); > com.lhs.ccb.soi.types.NvElementI namedValue = new com.lhs.ccb.soi.types.NvElementI("CS_CODE", any, 0, (short)0); > NvElementI[] input = new NvElementI[] {namedValue}; > NvElementI[] result = execute("CUSTOMERS.SEARCH", input); > //----------------------------------------- > > My problem is that I don't know how to create an object of NvElement with the above values. > > I have the following types: > > %%------------------------------------------ > 254> com_lhs_ccb_soi_types_NvElementI:tc(). > {tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", > "NvElementI", > [{"name",{tk_string,0}}, > {"value",tk_any}, > {"flags",tk_long}, > {"length",tk_short}]} > 255> com_lhs_ccb_soi_types_NvElementListI:tc(). > {tk_sequence,{tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", > "NvElementI", > [{"name",{tk_string,0}}, > {"value",tk_any}, > {"flags",tk_long}, > {"length",tk_short}]}, > 0} > %%------------------------------------------ > > I tried the following but I got 'MARSHAL' exception. > > 244> AnyStr=any:create(orber_tc:wstring(3),"1.1"). > {any,{tk_wstring,3},"1.1"} > 245> NV=any:create(com_lhs_ccb_soi_types_NvElementI:tc(),["CS_CODE",AnyStr,0,0]). > {any,{tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", > "NvElementI", > [{"name",{tk_string,0}}, > {"value",tk_any}, > {"flags",tk_long}, > {"length",tk_short}]}, > ["CS_CODE",{any,{tk_wstring,3},"1.1"},0,0]} > 246> com_lhs_ccb_soi_ServiceObjectI:executeI(CMIObj,"CUSTOMERS.SEARCH",[NV]). > > ** exception throw: {'EXCEPTION',{'MARSHAL',[],1163001856,'COMPLETED_NO'}} > in function corba:raise/1 > in call from orber_iiop:request/8 > > > So my question is, how to create Corba types in Erlang? > > > Thanks, > Ali > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mevans@REDACTED Thu Aug 5 15:55:27 2010 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 5 Aug 2010 09:55:27 -0400 Subject: [erlang-questions] erlang woes In-Reply-To: References: Message-ID: Are you using pg2 in a distributed system? There is a known (should be fixed) bug where pg2 was eating memory like crazy. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Arun Suresh Sent: Thursday, August 05, 2010 1:37 AM To: erlang-questions@REDACTED Subject: [erlang-questions] erlang woes Hello folks.. Ive been using erlang for a while now.. and ive got a production system up and running from scratch.. but there are some really annoying aspects of the platform.. the most fundamental of which is the fact that when a node crashes it is very hard to figure out exactly why.. Almost ALL the time what i see in the crash dump is something akin to : =erl_crash_dump:0.1 Wed Aug 4 21:50:01 2010 Slogan: eheap_alloc: Cannot allocate 1140328500 bytes of memory (of type "heap"). System version: Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Compiled: Tue May 11 12:37:38 2010 Taints: at which point I start to comb the sasl logs... and 9 out of 10 times... it is because some critical process has died and the supervisor is busy restarting it.. for example, the other day.. my node crashed and from the sasl logs.. i see that the http manager for a profile I had created had crashed like so : =CRASH REPORT==== 4-Aug-2010::21:47:09 === crasher: initial call: httpc_manager:init/1 pid: <0.185.0> registered_name: httpc_manager_store exception exit: {{case_clause, [{handler_info,#Ref<0.0.17.61372>,<0.17225.36>, undefined,<0.15665.36>,initiating}]}, [{httpc_manager,handle_connect_and_send,5}, {httpc_manager,handle_info,2}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} in function gen_server:terminate/6 ancestors: [httpc_profile_sup,httpc_sup,inets_sup,<0.46.0>] messages: [{'EXIT',<0.16755.36>,normal}, {connect_and_send,<0.16752.36>,#Ref<0.0.17.61366>, and subsequent messages were related to the supervisor trying to restart the profile manager... and failing.. Now my point is... why did the node have to crash.. just because the manager has to be restarted ? and why does the crash.dump always keep telling me im out of memory.. The problem is.. I thought erlang was built to be fault tolerant.. the choice of me using erlang had a LOT to do with doing away with the having to code defensively.. "let it crash" and all that .. just make sure u have a supervisor that restarts ur process and everything will just work fine... but my experience is that most of the time.. simple process restarts bring the whole node crashing down... Would deeply appreciate it someone could tell me if there is something fundamentally wrong with the way im doing things.. or if anyones been in my situation and have had some enlightenments. thanks in advance -Arun From ulf.wiger@REDACTED Thu Aug 5 16:31:08 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 05 Aug 2010 16:31:08 +0200 Subject: [erlang-questions] record type as variable In-Reply-To: <6c38617c4974496c097c0ddcddd4e8ae.squirrel@chasm.otago.ac.nz> References: <6c38617c4974496c097c0ddcddd4e8ae.squirrel@chasm.otago.ac.nz> Message-ID: <4C5ACB2C.3070701@erlang-solutions.com> ok@REDACTED wrote: >> On Wed, Aug 4, 2010 at 12:05 PM, Noel Bush wrote: >>> Is there any way to specify the type of a record using a variable? I >>> want to be able to do this sort of thing: >>> >>> create(Type, Name) -> >>> #Type{name = Name}. > > I'm trying to think of a use case for this. I did something along those lines in my DIAMETER stack while at Ericsson. http://www.erlang.org/cgi-bin/ezmlm-cgi/4/33563 Not to say that it couldn't be solved in other ways. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From mixolyde@REDACTED Thu Aug 5 17:30:35 2010 From: mixolyde@REDACTED (Brian Williams) Date: Thu, 5 Aug 2010 11:30:35 -0400 Subject: Erlang/Message-Passing Modeling Language/Tool Message-ID: Has there been any work done in the creation of a modeling language or tool that's specifically geared toward Erlang or other systems that use message passing for concurrency? I'm curious if everyone is using circle and stick diagrams, or something more sophisticated? I imagine that most UML is not really appropriate for modeling supervision hierarchies and top-level functions and things like that. Brian -- Brian E. Williams mixolyde@REDACTED http://www.techhouse.us/wordpress-mu/brianw "Never attribute to malice that which can be adequately explained by stupidity." - Hanlon's Razor From luismarianoguerra@REDACTED Thu Aug 5 17:50:52 2010 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Thu, 5 Aug 2010 12:50:52 -0300 Subject: [erlang-questions] idea: function meta data In-Reply-To: References: <837721.47969.qm@web31507.mail.mud.yahoo.com> <7348e9970711160029p213783b6oe82b801e49d6b79e@mail.gmail.com> <9353296C-B172-453B-8797-A66518F5593C@cs.otago.ac.nz> Message-ID: On Thu, Aug 5, 2010 at 4:08 AM, Joe Armstrong wrote: > 2010/8/3 Bob Ippolito : >> On Tuesday, August 3, 2010, Richard O'Keefe wrote: >>>> >>>> On Nov 16, 2007 4:49 AM, Vat Raghavan wrote: >>>>> i REALLY REALLY like the idea of meta doc strings. >>>>> one possibility for a syntax for it is like pythons', which is something like this -> >>>>> >>>>> def func( bar ) >>>>> """ ?This is the docstring for this function >>>>> """ >>>>> and then as someone else said, in the string you can do : >>>>> >>>>> help(Module, func). and the shell emits ?-> >>>>> "This is the docstring for this function" >>> >>> Lisp-style docstrings do not make sense for a language like >>> ML or CAML or Clean or Haskell or Mercury or Erlang where a >>> function (A) may have multiple clauses -- so there is no >>> obvious unique place to *put* the docstring and (B) normally >>> use pattern matching -- so the arguments often fail to have >>> *names* that the docstring can talk about. >>> >>> We're left with the Quintus Prolog style of documentation >>> comment, or with EDoc. ?Since we already *have* tools for >>> EDoc, let's stick with that. >>>>> >>>>> >>>>> then in the shell >>>>> help(Module, func). >>> >>> That should be something like help(Module, Name, Arity) >>> and there's not the least reason why that couldn't be >>> driven off EDoc. >> >> It'd be easier if edoc was in the BEAM which would imply keeping the >> comments or using an alternative syntax. Compiler directives could >> work but string quoting would be a chore. > > There are a couple of problems here - firstly the compiler ignores all > things inside comments > so edoc annotations are not noticed by the compiler. Things in > annotations (ie lines starting > '-') are parsed by the compiler and can be added to the compiled code > (usually retreivable > using module_info:... calls. > > Your problem could be solved in two ways - either pre-process a module > and turn the > parse tree of the edoc information into a function that could be > compiled togther with > the other functions in the module. Thus after compilation a function > Mod:edoc_info() would > appear by magic (similar to ?Mod:module_info() ...) > > Alternatively one could do something about quotes and annotations - > for a long time I'd > though it would be nice to have some kind of mechanism to turn off or > change the meaning of > of the characters inside a quoted string - something like: > > ? ? python raw strings ? r"......" > ? ? perl multiple quoting qq^hello joe^ > > There is scope for excessive creativity here - so a guess a few years > contemplation is in order here (unless somebody has already pondered > this for a few years and knows the answer) > > The next change would to allow annotation anywhere in the source, at > the moment they are only allowed before you start defining functions. > There was no deep thinking about this. > > With a change to the quoting conventions and relaxing the restrictions > on the placement of > annotations then the default would be to compile all annotations into > the module_info function. > > /joe maybe this can add some ideas to the discussion. in efene[0] there are local[1] and global[2] attributes. local attributes get the function name and arity prepended at compile time and are moved to the top of the ast to make it possible to compile. the @doc("doc string") is a "well known" local attribute that is used by the mod.doc function to generate the documentation, example of documentation generated using this annotation (and @@moddoc for module documentation) is here: http://marianoguerra.com.ar/efene/docs/lib/index.html disclaimer: not too much documentation but works as an example here is an example in efene and its transformation to erlang: mariano@REDACTED:~$ cat test.ifn @public @doc("a function that does something") foo = fn () io.format("hi!~n") mariano@REDACTED:~$ fnc -t erl test.ifn -module(test). -export([foo/0]). -doc({{foo, 0}, "a function that does something"}). foo() -> io:format("hi!~n"). [0] http://github.com/marianoguerra/efene [1] http://marianoguerra.com.ar/efene/docs/reference/toplevel/localattrs.html [2] http://marianoguerra.com.ar/efene/docs/reference/toplevel/globalattrs.html From arun.suresh@REDACTED Thu Aug 5 19:07:40 2010 From: arun.suresh@REDACTED (Arun Suresh) Date: Thu, 5 Aug 2010 10:07:40 -0700 Subject: [erlang-questions] erlang woes In-Reply-To: References: Message-ID: Hello Joe.. Thank you for the detailed response... the thing is... the example I gave was just a specific instance.. I shall give you another one... We use RabbitMQ.. and we maintain a couple of persistent connections to the sever. We also have many consumers for messages that might appear in rabbitmq queues.. So now all these connection processes (max 10) are started by a supervisor which has a 'one_for_all' policy (this is so that socket errors ensure that all connections to the host are torn down and recreated) Now it so happens that every once a while.. there is a connection issues.. and all connections are recreated.. at which point we notice that sometimes the node just crashes.. with the same "unable to allocate memory" error... and the funny thing is.. up until that point, the memory usage of the node doesnt exceed 10 -20 % of total RAM.. so why should this bomb ? Basically, what I trying to point out is... almost all the times the node has crashed, what i see in the sasl logs prior to crashing is that some process has dies.. and it has brought down a lot of other process and the supervisor was in the middle of restarting them.. regards -Arun On Thu, Aug 5, 2010 at 3:27 AM, Joe Armstrong wrote: > On Thu, Aug 5, 2010 at 7:37 AM, Arun Suresh wrote: > > Hello folks.. > > > > Ive been using erlang for a while now.. and ive got a production system > up > > and running from scratch.. but there are some really annoying aspects of > the > > platform.. the most fundamental of which is the fact that when a node > > crashes it is very hard to figure out exactly why.. Almost ALL the time > what > > i see in the crash dump is something akin to : > > > > =erl_crash_dump:0.1 > > Wed Aug 4 21:50:01 2010 > > Slogan: eheap_alloc: Cannot allocate 1140328500 bytes of memory (of type > > "heap"). > > System version: Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] > > [async-threads:0] [hipe] [kernel-poll:false] > > Compiled: Tue May 11 12:37:38 2010 > > Taints: > > > > > > at which point I start to comb the sasl logs... and 9 out of 10 times... > it > > is because some critical process has died and the supervisor is busy > > restarting it.. for example, the other day.. my node crashed and from the > > sasl logs.. i see that the http manager for a profile I had created had > > crashed like so : > > > > =CRASH REPORT==== 4-Aug-2010::21:47:09 === > > crasher: > > initial call: httpc_manager:init/1 > > pid: <0.185.0> > > registered_name: httpc_manager_store > > exception exit: {{case_clause, > > [{handler_info,#Ref<0.0.17.61372>,<0.17225.36>, > > undefined,<0.15665.36>,initiating}]}, > > [{httpc_manager,handle_connect_and_send,5}, > > {httpc_manager,handle_info,2}, > > {gen_server,handle_msg,5}, > > {proc_lib,init_p_do_apply,3}]} > > in function gen_server:terminate/6 > > ancestors: [httpc_profile_sup,httpc_sup,inets_sup,<0.46.0>] > > messages: [{'EXIT',<0.16755.36>,normal}, > > {connect_and_send,<0.16752.36>,#Ref<0.0.17.61366>, > > > > > > and subsequent messages were related to the supervisor trying to restart > the > > profile manager... and failing.. > > > > Now my point is... why did the node have to crash.. just because the > manager > > has to be restarted ? > > and why does the crash.dump always keep telling me im out of memory.. > > > > The problem is.. I thought erlang was built to be fault tolerant.. the > > choice of me using erlang had a LOT to do with doing away with the having > to > > code defensively.. "let it crash" and all that .. just make sure u have a > > supervisor that restarts ur process and everything will just work fine... > > but my experience is that most of the time.. simple process restarts > bring > > the whole node crashing down... > > Erlang was designed to be fault-tolerant. The point is that if you crash > then > "somebody else" has to diagnose the error - "you" can't diagnose the > error because > you are dead. Now if a process crashes, some other process can detect > and fix the error. > If you use otp behaviors this mechanism is hidden from you in the > supervisor hierarchy. > But if the entire node crashes then some other node must fix the > error. In raw erlang > (ie not using the OTP libraries) their is a spawn_link/4 primitive > precisely to propagate > errors over node boundaries. > > So to make an entire node fault-tolerant you need more than one node - > the other node(s) > should fix the error. > > The problem with error is that some you can fix at run-time others you > can't. Running out of > memory is something that you typically can't easily fix. If you run > out of memory you have to > kill something to reclaim memory - but what should you kill? - this > depends upon your application. In such circumstances Erlang decides it > can't do anything sensible and the whole node dies - with hopefully a > helpful error message. > > When things crash you're supposed to leave enough information behind > so you can figure out > why things went wrong - the default behavior of the system is to try > and leave some helpful clues as to what went wrong (so you don't have > to code this yourself). Mostly you don't > have to mess with this. For example, if you try to open a non existent > file you might generate > an 'enoent' exception, if your application only opens a single file > then this information will be > enough for you to discover and fix your error. But if your application > opens many different > files you'll have to add an explicit exception exit({missingFile, > File}) at appropriate places > in your code. > > No matter how good any exception and error recovery strategy is there > will always be corner > cases where automatic strategies fail, and here you're back to manual > debugging. > > You have a pretty good clue - you've got a memory leak - some process > is consuming too > much memory - so now you have to start using various tools to figure > out which process run > wild. > > I suspect this is an uncommon error - if it were common then you'd > find more tools > for detecting rogue processes in the standard libraries - their > absence indicates that either this > is not a common problem, or that it is a common problem, but resolving > it it easy. > > I know that none of what I've said helps you find your specific > problem - but you have > to understand the limitations of the system - a dead man cannot > determine the reason why they themselves died - somebody else has to > do this ... > > Good luck finding your error .. > > Cheers > > /Joe > > > > > > Would deeply appreciate it someone could tell me if there is something > > fundamentally wrong with the way im doing things.. or if anyones been in > my > > situation and have had some enlightenments. > > > > thanks in advance > > -Arun > > > From arun.suresh@REDACTED Thu Aug 5 19:09:14 2010 From: arun.suresh@REDACTED (Arun Suresh) Date: Thu, 5 Aug 2010 10:09:14 -0700 Subject: [erlang-questions] erlang woes In-Reply-To: References: Message-ID: Thank you for the suggestion Joseph.. But i was aware of this bug.. it was fixed in inets-5.3.2.. my prod system is currently running with the patched inets... so i dont think that is cause of the crash.. -Arun On Thu, Aug 5, 2010 at 7:21 AM, Joseph Wayne Norton wrote: > > Please check this patch for R13B04 - I believe it will fix your memory > leak. > > > http://hibari.git.sourceforge.net/git/gitweb.cgi?p=hibari/patches;a=blob;f=otp_src_R13B04-httpc-memoryleak.patch;h=14eb7a603dc7f0f49f6685260dd0d0bebddc2f4c;hb=HEAD > > This patch was posted previously to one of the erlang.org mailing lists. > > I don't know your application's requirements but I would recommend most of > these patches for a vanilla R13B04 erlang system: > > http://hibari.git.sourceforge.net/git/gitweb.cgi?p=hibari/patches;a=tree > > Instructions for applying these patches can found here: > > > http://hibari.sourceforge.net/webpage-README-ERL.html#_erlang_otp_to_download > > regards, > > > > On Thu, 05 Aug 2010 22:55:27 +0900, Evans, Matthew > wrote: > > Are you using pg2 in a distributed system? There is a known (should be >> fixed) bug where pg2 was eating memory like crazy. >> >> -----Original Message----- >> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On >> Behalf Of Arun Suresh >> Sent: Thursday, August 05, 2010 1:37 AM >> To: erlang-questions@REDACTED >> Subject: [erlang-questions] erlang woes >> >> Hello folks.. >> >> Ive been using erlang for a while now.. and ive got a production system up >> and running from scratch.. but there are some really annoying aspects of >> the >> platform.. the most fundamental of which is the fact that when a node >> crashes it is very hard to figure out exactly why.. Almost ALL the time >> what >> i see in the crash dump is something akin to : >> >> =erl_crash_dump:0.1 >> Wed Aug 4 21:50:01 2010 >> Slogan: eheap_alloc: Cannot allocate 1140328500 bytes of memory (of type >> "heap"). >> System version: Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] >> [async-threads:0] [hipe] [kernel-poll:false] >> Compiled: Tue May 11 12:37:38 2010 >> Taints: >> >> >> at which point I start to comb the sasl logs... and 9 out of 10 times... >> it >> is because some critical process has died and the supervisor is busy >> restarting it.. for example, the other day.. my node crashed and from the >> sasl logs.. i see that the http manager for a profile I had created had >> crashed like so : >> >> =CRASH REPORT==== 4-Aug-2010::21:47:09 === >> crasher: >> initial call: httpc_manager:init/1 >> pid: <0.185.0> >> registered_name: httpc_manager_store >> exception exit: {{case_clause, >> [{handler_info,#Ref<0.0.17.61372>,<0.17225.36>, >> undefined,<0.15665.36>,initiating}]}, >> [{httpc_manager,handle_connect_and_send,5}, >> {httpc_manager,handle_info,2}, >> {gen_server,handle_msg,5}, >> {proc_lib,init_p_do_apply,3}]} >> in function gen_server:terminate/6 >> ancestors: [httpc_profile_sup,httpc_sup,inets_sup,<0.46.0>] >> messages: [{'EXIT',<0.16755.36>,normal}, >> {connect_and_send,<0.16752.36>,#Ref<0.0.17.61366>, >> >> >> and subsequent messages were related to the supervisor trying to restart >> the >> profile manager... and failing.. >> >> Now my point is... why did the node have to crash.. just because the >> manager >> has to be restarted ? >> and why does the crash.dump always keep telling me im out of memory.. >> >> The problem is.. I thought erlang was built to be fault tolerant.. the >> choice of me using erlang had a LOT to do with doing away with the having >> to >> code defensively.. "let it crash" and all that .. just make sure u have a >> supervisor that restarts ur process and everything will just work fine... >> but my experience is that most of the time.. simple process restarts bring >> the whole node crashing down... >> >> Would deeply appreciate it someone could tell me if there is something >> fundamentally wrong with the way im doing things.. or if anyones been in >> my >> situation and have had some enlightenments. >> >> thanks in advance >> -Arun >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > -- > norton@REDACTED > From arun.suresh@REDACTED Thu Aug 5 19:13:11 2010 From: arun.suresh@REDACTED (Arun Suresh) Date: Thu, 5 Aug 2010 10:13:11 -0700 Subject: [erlang-questions] erlang woes In-Reply-To: References: Message-ID: Incidentally, I do use this exact same command line... But I am more concerned about which proces actually killed the node... If you look at the crash.dump file... I grep on all lines with 'Stack+heap: .......' (basically all processes with Stack+heap > 7 digits) This gives me a fair account of which were the memory hungry processes when the node went down -but- It does NOT tell me which process had made the request for the memory.. which could not be allocated.. and which brought the node down !! -Arun On Thu, Aug 5, 2010 at 3:42 AM, Attila Rajmund Nohl wrote: > 2010/8/5, Joe Armstrong : > [...] > > I suspect this is an uncommon error - if it were common then you'd > > find more tools > > for detecting rogue processes in the standard libraries - their > > absence indicates that either this > > is not a common problem, or that it is a common problem, but resolving > > it it easy. > > I've seen two tools that can be used to detect rogue processes. I also > have some one-liners on the local project Wikipage. So I'd go for the > second option: it's a common problem, but not that hard to resolve (or > at least to find the offending process - it's rather more complicated > to find out why it leaks and I guess it's too hard to write an > automated tool). > > By the way, the one-liners (in case some of you find it useful): > > Which process uses the most memory: > lists:reverse(lists:keysort(2,[{P, erlang:process_info(P, heap_size)} > || P <- erlang:processes()])). > > Which ets table uses the most memory: > lists:reverse(lists:keysort(2,[{T, ets:info(T, memory)} || T <- > ets:all()])). > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From fritchie@REDACTED Thu Aug 5 21:57:18 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 05 Aug 2010 14:57:18 -0500 Subject: [erlang-questions] erlang woes In-Reply-To: Message of "Thu, 05 Aug 2010 10:13:11 PDT." Message-ID: <77636.1281038238@snookles.snookles.com> Arun Suresh wrote: as> It does NOT tell me which process had made the request for the as> memory.. which could not be allocated.. and which brought the node as> down !! Sure it does, you just need to know where to look. :-) If your eyes haven't adjusted to looking at those dump files yet, I recommend the crashdump_viewer tool. http://www.erlang.org/doc/apps/observer/crashdump_ug.html http://www.erlang.org/doc/man/crashdump_viewer.html -Scott From arun.suresh@REDACTED Thu Aug 5 22:22:12 2010 From: arun.suresh@REDACTED (Arun Suresh) Date: Thu, 5 Aug 2010 13:22:12 -0700 Subject: [erlang-questions] erlang woes In-Reply-To: <77636.1281038238@snookles.snookles.com> References: <77636.1281038238@snookles.snookles.com> Message-ID: hello scott.. thanx for the recommendation... but suppose i dont have acces to the crashdump viewer on the prod machine... and moving the crashdump between machines is not an option... where in the crash dump files should i look for the process that asked for the memory ? -Arun On Thu, Aug 5, 2010 at 12:57 PM, Scott Lystig Fritchie < fritchie@REDACTED> wrote: > Arun Suresh wrote: > > as> It does NOT tell me which process had made the request for the > as> memory.. which could not be allocated.. and which brought the node > as> down !! > > Sure it does, you just need to know where to look. :-) If your eyes > haven't adjusted to looking at those dump files yet, I recommend the > crashdump_viewer tool. > > http://www.erlang.org/doc/apps/observer/crashdump_ug.html > http://www.erlang.org/doc/man/crashdump_viewer.html > > -Scott > From nem@REDACTED Fri Aug 6 00:45:04 2010 From: nem@REDACTED (Geoff Cant) Date: Thu, 05 Aug 2010 15:45:04 -0700 Subject: [erlang-questions] erlang woes In-Reply-To: (Arun Suresh's message of "Thu, 5 Aug 2010 13:22:12 -0700") References: <77636.1281038238@snookles.snookles.com> Message-ID: Arun Suresh writes: > On Thu, Aug 5, 2010 at 12:57 PM, Scott Lystig Fritchie < > fritchie@REDACTED> wrote: > >> Arun Suresh wrote: >> >> as> It does NOT tell me which process had made the request for the >> as> memory.. which could not be allocated.. and which brought the node >> as> down !! In a system like erlang, the particular thing that tries an allocation that crashes the system is more often than not _unlikely_ to be the real culprit. Examples scenarios include: A) The allocator need not be a process. ETS tables, drivers, and a host of other thing allocate memory, and these are not all tied to specific processes. B) If process A allocates 11Gb (all remaining memory) and then process B asks for 2 bytes, and the system crashes because this allocation can't be honoured - process B is not the real culprit. The crashdumps that you have almost certainly contain enough information for you to diagnose the problem. Load them up in crashdump viewer and browse around. Maybe there's a process with a huge heap - then either this process is broken, or maybe it runs too slowly (if it's the message queue for the process that's consuming all the heap space). If its message queue is consuming the memory, then maybe it's a downstream process not keeping up and this process is just buffering for the rest of the system. Maybe it's not a process at all - it could be an ETS table or two that are not being cleaned out in a timely way. The crashdump viewer lets you investigate all of this easily. I wouldn't try to examine them by hand. Cheers, -- Geoff Cant From allanwegan@REDACTED Fri Aug 6 01:12:16 2010 From: allanwegan@REDACTED (Allan Wegan) Date: Fri, 06 Aug 2010 01:12:16 +0200 Subject: [erlang-questions] record type as variable In-Reply-To: References: Message-ID: <4C5B4550.1000600@allanwegan.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2010-08-04 19:05, Noel Bush wrote: > Is there any way to specify the type of a record using a variable? I > want to be able to do this sort of thing: > > create(Type, Name) -> > #Type{name = Name}. No, there isn't (at least no easy one, as i think). But what you can do is: Use a generic record type with the subtype as an atom field defined in a common header file. Build one module for each subtype named as the subtype and containing all needed methods for using the subtype. Build one API module that dispatches function calls to the more specific functions in the subtype modules. % Generic type specification in the header file that is included in the API and subtype modules: - -record('sometype', { subtype :: atom(), data :: term() }). % Dispatcher functions in API module named 'sometype': get_name(#sometype{subtype = M} = O) -> M:get_name(O). set_name(#sometype{subtype = M} = O, Name) -> M:set_name(O, Name). % Specification of the data in the subtype1 module: - -record('subtype1_data', { property1 :: term(), name :: term(), propertyN :: term() }). % Functions in 'subtype1' module: get_name( #sometype{subtype = 'subtype1', data = #subtype1_data{name = Name} ) -> Name . set_name( #sometype{subtype = 'subtype1', data = #subtype1_data{} = Data}, Name ) -> #sometype{subtype = 'subtype1', data = Data#subtype1_data{name = Name} . In regular code you would use something like that to access the name property: Name = sometype:get_name(I), % where I is an instance of #sometype{} with any subtype. New_i = sometype:set_name(I, Name), % where I is an instance of #sometype{} with any subtype and name is the new name. This pattern allows to use records for sets of rather generic types that share a common API facade. You are obviously not limited to dispatchers in the API module. It is a convenient place for higher level functions, that do not have to know the exact implementation of each subtype. It's a little like abstract classes, interfaces, and their implementations in OOP - but without any inheritance or other language support of course... ;) - -- Allan Wegan Jabber: allanwegan@REDACTED ICQ: 209459114 Phone: +49 40 6732962 Sch?neberger Strasse 60, 22149 Hamburg -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (MingW32) iQEcBAEBAgAGBQJMW0VQAAoJENm5axHh7AcxdgoH/0+Z+M2OKpy0INS0/cvbTY78 /VLUStFwwrYO52nZgzyRF1/HBZNaQCmihUH+mP2hvLGZf9bicLwXk+pBc+vG6SCL wyN69/EcNv+gjFhf1db+MSK2i6z4tccl+7Ia8mKhEDDUuc8G9s1/4vIm8Lez8I1R LtBNNU9D4QdihZsb60TkXOXW5yaEByPiiOBuYzoIw8iKV35Zbpo4Q9PzciIthBmy O/CME44lU0xLiFnRBHN2lqoNU9F4SB4bFyGKTy2Z7P5j0oy1uqYyh5E71AomDE0j I9a/Tx4OBizY31lEFlSWDVDXh55GLHVMK7sqkamuLnylIJ6wwz3vKo4WbWPsWFo= =V80L -----END PGP SIGNATURE----- From norton@REDACTED Thu Aug 5 16:21:19 2010 From: norton@REDACTED (Joseph Wayne Norton) Date: Thu, 05 Aug 2010 23:21:19 +0900 Subject: [erlang-questions] erlang woes In-Reply-To: References: Message-ID: Please check this patch for R13B04 - I believe it will fix your memory leak. http://hibari.git.sourceforge.net/git/gitweb.cgi?p=hibari/patches;a=blob;f=otp_src_R13B04-httpc-memoryleak.patch;h=14eb7a603dc7f0f49f6685260dd0d0bebddc2f4c;hb=HEAD This patch was posted previously to one of the erlang.org mailing lists. I don't know your application's requirements but I would recommend most of these patches for a vanilla R13B04 erlang system: http://hibari.git.sourceforge.net/git/gitweb.cgi?p=hibari/patches;a=tree Instructions for applying these patches can found here: http://hibari.sourceforge.net/webpage-README-ERL.html#_erlang_otp_to_download regards, On Thu, 05 Aug 2010 22:55:27 +0900, Evans, Matthew wrote: > Are you using pg2 in a distributed system? There is a known (should be > fixed) bug where pg2 was eating memory like crazy. > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] > On Behalf Of Arun Suresh > Sent: Thursday, August 05, 2010 1:37 AM > To: erlang-questions@REDACTED > Subject: [erlang-questions] erlang woes > > Hello folks.. > > Ive been using erlang for a while now.. and ive got a production system > up > and running from scratch.. but there are some really annoying aspects of > the > platform.. the most fundamental of which is the fact that when a node > crashes it is very hard to figure out exactly why.. Almost ALL the time > what > i see in the crash dump is something akin to : > > =erl_crash_dump:0.1 > Wed Aug 4 21:50:01 2010 > Slogan: eheap_alloc: Cannot allocate 1140328500 bytes of memory (of type > "heap"). > System version: Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > Compiled: Tue May 11 12:37:38 2010 > Taints: > > > at which point I start to comb the sasl logs... and 9 out of 10 times... > it > is because some critical process has died and the supervisor is busy > restarting it.. for example, the other day.. my node crashed and from the > sasl logs.. i see that the http manager for a profile I had created had > crashed like so : > > =CRASH REPORT==== 4-Aug-2010::21:47:09 === > crasher: > initial call: httpc_manager:init/1 > pid: <0.185.0> > registered_name: httpc_manager_store > exception exit: {{case_clause, > [{handler_info,#Ref<0.0.17.61372>,<0.17225.36>, > undefined,<0.15665.36>,initiating}]}, > [{httpc_manager,handle_connect_and_send,5}, > {httpc_manager,handle_info,2}, > {gen_server,handle_msg,5}, > {proc_lib,init_p_do_apply,3}]} > in function gen_server:terminate/6 > ancestors: [httpc_profile_sup,httpc_sup,inets_sup,<0.46.0>] > messages: [{'EXIT',<0.16755.36>,normal}, > {connect_and_send,<0.16752.36>,#Ref<0.0.17.61366>, > > > and subsequent messages were related to the supervisor trying to restart > the > profile manager... and failing.. > > Now my point is... why did the node have to crash.. just because the > manager > has to be restarted ? > and why does the crash.dump always keep telling me im out of memory.. > > The problem is.. I thought erlang was built to be fault tolerant.. the > choice of me using erlang had a LOT to do with doing away with the > having to > code defensively.. "let it crash" and all that .. just make sure u have a > supervisor that restarts ur process and everything will just work fine... > but my experience is that most of the time.. simple process restarts > bring > the whole node crashing down... > > Would deeply appreciate it someone could tell me if there is something > fundamentally wrong with the way im doing things.. or if anyones been in > my > situation and have had some enlightenments. > > thanks in advance > -Arun > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- norton@REDACTED From zvi.avraham@REDACTED Fri Aug 6 13:47:35 2010 From: zvi.avraham@REDACTED (Zvi) Date: Fri, 6 Aug 2010 04:47:35 -0700 (PDT) Subject: Corba Help In-Reply-To: References: <591FA96352AC9946A403D860A60166520105941A@ESESSCMS0357.eemea.ericsson.se> Message-ID: <9b1b921f-5ee0-4e6d-a52c-b0f5c0c6b2a4@q35g2000yqn.googlegroups.com> Hi Niclas, I need a minimal example of using CORBA from Erlang. I.e. simple "Hello World" stuff using orber application. Can you give me any pointers / examples? Thanks in advance. Zvi On Aug 5, 3:31?pm, Niclas Eklund wrote: > Hello! > > Check out the chapter "6.8 Typecode, Identity and Name Access Functions" > in Orber's User's Guide: > > http://www.erlang.org/doc/apps/orber/ch_idl_to_erlang_mapping.html#id... > > The record is defined in a generated hrl-file after you've compiled the > IDL-file. Remember to include that file. > > In fact, you should read all in "6 OMG IDL to Erlang Mapping", since it > describe step by step how one should do. > > The debugging chapter should also be helpfull -http://www.erlang.org/doc/apps/orber/ch_debugging.html > > You will probably find the IC backend erl_template rather usefull (will > generate a complete call-back module). Note, it will overwrite any > existing version of the *impl.erl file. Read more about it in chapter 6. > > If possible you should use Light IFR when compiling the IDL files and > starting Orber. > > Best regards, > > Niclas @ Erlang/OTP > > > > On Thu, 5 Aug 2010, Ali Yakout wrote: > > Hi, > > > I'm new to Corba and I can't find my way through... > > I want to do the same operation done with the following Java code: > > > //----------------------------------------- > > org.omg.CORBA.Any any = _orb.create_any(); > > any.insert_wstring("1.1"); > > com.lhs.ccb.soi.types.NvElementI namedValue = new com.lhs.ccb.soi.types.NvElementI("CS_CODE", any, 0, (short)0); > > NvElementI[] input = new NvElementI[] {namedValue}; > > NvElementI[] result = execute("CUSTOMERS.SEARCH", input); > > //----------------------------------------- > > > My problem is that I don't know how to create an object of NvElement with the above values. > > > I have the following types: > > > %%------------------------------------------ > > 254> com_lhs_ccb_soi_types_NvElementI:tc(). > > {tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", > > ? ? ? ? ? "NvElementI", > > ? ? ? ? ? [{"name",{tk_string,0}}, > > ? ? ? ? ? ?{"value",tk_any}, > > ? ? ? ? ? ?{"flags",tk_long}, > > ? ? ? ? ? ?{"length",tk_short}]} > > 255> com_lhs_ccb_soi_types_NvElementListI:tc(). > > {tk_sequence,{tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", > > ? ? ? ? ? ? ? ? ? ? ? ?"NvElementI", > > ? ? ? ? ? ? ? ? ? ? ? ?[{"name",{tk_string,0}}, > > ? ? ? ? ? ? ? ? ? ? ? ? {"value",tk_any}, > > ? ? ? ? ? ? ? ? ? ? ? ? {"flags",tk_long}, > > ? ? ? ? ? ? ? ? ? ? ? ? {"length",tk_short}]}, > > ? ? ? ? ? ? 0} > > %%------------------------------------------ > > > I tried the following but I got 'MARSHAL' exception. > > > 244> AnyStr=any:create(orber_tc:wstring(3),"1.1"). > > {any,{tk_wstring,3},"1.1"} > > 245> NV=any:create(com_lhs_ccb_soi_types_NvElementI:tc(),["CS_CODE",AnyStr,0,0]). > > {any,{tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", > > ? ? ? ? ? ? ? ?"NvElementI", > > ? ? ? ? ? ? ? ?[{"name",{tk_string,0}}, > > ? ? ? ? ? ? ? ? {"value",tk_any}, > > ? ? ? ? ? ? ? ? {"flags",tk_long}, > > ? ? ? ? ? ? ? ? {"length",tk_short}]}, > > ? ? ["CS_CODE",{any,{tk_wstring,3},"1.1"},0,0]} > > 246> com_lhs_ccb_soi_ServiceObjectI:executeI(CMIObj,"CUSTOMERS.SEARCH",[NV]). > > > ** exception throw: {'EXCEPTION',{'MARSHAL',[],1163001856,'COMPLETED_NO'}} > > ? ? in function ?corba:raise/1 > > ? ? in call from orber_iiop:request/8 > > > So my question is, how to create Corba types in Erlang? > > > Thanks, > > Ali > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > Seehttp://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED From sgardell@REDACTED Fri Aug 6 18:49:43 2010 From: sgardell@REDACTED (Gardell, Steven) Date: Fri, 6 Aug 2010 12:49:43 -0400 Subject: SMP active? Message-ID: <6B3E29721F78364AA2C43697AFB15D9102F78EF1@sonusmail07.sonusnet.com> I am running Erlang on a 4 Core Dell machine: Linux xxx.sonusnet.com 2.6.9-55.ELsmp #1 SMP Fri Apr 20 16:36:54 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] When I run my Erlang program (a Megaco test program), top shows only only one of the cores being used to any meaningful level. Something like (5%, 97%, 97%, 97%) idle. Push it harder and, not too surprisingly, it starts presenting errors. I have tried the -smp and -S options, but there is no change in behavior (which is consistent with documentation, since smp should be enabled by default). Suggestions? Thanks! From fritchie@REDACTED Fri Aug 6 19:04:08 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Fri, 06 Aug 2010 12:04:08 -0500 Subject: [erlang-questions] erlang woes In-Reply-To: Message of "Thu, 05 Aug 2010 13:22:12 PDT." Message-ID: <36522.1281114248@snookles.snookles.com> Arun, the erl_crash.dump file uses the regexp "^=" to delimit section- or subsection-like things. Or delimit on a process basis. Look for big numbers in "Stack+heap:" lines under the per-process summaries. -Scott From rapsey@REDACTED Fri Aug 6 19:24:22 2010 From: rapsey@REDACTED (Rapsey) Date: Fri, 6 Aug 2010 19:24:22 +0200 Subject: [erlang-questions] SMP active? In-Reply-To: <6B3E29721F78364AA2C43697AFB15D9102F78EF1@sonusmail07.sonusnet.com> References: <6B3E29721F78364AA2C43697AFB15D9102F78EF1@sonusmail07.sonusnet.com> Message-ID: If your test program only uses one process for the work that it does, erlang will only use up one process on the machine. Sergej On Fri, Aug 6, 2010 at 6:49 PM, Gardell, Steven wrote: > I am running Erlang on a 4 Core Dell machine: > > Linux xxx.sonusnet.com 2.6.9-55.ELsmp #1 SMP Fri Apr 20 16:36:54 EDT > 2007 x86_64 x86_64 x86_64 GNU/Linux > > Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:0] [hipe] [kernel-poll:false] > > When I run my Erlang program (a Megaco test program), top shows only > only one of the cores being used to any meaningful level. Something > like (5%, 97%, 97%, 97%) idle. Push it harder and, not too surprisingly, > it starts presenting errors. I have tried the -smp and -S options, but > there is > no change in behavior (which is consistent with documentation, since smp > should > be enabled by default). > > Suggestions? Thanks! > From sgardell@REDACTED Fri Aug 6 20:17:29 2010 From: sgardell@REDACTED (Gardell, Steven) Date: Fri, 6 Aug 2010 14:17:29 -0400 Subject: [erlang-questions] SMP active? In-Reply-To: References: <6B3E29721F78364AA2C43697AFB15D9102F78EF1@sonusmail07.sonusnet.com> Message-ID: <6B3E29721F78364AA2C43697AFB15D9102152A5D@sonusmail07.sonusnet.com> Thanks. There are lots of processes overall (one per call ~20K), but I imagine that this is still the root problem. Duh. At the core I gather there is a single process responsible for dispatching incoming megaco requests back to my waiting Erlang process. Unfortunately this performance hit seems be quite dependant the number of current calls (as well, of course, as the call rate). I start seeing failures with an ets lookup being done by the stack. Since ets is, I gather, singly threaded, this might be as good as it gets without using multiple UDP ports and multiple megaco connection handlers. But depending on ets, even that might not help... -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Rapsey Sent: Friday, August 06, 2010 1:24 PM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] SMP active? If your test program only uses one process for the work that it does, erlang will only use up one process on the machine. Sergej On Fri, Aug 6, 2010 at 6:49 PM, Gardell, Steven wrote: > I am running Erlang on a 4 Core Dell machine: > > Linux xxx.sonusnet.com 2.6.9-55.ELsmp #1 SMP Fri Apr 20 16:36:54 EDT > 2007 x86_64 x86_64 x86_64 GNU/Linux > > Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:0] [hipe] [kernel-poll:false] > > When I run my Erlang program (a Megaco test program), top shows only > only one of the cores being used to any meaningful level. Something > like (5%, 97%, 97%, 97%) idle. Push it harder and, not too > surprisingly, it starts presenting errors. I have tried the -smp and > -S options, but there is no change in behavior (which is consistent > with documentation, since smp should be enabled by default). > > Suggestions? Thanks! > From jwilberding@REDACTED Fri Aug 6 21:03:50 2010 From: jwilberding@REDACTED (Jordan Wilberding) Date: Fri, 6 Aug 2010 14:03:50 -0500 Subject: [ANN] ErlangCamp Chicago 2010 Message-ID: We are pleased to announce ErlangCamp Chicago 2010 (http://erlangcamp.com/) has gone live and is ready to take registration. If you want to learn how to build production ready Erlang/OTP systems, this workshop is for you. Thanks, Jordan Wilberding From comptekki@REDACTED Fri Aug 6 23:31:51 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 6 Aug 2010 15:31:51 -0600 Subject: gen_smtp question Message-ID: I'm trying to get gen_smtp from: http://github.com/Vagabond/gen_smtp to send a message to my email, but it just seems to send the text to the console. Has anyone used this to send emails? I start the server then use the gen_smtp_client to send a test message. The client works if I use another mail relay, but If I set the rely to the gen_smtp_server (my machine) it doesn't seem to go beyond the console, i.e., the output shows on the console, but I'm not sure if it supposed to send it out to the To: email address also. Initially I started the server which it defaults to localhost, 0.0.0.0. I change the default to use my computers IP, but it still works the same. Maybe that's what it's supposed to do using the gen_smtp_server_example. Any ideas? thx, -wes From kaiduanx@REDACTED Sat Aug 7 02:35:12 2010 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Fri, 6 Aug 2010 20:35:12 -0400 Subject: Interesting result of performance with string ++ vs prepend to list of string to append a list of string Message-ID: Hi, all, For a common operation like appending a list of string to get a new string, I tried two ways, simple ++, add the string to the head of a list of string and then reverse as below, L1 = "Welcome to erlang land\r\n", L2 = "Welcome to Canada\r\n", 1) L = L1 ++ L2. 2) R1 = [L1]; R2 = [L2 | R1]; lists:reverse(R2) I expect 2) would be faster than 1) because everyone is talking ++ is slow. However, the test result shows 2) is even faster than 1). 12> test:test_string_contact(1000000). string++ time: 0.031/0.031 micro-seconds (runtime/wallclock) ok 13> test:test_list_contact(1000000). list contact time: 0.125/0.124 micro-seconds (runtime/wallclock) Any idea why? Thanks, Kaiduan -------------- next part -------------- A non-text attachment was scrubbed... Name: test.erl Type: application/octet-stream Size: 3119 bytes Desc: not available URL: From kaiduanx@REDACTED Sat Aug 7 02:37:30 2010 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Fri, 6 Aug 2010 20:37:30 -0400 Subject: Interesting result of performance with string ++ vs prepend to list of string to append a list of string In-Reply-To: References: Message-ID: Sorry a typo, the result is that 1) is faster than 2). On Fri, Aug 6, 2010 at 8:35 PM, Kaiduan Xie wrote: > Hi, all, > > For a common operation like appending a list of string to get a new > string, I tried two ways, simple ++, add the string to the head of a > list of string and then reverse as below, > > L1 = "Welcome to erlang land\r\n", > L2 = "Welcome to Canada\r\n", > > 1) L = L1 ++ L2. > > 2) R1 = [L1]; > ? ?R2 = [L2 ?| R1]; > ? ?lists:reverse(R2) > > I expect 2) would be faster than 1) because everyone is talking ++ is > slow. However, the test result shows 2) is even faster than 1). > > 12> test:test_string_contact(1000000). > string++ time: 0.031/0.031 micro-seconds (runtime/wallclock) > ok > 13> test:test_list_contact(1000000). > list contact time: 0.125/0.124 micro-seconds (runtime/wallclock) > > Any idea why? > > Thanks, > > Kaiduan > From rvirding@REDACTED Sat Aug 7 03:19:27 2010 From: rvirding@REDACTED (Robert Virding) Date: Sat, 7 Aug 2010 03:19:27 +0200 Subject: [erlang-questions] Re: Interesting result of performance with string ++ vs prepend to list of string to append a list of string In-Reply-To: References: Message-ID: Two things: - the two operations you are doing are not equivalent, if you check the resultant lists you will see they are different. - I would not initialise the strings in each test function as you have done. While I think the BEAM does not build strings each time as it did before, it still feels wrong to do it that way. :-) And maybe it is because ++ isn't as bad as people think, as long as you use it wisely. In your case their is not really much else you can do if you actually do want to concatenate the strings. Robert On 7 August 2010 02:37, Kaiduan Xie wrote: > Sorry a typo, the result is that 1) is faster than 2). > > On Fri, Aug 6, 2010 at 8:35 PM, Kaiduan Xie wrote: >> Hi, all, >> >> For a common operation like appending a list of string to get a new >> string, I tried two ways, simple ++, add the string to the head of a >> list of string and then reverse as below, >> >> L1 = "Welcome to erlang land\r\n", >> L2 = "Welcome to Canada\r\n", >> >> 1) L = L1 ++ L2. >> >> 2) R1 = [L1]; >> ? ?R2 = [L2 ?| R1]; >> ? ?lists:reverse(R2) >> >> I expect 2) would be faster than 1) because everyone is talking ++ is >> slow. However, the test result shows 2) is even faster than 1). >> >> 12> test:test_string_contact(1000000). >> string++ time: 0.031/0.031 micro-seconds (runtime/wallclock) >> ok >> 13> test:test_list_contact(1000000). >> list contact time: 0.125/0.124 micro-seconds (runtime/wallclock) >> >> Any idea why? >> >> Thanks, >> >> Kaiduan >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From comptekki@REDACTED Sat Aug 7 05:04:08 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 6 Aug 2010 21:04:08 -0600 Subject: gen_smtp question In-Reply-To: References: Message-ID: On Fri, Aug 6, 2010 at 3:31 PM, Wes James wrote: > I'm trying to get gen_smtp from: > > http://github.com/Vagabond/gen_smtp > > to send a message to my email, but it just seems to send the text to > the console. I got a message from the author - that is what it's supposed to do. thx, -wes From kaiduanx@REDACTED Sat Aug 7 06:19:27 2010 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Sat, 7 Aug 2010 00:19:27 -0400 Subject: [erlang-questions] Re: Interesting result of performance with string ++ vs prepend to list of string to append a list of string In-Reply-To: References: Message-ID: Robert, Thank you very much for the reply. I know the two operations are not equivalent. For the second case, lists:flatten(lists:reverse(R2)) achieves the same result as 1), but lists:flatten(lists:reverse(R2)) even consumes more time than 1). So ++ is the best choice if I want to concatenate strings. Thanks, Kaiduan On Fri, Aug 6, 2010 at 9:19 PM, Robert Virding wrote: > Two things: > > - the two operations you are doing are not equivalent, if you check > the resultant lists you will see they are different. > > - I would not initialise the strings in each test function as you have > done. While I think the BEAM does not build strings each time as it > did before, it still feels wrong to do it that way. :-) > > And maybe it is because ++ isn't as bad as people think, as long as > you use it wisely. In your case their is not really much else you can > do if you actually do want to concatenate the strings. > > Robert > > On 7 August 2010 02:37, Kaiduan Xie wrote: >> Sorry a typo, the result is that 1) is faster than 2). >> >> On Fri, Aug 6, 2010 at 8:35 PM, Kaiduan Xie wrote: >>> Hi, all, >>> >>> For a common operation like appending a list of string to get a new >>> string, I tried two ways, simple ++, add the string to the head of a >>> list of string and then reverse as below, >>> >>> L1 = "Welcome to erlang land\r\n", >>> L2 = "Welcome to Canada\r\n", >>> >>> 1) L = L1 ++ L2. >>> >>> 2) R1 = [L1]; >>> ? ?R2 = [L2 ?| R1]; >>> ? ?lists:reverse(R2) >>> >>> I expect 2) would be faster than 1) because everyone is talking ++ is >>> slow. However, the test result shows 2) is even faster than 1). >>> >>> 12> test:test_string_contact(1000000). >>> string++ time: 0.031/0.031 micro-seconds (runtime/wallclock) >>> ok >>> 13> test:test_list_contact(1000000). >>> list contact time: 0.125/0.124 micro-seconds (runtime/wallclock) >>> >>> Any idea why? >>> >>> Thanks, >>> >>> Kaiduan >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From bob@REDACTED Sat Aug 7 07:13:52 2010 From: bob@REDACTED (Bob Ippolito) Date: Sat, 7 Aug 2010 13:13:52 +0800 Subject: [erlang-questions] Re: Interesting result of performance with string ++ vs prepend to list of string to append a list of string In-Reply-To: References: Message-ID: The best choice is to not concatenate them at all if you don't have to, but if you do have to only concatenate two strings then of course ++ is the best way to do it. On Sat, Aug 7, 2010 at 12:19 PM, Kaiduan Xie wrote: > Robert, > > Thank you very much for the reply. I know the two operations are not > equivalent. For the second case, lists:flatten(lists:reverse(R2)) > achieves the same result as 1), but lists:flatten(lists:reverse(R2)) > even consumes more time than 1). > > So ++ is the best choice if I want to concatenate strings. > > Thanks, > > Kaiduan > > On Fri, Aug 6, 2010 at 9:19 PM, Robert Virding wrote: >> Two things: >> >> - the two operations you are doing are not equivalent, if you check >> the resultant lists you will see they are different. >> >> - I would not initialise the strings in each test function as you have >> done. While I think the BEAM does not build strings each time as it >> did before, it still feels wrong to do it that way. :-) >> >> And maybe it is because ++ isn't as bad as people think, as long as >> you use it wisely. In your case their is not really much else you can >> do if you actually do want to concatenate the strings. >> >> Robert >> >> On 7 August 2010 02:37, Kaiduan Xie wrote: >>> Sorry a typo, the result is that 1) is faster than 2). >>> >>> On Fri, Aug 6, 2010 at 8:35 PM, Kaiduan Xie wrote: >>>> Hi, all, >>>> >>>> For a common operation like appending a list of string to get a new >>>> string, I tried two ways, simple ++, add the string to the head of a >>>> list of string and then reverse as below, >>>> >>>> L1 = "Welcome to erlang land\r\n", >>>> L2 = "Welcome to Canada\r\n", >>>> >>>> 1) L = L1 ++ L2. >>>> >>>> 2) R1 = [L1]; >>>> ? ?R2 = [L2 ?| R1]; >>>> ? ?lists:reverse(R2) >>>> >>>> I expect 2) would be faster than 1) because everyone is talking ++ is >>>> slow. However, the test result shows 2) is even faster than 1). >>>> >>>> 12> test:test_string_contact(1000000). >>>> string++ time: 0.031/0.031 micro-seconds (runtime/wallclock) >>>> ok >>>> 13> test:test_list_contact(1000000). >>>> list contact time: 0.125/0.124 micro-seconds (runtime/wallclock) >>>> >>>> Any idea why? >>>> >>>> Thanks, >>>> >>>> Kaiduan >>>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From hynek@REDACTED Sat Aug 7 10:42:18 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Sat, 7 Aug 2010 10:42:18 +0200 Subject: [erlang-questions] Re: Interesting result of performance with string ++ vs prepend to list of string to append a list of string In-Reply-To: References: Message-ID: As Robert pointed out, when you initialize strings in same function as concatenate them you can be surprised by compiler optimizations. You can check it easily using 'S' option to compiler erlc +\'S\' test.erl --------- 8< ------------ {function, string_contact, 0, 8}. {label,7}. {func_info,{atom,test},{atom,string_contact},0}. {label,8}. {move,{literal,"INVITE sip:1000@REDACTED SIP/2.0\r\nFrom: sip:1000@REDACTED\r\nTo: sip:bob@REDACTED\r\nVia: SIP/2.0/UDP serv er.biloxi.com;branch=z9hgbK4bawer\r\nMax-Forwards: 70\r\nCall-ID: a84b4c76e66710\r\nCSeq: 314519 INVITE\r\nContact: \r\nContent-TYpe: application/sdp\r\nContent-Length: 0\r\n"}, {x,0}}. return. --------- 8< ------------ So you don't concatenate those string at all. It is done in compile time. On Sat, Aug 7, 2010 at 3:19 AM, Robert Virding wrote: > Two things: > > - the two operations you are doing are not equivalent, if you check > the resultant lists you will see they are different. > > - I would not initialise the strings in each test function as you have > done. While I think the BEAM does not build strings each time as it > did before, it still feels wrong to do it that way. :-) > > And maybe it is because ++ isn't as bad as people think, as long as > you use it wisely. In your case their is not really much else you can > do if you actually do want to concatenate the strings. > > Robert > > On 7 August 2010 02:37, Kaiduan Xie wrote: >> Sorry a typo, the result is that 1) is faster than 2). >> >> On Fri, Aug 6, 2010 at 8:35 PM, Kaiduan Xie wrote: >>> Hi, all, >>> >>> For a common operation like appending a list of string to get a new >>> string, I tried two ways, simple ++, add the string to the head of a >>> list of string and then reverse as below, >>> >>> L1 = "Welcome to erlang land\r\n", >>> L2 = "Welcome to Canada\r\n", >>> >>> 1) L = L1 ++ L2. >>> >>> 2) R1 = [L1]; >>> ? ?R2 = [L2 ?| R1]; >>> ? ?lists:reverse(R2) >>> >>> I expect 2) would be faster than 1) because everyone is talking ++ is >>> slow. However, the test result shows 2) is even faster than 1). >>> >>> 12> test:test_string_contact(1000000). >>> string++ time: 0.031/0.031 micro-seconds (runtime/wallclock) >>> ok >>> 13> test:test_list_contact(1000000). >>> list contact time: 0.125/0.124 micro-seconds (runtime/wallclock) >>> >>> Any idea why? >>> >>> Thanks, >>> >>> Kaiduan >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From pablo.platt@REDACTED Sun Aug 8 03:21:16 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Sat, 7 Aug 2010 18:21:16 -0700 (PDT) Subject: cost of passing a gen_server state between functions Message-ID: <198711.41599.qm@web112616.mail.gq1.yahoo.com> Hi, Is there a difference in manipulating the state of a gen_server inside a single function and passing the state to another function that manipulate it? Is there a difference if the othe function is in another module? %The state is been manipulated in the handl_cast function: handle_cast({something, I}, State1) -> OldList = State1#state.l, NewList = [I | OldList], State2 = State#state1{l=NewList}, {noreply, State2}. %handle_cast pass the state to another function that changes it: handle_cast({something, I}, State) -> State2 = do_something(I, State), {noreply, State2}. do_something(I, State) -> State2 = State#state1{l=NewList}, State2. Thanks From bgustavsson@REDACTED Sun Aug 8 09:33:35 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sun, 8 Aug 2010 09:33:35 +0200 Subject: [erlang-questions] Interesting result of performance with string ++ vs prepend to list of string to append a list of string In-Reply-To: References: Message-ID: On Sat, Aug 7, 2010 at 2:35 AM, Kaiduan Xie wrote: > Hi, all, > > For a common operation like appending a list of string to get a new > string, I tried two ways, simple ++, add the string to the head of a > list of string and then reverse as below, > > L1 = "Welcome to erlang land\r\n", > L2 = "Welcome to Canada\r\n", > > 1) L = L1 ++ L2. > > 2) R1 = [L1]; > ? ?R2 = [L2 ?| R1]; > ? ?lists:reverse(R2) > > I expect 2) would be faster than 1) because everyone is talking ++ is > slow. However, the test result shows 2) is even faster than 1). See Myth 2.4 in the Efficiency Guide: http://www.erlang.org/doc/efficiency_guide/myths.html ++ is only bad if you use it improperly in a loop, such as in the naive_reverse/1 example. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From fdmanana@REDACTED Sun Aug 8 15:47:27 2010 From: fdmanana@REDACTED (Filipe David Manana) Date: Sun, 8 Aug 2010 14:47:27 +0100 Subject: Question about parametrized modules Message-ID: I have a function_clause error when calling some functions of a parametrized module which use one of the module parameters. The module in question (from mochiweb) is like: -module(mochiweb_request, [Socket, Method, RawPath, Version, Headers]). -export([get/1]). % ... get(socket) -> Socket; get(scheme) -> case mochiweb_socket:type(Socket) of plain -> http; ssl -> https end; get(method) -> Method; get(raw_path) -> RawPath; get(version) -> Version; get(headers) -> Headers; Then, from another module, which has an instance of that parametrized module (mochiweb_request) I have: 1) a call like MochiReq:get(socket) - this one succeeds 2) a call like MochiReq:get(scheme) - it fails with the stack trace bellow: {error,function_clause, [{mochiweb_request,get, [scheme, {mochiweb_request,#Port<0.2224>,'GET',"/_utils", {1,1}, {10, {"host", {'Host',"localhost:5984"}, {"accept", {'Accept', "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"}, nil, {"accept-language", {'Accept-Language',"en-us,en;q=0.5"}, {"accept-encoding", {'Accept-Encoding',"gzip,deflate"}, {"accept-charset", {'Accept-Charset', "ISO-8859-1,utf-8;q=0.7,*;q=0.7"}, nil,nil}, nil}, {"connection", {'Connection',"keep-alive"}, {"cache-control", {'Cache-Control',"max-age=0"}, nil,nil}, {"cookie", {'Cookie', "5984_recent=testdb; AuthSession=ZmRtYW5hbmE6NEM1RUEyMkE60TPWV73IdXUYV68s4MUDLuI4P6o"}, nil,nil}}}}, {"user-agent", {'User-Agent', "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8"}, {"keep-alive",{'Keep-Alive',"115"},nil,nil}, nil}}}}]}, I suspect there's something missing in mochiweb_request:get(scheme) when referencing the module parameter Socket. Does anyone have an idea an can explain me what's wrong? cheers -- Filipe David Manana, fdmanana@REDACTED "Reasonable men adapt themselves to the world. Unreasonable men adapt the world to themselves. That's why all progress depends on unreasonable men." From wallentin.dahlberg@REDACTED Sun Aug 8 16:19:29 2010 From: wallentin.dahlberg@REDACTED (Wallentin Dahlberg) Date: Sun, 8 Aug 2010 16:19:29 +0200 Subject: [erlang-questions] SMP active? In-Reply-To: <6B3E29721F78364AA2C43697AFB15D9102152A5D@sonusmail07.sonusnet.com> References: <6B3E29721F78364AA2C43697AFB15D9102F78EF1@sonusmail07.sonusnet.com> <6B3E29721F78364AA2C43697AFB15D9102152A5D@sonusmail07.sonusnet.com> Message-ID: There is obviously a serialization point somewhere in your test or some other place underneath. ETS is not single-threaded, it uses the same context as the calling process, or rather it is the process and the corresponding thread. There are some tweaks with ETS, I suggest you read-up on them, but in essence there is one rw-lock per table and a namelookup lock (the metaname lock). Since there is speculation on what the problem is you should use one of the profilers provided in OTP. My favorite is the lock counter, lcnt, (though i may be biased). The lock counter is not hard to use but you need some common sense to interpret the results. You also need to recompile the vm (emulator directory) with lcnt enabled. It has low cpu overhead but will consume a bit of memory. lcnt will identify which locks give the most serialization, if it is a process it will give you the registered name or pid. All locks in the emulator are named (almost all are named with their purpose), and that will give you an idea of whats going on. Percept may also be useful but that one is a bit harder to interpret. Eprof has low overhead (from R14A) and will tell you which functions take up the most time, that is also helpful in identifying bad code. // Bj?rn-Egil 2010/8/6 Gardell, Steven > Thanks. > > There are lots of processes overall (one per call ~20K), but I imagine > that this is still the root problem. Duh. At the core I gather there is > a single process responsible for dispatching incoming megaco requests > back to my waiting Erlang process. Unfortunately this performance hit > seems be quite dependant the number of current calls (as well, of > course, as the call rate). I start seeing failures with an ets lookup > being done by the stack. Since ets is, I gather, singly threaded, this > might be as good as it gets without using multiple UDP ports and > multiple megaco connection handlers. But depending on ets, even that > might not help... > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] > On Behalf Of Rapsey > Sent: Friday, August 06, 2010 1:24 PM > To: erlang-questions@REDACTED > Subject: Re: [erlang-questions] SMP active? > > If your test program only uses one process for the work that it does, > erlang will only use up one process on the machine. > > > Sergej > > On Fri, Aug 6, 2010 at 6:49 PM, Gardell, Steven > wrote: > > > I am running Erlang on a 4 Core Dell machine: > > > > Linux xxx.sonusnet.com 2.6.9-55.ELsmp #1 SMP Fri Apr 20 16:36:54 EDT > > 2007 x86_64 x86_64 x86_64 GNU/Linux > > > > Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] > > [async-threads:0] [hipe] [kernel-poll:false] > > > > When I run my Erlang program (a Megaco test program), top shows only > > only one of the cores being used to any meaningful level. Something > > like (5%, 97%, 97%, 97%) idle. Push it harder and, not too > > surprisingly, it starts presenting errors. I have tried the -smp and > > -S options, but there is no change in behavior (which is consistent > > with documentation, since smp should be enabled by default). > > > > Suggestions? Thanks! > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From hynek@REDACTED Sun Aug 8 19:45:56 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Sun, 8 Aug 2010 19:45:56 +0200 Subject: [erlang-questions] cost of passing a gen_server state between functions In-Reply-To: <198711.41599.qm@web112616.mail.gq1.yahoo.com> References: <198711.41599.qm@web112616.mail.gq1.yahoo.com> Message-ID: There should not be difference because if function do_something is short (not in term of source code but result code) compiler will often inline it and then do same optimization as if work done in function. Even compiler don't inline it should not be big difference because state will be pass by pointer to it and call overhead should be low compared to function itself. Biggest difference will be when you do external call (m:f() call form or worse M:f() ot M:F()) when overhead is bigger and compiler can't inline and can't do any optimization because it would violate hot code swap promise. Note that state is never copied in function call in Erlang. Erlang do copy on write when state is modified but not in function calls even for external calls. If you are curious you can do a lot of things. For example 1/ compile using 'S' option and look at result 2/ write some sort of benchmark and ask if you will see something unexpected On Sun, Aug 8, 2010 at 3:21 AM, Pablo Platt wrote: > Hi, > > Is there a difference in manipulating the state of a gen_server inside a single > function > and passing the state to another function that manipulate it? > Is there a difference if the othe function is in another module? > > %The state is been manipulated in the handl_cast function: > handle_cast({something, I}, State1) -> > ? ?OldList = State1#state.l, > ? ?NewList = [I | OldList], > ? ?State2 = State#state1{l=NewList}, > ? ?{noreply, State2}. > > %handle_cast pass the state to another function that changes it: > handle_cast({something, I}, State) -> > ? ?State2 = do_something(I, State), > ? ?{noreply, State2}. > > do_something(I, State) -> > ? ?State2 = State#state1{l=NewList}, > ? ?State2. > > Thanks > > > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From wiener.guy@REDACTED Sun Aug 8 21:27:32 2010 From: wiener.guy@REDACTED (Guy Wiener) Date: Sun, 8 Aug 2010 22:27:32 +0300 Subject: OOP in Erlang Message-ID: Hello everyone, I came across the mentioning of two undocumented features in Erlang - Parameterized modules and module inheritance. 1. From which version and on are there features included in the Erlang distro? 2. Are these features considered stable? Experimental? Risky? Thanks, Guy From tony.arcieri@REDACTED Sun Aug 8 22:32:06 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Sun, 8 Aug 2010 14:32:06 -0600 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: Message-ID: To my knowledge both features are considered experimental. Neither of these are really "OOP" features. Parameterized modules merely make the module name and a prespecified set of variables implicit/hidden. However, a call to a parameterized module is still just a function call. Objects interpret and dispatch incoming messages. Module inheritance doesn't provide inheritance in the traditional style of OOP. It's more like "mix-ins", if you're familiar with those. If you're really looking for OOP in Erlang, I suggest you take a look at my language Reia, which provides a true "everything is an object"-style object orientation, true single inheritance, real hidden state with better handling of code change, and more. More on Reia: http://wiki.reia-lang.org/wiki/Reia_Programming_Language More on Reia's object orientation: http://www.unlimitednovelty.com/2010/06/reia-everything-is-object.html On Sun, Aug 8, 2010 at 1:27 PM, Guy Wiener wrote: > Hello everyone, > I came across the mentioning of two undocumented features in Erlang - > Parameterized modules and module inheritance. > 1. From which version and on are there features included in the Erlang > distro? > 2. Are these features considered stable? Experimental? Risky? > > Thanks, > Guy > -- Tony Arcieri Medioh! A Kudelski Brand From rvirding@REDACTED Mon Aug 9 02:53:33 2010 From: rvirding@REDACTED (Robert Virding) Date: Mon, 9 Aug 2010 02:53:33 +0200 Subject: [erlang-questions] cost of passing a gen_server state between functions In-Reply-To: <198711.41599.qm@web112616.mail.gq1.yahoo.com> References: <198711.41599.qm@web112616.mail.gq1.yahoo.com> Message-ID: The cost is negligible, literally the cost of one function call. A call to a function in another module costs a little more, *very* little more. This is irrespective of the size of the gen_server state. You should do that which results in the cleanest code, it will be a Big Win in the future. Robert P.S. What is the cleanest code is, of course, very subjective. I, for one, tend to avoid very short, one line, functions. This not because of efficiency but because too many small functions tend to clutter up the code. I think. On 8 August 2010 03:21, Pablo Platt wrote: > Hi, > > Is there a difference in manipulating the state of a gen_server inside a single > function > and passing the state to another function that manipulate it? > Is there a difference if the othe function is in another module? > > %The state is been manipulated in the handl_cast function: > handle_cast({something, I}, State1) -> > ? ?OldList = State1#state.l, > ? ?NewList = [I | OldList], > ? ?State2 = State#state1{l=NewList}, > ? ?{noreply, State2}. > > %handle_cast pass the state to another function that changes it: > handle_cast({something, I}, State) -> > ? ?State2 = do_something(I, State), > ? ?{noreply, State2}. > > do_something(I, State) -> > ? ?State2 = State#state1{l=NewList}, > ? ?State2. > > Thanks > > > > From pablo.platt@REDACTED Mon Aug 9 03:10:43 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Sun, 8 Aug 2010 18:10:43 -0700 (PDT) Subject: [erlang-questions] cost of passing a gen_server state between functions In-Reply-To: References: <198711.41599.qm@web112616.mail.gq1.yahoo.com> Message-ID: <996091.97816.qm@web112610.mail.gq1.yahoo.com> Thank you Hynek and Robert. The reason I asked this question is because I have one gen_server that handles several sessions for the same user. I have several session types for different types of connections: socket, http... When the gen_server receive a message it passes it to an handler?along with the gen_server's state. Each session type?defines a different handler (module). The?handler process the message and reply with a new state. Most of the operations on the state are setting a new value in a record, reading values and adding new items to lists. So from your explanation I understand that there is no copy. The state is passed by reference and that as long as I append new values to the head of lists in the state I'm safe. Simplified example: % In the gen_server: handle_cast({message, Message, SessionType}, State) -> ??? NewState = case SessionType of ??????? socket -> ??????????? socket_session:handle_message(State, Message); ??????? other_type -> ??? ??? ??? other_type_session:handle_message(State, Message); ??????? _ -> ??????????? State ??? end, ??? {noreply, NewState}. % inside the socket session module: handle_message(State, Message) -> ??? do_something_with_message(Message), ??? Log = State#state.log, ??? NewLog = [Message | Log], ??? NewState = State#state{log=NewLog}, ??? NewState. ________________________________ From: Robert Virding To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Mon, August 9, 2010 3:53:33 AM Subject: Re: [erlang-questions] cost of passing a gen_server state between functions The cost is negligible, literally the cost of one function call. A call to a function in another module costs a little more, *very* little more. This is irrespective of the size of the gen_server state. You should do that which results in the cleanest code, it will be a Big Win in the future. Robert P.S. What is the cleanest code is, of course, very subjective. I, for one, tend to avoid very short, one line, functions. This not because of efficiency but because too many small functions tend to clutter up the code. I think. On 8 August 2010 03:21, Pablo Platt wrote: > Hi, > > Is there a difference in manipulating the state of a gen_server inside a single > function > and passing the state to another function that manipulate it? > Is there a difference if the othe function is in another module? > > %The state is been manipulated in the handl_cast function: > handle_cast({something, I}, State1) -> > ? ?OldList = State1#state.l, > ? ?NewList = [I | OldList], > ? ?State2 = State#state1{l=NewList}, > ? ?{noreply, State2}. > > %handle_cast pass the state to another function that changes it: > handle_cast({something, I}, State) -> > ? ?State2 = do_something(I, State), > ? ?{noreply, State2}. > > do_something(I, State) -> > ? ?State2 = State#state1{l=NewList}, > ? ?State2. > > Thanks > > > > From bgustavsson@REDACTED Mon Aug 9 05:49:11 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 9 Aug 2010 05:49:11 +0200 Subject: [erlang-questions] cost of passing a gen_server state between functions In-Reply-To: References: <198711.41599.qm@web112616.mail.gq1.yahoo.com> Message-ID: On Sun, Aug 8, 2010 at 7:45 PM, Hynek Vychodil wrote: > There should not be difference because if function do_something is > short (not in term of source code but result code) compiler will often > inline it and then do same optimization as if work done in function. The compiler does not do any inlining unless inlining has been explicitly enabled (with a compiler option). -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From nick@REDACTED Mon Aug 9 10:30:34 2010 From: nick@REDACTED (Niclas Eklund) Date: Mon, 9 Aug 2010 10:30:34 +0200 (CEST) Subject: [erlang-questions] Re: Corba Help In-Reply-To: <9b1b921f-5ee0-4e6d-a52c-b0f5c0c6b2a4@q35g2000yqn.googlegroups.com> References: <591FA96352AC9946A403D860A60166520105941A@ESESSCMS0357.eemea.ericsson.se> <9b1b921f-5ee0-4e6d-a52c-b0f5c0c6b2a4@q35g2000yqn.googlegroups.com> Message-ID: Hello! Chapter 6 should take you through the process of creating a service rather easy, but here comes a very slimmed version. Run the example in an empty directory. %% Compile the attached file (my.idl) using the light_ifr option (reduces the footprints regarding memory usage): shell> erlc +"{light_ifr,true}" my.idl %% Create the call-back module (equivalent to *Impl.java): shell> erlc +"{be,erl_template}" my.idl %% Edit the module my_server_impl.erl echo(State, Str) -> io:format("~s\n", [Str]), %% Add this line. {reply, ok, State}. %% Compile shell> erlc *.erl %% "Jump-start" Orber. erl> orber:jump_start([{flags,16#80},{iiop_port,2809}]). %% Register interface in the IFR. erl> oe_my:oe_register(). %% Create an object instance. erl> Obj = my_server:oe_create([]). %% Invoke the echo operation. erl> my_server:echo(Obj, "It is easy to use CORBA"). It is easy to use CORBA To gain access to a server instance, simply stor the object reference in the name service and use the "corbaname" schema from the peer to get hold of it. I.e. the same way you'd do using, for example, a Java server. Best regards, Niclas On Fri, 6 Aug 2010, Zvi wrote: > Hi Niclas, > > I need a minimal example of using CORBA from Erlang. I.e. simple > "Hello World" stuff using orber application. > Can you give me any pointers / examples? > > Thanks in advance. > Zvi > > On Aug 5, 3:31?pm, Niclas Eklund wrote: >> Hello! >> >> Check out the chapter "6.8 Typecode, Identity and Name Access Functions" >> in Orber's User's Guide: >> >> http://www.erlang.org/doc/apps/orber/ch_idl_to_erlang_mapping.html#id... >> >> The record is defined in a generated hrl-file after you've compiled the >> IDL-file. Remember to include that file. >> >> In fact, you should read all in "6 OMG IDL to Erlang Mapping", since it >> describe step by step how one should do. >> >> The debugging chapter should also be helpfull -http://www.erlang.org/doc/apps/orber/ch_debugging.html >> >> You will probably find the IC backend erl_template rather usefull (will >> generate a complete call-back module). Note, it will overwrite any >> existing version of the *impl.erl file. Read more about it in chapter 6. >> >> If possible you should use Light IFR when compiling the IDL files and >> starting Orber. >> >> Best regards, >> >> Niclas @ Erlang/OTP >> >> >> >> On Thu, 5 Aug 2010, Ali Yakout wrote: >>> Hi, >> >>> I'm new to Corba and I can't find my way through... >>> I want to do the same operation done with the following Java code: >> >>> //----------------------------------------- >>> org.omg.CORBA.Any any = _orb.create_any(); >>> any.insert_wstring("1.1"); >>> com.lhs.ccb.soi.types.NvElementI namedValue = new com.lhs.ccb.soi.types.NvElementI("CS_CODE", any, 0, (short)0); >>> NvElementI[] input = new NvElementI[] {namedValue}; >>> NvElementI[] result = execute("CUSTOMERS.SEARCH", input); >>> //----------------------------------------- >> >>> My problem is that I don't know how to create an object of NvElement with the above values. >> >>> I have the following types: >> >>> %%------------------------------------------ >>> 254> com_lhs_ccb_soi_types_NvElementI:tc(). >>> {tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", >>> ? ? ? ? ? "NvElementI", >>> ? ? ? ? ? [{"name",{tk_string,0}}, >>> ? ? ? ? ? ?{"value",tk_any}, >>> ? ? ? ? ? ?{"flags",tk_long}, >>> ? ? ? ? ? ?{"length",tk_short}]} >>> 255> com_lhs_ccb_soi_types_NvElementListI:tc(). >>> {tk_sequence,{tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", >>> ? ? ? ? ? ? ? ? ? ? ? ?"NvElementI", >>> ? ? ? ? ? ? ? ? ? ? ? ?[{"name",{tk_string,0}}, >>> ? ? ? ? ? ? ? ? ? ? ? ? {"value",tk_any}, >>> ? ? ? ? ? ? ? ? ? ? ? ? {"flags",tk_long}, >>> ? ? ? ? ? ? ? ? ? ? ? ? {"length",tk_short}]}, >>> ? ? ? ? ? ? 0} >>> %%------------------------------------------ >> >>> I tried the following but I got 'MARSHAL' exception. >> >>> 244> AnyStr=any:create(orber_tc:wstring(3),"1.1"). >>> {any,{tk_wstring,3},"1.1"} >>> 245> NV=any:create(com_lhs_ccb_soi_types_NvElementI:tc(),["CS_CODE",AnyStr,0,0]). >>> {any,{tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", >>> ? ? ? ? ? ? ? ?"NvElementI", >>> ? ? ? ? ? ? ? ?[{"name",{tk_string,0}}, >>> ? ? ? ? ? ? ? ? {"value",tk_any}, >>> ? ? ? ? ? ? ? ? {"flags",tk_long}, >>> ? ? ? ? ? ? ? ? {"length",tk_short}]}, >>> ? ? ["CS_CODE",{any,{tk_wstring,3},"1.1"},0,0]} >>> 246> com_lhs_ccb_soi_ServiceObjectI:executeI(CMIObj,"CUSTOMERS.SEARCH",[NV]). >> >>> ** exception throw: {'EXCEPTION',{'MARSHAL',[],1163001856,'COMPLETED_NO'}} >>> ? ? in function ?corba:raise/1 >>> ? ? in call from orber_iiop:request/8 >> >>> So my question is, how to create Corba types in Erlang? >> >>> Thanks, >>> Ali >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> Seehttp://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> Seehttp://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -------------- next part -------------- A non-text attachment was scrubbed... Name: my.idl Type: text/x-idl Size: 81 bytes Desc: URL: From rtrlists@REDACTED Mon Aug 9 13:02:36 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 9 Aug 2010 12:02:36 +0100 Subject: [erlang-questions] Re: gen_smtp question In-Reply-To: References: Message-ID: On Sat, Aug 7, 2010 at 4:04 AM, Wes James wrote: > On Fri, Aug 6, 2010 at 3:31 PM, Wes James wrote: > > I'm trying to get gen_smtp from: > > > > http://github.com/Vagabond/gen_smtp > > > > to send a message to my email, but it just seems to send the text to > > the console. > > I got a message from the author - that is what it's supposed to do. > > thx, > > -wes > > > Yeah, the way I understand it, you can use gen_smtp to implement your own desired server behaviour. But it doesn't come with a full mail server itself, just the text based example one. Saying that, it is pretty easy to modify that to suit your needs. Robby From rtrlists@REDACTED Mon Aug 9 13:08:41 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 9 Aug 2010 12:08:41 +0100 Subject: [erlang-questions] Question about parametrized modules In-Reply-To: References: Message-ID: On Sun, Aug 8, 2010 at 2:47 PM, Filipe David Manana wrote: > I have a function_clause error when calling some functions of a > parametrized > module which use one of the module parameters. > The module in question (from mochiweb) is like: > > -module(mochiweb_request, [Socket, Method, RawPath, Version, Headers]). > -export([get/1]). > > % ... > > get(socket) -> > Socket; > get(scheme) -> > case mochiweb_socket:type(Socket) of > plain -> > http; > ssl -> > https > end; > get(method) -> > Method; > get(raw_path) -> > RawPath; > get(version) -> > Version; > get(headers) -> > Headers; > > Then, from another module, which has an instance of that parametrized > module > (mochiweb_request) I have: > > 1) a call like MochiReq:get(socket) - this one succeeds > > 2) a call like MochiReq:get(scheme) - it fails with the stack trace bellow: > > {error,function_clause, > [{mochiweb_request,get, > [scheme, > {mochiweb_request,#Port<0.2224>,'GET',"/_utils", > > Stupid question, sorry, but you did recompile and are running your new version, yes? Because that error looks like the mochiweb_request:get function simply doesn't understand you arguments. And my first guess would be that it doesn't understand 'scheme'. Robby From aidankane@REDACTED Mon Aug 9 16:47:14 2010 From: aidankane@REDACTED (Aidan Kane) Date: Mon, 9 Aug 2010 22:47:14 +0800 Subject: hello..b Message-ID: 0Hello: Last week, I have ordered china product Apple iPad 64GB this w e bsite:dhsellso.com I've received the item today It's amazing! The item is original, brand new and has high quality, but it's much cheap I'm pleased to share this good news with you! I believe you will find what you want there and have an good experience on shopping from them Regards! From dawid.figiel@REDACTED Mon Aug 9 17:04:06 2010 From: dawid.figiel@REDACTED (Dawid Figiel) Date: Mon, 9 Aug 2010 17:04:06 +0200 Subject: [erlang-questions] hello..b In-Reply-To: References: Message-ID: How is this great news related to Erlang ?;> On Mon, Aug 9, 2010 at 4:47 PM, Aidan Kane wrote: > 0Hello: > Last week, I have ordered china product Apple iPad 64GB > this w e bsite:dhsellso.com > I've received the item today > It's amazing! The item is original, brand new and has high quality, > but it's much cheap I'm pleased to share this good news with you! > I believe you will find what you want there and have an good experience > on shopping from them > Regards! > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Dawid From marcel.meyer@REDACTED Mon Aug 9 17:24:25 2010 From: marcel.meyer@REDACTED (Marcel Meyer) Date: Mon, 9 Aug 2010 11:24:25 -0400 Subject: [erlang-questions] hello..b In-Reply-To: References: Message-ID: Perhaps the spam bot was written in Erlang... ;-) On 9 August 2010 11:04, Dawid Figiel wrote: > How is this great news related to Erlang ?;> > > > > On Mon, Aug 9, 2010 at 4:47 PM, Aidan Kane wrote: > > > 0Hello: > > Last week, I have ordered china product Apple iPad 64GB > > this w e bsite:dhsellso.com > > I've received the item today > > It's amazing! The item is original, brand new and has high quality, > > but it's much cheap I'm pleased to share this good news with you! > > I believe you will find what you want there and have an good experience > > on shopping from them > > Regards! > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > -- > Dawid > From g9414002.pccu.edu.tw@REDACTED Mon Aug 9 17:30:59 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Mon, 9 Aug 2010 23:30:59 +0800 Subject: [erlang-questions] hello..b In-Reply-To: References: Message-ID: May Erlang work on iPhone/iPad, one day. On Mon, Aug 9, 2010 at 11:24 PM, Marcel Meyer wrote: > Perhaps the spam bot was written in Erlang... ;-) > > On 9 August 2010 11:04, Dawid Figiel wrote: > > > How is this great news related to Erlang ?;> > > > > > > > > On Mon, Aug 9, 2010 at 4:47 PM, Aidan Kane wrote: > > > > > 0Hello: > > > Last week, I have ordered china product Apple iPad 64GB > > > this w e bsite:dhsellso.com > > > I've received the item today > > > It's amazing! The item is original, brand new and has high quality, > > > but it's much cheap I'm pleased to share this good news with you! > > > I believe you will find what you want there and have an good experience > > > on shopping from them > > > Regards! > > > > > > ________________________________________________________________ > > > erlang-questions (at) erlang.org mailing list. > > > See http://www.erlang.org/faq.html > > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > > > > > > -- > > Dawid > > > From andrew@REDACTED Mon Aug 9 17:42:39 2010 From: andrew@REDACTED (Andrew Thompson) Date: Mon, 9 Aug 2010 11:42:39 -0400 Subject: [erlang-questions] Re: gen_smtp question In-Reply-To: References: Message-ID: <20100809154239.GD20513@hijacked.us> On Mon, Aug 09, 2010 at 12:02:36PM +0100, Robert Raschke wrote: > Yeah, the way I understand it, you can use gen_smtp to implement your own > desired server behaviour. But it doesn't come with a full mail server > itself, just the text based example one. Saying that, it is pretty easy to > modify that to suit your needs. > Yeah, that was the plan. I wanted to let people write their own backends, since that makes it easy to add a SMTP server to whatever you need (which is why I wrote it in the first place). You just handle the various SMTP verbs and return a response, and all the icky SMTP is handled for you. You can also advertise your own ESMTP extensions (or the standard ones I didn't implement) and implement them via the handle_other/3 callback, which gives you the verb, the data and your state. So you could implement ETRN, HELP, DSN, etc. Andrew From anthonym@REDACTED Mon Aug 9 19:50:45 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Mon, 9 Aug 2010 10:50:45 -0700 Subject: [erlang-questions] erlang woes In-Reply-To: References: Message-ID: <20100809175045.GC51936@alumni.caltech.edu> Even the patched inets caused me problems with memory, see here http://erlang.2086793.n4.nabble.com/Possible-leak-in-inets-httpc-td2282076.html#a2282805 I ended up switching to ibrowse, it was very painless and had much better memory usage patterns. -Anthony On Thu, Aug 05, 2010 at 10:09:14AM -0700, Arun Suresh wrote: > Thank you for the suggestion Joseph.. > > But i was aware of this bug.. it was fixed in inets-5.3.2.. my prod system > is currently running with the patched inets... so i dont think that is cause > of the crash.. > > -Arun > > On Thu, Aug 5, 2010 at 7:21 AM, Joseph Wayne Norton wrote: > > > > > Please check this patch for R13B04 - I believe it will fix your memory > > leak. > > > > > > http://hibari.git.sourceforge.net/git/gitweb.cgi?p=hibari/patches;a=blob;f=otp_src_R13B04-httpc-memoryleak.patch;h=14eb7a603dc7f0f49f6685260dd0d0bebddc2f4c;hb=HEAD > > > > This patch was posted previously to one of the erlang.org mailing lists. > > > > I don't know your application's requirements but I would recommend most of > > these patches for a vanilla R13B04 erlang system: > > > > http://hibari.git.sourceforge.net/git/gitweb.cgi?p=hibari/patches;a=tree > > > > Instructions for applying these patches can found here: > > > > > > http://hibari.sourceforge.net/webpage-README-ERL.html#_erlang_otp_to_download > > > > regards, > > > > > > > > On Thu, 05 Aug 2010 22:55:27 +0900, Evans, Matthew > > wrote: > > > > Are you using pg2 in a distributed system? There is a known (should be > >> fixed) bug where pg2 was eating memory like crazy. > >> > >> -----Original Message----- > >> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > >> Behalf Of Arun Suresh > >> Sent: Thursday, August 05, 2010 1:37 AM > >> To: erlang-questions@REDACTED > >> Subject: [erlang-questions] erlang woes > >> > >> Hello folks.. > >> > >> Ive been using erlang for a while now.. and ive got a production system up > >> and running from scratch.. but there are some really annoying aspects of > >> the > >> platform.. the most fundamental of which is the fact that when a node > >> crashes it is very hard to figure out exactly why.. Almost ALL the time > >> what > >> i see in the crash dump is something akin to : > >> > >> =erl_crash_dump:0.1 > >> Wed Aug 4 21:50:01 2010 > >> Slogan: eheap_alloc: Cannot allocate 1140328500 bytes of memory (of type > >> "heap"). > >> System version: Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] > >> [async-threads:0] [hipe] [kernel-poll:false] > >> Compiled: Tue May 11 12:37:38 2010 > >> Taints: > >> > >> > >> at which point I start to comb the sasl logs... and 9 out of 10 times... > >> it > >> is because some critical process has died and the supervisor is busy > >> restarting it.. for example, the other day.. my node crashed and from the > >> sasl logs.. i see that the http manager for a profile I had created had > >> crashed like so : > >> > >> =CRASH REPORT==== 4-Aug-2010::21:47:09 === > >> crasher: > >> initial call: httpc_manager:init/1 > >> pid: <0.185.0> > >> registered_name: httpc_manager_store > >> exception exit: {{case_clause, > >> [{handler_info,#Ref<0.0.17.61372>,<0.17225.36>, > >> undefined,<0.15665.36>,initiating}]}, > >> [{httpc_manager,handle_connect_and_send,5}, > >> {httpc_manager,handle_info,2}, > >> {gen_server,handle_msg,5}, > >> {proc_lib,init_p_do_apply,3}]} > >> in function gen_server:terminate/6 > >> ancestors: [httpc_profile_sup,httpc_sup,inets_sup,<0.46.0>] > >> messages: [{'EXIT',<0.16755.36>,normal}, > >> {connect_and_send,<0.16752.36>,#Ref<0.0.17.61366>, > >> > >> > >> and subsequent messages were related to the supervisor trying to restart > >> the > >> profile manager... and failing.. > >> > >> Now my point is... why did the node have to crash.. just because the > >> manager > >> has to be restarted ? > >> and why does the crash.dump always keep telling me im out of memory.. > >> > >> The problem is.. I thought erlang was built to be fault tolerant.. the > >> choice of me using erlang had a LOT to do with doing away with the having > >> to > >> code defensively.. "let it crash" and all that .. just make sure u have a > >> supervisor that restarts ur process and everything will just work fine... > >> but my experience is that most of the time.. simple process restarts bring > >> the whole node crashing down... > >> > >> Would deeply appreciate it someone could tell me if there is something > >> fundamentally wrong with the way im doing things.. or if anyones been in > >> my > >> situation and have had some enlightenments. > >> > >> thanks in advance > >> -Arun > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > >> > > > > -- > > norton@REDACTED > > -- ------------------------------------------------------------------------ Anthony Molinaro From anthonym@REDACTED Mon Aug 9 20:08:08 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Mon, 9 Aug 2010 11:08:08 -0700 Subject: Using ETS for large amounts of data? Message-ID: <20100809180808.GD51936@alumni.caltech.edu> Hi, I've got some data in a file of the form start_integer|end_integer|data_field1|data_field2|..|data_fieldN Where N is 12. Most fields are actually smallish integers. The total number of entries in the file is 9964217 lines. and the total file size is 752265457. I want to load these into an erlang process and essentially do lookups of the form given an integer M if M >= start_integer && M <= end_integer then return data I was planning on using ets with entries of the form #data { key = #range { begin = integer(), end = integer() }, data_field1 = Data1, ... data_fieldN = DataN } Then using a match_spec to get the appropriate entry. However, I'm running into problems. I can't seem to actually load up that many entries into an ets table. First I was trying an ordered_set because it seemed like that might be faster, however the load seems to be okay for about the first 2 million entries, then gets very slow and eventually crashes. So I tried just set, but that seems to not even make it that far before it chews up a lot of memory and then causes my machine to swap. So is ets up to this task at all? and if not any suggestions? Thanks, -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From rvirding@REDACTED Mon Aug 9 20:21:47 2010 From: rvirding@REDACTED (Robert Virding) Date: Mon, 9 Aug 2010 20:21:47 +0200 Subject: [erlang-questions] cost of passing a gen_server state between functions In-Reply-To: <996091.97816.qm@web112610.mail.gq1.yahoo.com> References: <198711.41599.qm@web112616.mail.gq1.yahoo.com> <996091.97816.qm@web112610.mail.gq1.yahoo.com> Message-ID: On 9 August 2010 03:10, Pablo Platt wrote: > Thank you Hynek and Robert. > > The reason I asked this question is because I have one gen_server that > handles several sessions for the same user. > I have several session types for different types of connections: socket, > http... > When the gen_server receive a message it passes it to an handler?along with > the gen_server's state. > Each session type?defines a different handler (module). > The?handler process the message and reply with a new state. > > Most of the operations on the state are setting a new value in a record, > reading values and adding new items to lists. > So from your explanation I understand that there is no copy. The state is > passed by reference > and that as long as I append new values to the head of lists in the state > I'm safe. The new state is also returned by reference. One thing you may consider is that if the state contains groups of values for each session type then create separate records for them: -record(state, {socket,other_type,...}). -record(socket_state, {...}). -record(other_type_state, {...}). ... You can then in the can in the top dispatcher just extra/reinsert just that part of the state. It will make it easier to keep each session type module independent. This will, of course, not work if there are state values which are shared between the different session types. > % inside the socket session module: > handle_message(State, Message) -> > ??? do_something_with_message(Message), > ??? Log = State#state.log, > ??? NewLog = [Message | Log], > ??? NewState = State#state{log=NewLog}, > ??? NewState. You don't really need to save all the intermediate values in variables, it could just as well have been written: handle_message(State, Message) -> do_something_with_message(Message), NewLog = [Message | State#state.log], State#state{log=NewLog}. . The resulting code will more or less be the same so here again it is a matter of taste and which feels best. Robert From rvirding@REDACTED Mon Aug 9 20:25:32 2010 From: rvirding@REDACTED (Robert Virding) Date: Mon, 9 Aug 2010 20:25:32 +0200 Subject: [erlang-questions] Custom module attributes In-Reply-To: References: Message-ID: Because. Originally module attributes were more module oriented and we felt it was better and clearer to put them in a block at the beginning of the module. Robert From jesper.louis.andersen@REDACTED Mon Aug 9 20:44:25 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 9 Aug 2010 20:44:25 +0200 Subject: [erlang-questions] Using ETS for large amounts of data? In-Reply-To: <20100809180808.GD51936@alumni.caltech.edu> References: <20100809180808.GD51936@alumni.caltech.edu> Message-ID: On Mon, Aug 9, 2010 at 8:08 PM, Anthony Molinaro wrote: > Hi, > > ?I've got some data in a file of the form > > start_integer|end_integer|data_field1|data_field2|..|data_fieldN > > Where N is 12. ?Most fields are actually smallish integers. > The total number of entries in the file is 9964217 lines. > and the total file size is 752265457. Some math to get you started: Assume the 12 fields are smallish integers. Integers are *at least* 8 bytes in size in our guessing game: 1> Lines = 9964217. 9964217 ... 4> Lines * 14 * 8 / (1024 * 1024). 1064.293197631836 So a reasonable lower bound on your data is 1Gb. Now, that is assuming we can pack the integers optimally. A more precise bet on the bound can be had: 1> byte_size(term_to_binary({data, 1, 2, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12})). 38 So: 4> Lines * 38 / (1024 * 1024). 361.09947776794434 Or perhaps even: 5> byte_size(term_to_binary({data, 1000000, 2000000, 1000000, 2000000, 300, 4000, 5000, 600, 7000, 80000, 90000000, 10000000, 11000000, 12000000})). 80 Yielding: 6> Lines * 80 / (1024 * 1024). 760.2094268798828 One of the VM-people will definitely be able to shed more light on what the implementation does. The real killer is if your data is much larger than this and nothing is done to compress stored terms in ETS. -- J. From vladdu55@REDACTED Mon Aug 9 20:49:03 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 9 Aug 2010 20:49:03 +0200 Subject: [erlang-questions] Custom module attributes In-Reply-To: References: Message-ID: On Mon, Aug 9, 2010 at 20:25, Robert Virding wrote: > Because. > > Originally module attributes were more module oriented and we felt it > was better and clearer to put them in a block at the beginning of the > module. Thanks. Yes, but now -specs break this rule, so maybe it wouldn't be a big deal to set all of them free. regards, Vlad From anthonym@REDACTED Mon Aug 9 21:15:33 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Mon, 9 Aug 2010 12:15:33 -0700 Subject: [erlang-questions] Using ETS for large amounts of data? In-Reply-To: References: <20100809180808.GD51936@alumni.caltech.edu> Message-ID: <20100809191533.GE51936@alumni.caltech.edu> On Mon, Aug 09, 2010 at 08:44:25PM +0200, Jesper Louis Andersen wrote: > On Mon, Aug 9, 2010 at 8:08 PM, Anthony Molinaro > wrote: > > Hi, > > > > ?I've got some data in a file of the form > > > > start_integer|end_integer|data_field1|data_field2|..|data_fieldN > > > > Where N is 12. ?Most fields are actually smallish integers. > > The total number of entries in the file is 9964217 lines. > > and the total file size is 752265457. > > Some math to get you started: > > Assume the 12 fields are smallish integers. Integers are *at least* 8 > bytes in size in our guessing game: > > 1> Lines = 9964217. > 9964217 > ... > 4> Lines * 14 * 8 / (1024 * 1024). > 1064.293197631836 > > So a reasonable lower bound on your data is 1Gb. Now, that is assuming > we can pack the integers optimally. Actually, while they are small integers, in the file they are ascii, so 750megs of ascii numbers. I'm actually attempting to store them as ascii currently (just because I was curious if I could store that much data in ETS, and figured binary would be smaller anyway, so this gives a better upper bound and at least a few of the fields are strings). But anyway, I'm fine with 1 or even 2 Gb of space. > A more precise bet on the bound can be had: > > 1> byte_size(term_to_binary({data, 1, 2, 1, 2, 3, 4, 5, 6, 7, 8, 9, > 10, 11, 12})). > 38 > > So: > > 4> Lines * 38 / (1024 * 1024). > 361.09947776794434 > > Or perhaps even: > > 5> byte_size(term_to_binary({data, 1000000, 2000000, 1000000, 2000000, > 300, 4000, 5000, 600, 7000, 80000, 90000000, 10000000, 11000000, > 12000000})). > 80 > > Yielding: > > 6> Lines * 80 / (1024 * 1024). > 760.2094268798828 > > > One of the VM-people will definitely be able to shed more light on > what the implementation does. > > The real killer is if your data is much larger than this and nothing > is done to compress stored terms in ETS. I actually already do some compression by taking common values and having some lookup tables (well tuples with all the end values, and use element to index the right value). That's what got me to integer values. I guess the real question I had is is this amount of data even feasible in ETS. Has anyone out their tried to store 10 million values in ETS? What other solutions have others tried? Thanks for the help, -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From joaohf@REDACTED Mon Aug 9 21:35:39 2010 From: joaohf@REDACTED (=?UTF-8?Q?Jo=C3=A3o_Henrique_Freitas?=) Date: Mon, 9 Aug 2010 16:35:39 -0300 Subject: writing a linked in driver by hand or using a tool. Message-ID: Hi, I need to interface with posix queue and system V. To achieve this I need to code a linked in driver (I choice this option). My doubt is: What is the best choice: use a tool (like edtk or dryerl) or code by hand? What is the modern way to do it? Thanks. -- ----------------------------------------------------------- Jo?o Henrique Freitas - joaohf_at_gmail.com Campinas-SP-Brasil BSD051283 LPI 1 http://www.joaohfreitas.eti.br From joaohf@REDACTED Mon Aug 9 21:35:39 2010 From: joaohf@REDACTED (=?UTF-8?Q?Jo=C3=A3o_Henrique_Freitas?=) Date: Mon, 9 Aug 2010 16:35:39 -0300 Subject: writing a linked in driver by hand or using a tool. Message-ID: Hi, I need to interface with posix queue and system V. To achieve this I need to code a linked in driver (I choice this option). My doubt is: What is the best choice: use a tool (like edtk or dryerl) or code by hand? What is the modern way to do it? Thanks. -- ----------------------------------------------------------- Jo?o Henrique Freitas - joaohf_at_gmail.com Campinas-SP-Brasil BSD051283 LPI 1 http://www.joaohfreitas.eti.br From rapsey@REDACTED Mon Aug 9 21:56:21 2010 From: rapsey@REDACTED (Rapsey) Date: Mon, 9 Aug 2010 21:56:21 +0200 Subject: [erlang-questions] writing a linked in driver by hand or using a tool. In-Reply-To: References: Message-ID: I would say the modern way is to use a NIF. Especially in the upcoming R14, it can replace linked-in drivers in most situations. It can be coupled much more tightly with your erlang code and it allows you to really use C just for what is necessary. The resource data type with garbage collection is a really kick-ass feature. I'm in the process of rewriting my pretty complex linked-in driver (written plain without edtk/dryerl) with a NIF, simply because it allows me to put more of the functionality on the erlang side and it makes the entire codebase much easier to handle and work with. Sergej 2010/8/9 Jo?o Henrique Freitas > Hi, > > I need to interface with posix queue and system V. To achieve this I > need to code a linked in driver (I choice this option). My doubt is: > What is the best choice: use a tool (like edtk or dryerl) or code by > hand? What is the modern way to do it? > > Thanks. > > -- > ----------------------------------------------------------- > Jo?o Henrique Freitas - joaohf_at_gmail.com > Campinas-SP-Brasil > BSD051283 > LPI 1 > http://www.joaohfreitas.eti.br > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From max.lapshin@REDACTED Mon Aug 9 22:12:29 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 10 Aug 2010 00:12:29 +0400 Subject: [erlang-questions] writing a linked in driver by hand or using a tool. In-Reply-To: References: Message-ID: The only thing for nif, perhaps, is custom IO handlers, bound to real sockets. But I hardly can imagine, when I've needed it. From reachsaurabhnarula@REDACTED Mon Aug 9 22:15:08 2010 From: reachsaurabhnarula@REDACTED (Saurabh Narula) Date: Tue, 10 Aug 2010 01:45:08 +0530 Subject: recover from unexpected errors from inside NIF Message-ID: Hello Everyone, I am writing a gen server that makes use of certain native implemented functions (NIFs) that I have written in C, can anybody tell me how to recover from unexpected errors that might happen in my C program where my C program tends to crash at the OS level? Would appreciate any kind help! Saurabh From tony@REDACTED Mon Aug 9 21:20:09 2010 From: tony@REDACTED (Tony Rogvall) Date: Mon, 9 Aug 2010 21:20:09 +0200 Subject: [erlang-questions] hello..b In-Reply-To: References: Message-ID: <38BF52EE-BB77-4236-AFB2-D62AE387AD24@rogvall.se> Nope. /Tony On 9 aug 2010, at 17.30, ??? (Yau-Hsien Huang) wrote: > May Erlang work on iPhone/iPad, one day. > > On Mon, Aug 9, 2010 at 11:24 PM, Marcel Meyer wrote: > >> Perhaps the spam bot was written in Erlang... ;-) >> >> On 9 August 2010 11:04, Dawid Figiel wrote: >> >>> How is this great news related to Erlang ?;> >>> >>> >>> >>> On Mon, Aug 9, 2010 at 4:47 PM, Aidan Kane wrote: >>> >>>> 0Hello: >>>> Last week, I have ordered china product Apple iPad 64GB >>>> this w e bsite:dhsellso.com >>>> I've received the item today >>>> It's amazing! The item is original, brand new and has high quality, >>>> but it's much cheap I'm pleased to share this good news with you! >>>> I believe you will find what you want there and have an good experience >>>> on shopping from them >>>> Regards! >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> >>> >>> >>> -- >>> Dawid >>> >> From ulf.wiger@REDACTED Mon Aug 9 22:26:01 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 09 Aug 2010 22:26:01 +0200 Subject: [erlang-questions] Using ETS for large amounts of data? In-Reply-To: <20100809191533.GE51936@alumni.caltech.edu> References: <20100809180808.GD51936@alumni.caltech.edu> <20100809191533.GE51936@alumni.caltech.edu> Message-ID: <4C606459.1040005@erlang-solutions.com> The better function for checking the size of a data object is erts_debug:flat_size(Data), although it will give misleading results if Data contains binaries. It returns the size in heap words (4 bytes in 32-bit Erlang and 8 in 64-bit). I have stored 5 million records in an ets table (http://www.erlang.org/pipermail/erlang-questions/2005-November/017728.html) without any problems. I'm not sure what you mean exactly when you say you are storing ascii values. If you happen to be storing strings, you should be aware that each smallint in an Erlang list occupies two heap words (8 or 16 bytes depending on word length). Anyway, the above function will help you figure out how large your objects are. You might also want to look at how you move the data from the file to the ets table. If you pull one line at a time, you should be ok, but if you pull in all the data first, then start moving it to ets, it's probably not ets that's giving you grief. :) BR, Ulf W Anthony Molinaro wrote: > On Mon, Aug 09, 2010 at 08:44:25PM +0200, Jesper Louis Andersen wrote: >> On Mon, Aug 9, 2010 at 8:08 PM, Anthony Molinaro >> wrote: >>> Hi, >>> >>> I've got some data in a file of the form >>> >>> start_integer|end_integer|data_field1|data_field2|..|data_fieldN >>> >>> Where N is 12. Most fields are actually smallish integers. >>> The total number of entries in the file is 9964217 lines. >>> and the total file size is 752265457. >> Some math to get you started: >> >> Assume the 12 fields are smallish integers. Integers are *at least* 8 >> bytes in size in our guessing game: >> >> 1> Lines = 9964217. >> 9964217 >> ... >> 4> Lines * 14 * 8 / (1024 * 1024). >> 1064.293197631836 >> >> So a reasonable lower bound on your data is 1Gb. Now, that is assuming >> we can pack the integers optimally. > > Actually, while they are small integers, in the file they are ascii, so > 750megs of ascii numbers. I'm actually attempting to store them as ascii > currently (just because I was curious if I could store that much data in > ETS, and figured binary would be smaller anyway, so this gives a better > upper bound and at least a few of the fields are strings). But anyway, > I'm fine with 1 or even 2 Gb of space. > >> A more precise bet on the bound can be had: >> >> 1> byte_size(term_to_binary({data, 1, 2, 1, 2, 3, 4, 5, 6, 7, 8, 9, >> 10, 11, 12})). >> 38 >> >> So: >> >> 4> Lines * 38 / (1024 * 1024). >> 361.09947776794434 >> >> Or perhaps even: >> >> 5> byte_size(term_to_binary({data, 1000000, 2000000, 1000000, 2000000, >> 300, 4000, 5000, 600, 7000, 80000, 90000000, 10000000, 11000000, >> 12000000})). >> 80 >> >> Yielding: >> >> 6> Lines * 80 / (1024 * 1024). >> 760.2094268798828 >> >> >> One of the VM-people will definitely be able to shed more light on >> what the implementation does. >> >> The real killer is if your data is much larger than this and nothing >> is done to compress stored terms in ETS. > > I actually already do some compression by taking common values and having > some lookup tables (well tuples with all the end values, and use element > to index the right value). That's what got me to integer values. I guess > the real question I had is is this amount of data even feasible in ETS. > Has anyone out their tried to store 10 million values in ETS? What other > solutions have others tried? > > Thanks for the help, > > -Anthony > > -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From paul-trapexit@REDACTED Mon Aug 9 23:22:15 2010 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Mon, 9 Aug 2010 14:22:15 -0700 (PDT) Subject: [erlang-questions] Using ETS for large amounts of data? In-Reply-To: <20100809180808.GD51936@alumni.caltech.edu> References: <20100809180808.GD51936@alumni.caltech.edu> Message-ID: On Mon, 9 Aug 2010, Anthony Molinaro wrote: > Hi, > > I've got some data in a file of the form > > start_integer|end_integer|data_field1|data_field2|..|data_fieldN > > Where N is 12. Most fields are actually smallish integers. > The total number of entries in the file is 9964217 lines. > and the total file size is 752265457. > > I want to load these into an erlang process and essentially do lookups > of the form > > given an integer M > > if M >= start_integer && M <= end_integer then > return data ... > So is ets up to this task at all? and if not any suggestions? Here's my initial thoughts: 1) use an interval tree to encode (start, end) -> index 1a) http://en.wikipedia.org/wiki/Interval_tree 1b) if your intervals do not overlap you can use ets ordered set for this part, but use the beginning of the interval for the key, and then use ets:prev/2 to find the entry 2) put the rest of the data fields in a huge flat binary and look them up by reading the index from the interval tree ... or hey just seek around a disk file, and let the OS cache keep the hot spots warm. -- p From pablo.platt@REDACTED Mon Aug 9 22:37:19 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 9 Aug 2010 13:37:19 -0700 (PDT) Subject: [erlang-questions] cost of passing a gen_server state between functions In-Reply-To: References: <198711.41599.qm@web112616.mail.gq1.yahoo.com> <996091.97816.qm@web112610.mail.gq1.yahoo.com> Message-ID: <522632.33137.qm@web112608.mail.gq1.yahoo.com> Shorter functions is always more readable so I like your version better. Thank you for the clarifications and coding style suggestions. ________________________________ From: Robert Virding To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Mon, August 9, 2010 9:21:47 PM Subject: Re: [erlang-questions] cost of passing a gen_server state between functions On 9 August 2010 03:10, Pablo Platt wrote: > Thank you Hynek and Robert. > > The reason I asked this question is because I have one gen_server that > handles several sessions for the same user. > I have several session types for different types of connections: socket, > http... > When the gen_server receive a message it passes it to an handler?along with > the gen_server's state. > Each session type?defines a different handler (module). > The?handler process the message and reply with a new state. > > Most of the operations on the state are setting a new value in a record, > reading values and adding new items to lists. > So from your explanation I understand that there is no copy. The state is > passed by reference > and that as long as I append new values to the head of lists in the state > I'm safe. The new state is also returned by reference. One thing you may consider is that if the state contains groups of values for each session type then create separate records for them: -record(state, {socket,other_type,...}). -record(socket_state, {...}). -record(other_type_state, {...}). ... You can then in the can in the top dispatcher just extra/reinsert just that part of the state. It will make it easier to keep each session type module independent. This will, of course, not work if there are state values which are shared between the different session types. > % inside the socket session module: > handle_message(State, Message) -> > ??? do_something_with_message(Message), > ??? Log = State#state.log, > ??? NewLog = [Message | Log], > ??? NewState = State#state{log=NewLog}, > ??? NewState. You don't really need to save all the intermediate values in variables, it could just as well have been written: handle_message(State, Message) -> ? ? do_something_with_message(Message), ? ? NewLog = [Message | State#state.log], ? ? State#state{log=NewLog}. . The resulting code will more or less be the same so here again it is a matter of taste and which feels best. Robert From anthonym@REDACTED Tue Aug 10 01:13:25 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Mon, 9 Aug 2010 16:13:25 -0700 Subject: [erlang-questions] Using ETS for large amounts of data? In-Reply-To: References: <20100809180808.GD51936@alumni.caltech.edu> Message-ID: <20100809231325.GF51936@alumni.caltech.edu> On Mon, Aug 09, 2010 at 02:22:15PM -0700, Paul Mineiro wrote: > On Mon, 9 Aug 2010, Anthony Molinaro wrote: > > > Hi, > > > > I've got some data in a file of the form > > > > start_integer|end_integer|data_field1|data_field2|..|data_fieldN > > > > Where N is 12. Most fields are actually smallish integers. > > The total number of entries in the file is 9964217 lines. > > and the total file size is 752265457. > > > > I want to load these into an erlang process and essentially do lookups > > of the form > > > > given an integer M > > > > if M >= start_integer && M <= end_integer then > > return data > ... > > So is ets up to this task at all? and if not any suggestions? > > Here's my initial thoughts: > > 1) use an interval tree to encode (start, end) -> index > 1a) http://en.wikipedia.org/wiki/Interval_tree > 1b) if your intervals do not overlap you can use ets ordered set for > this part, but use the beginning of the interval for the key, and then use > ets:prev/2 to find the entry > 2) put the rest of the data fields in a huge flat binary and look them up > by reading the index from the interval tree ... or hey just seek around a > disk file, and let the OS cache keep the hot spots warm. Since my intervals are closed and non intersecting, it looks like storing just the interval start and an index is a win. I was able to create an ets file with the starts and random indexes, then save it with ets:tab2file, then load it quickly and do ets:prev calls. So now for the data file. It looks like my structure is actually 4 uint8 integers 5 uint16 integers 1 uint32 integer 2 floats (4 byte precision should be fine) So fixed length record size of 26 bytes for a total of 259069642 bytes. Since I'm generating these files from perl, I'll use pack on the encode side, then the bit syntax to decode, but I'm wondering what the most efficient way of pulling the individual record out of the large binary are (I plan to file:read_file/1 to get the binary)? It is using bit syntax to pull out 26 bytes? BigBinary = <<_:Index, Record:26>> Record = <> or is it better to using something like split_binary/2? {Before, At} -> split_binary (BigBinary, Index), At = <> I was going to try some timing tests, but figured I'd see if anyone has any idea (I'm using R13B04 if that matters). Thanks for the help, -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From asit.cse.psu@REDACTED Tue Aug 10 03:21:45 2010 From: asit.cse.psu@REDACTED (Asit Mishra) Date: Mon, 9 Aug 2010 18:21:45 -0700 Subject: where to modify sockets code Message-ID: Hi all When communicating between two erlang nodes using the ! construct, the underlying system takes care of where to send the message i.e. if two nodes are on the same physical machine (say a@REDACTED and b@REDACTED), the system does the address translation and sends message to the appropriate destination. Similarly, when the nodes are on physically different machines (say a@REDACTED b@REDACTED), the system again identifies the proper destination and sends it a message. I am interested in figuring out where this code is and how it is implemented. Is this implementation in C or Erlang? I am interested in hacking this piece of code and re-route packets i.e. even if the user specifies a remote destination (to nodeB) I want to route the message packet to a local erlang node (to nodeA) on the same machines -- just a research project of mine at Intel here to leverage multicores. The reason I dont want to do this in erlang is that I want the same piece of code to run transparently on both the unmodified and modified Erlang VM. I know there is a load-balancer in Erlang library but I dont want to use it as well. Any other suggestions on how I should proceed is appreciated? Can someone tell me where to hack the address translation code? How is message passing actually implemented in a shared memory system? Thanks a bunch Asit From gleber.p@REDACTED Tue Aug 10 11:29:09 2010 From: gleber.p@REDACTED (Gleb Peregud) Date: Tue, 10 Aug 2010 11:29:09 +0200 Subject: [erlang-questions] recover from unexpected errors from inside NIF In-Reply-To: References: Message-ID: AFAIk there is no good way to recover from unexpected errors inside NIFs. You have to make sure to handle manually all extreme cases inside NIF's code, because NIFs are run directly in scheduler thread of BEAM, so any errors can crash whole BEAM. If you have some unstable library and there is no way to make it stable, I'd suggest using it through external program connected to Erlang via port mechanism. On Mon, Aug 9, 2010 at 22:15, Saurabh Narula wrote: > Hello Everyone, > > I am writing a gen server that makes use of certain native implemented > functions (NIFs) that I have written in C, can anybody tell me how to > recover from unexpected errors that might happen in my C program where my C > program tends to crash at the OS level? > > Would appreciate any kind help! > > Saurabh > From hynek@REDACTED Tue Aug 10 11:50:13 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Tue, 10 Aug 2010 11:50:13 +0200 Subject: [erlang-questions] Using ETS for large amounts of data? In-Reply-To: <20100809231325.GF51936@alumni.caltech.edu> References: <20100809180808.GD51936@alumni.caltech.edu> <20100809231325.GF51936@alumni.caltech.edu> Message-ID: Absolutely fastest (including startup) should be nif with mmaped file. But if you want stay in erlang you can use file:read_file/1 if occupied memory is not problem and than binary:at/2 if you can use R14. Otherwise I would use <<_:Index, V1:1, V2:1, V3:1,..., _/binary>> = Bin. If occupied memory is problem I would use file:open/2 with [read, raw, binary, read_ahead] and file:pread/3 and may be tailor mine own LRU record or block cache. Anyway, if you are curious, tailor your own benchmark and measure. It is worth of thousands of advices. You should start with simplest solution and than measure. If not fast enough, then try another approach and then measure and compare and again and again unless you are satisfied. You will learn a lot during this process. On Tue, Aug 10, 2010 at 1:13 AM, Anthony Molinaro wrote: > > On Mon, Aug 09, 2010 at 02:22:15PM -0700, Paul Mineiro wrote: >> On Mon, 9 Aug 2010, Anthony Molinaro wrote: >> >> > Hi, >> > >> > ? I've got some data in a file of the form >> > >> > start_integer|end_integer|data_field1|data_field2|..|data_fieldN >> > >> > Where N is 12. ?Most fields are actually smallish integers. >> > The total number of entries in the file is 9964217 lines. >> > and the total file size is 752265457. >> > >> > I want to load these into an erlang process and essentially do lookups >> > of the form >> > >> > ? given an integer M >> > >> > ? if M >= start_integer && M <= end_integer then >> > ? ? return data >> ... >> > So is ets up to this task at all? and if not any suggestions? >> >> Here's my initial thoughts: >> >> 1) use an interval tree to encode (start, end) -> index >> ? ?1a) http://en.wikipedia.org/wiki/Interval_tree >> ? ?1b) if your intervals do not overlap you can use ets ordered set for >> this part, but use the beginning of the interval for the key, and then use >> ets:prev/2 to find the entry >> 2) put the rest of the data fields in a huge flat binary and look them up >> by reading the index from the interval tree ... or hey just seek around a >> disk file, and let the OS cache keep the hot spots warm. > > Since my intervals are closed and non intersecting, it looks like storing > just the interval start and an index is a win. ? I was able to create an > ets file with the starts and random indexes, then save it with ets:tab2file, > then load it quickly and do ets:prev calls. > > So now for the data file. ?It looks like my structure is actually > > 4 uint8 integers > 5 uint16 integers > 1 uint32 integer > 2 floats (4 byte precision should be fine) > > So fixed length record size of 26 bytes for a total of 259069642 bytes. > > Since I'm generating these files from perl, I'll use pack on the encode > side, then the bit syntax to decode, but I'm wondering what the most > efficient way of pulling the individual record out of the large binary > are (I plan to file:read_file/1 to get the binary)? > > It is using bit syntax to pull out 26 bytes? > > BigBinary = <<_:Index, Record:26>> > Record = <> > > or is it better to using something like split_binary/2? > > {Before, At} -> split_binary (BigBinary, Index), > At = <> > > I was going to try some timing tests, but figured I'd see if anyone has any > idea (I'm using R13B04 if that matters). > > Thanks for the help, > > -Anthony > > -- > ------------------------------------------------------------------------ > Anthony Molinaro ? ? ? ? ? ? ? ? ? ? ? ? ? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From joaohf@REDACTED Tue Aug 10 13:35:40 2010 From: joaohf@REDACTED (=?UTF-8?Q?Jo=C3=A3o_Henrique_Freitas?=) Date: Tue, 10 Aug 2010 08:35:40 -0300 Subject: [erlang-questions] writing a linked in driver by hand or using a tool. In-Reply-To: References: Message-ID: Hi, I thought in use NIF but some functions like msgrcv and mq_receive (by default) blocks until a message becomes available and consequently will block the VM. With linked in driver, I can to create a specific thread to unqueue the data and sends it to erlang process. But, my question is about using edtk or dryerl to write a driver. I am searching arguments to convince me to use them. Thanks. -- ----------------------------------------------------------- Jo?o Henrique Freitas - joaohf_at_gmail.com Campinas-SP-Brasil BSD051283 LPI 1 http://www.joaohfreitas.eti.br From rapsey@REDACTED Tue Aug 10 14:44:18 2010 From: rapsey@REDACTED (Rapsey) Date: Tue, 10 Aug 2010 14:44:18 +0200 Subject: [erlang-questions] writing a linked in driver by hand or using a tool. In-Reply-To: References: Message-ID: R14 added support for sending messages from a NIF thread to an erlang process. Sergej 2010/8/10 Jo?o Henrique Freitas > Hi, > > I thought in use NIF but some functions like msgrcv and mq_receive > (by default) blocks until a message becomes available and consequently > will block the VM. > > With linked in driver, I can to create a specific thread to unqueue > the data and sends it to erlang process. > > But, my question is about using edtk or dryerl to write a driver. I am > searching arguments to convince me to use them. > > Thanks. > > > -- > ----------------------------------------------------------- > Jo?o Henrique Freitas - joaohf_at_gmail.com > Campinas-SP-Brasil > BSD051283 > LPI 1 > http://www.joaohfreitas.eti.br > From max.lapshin@REDACTED Tue Aug 10 15:20:27 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 10 Aug 2010 17:20:27 +0400 Subject: [erlang-questions] writing a linked in driver by hand or using a tool. In-Reply-To: References: Message-ID: On Tue, Aug 10, 2010 at 4:44 PM, Rapsey wrote: > R14 added support for sending messages from a NIF thread to an erlang > process. > But what convenient way of sending messages into NIF thread? From daniel.goertzen@REDACTED Tue Aug 10 15:39:43 2010 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Tue, 10 Aug 2010 08:39:43 -0500 Subject: open_port() on FD 0,1 problems Message-ID: I am trying to operate an Erlang Port for stdin and stdout [open_port({fd,0,1}...]. The program below is to receive two integers, multiply them, and write the result. Every other request vanishes without a reply, and I don't know why. Details: - Windows 7 64 bit. - Erlang launched with "-noshell -s erlang_rpc_benchmark" - I have to wait a moment after launching erlang before sending it data, or I definitely get dataloss. - Exactly every other packet gets lost, starting with the first one. - The byte counters on Task Manager show that the request is getting to erl.exe, but not to my code. - I've added debug code to verify that there are no other messages appearing at my receive expression (not currently in code below). - I wrote a Python version of this program, and it worked without issue. Background: I plan to write a Python GUI program with an Erlang child process. The test program below is to assess Python->Erlang messaging performance. I am open to other IPC approaches if there is something better. Thanks, Dan. -module(erlang_rpc_benchmark). -export([start/0, std_server/0]). start() -> spawn(fun std_server/0). std_server() -> Port = open_port( {fd,0,1}, [{packet, 4}, binary ] ), loop(Port). loop(Port) -> receive {Port, { data, <> }} -> Result = A*B, Port ! {self(), {command, <>}}; end, loop(Port). From hynek@REDACTED Tue Aug 10 16:05:53 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Tue, 10 Aug 2010 16:05:53 +0200 Subject: [erlang-questions] open_port() on FD 0,1 problems In-Reply-To: References: Message-ID: I don't know how on Windows but try -noinput parameter. On Tue, Aug 10, 2010 at 3:39 PM, Daniel Goertzen wrote: > I am trying to operate an Erlang Port for stdin and stdout > [open_port({fd,0,1}...]. ?The program below is to receive two integers, > multiply them, and write the result. Every other request vanishes without a > reply, and I don't know why. ?Details: > > - Windows 7 64 bit. > - Erlang launched with "-noshell -s erlang_rpc_benchmark" > - I have to wait a moment after launching erlang before sending it data, or > I definitely get dataloss. > - Exactly every other packet gets lost, starting with the first one. > - The byte counters on Task Manager show that the request is getting to > erl.exe, but not to my code. > - I've added debug code to verify that there are no other messages appearing > at my receive expression (not currently in code below). > - I wrote a Python version of this program, and it worked without issue. > > > Background: ?I plan to write a Python GUI program with an Erlang child > process. ?The test program below is to assess Python->Erlang messaging > performance. ?I am open to other IPC approaches if there is something > better. > > Thanks, > Dan. > > > -module(erlang_rpc_benchmark). > -export([start/0, std_server/0]). > > start() -> > ?spawn(fun std_server/0). > > std_server() -> > ?Port = open_port( {fd,0,1}, [{packet, 4}, binary ] ), > ?loop(Port). > > loop(Port) -> > ?receive > ? ?{Port, { data, <> }} -> > ? ? ?Result = A*B, > ? ? ?Port ! {self(), {command, <>}}; > ?end, > ?loop(Port). > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From paul.joseph.davis@REDACTED Tue Aug 10 18:00:15 2010 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Tue, 10 Aug 2010 12:00:15 -0400 Subject: [erlang-questions] writing a linked in driver by hand or using a tool. In-Reply-To: References: Message-ID: On Tue, Aug 10, 2010 at 9:20 AM, Max Lapshin wrote: > On Tue, Aug 10, 2010 at 4:44 PM, Rapsey wrote: >> R14 added support for sending messages from a NIF thread to an erlang >> process. >> > > But what convenient way of sending messages into NIF thread? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > NIF's allow you to create threads. So you can just create a worker thread when the NIF is loaded. When calling a NIF function from Erlang, you just push the request onto a queue for the thread and let the thread handle it and send the response back to the requesting pid. From max.lapshin@REDACTED Tue Aug 10 18:02:31 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 10 Aug 2010 20:02:31 +0400 Subject: [erlang-questions] writing a linked in driver by hand or using a tool. In-Reply-To: References: Message-ID: > NIF's allow you to create threads. So you can just create a worker > thread when the NIF is loaded. When calling a NIF function from > Erlang, you just push the request onto a queue for the thread and let > the thread handle it and send the response back to the requesting pid. This is the question: which is the best way to add queue for thread? Don't forget, that thread may block when there is nothing to handle and resume, when new task appears. From daniel.goertzen@REDACTED Tue Aug 10 18:44:42 2010 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Tue, 10 Aug 2010 11:44:42 -0500 Subject: [erlang-questions] open_port() on FD 0,1 problems In-Reply-To: References: Message-ID: Thanks! That helps a lot. New issue is that erl.exe is translating incoming 0x0d bytes to 0x0a. Windows line-end conversions, grumble, grumble. Is there a nice way to turn that off? Or am I at a dead-end? Thanks, Dan. On Tue, Aug 10, 2010 at 9:05 AM, Hynek Vychodil wrote: > I don't know how on Windows but try -noinput parameter. > > On Tue, Aug 10, 2010 at 3:39 PM, Daniel Goertzen > wrote: > > I am trying to operate an Erlang Port for stdin and stdout > > [open_port({fd,0,1}...]. The program below is to receive two integers, > > multiply them, and write the result. Every other request vanishes without > a > > reply, and I don't know why. Details: > > > > - Windows 7 64 bit. > > - Erlang launched with "-noshell -s erlang_rpc_benchmark" > > - I have to wait a moment after launching erlang before sending it data, > or > > I definitely get dataloss. > > - Exactly every other packet gets lost, starting with the first one. > > - The byte counters on Task Manager show that the request is getting to > > erl.exe, but not to my code. > > - I've added debug code to verify that there are no other messages > appearing > > at my receive expression (not currently in code below). > > - I wrote a Python version of this program, and it worked without issue. > > > > > > Background: I plan to write a Python GUI program with an Erlang child > > process. The test program below is to assess Python->Erlang messaging > > performance. I am open to other IPC approaches if there is something > > better. > > > > Thanks, > > Dan. > > > > > > -module(erlang_rpc_benchmark). > > -export([start/0, std_server/0]). > > > > start() -> > > spawn(fun std_server/0). > > > > std_server() -> > > Port = open_port( {fd,0,1}, [{packet, 4}, binary ] ), > > loop(Port). > > > > loop(Port) -> > > receive > > {Port, { data, <> }} -> > > Result = A*B, > > Port ! {self(), {command, <>}}; > > end, > > loop(Port). > > > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try GoodData now for free: www.gooddata.com > -- Daniel Goertzen ----------------- dang@REDACTED (work) daniel.goertzen@REDACTED (home) ----------------- 1 204 272 6149 (home/office) 1 204 470 8360 (mobile) ----------------- From rapsey@REDACTED Tue Aug 10 19:18:33 2010 From: rapsey@REDACTED (Rapsey) Date: Tue, 10 Aug 2010 19:18:33 +0200 Subject: [erlang-questions] writing a linked in driver by hand or using a tool. In-Reply-To: References: Message-ID: On Tue, Aug 10, 2010 at 6:02 PM, Max Lapshin wrote: > > NIF's allow you to create threads. So you can just create a worker > > thread when the NIF is loaded. When calling a NIF function from > > Erlang, you just push the request onto a queue for the thread and let > > the thread handle it and send the response back to the requesting pid. > > This is the question: which is the best way to add queue for thread? > Don't forget, that thread may block when there is nothing to handle > and resume, when new task appears. > Since my worker thread is a kqueue event thread, I use a pipe to communicate with it. Sergej From mikpe@REDACTED Tue Aug 10 20:51:55 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Tue, 10 Aug 2010 20:51:55 +0200 Subject: [erlang-questions] Hipe and memory alignment (Yikes! erts_alloc() returned misaligned address) In-Reply-To: References: <33495D81-33AC-4AC8-9394-6377C01760CD@kallisys.net> <19545.55651.737243.453894@pilspetsen.it.uu.se> <8E2689F9-91F4-43AB-9D6B-7F29B9584AB0@kallisys.net> Message-ID: <19553.40907.521909.895014@pilspetsen.it.uu.se> Tony Finch writes: > On Wed, 4 Aug 2010, Paul Guyot wrote: > > > > Yikes! erts_alloc() returned 0x80169917e for align 8 nrbytes 0 > > According to the C standard it isn't possible for C objects to have an > alignment larger than their size, so the check leading to this error > message is incorrect. The check is correct. It is however presented with pairs of parameters that are incorrect for a specific boundary case. From vladdu55@REDACTED Tue Aug 10 22:24:25 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 10 Aug 2010 22:24:25 +0200 Subject: [erlang-questions] RFC: parse_trans for code generation In-Reply-To: <4C2DC303.7060408@erlang-solutions.com> References: <4C2DC303.7060408@erlang-solutions.com> Message-ID: On Fri, Jul 2, 2010 at 12:44, Ulf Wiger wrote: > I realize that quite the minority of list readers take an > interest in parse transforms. Nevertheless, I thought I'd > highlight a small addition to the parse_trans project to > assist code generation. > > http://github.com/esl/parse_trans Hi Ulf, I think it's a very nice piece of work. I had once started something similar, but didn't get even this far. I'm glad that you did :-) The problem as I see it is that even if this was unanimously voted as the greatest thing since sliced bread, it has some usability issues. Without a simpler syntax, it's simply not going to be used, unfortunately. Additionally, parse transforms (for all their coolness) have the disadvantage that they make the code very difficult to debug (except for the toy examples). So what is needed is extensible syntax and tooling support ("un-parse transforms"). On the other hand, there is a place where this stuff is already usable, working (maybe not 100%, but close) and much more amenable to . I'm referring to LFE, of course. (Even the other beam-based languages can be considered here, but I haven't used tham) I'm guessing that the tooling support (with the debugger as the most important part) is lagging behind, but my guess is that it would be easier to develop than for Erlang-proper. The big question is when someone will get an itch severe enough to actually start scratching at it... best regards, Vlad From vladdu55@REDACTED Tue Aug 10 22:24:25 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 10 Aug 2010 22:24:25 +0200 Subject: [erlang-questions] RFC: parse_trans for code generation In-Reply-To: <4C2DC303.7060408@erlang-solutions.com> References: <4C2DC303.7060408@erlang-solutions.com> Message-ID: On Fri, Jul 2, 2010 at 12:44, Ulf Wiger wrote: > I realize that quite the minority of list readers take an > interest in parse transforms. Nevertheless, I thought I'd > highlight a small addition to the parse_trans project to > assist code generation. > > http://github.com/esl/parse_trans Hi Ulf, I think it's a very nice piece of work. I had once started something similar, but didn't get even this far. I'm glad that you did :-) The problem as I see it is that even if this was unanimously voted as the greatest thing since sliced bread, it has some usability issues. Without a simpler syntax, it's simply not going to be used, unfortunately. Additionally, parse transforms (for all their coolness) have the disadvantage that they make the code very difficult to debug (except for the toy examples). So what is needed is extensible syntax and tooling support ("un-parse transforms"). On the other hand, there is a place where this stuff is already usable, working (maybe not 100%, but close) and much more amenable to . I'm referring to LFE, of course. (Even the other beam-based languages can be considered here, but I haven't used tham) I'm guessing that the tooling support (with the debugger as the most important part) is lagging behind, but my guess is that it would be easier to develop than for Erlang-proper. The big question is when someone will get an itch severe enough to actually start scratching at it... best regards, Vlad From vladdu55@REDACTED Tue Aug 10 22:24:25 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 10 Aug 2010 22:24:25 +0200 Subject: [erlang-questions] RFC: parse_trans for code generation In-Reply-To: <4C2DC303.7060408@erlang-solutions.com> References: <4C2DC303.7060408@erlang-solutions.com> Message-ID: On Fri, Jul 2, 2010 at 12:44, Ulf Wiger wrote: > I realize that quite the minority of list readers take an > interest in parse transforms. Nevertheless, I thought I'd > highlight a small addition to the parse_trans project to > assist code generation. > > http://github.com/esl/parse_trans Hi Ulf, I think it's a very nice piece of work. I had once started something similar, but didn't get even this far. I'm glad that you did :-) The problem as I see it is that even if this was unanimously voted as the greatest thing since sliced bread, it has some usability issues. Without a simpler syntax, it's simply not going to be used, unfortunately. Additionally, parse transforms (for all their coolness) have the disadvantage that they make the code very difficult to debug (except for the toy examples). So what is needed is extensible syntax and tooling support ("un-parse transforms"). On the other hand, there is a place where this stuff is already usable, working (maybe not 100%, but close) and much more amenable to . I'm referring to LFE, of course. (Even the other beam-based languages can be considered here, but I haven't used tham) I'm guessing that the tooling support (with the debugger as the most important part) is lagging behind, but my guess is that it would be easier to develop than for Erlang-proper. The big question is when someone will get an itch severe enough to actually start scratching at it... best regards, Vlad From vladdu55@REDACTED Tue Aug 10 22:24:25 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 10 Aug 2010 22:24:25 +0200 Subject: [erlang-questions] RFC: parse_trans for code generation In-Reply-To: <4C2DC303.7060408@erlang-solutions.com> References: <4C2DC303.7060408@erlang-solutions.com> Message-ID: On Fri, Jul 2, 2010 at 12:44, Ulf Wiger wrote: > I realize that quite the minority of list readers take an > interest in parse transforms. Nevertheless, I thought I'd > highlight a small addition to the parse_trans project to > assist code generation. > > http://github.com/esl/parse_trans Hi Ulf, I think it's a very nice piece of work. I had once started something similar, but didn't get even this far. I'm glad that you did :-) The problem as I see it is that even if this was unanimously voted as the greatest thing since sliced bread, it has some usability issues. Without a simpler syntax, it's simply not going to be used, unfortunately. Additionally, parse transforms (for all their coolness) have the disadvantage that they make the code very difficult to debug (except for the toy examples). So what is needed is extensible syntax and tooling support ("un-parse transforms"). On the other hand, there is a place where this stuff is already usable, working (maybe not 100%, but close) and much more amenable to . I'm referring to LFE, of course. (Even the other beam-based languages can be considered here, but I haven't used tham) I'm guessing that the tooling support (with the debugger as the most important part) is lagging behind, but my guess is that it would be easier to develop than for Erlang-proper. The big question is when someone will get an itch severe enough to actually start scratching at it... best regards, Vlad From ulf.wiger@REDACTED Tue Aug 10 23:11:58 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 10 Aug 2010 23:11:58 +0200 Subject: [erlang-questions] RFC: parse_trans for code generation In-Reply-To: References: <4C2DC303.7060408@erlang-solutions.com> Message-ID: <4C61C09E.8020606@erlang-solutions.com> Hi Vlad, Thank you for that endorsement. :) I agree with the usability issues, but still think it's worthwhile to maintain a decent parse transform library for Erlang proper. I will leave it to others to explore the wonders of LFE. BR, Ulf W Vlad Dumitrescu wrote: > On Fri, Jul 2, 2010 at 12:44, Ulf Wiger wrote: >> I realize that quite the minority of list readers take an >> interest in parse transforms. Nevertheless, I thought I'd >> highlight a small addition to the parse_trans project to >> assist code generation. >> >> http://github.com/esl/parse_trans > > Hi Ulf, > > I think it's a very nice piece of work. I had once started something > similar, but didn't get even this far. I'm glad that you did :-) > > The problem as I see it is that even if this was unanimously voted as > the greatest thing since sliced bread, it has some usability issues. > Without a simpler syntax, it's simply not going to be used, > unfortunately. Additionally, parse transforms (for all their coolness) > have the disadvantage that they make the code very difficult to debug > (except for the toy examples). So what is needed is extensible syntax > and tooling support ("un-parse transforms"). > > On the other hand, there is a place where this stuff is already > usable, working (maybe not 100%, but close) and much more amenable to > . I'm referring to LFE, of course. (Even the other beam-based > languages can be considered here, but I haven't used tham) I'm > guessing that the tooling support (with the debugger as the most > important part) is lagging behind, but my guess is that it would be > easier to develop than for Erlang-proper. The big question is when > someone will get an itch severe enough to actually start scratching at > it... > > best regards, > Vlad -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From ngocdaothanh@REDACTED Wed Aug 11 04:17:43 2010 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Wed, 11 Aug 2010 11:17:43 +0900 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: Message-ID: The mostly used feature of OOP is probably encapsulating code and data. Have a look at gen_server: http://www.erlang.org/doc/man/gen_server.html From rzezeski@REDACTED Wed Aug 11 04:33:41 2010 From: rzezeski@REDACTED (Ryan Zezeski) Date: Tue, 10 Aug 2010 22:33:41 -0400 Subject: [erlang-questions] Reinserting a message into a gen_server's mailbox In-Reply-To: References: Message-ID: On Thu, Jul 29, 2010 at 11:15 AM, Juan Jose Comellas wrote: > I have a situation where for one specific gen_server:call/2 invocation I > have to perform an action that may take a few seconds (e.g. reading from a > DB; reading from external files). I don't want to block the gen_server > process while I do this, so I want to spawn a helper process that takes > care > of reading the data I need. The problem is that I can receive several > gen_server:call/2 invocations that depend on this data being present at the > same time and I don't want to have them fail because the data is not yet > ready. I've thought of two ways of solving this problem: > > > I'm currently writing an application that needs similar behavior, i.e. high request throughput on single gen_server, but the requests may take a long time to service. I approached the problem by creating a gen_server name req_delegate (under a supervisor w/ one_for_one restart policy) whose sole purpose in life is to spawn req_hdlr processes and kick them off. This way the only time spent in the gen_server is the time needed to create a new process and send a msg to it. The req_hdlr is a worker underneath a supervisor (underneath the main supervisor) with a simple_one_for_one restart policy. When a new one is created it is given a reference to the process that made the request and it uses this to msg back the result. When the req_hdlr is done it simply dies. My req_delegate has a function that looks like this: handle_cast({new_request, Req}, State) -> %% Start new req_hdlr to process request {ok, Pid} = supervisor:start_child(req_hdlr_sup, [Req]), %% Tell it to process the request req_hdlr:process(Pid), {noreply, State}. The caller then waits for a response from the newly spawned req_hdlr worker. I wrote a library module to make this easier. This has worked great for me so far. However, the next thing I want to address (as I think you hinted to as well) is how to only have one req_hdlr do the work when there are multiple requests for the same data. In my case, I'm using this app as the engine behind a web application. I want to be able to take 10 concurrent requests for the same data, and perform the work only once. I haven't solved this yet, but I have some ideas. One idea is to normalize every request to an ID that represents what it's asking for. If the req_delegate notices there is already a request for a particular ID then it can have the req_hdlr piggy back off the request that is already in progress. Anyways, I hope I'm making sense. I'm still _very_ new to Erlang and trying to learn. -Ryan From igarai@REDACTED Wed Aug 11 04:37:11 2010 From: igarai@REDACTED (=?UTF-8?B?ScOxYWtpIEdhcmF5?=) Date: Tue, 10 Aug 2010 23:37:11 -0300 Subject: [erlang-questions] Using ETS for large amounts of data? In-Reply-To: <20100809231325.GF51936@alumni.caltech.edu> References: <20100809180808.GD51936@alumni.caltech.edu> <20100809231325.GF51936@alumni.caltech.edu> Message-ID: This is fascinating. Please do post your timing results. -- I?aki Garay. From hynek@REDACTED Wed Aug 11 08:46:03 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Wed, 11 Aug 2010 08:46:03 +0200 Subject: [erlang-questions] Reinserting a message into a gen_server's mailbox In-Reply-To: References: Message-ID: On Wed, Aug 11, 2010 at 4:33 AM, Ryan Zezeski wrote: > On Thu, Jul 29, 2010 at 11:15 AM, Juan Jose Comellas wrote: > >> I have a situation where for one specific gen_server:call/2 invocation I >> have to perform an action that may take a few seconds (e.g. reading from a >> DB; reading from external files). I don't want to block the gen_server >> process while I do this, so I want to spawn a helper process that takes >> care >> of reading the data I need. The problem is that I can receive several >> gen_server:call/2 invocations that depend on this data being present at the >> same time and I don't want to have them fail because the data is not yet >> ready. I've thought of two ways of solving this problem: >> >> >> > I'm currently writing an application that needs similar behavior, i.e. high > request throughput on single gen_server, but the requests may take a long > time to service. ?I approached the problem by creating a gen_server name > req_delegate (under a supervisor w/ one_for_one restart policy) whose sole > purpose in life is to spawn req_hdlr processes and kick them off. ?This way > the only time spent in the gen_server is the time needed to create a > new process and send a msg to it. ?The req_hdlr is a worker underneath a > supervisor (underneath the main supervisor) with a simple_one_for_one > restart policy. ?When a new one is created it is given a reference to the > process that made the request and it uses this to msg back the result. ?When > the req_hdlr is done it simply dies. > > My req_delegate has a function that looks like this: > > handle_cast({new_request, Req}, State) -> > ? ?%% Start new req_hdlr to process request > ? ?{ok, Pid} = supervisor:start_child(req_hdlr_sup, [Req]), > ? ?%% Tell it to process the request > ? ?req_hdlr:process(Pid), > ? ?{noreply, State}. > > The caller then waits for a response from the newly spawned req_hdlr worker. > ?I wrote a library module to make this easier. > > This has worked great for me so far. ?However, the next thing I want to > address (as I think you hinted to as well) is how to only have one req_hdlr > do the work when there are multiple requests for the same data. ?In my case, > I'm using this app as the engine behind a web application. ?I want to be > able to take 10 concurrent requests for the same data, and perform the work > only once. ?I haven't solved this yet, but I have some ideas. ?One idea is > to normalize every request to an ID that represents what it's asking for. > ?If the req_delegate notices there is already a request for a particular ID > then it can have the req_hdlr piggy back off the request that is already in > progress. > > Anyways, I hope I'm making sense. ?I'm still _very_ new to Erlang and trying > to learn. > > -Ryan > One idea comes on mine mind, why there is "high request throughput on *single* gen_server"? Where those requests comes from? If I read handle_cast({new_request, Req}, State) -> %% Start new req_hdlr to process request {ok, Pid} = supervisor:start_child(req_hdlr_sup, [Req]), %% Tell it to process the request req_hdlr:process(Pid), {noreply, State}. I can't see any reason why this request arrived to this single gen_server at all. There is known anti-pattern in the linux kernel development named "midlayer mistake" and common advice is, if there is not a reason for midlayer use a library Luke. Translated to Erlang: if there is not a reason for process use a module Luke. I know it is strong temptation add some abstract layer and separate things and server is first thing what comes on mind in Erlang but stateless module often serves too. Very similar advice is written in subsection A Case Study on Concurrency-Oriented Programming of Chapter 4: Concurrent Programming of famous Cesarini and Thompson's book Erlang Programming (page 110). Where "high request throughput" comes from? If it is from already concurrent activity (web server request handler I guess) keep it then. I did a lot of same mistakes and will do but I hope I will wrap mine head around and will do it less and less in future. -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From wiener.guy@REDACTED Wed Aug 11 09:44:22 2010 From: wiener.guy@REDACTED (Guy Wiener) Date: Wed, 11 Aug 2010 10:44:22 +0300 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: Message-ID: I have to stand up for OOP at this point: OOP does not start and end with "encapsulating code and data". The OOP feature that has the strongest impact on code brevity is *inheritance*. Thus, the interest in extending modules. Also, IMHO, parameterized modules are a more compact way to encapsulate code and data, since that you don't have to pass the data structure around as an explicit state (assuming, of course, that the data is immutable). Guy On Wed, Aug 11, 2010 at 5:17 AM, Ngoc Dao wrote: > The mostly used feature of OOP is probably encapsulating code and > data. Have a look at gen_server: > http://www.erlang.org/doc/man/gen_server.html > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From hynek@REDACTED Wed Aug 11 10:37:33 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Wed, 11 Aug 2010 10:37:33 +0200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: Message-ID: On Wed, Aug 11, 2010 at 9:44 AM, Guy Wiener wrote: > I have to stand up for OOP at this point: OOP does not start and end with > "encapsulating code and data". The OOP feature that has the strongest impact > on code brevity is *inheritance*. Thus, the interest in extending modules. I can't agree with you. Inheritance is only way how some OOP language copes with complexity of OO systems. Inheritance is neither core nor mandatory feature of OOP. What worse, many of OO affected people what I met stated that inheritance is most controversial and mostly misused feature of OOP. I also observe that there is positive correlation with person experience and probability that will tell or agree with above statement. > > Also, IMHO, parameterized modules are a more compact way to encapsulate code > and data, since that you don't have to pass the data structure around as an > explicit state (assuming, of course, that the data is immutable). > > Guy > > > On Wed, Aug 11, 2010 at 5:17 AM, Ngoc Dao wrote: > >> The mostly used feature of OOP is probably encapsulating code and >> data. Have a look at gen_server: >> http://www.erlang.org/doc/man/gen_server.html >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From jesper.louis.andersen@REDACTED Wed Aug 11 11:03:13 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 11 Aug 2010 11:03:13 +0200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: Message-ID: On Wed, Aug 11, 2010 at 10:37 AM, Hynek Vychodil wrote: > On Wed, Aug 11, 2010 at 9:44 AM, Guy Wiener wrote: >> I have to stand up for OOP at this point: OOP does not start and end with >> "encapsulating code and data". The OOP feature that has the strongest impact >> on code brevity is *inheritance*. Thus, the interest in extending modules. > > I can't agree with you. Me neither. To provoke: Inheritance (In Java/C++ style OO) has a strong impact, but it is negative rather than positive. In my experience, all languages with no inheritance features whatsoever beat those with in brevity. In Erlang, whenever somebody thinks inheritance, you should take a step back and think "closure" or "higher order function". The whole concept of "Red car inherits from car" does not work, save for introductory Java courses. So in practice, inheritance is used to facilitate code reuse and composition - and in my experience, functional programming tend to have much better tools for composing code[1]. Another point is google Go, which has no inheritance per se, but uses an interface concept: an object can belong to zero, one or many interfaces by implementing the methods in that interface. The declaration is implicit and inferred so the programmer is saved from doing boilerplate notation. That is, interface implementation is purely structural. "If you implement these methods with these types, you are an implementor of the interface and can be used whenever the interface is used." There is a way to embed all methods from one object into another which gives somewhat the inheritance concept, but it is subtly different. Structurally constructing the type hierarchy is not new. Ocaml has had it for years. Parameterized modules make me warm and fuzzy because they seem to give a way to implement ML-style functors (maps from modules to modules). It may look like an object, but I don't think it is. Module parameters are static unless you jump hoops for instance. [1] Erlang, even with a lack of currying, fares better here. -- J. From ulf.wiger@REDACTED Wed Aug 11 11:29:24 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 11 Aug 2010 11:29:24 +0200 Subject: [erlang-questions] Reinserting a message into a gen_server's mailbox In-Reply-To: References: Message-ID: <4C626D74.8050208@erlang-solutions.com> Hynek Vychodil wrote: > I can't see any reason why this request arrived to this single > gen_server at all. There is known anti-pattern in the linux kernel > development named "midlayer mistake" and common advice is, if there is > not a reason for midlayer use a library Luke. Translated to Erlang: if > there is not a reason for process use a module Luke. I know it is > strong temptation add some abstract layer and separate things and > server is first thing what comes on mind in Erlang but stateless > module often serves too. I agree, and in my experience, this extra layer is often made necessary by the simple fact that you cannot use complex terms as locally registered names. Thus, you end up having to 'index' the workers somehow. Providing a generic solution to this was the main driving force behind gproc (http://github.com/esl/gproc). If a dispatching server ends up only looking at a request and forwarding it to the right server, it may well be that the processes doing the actual work could simply register themselves appropriately in gproc and eliminate the dispatcher altogether. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From rumata-estor@REDACTED Wed Aug 11 09:29:16 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Wed, 11 Aug 2010 11:29:16 +0400 Subject: [erlang-questions] open_port() on FD 0,1 problems In-Reply-To: References: Message-ID: <4C62514C.4070706@nm.ru> You can try to change file mode from text to binary. Dmitry Belyaev On 08/10/2010 08:44 PM, Daniel Goertzen wrote: > Thanks! That helps a lot. > > New issue is that erl.exe is translating incoming 0x0d bytes to 0x0a. > Windows line-end conversions, grumble, grumble. Is there a nice way to > turn that off? Or am I at a dead-end? > > Thanks, > Dan. > > On Tue, Aug 10, 2010 at 9:05 AM, Hynek Vychodil wrote: > > >> I don't know how on Windows but try -noinput parameter. >> >> On Tue, Aug 10, 2010 at 3:39 PM, Daniel Goertzen >> wrote: >> >>> I am trying to operate an Erlang Port for stdin and stdout >>> [open_port({fd,0,1}...]. The program below is to receive two integers, >>> multiply them, and write the result. Every other request vanishes without >>> >> a >> >>> reply, and I don't know why. Details: >>> >>> - Windows 7 64 bit. >>> - Erlang launched with "-noshell -s erlang_rpc_benchmark" >>> - I have to wait a moment after launching erlang before sending it data, >>> >> or >> >>> I definitely get dataloss. >>> - Exactly every other packet gets lost, starting with the first one. >>> - The byte counters on Task Manager show that the request is getting to >>> erl.exe, but not to my code. >>> - I've added debug code to verify that there are no other messages >>> >> appearing >> >>> at my receive expression (not currently in code below). >>> - I wrote a Python version of this program, and it worked without issue. >>> >>> >>> Background: I plan to write a Python GUI program with an Erlang child >>> process. The test program below is to assess Python->Erlang messaging >>> performance. I am open to other IPC approaches if there is something >>> better. >>> >>> Thanks, >>> Dan. >>> >>> >>> -module(erlang_rpc_benchmark). >>> -export([start/0, std_server/0]). >>> >>> start() -> >>> spawn(fun std_server/0). >>> >>> std_server() -> >>> Port = open_port( {fd,0,1}, [{packet, 4}, binary ] ), >>> loop(Port). >>> >>> loop(Port) -> >>> receive >>> {Port, { data,<> }} -> >>> Result = A*B, >>> Port ! {self(), {command,<>}}; >>> end, >>> loop(Port). >>> >>> >> >> >> -- >> --Hynek (Pichi) Vychodil >> >> Analyze your data in minutes. Share your insights instantly. Thrill >> your boss. Be a data hero! >> Try GoodData now for free: www.gooddata.com >> >> > > > From ulf.wiger@REDACTED Wed Aug 11 12:24:31 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 11 Aug 2010 12:24:31 +0200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: Message-ID: <4C627A5F.9060802@erlang-solutions.com> Jesper Louis Andersen wrote: > On Wed, Aug 11, 2010 at 10:37 AM, Hynek Vychodil wrote: >> On Wed, Aug 11, 2010 at 9:44 AM, Guy Wiener wrote: >>> I have to stand up for OOP at this point: OOP does not start and end with >>> "encapsulating code and data". The OOP feature that has the strongest impact >>> on code brevity is *inheritance*. Thus, the interest in extending modules. >> I can't agree with you. > > Me neither. To provoke: Inheritance (In Java/C++ style OO) has a > strong impact, but it is negative rather than positive. In my > experience, all languages with no inheritance features whatsoever beat > those with in brevity. In Erlang, whenever somebody thinks > inheritance, you should take a step back and think "closure" or > "higher order function". The whole concept of "Red car inherits from > car" does not work, save for introductory Java courses. So in > practice, inheritance is used to facilitate code reuse and composition > - and in my experience, functional programming tend to have much > better tools for composing code[1]. It's certainly useful to have the slightly abstracted discussion: What types of programs are a breeze to write in language X, but cumbersome in Erlang? What parts of Erlang make it cumbersome? Could they be fixed? Do we care enough to fix it? There are certainly problems that are not solved as easily in Erlang as in other languages. Backtracking a bit, we can observe that one of the first areas where Erlang departed from its main inspiration, Prolog, was ...uhm, backtracking. This was not done because backtracking is bad per se, but because it doesn't play that well with soft real-time characteristics and hardware control. Similarly there have been discussions on other features that are much liked by proponents of Language X, but not found in Erlang: - Strong static (a la Haskell & ML) typing is praised by many as a great boon to software quality and productivity. Explaining why Erlang doesn't have it sometimes borders on revisionism, but when defending his PhD thesis, Joe was asked about it and basically said that at the time, they didn't know that much about static typing, and were mainly inspired by dynamically typed languages like LISP and Prolog. In other words, it wasn't so much a question of deciding *against* static typing as getting inspiration elsewhere. Still, managing large projects with loosely coupled components that need to be upgradable in service is not an easy problem to solve, and largely still a research area. - Lazy evaluation is superb for some problems, and e.g. QuickCheck relies heavily on it. While it can be done in Erlang, you have to make do with "poor-man's lazy evaluation", rolling your own. Again, going all the way seems to be somewhat at odds with soft real-time characteristics and predictable memory utilization. - Shared-memory concurrency is the cat's miau for some problems, but Erlang carefully stays away from it, since it was designed for problems where shared memory is more trouble than it's worth - not least because all bets are off in terms of fault tolerance if a process with write access to your memory dies in the process of modifying it. Finally, OO was on the plate even during the language experiments that led to Erlang. It received favourable mention, but the old papers (the ones I've read) don't clearly describe why OO features were left out. I believe the main reason was that the team decided to use Prolog as a base - as it was closest to what they imagined. It was only later that they decided to make a language in its own right. My own experience from telecoms tells me that OO gives the wrong signals to people in complex projects. They tend to want to maximize design-time dependencies rather than going with a style more similar to electrical component design - black boxes, with well-defined interfaces. While expert OO programmers tend to violently disagree with me, my opinion is based more on what mere mortals tend to do with OO than on what the experts do. I've seen too many occasions where C++ hackers have come up with a great-looking prototype, offering superb performance and seemingly ultra-compact code, although carefully ignoring many of the harder problems and exception flows. The problem is that the prototype is then handed off to "normal" programmers, who have to address all the trickiness that the prototype didn't cover, and that's when it tends to go south... In short, I have not missed OO when using Erlang in its original domain. I have, on occasion, missed being able to conveniently extend modules and override individual functions when working on other things (e.g. like extending mnesia). OTOH, I've also dug into some Ruby-on-Rails code to see if I could extend some Redmine plugins, and that experience only strengthened my opinion that you can go horribly wrong with OO and method overriding - especially when there can be many modules overriding the same function, and it's not especially clear in which order they are processed[1]. The hardest part of designing a language is probably not what to include, but what to leave out. Erlang is not a multi-paradigm language[2], and shouldn't strive to become one IMHO. BR, Ulf W [1] Actually it is clearly defined, but it took some time for me to find it. RoR defines one order, and Redmine modifies it. Maybe it's an acquired taste, but I must say I didn't immediately take to it. [2] There are such languages; e.g. Oz, Scala and perhaps LISP (although LISP probably deserves a classification of its own). -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From ulf.wiger@REDACTED Wed Aug 11 12:38:15 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 11 Aug 2010 12:38:15 +0200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: <4C627A5F.9060802@erlang-solutions.com> References: <4C627A5F.9060802@erlang-solutions.com> Message-ID: <4C627D97.2080205@erlang-solutions.com> Ulf Wiger wrote: > ...Still, managing large projects [WELL] > with loosely coupled components [WRITTEN IN A STATICALLY > TYPED LANGUAGE] that need to be upgradable in > service is not an easy problem to solve, and largely still a > research area. I accidentally left out the parts in brackets above. I think Erlang solves it pretty well, albeit often through convention and intuitive patterns rather than with guarantees. It could well be that we will eventually see solutions based on statically typed languages that turn out to be superior, if they can come up with ways to write concise and readable code that is statically checkable to an extent that Erlang code possibly never will be. It'd be sad if that didn't happen, actually. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From ngocdaothanh@REDACTED Wed Aug 11 13:20:56 2010 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Wed, 11 Aug 2010 20:20:56 +0900 Subject: [erlang-questions] OOP in Erlang In-Reply-To: <4C627D97.2080205@erlang-solutions.com> References: <4C627A5F.9060802@erlang-solutions.com> <4C627D97.2080205@erlang-solutions.com> Message-ID: > when defending his PhD thesis, Joe was asked about it and basically said that at the time >From http://www.infoq.com/news/2010/07/objects-smalltalk-erlang -------------------------- Joe Armstrong's thesis advisor is quoted making a very similar argument: I started wondering about what object oriented programming was and I thought Erlang wasn't object oriented, it was a functional programming language. Then, my thesis supervisor said "But you're wrong, Erlang is extremely object oriented". He said object oriented languages aren't object oriented. I might think, though I'm not quite sure if I believe this or not, but Erlang might be the only object oriented language because the 3 tenets of object oriented programming are that it's based on message passing, that you have isolation between objects and have polymorphism. Dr. Armstrong indicates he is not entirely convinced by his advisor's arguments, but does seem to think that Erlang "might be the only object oriented language." -------------------------- From e_d_k@REDACTED Wed Aug 11 13:04:39 2010 From: e_d_k@REDACTED (Ed Keith) Date: Wed, 11 Aug 2010 04:04:39 -0700 (PDT) Subject: [erlang-questions] OOP in Erlang In-Reply-To: Message-ID: <611396.69573.qm@web120515.mail.ne1.yahoo.com> --- On Wed, 8/11/10, Hynek Vychodil wrote: > From: Hynek Vychodil > Subject: Re: [erlang-questions] OOP in Erlang > To: "Guy Wiener" > Cc: "Erlang Questions" > Date: Wednesday, August 11, 2010, 4:37 AM > On Wed, Aug 11, 2010 at 9:44 AM, Guy > Wiener > wrote: > > I have to stand up for OOP at this point: OOP does not > start and end with > > "encapsulating code and data". The OOP feature that > has the strongest impact > > on code brevity is *inheritance*. Thus, the interest > in extending modules. > > I can't agree with you. Inheritance is only way how some > OOP language > copes with complexity of OO systems. Inheritance is neither > core nor > mandatory feature of OOP. What worse, many of OO affected > people what > I met stated that inheritance is most controversial and > mostly misused > feature of OOP. I also observe that there is positive > correlation with > person experience and probability that will tell or agree > with above > statement. > > > > > Also, IMHO, parameterized modules are a more compact > way to encapsulate code > > and data, since that you don't have to pass the data > structure around as an > > explicit state (assuming, of course, that the data is > immutable). > > > > Guy > > > > > > On Wed, Aug 11, 2010 at 5:17 AM, Ngoc Dao > wrote: > > > >> The mostly used feature of OOP is probably > encapsulating code and > >> data. Have a look at gen_server: > >> http://www.erlang.org/doc/man/gen_server.html > >> > >> When I first studied OOP, in 1990, I was taught that OO was defined by four features: Encapsulation, Abstraction, Inheritance, and Polymorphism. That if any of these were missing it was not OO. -EdK Ed Keith e_d_k@REDACTED Blog: edkeith.blogspot.com From attila.r.nohl@REDACTED Wed Aug 11 14:08:40 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 11 Aug 2010 14:08:40 +0200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: <4C627A5F.9060802@erlang-solutions.com> References: <4C627A5F.9060802@erlang-solutions.com> Message-ID: 2010/8/11, Ulf Wiger : [...] > In short, I have not missed OO when using Erlang in its original > domain. I have, on occasion, missed being able to conveniently > extend modules and override individual functions when working > on other things (e.g. like extending mnesia). This is something I also miss. I've seen way too many instances of code simply copied and only 2 functions were substantially changed out of 16. Of course, this can be refactored, but it just doesn't feel elegant. But it's nothing compared to the headache caused by the four different ways to end a "statement": comma, full stop, semicolon or nothing. From mononcqc@REDACTED Wed Aug 11 14:25:21 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 11 Aug 2010 08:25:21 -0400 Subject: [erlang-questions] OOP in Erlang In-Reply-To: <4C627A5F.9060802@erlang-solutions.com> References: <4C627A5F.9060802@erlang-solutions.com> Message-ID: On Wed, Aug 11, 2010 at 6:24 AM, Ulf Wiger wrote: > > In short, I have not missed OO when using Erlang in its original > domain. I have, on occasion, missed being able to conveniently > extend modules and override individual functions when working > on other things (e.g. like extending mnesia). > > BR, > Ulf W > I believe you use mnesia:activity/4 to do this (change the callback module), but wouldn't the -extend(Mod) module attribute solve your problem (even in the case of mnesia?). As far as I know, it does exactly that: override functions and leave the others to be called from 'Mod' when they are undefined. Any reason why this wouldn't do or why you [maybe] dislike the feature? From rzezeski@REDACTED Wed Aug 11 14:39:18 2010 From: rzezeski@REDACTED (Ryan Zezeski) Date: Wed, 11 Aug 2010 08:39:18 -0400 Subject: [erlang-questions] Reinserting a message into a gen_server's mailbox In-Reply-To: References: Message-ID: On Wed, Aug 11, 2010 at 2:46 AM, Hynek Vychodil wrote: > > > I can't see any reason why this request arrived to this single > gen_server at all. There is known anti-pattern in the linux kernel > development named "midlayer mistake" and common advice is, if there is > not a reason for midlayer use a library Luke. Translated to Erlang: if > there is not a reason for process use a module Luke. I know it is > strong temptation add some abstract layer and separate things and > server is first thing what comes on mind in Erlang but stateless > module often serves too. Very similar advice is written in subsection > A Case Study on Concurrency-Oriented Programming of Chapter 4: > Concurrent Programming of famous Cesarini and Thompson's book Erlang > Programming (page 110). Where "high request throughput" comes from? If > it is from already concurrent activity (web server request handler I > guess) keep it then. I did a lot of same mistakes and will do but I > hope I will wrap mine head around and will do it less and less in > future. > > As I thought about your response this morning I realized--you are absolutely right! There is no reason that I need this intermediate gen_server in my design. I think my reason for having it was more by accident as I evolved the application. Originally I had one gen_server satisfying all requests to my app. At some point I realized this could cause a bottle-neck if a request was long running, so I then took my above approach. Now, it's obvious that I could just use a library module to achieve the same thing. -Ryan From jesper.louis.andersen@REDACTED Wed Aug 11 14:44:47 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 11 Aug 2010 14:44:47 +0200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: <4C627A5F.9060802@erlang-solutions.com> References: <4C627A5F.9060802@erlang-solutions.com> Message-ID: On Wed, Aug 11, 2010 at 12:24 PM, Ulf Wiger wrote: I simply cannot resist replying to this one :) > It's certainly useful to have the slightly abstracted discussion: > What types of programs are a breeze to write in language X, but > cumbersome in Erlang? What parts of Erlang make it cumbersome? > Could they be fixed? Do we care enough to fix it? I think your last paragraph is the most important. A language is defined by what you leave out rather than what you let slip in. So it is entirely possible to punt on certain problems and eschew their addition to the programming language in question. There are two kinds of programs I still prefer to write in Ocaml rather than Erlang: 1) Programs which are relying heavily on algebraic datatypes (ADTs). That is, symbolic manipulation. A struct where the first field is tagged with an atom() type simply just doesn't feel the same as having a rigorously defined variant type with fast member access. 2) Programs with a computationally intensive kernel. An optimization in Ocaml makes arrays of floats be unboxed by default. This means that numeric code actually runs fast in ocaml. Add all the virtues of functional programming on top of this and you have an imperative/functional hybrid which rips apart most other programming languages. One way around this trouble can be achieved by using NIFs and write the kernels in C. Ocaml doesn't fare well on concurrency however. It can do parallellism with some trouble (There is a rather nice MPI-interface), but for concurrency, I'd much rather have Erlang. The only way to add this to Erlang is to open up Pandoras box. This box contain the destructive updates and I would rather that it is not opened. [...] > - Strong static (a la Haskell & ML) typing is praised by many > ?as a great boon to software quality and productivity. With a channel-primitive, it is possible to add strong static typing to a concurrent language. The upgradeable processes part is far more interesting however. One simple way to achieve it is to postpone type-checking till on-load-time but you will still need to resolve types around channels if they change. In principle you need functionality akin to code_change/3 on the type level - and perhaps some fairly deep dependency tracking features if more than one module is loaded. Indeed, the ramifications are deep. > - Lazy evaluation is superb for some problems, and e.g. QuickCheck > ?relies heavily on it. While it can be done in Erlang, you have > ?to make do with "poor-man's lazy evaluation", rolling your own. > ?Again, going all the way seems to be somewhat at odds with soft > ?real-time characteristics and predictable memory utilization. Lazy evaluation is also a curse for other problems. Lazily evaluating is slower so you need strictness analysis to get it to run fast. If the strictness analyzers can't "see" what is going on in the program you end up with a program that in the worst case fills up all of your memory. In the common case, the program will fill up memory and then consume it, fill up, consume, ..., and so on. This way of computing is hard on the memory bandwidth - the scarcest resource we have in modern computers. Also, writing daemons which do not leak takes time as you have to go over all the details of strictness in the program. I much prefer strict evaluation and then the ability to "go lazy" on a by-need basis. Note that rpc:async_call/4 and its cousin rpc:yield/1 defines a lazy promise if the call is *not* executed by a separate thread in the background. Alice ML uses lazy promises like this for lazy evaluation and in addition for concurrent background work as in Erlang. > - Shared-memory concurrency is the cat's miau for some problems, > ?but Erlang carefully stays away from it, since it was designed > ?for problems where shared memory is more trouble than it's > ?worth - not least because all bets are off in terms of fault > ?tolerance if a process with write access to your memory dies > ?in the process of modifying it. There is one idea here I have been toying with. One problem of Erlangs memory model is that sending a large datastructure as a capability to another process, several megabytes in size, will mean a copy. In the default VM setup that is. But if you had a region into which it got allocated, then that region could safely be sent under a proof that the original process will not touch it anymore. It is possible to infer regions like these (See Tofte/Talpin and later Niss/Makholm/Henglein) but I would opt for a simpler variant with explicit mention of regions for starters. If the (albeit cumbersome) model of explicit region management can't work out, why have any hope for an automatic variant? There is also a tangent here with substructural type systems, linear types or uniqueness types in particular. A destructive update can be allowed if we can prove we are the only one holding a reference to the data. But unless such a system works out rahter clearly, it is not worth the hassle of adding. Practical usefulness is more important for Erlang - so it should be kept in a theoretic toy language first for play, fun, and profit. > My own experience from telecoms tells me that OO gives the wrong > signals to people in complex projects. They tend to want to > maximize design-time dependencies rather than going with a style > more similar to electrical component design - black boxes, with > well-defined interfaces. This is my experience as well. The trap of OO is to be an architecture astronaut and define a wild system - only to find that your wrapping of objects 4-5 times is what kills your Java heap. And Garbage collectors with one big heap greatly dislikes having to walk large amounts of live data... > The hardest part of designing a language is probably not what to > include, but what to leave out. Erlang is not a multi-paradigm > language[2], and shouldn't strive to become one IMHO. Precisely. It is about choosing a few concepts which can support each other well with little overlap. And aggressively shaving off everything that Occam's Razor suggest. For that reason, Joe Armstrongs idea of removing something whenever you add something is a good one. Improvement should come from "silent" optimizations, like the gen_server:call/2,3 mailbox optimization recently added. I for one have enjoyed that every time a new version of Erlang came out, my program got faster. -- J. From rzezeski@REDACTED Wed Aug 11 14:45:56 2010 From: rzezeski@REDACTED (Ryan Zezeski) Date: Wed, 11 Aug 2010 08:45:56 -0400 Subject: [erlang-questions] Reinserting a message into a gen_server's mailbox In-Reply-To: <4C626D74.8050208@erlang-solutions.com> References: <4C626D74.8050208@erlang-solutions.com> Message-ID: On Wed, Aug 11, 2010 at 5:29 AM, Ulf Wiger wrote: > > > I agree, and in my experience, this extra layer is often made > necessary by the simple fact that you cannot use complex terms > as locally registered names. Thus, you end up having to 'index' > the workers somehow. > > Providing a generic solution to this was the main driving force > behind gproc (http://github.com/esl/gproc). > > If a dispatching server ends up only looking at a request and > forwarding it to the right server, it may well be that the > processes doing the actual work could simply register themselves > appropriately in gproc and eliminate the dispatcher altogether. > Ulf, gproc looks like it could be used for my many redundant requests/only do the work once problem? As long as I can normalize the request to some type of term to act as an identifier I would be able to discern if work was already being done for that request and if so I could use it to satisfy multiple clients? This is a little more transient than "looking at a request and forwarding it to the right server" as these are short lived "servers" but it seems to be the same idea, no? From steven.charles.davis@REDACTED Wed Aug 11 15:20:46 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 11 Aug 2010 06:20:46 -0700 (PDT) Subject: OOP in Erlang In-Reply-To: <611396.69573.qm@web120515.mail.ne1.yahoo.com> References: <611396.69573.qm@web120515.mail.ne1.yahoo.com> Message-ID: <1858bc4d-497b-4953-b083-f0d4df21f9d5@k10g2000yqa.googlegroups.com> On Aug 11, 6:04?am, Ed Keith wrote: > When I first studied OOP, in 1990, I was taught that OO was defined by four features: Encapsulation, Abstraction, Inheritance, and Polymorphism. That if any of these were missing it was not OO. > > ? ? ? -EdK I was taught this also. However, I'm pretty certain that Alan Kay doesn't subscribe to this definition. Java-style OOP, of course, does. Let's not ignore what java objects are. Instances of user-defined types that are bundled with a number of transformations (methods) that are legal (hopefully) on that type. Looking at this another way, there's really only two things going on here: 1) Data structures and 2) Transformations of data structures. The class type definition defines a data structure, often as a composite of other, previously defined structures. "Encapsulation" insulates you from this detail. Thus you can program in a world at a higher level, and that world of "Abstraction" was defined by one or more programmers each with a different view of correctness and applicability. So you'd better hope that all the detail of the implementation was appropriate and well thought through. Of course it usually isn't entirely appropriate to *your* problem and *your* current notion of applicability... ...but never mind along comes a toolset to change all that."Inheritance" will allow you to add to or modify the underlying data structure and define or redefine the applicable transformations. You can re-invent your abstraction as you please. Obviously, you are taking care to make sure everything is correct and applicable to the problem domain (at least, the one you are currently aware of). Any mis- step in this process has ramifications way beyond what you can currently appreciate, when other programmers program in the world of your new abstractions, along with any other issues inherited by it. While you are making this wonderful re-useable software by extension of other data structures and transformations you can do some radical things to fix up problems of the past. Overloading, to allow you to parameterize your transformations with other structures that may impact the transformation of the subject of the transformation. With overriding, you can just redefine the entire transformation. Somehow you want to allow all these (different) types to continue to be used with this new transformations, so you need type "Polymorphism". You are careful to be sure that your new transformation (method) will be applicable and correct for all data structures that could be represented by the polymorphic type (interface), so I guess you really need to know quite a bit about the abstracted and encapsulated data structures so that your system doesn't end up in an illegal state of some kind. Of course when you don't know what type to use, just can just shove in a 'null'. Having built your beautiful cathedral of types and transformations, you may then need to define and implement a protocol to send it as binaries over the network. To conform to the contract of the protocol, you'll need to account for all that hidden state and implicit transformations of it to keep the system consistent. So... does OOP (Java-style) sound complicated and hard to manage? It is. You are usually dealing with in a language of type definitions, invented by all the programmers who touched the code, each with different motivations, and which you do not (and probably cannot) fully understand. You have lost the ability to reason effectively about your program and your system. /s From mononcqc@REDACTED Wed Aug 11 15:59:01 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 11 Aug 2010 09:59:01 -0400 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: <4C627A5F.9060802@erlang-solutions.com> Message-ID: On Wed, Aug 11, 2010 at 8:44 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > There is one idea here I have been toying with. One problem of Erlangs > memory model is that sending a large datastructure as a capability to > another process, several megabytes in size, will mean a copy. In the > default VM setup that is. But if you had a region into which it got > allocated, then that region could safely be sent under a proof that > the original process will not touch it anymore. It is possible to > infer regions like these (See Tofte/Talpin and later > Niss/Makholm/Henglein) but I would opt for a simpler variant with > explicit mention of regions for starters. If the (albeit cumbersome) > model of explicit region management can't work out, why have any hope > for an automatic variant? There is also a tangent here with > substructural type systems, linear types or uniqueness types in > particular. A destructive update can be allowed if we can prove we are > the only one holding a reference to the data. > > But unless such a system works out rahter clearly, it is not worth the > hassle of adding. Practical usefulness is more important for Erlang - > so it should be kept in a theoretic toy language first for play, fun, > and profit. > > One interesting point of *always* copying data structures is that you need to plan for small messages (as far as possible) whether you are on a single node or in a distributed setting. Moving up from a [partially] shared memory model to a fully isolated one when going distributed is likely going to have its share of performance problems and might create a dissonance between "what is acceptable locally" and "what is acceptable when distributed". I believe forcing messages to always be copied (except for large binaries) reinforce the transparency and predictability of distribution in Erlang, on top of keeping it all conceptually simpler on all levels. I'm not sure I would necessarily like to give up these hard-to-notice-but-still-pretty-useful advantages for the sake of speed on local nodes. From rvirding@REDACTED Wed Aug 11 16:02:22 2010 From: rvirding@REDACTED (Robert Virding) Date: Wed, 11 Aug 2010 16:02:22 +0200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: <4C627A5F.9060802@erlang-solutions.com> Message-ID: On 11 August 2010 14:08, Attila Rajmund Nohl wrote: > 2010/8/11, Ulf Wiger : > [...] >> In short, I have not missed OO when using Erlang in its original >> domain. I have, on occasion, missed being able to conveniently >> extend modules and override individual functions when working >> on other things (e.g. like extending mnesia). > > This is something I also miss. I've seen way too many instances of > code simply copied and only 2 functions were substantially changed out > of 16. Of course, this can be refactored, but it just doesn't feel > elegant. That's just poor programming. > But it's nothing compared to the headache caused by the four different > ways to end a "statement": comma, full stop, semicolon or nothing. Well, we have expressions. Look on the bright side you don't have to wrap with unnecessary begin ... end blocks and have needless connecting words like 'else' spread everywhere like you do in less well-designed languages. If you really need a more classic style syntax look at Lisp Flavoured Erlang or Effene (albeit different classic). Robert From daniel.goertzen@REDACTED Wed Aug 11 16:45:58 2010 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 11 Aug 2010 09:45:58 -0500 Subject: [erlang-questions] open_port() on FD 0,1 problems In-Reply-To: References: Message-ID: Snooping through the R14A source code shows that fd_driver on Windows is hardcoded to perform CR translation. Boo! Will post details to the bug list and maybe try for a patch if I can get Erlang compiling on Windows. Dan. On Tue, Aug 10, 2010 at 11:44 AM, Daniel Goertzen wrote: > Thanks! That helps a lot. > > New issue is that erl.exe is translating incoming 0x0d bytes to 0x0a. > Windows line-end conversions, grumble, grumble. Is there a nice way to > turn that off? Or am I at a dead-end? > > Thanks, > Dan. > > > On Tue, Aug 10, 2010 at 9:05 AM, Hynek Vychodil wrote: > >> I don't know how on Windows but try -noinput parameter. >> >> On Tue, Aug 10, 2010 at 3:39 PM, Daniel Goertzen >> wrote: >> > I am trying to operate an Erlang Port for stdin and stdout >> > [open_port({fd,0,1}...]. The program below is to receive two integers, >> > multiply them, and write the result. Every other request vanishes >> without a >> > reply, and I don't know why. Details: >> > >> > - Windows 7 64 bit. >> > - Erlang launched with "-noshell -s erlang_rpc_benchmark" >> > - I have to wait a moment after launching erlang before sending it data, >> or >> > I definitely get dataloss. >> > - Exactly every other packet gets lost, starting with the first one. >> > - The byte counters on Task Manager show that the request is getting to >> > erl.exe, but not to my code. >> > - I've added debug code to verify that there are no other messages >> appearing >> > at my receive expression (not currently in code below). >> > - I wrote a Python version of this program, and it worked without issue. >> > >> > >> > Background: I plan to write a Python GUI program with an Erlang child >> > process. The test program below is to assess Python->Erlang messaging >> > performance. I am open to other IPC approaches if there is something >> > better. >> > >> > Thanks, >> > Dan. >> > >> > >> > -module(erlang_rpc_benchmark). >> > -export([start/0, std_server/0]). >> > >> > start() -> >> > spawn(fun std_server/0). >> > >> > std_server() -> >> > Port = open_port( {fd,0,1}, [{packet, 4}, binary ] ), >> > loop(Port). >> > >> > loop(Port) -> >> > receive >> > {Port, { data, <> }} -> >> > Result = A*B, >> > Port ! {self(), {command, <>}}; >> > end, >> > loop(Port). >> > >> >> >> >> -- >> --Hynek (Pichi) Vychodil >> >> Analyze your data in minutes. Share your insights instantly. Thrill >> your boss. Be a data hero! >> Try GoodData now for free: www.gooddata.com >> > > > > -- > Daniel Goertzen > ----------------- > dang@REDACTED (work) > daniel.goertzen@REDACTED (home) > ----------------- > 1 204 272 6149 (home/office) > 1 204 470 8360 (mobile) > ----------------- > > > > -- Daniel Goertzen ----------------- dang@REDACTED (work) daniel.goertzen@REDACTED (home) ----------------- 1 204 272 6149 (home/office) 1 204 470 8360 (mobile) ----------------- From hynek@REDACTED Wed Aug 11 17:57:30 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Wed, 11 Aug 2010 17:57:30 +0200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: <611396.69573.qm@web120515.mail.ne1.yahoo.com> References: <611396.69573.qm@web120515.mail.ne1.yahoo.com> Message-ID: On Wed, Aug 11, 2010 at 1:04 PM, Ed Keith wrote: > --- On Wed, 8/11/10, Hynek Vychodil wrote: > >> From: Hynek Vychodil >> Subject: Re: [erlang-questions] OOP in Erlang >> To: "Guy Wiener" >> Cc: "Erlang Questions" >> Date: Wednesday, August 11, 2010, 4:37 AM >> On Wed, Aug 11, 2010 at 9:44 AM, Guy >> Wiener >> wrote: >> > I have to stand up for OOP at this point: OOP does not >> start and end with >> > "encapsulating code and data". The OOP feature that >> has the strongest impact >> > on code brevity is *inheritance*. Thus, the interest >> in extending modules. >> >> I can't agree with you. Inheritance is only way how some >> OOP language >> copes with complexity of OO systems. Inheritance is neither >> core nor >> mandatory feature of OOP. What worse, many of OO affected >> people what >> I met stated that inheritance is most controversial and >> mostly misused >> feature of OOP. I also observe that there is positive >> correlation with >> person experience and probability that will tell or agree >> with above >> statement. >> >> > >> > Also, IMHO, parameterized modules are a more compact >> way to encapsulate code >> > and data, since that you don't have to pass the data >> structure around as an >> > explicit state (assuming, of course, that the data is >> immutable). >> > >> > Guy >> > >> > >> > On Wed, Aug 11, 2010 at 5:17 AM, Ngoc Dao >> wrote: >> > >> >> The mostly used feature of OOP is probably >> encapsulating code and >> >> data. Have a look at gen_server: >> >> http://www.erlang.org/doc/man/gen_server.html >> >> >> >> > > > When I first studied OOP, in 1990, I was taught that OO was defined by four features: Encapsulation, Abstraction, Inheritance, and Polymorphism. That if any of these were missing it was not OO. Same to me. It was in all books what I read in 1990s and it is still in many books. But I learnt that not all what they taught me and not all what is written in books is true. http://en.wikipedia.org/wiki/Object-oriented_programming#History Starts from 1960 through 1970s when "The Smalltalk language, which was developed at Xerox PARC (by Alan Kay and others) in the 1970s, introduced the term object-oriented programming to represent the pervasive use of objects and messages as the basis for computation." Smalltalk contains inheritance but Alan Kay doesn't state that inheritance is key feature of OOP. Especially in these days he is telling something different. (For example: http://www.google.co.uk/search?q=alan+kay+message+passing) > > ? ? ?-EdK > > Ed Keith > e_d_k@REDACTED > > Blog: edkeith.blogspot.com > > > > > > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From fritchie@REDACTED Wed Aug 11 18:35:10 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 11 Aug 2010 11:35:10 -0500 Subject: [erlang-questions] OOP in Erlang In-Reply-To: Message of "Wed, 11 Aug 2010 14:44:47 +0200." Message-ID: <18654.1281544510@snookles.snookles.com> Jesper Louis Andersen wrote: jls> [...] One problem of jls> Erlangs memory model is that sending a large datastructure as a jls> capability to another process, several megabytes in size, will mean jls> a copy. If I can add a pair of small but important points ... First, the *semantics* require a copy. jls> In the default VM setup that is. Second, the implementation is free to do whatever it wishes as long as it preserves the semantics. Depending on how new you are to the Erlang world, you may not know about the hybrid heap VM, which can avoid the copy-between-processes behavior you mention above. It is available, if you specifically build for it ("./configure --enable-hybrid-heap") and then start a VM using "erl -hybrid"(*). To start hunting for research papers, ask a Web-search-gizmo about "erlang hybrid heap" and/or "erlang shared heap". Drifting farther afield, the Erjang implementation of Erlang has some impressive speed gains by avoiding inter-process copying. -Scott (*) Odd, the "-hybrid" flag is no longer mentioned in the R13B04 "erl" reference guide ... except an oblique mention vis a vis the "-smp" flag. The hybrid heap isn't fully supported by Ericsson right now, IIRC, so I'm not very surprised. From jesper.louis.andersen@REDACTED Wed Aug 11 18:44:07 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 11 Aug 2010 18:44:07 +0200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: <18654.1281544510@snookles.snookles.com> References: <18654.1281544510@snookles.snookles.com> Message-ID: On Wed, Aug 11, 2010 at 6:35 PM, Scott Lystig Fritchie wrote: > Jesper Louis Andersen wrote: > Second, the implementation is free to do whatever it wishes as long as > it preserves the semantics. ?Depending on how new you are to the Erlang > world, you may not know about the hybrid heap VM, which can avoid the > copy-between-processes behavior you mention above. It was exactly for the reason of the Hybrid-heap VM I worded my sentence as I did. But I think I would prefer the current VM simply on the basis that modern CPU architectures probably has to let go of cache-coherency. One large heap without cache coherency sounds like trouble. -- J. From emile@REDACTED Wed Aug 11 18:59:41 2010 From: emile@REDACTED (Emile Joubert) Date: Wed, 11 Aug 2010 17:59:41 +0100 Subject: enforcing ssl trust chain Message-ID: <4C62D6FD.4010208@rabbitmq.com> Hi, I read in the latest ssl documentation and SSL 3.10.3 release notes that an unknown CA is not considered a validation error. What is the motivation for this default? In a production environment I want to prevent clients without certificates signed by a known CA from connecting. Is there any way of getting this behaviour by using configuration files? The only way I can find is to set verify_fun to an appropriate function, but this is unappealing because I want to change my mind without needing to recompile. Thanks Emile From raould@REDACTED Wed Aug 11 19:56:22 2010 From: raould@REDACTED (Raoul Duke) Date: Wed, 11 Aug 2010 10:56:22 -0700 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: <4C627A5F.9060802@erlang-solutions.com> Message-ID: On Wed, Aug 11, 2010 at 5:08 AM, Attila Rajmund Nohl wrote: > But it's nothing compared to the headache caused by the four different > ways to end a "statement": comma, full stop, semicolon or nothing. lfe? reia? From tony.arcieri@REDACTED Wed Aug 11 19:59:53 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Wed, 11 Aug 2010 11:59:53 -0600 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: Message-ID: On Wed, Aug 11, 2010 at 2:37 AM, Hynek Vychodil wrote: > I can't agree with you. Inheritance is only way how some OOP language > copes with complexity of OO systems. Inheritance is neither core nor > mandatory feature of OOP. I would say some type of polymorphism is a mandatory requirement of OOP. Inheritance is one way to provide polymorphism, but certainly not the only one. -- Tony Arcieri Medioh! A Kudelski Brand From info@REDACTED Wed Aug 11 19:44:30 2010 From: info@REDACTED (info) Date: Wed, 11 Aug 2010 19:44:30 +0200 Subject: decoding nmea messages Message-ID: <201008111944290468939@its3.ch> Hi All, According to your experience, what is the best method for decoding an NMEA 0183 message ? I mean: - extract each field, - detect those which are missing, - check syntax I see binary module or pattern matching. Other methods ? examples ? Thank's John From max.lapshin@REDACTED Wed Aug 11 20:06:34 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 11 Aug 2010 22:06:34 +0400 Subject: [erlang-questions] decoding nmea messages In-Reply-To: <201008111944290468939@its3.ch> References: <201008111944290468939@its3.ch> Message-ID: I've written even a Ragel parser for NMEA and advise you to use regexps. From raould@REDACTED Wed Aug 11 20:07:09 2010 From: raould@REDACTED (Raoul Duke) Date: Wed, 11 Aug 2010 11:07:09 -0700 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: Message-ID: On Wed, Aug 11, 2010 at 10:59 AM, Tony Arcieri wrote: > I would say some type of polymorphism is a mandatory requirement of OOP. > Inheritance is one way to provide polymorphism, but certainly not the only > one. there are quite a lot of different kinds of polymorphism. not true that e.g. a language with only parametric polymorphism is guaranteed OO (cf. haskell), so i suspect you have to narrow it down a little. maybe something more like "OO requires messaging, and different objects can react differently to the same message" (cf. typeclasses)? which isn't anything about inheritance. sincerely. From max.lapshin@REDACTED Wed Aug 11 22:53:20 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 12 Aug 2010 00:53:20 +0400 Subject: https request with checking peer Message-ID: Hi! I need to make https requests to https server and I need to check validity of remote server. When I type https://.. in browser, it takes somewhere public root certificates and checkes server signature. How should I do the same in erlang to verify remote server? > application:start(crypto). ok > application:start(ssl). {error,{not_started,public_key}} Should I carry that server's public key with me, or I need to do something else. I would be very pleased, if someone points me where to read. Erlang SSL guide haven't helped me =( From info@REDACTED Thu Aug 12 01:08:48 2010 From: info@REDACTED (info) Date: Thu, 12 Aug 2010 00:08:48 +0100 Subject: [erlang-questions] decoding nmea messages References: <201008111944290468939@its3.ch>, Message-ID: <201008120008469062601@its3.ch> Hello Max, What do you mean ? I should write a Ragel compiler with erlang ? or erlang + regexps = Ragel ? John I've written even a Ragel parser for NMEA and advise you to use regexps. ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From steven.charles.davis@REDACTED Thu Aug 12 01:37:15 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 11 Aug 2010 16:37:15 -0700 (PDT) Subject: https request with checking peer In-Reply-To: References: Message-ID: <0830ce17-29ef-406d-8446-5f91117c4461@p7g2000yqa.googlegroups.com> Hi Max, It seems that ssl:start() isn't synonymous with application:start(ssl). You can use ssl:start() or, doing it manually, the public_key application needs to be started up as well as crypto. application:start(crypto). application:start(public_key). application:start(ssl). hth /s On Aug 11, 3:53?pm, Max Lapshin wrote: > Hi! > > I need to make https requests to https server and I need to check > validity of remote server. > When I type https://.. in browser, it takes somewhere public root > certificates and checkes server signature. > How should I do the same in erlang to verify remote server? > > > application:start(crypto). > ok > > application:start(ssl). > > {error,{not_started,public_key}} > > Should I carry that server's public key with me, or I need to do > something else. > > I would be very pleased, if someone points me where to read. Erlang > SSL guide haven't helped me =( > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED From james.aimonetti@REDACTED Thu Aug 12 03:04:03 2010 From: james.aimonetti@REDACTED (James Aimonetti) Date: Wed, 11 Aug 2010 18:04:03 -0700 Subject: CNodes, FQDNs, and -name Message-ID: How does the Erlang VM determine the FQDN when started with the -name option? In my bash shell, hostname and hostname -f both return the shortname of my laptop. I'm using the mod_erlang_event module for freeSWITCH and even though I've set using sname to false, the module registers the shortname (I think it uses the hostname call internally to figure out what the node will be named) and my Erlang VM cannot communicate with it. I read about inet_res (http://erldocs.com/R14A/kernel/inet_res.html) and it looks like the VM might read from /etc/resolv.conf? I do have "domain mydomain.org" in that file so its not surprising that the VM picks it up (if that's even the mechanism by which the VM figures out the domain). So, in the mod_erlang_event C code, how would I be able to mimic the way the VM has figured out the longname of my laptop? (I do not have a DNS entry for the laptop at this domain). I read http://www.erlang.org/doc/tutorial/cnode.html and it looked like the examples had the long name hardcoded into the C ei_connect_xinit(). Sorry if this doesn't make sense; this is my first foray into interacting with a Cnode. James From ok@REDACTED Thu Aug 12 04:49:56 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 12 Aug 2010 14:49:56 +1200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: Message-ID: <39CACCD6-62B2-48B8-A03C-24621E14A6E5@cs.otago.ac.nz> On Aug 11, 2010, at 7:44 PM, Guy Wiener wrote: > I have to stand up for OOP at this point: OOP does not start and end with > "encapsulating code and data". The OOP feature that has the strongest impact > on code brevity is *inheritance*. Thus, the interest in extending modules. I was recently reading a collection of essays by Kent Beck, one of the big names in the Smalltalk and Patterns worlds. Oddly enough, he said that he didn't think inheritance was all that big a deal. Having worked on a Smalltalk system for several years, I've found that every time a new subclass is created you have to revise up and down the inheritance hierarchy to make sure that everything still makes sense. The more goodies you put in the parent class, the harder it is to make a child class that gets them all exactly right. That is, if you *bother*. I love the Smalltalk language, but one thing that bugs me about all the actual Smalltalk systems (other than my own, of course) that I've tried is the high frequency of stuff in superclasses that does NOT in fact work with subclasses. (What bugs me about _my_ Smalltalk is the effort it takes to make sure that doesn't happen.) Here's the example that currently annoys me. (0) Squeak 4.1 has a #closed method in Stream, that you can send to a stream to find out whether it needs closing or not. Some of the streams in VisualWorks 7.5 non-commercial also have this. (1) ('abc' readStream) close; closed returns false in Squeak. Now the way I read the ANSI Smalltalk standard, closing a stream is supposed to release all the resources it is holding onto. A ReadStream is holding onto the collection it is streaming over, so closing it ought to release that. In fact #closed is giving the right answer here, because #close does NOT close a ReadStream. It's worse than that: when I tried to fix this by defining #close suitably for ReadStream, a core part of the system broke, because it relied on #close *NOT* closing certain derived streams! (2) In VisualWorks it just raises an exception because file streams do have #closed but data structure streams don't. (3) I thought that #closed sounded like a neat idea and decided to add it to my system. This turned out to require the careful reinspection of some 75 classes, and the addition of some new ones. On the whole, I'm pleased that I'm doing this (I've done all the input streams and am working on the output ones). Previously I had taken far too much advantage of the fact that ANSI says sending anything to a closed stream has "undefined" effects. Now either something sensible will happen or you will get a defined exception. Adding more test cases did no harm either. The problem is that inheritance per se is easy. Making sure that all inherited code works in all places is hard. Conclusion: inheritance is NOT an unmixed blessing. It can, in fact, create maintenance nightmares. Then there's the Standard Template Library in C++. Alexander Stepanov is on record as not having found inheritance of much use for that. The book about programming he published recently makes much use of "types belonging to CONCEPTS" but none of types belonging to inheritance hierarchies. Then of course there's CORBA, with inheritance of _interfaces_, but no inheritance of code. There's the interesting distinction between Haskell type classes (which do provide 'default implementations' that you can inherit) and the otherwise identical Clean type classes (which don't). Oh yes, one other thing. In developing stream classes, inheritance in Smalltalk is less help than you might expect. For example, ReadWriteStream must provide all the methods that ReadStream and WriteStream do, but can inherit from only one of them. And I have a special case of that, StringReadWriteStream, which ought to inherit from String as well, but can't. There are several other examples of this kind. I have four copies of some methods. Until my compiler supports Traits, I've considered using file inclusion to handle such duplicate methods. Erlang _has_ file inclusion, so we can use that. In a language with hot loading, it's not completely clear what module extension MEANS. Suppose we have module P module C extends P load C Does P also get loaded, or is it just the compiler that's aware of it? change P load P Should C change? What if some of the functions of P have been inlined in C? > Also, IMHO, parameterized modules are a more compact way to encapsulate code > and data, since that you don't have to pass the data structure around as an > explicit state (assuming, of course, that the data is immutable). When I called myself an ML programmer, I *had* modules which took values, types, and modules as parameters (called 'functors'). When I switched to Haskell, I found that I never missed them. In fact with Erlang parameterised modules you DO have to pass around the explicit state. As noted before in this mailing list, it's just as easy, if not easier, to put functions inside a data structure. Parameterised modules solve a problem that Erlang did not actually have. From ok@REDACTED Thu Aug 12 05:21:36 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 12 Aug 2010 15:21:36 +1200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: <611396.69573.qm@web120515.mail.ne1.yahoo.com> References: <611396.69573.qm@web120515.mail.ne1.yahoo.com> Message-ID: <697AAA9C-6D58-4383-80E4-74F698E0FAD4@cs.otago.ac.nz> On Aug 11, 2010, at 11:04 PM, Ed Keith wrote: > > > When I first studied OOP, in 1990, I was taught that OO was defined by four features: Encapsulation, Abstraction, Inheritance, and Polymorphism. That if any of these were missing it was not OO. And other teachers taught other students different slogans. I was taught that the defining characteristics of an object were "identity, state, and behaviour" and if a language had objects it was an OO language. This is arguing over terminology; I hope we can move past that to argue about semantics. It fundamentally doesn't MATTER whether something is CALLED Object-Oriented or not. Take Ada 95, for example. It has encapsulation (associated with modules, not classes), abstraction (associated with generics, not classes), inheritance (associated with classes), and two kinds of polymorphism (parametric polymorphism via generics and inheritance polymorphism via classes). Does the fact that classes per se are NOT encapsulated mean that it isn't OO? Ada encapsulation is actually pretty good, and not only can be used to encapsulate class information, but normally is. But it's not OBJECT-based encapsulation. Oh yes, I lied when I said "classes". Ada "methods" are defined *outside* the hierarchically structured types, not inside. Doesn't matter. Who cares what we call it? If you want to program in an OO style, Ada 95 is perfectly adequate for the job. If any language is an OO language, surely Smalltalk 80 is. In normal use, it's well encapsulated, but thanks to its introspection facilities, that encapsulation amounts to "as long as nobody really REALLY wants to poke their finger through the tissue paper." Anyone who said *the* classic OO language wasn't really OO would be laughed at. For that matter, anyone who pointed out that because you can cast any pointer to char* and thus scribble on anything, C++ doesn't really have any *trustworthy* encapsulation to speak of, C++ cannot be OO either, would get some strange and some hostile looks. I don't suppose Brad Cox would be too pleased to be told that Objective C wasn't really OO either. The question for Erlang is not "OOP in Erlang", but "what ideas can we steal from _any_ other source that would be useful _for Erlang_." And for any such idea, "what exactly are we proposing that Erlang should salvage from the other language, in what form, and why do we need it?" From ok@REDACTED Thu Aug 12 05:29:38 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 12 Aug 2010 15:29:38 +1200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: <4C627A5F.9060802@erlang-solutions.com> Message-ID: <15D714BA-EFA7-4D3D-B124-F26091D097C8@cs.otago.ac.nz> On Aug 12, 2010, at 12:08 AM, Attila Rajmund Nohl wrote: > But it's nothing compared to the headache caused by the four different > ways to end a "statement": comma, full stop, semicolon or nothing. I'm a little puzzled by this. Allowing expressions to be called statements, Erlang doesn't have *ANY* way to "end" them. (Of course every kind of opening bracket has a corresponding closing bracket, but I'm not counting ")" as a way to end a "statement".) Erlang uses the full stop to terminate top level forms. That is the ONLY way they can end. They are the ONLY things so terminated. Commas and semicolons are not terminators, they are separators. Semicolon is the "else" that someone recently said Erlang didn't have. Much confusion can be saved by putting semicolons at the beginning of lines, making it clear that ";" is no more of a statement terminator in Erlang than "else" can be called a statement terminator in Pascal. From tony.arcieri@REDACTED Thu Aug 12 05:35:02 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Wed, 11 Aug 2010 21:35:02 -0600 Subject: [erlang-questions] OOP in Erlang In-Reply-To: <39CACCD6-62B2-48B8-A03C-24621E14A6E5@cs.otago.ac.nz> References: <39CACCD6-62B2-48B8-A03C-24621E14A6E5@cs.otago.ac.nz> Message-ID: On Wed, Aug 11, 2010 at 8:49 PM, Richard O'Keefe wrote: > I was recently reading a collection of essays by Kent Beck, one of the > big names in the Smalltalk and Patterns worlds. Oddly enough, he said > that he didn't think inheritance was all that big a deal. I must respectfully disagree here, especially in an everything-is-an-onject language like Smalltalk. With everything-is-an-object and inheritance, all of your objects speak a common object protocol, which can provide all sorts of things to the end user by default for every object, including powerful introspection methods which let you peruse and probe a particular object's state at runtime. Is this possible without inheritance? Sure. But I find it to be particularly powerful and elegant in languages like Ruby. -- Tony Arcieri Medioh! A Kudelski Brand From ok@REDACTED Thu Aug 12 05:47:15 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 12 Aug 2010 15:47:15 +1200 Subject: [erlang-questions] decoding nmea messages In-Reply-To: <201008111944290468939@its3.ch> References: <201008111944290468939@its3.ch> Message-ID: <9B195EEE-4A6E-442B-8E6B-854D47A4E448@cs.otago.ac.nz> On Aug 12, 2010, at 5:44 AM, info wrote: > Hi All, > According to your experience, what is the best method for decoding an NMEA 0183 message ? I mean: > - extract each field, > - detect those which are missing, > - check syntax You mean things like the example in the Wikipedia? $GPAAM,A,A,0.10,N,WPTNME*32 Binary matching will let you - check for $ - extract GP (talker) - extract AAM (type) - extract the rest (because you know how long the binary is) - check for * - extract the two hex digits - check for "\r\n" As for "the rest", I haven't read the standard, so I don't know whether the fields can contain commas or not. If not, you can break "the rest" up using binary:split/[2,3]. Where you go from there depends on what the kind of message at hand expects to find. From geoffrey.biggs@REDACTED Thu Aug 12 05:51:33 2010 From: geoffrey.biggs@REDACTED (Geoffrey Biggs) Date: Thu, 12 Aug 2010 12:51:33 +0900 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: <4C627A5F.9060802@erlang-solutions.com> Message-ID: <4C636FC5.8080308@aist.go.jp> I've never really had the problems that others seem to with Erlang's punctuation. I've always thought of an Erlang function as like a sentence. Just like in sentences, you use different punctuation to separate the different parts. When you see the full stop, the sentence ends. A single line is not necessarily a single, self-contained "statement". Geoff On 11/08/10 21:08, Attila Rajmund Nohl wrote: > But it's nothing compared to the headache caused by the four different > ways to end a "statement": comma, full stop, semicolon or nothing. From mjtruog@REDACTED Thu Aug 12 06:12:54 2010 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 11 Aug 2010 21:12:54 -0700 Subject: [erlang-questions] OOP in Erlang In-Reply-To: <39CACCD6-62B2-48B8-A03C-24621E14A6E5@cs.otago.ac.nz> References: <39CACCD6-62B2-48B8-A03C-24621E14A6E5@cs.otago.ac.nz> Message-ID: <4C6374C6.2060905@gmail.com> On 08/11/2010 07:49 PM, Richard O'Keefe wrote: > On Aug 11, 2010, at 7:44 PM, Guy Wiener wrote: > > >> I have to stand up for OOP at this point: OOP does not start and end with >> "encapsulating code and data". The OOP feature that has the strongest impact >> on code brevity is *inheritance*. Thus, the interest in extending modules. >> > I was recently reading a collection of essays by Kent Beck, one of the > big names in the Smalltalk and Patterns worlds. Oddly enough, he said > that he didn't think inheritance was all that big a deal. > I agree that inheritance can be more trouble than it is worth in large systems. A good example is the Microsoft Foundation Classes which were replaced by .NET (i.e., if MFC was great, it would not need to be rewritten). Has-A relationships are much easier to understand with proper scoping than Is-A relationships. The most useful book I have found for designing large systems that are maintainable is "Large Scale C++" by Lakos (http://www.amazon.com/Large-Scale-Software-Design-John-Lakos/dp/0201633620). However, these types of design issues are much easier to find in C++ than in Erlang, since you can accomplish more with less source code (i.e., smaller systems when judging by a non-whitespace line count) in Erlang (and Erlang does not provide inheritance, but rather has the simpler concept of behaviours). A common mistake that is easy to make in C++ is to have a small fine grained type of object that is abstract (only virtual methods) with a very large vertical inheritance hierarchy of perhaps 5 or more levels. I think such object hierarchies make the system less scalable because of the virtual function calls with extreme runtime polymorphism. Usually source code that is written like this was over-engineered and only exists to waste both development-time and run-time. Erlang would not benefit from the added complexity of OOP inheritance. From mjtruog@REDACTED Thu Aug 12 06:17:36 2010 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 11 Aug 2010 21:17:36 -0700 Subject: [erlang-questions] OOP in Erlang In-Reply-To: <4C636FC5.8080308@aist.go.jp> References: <4C627A5F.9060802@erlang-solutions.com> <4C636FC5.8080308@aist.go.jp> Message-ID: <4C6375E0.80605@gmail.com> I also really like the Erlang syntax with commas and semicolons because it is more of a logical language with the comma as an "and" and the semicolon as an "or" (like in Prolog). Enforcing a type of indentation like Python, might help create more order within the source code. However, the Erlang language itself already contains much logical order as a functional language. On 08/11/2010 08:51 PM, Geoffrey Biggs wrote: > I've never really had the problems that others seem to with Erlang's > punctuation. I've always thought of an Erlang function as like a > sentence. Just like in sentences, you use different punctuation to > separate the different parts. When you see the full stop, the sentence > ends. A single line is not necessarily a single, self-contained "statement". > > Geoff > > On 11/08/10 21:08, Attila Rajmund Nohl wrote: > >> But it's nothing compared to the headache caused by the four different >> ways to end a "statement": comma, full stop, semicolon or nothing. >> > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From ok@REDACTED Thu Aug 12 08:09:14 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 12 Aug 2010 18:09:14 +1200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: <39CACCD6-62B2-48B8-A03C-24621E14A6E5@cs.otago.ac.nz> Message-ID: <9EBC7ECC-57DD-411E-915C-B3AE39F84FBE@cs.otago.ac.nz> On Aug 12, 2010, at 3:35 PM, Tony Arcieri wrote: > On Wed, Aug 11, 2010 at 8:49 PM, Richard O'Keefe wrote: > >> I was recently reading a collection of essays by Kent Beck, one of the >> big names in the Smalltalk and Patterns worlds. Oddly enough, he said >> that he didn't think inheritance was all that big a deal. > > > I must respectfully disagree here, especially in an everything-is-an-onject > language like Smalltalk. It wasn't me. It was a great big Smalltalk guru saying that. > > With everything-is-an-object and inheritance, all of your objects speak a > common object protocol, No they don't. Oh sure, there are a bunch of methods in Object. VisualWorks 7.5 has 253 methods available in Object. Squeak-3.8Full has 441 methods available in Object. My own Smalltalk has 98 methods available in Object, and that's despite trying to keep the number down. (It's about 80 if you exclude private error reporting methods.) Of course, *without* inheritance, all my objects would *still* support all these methods. There is nothing in the definitions of "OO" that have been bandied around to this point that implies "everything is an object" or "has a single root". I repeat: inheritance can be a great burden. Imagine me adding a new class to Squeak 3.8. "What should my object do for each of these 441 methods?" I don't even know what most of them *are*, and when I look at them, the documentation usually isn't there so that I don't really understand what the methods do for *any* object. Take just one example, which I had never looked at before. actAsExecutor "Prepare the receiver to act as executor for any resources associated with it." self breakDependents What should that do for MY object? Hmm, what does everyone else do? Oh, only StandardFileStream overrides it, and that just calls super and then sets the name to nil. Who sends it? executor "Return an object which can act as executor for finalization of the receiver." ^self shallowCopy actAsExecutor I am beginning to get a glimmer here. If _this_ object is going to be finalised, then the message about it is going to be sent to some *other* object which has a copy of this object's internal state. Let's see what breakDependents does. breakDependents "Remove all of the receiver's dependents." self myDependents: nil Got that. Oh dear. I haven't done anything about #myDependents:. That's OK, turns out I don't need to. Looking further, #executor is called in WeakRegistry, where add: anObject "Add anObject to the receiver. Store the object as well as the associated executor." |executor| executor := anObject executor. self protected: [ valueDictionary at: anObject put: executor]. ^anObject I see. #actAsExecutor is something I have to worry about if and only if my object is ever added to a WeakRegistry. Whoops! I don't and can't know if that will ever happen, so I had better worry about it for ALL classes. I don't understand how things are supposed to work if my object has a reference to something, an executor for it is made (which holds onto that reference), and then my object replaces that reference by a different one. Won't the executor use the wrong object? Now I am thoroughly confused and worried (and happier than ever to have my own Smalltalk). Multiply this by about 100 to discover how it feels to be inherited at by something with so many methods. Now suppose you want to make something that is a subclass of OrderedCollection. Oh my: there are 719 methods. I have to make sure I don't break -any- of them. I have actually done this, and BOY was it a pain. Being inherited upon from a great height is no fun at all. The more there is to inherit, the harder it is. I'm afraid the Smalltalk tradition seems to be "check that the methods I currently want to use work and the H**L with the rest of them". Case in point: RunArray. When I came to implement RunArray, I found it difficult to implement certain methods that the ANSI standard requires sequences to have. So I wondered how other Smalltalks had done them. Turned out they hadn't bothered. > which can provide all sorts of things to the end > user by default for every object, It isn't inheritance that makes all sorts of things available to the user, it is conformance to common interfaces. It's wonderful for whoever gets to use the stuff. It's painful for whoever has to implement it. Amongst other things, just because you didn't override a method doesn't mean you have no need to test it. Any time you make a new class, you need to test *ALL* its methods, and if you inherited 441 of them, that's 441 methods needing several test cases each. You *might* be able to inherit some test cases, but it's practically certain you will have to write a heck of a lot of tests for things you personally aren't responsible for and wish weren't there. > including powerful introspection methods > which let you peruse and probe a particular object's state at runtime. > > Is this possible without inheritance? Sure. But I find it to be particularly > powerful and elegant in languages like Ruby. And Ruby copied practically all of it from Smalltalk. Yes, introspection is possible without inheritance. For example, my Smalltalk can find - the class of an object - the name of the class - the comment of the class (if there is one) - the methods defined in that class - the methods available in that class and can - call any of those methods and as it happens, the way this works makes no use of inheritance. ALL of the key methods are in Object or Class and are not overridden. It so happens that I don't provide X the names of the instance variables X access to the instance variables of another object but an object can - access its own instance variables in an array-like way so that - the persistence machinery can do this. Persistence *does* make use of inheritance, but if inheritance had not been available it could have been done another way. I'm not saying that inheritance is always a bad thing. Just that it has fairly massive costs as well as benefits, so that it isn't always an overall win. From max.lapshin@REDACTED Thu Aug 12 08:31:00 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 12 Aug 2010 10:31:00 +0400 Subject: [erlang-questions] Re: https request with checking peer In-Reply-To: <0830ce17-29ef-406d-8446-5f91117c4461@p7g2000yqa.googlegroups.com> References: <0830ce17-29ef-406d-8446-5f91117c4461@p7g2000yqa.googlegroups.com> Message-ID: But if I do so: ssl:start(). inets:start(). httpc:request(get, {"https://zzz.com", []}, [{ssl,[{verify,1}]}], []). I get response, while I want to get error: "untrusted certificate" Is there any simple way automatically check peer certificate against some root cert? From mikma264@REDACTED Thu Aug 12 08:08:15 2010 From: mikma264@REDACTED (Mikael Magnusson) Date: Thu, 12 Aug 2010 08:08:15 +0200 Subject: [erlang-questions] CNodes, FQDNs, and -name In-Reply-To: References: Message-ID: <1281593295.5139.620.camel@fowley.hem.za.org> On Wed, 2010-08-11 at 18:04 -0700, James Aimonetti wrote: > How does the Erlang VM determine the FQDN when started with the -name > option? In my bash shell, hostname and hostname -f both return the > shortname of my laptop. I'm using the mod_erlang_event module for > freeSWITCH and even though I've set using sname to false, the module > registers the shortname (I think it uses the hostname call internally > to figure out what the node will be named) and my Erlang VM cannot > communicate with it. > > I read about inet_res (http://erldocs.com/R14A/kernel/inet_res.html) > and it looks like the VM might read from /etc/resolv.conf? I do have > "domain mydomain.org" in that file so its not surprising that the VM > picks it up (if that's even the mechanism by which the VM figures out > the domain). So, in the mod_erlang_event C code, how would I be able > to mimic the way the VM has figured out the longname of my laptop? (I > do not have a DNS entry for the laptop at this domain). > You should make sure that the first name listed for your IP address in /etc/hosts is the FQDN. That should fix the problem with hostname -f not giving you the FQDN. /Mikael From dbarker@REDACTED Thu Aug 12 09:25:13 2010 From: dbarker@REDACTED (Deryk Barker) Date: Thu, 12 Aug 2010 00:25:13 -0700 Subject: [erlang-questions] OOP in Erlang In-Reply-To: <4C636FC5.8080308@aist.go.jp> References: <4C627A5F.9060802@erlang-solutions.com> <4C636FC5.8080308@aist.go.jp> Message-ID: <4C63A1D9.1000300@camosun.bc.ca> Geoffrey Biggs wrote: > I've never really had the problems that others seem to with Erlang's > punctuation. I've always thought of an Erlang function as like a > sentence. Just like in sentences, you use different punctuation to > separate the different parts. When you see the full stop, the sentence > ends. A single line is not necessarily a single, self-contained "statement". > Hear, hear! I've always found (well-laid out) erlang code to be particularly elegant. deryk From schramm.ingo@REDACTED Thu Aug 12 09:38:13 2010 From: schramm.ingo@REDACTED (ingo.schramm) Date: Thu, 12 Aug 2010 00:38:13 -0700 (PDT) Subject: OOP in Erlang In-Reply-To: References: Message-ID: <7e55b389-5d25-47fd-b3cf-b613dca1c680@u26g2000yqu.googlegroups.com> On Aug 11, 11:03?am, Jesper Louis Andersen wrote: > Parameterized modules make me warm and fuzzy because they seem to give > a way to implement ML-style functors (maps from modules to modules). Yes, they do. This is the real functional fun :) From michael.eugene.turner@REDACTED Thu Aug 12 09:46:26 2010 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 12 Aug 2010 16:46:26 +0900 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: <611396.69573.qm@web120515.mail.ne1.yahoo.com> Message-ID: Hynek Vychodil wrote: "Smalltalk contains inheritance but Alan Kay doesn't state that inheritance is key feature of OOP. Especially in these days he is telling something different." Not just recently but originally. http://forum.trapexit.org/viewtopic.php?p=51130&sid=bf412b1a24a7b9e777beeb2035d5f7a0 IIRC, the affirmative response immediately following was from Joe Armstrong, not from "Guest". Ah, yes, I *do* RC: http://groups.google.com/group/erlang-programming/msg/546621f6b27970bb?hl=en Possibly off-topic: can someone please tell me why TrapExit mirrors this list in a "forum" that's not even really a forum? Multi-threaded, searchable mailing list archives of the usual kind would not only be easier to use, but actually an honest representation of the origin of the discussion. I suppose the counter-argument will be that "a mailing list is a subclass of forum", but if so why did the typecast of one to another reduce Joe Armstrong to a mere guest? -michael turner On Thu, Aug 12, 2010 at 12:57 AM, Hynek Vychodil wrote: > On Wed, Aug 11, 2010 at 1:04 PM, Ed Keith wrote: > > --- On Wed, 8/11/10, Hynek Vychodil wrote: > > > >> From: Hynek Vychodil > >> Subject: Re: [erlang-questions] OOP in Erlang > >> To: "Guy Wiener" > >> Cc: "Erlang Questions" > >> Date: Wednesday, August 11, 2010, 4:37 AM > >> On Wed, Aug 11, 2010 at 9:44 AM, Guy > >> Wiener > >> wrote: > >> > I have to stand up for OOP at this point: OOP does not > >> start and end with > >> > "encapsulating code and data". The OOP feature that > >> has the strongest impact > >> > on code brevity is *inheritance*. Thus, the interest > >> in extending modules. > >> > >> I can't agree with you. Inheritance is only way how some > >> OOP language > >> copes with complexity of OO systems. Inheritance is neither > >> core nor > >> mandatory feature of OOP. What worse, many of OO affected > >> people what > >> I met stated that inheritance is most controversial and > >> mostly misused > >> feature of OOP. I also observe that there is positive > >> correlation with > >> person experience and probability that will tell or agree > >> with above > >> statement. > >> > >> > > >> > Also, IMHO, parameterized modules are a more compact > >> way to encapsulate code > >> > and data, since that you don't have to pass the data > >> structure around as an > >> > explicit state (assuming, of course, that the data is > >> immutable). > >> > > >> > Guy > >> > > >> > > >> > On Wed, Aug 11, 2010 at 5:17 AM, Ngoc Dao > >> wrote: > >> > > >> >> The mostly used feature of OOP is probably > >> encapsulating code and > >> >> data. Have a look at gen_server: > >> >> http://www.erlang.org/doc/man/gen_server.html > >> >> > >> >> > > > > > > When I first studied OOP, in 1990, I was taught that OO was defined by > four features: Encapsulation, Abstraction, Inheritance, and Polymorphism. > That if any of these were missing it was not OO. > > Same to me. It was in all books what I read in 1990s and it is still > in many books. But I learnt that not all what they taught me and not > all what is written in books is true. > > http://en.wikipedia.org/wiki/Object-oriented_programming#History > > Starts from 1960 through 1970s when "The Smalltalk language, which was > developed at Xerox PARC (by Alan Kay and others) in the 1970s, > introduced the term object-oriented programming to represent the > pervasive use of objects and messages as the basis for computation." > > Smalltalk contains inheritance but Alan Kay doesn't state that > inheritance is key feature of OOP. Especially in these days he is > telling something different. (For example: > http://www.google.co.uk/search?q=alan+kay+message+passing) > > > > > -EdK > > > > Ed Keith > > e_d_k@REDACTED > > > > Blog: edkeith.blogspot.com > > > > > > > > > > > > > > > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try GoodData now for free: www.gooddata.com > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ulf.wiger@REDACTED Thu Aug 12 10:01:28 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 12 Aug 2010 10:01:28 +0200 Subject: shared data areas (was Re: [erlang-questions] OOP in Erlang) In-Reply-To: References: <4C627A5F.9060802@erlang-solutions.com> Message-ID: <4C63AA58.3040301@erlang-solutions.com> Fred Hebert wrote: > > On Wed, Aug 11, 2010 at 8:44 AM, Jesper Louis Andersen > > wrote: > > > There is one idea here I have been toying with. One problem of Erlangs > memory model is that sending a large datastructure as a capability to > another process, several megabytes in size, will mean a copy. In the > default VM setup that is. But if you had a region into which it got > allocated, then that region could safely be sent under a proof that > the original process will not touch it anymore. [...] > > One interesting point of *always* copying data structures is that you > need to plan for small messages (as far as possible) whether you are on > a single node or in a distributed setting. Moving up from a [partially] > shared memory model to a fully isolated one when going distributed is > likely going to have its share of performance problems and might create > a dissonance between "what is acceptable locally" and "what is > acceptable when distributed". So a number of different variations on this theme have been tried in the past and discussed as future extensions: - Robert Virding used to have his own implementation called VEE. It had a global shared heap and incremental GC. - A 'shared heap' option in BEAM once had experimental status. It passed messages by reference. Note that in neither of these cases is there any change in semantics - conceptually, msg passing was still copying. The main problem with this version was that it still used the old copying garbage collector. The idea was to implement a reference-counting GC, but for various reasons, it didn't happen. When multicore started becoming interesting, the shared-heap version was left behind. - Hybrid heap was an evolution of 'shared heap', where only data sent in messages were put on a global heap. In the first implementation, data was copied to the global heap on send (unless already there) instead of being copied to the receiver's heap. This implementation was also broken by SMP. - Lately, some exploration has gone into allowing a set of processes to share the same heap. This could be done in (at least) two ways: a) either co-locate all processes in the group on the same scheduler. This would ensure mutual exclusion and mainly serve to reduce message-passing cost in a process group. b) allow processes in a group to run on different schedulers, using mutexes to protect accesses to the heap data. This could allow for parallel processing, but the locking granularity would either be heap-level or ...very subtle, I guess. I favour option (a). I think it is appropriate to use under-the-cover tricks to speed up message passing as much as possible, as long as the semantics stay the same. In other words, in all the above cases, isolation has been a given, and conceptually, messages are still copied. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From ulf.wiger@REDACTED Thu Aug 12 10:03:04 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 12 Aug 2010 10:03:04 +0200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: <4C627A5F.9060802@erlang-solutions.com> Message-ID: <4C63AAB8.3060206@erlang-solutions.com> I agree that -extend(Mod) seems like it would solve this. I have not used it myself yet and am not yet ready to have a firm opinion about whether that is the way to go. To be clear, it does address something that has occasionally bugged me, but I've not considered it a big enough problem that I've made it a priority to evaluate it. BR, Ulf W Fred Hebert wrote: > On Wed, Aug 11, 2010 at 6:24 AM, Ulf Wiger > > > wrote: > > > In short, I have not missed OO when using Erlang in its original > domain. I have, on occasion, missed being able to conveniently > extend modules and override individual functions when working > on other things (e.g. like extending mnesia). > > BR, > Ulf W > > > I believe you use mnesia:activity/4 to do this (change the callback > module), but wouldn't the -extend(Mod) module attribute solve your > problem (even in the case of mnesia?). As far as I know, it does exactly > that: override functions and leave the others to be called from 'Mod' > when they are undefined. Any reason why this wouldn't do or why you > [maybe] dislike the feature? -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From matthias@REDACTED Thu Aug 12 10:21:14 2010 From: matthias@REDACTED (Matthias Lang) Date: Thu, 12 Aug 2010 10:21:14 +0200 Subject: [erlang-questions] Closures across module reloads, undocumented behavior? In-Reply-To: References: Message-ID: <20100812082114.GA3078@corelatus.se> On Thursday, July 29, Nahuel Greco wrote: > I found an strange behavior, probably undocumented: I think this is the same bug as reported here: http://erlang.org/pipermail/erlang-bugs/2007-June/000368.html The root cause is that a fun with the same hash as an existing fun will always overwrite the existing fun on code load. I.e. the "is the hash the same?" test is too weak. I provided a patch a few years ago, but it was never incorporated. The patch itself has also been stripped from the mailing list archive. http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:32340:200801:aifmekcigbbmiidkfbgo Patch is appended again below, it was originally written for R12B, but applies cleanly to R14A and (brief check) seems to work. Does the output below from the version with the patch correspond to what you expect? Matthias ---------------------------------------------------------------------- Your test case running on the official R14A: - Unloading m old and current - Loading m__base and starting process Started a() from version 1 >From closure, version() -> 1, B -> first - Loaded m__2 , differences with base: Increments the version - Continuing process >From closure, version() -> 2, B -> first - Unloading m old and current - Loading m__base and starting process Started a() from version 1 >From closure, version() -> 1, B -> first - Loaded m__3 , differences with base: Increments the version, version/0 changed to another_version/0 - Continuing process >From closure, version() -> 1, B -> first - Unloading m old and current - Loading m__base and starting process Started a() from version 1 >From closure, version() -> 1, B -> first - Loaded m__4 , differences with base: Increments the version, changed text literal in closure - Continuing process >From closure, version() -> 1, B -> first - Unloading m old and current - Loading m__base and starting process Started a() from version 1 >From closure, version() -> 1, B -> first - Loaded m__5 , differences with base: Increments the version, changes B assigned atom to 'second' - Continuing process >From closure, version() -> 2, B -> second [undef,undef,undef,undef] ---------------------------------------------------------------------- Your test case running on R14A with my patch - Unloading m old and current - Loading m__base and starting process Started a() from version 1 >From closure, version() -> 1, B -> first - Loaded m__2 , differences with base: Increments the version - Continuing process >From closure, version() -> 1, B -> first - Unloading m old and current - Loading m__base and starting process Started a() from version 1 >From closure, version() -> 1, B -> first - Loaded m__3 , differences with base: Increments the version, version/0 changed to another_version/0 - Continuing process >From closure, version() -> 1, B -> first - Unloading m old and current - Loading m__base and starting process Started a() from version 1 >From closure, version() -> 1, B -> first - Loaded m__4 , differences with base: Increments the version, changed text literal in closure - Continuing process >From closure, version() -> 1, B -> first - Unloading m old and current - Loading m__base and starting process Started a() from version 1 >From closure, version() -> 1, B -> first - Loaded m__5 , differences with base: Increments the version, changes B assigned atom to 'second' - Continuing process >From closure, version() -> 1, B -> first [undef,undef,undef,undef] ---------------------------------------------------------------------- The patch: --- erl_fun.c.orig 2008-01-17 00:03:10.000000000 +0100 +++ erl_fun.c 2008-01-17 00:23:23.000000000 +0100 @@ -97,6 +97,7 @@ long refc; ASSERT(is_atom(mod)); template.old_uniq = uniq; + sys_memset(template.uniq, 0, sizeof(template.uniq)); template.old_index = index; template.module = mod; erts_fun_write_lock(); @@ -120,6 +121,7 @@ ASSERT(is_atom(mod)); template.old_uniq = old_uniq; + sys_memcpy(template.uniq, uniq, sizeof(template.uniq)); template.old_index = old_index; template.module = mod; erts_fun_write_lock(); @@ -279,7 +281,10 @@ static HashValue fun_hash(ErlFunEntry* obj) { - return (HashValue) (obj->old_uniq ^ obj->old_index ^ atom_val(obj->module)); + return (HashValue) (obj->old_uniq ^ + *(int*)obj->uniq ^ + obj->old_index ^ + atom_val(obj->module)); } static int @@ -287,6 +292,7 @@ { return !(obj1->module == obj2->module && obj1->old_uniq == obj2->old_uniq && + sys_memcmp(obj1->uniq, obj2->uniq, sizeof(obj1->uniq)) == 0 && obj1->old_index == obj2->old_index); } @@ -297,6 +303,7 @@ sizeof(ErlFunEntry)); obj->old_uniq = template->old_uniq; + sys_memcpy(obj->uniq, template->uniq, sizeof(template->uniq)); obj->old_index = template->old_index; obj->module = template->module; erts_refc_init(&obj->refc, -1); From mazen.harake@REDACTED Thu Aug 12 10:31:58 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Thu, 12 Aug 2010 11:31:58 +0300 Subject: Best way to kill an async thread in a linked in driver? Message-ID: <4C63B17E.5060901@erlang-solutions.com> It was late yesterday and despite 2 coffees and a nice breakfast I still can't solve this one. Maybe a nudge in the right direction anyone? Scenario: I have a linked in driver; cecho [1], when ever I want to read input I disable input to Erlang (-noinput) and use the getch() function in C instead (never mind why ;)). This function is run in a seperate async thread using driver_async() [2][3], and blocks at getch(). When a character is read it sends a message back to Erlang (driver_output()) and then the thread exits, when the message comes back to Erlang the character is sent to the requester and the thread is restarted (which again blocks at getch()). Later if I do an erlang:halt(0) ("graceful halt") the thread is still lingering despite having closed the port[4] (which, Afaik should close the thread as well) but it seems stay alive somehow. erlang:halt(1) kill it immediately so it is not a problem. Questions: 1) Is my "analysis" correct? Is this infact what is happening? Anyone who can explain this behaviour (I.e. why it lingers on)? 2) If Question 1 then; what is the best way to kill the thread when I exit? driver_async_cancel() only works for threads that are not executing (according to Documentation) so what to do? I did an ugly workaround which sets a global flag and then I timeout the getch() operation and read for ERR. If I get an ERR I just read the abort flag and if it is set I just die otherwise I try to getch() again. This avoids hanging around for more then a tenth of a second but it is pretty ugly in my oppinion and I'm forced to go into halfdelay mode which means I can not allow this option outside of the driver because I become dependent on it. I'd rather do it the "right way" if there is such a way. References: (Links given as commit Ids so that if someone searches then they get relevant code and not the latest) [1] http://github.com/mazenharake/cecho/tree/199fd342ba186e1154c32dbfdcf2d9222e25f9f4 [2] http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/c_src/cecho.c#L119 static void request(ErlDrvData drvstate, char *buf, int buflen) { state *st = (state *)drvstate; driver_async(st->drv_port, NULL, loop_getch, (void *)st, NULL); } [3] http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/c_src/cecho.c#L557 void loop_getch(void *arg) { state *st = (state *)arg; ei_x_buff eixb; int keycode; ei_x_new_with_version(&eixb); keycode = getch(); integer(&eixb, keycode); driver_output(st->drv_port, eixb.buff, eixb.index); } [4] http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/src/cecho_srv.erl#L76 terminate(_Reason, State) -> do_call(State#state.port, ?ENDWIN), do_call(State#state.port, ?CURS_SET, ?ceCURS_NORMAL), erlang:port_close(State#state.port), erl_ddll:unload("cecho"). From krab@REDACTED Thu Aug 12 13:42:00 2010 From: krab@REDACTED (Kresten Krab Thorup) Date: Thu, 12 Aug 2010 13:42:00 +0200 Subject: [erlang-questions] Best way to kill an async thread in a linked in driver? In-Reply-To: <4C63B17E.5060901@erlang-solutions.com> References: <4C63B17E.5060901@erlang-solutions.com> Message-ID: I'd write the driver using select and non-blocking reads; that way you would not need async threads at all. Kresten On Aug 12, 2010, at 10:31 , Mazen Harake wrote: > It was late yesterday and despite 2 coffees and a nice breakfast I > still can't solve this one. Maybe a nudge in the right direction anyone? > > Scenario: > I have a linked in driver; cecho [1], when ever I want to read input I > disable input to Erlang (-noinput) and use the getch() function in C > instead (never mind why ;)). This function is run in a seperate async > thread using driver_async() [2][3], and blocks at getch(). When a > character is read it sends a message back to Erlang (driver_output()) > and then the thread exits, when the message comes back to Erlang the > character is sent to the requester and the thread is restarted (which > again blocks at getch()). > > Later if I do an erlang:halt(0) ("graceful halt") the thread is still > lingering despite having closed the port[4] (which, Afaik should close > the thread as well) but it seems stay alive somehow. erlang:halt(1) kill > it immediately so it is not a problem. > > Questions: > 1) Is my "analysis" correct? Is this infact what is happening? Anyone > who can explain this behaviour (I.e. why it lingers on)? > > 2) If Question 1 then; what is the best way to kill the thread when I > exit? driver_async_cancel() only works for threads that are not > executing (according to Documentation) so what to do? I did an ugly > workaround which sets a global flag and then I timeout the getch() > operation and read for ERR. If I get an ERR I just read the abort flag > and if it is set I just die otherwise I try to getch() again. This > avoids hanging around for more then a tenth of a second but it is pretty > ugly in my oppinion and I'm forced to go into halfdelay mode which means > I can not allow this option outside of the driver because I become > dependent on it. I'd rather do it the "right way" if there is such a way. > > References: (Links given as commit Ids so that if someone searches then > they get relevant code and not the latest) > [1] > http://github.com/mazenharake/cecho/tree/199fd342ba186e1154c32dbfdcf2d9222e25f9f4 > > [2] > http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/c_src/cecho.c#L119 > static void request(ErlDrvData drvstate, char *buf, int buflen) { > state *st = (state *)drvstate; > driver_async(st->drv_port, NULL, loop_getch, (void *)st, NULL); > } > > [3] > http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/c_src/cecho.c#L557 > void loop_getch(void *arg) { > state *st = (state *)arg; > ei_x_buff eixb; > int keycode; > ei_x_new_with_version(&eixb); > keycode = getch(); > integer(&eixb, keycode); > driver_output(st->drv_port, eixb.buff, eixb.index); > } > > [4] > http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/src/cecho_srv.erl#L76 > terminate(_Reason, State) -> > do_call(State#state.port, ?ENDWIN), > do_call(State#state.port, ?CURS_SET, ?ceCURS_NORMAL), > erlang:port_close(State#state.port), > erl_ddll:unload("cecho"). > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Kresten Krab Thorup, CTO, Trifork From max.lapshin@REDACTED Thu Aug 12 14:26:13 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 12 Aug 2010 16:26:13 +0400 Subject: [erlang-questions] Best way to kill an async thread in a linked in driver? In-Reply-To: References: <4C63B17E.5060901@erlang-solutions.com> Message-ID: On Thu, Aug 12, 2010 at 3:42 PM, Kresten Krab Thorup wrote: > I'd write the driver using select and non-blocking reads; that way you would not need async threads at all. > Maybe author is writing something like driver to libsqlite3 or some other code, that is non-IO and highly computable. For example, libavcodec. From mixolyde@REDACTED Thu Aug 12 14:30:15 2010 From: mixolyde@REDACTED (Brian Williams) Date: Thu, 12 Aug 2010 08:30:15 -0400 Subject: Different code layout examples Message-ID: Are there any examples out there of different code layouts that may (or may not) look better to someone confused by the punctuation? Brian On Thu, Aug 12, 2010 at 3:25 AM, Deryk Barker wrote: > Geoffrey Biggs wrote: >> >> I've never really had the problems that others seem to with Erlang's >> punctuation. I've always thought of an Erlang function as like a >> sentence. Just like in sentences, you use different punctuation to >> separate the different parts. When you see the full stop, the sentence >> ends. A single line is not necessarily a single, self-contained >> "statement". >> > > Hear, hear! > > I've always found (well-laid out) erlang code to be particularly elegant. > > deryk > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Brian E. Williams mixolyde@REDACTED http://www.techhouse.us/wordpress-mu/brianw "Never attribute to malice that which can be adequately explained by stupidity." - Hanlon's Razor From kaiduanx@REDACTED Thu Aug 12 15:51:08 2010 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Thu, 12 Aug 2010 09:51:08 -0400 Subject: Wondering on active TCP socket Message-ID: Hi, all, >From Joe's book, "Once an active socket has been created, the controlling process will be sent {tcp, Socket, Data} messages as data is received. There is no way the controlling process can control the flow of these messages. A rogue client could send thousands of messages to the system, and these would all be sent to the controlling process. The controlling process cannot stop this flow of messages." "This process cannot control the flow of messages to the server loop. If the client produces data faster than the server can consume this data, then the system can be flooded with messages?the message buffers will fill up, and the system might crash or behave strangely." For this case, I think that the flow control of underlying OS TCP stack will kick in, and the sender can not send more packets. How the system can be flooded with messages in erlang? Can someone elaborate more on this point? Thanks, Kaiduan From ngreco@REDACTED Thu Aug 12 16:00:24 2010 From: ngreco@REDACTED (Nahuel Greco) Date: Thu, 12 Aug 2010 11:00:24 -0300 Subject: [erlang-questions] Wondering on active TCP socket In-Reply-To: References: Message-ID: On Thu, Aug 12, 2010 at 10:51 AM, Kaiduan Xie wrote: > For this case, I think that the flow control of underlying OS TCP > stack will kick in, and the sender can not send more packets. How the > system can be flooded with messages in erlang? > The flow control of the TCP stack will not kick in, because the OS process (the Erlang VM) already consumed the data from the TCP incoming buffer, that consumed data will sit in the controlling process message queue, inside the VM. Saludos, Nahuel Greco. From info@REDACTED Thu Aug 12 16:19:06 2010 From: info@REDACTED (info) Date: Thu, 12 Aug 2010 16:19:06 +0200 Subject: [erlang-questions] decoding nmea messages References: <201008111944290468939@its3.ch>, <9B195EEE-4A6E-442B-8E6B-854D47A4E448@cs.otago.ac.nz> Message-ID: <201008121619055993982@its3.ch> Hi Richard, The fields don't contain commas but sometimes they are empty. Therefore the size of the messages is not a constant. And what about the syntax analysis if each field is considered as a binary ? Someone suggest to use regexp ... John On Aug 12, 2010, at 5:44 AM, info wrote: > Hi All, > According to your experience, what is the best method for decoding an NMEA 0183 message ? I mean: > - extract each field, > - detect those which are missing, > - check syntax You mean things like the example in the Wikipedia? $GPAAM,A,A,0.10,N,WPTNME*32 Binary matching will let you - check for $ - extract GP (talker) - extract AAM (type) - extract the rest (because you know how long the binary is) - check for * - extract the two hex digits - check for "\r\n" As for "the rest", I haven't read the standard, so I don't know whether the fields can contain commas or not. If not, you can break "the rest" up using binary:split/[2,3]. Where you go from there depends on what the kind of message at hand expects to find. From james.aimonetti@REDACTED Thu Aug 12 16:53:13 2010 From: james.aimonetti@REDACTED (James Aimonetti) Date: Thu, 12 Aug 2010 07:53:13 -0700 Subject: [erlang-questions] CNodes, FQDNs, and -name In-Reply-To: <1281593295.5139.620.camel@fowley.hem.za.org> References: <1281593295.5139.620.camel@fowley.hem.za.org> Message-ID: On Wed, Aug 11, 2010 at 11:08 PM, Mikael Magnusson wrote: > On Wed, 2010-08-11 at 18:04 -0700, James Aimonetti wrote: >> How does the Erlang VM determine the FQDN when started with the -name >> option? In my bash shell, hostname and hostname -f both return the >> shortname of my laptop. I'm using the mod_erlang_event module for >> freeSWITCH and even though I've set using sname to false, the module >> registers the shortname (I think it uses the hostname call internally >> to figure out what the node will be named) and my Erlang VM cannot >> communicate with it. >> >> I read about inet_res (http://erldocs.com/R14A/kernel/inet_res.html) >> and it looks like the VM might read from /etc/resolv.conf? I do have >> "domain mydomain.org" in that file so its not surprising that the VM >> picks it up (if that's even the mechanism by which the VM figures out >> the domain). So, in the mod_erlang_event C code, how would I be able >> to mimic the way the VM has figured out the longname of my laptop? (I >> do not have a DNS entry for the laptop at this domain). >> > > You should make sure that the first name listed for your IP address > in /etc/hosts is the FQDN. That should fix the problem with hostname -f > not giving you the FQDN. > > /Mikael With my laptop, the FQDN is rarely valid. When at work, the resolv.conf has the work domain listed. Now that I'm home on DSL, the resolv.conf lacks any domain line. Also, now that I'm home, I can no longer start my VM with the -name option with just a node name; the VM errors with "Can't set long node name". I don't think setting the hosts file with something helped (unless it can't be arbitrary). I can put a full node@REDACTED for the -name parameter, but as I'd like to put my applications on several boxes, I'd like it to get picked up automatically. This issue will be moot once I deploy, but its a small gotcha when doing local development that I'd like to eliminate for the CNode code. I guess I would like to update the CNode to respond more like the VM in determining the hostname for the node, but am not sure the mechanism by which the VM sets the node's name when using -sname and -name with just a node and no host. Thanks for any pointers/ideas. James From kaiduanx@REDACTED Thu Aug 12 16:58:46 2010 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Thu, 12 Aug 2010 10:58:46 -0400 Subject: [erlang-questions] Wondering on active TCP socket In-Reply-To: References: Message-ID: Nahuel, thank you for the insightful clarification. How erlang VM retrieve packet from TCP stack for active socket? Does it retrieve the packet, wrap it as a message and send to the controlling process every time underlying OS TCP stack receives a packet (here packet is a stream of bytes) from network? Where can I find more information? Which moudles/source files? Thanks, On Thu, Aug 12, 2010 at 10:00 AM, Nahuel Greco wrote: > On Thu, Aug 12, 2010 at 10:51 AM, Kaiduan Xie wrote: >> For this case, I think that the flow control of underlying OS TCP >> stack will kick in, and the sender can not send more packets. How the >> system can be flooded with messages in erlang? >> > > The flow control of the TCP stack will not kick in, because the OS > process (the Erlang VM) already consumed the data from the TCP > incoming buffer, that consumed data will sit in the controlling > process message queue, inside the VM. > > Saludos, > Nahuel Greco. > From ngreco@REDACTED Thu Aug 12 17:21:25 2010 From: ngreco@REDACTED (Nahuel Greco) Date: Thu, 12 Aug 2010 12:21:25 -0300 Subject: [erlang-questions] Wondering on active TCP socket In-Reply-To: References: Message-ID: On Thu, Aug 12, 2010 at 11:58 AM, Kaiduan Xie wrote: > Nahuel, thank you for the insightful clarification. > > How erlang VM retrieve packet from TCP stack for active socket? Does > it retrieve the packet, wrap it as a message and send to the > controlling process every time underlying OS TCP stack receives a > packet (here packet is a stream of bytes) from network? Where can I > find more information? Which moudles/source files? > It collects data from the TCP incoming buffer, and when a packet is complete wraps it in a message and sends it to the controlling process. What a packet is depends on the options you set to the socket. See the {packet, PacketType} option here: http://erldocs.com/R14A/kernel/inet.html?i=0&search=inet:setopts#setopts/2 See also the gen_tcp documentation. Saludos, Nahuel Greco. From juanjo@REDACTED Thu Aug 12 17:24:22 2010 From: juanjo@REDACTED (Juan Jose Comellas) Date: Thu, 12 Aug 2010 12:24:22 -0300 Subject: [erlang-questions] Wondering on active TCP socket In-Reply-To: References: Message-ID: BTW, using {active, once} every time the process is ready to receive data from the socket would avoid flooding the process' mailbox. On Thu, Aug 12, 2010 at 11:00 AM, Nahuel Greco wrote: > On Thu, Aug 12, 2010 at 10:51 AM, Kaiduan Xie wrote: > > For this case, I think that the flow control of underlying OS TCP > > stack will kick in, and the sender can not send more packets. How the > > system can be flooded with messages in erlang? > > > > The flow control of the TCP stack will not kick in, because the OS > process (the Erlang VM) already consumed the data from the TCP > incoming buffer, that consumed data will sit in the controlling > process message queue, inside the VM. > > Saludos, > Nahuel Greco. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From attila.r.nohl@REDACTED Thu Aug 12 21:31:05 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 12 Aug 2010 21:31:05 +0200 Subject: [erlang-questions] OOP in Erlang In-Reply-To: References: <4C627A5F.9060802@erlang-solutions.com> Message-ID: 2010/8/11, Fred Hebert : > On Wed, Aug 11, 2010 at 8:08 AM, Attila Rajmund Nohl < > attila.r.nohl@REDACTED> wrote: > >> >> But it's nothing compared to the headache caused by the four different >> ways to end a "statement": comma, full stop, semicolon or nothing. >> >> Maybe this can help? http://ferd.ca/on-erlang-s-syntax.html > I wrote this a while ago especially for such complaints, let me know what > you think. It addresses the actual problem: "unable to swap lines of code without changing the token at the end of the line". I only write new code in about 1% of my work time, the rest is about fixing bugs in existing code and refactoring. For example I have a macro in vim that generates an io:format printing the value of the variable under the cursor. It works fine when the line ends with comma, but doesn't work that fine if the variable is on the last line of a case clause or a function clause. It's not a huge deal, but a constant annoyance. From hpjcon@REDACTED Thu Aug 12 21:14:50 2010 From: hpjcon@REDACTED (Jan Jacobs) Date: Thu, 12 Aug 2010 21:14:50 +0200 Subject: YAWS, SOAP and SSL under load result in SSL socket closing with bad_info Message-ID: <4C64482A.9010008@mweb.co.za> Hi, I have an application which uses YAWS(1.88) in embedded mode with SOAP over SSL. It is using R13B04 on windows platforms (XP 32 Bit, Vista 32 Bit and Windows 2008 64 Bit). I am getting the following error: (It looks like a SSL error) =ERROR REPORT==== 4-Aug-2010::11:37:19 === ** State machine <0.7595.0> terminating ** Last message in was {tcp_error,#Port<0.17333>,econnaborted} ** When State == connection ... undefined,undefined,#Ref<0.0.0.223980>, {<0.7592.0>,#Ref<0.0.0.232900>}, 0,<<>>,true} ** Reason for termination = ** bad_info (which then kicks a yaws error) =ERROR REPORT==== 4-Aug-2010::11:37:19 === Yaws process died: {bad_info,{gen_fsm,sync_send_all_state_event, [<0.7595.0>,{recv,0},30000]}} This happens under load or when more than 300000 SOAP transaction has been processed. I then have to stop SSL and YAWS applications and start them again to accept more transactions. YAWS Config: GL = [ {enable_soap, true} ,{trace, false} ,{tmpdir, DirLog} ,{logdir, DirLog} ,{ebin_dir, [DirEbin]} ,{include_dir, DirInc} ,{max_connections, nolimit} ,{use_old_ssl, false} ,{flags, [ {auth_log, true} ,{tty_trace, true} ,{copy_errlog, false} ]} ], SL = [ {port, PortYaws} ,{servername, MyHostname} ,{dir_listings, true} ,{listen, {0,0,0,0}} ,{appmods, [{"/OpenAPI",maxman_km_service_soap}]} ,{flags, [ {access_log,true} ]} ,ssl() ], YAWS SSL Config: {ssl, [ {keyfile,Keyfile} ,{certfile,Certfile} ,{verify,0} ,{depth,0} ]} Any suggestions and/or comments will be appreciated. Thanks Jan From ngreco@REDACTED Thu Aug 12 22:14:43 2010 From: ngreco@REDACTED (Nahuel Greco) Date: Thu, 12 Aug 2010 17:14:43 -0300 Subject: [erlang-questions] Closures across module reloads, undocumented behavior? In-Reply-To: <20100812082114.GA3078@corelatus.se> References: <20100812082114.GA3078@corelatus.se> Message-ID: I think you are right, seems to be the same bug. I applied your patch to R14A and got the same results. Also, after patching, I tested if code:purge/1 kills the processes having a closure referencing the old module version, and yes, they are killed. I expected a consistent behaviour, so I think the patch is good for me :) But the desired behaviour is at least undocumented, it needs to be defined and/or documented officially. Thanks. Saludos, Nahuel Greco. On Thu, Aug 12, 2010 at 5:21 AM, Matthias Lang wrote: > On Thursday, July 29, Nahuel Greco wrote: >> I found an strange behavior, probably undocumented: > > I think this is the same bug as reported here: > > ?http://erlang.org/pipermail/erlang-bugs/2007-June/000368.html > > The root cause is that a fun with the same hash as an existing fun > will always overwrite the existing fun on code load. I.e. the "is the > hash the same?" test is too weak. > > I provided a patch a few years ago, but it was never incorporated. The > patch itself has also been stripped from the mailing list archive. > > ?http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:32340:200801:aifmekcigbbmiidkfbgo > > Patch is appended again below, it was originally written for R12B, but > applies cleanly to R14A and (brief check) seems to work. > > Does the output below from the version with the patch correspond to > what you expect? > > Matthias > > ---------------------------------------------------------------------- > Your test case running on the official R14A: > > - Unloading m old and current > - Loading m__base and starting process > Started a() from version 1 > From closure, version() -> 1, B -> first > - Loaded m__2 , differences with base: Increments the version > - Continuing process > From closure, version() -> 2, B -> first > > - Unloading m old and current > - Loading m__base and starting process > Started a() from version 1 > From closure, version() -> 1, B -> first > - Loaded m__3 , differences with base: Increments the version, version/0 changed to another_version/0 > - Continuing process > From closure, version() -> 1, B -> first > > - Unloading m old and current > - Loading m__base and starting process > Started a() from version 1 > From closure, version() -> 1, B -> first > - Loaded m__4 , differences with base: Increments the version, changed text literal in closure > - Continuing process > From closure, version() -> 1, B -> first > > - Unloading m old and current > - Loading m__base and starting process > Started a() from version 1 > From closure, version() -> 1, B -> first > - Loaded m__5 , differences with base: Increments the version, changes B assigned atom to 'second' > - Continuing process > From closure, version() -> 2, B -> second > [undef,undef,undef,undef] > > ---------------------------------------------------------------------- > Your test case running on R14A with my patch > > - Unloading m old and current > - Loading m__base and starting process > Started a() from version 1 > From closure, version() -> 1, B -> first > - Loaded m__2 , differences with base: Increments the version > - Continuing process > From closure, version() -> 1, B -> first > > - Unloading m old and current > - Loading m__base and starting process > Started a() from version 1 > From closure, version() -> 1, B -> first > - Loaded m__3 , differences with base: Increments the version, version/0 changed to another_version/0 > - Continuing process > From closure, version() -> 1, B -> first > > - Unloading m old and current > - Loading m__base and starting process > Started a() from version 1 > From closure, version() -> 1, B -> first > - Loaded m__4 , differences with base: Increments the version, changed text literal in closure > - Continuing process > From closure, version() -> 1, B -> first > > - Unloading m old and current > - Loading m__base and starting process > Started a() from version 1 > From closure, version() -> 1, B -> first > - Loaded m__5 , differences with base: Increments the version, changes B assigned atom to 'second' > - Continuing process > From closure, version() -> 1, B -> first > [undef,undef,undef,undef] > > ---------------------------------------------------------------------- > The patch: > > --- erl_fun.c.orig ? ? ?2008-01-17 00:03:10.000000000 +0100 > +++ erl_fun.c ? 2008-01-17 00:23:23.000000000 +0100 > @@ -97,6 +97,7 @@ > ? ? long refc; > ? ? ASSERT(is_atom(mod)); > ? ? template.old_uniq = uniq; > + ? ?sys_memset(template.uniq, 0, sizeof(template.uniq)); > ? ? template.old_index = index; > ? ? template.module = mod; > ? ? erts_fun_write_lock(); > @@ -120,6 +121,7 @@ > > ? ? ASSERT(is_atom(mod)); > ? ? template.old_uniq = old_uniq; > + ? ?sys_memcpy(template.uniq, uniq, sizeof(template.uniq)); > ? ? template.old_index = old_index; > ? ? template.module = mod; > ? ? erts_fun_write_lock(); > @@ -279,7 +281,10 @@ > ?static HashValue > ?fun_hash(ErlFunEntry* obj) > ?{ > - ? ?return (HashValue) (obj->old_uniq ^ obj->old_index ^ atom_val(obj->module)); > + ? ?return (HashValue) (obj->old_uniq ^ > + ? ? ? ? ? ? ? ? ? ? ? ?*(int*)obj->uniq ^ > + ? ? ? ? ? ? ? ? ? ? ? ?obj->old_index ^ > + ? ? ? ? ? ? ? ? ? ? ? ?atom_val(obj->module)); > ?} > > ?static int > @@ -287,6 +292,7 @@ > ?{ > ? ? return !(obj1->module == obj2->module && > ? ? ? ? ? ? obj1->old_uniq == obj2->old_uniq && > + ? ? ? ? ? ? sys_memcmp(obj1->uniq, obj2->uniq, sizeof(obj1->uniq)) == 0 && > ? ? ? ? ? ? obj1->old_index == obj2->old_index); > ?} > > @@ -297,6 +303,7 @@ > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?sizeof(ErlFunEntry)); > > ? ? obj->old_uniq = template->old_uniq; > + ? ?sys_memcpy(obj->uniq, template->uniq, sizeof(template->uniq)); > ? ? obj->old_index = template->old_index; > ? ? obj->module = template->module; > ? ? erts_refc_init(&obj->refc, -1); > > > From hynek@REDACTED Fri Aug 13 00:00:51 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Fri, 13 Aug 2010 00:00:51 +0200 Subject: [erlang-questions] decoding nmea messages In-Reply-To: <201008121619055993982@its3.ch> References: <201008111944290468939@its3.ch> <9B195EEE-4A6E-442B-8E6B-854D47A4E448@cs.otago.ac.nz> <201008121619055993982@its3.ch> Message-ID: Basic message parsing is very simple -module(nmea0183). -export([parse_msg/1, split/1]). split(Bin) -> [ X || <<$$,_/binary>> = X <- binary:split(Bin, <<"\r\n">>)]. parse_msg(Bin) -> [<<$$, Msg/binary>>, Crc] = binary:split(Bin, <<$*>>), IsCrc = checkcrc(Msg, Crc), <> = Msg, Values = case Rest of <<>> -> []; _ -> tl(binary:split(Rest, <<$,>>, [global])) end, {Sender, Type, Values, IsCrc}. checkcrc(_, <<>>) -> nocrc; checkcrc(Msg, <>) -> Crc = hdv(H)*16 + hdv(L), Crc = crc(Msg), crc. crc(Msg) -> lists:foldl(fun(X, A) -> X bxor A end, 0, binary_to_list(Msg)). hdv(X) when X >= $0, X =< $9 -> X band 15; hdv(X) -> 9 + (X band 15). 43> nmea0183:parse_msg(<<"$GPGSV,2,1,08,11,74,137,45,20,58,248,43,07,27,309,00,14,23,044,36*7A">>). {<<"GP">>,<<"GSV">>, [<<"2">>,<<"1">>,<<"08">>,<<"11">>,<<"74">>,<<"137">>, <<"45">>,<<"20">>,<<"58">>,<<"248">>,<<"43">>,<<"07">>, <<"27">>,<<"309">>,<<"00">>,<<"14">>,<<"23">>,<<"044">>, <<"36">>], crc} 44> nmea0183:parse_msg(<<"$GPGSA,A,3,20,11,25,01,14,31,,,,,,,2.6,1.7,1.9*3B">>). {<<"GP">>,<<"GSA">>, [<<"A">>,<<"3">>,<<"20">>,<<"11">>,<<"25">>,<<"01">>, <<"14">>,<<"31">>,<<>>,<<>>,<<>>,<<>>,<<>>,<<>>,<<"2.6">>, <<"1.7">>,<<"1.9">>], crc} 45> nmea0183:parse_msg(<<"$GPAAM,A,A,0.10,N,WPTNME*32">>)). {<<"GP">>,<<"AAM">>, [<<"A">>,<<"A">>,<<"0.10">>,<<"N">>,<<"WPTNME">>], crc} But if you want parse value of each field you have to do more work. On Thu, Aug 12, 2010 at 4:19 PM, info wrote: > Hi Richard, > The fields don't contain commas but sometimes they are empty. Therefore the size of the messages is not a constant. > And what about the syntax analysis if each field is considered as a binary ? > Someone suggest to use regexp ... > John > On Aug 12, 2010, at 5:44 AM, info wrote: > >> Hi All, >> According to your experience, what is the best method for decoding an NMEA 0183 message ? I mean: >> - extract each field, >> - detect those which are missing, >> - check syntax > > You mean things like the example in the Wikipedia? > > $GPAAM,A,A,0.10,N,WPTNME*32 > > Binary matching will let you > ?- check for $ > ?- extract GP (talker) > ?- extract AAM (type) > ?- extract the rest (because you know how long the binary is) > ?- check for * > ?- extract the two hex digits > ?- check for "\r\n" > As for "the rest", I haven't read the standard, so I don't know > whether the fields can contain commas or not. ?If not, you can > break "the rest" up using binary:split/[2,3]. ?Where you go from > there depends on what the kind of message at hand expects to find. > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From behdad.forghani@REDACTED Fri Aug 13 00:09:28 2010 From: behdad.forghani@REDACTED (Behdad Forghani) Date: Thu, 12 Aug 2010 15:09:28 -0700 Subject: Transfering large data Message-ID: <4c64711b.049e720a.21de.76fc@mx.google.com> Hi, I am writing a remote collection agent software in Erlang. I have gen_server application that reads binary data (protocol traces) and then sends the contents to another gen_server. The way I have designed it is that I process a file and write it to an output file. Then I read the output file using file:read_file and then send it to the central server with gen_server:call. The reason I chose this model is: 1 - If I send data packet by packet with a gen_server:call, the resulting "stop and go" protocol would slower than sending bigger chunk. 2 - This is an atomic one shot operation and I will not have to keep track of which collection agent is sending the file and keep a table of processes and file handles and take care of closing file handles if the process dies. My question is: 1- Shall I be worried if the file is too large. Will I run into limitation of how much data gen_server:call can embed? Is there a limit? 2- Is there a better way of doing this in Erlang. I appreciate your comments. Regards, Behdad From mazen.harake@REDACTED Fri Aug 13 00:14:27 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Fri, 13 Aug 2010 01:14:27 +0300 Subject: [erlang-questions] Best way to kill an async thread in a linked in driver? In-Reply-To: References: <4C63B17E.5060901@erlang-solutions.com> Message-ID: <4C647243.3050409@erlang-solutions.com> Hmm... I'm not sure what you mean but the part I do understand is that select() blocks... right? I can't have anything that blocks in the driver since I must be able to interact with it as I'm waiting for input. E.g. I need to update a certain part of the screen even if the user is not pressing any buttons. I don't see how select() would make any difference? Perhaps I misunderstood you, could you explain a bit more on how that would work? Or perhaps point me to some direction? Thanks for feedback, /M On 12/08/2010 14:42, Kresten Krab Thorup wrote: > I'd write the driver using select and non-blocking reads; that way you would not need async threads at all. > > Kresten > > On Aug 12, 2010, at 10:31 , Mazen Harake wrote: > >> It was late yesterday and despite 2 coffees and a nice breakfast I >> still can't solve this one. Maybe a nudge in the right direction anyone? >> >> Scenario: >> I have a linked in driver; cecho [1], when ever I want to read input I >> disable input to Erlang (-noinput) and use the getch() function in C >> instead (never mind why ;)). This function is run in a seperate async >> thread using driver_async() [2][3], and blocks at getch(). When a >> character is read it sends a message back to Erlang (driver_output()) >> and then the thread exits, when the message comes back to Erlang the >> character is sent to the requester and the thread is restarted (which >> again blocks at getch()). >> >> Later if I do an erlang:halt(0) ("graceful halt") the thread is still >> lingering despite having closed the port[4] (which, Afaik should close >> the thread as well) but it seems stay alive somehow. erlang:halt(1) kill >> it immediately so it is not a problem. >> >> Questions: >> 1) Is my "analysis" correct? Is this infact what is happening? Anyone >> who can explain this behaviour (I.e. why it lingers on)? >> >> 2) If Question 1 then; what is the best way to kill the thread when I >> exit? driver_async_cancel() only works for threads that are not >> executing (according to Documentation) so what to do? I did an ugly >> workaround which sets a global flag and then I timeout the getch() >> operation and read for ERR. If I get an ERR I just read the abort flag >> and if it is set I just die otherwise I try to getch() again. This >> avoids hanging around for more then a tenth of a second but it is pretty >> ugly in my oppinion and I'm forced to go into halfdelay mode which means >> I can not allow this option outside of the driver because I become >> dependent on it. I'd rather do it the "right way" if there is such a way. >> >> References: (Links given as commit Ids so that if someone searches then >> they get relevant code and not the latest) >> [1] >> http://github.com/mazenharake/cecho/tree/199fd342ba186e1154c32dbfdcf2d9222e25f9f4 >> >> [2] >> http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/c_src/cecho.c#L119 >> static void request(ErlDrvData drvstate, char *buf, int buflen) { >> state *st = (state *)drvstate; >> driver_async(st->drv_port, NULL, loop_getch, (void *)st, NULL); >> } >> >> [3] >> http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/c_src/cecho.c#L557 >> void loop_getch(void *arg) { >> state *st = (state *)arg; >> ei_x_buff eixb; >> int keycode; >> ei_x_new_with_version(&eixb); >> keycode = getch(); >> integer(&eixb, keycode); >> driver_output(st->drv_port, eixb.buff, eixb.index); >> } >> >> [4] >> http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/src/cecho_srv.erl#L76 >> terminate(_Reason, State) -> >> do_call(State#state.port, ?ENDWIN), >> do_call(State#state.port, ?CURS_SET, ?ceCURS_NORMAL), >> erlang:port_close(State#state.port), >> erl_ddll:unload("cecho"). >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > Kresten Krab Thorup, CTO, Trifork > From mazen.harake@REDACTED Fri Aug 13 00:21:14 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Fri, 13 Aug 2010 01:21:14 +0300 Subject: [erlang-questions] Best way to kill an async thread in a linked in driver? In-Reply-To: <4C647243.3050409@erlang-solutions.com> References: <4C63B17E.5060901@erlang-solutions.com> <4C647243.3050409@erlang-solutions.com> Message-ID: <4C6473DA.9000609@erlang-solutions.com> Sorry, I should have been a bit more clear: I need to do things in Erlang space while I'm waiting for input, such as telling the driver to print out a character at pos y,x. This is the reason I can't be blocked in the driver. Still... I may have missed something, you are more than welcome to enlighten me :):) /M On 13/08/2010 01:14, Mazen Harake wrote: > Hmm... I'm not sure what you mean but the part I do understand is > that select() blocks... right? I can't have anything that blocks in > the driver since I must be able to interact with it as I'm waiting for > input. E.g. I need to update a certain part of the screen even if the > user is not pressing any buttons. > > I don't see how select() would make any difference? Perhaps I > misunderstood you, could you explain a bit more on how that would > work? Or perhaps point me to some direction? > > Thanks for feedback, > > /M > > On 12/08/2010 14:42, Kresten Krab Thorup wrote: >> I'd write the driver using select and non-blocking reads; that way >> you would not need async threads at all. >> >> Kresten >> >> On Aug 12, 2010, at 10:31 , Mazen Harake wrote: >> >>> It was late yesterday and despite 2 coffees and a nice breakfast I >>> still can't solve this one. Maybe a nudge in the right direction >>> anyone? >>> >>> Scenario: >>> I have a linked in driver; cecho [1], when ever I want to read input I >>> disable input to Erlang (-noinput) and use the getch() function in C >>> instead (never mind why ;)). This function is run in a seperate async >>> thread using driver_async() [2][3], and blocks at getch(). When a >>> character is read it sends a message back to Erlang (driver_output()) >>> and then the thread exits, when the message comes back to Erlang the >>> character is sent to the requester and the thread is restarted (which >>> again blocks at getch()). >>> >>> Later if I do an erlang:halt(0) ("graceful halt") the thread is still >>> lingering despite having closed the port[4] (which, Afaik should close >>> the thread as well) but it seems stay alive somehow. erlang:halt(1) >>> kill >>> it immediately so it is not a problem. >>> >>> Questions: >>> 1) Is my "analysis" correct? Is this infact what is happening? Anyone >>> who can explain this behaviour (I.e. why it lingers on)? >>> >>> 2) If Question 1 then; what is the best way to kill the thread when I >>> exit? driver_async_cancel() only works for threads that are not >>> executing (according to Documentation) so what to do? I did an ugly >>> workaround which sets a global flag and then I timeout the getch() >>> operation and read for ERR. If I get an ERR I just read the abort flag >>> and if it is set I just die otherwise I try to getch() again. This >>> avoids hanging around for more then a tenth of a second but it is >>> pretty >>> ugly in my oppinion and I'm forced to go into halfdelay mode which >>> means >>> I can not allow this option outside of the driver because I become >>> dependent on it. I'd rather do it the "right way" if there is such a >>> way. >>> >>> References: (Links given as commit Ids so that if someone searches then >>> they get relevant code and not the latest) >>> [1] >>> http://github.com/mazenharake/cecho/tree/199fd342ba186e1154c32dbfdcf2d9222e25f9f4 >>> >>> >>> [2] >>> http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/c_src/cecho.c#L119 >>> >>> static void request(ErlDrvData drvstate, char *buf, int buflen) { >>> state *st = (state *)drvstate; >>> driver_async(st->drv_port, NULL, loop_getch, (void *)st, NULL); >>> } >>> >>> [3] >>> http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/c_src/cecho.c#L557 >>> >>> void loop_getch(void *arg) { >>> state *st = (state *)arg; >>> ei_x_buff eixb; >>> int keycode; >>> ei_x_new_with_version(&eixb); >>> keycode = getch(); >>> integer(&eixb, keycode); >>> driver_output(st->drv_port, eixb.buff, eixb.index); >>> } >>> >>> [4] >>> http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/src/cecho_srv.erl#L76 >>> >>> terminate(_Reason, State) -> >>> do_call(State#state.port, ?ENDWIN), >>> do_call(State#state.port, ?CURS_SET, ?ceCURS_NORMAL), >>> erlang:port_close(State#state.port), >>> erl_ddll:unload("cecho"). >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> Kresten Krab Thorup, CTO, Trifork >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ok@REDACTED Fri Aug 13 02:08:22 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 13 Aug 2010 12:08:22 +1200 Subject: [erlang-questions] decoding nmea messages In-Reply-To: <201008121619055993982@its3.ch> References: <201008111944290468939@its3.ch>, <9B195EEE-4A6E-442B-8E6B-854D47A4E448@cs.otago.ac.nz> <201008121619055993982@its3.ch> Message-ID: On Aug 13, 2010, at 2:19 AM, info wrote: > Hi Richard, > The fields don't contain commas but sometimes they are empty. Therefore the size of the messages is not a constant. That's why I pointed out that you can "extract the rest (because you know HOW LONG the binary is)". I didn't mean that you know _without looking_, but that first you _find out_ how long the binary is and _then_ you do the binary match for the fixed length parts. > And what about the syntax analysis if each field is considered as a binary ? > Someone suggest to use regexp ... The question about how to analyse the fields depends on what the fields are supposed to look like. You'd really like to stick fairly close to the specification. For example, suppose you are given something like this: DBS Depth Below Surface 1 2 3 4 5 6 7 | | | | | | | $--DBS,x.x,f,x.x,M,x.x,F*hh 1) Depth, feet 2) f = feet 3) Depth, meters 4) M = meters 5) Depth, Fathoms 6) F = Fathoms 7) Checksum (NMEAdescription.pdf, page 7, found on the web) I'd actually think about writing a tiny pattern compiler that took this and generated %% DBS Depth Below Surface %% $--DBS,x.x,f,x.x,M,x.x,F*hh depth_below_surface(Talker, N, <<"$", Talker/binary:2, "DBS,", Data/binary:(N-10), _/binary>> ) -> case split(Data) of [F1,<<"f">>,F3,<<"M">>,F5,<<"F">>] -> case {nchk(F1),nck(F3),nchk(F5)} of {N1,N3,N5} when is_number(N1),is_number(N3),is_number(N5) -> {Talker,dbs,N1,N3,N5} ; fail end ; _ -> fail end; depth_below_surface(_, _, _) -> fail. or something like that. It would be necessary to read the specification of all the items one might want to recognise to come up with a suitable vocabulary. There are - distance (a number) - angle (a number) - waypoint id (no idea of format) - hexadecimal of specified length - fixed letter - one of several letters - mode - satellite ID - time - time difference - frequency ... Now regular expressions CAN be used to match these, but not to decode them. And supposing you do decide to use regular expressions for taking things apart, (a) it would be better not to do this by hand. Complex regular expressions are not easy to get right. It is much better to write a little AWK or Python (or Erlang!) program to read the specifications and do the conversions. It is so much easier to check that you have copied information verbatim than to check that you have translated it correctly. If a match or translation for some field type is wrong, it is so much easier to fix it ONCE in the pattern compiler than to fix every occurrence of that pattern. And with a pattern compiler (NOT a complex program in this case), it is so much easier for maintainers to add new formats or revise old ones. (b) the different semantic types imply _range_ checks that are difficult to do with regular expressions, so you would STILL need custom code to validate what you found, which for the reasons given in (a) should be generated, not hand written. Which means that regular expressions really give you very very little for this problem. I note that these messages are limited to 80 bytes, so I would not be scared to turn them into lists and parse them that way. And it appears that the fields cannot contain commas, so breaking one of these things into fields is easy, it's just checking and converting them that's tricky. From zeno490@REDACTED Fri Aug 13 02:29:23 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Thu, 12 Aug 2010 20:29:23 -0400 Subject: [erlang-questions] Best way to kill an async thread in a linked in driver? In-Reply-To: <4C6473DA.9000609@erlang-solutions.com> References: <4C63B17E.5060901@erlang-solutions.com> <4C647243.3050409@erlang-solutions.com> <4C6473DA.9000609@erlang-solutions.com> Message-ID: It is pretty standard to pool for a shared variable and exit/clean up on state change. I've done this hundred of times. Alternatively, you might be able to flag that thread as a daemon thread. Typically programs exit when only daemon threads remain and all non-daemon threads have completed/exited. I know they have that with C++ on solaris, java, python, etc. Not sure about your specifics though but worth checking it out. You can play with the timeout value of your polling. High values mean that you'll be penalized when exiting but you'll be decreasing the delay in reading while lower values will do the opposite. Just find a decent balance for your app but afaik, few people have problems waiting 1-2s for an app to exit. Nicholas On Thu, Aug 12, 2010 at 6:21 PM, Mazen Harake < mazen.harake@REDACTED> wrote: > Sorry, I should have been a bit more clear: > I need to do things in Erlang space while I'm waiting for input, such as > telling the driver to print out a character at pos y,x. > > This is the reason I can't be blocked in the driver. > > Still... I may have missed something, you are more than welcome to > enlighten me :):) > > /M > > > On 13/08/2010 01:14, Mazen Harake wrote: > >> Hmm... I'm not sure what you mean but the part I do understand is that >> select() blocks... right? I can't have anything that blocks in the driver >> since I must be able to interact with it as I'm waiting for input. E.g. I >> need to update a certain part of the screen even if the user is not pressing >> any buttons. >> >> I don't see how select() would make any difference? Perhaps I >> misunderstood you, could you explain a bit more on how that would work? Or >> perhaps point me to some direction? >> >> Thanks for feedback, >> >> /M >> >> On 12/08/2010 14:42, Kresten Krab Thorup wrote: >> >>> I'd write the driver using select and non-blocking reads; that way you >>> would not need async threads at all. >>> >>> Kresten >>> >>> On Aug 12, 2010, at 10:31 , Mazen Harake wrote: >>> >>> It was late yesterday and despite 2 coffees and a nice breakfast I >>>> still can't solve this one. Maybe a nudge in the right direction anyone? >>>> >>>> Scenario: >>>> I have a linked in driver; cecho [1], when ever I want to read input I >>>> disable input to Erlang (-noinput) and use the getch() function in C >>>> instead (never mind why ;)). This function is run in a seperate async >>>> thread using driver_async() [2][3], and blocks at getch(). When a >>>> character is read it sends a message back to Erlang (driver_output()) >>>> and then the thread exits, when the message comes back to Erlang the >>>> character is sent to the requester and the thread is restarted (which >>>> again blocks at getch()). >>>> >>>> Later if I do an erlang:halt(0) ("graceful halt") the thread is still >>>> lingering despite having closed the port[4] (which, Afaik should close >>>> the thread as well) but it seems stay alive somehow. erlang:halt(1) kill >>>> it immediately so it is not a problem. >>>> >>>> Questions: >>>> 1) Is my "analysis" correct? Is this infact what is happening? Anyone >>>> who can explain this behaviour (I.e. why it lingers on)? >>>> >>>> 2) If Question 1 then; what is the best way to kill the thread when I >>>> exit? driver_async_cancel() only works for threads that are not >>>> executing (according to Documentation) so what to do? I did an ugly >>>> workaround which sets a global flag and then I timeout the getch() >>>> operation and read for ERR. If I get an ERR I just read the abort flag >>>> and if it is set I just die otherwise I try to getch() again. This >>>> avoids hanging around for more then a tenth of a second but it is pretty >>>> ugly in my oppinion and I'm forced to go into halfdelay mode which means >>>> I can not allow this option outside of the driver because I become >>>> dependent on it. I'd rather do it the "right way" if there is such a >>>> way. >>>> >>>> References: (Links given as commit Ids so that if someone searches then >>>> they get relevant code and not the latest) >>>> [1] >>>> >>>> http://github.com/mazenharake/cecho/tree/199fd342ba186e1154c32dbfdcf2d9222e25f9f4 >>>> >>>> [2] >>>> >>>> http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/c_src/cecho.c#L119 >>>> static void request(ErlDrvData drvstate, char *buf, int buflen) { >>>> state *st = (state *)drvstate; >>>> driver_async(st->drv_port, NULL, loop_getch, (void *)st, NULL); >>>> } >>>> >>>> [3] >>>> >>>> http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/c_src/cecho.c#L557 >>>> void loop_getch(void *arg) { >>>> state *st = (state *)arg; >>>> ei_x_buff eixb; >>>> int keycode; >>>> ei_x_new_with_version(&eixb); >>>> keycode = getch(); >>>> integer(&eixb, keycode); >>>> driver_output(st->drv_port, eixb.buff, eixb.index); >>>> } >>>> >>>> [4] >>>> >>>> http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/src/cecho_srv.erl#L76 >>>> terminate(_Reason, State) -> >>>> do_call(State#state.port, ?ENDWIN), >>>> do_call(State#state.port, ?CURS_SET, ?ceCURS_NORMAL), >>>> erlang:port_close(State#state.port), >>>> erl_ddll:unload("cecho"). >>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> Kresten Krab Thorup, CTO, Trifork >>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From g9414002.pccu.edu.tw@REDACTED Fri Aug 13 05:03:45 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Fri, 13 Aug 2010 11:03:45 +0800 Subject: [erlang-questions] Re: OOP in Erlang In-Reply-To: <1858bc4d-497b-4953-b083-f0d4df21f9d5@k10g2000yqa.googlegroups.com> References: <611396.69573.qm@web120515.mail.ne1.yahoo.com> <1858bc4d-497b-4953-b083-f0d4df21f9d5@k10g2000yqa.googlegroups.com> Message-ID: On Wed, Aug 11, 2010 at 9:20 PM, Steve Davis wrote: > things going on > here: > 1) Data structures and > 2) Transformations of data structures. > > I agree. Mandatory features of OOP includes encapsulation, abstraction, inheritance, and polymorphism. After reviewing this thread, opinions are summarized that including * if we do OOP in Erlang syntax, * if features of OOP are able to be implemented in Erlang, * and if exactly Erlang is an OOP language or OO language. Now knowledge of OOP is mixture of origin concepts, languages such as Smalltalk or Java, and UML modelling methods. Recently I read the book "Design Patterns Explained," which hints that OOP/OOD is to treat everything, including primitive datatypes, as object. And he explained how abstraction is different from implementation. Then I thought that Erlang treats everything as function or process, but really? And, Erlang is not toward OOP, so it may not be OOP-like. Let's consider how to implement OOP in Erlang, either using Erlang syntax or realizing OOP system in Erlang. Encapsulation: values and functions should be organized as an object structure tagged as an object name, belong to some class. It is done by Erlang record syntax or as a module, and somehow messages are sent to the object or to some member of the structure. Functions in a structure may call other fields or functions in other structures. Encapsulation means how an object is structured. Any object is generated by following a object specification, the class. Object and class are implemented in the same way: structure. Abstraction: it's separated from implementation. Abstraction is done in Erlang by hot code replacement. Inheritance: it should be done by taking objects by following both superclass and subclass specifications and then overriding the superclass object with the subclass object, that is to merge base functions into extended functions effectively. Inheritance shall not be as copying of superclass object, but keeping a newest copy of a extended function while keeping the reference to its base function. Inheritance semantics shall be consistent. Polymorphism: it means that different objects, even if they are in the same class, may behave separately. It's not a problem because Erlang has this feature. On real OOP, I think that OOP is a feature, and somehow it's the problem itself. So, when talking about OOP in Erlang, you can have a try to write Erlang in OOP style, however, Erlang need not be too OOP-like. From vinoski@REDACTED Fri Aug 13 05:43:33 2010 From: vinoski@REDACTED (Steve Vinoski) Date: Thu, 12 Aug 2010 23:43:33 -0400 Subject: [erlang-questions] Best way to kill an async thread in a linked in driver? In-Reply-To: <4C647243.3050409@erlang-solutions.com> References: <4C63B17E.5060901@erlang-solutions.com> <4C647243.3050409@erlang-solutions.com> Message-ID: On Thu, Aug 12, 2010 at 6:14 PM, Mazen Harake wrote: > On 12/08/2010 14:42, Kresten Krab Thorup wrote: >> >> I'd write the driver using select and non-blocking reads; that way you >> would not need async threads at all. > > ?Hmm... I'm not sure what you mean but the part I do understand is that > select() blocks... right? I can't have anything that blocks in the driver > since I must be able to interact with it as I'm waiting for input. E.g. I > need to update a certain part of the screen even if the user is not pressing > any buttons. > > I don't see how select() would make any difference? Perhaps I misunderstood > you, could you explain a bit more on how that would work? Or perhaps point > me to some direction? What Kresten is essentially recommending is that you make the file descriptor from which you're reading characters non-blocking and then register it into the emulator's event loop via driver_select(). Once your file descriptor has something to read, the emulator will call your code via your driver's ready_input callback. At that point you can read your character(s) from your fd and process them pretty much like you're doing now. You can read the fd until you get EAGAIN or EWOULDBLOCK, at which point there's nothing left to read, and you can then return from your ready_input callback. The emulator will call you every time there's something to read until you unregister the fd. As Kresten said, no async thread required. --steve From ulf.wiger@REDACTED Fri Aug 13 10:18:22 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 13 Aug 2010 10:18:22 +0200 Subject: [erlang-questions] Re: OOP in Erlang In-Reply-To: References: <611396.69573.qm@web120515.mail.ne1.yahoo.com> <1858bc4d-497b-4953-b083-f0d4df21f9d5@k10g2000yqa.googlegroups.com> Message-ID: <4C64FFCE.4080301@erlang-solutions.com> ??? (Yau-Hsien Huang) wrote: > > Recently I read the book > "Design Patterns Explained," which hints that OOP/OOD is to treat > everything, including primitive datatypes, as object. > And he explained how abstraction > is different from implementation. Then I thought that Erlang > treats everything as function or process, but really? Yeah, well, I've had some pretty interesting discussions with proponents of that view, and I would rather say that it is possible to deal with abstractions as wholly separate from implementation, but it is wrong to say that it holds in general (at least for any useful definition of implementation). The idea, as I understand it, is that you want to adapt to the common pattern that you have separate teams that handle requirements, architecture, and programming (oh, and yet another team doing testing, usually). I am very well familiar with that way of working, and will go as far as saying that it can sometimes be extremely difficult to avoid. The real danger with that setup, and the viewing of abstraction as somehow separate from implementation, is that you tend to to do "abstraction" for its own sake. The quotation marks are there because in many cases, people confuse drawing visual symbols with abstraction; while it may be in some dimension, it is not abstraction in any useful sense for a programmer. "The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise." Edsger W. Dijkstra, "The Humble Programmer", October 1972 To do abstraction in the way Dijkstra intends, you should have a very good intuition about implementation issues. You might say that abstraction grows from implementation. The key here is that you need to *pick* your abstractions carefully based on your understanding of the domain *and* details, opportunities and limitations of the implementation layer. Erlang provides some abstractions that are a bit unusual, but have been found in practice to work exceedingly well in some domains. Some of those abstractions tend to be sub-optimal if your aim is to write computational code that goes blazingly fast. OTOH, some of the abstractions that have traditionally fared well in that regard are suffering now, as the hardware becomes increasingly parallel - something that happens to suit Erlang just fine. Programming is all about abstraction - layers on top of layers of abstraction, reaching all the way deep down into the hardware your program runs on. > Abstraction: it's separated from implementation. Abstraction > is done in Erlang by hot code replacement. No. Abstraction is when you make a model of something by hiding underlying details. Abstraction is all over the place in programming. Any Erlang data type is an abstraction - as Erlang doesn't allow you do address memory directly, you cannot control how many bits are assigned, how the data is laid out, where it is, whether it is shared, etc. Nor are you supposed to care. "Programming is both a top down activity from requirements and overall structure of the system and a bottom up activity based on the abstractions provided by the programming language. Abstractions such as modules, processes, higher order functions, etc. are to the programmer like transistors, capacitors, resistors, etc. to the hardware designer. It is important that the abstractions be few, simple to understand and yet powerful and provide the needed functionality. For example, if the application is inherently concurrent it would be very dfficult to program without some process concept. In that case the application program itself would have to include some form of scheduler. Distribution, error handling, and hot code loading are extremely complicated requirements and the support for them provided by Erlang enables the programmer to concentrate on the application rather than on the basic programming technology." (Bjarne Dacker, http://www.erlang.se/publications/bjarnelic.pdf, pg 26) BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From ttom.kelly@REDACTED Fri Aug 13 12:20:09 2010 From: ttom.kelly@REDACTED (tom kelly) Date: Fri, 13 Aug 2010 11:20:09 +0100 Subject: Faster ets:lookups Message-ID: Hi List, I've just been reading Scott Lystig Fritchies paper "A Study of Erlang ETS Table Implementations and Performance" and was wondering whatever became of the "judy" option for ets table types. It's not in any OTP release I've worked with (R11 onwards) even though the research was done on R9, I assume this was considered but not accepted by OTP, anyone know the reasons? I found this when looking for ways to improve our cache system, we uses ets:lookup very intensively (table contains ~7000 entries indexed by a tuple of two integers representing an adler, table is type "bag") and our profiling has shown that the lookup uses the largest proportion of our processing time. Does anyone have any suggestions on how to optimise the lookup? Thanks in advance! //Tom. From krab@REDACTED Fri Aug 13 12:12:06 2010 From: krab@REDACTED (Kresten Krab Thorup) Date: Fri, 13 Aug 2010 12:12:06 +0200 Subject: [erlang-questions] Wondering on active TCP socket In-Reply-To: References: Message-ID: <86E6BE27-2022-46B7-8784-C83D229246E7@trifork.com> Hi, In "active" mode, a socket acts like an infinite loop that simply forwards incoming data to the designated target process. Incoming data is repackaged according to the socket's current packet mode. There is no flow control, unlike normal message sends where the sender is punished with reductions proportional to the size of the message queue. The port driver does not have a concept of reductions that will slow it down, and most of it's operations run in the asynch threads that are independent of the normal scheduling of Erlang processes; so driver's are at liberty to run infinitely. So perhaps you can insert something like this in your consume loop to do your own pseudo flow control - this will switch active on/off according to some high/low watermark for the consuming process queue length. But since the port runs independently, there is no guarantee that the queue does not overrun. Kresten init() -> Socket = inets:accept(...); ControlState = set_controlled_active(Socket, 100, 200), main_loop(ControlState). main_loop(CS) -> CS2 = control_active(true, CS), receive {data, DataPkt} -> ... main_loop(CS2) ... %%%%%%%%%%%%%%%%%%%%%%% %% pseudo flow control for active sockets %% -record(flow_control,{socket,is_active,low,high}). set_controlled_active(Socket, LowWaterMark, HighWaterMark) -> [{active, IsActive}] = inet:getopts(Socket, [active]), control_active(true, #flow_control { socket=Socket, is_active=IsActive, low=LowWaterMark, high=HighWaterMark}). %% you want it to be active, and it is currently set to active; test %% if our queue size is above the high watermark control_active(true, #flow_control{is_active=true, high=HW}=State) -> {message_queue_len, QLen} = erlang:process_info(message_queue_len), case QLen > HW of true -> ok = inet:setopts(State#flow_control.socket, [{active, false}]), State#flow_control{is_active=false}; false -> State end; %% you want it to be active, but it is currently set to non-active; test %% if our queue size is below the LowWatermark control_active(true, #flow_control{is_active=false, low=LW}=State) -> {message_queue_len, QLen} = erlang:process_info(message_queue_len), case QLen < LW of true -> ok = inet:setopts(State#flow_control.socket, [{active, true}]), State#flow_control{is_active=true}; false -> State end; %% setting active to false control_active(false, #flow_control{socket=Socket}=State ) -> ok = inet:setopts(Socket,[{active, false}]), State#flow_control{is_active=false}. On Aug 12, 2010, at 15:51 , Kaiduan Xie wrote: > Hi, all, > > From Joe's book, > > "Once an active socket has been created, the controlling process will > be sent {tcp, Socket, Data} messages as data is received. There > is no way the controlling process can control the flow of these > messages. A rogue client could send thousands of messages to > the system, and these would all be sent to the controlling process. > The controlling process cannot stop this flow of messages." > > "This process cannot control the flow of messages to the server loop. > If the client produces data faster than the server can consume this > data, > then the system can be flooded with messages?the message buffers will > fill up, and the system might crash or behave strangely." > > For this case, I think that the flow control of underlying OS TCP > stack will kick in, and the sender can not send more packets. How the > system can be flooded with messages in erlang? > > Can someone elaborate more on this point? > > Thanks, > > Kaiduan > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Kresten Krab Thorup, CTO, Trifork From kaiduanx@REDACTED Fri Aug 13 15:34:55 2010 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Fri, 13 Aug 2010 09:34:55 -0400 Subject: [erlang-questions] Wondering on active TCP socket In-Reply-To: <964012D0-F9E6-4E20-BCAC-7AFCB89360A8@rogvall.se> References: <86E6BE27-2022-46B7-8784-C83D229246E7@trifork.com> <964012D0-F9E6-4E20-BCAC-7AFCB89360A8@rogvall.se> Message-ID: Thanks everyone. As Joe pointed out we can introduce traffic control using {active, once}. My point here is to understand how Erlang gen_tcp interacts with the underlying OS TCP stack. More pointers on this are welcome. Kaiduan On Fri, Aug 13, 2010 at 8:49 AM, Tony Rogvall wrote: > {active, once} > > /Tony > > On 13 aug 2010, at 12.12, Kresten Krab Thorup wrote: > >> Hi, >> >> In "active" mode, a socket acts like an infinite loop that simply forwards incoming data to the designated target process. ?Incoming data is repackaged according to the socket's current packet mode. >> >> There is no flow control, unlike normal message sends where the sender is punished with reductions proportional to the size of the message queue. ?The port driver does not have a concept of reductions that will slow it down, and most of it's operations run in the asynch threads that are independent of the normal scheduling of Erlang processes; so driver's are at liberty to run infinitely. >> >> So perhaps you can insert something like this in your consume loop to do your own pseudo flow control - this will ?switch active on/off according to some high/low watermark for the consuming process queue length. ?But since the port runs independently, there is no guarantee that the queue does not overrun. >> >> Kresten >> >> >> init() -> >> ? Socket = inets:accept(...); >> ? ControlState = set_controlled_active(Socket, 100, 200), >> ? main_loop(ControlState). >> >> main_loop(CS) -> >> ? ? CS2 = control_active(true, CS), >> ? ? receive >> ? ? ? ?{data, DataPkt} -> ... >> ? ? ? ? ? ?main_loop(CS2) >> ? ? ? ?... >> >> >> %%%%%%%%%%%%%%%%%%%%%%% >> %% pseudo flow control for active sockets %% >> >> -record(flow_control,{socket,is_active,low,high}). >> >> >> set_controlled_active(Socket, LowWaterMark, HighWaterMark) -> >> ? ?[{active, IsActive}] = inet:getopts(Socket, [active]), >> ? ?control_active(true, >> ? ? ? ? ? ? ? ? ?#flow_control { socket=Socket, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?is_active=IsActive, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?low=LowWaterMark, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?high=HighWaterMark}). >> >> >> >> %% you want it to be active, and it is currently set to active; test >> %% if our queue size is above the high watermark >> control_active(true, #flow_control{is_active=true, high=HW}=State) -> >> ? ?{message_queue_len, QLen} = erlang:process_info(message_queue_len), >> ? ?case QLen > HW of >> ? ? ? true -> >> ? ? ? ? ? ok = inet:setopts(State#flow_control.socket, [{active, false}]), >> ? ? ? ? ? State#flow_control{is_active=false}; >> ? ? ? false -> >> ? ? ? ? ? State >> ? ?end; >> >> %% you want it to be active, but it is currently set to non-active; test >> %% if our queue size is below the LowWatermark >> control_active(true, #flow_control{is_active=false, low=LW}=State) -> >> ? ?{message_queue_len, QLen} = erlang:process_info(message_queue_len), >> ? ?case QLen < LW of >> ? ? ? true -> >> ? ? ? ? ? ok = inet:setopts(State#flow_control.socket, [{active, true}]), >> ? ? ? ? ? State#flow_control{is_active=true}; >> ? ? ? false -> >> ? ? ? ? ? State >> ? ?end; >> >> %% setting active to false >> control_active(false, #flow_control{socket=Socket}=State ) -> >> ? ?ok = inet:setopts(Socket,[{active, false}]), >> ? ?State#flow_control{is_active=false}. >> >> >> >> On Aug 12, 2010, at 15:51 , Kaiduan Xie wrote: >> >>> Hi, all, >>> >>> From Joe's book, >>> >>> "Once an active socket has been created, the controlling process will >>> be sent {tcp, Socket, Data} messages as data is received. There >>> is no way the controlling process can control the flow of these >>> messages. A rogue client could send thousands of messages to >>> the system, and these would all be sent to the controlling process. >>> The controlling process cannot stop this flow of messages." >>> >>> "This process cannot control the flow of messages to the server loop. >>> If the client produces data faster than the server can consume this >>> data, >>> then the system can be flooded with messages?the message buffers will >>> fill up, and the system might crash or behave strangely." >>> >>> For this case, I think that the flow control of underlying OS TCP >>> stack will kick in, and the sender can not send more packets. How the >>> system can be flooded with messages in erlang? >>> >>> Can someone elaborate more on this point? >>> >>> Thanks, >>> >>> Kaiduan >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> Kresten Krab Thorup, CTO, Trifork >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > From tony@REDACTED Fri Aug 13 14:49:42 2010 From: tony@REDACTED (Tony Rogvall) Date: Fri, 13 Aug 2010 14:49:42 +0200 Subject: [erlang-questions] Wondering on active TCP socket In-Reply-To: <86E6BE27-2022-46B7-8784-C83D229246E7@trifork.com> References: <86E6BE27-2022-46B7-8784-C83D229246E7@trifork.com> Message-ID: <964012D0-F9E6-4E20-BCAC-7AFCB89360A8@rogvall.se> {active, once} /Tony On 13 aug 2010, at 12.12, Kresten Krab Thorup wrote: > Hi, > > In "active" mode, a socket acts like an infinite loop that simply forwards incoming data to the designated target process. Incoming data is repackaged according to the socket's current packet mode. > > There is no flow control, unlike normal message sends where the sender is punished with reductions proportional to the size of the message queue. The port driver does not have a concept of reductions that will slow it down, and most of it's operations run in the asynch threads that are independent of the normal scheduling of Erlang processes; so driver's are at liberty to run infinitely. > > So perhaps you can insert something like this in your consume loop to do your own pseudo flow control - this will switch active on/off according to some high/low watermark for the consuming process queue length. But since the port runs independently, there is no guarantee that the queue does not overrun. > > Kresten > > > init() -> > Socket = inets:accept(...); > ControlState = set_controlled_active(Socket, 100, 200), > main_loop(ControlState). > > main_loop(CS) -> > CS2 = control_active(true, CS), > receive > {data, DataPkt} -> ... > main_loop(CS2) > ... > > > %%%%%%%%%%%%%%%%%%%%%%% > %% pseudo flow control for active sockets %% > > -record(flow_control,{socket,is_active,low,high}). > > > set_controlled_active(Socket, LowWaterMark, HighWaterMark) -> > [{active, IsActive}] = inet:getopts(Socket, [active]), > control_active(true, > #flow_control { socket=Socket, > is_active=IsActive, > low=LowWaterMark, > high=HighWaterMark}). > > > > %% you want it to be active, and it is currently set to active; test > %% if our queue size is above the high watermark > control_active(true, #flow_control{is_active=true, high=HW}=State) -> > {message_queue_len, QLen} = erlang:process_info(message_queue_len), > case QLen > HW of > true -> > ok = inet:setopts(State#flow_control.socket, [{active, false}]), > State#flow_control{is_active=false}; > false -> > State > end; > > %% you want it to be active, but it is currently set to non-active; test > %% if our queue size is below the LowWatermark > control_active(true, #flow_control{is_active=false, low=LW}=State) -> > {message_queue_len, QLen} = erlang:process_info(message_queue_len), > case QLen < LW of > true -> > ok = inet:setopts(State#flow_control.socket, [{active, true}]), > State#flow_control{is_active=true}; > false -> > State > end; > > %% setting active to false > control_active(false, #flow_control{socket=Socket}=State ) -> > ok = inet:setopts(Socket,[{active, false}]), > State#flow_control{is_active=false}. > > > > On Aug 12, 2010, at 15:51 , Kaiduan Xie wrote: > >> Hi, all, >> >> From Joe's book, >> >> "Once an active socket has been created, the controlling process will >> be sent {tcp, Socket, Data} messages as data is received. There >> is no way the controlling process can control the flow of these >> messages. A rogue client could send thousands of messages to >> the system, and these would all be sent to the controlling process. >> The controlling process cannot stop this flow of messages." >> >> "This process cannot control the flow of messages to the server loop. >> If the client produces data faster than the server can consume this >> data, >> then the system can be flooded with messages?the message buffers will >> fill up, and the system might crash or behave strangely." >> >> For this case, I think that the flow control of underlying OS TCP >> stack will kick in, and the sender can not send more packets. How the >> system can be flooded with messages in erlang? >> >> Can someone elaborate more on this point? >> >> Thanks, >> >> Kaiduan >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > Kresten Krab Thorup, CTO, Trifork > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From kenrobinsonster@REDACTED Fri Aug 13 16:02:49 2010 From: kenrobinsonster@REDACTED (Ken Robinson) Date: Sat, 14 Aug 2010 00:02:49 +1000 Subject: Stopping of Erlang app hangs when mnesia stopped from within the program Message-ID: Hi all, I've run into a problem where I cannot stop mnesia within my program without causing the app to hang. I'm presently doing prototyping of mnesia within my erlang app. In my jaus_app.erl file the start() calls: {atomic, ok} = mnesia:load_textfile("priv/mnesia_prototype.txt") My stop() function calls: mnesia:dump_to_textfile("priv/mnesia_prototype_res.txt"), mnesia:stop(), When I comment out these lines and start and stop mnesia from the erlang prompt, I am able to stop my application cleanly. Should I not use these prototype functions within a fully fledged erlang app? Ken. From igarai@REDACTED Fri Aug 13 17:11:04 2010 From: igarai@REDACTED (=?UTF-8?B?ScOxYWtpIEdhcmF5?=) Date: Fri, 13 Aug 2010 12:11:04 -0300 Subject: A matter of style? Message-ID: Hello everyone, while coding a spatial indexing tree, I often use a small helper function within/9, which given three sets of xyz coordinates describing three points, returns true when first point lies within the bounding box created by the other two points. This is nothing more than: within(X, Y, Z, Xmin, Ymin, Zmin, Xmax, Ymax, Zmax) -> (Xmin =< X) and (X =< Xmax) and (Ymin =< Y) and (Y =< Ymax) and (Zmin =< Z) and (Z =< Zmax). I ask of you this, what difference, if any, is there between using an if clause and this function, and inserting the body of this function as a guard? There is the obvious code reuse when using the function, instead of copy-pasting the body. But it makes me do this: Within = within(X, Y, Z, Xmin, Ymin, Zmin, Xmax, Ymax, Zmax), if Within -> do_something() true -> {error,out_of_bounds} end. which I dislike for personal reasons. Is using a guard more efficient than calling a function? Is the impact negligible even when done many times? Or is it a matter of style? Do you see a better way? thank you for your time, I?aki Garay. -----BEGIN GEEK CODE BLOCK----- Version: 3.1 GCS/GMU d- s:- a25 C++ UL+++ P--- E- W+ PS+++ PE- Y+ t++ 5+ tv- b++ DI+ D--- G e h! r y++ ------END GEEK CODE BLOCK------ From lenartlad@REDACTED Fri Aug 13 17:33:18 2010 From: lenartlad@REDACTED (Ladislav Lenart) Date: Fri, 13 Aug 2010 17:33:18 +0200 Subject: [erlang-questions] A matter of style? In-Reply-To: References: Message-ID: <4C6565BE.7060003@volny.cz> I?aki Garay wrote: > Hello everyone, > while coding a spatial indexing tree, I often use a small helper > function within/9, which given three sets of xyz coordinates > describing three points, returns true when first point lies within the > bounding box created by the other two points. > > This is nothing more than: > > within(X, Y, Z, Xmin, Ymin, Zmin, Xmax, Ymax, Zmax) -> > (Xmin =< X) and > (X =< Xmax) and > (Ymin =< Y) and > (Y =< Ymax) and > (Zmin =< Z) and > (Z =< Zmax). > > I ask of you this, what difference, if any, is there between using an > if clause and this function, and inserting the body of this function > as a guard? > There is the obvious code reuse when using the function, instead of > copy-pasting the body. But it makes me do this: > > Within = within(X, Y, Z, Xmin, Ymin, Zmin, Xmax, Ymax, Zmax), > if > Within -> > do_something() > true -> > {error,out_of_bounds} > end. > > which I dislike for personal reasons. > Is using a guard more efficient than calling a function? Is the impact > negligible even when done many times? Or is it a matter of style? > Do you see a better way? Hello, I would use "case" instead of "if" precisely because case allows full expressions and not only guards: case within(...) of true -> do_something(); false -> {error, ouf_of_bounds} end. HTH, Ladislav Lenart From alan@REDACTED Fri Aug 13 17:54:06 2010 From: alan@REDACTED (Alan Hoffman) Date: Fri, 13 Aug 2010 11:54:06 -0400 Subject: Job Posting -- Work at Cloudant Message-ID: Hi All- Cloudant is looking for seriously talented developers that think concurrently to join our team and help us extend our hosted CouchDB data platform. You would be working with experts in distributed erlang, CouchDB, search, and high-availability cloud services. Check out our job posting below, and send me an email if you're interested. Cheers Alan Hoffman Co-founder, Cloudant +++++++++++++++++++ At Cloudant we're building a hosted data platform based on Apache CouchDB. We're serious about data management, analytics, search and distributed systems. Our founding team consists of 3 physics PhDs from MIT and our company includes experts in database tech, devops, analytics, UI/UX, and search. We're looking for talented and passionate software engineers to join us at our Boston and Seattle offices. You should be a technical generalist ready and eager to jump into any problem and able to learn the skills necessary to do so. You will be working closely with our CTO on the core parts of our data platform and have the opportunity to influence product development at the highest levels. Our only requirement is that you have experience writing Erlang/OTP applications. While not necessary, experience with the other tools we use for our platform (C/C++, CouchDB, Chef) is a huge plus. We're more impressed by talent, passion, and ability to execute well in a small team (since we are a small team). Here is a list of requirements and other things that can't hurt your chances: REQUIREMENTS Mastery of Erlang/OTP Applications for distributed systems LIST OF THINGS THAT CAN'T HURT Experience developing in C/C++ Familiarity with CouchDB and other NoSQL databases Notable experience working on open-source projects An advanced degree in Computer Science or equivalent technical field Cloudant is a VC-funded software startup with offices in Boston and Seattle. Compensation includes full time salary, generous equity compensation plan, and benefits. More info at: http://cloudant.com To apply, visit cloudant.theresumator.com and include a resume and why you'd be a great fit for Cloudant From mazen.harake@REDACTED Fri Aug 13 18:17:33 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Fri, 13 Aug 2010 19:17:33 +0300 Subject: [erlang-questions] Best way to kill an async thread in a linked in driver? In-Reply-To: References: <4C63B17E.5060901@erlang-solutions.com> <4C647243.3050409@erlang-solutions.com> Message-ID: <4C65701D.3010906@erlang-solutions.com> Ah! I had completely over looked the driver_select() and thought that he was talking about simply select() which didn't make sense. I'll try this out later today or tomorrow, thanks for the explaination. /Mazen On 13/08/2010 06:43, Steve Vinoski wrote: > On Thu, Aug 12, 2010 at 6:14 PM, Mazen Harake > wrote: >> On 12/08/2010 14:42, Kresten Krab Thorup wrote: >>> I'd write the driver using select and non-blocking reads; that way you >>> would not need async threads at all. >> Hmm... I'm not sure what you mean but the part I do understand is that >> select() blocks... right? I can't have anything that blocks in the driver >> since I must be able to interact with it as I'm waiting for input. E.g. I >> need to update a certain part of the screen even if the user is not pressing >> any buttons. >> >> I don't see how select() would make any difference? Perhaps I misunderstood >> you, could you explain a bit more on how that would work? Or perhaps point >> me to some direction? > What Kresten is essentially recommending is that you make the file > descriptor from which you're reading characters non-blocking and then > register it into the emulator's event loop via driver_select(). Once > your file descriptor has something to read, the emulator will call > your code via your driver's ready_input callback. At that point you > can read your character(s) from your fd and process them pretty much > like you're doing now. You can read the fd until you get EAGAIN or > EWOULDBLOCK, at which point there's nothing left to read, and you can > then return from your ready_input callback. The emulator will call you > every time there's something to read until you unregister the fd. > > As Kresten said, no async thread required. > > --steve From mazen.harake@REDACTED Fri Aug 13 18:25:26 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Fri, 13 Aug 2010 19:25:26 +0300 Subject: [erlang-questions] Best way to kill an async thread in a linked in driver? In-Reply-To: References: <4C63B17E.5060901@erlang-solutions.com> <4C647243.3050409@erlang-solutions.com> <4C6473DA.9000609@erlang-solutions.com> Message-ID: <4C6571F6.8040005@erlang-solutions.com> You're actually spot on about the timeout "tuning", I was playing with that yesterday and my conclusion was (I read up on it a little) that most people aren't able to tell the small differences in delays that can occur between pressing a key and reacting to it. Anyway I did implement this version now and I'm going to try out the driver_select() version later to compare in terms of user-experience and complexity of the code. If the driver_select() version works out as well as the version I have now (not yet pushed to branch) I might go with that. Thanks for the feedback! /Mazen On 13/08/2010 03:29, Nicholas Frechette wrote: > It is pretty standard to pool for a shared variable and exit/clean up > on state change. I've done this hundred of times. > Alternatively, you might be able to flag that thread as a daemon > thread. Typically programs exit when only daemon threads remain and > all non-daemon threads have completed/exited. I know they have that > with C++ on solaris, java, python, etc. Not sure about your specifics > though but worth checking it out. > > You can play with the timeout value of your polling. High values mean > that you'll be penalized when exiting but you'll be decreasing the > delay in reading while lower values will do the opposite. Just find a > decent balance for your app but afaik, few people have problems > waiting 1-2s for an app to exit. > Nicholas > > On Thu, Aug 12, 2010 at 6:21 PM, Mazen Harake > > wrote: > > Sorry, I should have been a bit more clear: > I need to do things in Erlang space while I'm waiting for input, > such as telling the driver to print out a character at pos y,x. > > This is the reason I can't be blocked in the driver. > > Still... I may have missed something, you are more than welcome to > enlighten me :):) > > /M > > > On 13/08/2010 01:14, Mazen Harake wrote: > > Hmm... I'm not sure what you mean but the part I do > understand is that select() blocks... right? I can't have > anything that blocks in the driver since I must be able to > interact with it as I'm waiting for input. E.g. I need to > update a certain part of the screen even if the user is not > pressing any buttons. > > I don't see how select() would make any difference? Perhaps I > misunderstood you, could you explain a bit more on how that > would work? Or perhaps point me to some direction? > > Thanks for feedback, > > /M > > On 12/08/2010 14:42, Kresten Krab Thorup wrote: > > I'd write the driver using select and non-blocking reads; > that way you would not need async threads at all. > > Kresten > > On Aug 12, 2010, at 10:31 , Mazen Harake wrote: > > It was late yesterday and despite 2 coffees and a > nice breakfast I > still can't solve this one. Maybe a nudge in the right > direction anyone? > > Scenario: > I have a linked in driver; cecho [1], when ever I want > to read input I > disable input to Erlang (-noinput) and use the getch() > function in C > instead (never mind why ;)). This function is run in a > seperate async > thread using driver_async() [2][3], and blocks at > getch(). When a > character is read it sends a message back to Erlang > (driver_output()) > and then the thread exits, when the message comes back > to Erlang the > character is sent to the requester and the thread is > restarted (which > again blocks at getch()). > > Later if I do an erlang:halt(0) ("graceful halt") the > thread is still > lingering despite having closed the port[4] (which, > Afaik should close > the thread as well) but it seems stay alive somehow. > erlang:halt(1) kill > it immediately so it is not a problem. > > Questions: > 1) Is my "analysis" correct? Is this infact what is > happening? Anyone > who can explain this behaviour (I.e. why it lingers on)? > > 2) If Question 1 then; what is the best way to kill > the thread when I > exit? driver_async_cancel() only works for threads > that are not > executing (according to Documentation) so what to do? > I did an ugly > workaround which sets a global flag and then I timeout > the getch() > operation and read for ERR. If I get an ERR I just > read the abort flag > and if it is set I just die otherwise I try to getch() > again. This > avoids hanging around for more then a tenth of a > second but it is pretty > ugly in my oppinion and I'm forced to go into > halfdelay mode which means > I can not allow this option outside of the driver > because I become > dependent on it. I'd rather do it the "right way" if > there is such a way. > > References: (Links given as commit Ids so that if > someone searches then > they get relevant code and not the latest) > [1] > http://github.com/mazenharake/cecho/tree/199fd342ba186e1154c32dbfdcf2d9222e25f9f4 > > > [2] > http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/c_src/cecho.c#L119 > > static void request(ErlDrvData drvstate, char > *buf, int buflen) { > state *st = (state *)drvstate; > driver_async(st->drv_port, NULL, loop_getch, (void > *)st, NULL); > } > > [3] > http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/c_src/cecho.c#L557 > > void loop_getch(void *arg) { > state *st = (state *)arg; > ei_x_buff eixb; > int keycode; > ei_x_new_with_version(&eixb); > keycode = getch(); > integer(&eixb, keycode); > driver_output(st->drv_port, eixb.buff, eixb.index); > } > > [4] > http://github.com/mazenharake/cecho/blob/199fd342ba186e1154c32dbfdcf2d9222e25f9f4/src/cecho_srv.erl#L76 > > terminate(_Reason, State) -> > do_call(State#state.port, ?ENDWIN), > do_call(State#state.port, ?CURS_SET, ?ceCURS_NORMAL), > erlang:port_close(State#state.port), > erl_ddll:unload("cecho"). > > > ________________________________________________________________ > erlang-questions (at) erlang.org > mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; > mailto:erlang-questions-unsubscribe@REDACTED > > > Kresten Krab Thorup, CTO, Trifork > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From mazen.harake@REDACTED Fri Aug 13 18:32:07 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Fri, 13 Aug 2010 19:32:07 +0300 Subject: [erlang-questions] Stopping of Erlang app hangs when mnesia stopped from within the program In-Reply-To: References: Message-ID: <4C657387.1020801@erlang-solutions.com> The application controller is already busy trying to stop your app, when you do mnesia:stop() I suspect it is trying to call the application controller to stop it but the application controller is already busy trying to stop your app, when you do mnesia:stop() I suspect it is trying to call the application controller to stop it but the application controller is already busy trying to stop your app. :) Use a boot file to handle startup of your applications or a function which starts mnesia and then starts your app (and the reverse in a stop function). > Should I not use these prototype functions within a fully fledged erlang app? No you shouldn't /Mazen On 13/08/2010 17:02, Ken Robinson wrote: > Hi all, > > I've run into a problem where I cannot stop mnesia within my program > without causing the app to hang. > > I'm presently doing prototyping of mnesia within my erlang app. > > In my jaus_app.erl file the start() calls: > > {atomic, ok} = mnesia:load_textfile("priv/mnesia_prototype.txt") > > My stop() function calls: > > mnesia:dump_to_textfile("priv/mnesia_prototype_res.txt"), > mnesia:stop(), > > When I comment out these lines and start and stop mnesia from the > erlang prompt, I am able to stop my application cleanly. > > Should I not use these prototype functions within a fully fledged erlang app? > > Ken. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From mevans@REDACTED Fri Aug 13 20:13:16 2010 From: mevans@REDACTED (Evans, Matthew) Date: Fri, 13 Aug 2010 14:13:16 -0400 Subject: [erlang-questions] Faster ets:lookups In-Reply-To: References: Message-ID: What sort of results are you getting? In most tests that I have been doing ETS is as fast, if not faster than C++ map. I do know that a table of type bag is a bit slower than set or ordered set. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of tom kelly Sent: Friday, August 13, 2010 6:20 AM To: erlang-questions@REDACTED Subject: [erlang-questions] Faster ets:lookups Hi List, I've just been reading Scott Lystig Fritchies paper "A Study of Erlang ETS Table Implementations and Performance" and was wondering whatever became of the "judy" option for ets table types. It's not in any OTP release I've worked with (R11 onwards) even though the research was done on R9, I assume this was considered but not accepted by OTP, anyone know the reasons? I found this when looking for ways to improve our cache system, we uses ets:lookup very intensively (table contains ~7000 entries indexed by a tuple of two integers representing an adler, table is type "bag") and our profiling has shown that the lookup uses the largest proportion of our processing time. Does anyone have any suggestions on how to optimise the lookup? Thanks in advance! //Tom. From erlang@REDACTED Fri Aug 13 22:36:49 2010 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 13 Aug 2010 22:36:49 +0200 Subject: [erlang-questions] Re: OOP in Erlang In-Reply-To: <4C64FFCE.4080301@erlang-solutions.com> References: <611396.69573.qm@web120515.mail.ne1.yahoo.com> <1858bc4d-497b-4953-b083-f0d4df21f9d5@k10g2000yqa.googlegroups.com> <4C64FFCE.4080301@erlang-solutions.com> Message-ID: Interesting discussion.... The term OO seems to mean different things to different people. Since Alan Kay invented the term I think what he said should bear some weight. On several occasions Alan said that OOP was all about messaging not "all that other stuff". Let me give you a few quotes: Quote 1: OOP to me means only messaging, local retention and protection and hiding of state-process, and extreme late-binding of all things. It can be done in Smalltalk and in LISP. There are possibly other systems in which this is possible, but I'm not aware of them. Ref: http://userpage.fu-berlin.de/~ram/pub/pub_jf47ht81Ht/doc_kay_oop_en Quote 2: Folks -- Just a gentle reminder that I took some pains at the last OOPSLA to try to remind everyone that Smalltalk is not only NOT its syntax or the class library, it is not even about classes. I'm sorry that I long ago coined the term "objects" for this topic because it gets many people to focus on the lesser idea. The big idea is "messaging" -- that is what the kernal of Smalltalk/Squeak is all about Ref: http://lists.squeakfoundation.org/pipermail/squeak-dev/1998-October/017019.html Quote 3: ... In other words, always having real objects always retains the ability to simulate anything you want, and to send it around the planet. If you send data 1000 miles you have to send a manual and/or a programmer to make use of it. If you send the needed programs that can deal with the data, then you are sending an object (even if the design is poor). And RCATWD also provides perfect protection in both directions. We can see this in the hardware model of the Internet (possibly the only real object-oriented system in working order). You get language extensibility almost for free by simply agreeing on conventions for the message forms. My thought in the 70s was that the Internet we were all working on alongside personal computing was a really good scalable design, and that we should make a virtual internet of virtual machines that could be cached by the hardware machines. It?s really too bad that this didn?t happen. Ref: http://www.computerworld.com.au/article/352182/z_programming_languages_smalltalk-80 --- this last quote is from an interview in computer world in Australia. (read the full article it is full of wisdom It is pretty clean that when Alan dreamt up the idea of OO it was "all about messaging" - he says this time and time again. I talked to Dan Ingalls about this and he was in complete agreement. He (Dan) said in a lecture (paraphrased) "the great thing about messaging is that is decouples the sender from the receiver" In a sense the receiver of a message does not care who sent the message, and the sins of the sender are not visited on the receiver of the message. This is the key point. If your program can crash my program they we are totally screwed. This should never ever happen. never ever ever ever ever ever ... (repeat zillions of times). This is the property of isolation. The mechanism of isolation is message passing. (note: asynchronous MP - synchronous MP breaks isolation since the receiver could block the caller by refusing to read a message) Note also that communication though messaging is the easiest way possible to glue things together - think Unix pipes. (aside: read this: http://cm.bell-labs.com/cm/cs/who/dmr/mdmpipe.html this is Doug McLlroys memo to Dennis Richie and Ken Thompson quote: Summary--what's most important. To put my strongest concerns into a nutshell: 1. We should have some ways of connecting programs like garden hose--screw in another segment when it becomes when it becomes necessary to massage data in another way. This is the way of IO also. ... ) Wonderful stuff - "like a garden hose" ... "screw in another segment" ... This is *precisely* what Alan Kay was talking about. There is a stong conceptual link between Alan's ideas on messaaging and isolation and McIlroys pipes - they are the same idea in a different guise - Unix processes connected though pipes and sockets *are* communiction objects (by Kay's definitions) So we have the pipes (garden hoses) that pass messages - that is the big idea. Polymorphism - (all objects respond to a asString message) is there so make programming easy - so we can guess the names of things. All the rest - organisation into classes and objects has zilch do do with the central concept of OOP. As Alan Kay said, this was " the less idea " (his words). This is "just" how you organise code (this is not to belittle the point - but remember the *big* idea is messaging ) To me the Erlang angle has always been that of making things fault tolerant. "my stuff should not crash your stuff" - this necessitates message passing passing systems. (ie you must have two or more machines). In conclusion Erlang is very much an OOPL - since deep down, very deep down, it tries to make makes a passable job of getting message passing and isolation right. In this sense it is far more OO than languages like Java (etc.) that have no notion of isolation and thus cannot be considered OOPLs :-) /Cheers Joe 2010/8/13 Ulf Wiger : > ??? (Yau-Hsien Huang) wrote: >> >> Recently I read the book >> "Design Patterns Explained," which hints that OOP/OOD is to treat >> everything, including primitive datatypes, as object. >> And he explained how abstraction >> is different from implementation. Then I thought that Erlang >> treats everything as function or process, but really? > > Yeah, well, I've had some pretty interesting discussions with > proponents of that view, and I would rather say that it is > possible to deal with abstractions as wholly separate from > implementation, but it is wrong to say that it holds in general > (at least for any useful definition of implementation). > > The idea, as I understand it, is that you want to adapt to > the common pattern that you have separate teams that handle > requirements, architecture, and programming (oh, and yet > another team doing testing, usually). I am very well familiar > with that way of working, and will go as far as saying that > it can sometimes be extremely difficult to avoid. > > The real danger with that setup, and the viewing of abstraction > as somehow separate from implementation, is that you tend to > to do "abstraction" for its own sake. The quotation marks are > there because in many cases, people confuse drawing visual > symbols with abstraction; while it may be in some dimension, > it is not abstraction in any useful sense for a programmer. > > > "The purpose of abstraction is not to be vague, but to create a > new semantic level in which one can be absolutely precise." > Edsger W. Dijkstra, "The Humble Programmer", October 1972 > > > To do abstraction in the way Dijkstra intends, you should have > a very good intuition about implementation issues. You might > say that abstraction grows from implementation. > > The key here is that you need to *pick* your abstractions > carefully based on your understanding of the domain *and* > details, opportunities and limitations of the implementation > layer. Erlang provides some abstractions that are a bit > unusual, but have been found in practice to work exceedingly > well in some domains. > > Some of those abstractions tend to be sub-optimal if your > aim is to write computational code that goes blazingly > fast. OTOH, some of the abstractions that have traditionally > fared well in that regard are suffering now, as the > hardware becomes increasingly parallel - something that > happens to suit Erlang just fine. > > Programming is all about abstraction - layers on top of > layers of abstraction, reaching all the way deep down into > the hardware your program runs on. > >> Abstraction: it's separated from implementation. Abstraction >> is done in Erlang by hot code replacement. > > No. Abstraction is when you make a model of something by > hiding underlying details. Abstraction is all over the place > in programming. Any Erlang data type is an abstraction - as > Erlang doesn't allow you do address memory directly, you > cannot control how many bits are assigned, how the data is > laid out, where it is, whether it is shared, etc. Nor are you > supposed to care. > > "Programming is both a top down activity from requirements and > overall structure of the system and a bottom up activity based > on the abstractions provided by the programming language. > Abstractions such as modules, processes, higher order > functions, etc. are to the programmer like transistors, > capacitors, resistors, etc. to the hardware designer. > It is important that the abstractions be few, simple to > understand and yet powerful and provide the needed > functionality. For example, if the application is inherently > concurrent it would be very dfficult to program without > some process concept. In that case the application program > itself would have to include some form of scheduler. > Distribution, error handling, and hot code loading are > extremely complicated requirements and the support for them > provided by Erlang enables the programmer to concentrate on > the application rather than on the basic programming technology." > > (Bjarne Dacker, http://www.erlang.se/publications/bjarnelic.pdf, > pg 26) > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > http://www.erlang-solutions.com > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From erlang@REDACTED Fri Aug 13 22:44:58 2010 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 13 Aug 2010 22:44:58 +0200 Subject: [erlang-questions] Different code layout examples In-Reply-To: References: Message-ID: On Thu, Aug 12, 2010 at 2:30 PM, Brian Williams wrote: > Are there any examples out there of different code layouts that may > (or may not) look better to someone confused by the punctuation? > Brian > > On Thu, Aug 12, 2010 at 3:25 AM, Deryk Barker wrote: >> Geoffrey Biggs wrote: >>> >>> I've never really had the problems that others seem to with Erlang's >>> punctuation. I've always thought of an Erlang function as like a >>> sentence. Just like in sentences, you use different punctuation to >>> separate the different parts. When you see the full stop, the sentence >>> ends. A single line is not necessarily a single, self-contained >>> "statement". >>> >> >> Hear, hear! I agree. Commas as separators, are pretty common. We write functions calls thusly: foo(a,b,c) and not: foo(a,b,c,) And English clauses, with a trailing comma at the end of a sentence, would be very ugly,. /Joe >> >> I've always found (well-laid out) erlang code to be particularly elegant. >> >> deryk >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > > > -- > Brian E. Williams > mixolyde@REDACTED > http://www.techhouse.us/wordpress-mu/brianw > > "Never attribute to malice that which can be adequately > explained by stupidity." - Hanlon's Razor > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tony.arcieri@REDACTED Fri Aug 13 22:58:51 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Fri, 13 Aug 2010 14:58:51 -0600 Subject: [erlang-questions] Re: OOP in Erlang In-Reply-To: References: <611396.69573.qm@web120515.mail.ne1.yahoo.com> <1858bc4d-497b-4953-b083-f0d4df21f9d5@k10g2000yqa.googlegroups.com> Message-ID: I thought I'd note that what you're describing sounds a *lot* like how Reia implements immutable objects. However, Reia does not support instance-specific behavior. Polymorphism is only offered in the form of class-based inheritance. 2010/8/12 ??? (Yau-Hsien Huang) > On Wed, Aug 11, 2010 at 9:20 PM, Steve Davis < > steven.charles.davis@REDACTED > > wrote: > > > things going on > > here: > > 1) Data structures and > > 2) Transformations of data structures. > > > > > I agree. Mandatory features of OOP includes encapsulation, abstraction, > inheritance, and polymorphism. After reviewing this thread, opinions are > summarized that including > * if we do OOP in Erlang syntax, > * if features of OOP are able to be implemented in Erlang, > * and if exactly Erlang is an OOP language or OO language. > > Now knowledge of OOP is mixture of origin concepts, languages such as > Smalltalk or Java, and UML modelling methods. Recently I read the book > "Design Patterns Explained," which hints that OOP/OOD is to treat > everything, > including primitive datatypes, as object. And he explained how abstraction > is > different from implementation. Then I thought that Erlang treats everything > as > function or process, but really? And, Erlang is not toward OOP, so it may > not > be OOP-like. > > Let's consider how to implement OOP in Erlang, either using Erlang syntax > or realizing OOP system in Erlang. > > Encapsulation: values and functions should be organized as an object > structure tagged as an object name, belong to some class. It is done by > Erlang record syntax or as a module, and somehow messages are sent > to the object or to some member of the structure. Functions in a structure > may call other fields or functions in other structures. Encapsulation means > how an object is structured. Any object is generated by following a object > specification, the class. Object and class are implemented in the same > way: structure. > > Abstraction: it's separated from implementation. Abstraction is done in > Erlang > by hot code replacement. > > Inheritance: it should be done by taking objects by following > both superclass > and subclass specifications and then overriding the superclass object with > the subclass object, that is to merge base functions into extended > functions > effectively. Inheritance shall not be as copying of superclass object, but > keeping a newest copy of a extended function while keeping the reference to > its base function. Inheritance semantics shall be consistent. > > Polymorphism: it means that different objects, even if they are in the same > class, may behave separately. It's not a problem because Erlang has this > feature. > > On real OOP, I think that OOP is a feature, and somehow it's the problem > itself. > So, when talking about OOP in Erlang, you can have a try to write Erlang in > OOP style, however, Erlang need not be too OOP-like. > -- Tony Arcieri Medioh! A Kudelski Brand From steven.charles.davis@REDACTED Sat Aug 14 02:05:01 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Fri, 13 Aug 2010 17:05:01 -0700 (PDT) Subject: A matter of style? In-Reply-To: <4C6565BE.7060003@volny.cz> References: <4C6565BE.7060003@volny.cz> Message-ID: <2a49f209-d16c-4d0f-b0be-29442848aae3@w30g2000yqw.googlegroups.com> > I?aki Garay wrote: > which I dislike for personal reasons. It's not just you, and I think there's more to it than just a personal dislike. I never use the if construct, preferring case or pattern matching on the function head. /s From zeno490@REDACTED Sat Aug 14 02:23:36 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Fri, 13 Aug 2010 20:23:36 -0400 Subject: [erlang-questions] shared data areas (was Re: [erlang-questions] OOP in Erlang) In-Reply-To: <4C63AA58.3040301@erlang-solutions.com> References: <4C627A5F.9060802@erlang-solutions.com> <4C63AA58.3040301@erlang-solutions.com> Message-ID: Co-locating processes on a shared heap on 1 scheduler might be dangerous. What if a process migrates to another core/scheduler? Should that be allowed? Blocked? Should you pay the cost of a largely uncontested lock (azul claims it is almost free)? What if one of those processes forks another one? Should it belong to the flock and be bound to that scheduler? Should it be allowed to migrate to another scheduler? Should the message it is receiving dictate the scheduler it is going to process it on? (ie: on receive, check where the message originates from (heap wise) and force the process to reschedule on the proper scheduler so it can have access to the memory) etc. I think it will be quite hard to do transparently (sharing msg data between processes without copying) mainly because it might be hard to tell data that is a msg from data that isn't. If a process sends a message then dies, you'll have to either: allocate the msg from the shared heap, somehow knowing or copying, or copy it in case the process heap gets garbage collected/freed (or prevent that from happening, which could have a detrimental effect on the system if the receiving process has a very large queue and takes a very long time before consuming said message) This could be mitigated by adding a special syntax for message creation where they would be created on a shared heap (or a special syntax for shared heap allocation). Perhaps something like {Foo} = Whatever (or Foo }= Whatever.). Where 'Foo' would be copied on the shared heap. Binaries are already reference counted so it might not be too bad to implement. IMO though, the whole sending a large datastructure as a message is largely a non-issue. You can easily wrap it in a process and allocate it from that processe's heap and just pass the PID around. Sure you have to pay an overhead for accessing said data through messages but if you are clever, that isn't too bad. If you are going to communicate accross nodes, you don't have much of a choice anyway: either you copy to the other node or you pass the PID and pay the price of cross node communication. That will be largely determined by your overall design. How much data can you afford to transfer? How likely are you to query from it? What if the data node goes down? etc. The only case requiring shared memory i've run into (ever, programming, with or without erlang) is large number crunching. You either need to access large amounts of data concurrently or you need to access non trivial amounts of data from many processes/threads. If that is what you need to do, I would advice against using erlang (as much as I like it). You'll be better off making a C++/C#/Java node for those, or a driver. Chances are if you need to do number crunching or process large amounts of data you'll care about cache alignment and such and erlang isn't the tool for that particular job. I guess the only other viable alternative would be to have all processes allocated from a shared heap and implement a concurrent GC. That would make sharing messages quite easy as you would be guaranteed that a message has a live reference somewhere (the sender, the receiver or both). This would however impact just about every system out there and could be quite risky... Then again, concurrent GCs are the new hot stuff nowadays and erlang is in dire need of modernizing. Anyhow, 2cents. I like the current erlang design and probably wouldn't change it much if i could. Nicholas On Thu, Aug 12, 2010 at 4:01 AM, Ulf Wiger wrote: > Fred Hebert wrote: > >> >> On Wed, Aug 11, 2010 at 8:44 AM, Jesper Louis Andersen < >> jesper.louis.andersen@REDACTED > >> wrote: >> >> >> There is one idea here I have been toying with. One problem of Erlangs >> memory model is that sending a large datastructure as a capability to >> another process, several megabytes in size, will mean a copy. In the >> default VM setup that is. But if you had a region into which it got >> allocated, then that region could safely be sent under a proof that >> the original process will not touch it anymore. [...] >> >> One interesting point of *always* copying data structures is that you need >> to plan for small messages (as far as possible) whether you are on a single >> node or in a distributed setting. Moving up from a [partially] shared memory >> model to a fully isolated one when going distributed is likely going to have >> its share of performance problems and might create a dissonance between >> "what is acceptable locally" and "what is acceptable when distributed". >> > > So a number of different variations on this theme have been tried > in the past and discussed as future extensions: > > - Robert Virding used to have his own implementation called VEE. > It had a global shared heap and incremental GC. > > - A 'shared heap' option in BEAM once had experimental status. > It passed messages by reference. Note that in neither of these > cases is there any change in semantics - conceptually, msg > passing was still copying. The main problem with this version > was that it still used the old copying garbage collector. The > idea was to implement a reference-counting GC, but for various > reasons, it didn't happen. When multicore started becoming > interesting, the shared-heap version was left behind. > > - Hybrid heap was an evolution of 'shared heap', where only > data sent in messages were put on a global heap. In the first > implementation, data was copied to the global heap on send > (unless already there) instead of being copied to the receiver's > heap. This implementation was also broken by SMP. > > - Lately, some exploration has gone into allowing a set of > processes to share the same heap. This could be done in (at > least) two ways: > a) either co-locate all processes in the group on the same > scheduler. This would ensure mutual exclusion and mainly > serve to reduce message-passing cost in a process group. > b) allow processes in a group to run on different schedulers, > using mutexes to protect accesses to the heap data. This > could allow for parallel processing, but the locking > granularity would either be heap-level or ...very subtle, > I guess. I favour option (a). > > I think it is appropriate to use under-the-cover tricks to > speed up message passing as much as possible, as long as the > semantics stay the same. In other words, in all the above cases, > isolation has been a given, and conceptually, messages are > still copied. > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > http://www.erlang-solutions.com > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From dave.smith.to@REDACTED Sat Aug 14 05:25:10 2010 From: dave.smith.to@REDACTED (Dave Smith) Date: Fri, 13 Aug 2010 23:25:10 -0400 Subject: [erlang-questions] Re: OOP in Erlang In-Reply-To: References: <611396.69573.qm@web120515.mail.ne1.yahoo.com> <1858bc4d-497b-4953-b083-f0d4df21f9d5@k10g2000yqa.googlegroups.com> <4C64FFCE.4080301@erlang-solutions.com> Message-ID: Yes I agree, The lack of isolation is where the the big names of OOP went wrong; the likes of C++, Java, C# and JavaScript provide ways for state to 'leak out' of objects by means of pointers and references. What good is encapsulation if member variables are bound to shared objects. One can avoid this problem if he goes our of his way, but in practice, this is rarely done. Strict encapsulation should be the rule, not an options. Arguing that encapsulation is a virtue with respect to these languages seems very shallow indeed. Respectfully, I would not go as far as Joe to say that OO is a poor paradigm for modelling real world applications, as I've heard him argue in the past. I've made a career out of object modelling and writing software in C++ and Java -- it's very much the way I approach software design. The inheritance concept, on the other hand is crucial to OO design; not so much for the inherited functionality as for type polymorphism. Other languages provide this using algebraic data types and other distinguished unions, but in OO we work with classes, and we need polymorphic types to avoid the deluge of conditionals that we would see otherwise. --DS 2010/8/13 Joe Armstrong > ery much an OOPL - since deep down, very > deep down, it tries to make makes a passable job of getting message > passing and isolation right. In this sense it is far more OO than > languages like Java (etc.) that have no notion of isolation and thus > cannot be considered OO > From michael.eugene.turner@REDACTED Sat Aug 14 08:58:04 2010 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 14 Aug 2010 15:58:04 +0900 Subject: [erlang-questions] Re: OOP in Erlang In-Reply-To: References: <611396.69573.qm@web120515.mail.ne1.yahoo.com> <1858bc4d-497b-4953-b083-f0d4df21f9d5@k10g2000yqa.googlegroups.com> <4C64FFCE.4080301@erlang-solutions.com> Message-ID: "The inheritance concept, on the other hand is crucial to OO design; not so much for the inherited functionality as for type polymorphism. Other languages provide this using algebraic data types and other distinguished unions, but in OO we work with classes, and we need polymorphic types to avoid the deluge of conditionals that we would see otherwise." With all due respect, when I boiled this paragraph down to its essentials, it read to me like this: "The definition of 'object-oriented design' entails inheritance, because that's the way 'object-oriented' was defined." There's a particularly strenuous force-fitting of this definition in one of the comments on that Alan Kay interview -- it's first admitted that Alan Kay coined the term and contributed a lot of ideas, but then flatly asserted that object-orientation was defined by the architects of Simula -- even though Simula 67 lacked encapsulation (no class variables, even -- you had to use globals) and even though Simula 67's treatments of types and inheritance probably doesn't meet some people's definitions of polymorphism. There's a Humpty Dumpty feeling about a lot of these debates. ISTR a similar one, almost two decades ago, about whether C++ was truly "polymorphic" despite being strongly typed. It turned out that "polymorphic" had come to mean different things to different people, that there was no hard-and-fast formal definition. Whether you were going to "win" such arguments often depended on your staying power in debate. I think the Erlang community would do itself a world of good by finding somebody to go knock on Alan Kay's door in Glendale and spend 15 minutes with him showing how Erlang does things. As far as I can tell, he's never said anything about the language. (Perhaps because he's hardly heard of it. After all, if there's one thing the Erlang community seems dismayingly bad at, it's self-promotion.) If Alan Kay said, "Erlang looks like almost what I've been talking about since the mid-60s, I wonder how I missed it for so many years?" I think some of you might be surprised by how many highly influential people in the programming world would sit up and take notice. -michael turner 2010/8/14 Dave Smith > Yes I agree, The lack of isolation is where the the big names of OOP went > wrong; the likes of C++, Java, C# and JavaScript provide ways for state to > 'leak out' of objects by means of pointers and references. What good is > encapsulation if member variables are bound to shared objects. One can > avoid this problem if he goes our of his way, but in practice, this is > rarely done. Strict encapsulation should be the rule, not an options. > Arguing that encapsulation is a virtue with respect to these languages > seems > very shallow indeed. > > Respectfully, I would not go as far as Joe to say that OO is a poor > paradigm for modelling real world applications, as I've heard him argue in > the past. I've made a career out of object modelling and writing software > in C++ and Java -- it's very much the way I approach software design. > > The inheritance concept, on the other hand is crucial to OO design; not so > much for the inherited functionality as for type polymorphism. Other > languages provide this using algebraic data types and other distinguished > unions, but in OO we work with classes, and we need polymorphic types to > avoid the deluge of conditionals that we would see otherwise. > > --DS > > > 2010/8/13 Joe Armstrong > > > ery much an OOPL - since deep down, very > > deep down, it tries to make makes a passable job of getting message > > passing and isolation right. In this sense it is far more OO than > > languages like Java (etc.) that have no notion of isolation and thus > > cannot be considered OO > > > From bgustavsson@REDACTED Sat Aug 14 09:18:57 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sat, 14 Aug 2010 09:18:57 +0200 Subject: [erlang-questions] A matter of style? In-Reply-To: References: Message-ID: On Fri, Aug 13, 2010 at 5:11 PM, I?aki Garay wrote: > within(X, Y, Z, Xmin, Ymin, Zmin, Xmax, Ymax, Zmax) -> > ? ?(Xmin =< X) and > ? ?(X =< Xmax) and > ? ?(Ymin =< Y) and > ? ?(Y =< Ymax) and > ? ?(Zmin =< Z) and > ? ?(Z =< Zmax). Written like this, all conditions will be evaluated every time, even if a condition has evaluated to false. To ensure that the function returns false as soon as one condition evaluates to false, rewrite it like this: within(X, Y, Z, Xmin, Ymin, Zmin, Xmax, Ymax, Zmax) -> Xmin =< X andalso X =< Xmax andalso Ymin =< Y andalso Y =< Ymax andalso Zmin =< Z andalso Z =< Zmax. Alternately, it can be written like this: within(X, Y, Z, Xmin, Ymin, Zmin, Xmax, Ymax, Zmax) when Xmin =< X, X =< Xmax, Ymin =< Y, Y =< Ymax, Zmin =< Z, Z =< Zmax -> true; within(_, _, _, _, _, _, _, _, _) -> false. In general, boolean expressions in guards will be short-circuited unless that would break the semantics. For instance, in this function f(A, B) when (length(A) > 5) or (length(B) > 10) -> ok. both length(A) and length(B) will always be evaluated. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From tony.arcieri@REDACTED Sat Aug 14 10:11:07 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Sat, 14 Aug 2010 02:11:07 -0600 Subject: [erlang-questions] Re: OOP in Erlang In-Reply-To: References: <611396.69573.qm@web120515.mail.ne1.yahoo.com> <1858bc4d-497b-4953-b083-f0d4df21f9d5@k10g2000yqa.googlegroups.com> <4C64FFCE.4080301@erlang-solutions.com> Message-ID: 2010/8/13 Joe Armstrong > Erlang is very much an OOPL - since deep down, very > deep down, it tries to make makes a passable job of getting message > passing and isolation right. In this sense it is far more OO than > languages like Java (etc.) that have no notion of isolation and thus > cannot be considered OOPLs :-) I strongly agree with this, except when it comes to managing sequential, immutable state. The mechanisms Erlang provides, namely records and paramaterized modules, kind of suck. I think some type of object-like thing could really benefit Erlang in that regard. I've implemented such a thing (immutable objects) in Reia. -- Tony Arcieri Medioh! A Kudelski Brand From attila.r.nohl@REDACTED Sat Aug 14 10:58:30 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Sat, 14 Aug 2010 10:58:30 +0200 Subject: [erlang-questions] Different code layout examples In-Reply-To: References: Message-ID: 2010/8/13, Joe Armstrong : [...] > And English clauses, with a trailing comma at the end of a sentence, > would be very ugly,. On the other hand you don't leave the full stop from the end of the sentence at the end of the paragraph... From emailos@REDACTED Sat Aug 14 11:17:48 2010 From: emailos@REDACTED (Mikkel Jensen) Date: Sat, 14 Aug 2010 11:17:48 +0200 Subject: Common test: ct_ssh usage? Message-ID: Hi Im having a hard time using ct_ssh to just create a simple connection. This shouldn't be hard, but for some reason it is. I have been using http://www.erlang.org/doc/man/ct_ssh.html as the base of my efforts. My attempts at making a connection is shown below. If anyone could give me any tips or pointers to get me started, it would be much appreciated! 1. -module(simple). 2. -export([run/0]). 3. 4. -import(ct_ssh). 5. -import(ct_util). 6. 7. run() -> 8. ok = ct:require(mynodename,{sec,[ssh,"127.0.0.1"]}), 9. {ok, Handle} = ct_ssh:connect(mynodename), 10. Handle. 11. 12. 13. ERROR: 14. ------ 15. 16. 1> simple:run(). 17. ** exception error: bad argument 18. in function ct_util:call/1 19. in call from simple:run/0 20. 2> crypto:start(). 21. ok 22. 3> ssh:start(). 23. ok 24. 4> simple:run(). 25. ** exception error: bad argument 26. in function ct_util:call/1 27. in call from simple:run/0 28. 1. -module(solossh). 2. -export([soloconnect/0]). 3. -import(ct_ssh). 4. 5. soloconnect() -> 6. {ok, Handle} = ct_ssh:connect(host, ssh, [{ssh, "127.0.0.1"}, {port, "22"}, {user, "testserver"}, {password, "testpass"}]), 7. Handle. 8. 9. 10. error 11. ------ 12. 13. 1> solossh:soloconnect(). 14. ** exception error: bad argument 15. in function ets:select/2 16. called as ets:select(ct_attributes, 17. [{{ct_conf,'_','$2','$1',host,'_'},[],[{ {'$1','$2'}}]}]) 18. in call from ct_util:lookup_name/1 19. in call from ct_util:lookup_config/1 20. in call from ct_util:get_config/3 21. in call from ct_ssh:connect/3 22. in call from solossh:soloconnect/0 From ulf.wiger@REDACTED Sat Aug 14 16:45:57 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 14 Aug 2010 16:45:57 +0200 Subject: [erlang-questions] shared data areas (was Re: [erlang-questions] OOP in Erlang) In-Reply-To: References: <4C627A5F.9060802@erlang-solutions.com> <4C63AA58.3040301@erlang-solutions.com> Message-ID: <4C66AC25.8030309@erlang-solutions.com> Note: these are all hand-waving comments from me. Nicholas Frechette wrote: > Co-locating processes on a shared heap on 1 scheduler might be > dangerous. What if a process migrates to another core/scheduler? Should > that be allowed? Blocked? An implementation issue, but yes, I imagine it could be forbidden, or rather, the group would be considered as a whole for migration. > What if one of those > processes forks another one? Should it belong to the flock and be bound > to that scheduler? This could be at the parent's discretion, using a new spawn_opt option, e.g. 'same_heap', dictating that the child should use the parent's heap. > Should the message it is receiving dictate the scheduler it is going to > process it on? No, I think this would be complicated, and can't see the need for it. > I think it will be quite hard to do transparently (sharing msg data > between processes without copying) mainly because it might be hard to > tell data that is a msg from data that isn't. But a lot of thinking already went into this. Binaries are often off-heap, although in that case, you only need to look at the type tag to know. In the case I suggested, the only check needed is for whether the receiver is sharing the sender's heap. > If a process sends a > message then dies, you'll have to either: allocate the msg from the > shared heap, somehow knowing or copying, or copy it in case the process > heap gets garbage collected/freed (or prevent that from happening, which > could have a detrimental effect on the system if the receiving process > has a very large queue and takes a very long time before consuming said > message) To my uneducated eye, this seems fairly similar to normal process death + normal GC. If a process (not the last on that heap) dies, the extra problem is that some data can be "freed" that is still referenced by other processes. Something similar should have been necessary for the 'shared heap' emulator to work in the first place. > This could be mitigated by adding a special syntax for message creation > where they would be created on a shared heap (or a special syntax for > shared heap allocation). Perhaps something like {Foo} = Whatever (or Foo > }= Whatever.). Where 'Foo' would be copied on the shared heap. Binaries > are already reference counted so it might not be too bad to implement. IMHO - no language changes! An option to spawn_opt is the right place and scope. That's where you can peek under the covers and affect things like GC parameters, which are otherwise fully transparent. > IMO though, the whole sending a large datastructure as a message is > largely a non-issue. You can easily wrap it in a process and allocate it > from that processe's heap and just pass the PID around. But in my experience, from complex telephony and multimedia apps, it is in fact pretty common to have thousands of small 'clusters' of state machines, each cluster sharing a whopping big state record and taking turns processing different parts of the control flow. The suggestion is that this would work as an optimization for the tuning stage of such an application, much like it is now possible to tune heap size, GC thresholds, etc. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From erlang@REDACTED Sat Aug 14 17:22:56 2010 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 14 Aug 2010 17:22:56 +0200 Subject: [erlang-questions] Re: OOP in Erlang In-Reply-To: References: <611396.69573.qm@web120515.mail.ne1.yahoo.com> <1858bc4d-497b-4953-b083-f0d4df21f9d5@k10g2000yqa.googlegroups.com> <4C64FFCE.4080301@erlang-solutions.com> Message-ID: On Sat, Aug 14, 2010 at 8:58 AM, Michael Turner wrote: > "The inheritance concept, on the other hand is crucial to OO design; not > so?much for the ?inherited functionality as for type polymorphism. > ?Other?languages provide this using algebraic data types and other > distinguished?unions, but in OO we work with classes, and we need > polymorphic types to?avoid the deluge of conditionals that we would see > otherwise." > With all due respect, when I boiled this paragraph down to its essentials, > it read to me like this: "The definition of 'object-oriented design' entails > inheritance, because that's the way 'object-oriented' was defined." > There's a particularly strenuous force-fitting of this definition in one of > the comments on that Alan Kay interview -- it's first admitted that Alan Kay > coined the term and contributed a lot of ideas, but then flatly asserted > that object-orientation was defined by the architects of Simula -- even > though Simula 67 lacked encapsulation (no class variables, even -- you had > to use globals) and even though Simula 67's treatments of types and > inheritance probably doesn't meet some people's definitions of polymorphism. > There's a Humpty Dumpty feeling about a lot of these debates. ?ISTR a > similar one, almost two decades ago, about whether C++ was truly > "polymorphic" despite being strongly typed. ?It turned out that > "polymorphic" had come to mean different things to different people, that > there was no hard-and-fast formal definition. ?Whether you were going to > "win" such arguments often depended on your staying power in debate. > I think the Erlang community would do itself a world of good by finding > somebody to go knock on Alan Kay's door in Glendale and spend 15 minutes > with him showing how Erlang does things. ?As far as I can tell, he's never > said anything about the language. ?(Perhaps because he's hardly heard of it. > ?After all, if there's one thing the Erlang community seems dismayingly bad > at, it's self-promotion.) Erlang was developed in Sweden. In the Swedish culture self-promotion is considered extremely bad-manners. It's ok for *other people* to say nice things about your work, but not for you to do this, which is considered rude. For a Swede to even mention what they have done something is considered bad-manners and a violation of "Jantelagen" http://en.wikipedia.org/wiki/Jante_Law. Erlang evolved in a Swedish culture and has inherited this behaviour. In some cultures self-promotion is trained into children from birth, in Sweden the opposite is true, children are reprimanded for self-promotion. When adults of these two cultures meet cross-cultural confusion can occur. If somebody from a self-promotion culture meets a Sweden and says "this stuff I did is amazing" Swedes tend to literally believe this. Why is this, basically because Swedes will never say that what they did is good. If pressed they might admit to having done it at all (which is their way of saying it's great) so if they (a Swede) meets somebody who actually claims to have done something good they view this as such an exceptional circumstance that it must be true. They (the Swedes) realize that the person from the other culture is not bragging and do not apply Jantelagen to the foreigner but they *overcompensate* - since Swedes never say that what they did was great they lack a mechanism for calibrating what this means. In my experience if somebody from a self-effacing culture says they did something you should upgrade the significance of what they did, and if somebody from a self-promoting culture says something is great you should down-grade the significance. Janelagen is essential in understanding how Swedes reason - it is basically untranslatable. It really takes about five generations of living here and exchange of genetic material to understand. /Joe If Alan Kay said, "Erlang looks like almost what > I've been talking about since the mid-60s, I wonder how I missed it for so > many years?" I think some of you might be surprised by how many highly > influential people in the programming world would sit up and take notice. > -michael turner > 2010/8/14 Dave Smith >> >> Yes I agree, ?The lack of isolation is where the the big names of OOP went >> wrong; the likes of C++, Java, C# and JavaScript provide ways for state to >> 'leak out' of objects by means of pointers and references. ?What good is >> encapsulation if member variables are bound to shared objects. ?One can >> avoid this problem ?if he goes our of his way, but in practice, this is >> rarely done. ?Strict encapsulation should be the rule, not an options. >> Arguing that encapsulation is a virtue with respect to these languages >> seems >> very shallow indeed. >> >> ?Respectfully, I would not go as far as Joe to say that OO is a poor >> paradigm for modelling real world applications, as I've heard him argue in >> the past. ?I've made a career out of object modelling and writing software >> in ?C++ and Java -- it's very much the way I approach software design. >> >> The inheritance concept, on the other hand is crucial to OO design; not so >> much for the ?inherited functionality as for type polymorphism. ?Other >> languages provide this using algebraic data types and other distinguished >> unions, but in OO we work with classes, and we need polymorphic types to >> avoid the deluge of conditionals that we would see otherwise. >> >> --DS >> >> >> 2010/8/13 Joe Armstrong >> >> > ery much an OOPL - since deep down, very >> > deep down, it tries to make makes a passable job of getting message >> > passing and isolation right. ?In this sense it is far more OO than >> > languages like Java (etc.) that have no notion of isolation and thus >> > cannot be considered OO >> > > > From dave.smith.to@REDACTED Sat Aug 14 20:17:15 2010 From: dave.smith.to@REDACTED (Dave Smith) Date: Sat, 14 Aug 2010 14:17:15 -0400 Subject: [erlang-questions] Re: OOP in Erlang In-Reply-To: References: <611396.69573.qm@web120515.mail.ne1.yahoo.com> <1858bc4d-497b-4953-b083-f0d4df21f9d5@k10g2000yqa.googlegroups.com> <4C64FFCE.4080301@erlang-solutions.com> Message-ID: It wasn't my intention to try to define OO. Those of us who work in industry feel we know what a peer is talking about when he uses the term (irrespective of what was in Allan Kay's head when he coined the term 40 years ago). --DS On 14 August 2010 02:58, Michael Turner wrote: > "The inheritance concept, on the other hand is crucial to OO design; not > so much for the inherited functionality as for type polymorphism. > Other languages provide this using algebraic data types and other > distinguished unions, but in OO we work with classes, and we need > polymorphic types to avoid the deluge of conditionals that we would see > otherwise." > > With all due respect, when I boiled this paragraph down to its essentials, > it read to me like this: "The definition of 'object-oriented design' entails > inheritance, because that's the way 'object-oriented' was defined." > > There's a particularly strenuous force-fitting of this definition in one of > the comments on that Alan Kay interview -- it's first admitted that Alan Kay > coined the term and contributed a lot of ideas, but then flatly asserted > that object-orientation was defined by the architects of Simula -- even > though Simula 67 lacked encapsulation (no class variables, even -- you had > to use globals) and even though Simula 67's treatments of types and > inheritance probably doesn't meet some people's definitions of polymorphism. > > There's a Humpty Dumpty feeling about a lot of these debates. ISTR a > similar one, almost two decades ago, about whether C++ was truly > "polymorphic" despite being strongly typed. It turned out that > "polymorphic" had come to mean different things to different people, that > there was no hard-and-fast formal definition. Whether you were going to > "win" such arguments often depended on your staying power in debate. > > I think the Erlang community would do itself a world of good by finding > somebody to go knock on Alan Kay's door in Glendale and spend 15 minutes > with him showing how Erlang does things. As far as I can tell, he's never > said anything about the language. (Perhaps because he's hardly heard of it. > After all, if there's one thing the Erlang community seems dismayingly bad > at, it's self-promotion.) If Alan Kay said, "Erlang looks like almost what > I've been talking about since the mid-60s, I wonder how I missed it for so > many years?" I think some of you might be surprised by how many highly > influential people in the programming world would sit up and take notice. > > -michael turner > > 2010/8/14 Dave Smith > > Yes I agree, The lack of isolation is where the the big names of OOP went >> wrong; the likes of C++, Java, C# and JavaScript provide ways for state to >> 'leak out' of objects by means of pointers and references. What good is >> encapsulation if member variables are bound to shared objects. One can >> avoid this problem if he goes our of his way, but in practice, this is >> rarely done. Strict encapsulation should be the rule, not an options. >> Arguing that encapsulation is a virtue with respect to these languages >> seems >> very shallow indeed. >> >> Respectfully, I would not go as far as Joe to say that OO is a poor >> paradigm for modelling real world applications, as I've heard him argue in >> the past. I've made a career out of object modelling and writing software >> in C++ and Java -- it's very much the way I approach software design. >> >> The inheritance concept, on the other hand is crucial to OO design; not so >> much for the inherited functionality as for type polymorphism. Other >> languages provide this using algebraic data types and other distinguished >> unions, but in OO we work with classes, and we need polymorphic types to >> avoid the deluge of conditionals that we would see otherwise. >> >> --DS >> >> >> 2010/8/13 Joe Armstrong >> >> > ery much an OOPL - since deep down, very >> > deep down, it tries to make makes a passable job of getting message >> > passing and isolation right. In this sense it is far more OO than >> > languages like Java (etc.) that have no notion of isolation and thus >> > cannot be considered OO >> > >> > > From igarai@REDACTED Sat Aug 14 20:46:38 2010 From: igarai@REDACTED (=?UTF-8?B?ScOxYWtpIEdhcmF5?=) Date: Sat, 14 Aug 2010 15:46:38 -0300 Subject: [erlang-questions] A matter of style? In-Reply-To: References: Message-ID: Of course, how did I not think of that? Thank you all for the replies. 2010/8/14 Bj?rn Gustavsson : > On Fri, Aug 13, 2010 at 5:11 PM, I?aki Garay wrote: >> within(X, Y, Z, Xmin, Ymin, Zmin, Xmax, Ymax, Zmax) -> >> ? ?(Xmin =< X) and >> ? ?(X =< Xmax) and >> ? ?(Ymin =< Y) and >> ? ?(Y =< Ymax) and >> ? ?(Zmin =< Z) and >> ? ?(Z =< Zmax). > > Written like this, all conditions will be evaluated every time, > even if a condition has evaluated to false. > > To ensure that the function returns false as soon as one > condition evaluates to false, rewrite it like this: > > within(X, Y, Z, Xmin, Ymin, Zmin, Xmax, Ymax, Zmax) -> > ? Xmin =< X andalso > ? X =< Xmax andalso > ? Ymin =< Y andalso > ? Y =< Ymax andalso > ? Zmin =< Z andalso > ? Z =< Zmax. > > Alternately, it can be written like this: > > within(X, Y, Z, Xmin, Ymin, Zmin, Xmax, Ymax, Zmax) when > Xmin =< X, X =< Xmax, Ymin =< Y, > Y =< Ymax, Zmin =< Z, Z =< Zmax -> > ? ?true; > within(_, _, _, _, _, _, _, _, _) -> > ? ?false. > > In general, boolean expressions in guards will > be short-circuited unless that would break > the semantics. For instance, in this function > > f(A, B) when (length(A) > 5) or (length(B) > 10) -> > ? ?ok. > > both length(A) and length(B) will always be > evaluated. > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > -- I?aki Garay. -----BEGIN GEEK CODE BLOCK----- Version: 3.1 GCS/GMU d- s:- a25 C++ UL+++ P--- E- W+ PS+++ PE- Y+ t++ 5+ tv- b++ DI+ D--- G e h! r y++ ------END GEEK CODE BLOCK------ From ttom.kelly@REDACTED Sat Aug 14 20:46:53 2010 From: ttom.kelly@REDACTED (tom kelly) Date: Sat, 14 Aug 2010 19:46:53 +0100 Subject: [erlang-questions] Faster ets:lookups In-Reply-To: References: Message-ID: Hi Matthew, Maybe I should have said more about my scenario. I'm passing a ~30MB test file through my cache. For each byte I calculate a rolling adler for the 4K block that it is the start of and check my ets table to see if I already have it. If I get a hit I calculate the MD5 to make sure it's a genuine hit and if it is I can send a reference instead of the complete block. The table is a bag in case different blocks have the same adler. For the case I'm trying to optimise the file is not already in cache so I end up calculating ~30 million adlers and doing ~30 million lookups, which will all fail, on a table that grows to over 7,000 entries. Each entry contains an adler and MD5 of a 4k block of data stored elsewhere. Before I measured it I would have bet my house on the adler calculation taking most of the execution time, but it seems that the ets lookup takes just under three times as much! 4.5 seconds for the adler calculation, 12 seconds for the ets lookup. By now I don't really expect to improve on ets, I know it's just that our algorithm uses it so intensively. I'm just hoping beyond hope that maybe someone has a suggestion for a better way to keep and check this data? Our CTO isn't sold on Erlang yet and is considering re-writing the whole thing in C. I just can't let him win!! ;-) //Tom. On Fri, Aug 13, 2010 at 7:13 PM, Evans, Matthew wrote: > What sort of results are you getting? In most tests that I have been doing > ETS is as fast, if not faster than C++ map. I do know that a table of type > bag is a bit slower than set or ordered set. > > > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of tom kelly > Sent: Friday, August 13, 2010 6:20 AM > To: erlang-questions@REDACTED > Subject: [erlang-questions] Faster ets:lookups > > Hi List, > I've just been reading Scott Lystig Fritchies paper "A Study of Erlang ETS > Table Implementations and Performance" and was wondering whatever became of > the "judy" option for ets table types. It's not in any OTP release I've > worked with (R11 onwards) even though the research was done on R9, I assume > this was considered but not accepted by OTP, anyone know the reasons? > I found this when looking for ways to improve our cache system, we uses > ets:lookup very intensively (table contains ~7000 entries indexed by a > tuple > of two integers representing an adler, table is type "bag") and our > profiling has shown that the lookup uses the largest proportion of our > processing time. > Does anyone have any suggestions on how to optimise the lookup? > Thanks in advance! > //Tom. > From ttom.kelly@REDACTED Sat Aug 14 20:49:10 2010 From: ttom.kelly@REDACTED (tom kelly) Date: Sat, 14 Aug 2010 19:49:10 +0100 Subject: [erlang-questions] Common test: ct_ssh usage? In-Reply-To: References: Message-ID: Hi Mikkel, I've used ct_ssh in the past and found it to be an extremely useful wrapper for the ssh application! If I create a config file (and call it "ssh.config") for common test with the following entry: {nodeICanSshTo,[ {ssh,"127.0.0.1"}, {username,"myName"}, {password,"letMeIn"} ]}. Then from an erlang shell I can: 1> ct:install([{config,["ssh.config"]}]). ok 2> ct:start_interactive(). <0.43.0> 3> ct:get_config({nodeICanSshTo). [{ssh,"127.0.0.1"},{username,"myName"},{password,"letMeIn"}] 4> {ok,A} = ct_ssh:connect(nodeICanSshTo). {ok,<0.53.0>} 5> {ok,B} = ct_ssh:session_open(A). {ok,0} 6> If you're using ct:run there's no need for the "install" and "start_interactive" calls, I've just used interactive mode here for demonstration. Maybe you've got this far already? If this doesn't help include your config file data next time. //Tom. On Sat, Aug 14, 2010 at 10:17 AM, Mikkel Jensen wrote: > Hi > > Im having a hard time using ct_ssh to just create a simple connection. This > shouldn't be hard, but for some reason it is. I have been using > http://www.erlang.org/doc/man/ct_ssh.html as the base of my efforts. My > attempts at making a connection is shown below. > > If anyone could give me any tips or pointers to get me started, it would be > much appreciated! > > > 1. -module(simple). > 2. -export([run/0]). > 3. > 4. -import(ct_ssh). > 5. -import(ct_util). > 6. > 7. run() -> > 8. ok = ct:require(mynodename,{sec,[ssh,"127.0.0.1"]}), > 9. {ok, Handle} = ct_ssh:connect(mynodename), > 10. Handle. > 11. > 12. > 13. ERROR: > 14. ------ > 15. > 16. 1> simple:run(). > 17. ** exception error: bad argument > 18. in function ct_util:call/1 > 19. in call from simple:run/0 > 20. 2> crypto:start(). > 21. ok > 22. 3> ssh:start(). > 23. ok > 24. 4> simple:run(). > 25. ** exception error: bad argument > 26. in function ct_util:call/1 > 27. in call from simple:run/0 > 28. > > > > > 1. -module(solossh). > 2. -export([soloconnect/0]). > 3. -import(ct_ssh). > 4. > 5. soloconnect() -> > 6. {ok, Handle} = ct_ssh:connect(host, ssh, [{ssh, "127.0.0.1"}, > {port, "22"}, {user, "testserver"}, {password, "testpass"}]), > 7. Handle. > 8. > 9. > 10. error > 11. ------ > 12. > 13. 1> solossh:soloconnect(). > 14. ** exception error: bad argument > 15. in function ets:select/2 > 16. called as ets:select(ct_attributes, > 17. [{{ct_conf,'_','$2','$1',host,'_'},[],[{ > {'$1','$2'}}]}]) > 18. in call from ct_util:lookup_name/1 > 19. in call from ct_util:lookup_config/1 > 20. in call from ct_util:get_config/3 > 21. in call from ct_ssh:connect/3 > 22. in call from solossh:soloconnect/0 > From vances@REDACTED Sat Aug 14 21:23:32 2010 From: vances@REDACTED (Vance Shipley) Date: Sat, 14 Aug 2010 15:23:32 -0400 Subject: [erlang-questions] Re: OOP in Erlang Message-ID: <20100814192331.GV274@h216-235-12-174.host.egate.net> If you really want inheritance you can always build your own: http://www.trapexit.org/Cascading_Behaviours :) -- -Vance From zeno490@REDACTED Sat Aug 14 21:52:25 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Sat, 14 Aug 2010 15:52:25 -0400 Subject: [erlang-questions] Faster ets:lookups In-Reply-To: References: Message-ID: With that scenario, even if you were to rewrite the whole thing in C++, you'd find that the lookup into your hash bag or whatever structure you'll use will take the most time, as it does in erlang. Short of having a O(1) lookup, any language will exhibit this. Perhaps you would get better luck tuning your algorithm differently (ie: 64k blocks instead of 4k) or simply trying a new one altogether. You could also look into parallelizing the whole thing and splitting your ets table and merging the results later, removing duplicates. Nicholas On Sat, Aug 14, 2010 at 2:46 PM, tom kelly wrote: > Hi Matthew, > > Maybe I should have said more about my scenario. > > I'm passing a ~30MB test file through my cache. For each byte I calculate a > rolling adler for the 4K block that it is the start of and check my ets > table to see if I already have it. If I get a hit I calculate the MD5 to > make sure it's a genuine hit and if it is I can send a reference instead of > the complete block. The table is a bag in case different blocks have the > same adler. > > For the case I'm trying to optimise the file is not already in cache so I > end up calculating ~30 million adlers and doing ~30 million lookups, which > will all fail, on a table that grows to over 7,000 entries. Each entry > contains an adler and MD5 of a 4k block of data stored elsewhere. > > Before I measured it I would have bet my house on the adler calculation > taking most of the execution time, but it seems that the ets lookup takes > just under three times as much! 4.5 seconds for the adler calculation, 12 > seconds for the ets lookup. > > By now I don't really expect to improve on ets, I know it's just that our > algorithm uses it so intensively. I'm just hoping beyond hope that maybe > someone has a suggestion for a better way to keep and check this data? > > Our CTO isn't sold on Erlang yet and is considering re-writing the whole > thing in C. I just can't let him win!! ;-) > > //Tom. > > > On Fri, Aug 13, 2010 at 7:13 PM, Evans, Matthew > wrote: > > > What sort of results are you getting? In most tests that I have been > doing > > ETS is as fast, if not faster than C++ map. I do know that a table of > type > > bag is a bit slower than set or ordered set. > > > > > > > > -----Original Message----- > > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] > On > > Behalf Of tom kelly > > Sent: Friday, August 13, 2010 6:20 AM > > To: erlang-questions@REDACTED > > Subject: [erlang-questions] Faster ets:lookups > > > > Hi List, > > I've just been reading Scott Lystig Fritchies paper "A Study of Erlang > ETS > > Table Implementations and Performance" and was wondering whatever became > of > > the "judy" option for ets table types. It's not in any OTP release I've > > worked with (R11 onwards) even though the research was done on R9, I > assume > > this was considered but not accepted by OTP, anyone know the reasons? > > I found this when looking for ways to improve our cache system, we uses > > ets:lookup very intensively (table contains ~7000 entries indexed by a > > tuple > > of two integers representing an adler, table is type "bag") and our > > profiling has shown that the lookup uses the largest proportion of our > > processing time. > > Does anyone have any suggestions on how to optimise the lookup? > > Thanks in advance! > > //Tom. > > > From jesper.louis.andersen@REDACTED Sat Aug 14 22:33:25 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 14 Aug 2010 22:33:25 +0200 Subject: [erlang-questions] Faster ets:lookups In-Reply-To: References: Message-ID: On Sat, Aug 14, 2010 at 8:46 PM, tom kelly wrote: > > Before I measured it I would have bet my house on the adler calculation > taking most of the execution time, but it seems that the ets lookup takes > just under three times as much! 4.5 seconds for the adler calculation, 12 > seconds for the ets lookup. In modern architectures, what you pay for tend to be not clock cycles but memory locality and bandwidth abuse. Clearly accessing a fairly big ETS table might incur cache misses. Perhaps even more than calculating the adler checksum, which can be prefetched into memory. -- J. From jesper.louis.andersen@REDACTED Sat Aug 14 22:56:13 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 14 Aug 2010 22:56:13 +0200 Subject: [erlang-questions] Re: OOP in Erlang In-Reply-To: References: <611396.69573.qm@web120515.mail.ne1.yahoo.com> <1858bc4d-497b-4953-b083-f0d4df21f9d5@k10g2000yqa.googlegroups.com> <4C64FFCE.4080301@erlang-solutions.com> Message-ID: On Sat, Aug 14, 2010 at 8:17 PM, Dave Smith wrote: > It wasn't my intention to try to define OO. ?Those of us who work in > industry feel we know what a peer is talking about when he uses the term > (irrespective of what was in Allan Kay's head when he coined the term 40 > years ago). Beware of OO having a too specific meaning. Like most CS nomenclature, it is not well-defined. In particular, one should always keep in mind the rant by Jonathan Rees: http://www.paulgraham.com/reesoo.html Even if you know of what you speak about among your peers, it is good to have this as a perspective. It somewhat amazes me nobody has linked to the rant yet, as it seems very valuable as an input to the discussion. Personally, I try to avoid the OO term because of its vague meaning. Instead, I try to define the parts that make up OO in diferent languages and use them. Java for instance has a class-type system with explicit subtyping relations. Erlang is defined by isolated processes and message passing among them. And so on. If there is a mistake made in language theory, then it is precisely the lack of mathematical nomenclature and rigor. OO is more of a philosophical statement about how to structure programs and is a mighty fine example of a bug-ridden, informal, and weak notion. The devil is in the details - how you achieve and support the philosophy. I claim that we might take different paths in different languages, but in general end in the vicinity of the OO philosophy. Herein lies my hypothesis: because there are more than one path to the goal, the OO term will keep being vague. Hence I try to abstain from its use. End of rant. -- J. From mazen.harake@REDACTED Sun Aug 15 18:17:28 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Sun, 15 Aug 2010 19:17:28 +0300 Subject: [ANN] ntop - A top-like tool for monitoring Erlang nodes Message-ID: <4C681318.9020500@erlang-solutions.com> Hi all, I've created a top-like tool which shows processes and their information given an Erlang node. It is like "etop" but without the very annoying scrolling and it is used the same way as top. Read more about it here (including a nice screenshot ;)): Blog: http://mazenharake.wordpress.com/2010/08/15/announcing-ntop-a-top-like-monitoring-tool-for-erlang-nodes/ GitHub: http://github.com/mazenharake/ntop Please give me feedback if you have any ideas/suggestions/rants or what ever if you feel like it! /Mazen From scott@REDACTED Sun Aug 15 22:57:52 2010 From: scott@REDACTED (Scott Brooks) Date: Sun, 15 Aug 2010 14:57:52 -0600 Subject: [erlang-questions] [ANN] ntop - A top-like tool for monitoring Erlang nodes In-Reply-To: <4C681318.9020500@erlang-solutions.com> References: <4C681318.9020500@erlang-solutions.com> Message-ID: Looks like a great tool, my only suggestion is that there is already an ntop out there http://www.ntop.org/ If you refer to it as Node Top everywhere then that may help people trying to google for it, although googing for "ntop erlang" should work quite well. Scott Brooks On Sun, Aug 15, 2010 at 10:17 AM, Mazen Harake wrote: > ?Hi all, > I've created a top-like tool which shows processes and their information > given an Erlang node. It is like "etop" but without the very annoying > scrolling and it is used the same way as top. > > Read more about it here (including a nice screenshot ;)): > Blog: > http://mazenharake.wordpress.com/2010/08/15/announcing-ntop-a-top-like-monitoring-tool-for-erlang-nodes/ > GitHub: http://github.com/mazenharake/ntop > > Please give me feedback if you have any ideas/suggestions/rants or what ever > if you feel like it! > > /Mazen > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From steven.charles.davis@REDACTED Mon Aug 16 00:29:34 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 15 Aug 2010 15:29:34 -0700 (PDT) Subject: Removal of the keyword 'query'. Message-ID: I'd like to suggest that the keyword query be removed from Erlang. It's unused. It's been unused for a long time. It's highly inconvenient to avoid that atom when it's the right atom to use. I imagine that the only reason to keep it would be if anyone said they were still using it in production (given the longtime existence of qlc, why would you need this level of back-compatibility?). Any negative reactions? /s From ok@REDACTED Mon Aug 16 01:46:46 2010 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 16 Aug 2010 11:46:46 +1200 Subject: [erlang-questions] A matter of style? In-Reply-To: References: Message-ID: On Aug 14, 2010, at 3:11 AM, I?aki Garay wrote: > Hello everyone, > while coding a spatial indexing tree, I often use a small helper > function within/9, which given three sets of xyz coordinates > describing three points, returns true when first point lies within the > bounding box created by the other two points. > > This is nothing more than: > > within(X, Y, Z, Xmin, Ymin, Zmin, Xmax, Ymax, Zmax) -> > (Xmin =< X) and > (X =< Xmax) and > (Ymin =< Y) and > (Y =< Ymax) and > (Zmin =< Z) and > (Z =< Zmax). (1) It is a shame the way Erlang gets the relative precedence of '=<' and 'and' wrong. (2) This evaluates *every* test, which may not be what you want. > > I ask of you this, what difference, if any, is there between using an > if clause and this function, and inserting the body of this function > as a guard? (3) There is the obvious difference that the guard really ought to be Xmin =< X, X =< Xmax, Ymin =< Y, Y =< Ymax, Zmin =< Z, Z =< Zmax with commas instead of ands. > There is the obvious code reuse when using the function, instead of > copy-pasting the body. But it makes me do this: > > Within = within(X, Y, Z, Xmin, Ymin, Zmin, Xmax, Ymax, Zmax), > if > Within -> > do_something() > true -> > {error,out_of_bounds} > end. > > which I dislike for personal reasons. No, it doesn't make you do that. What it DOES make you do is case within(X, Y, Z, XMin, Ymin, Zmin, Xmax, Ymax, Zmax) of true -> do_something() ; _ -> {error, out_of_bounds} end By the way, a function with 9 arguments, all of them of the same type, in which the argument order is surprising (I would have expected X, Xmin, Xmax, Y, Ymin, Ymax, Z, Zmin, Zmax) is just begging for trouble that even type checking cannot help you with. > Is using a guard more efficient than calling a function? No single answer is possible. It depends on which guard and what function and what the data are like, _and_ the maturity of the compiler, the total amount of code you need to keep in your L1/I cache, and probably the phase of the moon. > Is the impact negligible even when done many times? No single answer is possible. In specific cases, measure it. This is a micro-optimisation question and system level issues are likely to be more important. > Do you see a better way? Abstract patterns answer this particular problem perfectly. (Why do I always seem to end up saying that?) As for what you can do now, if you find yourself having to do something awkward often, *redesign*. Why do you have to test containment in a box so often that it _matters_ how you do it? Can't you restructure the program so that most of the time you _know_? If you are doing some sort of spatial subdivision, most of the time you _do_ know 3 of the 6 sides and should not be retesting them. So what are you actually doing? It was proposed many years ago to extend Erlang with a 'cond' construct that would eliminate all the complaints about 'if'. I'm actually glad that hasn't happened, because it seems as if every time this kinds up there _is_ a better way, once we know enough about the real problem. From jwilberding@REDACTED Mon Aug 16 02:15:17 2010 From: jwilberding@REDACTED (Jordan Wilberding) Date: Sun, 15 Aug 2010 19:15:17 -0500 Subject: [erlang-questions] [ANN] ntop - A top-like tool for monitoring Erlang nodes In-Reply-To: References: <4C681318.9020500@erlang-solutions.com> Message-ID: I agree, it is unwise to use 'ntop', it is a very prominent program. I am horrible at naming, but maybe just go with erltop? Your only competition there is a sample program from wxErlang. With that said, great program. It might even be worth adapting wxErlang's erltop GUI for your code, so you can have a gui version too. JW On Sun, Aug 15, 2010 at 3:57 PM, Scott Brooks wrote: > Looks like a great tool, my only suggestion is that there is already > an ntop out there > http://www.ntop.org/ > > If you refer to it as Node Top everywhere then that may help people > trying to google for it, although googing for "ntop erlang" should > work quite well. > > Scott Brooks > > On Sun, Aug 15, 2010 at 10:17 AM, Mazen Harake > wrote: > > Hi all, > > I've created a top-like tool which shows processes and their information > > given an Erlang node. It is like "etop" but without the very annoying > > scrolling and it is used the same way as top. > > > > Read more about it here (including a nice screenshot ;)): > > Blog: > > > http://mazenharake.wordpress.com/2010/08/15/announcing-ntop-a-top-like-monitoring-tool-for-erlang-nodes/ > > GitHub: http://github.com/mazenharake/ntop > > > > Please give me feedback if you have any ideas/suggestions/rants or what > ever > > if you feel like it! > > > > /Mazen > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From andrew@REDACTED Mon Aug 16 05:36:23 2010 From: andrew@REDACTED (Andrew Thompson) Date: Sun, 15 Aug 2010 23:36:23 -0400 Subject: [erlang-questions] Removal of the keyword 'query'. In-Reply-To: References: Message-ID: <20100816033623.GA18924@hijacked.us> On Sun, Aug 15, 2010 at 03:29:34PM -0700, Steve Davis wrote: > I'd like to suggest that the keyword query be removed from Erlang. > > It's unused. It's been unused for a long time. It's highly > inconvenient to avoid that atom when it's the right atom to use. > +1 for this. Although, it'll be quite a while before it becomes safe to use. Might not hurt to review some of the other reserved keywords that have never been used, too (like feeing up 'let' for parse transforms). Andrew From alisdairsullivan@REDACTED Mon Aug 16 06:29:20 2010 From: alisdairsullivan@REDACTED (Alisdair Sullivan) Date: Sun, 15 Aug 2010 21:29:20 -0700 Subject: application:start crash, bug? Message-ID: <42D87025-59D9-482D-BC7C-326B19B9FC27@yahoo.ca> the sequence: application:start({application, crypto, [{mod, {crypto_app, []}}]}), application:stop(crypto), application:start({application, crypto, [{mod, {crypto_app, []}}]}). results in the following error: {"Kernel pid terminated",application_controller,{{badmatch,false},[{application_controller,handle_application_started,3},{gen_server,handle_msg,5}]}} and a hard crash. It doesn't matter which application I attempt to start, I encounter this error with all of them. I realize using application:start in this manner is undocumented, but from looking over application_controller.erl, I can't locate the source of the error. Any insight? Alisdair From ingela.andin@REDACTED Mon Aug 16 09:31:40 2010 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 16 Aug 2010 09:31:40 +0200 Subject: [erlang-questions] YAWS, SOAP and SSL under load result in SSL socket closing with bad_info In-Reply-To: <4C64482A.9010008@mweb.co.za> References: <4C64482A.9010008@mweb.co.za> Message-ID: Hi! Using the latest erlang ssl-version the ssl connection will be shutdown gracefully and you will get an info report about the tcp error. Regards Ingela Erlang/OTP team - Ericsson AB 2010/8/12 Jan Jacobs : > ?Hi, > > I have an application which uses YAWS(1.88) in embedded mode with SOAP over > SSL. > It is using R13B04 on windows platforms (XP 32 Bit, Vista 32 Bit and Windows > 2008 64 Bit). > > I am getting the following error: > > (It looks like a SSL error) > =ERROR REPORT==== 4-Aug-2010::11:37:19 === > ** State machine <0.7595.0> terminating > ** Last message in was {tcp_error,#Port<0.17333>,econnaborted} > ** When State == connection > > ... > > ? ? ? ? ? ? ? ? ? ? undefined,undefined,#Ref<0.0.0.223980>, > ? ? ? ? ? ? ? ? ? ? {<0.7592.0>,#Ref<0.0.0.232900>}, > ? ? ? ? ? ? ? ? ? ? 0,<<>>,true} > ** Reason for termination = > ** bad_info > > (which then kicks a yaws error) > =ERROR REPORT==== 4-Aug-2010::11:37:19 === > Yaws process died: {bad_info,{gen_fsm,sync_send_all_state_event, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[<0.7595.0>,{recv,0},30000]}} > > This happens under load or when more than 300000 SOAP transaction has been > processed. > I then have to stop SSL and YAWS applications and start them again to accept > more transactions. > > YAWS Config: > ?GL = > ?[ > ? ?{enable_soap, ? ? ? true} > ? ?,{trace, ? ? ? ? ? ?false} > ? ?,{tmpdir, ? ? ? ? ? DirLog} > ? ?,{logdir, ? ? ? ? ? DirLog} > ? ?,{ebin_dir, ? ? ? ? [DirEbin]} > ? ?,{include_dir, ? ? ?DirInc} > ? ?,{max_connections, ?nolimit} > ? ?,{use_old_ssl, ? ? ?false} > ? ?,{flags, ? ? ? ? ? ?[ > ? ? ? ? ? ? ? ? ? ? ? ? ?{auth_log, ? ? ?true} > ? ? ? ? ? ? ? ? ? ? ? ? ?,{tty_trace, ? ?true} > ? ? ? ? ? ? ? ? ? ? ? ? ?,{copy_errlog, ?false} > ? ? ? ? ? ? ? ? ? ? ? ?]} > ?], > > ?SL = > ?[ > ? ?{port, ? ? ? ? ? ? ?PortYaws} > ? ?,{servername, ? ? ? MyHostname} > ? ?,{dir_listings, ? ? true} > ? ?,{listen, ? ? ? ? ? {0,0,0,0}} > ? ?,{appmods, ? ? ? ? ?[{"/OpenAPI",maxman_km_service_soap}]} > ? ?,{flags, ? ? ? ? ? ?[ > ? ? ? ? ? ? ? ? ? ? ? ? ?{access_log,true} > ? ? ? ? ? ? ? ? ? ? ? ?]} > ? ?,ssl() > ?], > > > YAWS SSL Config: > ?{ssl, > ?[ > ? ?{keyfile,Keyfile} > ? ?,{certfile,Certfile} > ? ?,{verify,0} > ? ?,{depth,0} > ?]} > > Any suggestions and/or comments will be appreciated. > > Thanks > Jan > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mazen.harake@REDACTED Mon Aug 16 09:21:36 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 16 Aug 2010 10:21:36 +0300 Subject: [erlang-questions] [ANN] ntop - A top-like tool for monitoring Erlang nodes In-Reply-To: References: <4C681318.9020500@erlang-solutions.com> Message-ID: <4C68E700.2070009@erlang-solutions.com> Thanks for the feedback; You are both right. I'll consider erltop but I want to avoid "erl" in the name because I think it is corny :P I'll give it a day to think about it :) Regarding the wxErlang UI; it would defeat the purpose of why I created this. The main point of the tools I create is that they are to be used in an environment where you only have shell access and no GUI environment. I'm adressing the scenario of when you ssh to the server (or ssh to a gateway then ssh to another gateway then ssh to the server that hosts the server; it is pretty common :)) and run "ntop" there and you get the information in your terminal. It would be, however, cool if someone decided to rip out the viewer and replace it with a wxErlang frontend instead; that is why it is free software :) Thanks, /Mazen On 16/08/2010 03:15, Jordan Wilberding wrote: > I agree, it is unwise to use 'ntop', it is a very prominent program. > > I am horrible at naming, but maybe just go with erltop? Your only > competition there is a sample program from wxErlang. > > With that said, great program. It might even be worth adapting > wxErlang's erltop GUI for your code, so you can have a gui version too. > > JW > > On Sun, Aug 15, 2010 at 3:57 PM, Scott Brooks > wrote: > > Looks like a great tool, my only suggestion is that there is already > an ntop out there > http://www.ntop.org/ > > If you refer to it as Node Top everywhere then that may help people > trying to google for it, although googing for "ntop erlang" should > work quite well. > > Scott Brooks > > On Sun, Aug 15, 2010 at 10:17 AM, Mazen Harake > > wrote: > > Hi all, > > I've created a top-like tool which shows processes and their > information > > given an Erlang node. It is like "etop" but without the very > annoying > > scrolling and it is used the same way as top. > > > > Read more about it here (including a nice screenshot ;)): > > Blog: > > > http://mazenharake.wordpress.com/2010/08/15/announcing-ntop-a-top-like-monitoring-tool-for-erlang-nodes/ > > GitHub: http://github.com/mazenharake/ntop > > > > Please give me feedback if you have any ideas/suggestions/rants > or what ever > > if you feel like it! > > > > /Mazen > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From ingela@REDACTED Mon Aug 16 10:30:49 2010 From: ingela@REDACTED (Ingela Andin) Date: Mon, 16 Aug 2010 10:30:49 +0200 Subject: [erlang-questions] Re: https request with checking peer In-Reply-To: References: <0830ce17-29ef-406d-8446-5f91117c4461@p7g2000yqa.googlegroups.com> Message-ID: Hi! > I would be very pleased, if someone points me where to read. Erlang > SSL guide haven't helped me =( We are currently in the process of rewriting the documentation to fit the new ssl implementation, I hope the upcoming documentation will be more satisfying. > It seems that ssl:start() isn't synonymous with > application:start(ssl). No it is not, since the latest version off ssl also crypto and public_key will also be started if the are not already started. application:start/[1,2] may be used with embedded systems and may not start anything else then the application in question, but ssl:start/[1,2] is a utility function often used for testing in the shell and we wanted to make it simpler and more convenient. It probably would be more logical to use application:start on crypto, public_key, and ssl in a "dynamic" (not embedded) applications setup procedure, as for clean up you will have to stop all the applications. ssl:stop will only stop ssl. 2010/8/12 Max Lapshin : > But if I do so: > > ssl:start(). > inets:start(). > httpc:request(get, {"https://zzz.com", []}, [{ssl,[{verify,1}]}], []). > > I get response, while I want to get error: "untrusted certificate" > > Is there any simple way automatically check peer certificate against > some root cert? Well this depends on the default verify_fun. Try VerifyFun = fun([]) -> true; ([_| _]) -> false end, httpc:request(get, {"https://zzz.com", []}, [{ssl,[{verify,1}, {verify_fun, VerifyFun}]}], []). I also saw there was an other question about the default verify fun, I will reply to that thread next. Regards Ingela Erlang/OTP Team, Ericsson AB From matthias@REDACTED Mon Aug 16 10:11:24 2010 From: matthias@REDACTED (Matthias Lang) Date: Mon, 16 Aug 2010 10:11:24 +0200 Subject: [erlang-questions] Removal of the keyword 'query'. In-Reply-To: References: Message-ID: <20100816081124.GA2165@corelatus.se> On Sunday, August 15, Steve Davis wrote: > It's unused. It's been unused for a long time. It's highly > inconvenient to avoid that atom when it's the right atom to use. You can already have 'query' as an atom. 4> is_atom('query'). true (It could be that you were already aware of how to use single quotes, though the "highly inconvenient" description suggests otherwise.) Matt From gordon@REDACTED Mon Aug 16 10:34:45 2010 From: gordon@REDACTED (Gordon Guthrie) Date: Mon, 16 Aug 2010 09:34:45 +0100 Subject: Module For Enforcing Password Strength Message-ID: Folks I was sitting fingers poised on the keyboard to write a module to test password strength and I though, "haud on, somebody musta done this"... Does anyone know of a library module that checks passwords for: * length * dictionary attack * contains numbers/punctuation * etc, etc Or does anyone have the bones of one that could be turned into an open source library (I would take on the maintenance if that was too much of a chore). Gordon -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From matthias@REDACTED Mon Aug 16 10:41:07 2010 From: matthias@REDACTED (Matthias Lang) Date: Mon, 16 Aug 2010 10:41:07 +0200 Subject: [erlang-questions] Closures across module reloads, undocumented behavior? In-Reply-To: References: <20100812082114.GA3078@corelatus.se> Message-ID: <20100816084107.GA2694@corelatus.se> On Thursday, August 12, Nahuel Greco wrote: > I expected a consistent behaviour, so I think the patch is good for me Great. > But the desired behaviour is at least undocumented, it needs to be > defined and/or documented officially. It's not exactly official, but it's certainly useful: the "Barklund Draft" defines hot code loading quite carefully, starting on p.125. http://www.erlang.org/download/erl_spec47.ps.gz I think the behaviour you get from a patched system matches what the Barklund draft says will happen. Or have I misunderstood? Matt From maruthavanan_s@REDACTED Mon Aug 16 10:43:26 2010 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Mon, 16 Aug 2010 04:43:26 -0400 Subject: TCP Options to get exact Datagrams Message-ID: Hi, I am looking how can I access or read any pointers and bits in exact TCP datagrams, say for example can I set PUSH and URGENT flag of TCP datagram when sending through my application? Is there any options available for this? Thanks, Marutha From mazen.harake@REDACTED Mon Aug 16 12:33:40 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 16 Aug 2010 13:33:40 +0300 Subject: [erlang-questions] [ANN] ntop - A top-like tool for monitoring Erlang nodes In-Reply-To: <4C68E700.2070009@erlang-solutions.com> References: <4C681318.9020500@erlang-solutions.com> <4C68E700.2070009@erlang-solutions.com> Message-ID: <4C691404.5030308@erlang-solutions.com> I decided to name it "entop" :) http://mazenharake.wordpress.com/2010/08/16/renaming-ntop-to-entop/ /Mazen On 16/08/2010 10:21, Mazen Harake wrote: > Thanks for the feedback; You are both right. I'll consider erltop but > I want to avoid "erl" in the name because I think it is corny :P > I'll give it a day to think about it :) > > Regarding the wxErlang UI; it would defeat the purpose of why I > created this. The main point of the tools I create is that they are to > be used in an environment where you only have shell access and no GUI > environment. I'm adressing the scenario of when you ssh to the server > (or ssh to a gateway then ssh to another gateway then ssh to the > server that hosts the server; it is pretty common :)) and run "ntop" > there and you get the information in your terminal. > > It would be, however, cool if someone decided to rip out the viewer > and replace it with a wxErlang frontend instead; that is why it is > free software :) > > Thanks, > > /Mazen > > On 16/08/2010 03:15, Jordan Wilberding wrote: >> I agree, it is unwise to use 'ntop', it is a very prominent program. >> >> I am horrible at naming, but maybe just go with erltop? Your only >> competition there is a sample program from wxErlang. >> >> With that said, great program. It might even be worth adapting >> wxErlang's erltop GUI for your code, so you can have a gui version too. >> >> JW >> >> On Sun, Aug 15, 2010 at 3:57 PM, Scott Brooks > > wrote: >> >> Looks like a great tool, my only suggestion is that there is already >> an ntop out there >> http://www.ntop.org/ >> >> If you refer to it as Node Top everywhere then that may help people >> trying to google for it, although googing for "ntop erlang" should >> work quite well. >> >> Scott Brooks >> >> On Sun, Aug 15, 2010 at 10:17 AM, Mazen Harake >> > > wrote: >> > Hi all, >> > I've created a top-like tool which shows processes and their >> information >> > given an Erlang node. It is like "etop" but without the very >> annoying >> > scrolling and it is used the same way as top. >> > >> > Read more about it here (including a nice screenshot ;)): >> > Blog: >> > >> >> http://mazenharake.wordpress.com/2010/08/15/announcing-ntop-a-top-like-monitoring-tool-for-erlang-nodes/ >> > GitHub: http://github.com/mazenharake/ntop >> > >> > Please give me feedback if you have any ideas/suggestions/rants >> or what ever >> > if you feel like it! >> > >> > /Mazen >> > >> > ________________________________________________________________ >> > erlang-questions (at) erlang.org mailing list. >> > See http://www.erlang.org/faq.html >> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > >> > >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> > > From attila.r.nohl@REDACTED Mon Aug 16 13:34:36 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 16 Aug 2010 13:34:36 +0200 Subject: [erlang-questions] [ANN] ntop - A top-like tool for monitoring Erlang nodes In-Reply-To: <4C681318.9020500@erlang-solutions.com> References: <4C681318.9020500@erlang-solutions.com> Message-ID: 2010/8/15, Mazen Harake : > Hi all, > I've created a top-like tool which shows processes and their information > given an Erlang node. It is like "etop" but without the very annoying > scrolling and it is used the same way as top. > > Read more about it here (including a nice screenshot ;)): > Blog: > http://mazenharake.wordpress.com/2010/08/15/announcing-ntop-a-top-like-monitoring-tool-for-erlang-nodes/ > GitHub: http://github.com/mazenharake/ntop > > Please give me feedback if you have any ideas/suggestions/rants or what > ever if you feel like it! I don't know if you're aware of dtop at http://code.google.com/p/eper/. That is a similar tool. From steven.charles.davis@REDACTED Mon Aug 16 13:40:51 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 16 Aug 2010 06:40:51 -0500 Subject: [erlang-questions] Removal of the keyword 'query'. In-Reply-To: <20100816081124.GA2165@corelatus.se> References: <20100816081124.GA2165@corelatus.se> Message-ID: <4C6923C3.5070101@gmail.com> Yes, I was :) In an API, you'd probably be cursed by users for creating a so-named function, and forcing the frequent use of something like: my_datasource:'query'(...) In the general flow of development, using such an export, it would also be quite easy to write by mistake: my_datasource:query(...) ...since that's what you are thinking naturally. And that is just a distracting way of causing avoidable compile-time errors had you chosen a different atom... ...so one does tend to avoid it. It is inconvenient, albeit just an inconvenience. However, there seems no reason for it any more. Matthias Lang wrote: > On Sunday, August 15, Steve Davis wrote: > (It could be that you were already aware of how to use single quotes, > though the "highly inconvenient" description suggests otherwise.) From mazen.harake@REDACTED Mon Aug 16 12:44:29 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 16 Aug 2010 13:44:29 +0300 Subject: [erlang-questions] [ANN] ntop - A top-like tool for monitoring Erlang nodes In-Reply-To: References: <4C681318.9020500@erlang-solutions.com> Message-ID: <4C69168D.80108@erlang-solutions.com> Yeah I was looking around a bit and this is one that I know of but still it runs in a GUI environment. The main reason I made entop was because I need to monitor a node in the terminal only (and etop was annoying me as hell because it scrolls :)) Let me know which one you think is better and why, maybe I can improve entop :) /M On 16/08/2010 14:34, Attila Rajmund Nohl wrote: > 2010/8/15, Mazen Harake: >> Hi all, >> I've created a top-like tool which shows processes and their information >> given an Erlang node. It is like "etop" but without the very annoying >> scrolling and it is used the same way as top. >> >> Read more about it here (including a nice screenshot ;)): >> Blog: >> http://mazenharake.wordpress.com/2010/08/15/announcing-ntop-a-top-like-monitoring-tool-for-erlang-nodes/ >> GitHub: http://github.com/mazenharake/ntop >> >> Please give me feedback if you have any ideas/suggestions/rants or what >> ever if you feel like it! > I don't know if you're aware of dtop at > http://code.google.com/p/eper/. That is a similar tool. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From 12ukwn@REDACTED Mon Aug 16 13:45:50 2010 From: 12ukwn@REDACTED (Jean-Yves F. Barbier) Date: Mon, 16 Aug 2010 13:45:50 +0200 Subject: [erlang-questions] [ANN] ntop - A top-like tool for monitoring Erlang nodes In-Reply-To: <4C691404.5030308@erlang-solutions.com> References: <4C681318.9020500@erlang-solutions.com> <4C68E700.2070009@erlang-solutions.com> <4C691404.5030308@erlang-solutions.com> Message-ID: <20100816134550.6952b7ff@anubis.defcon1> Le Mon, 16 Aug 2010 13:33:40 +0300, Mazen Harake a ?crit : > I decided to name it "entop" :) You could have it also named to: harakentop or mazentop ;) JY -- From ingela@REDACTED Mon Aug 16 14:18:59 2010 From: ingela@REDACTED (Ingela Andin) Date: Mon, 16 Aug 2010 14:18:59 +0200 Subject: [erlang-questions] enforcing ssl trust chain In-Reply-To: <4C62D6FD.4010208@rabbitmq.com> References: <4C62D6FD.4010208@rabbitmq.com> Message-ID: Hi! 2010/8/11 Emile Joubert : > > Hi, > > I read in the latest ssl documentation and SSL 3.10.3 release notes that > an unknown CA is not considered a validation error. What is the > motivation for this default? My personal preference for default value was not to accept any path-validation errors as default, but the motivation was that it should be as easy as possible to get an ssl connection up and running. I am just back from vacation and I do not remember all the details of the discussion. We are of course interested in all user feedback we can get. So if you have any arguments for or against please let us know. > In a production environment I want to prevent clients without > certificates signed by a known CA from connecting. Is there any way of > getting this behaviour by using configuration files? The only way I can > find is to set verify_fun to an appropriate function, but this is > unappealing because I want to change my mind without needing to recompile. At the moment defining a verify fun would be your option to accomplish this. We might add some other configuration option if we find that it seems to be a good thing from a general point of view. Regards Ingela Erlang/OTP team - Ericsson AB From attila.r.nohl@REDACTED Mon Aug 16 14:26:11 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 16 Aug 2010 14:26:11 +0200 Subject: [erlang-questions] [ANN] ntop - A top-like tool for monitoring Erlang nodes In-Reply-To: <4C69168D.80108@erlang-solutions.com> References: <4C681318.9020500@erlang-solutions.com> <4C69168D.80108@erlang-solutions.com> Message-ID: Hello! dtop actually runs in a shell, so there's no need for a GUI. Other tools in that package need GUI though. I can't compare it to your tool, because I haven't used it yet. 2010/8/16, Mazen Harake : > Yeah I was looking around a bit and this is one that I know of but > still it runs in a GUI environment. The main reason I made entop was > because I need to monitor a node in the terminal only (and etop was > annoying me as hell because it scrolls :)) > > Let me know which one you think is better and why, maybe I can improve > entop :) > > /M > > On 16/08/2010 14:34, Attila Rajmund Nohl wrote: >> 2010/8/15, Mazen Harake: >>> Hi all, >>> I've created a top-like tool which shows processes and their information >>> given an Erlang node. It is like "etop" but without the very annoying >>> scrolling and it is used the same way as top. >>> >>> Read more about it here (including a nice screenshot ;)): >>> Blog: >>> http://mazenharake.wordpress.com/2010/08/15/announcing-ntop-a-top-like-monitoring-tool-for-erlang-nodes/ >>> GitHub: http://github.com/mazenharake/ntop >>> >>> Please give me feedback if you have any ideas/suggestions/rants or what >>> ever if you feel like it! >> I don't know if you're aware of dtop at >> http://code.google.com/p/eper/. That is a similar tool. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > From jesper.pettersson@REDACTED Mon Aug 16 14:30:23 2010 From: jesper.pettersson@REDACTED (Jesper Pettersson) Date: Mon, 16 Aug 2010 14:30:23 +0200 Subject: [erlang-questions] enforcing ssl trust chain In-Reply-To: References: <4C62D6FD.4010208@rabbitmq.com> Message-ID: > My personal preference for default value was not to accept any > path-validation errors as default, but the motivation was that it > should be as easy as possible to get an ssl connection up and > running. I am just back from vacation and I do not remember > all the details of the discussion. We are of course interested in all > user feedback we can get. > So if you have any arguments for or against please let us know. In my opinion the default behavior should be very strict with regards to certificate validation. It should honor keyUsage and AKI/SKI extensions, check all CAs in the chain, have the possibility to supply a CRL (or CRL location) etc. Then there could be options to allow "quick-and-dirty" SSL where only basic validation like the signature and validity time of the subject certificate is verified. In the path-validation case there could be an option specifying the maximum chain depth allowed where 0 could mean skip CA validation. By default the whole chain shoul be validated until we find a trusted self-signed root CA (or a trusted intermediary CA). Jesper Pettersson Klarna AB From mit@REDACTED Mon Aug 16 14:31:20 2010 From: mit@REDACTED (Andrey Mitroshin) Date: Mon, 16 Aug 2010 16:31:20 +0400 Subject: Erlang code analysis Message-ID: <45d20dbbc0d02e179882f0be0be18dbf.squirrel@webmail.akamit.com> Hello, Could you please recommend some tools for visualizing trace data to facilitate more ease understanding of complex erlang software projects with lack of code documentation? Thanks in advance -- mit From jwilberding@REDACTED Mon Aug 16 15:10:32 2010 From: jwilberding@REDACTED (Jordan Wilberding) Date: Mon, 16 Aug 2010 08:10:32 -0500 Subject: [erlang-questions] Module For Enforcing Password Strength In-Reply-To: References: Message-ID: You can do most of these except the dictionary lookup with a regular expression. For dictionary lookup, I'd just take whatever dictionary cracklib uses, load it into a table or dict, and you are good to go for lookups. Thanks! Jordan Wilberding On Mon, Aug 16, 2010 at 3:34 AM, Gordon Guthrie wrote: > Folks > > I was sitting fingers poised on the keyboard to write a module to test > password strength and I though, "haud on, somebody musta done this"... > > Does anyone know of a library module that checks passwords for: > * length > * dictionary attack > * contains numbers/punctuation > * etc, etc > > Or does anyone have the bones of one that could be turned into an open > source library (I would take on the maintenance if that was too much of a > chore). > > Gordon > > -- > Gordon Guthrie > CEO hypernumbers > > http://hypernumbers.com > t: hypernumbers > +44 7776 251669 > From hpjcon@REDACTED Sun Aug 15 09:43:08 2010 From: hpjcon@REDACTED (Jan Jacobs) Date: Sun, 15 Aug 2010 09:43:08 +0200 Subject: [erlang-questions] YAWS, SOAP and SSL under load result in SSL socket closing with bad_info In-Reply-To: <4C64482A.9010008@mweb.co.za> References: <4C64482A.9010008@mweb.co.za> Message-ID: <4C679A8C.3080300@mweb.co.za> Hi, I decided to upgraded to R14A, the results is looking good. I have done 1000000 transactions without the necessity to restart SSL or YAWS. The next step is to push it to the live environment. Cheers Jan From fritchie@REDACTED Mon Aug 16 17:02:13 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Mon, 16 Aug 2010 10:02:13 -0500 Subject: [erlang-questions] Faster ets:lookups In-Reply-To: Message of "Fri, 13 Aug 2010 11:20:09 BST." Message-ID: <92248.1281970933@snookles.snookles.com> tom kelly wrote: tk> [...] I assume this was tk> considered but not accepted by OTP, anyone know the reasons? Tom, the R9 days predate Ericsson's workflow with GitHub. I hadn't cleaned up that work so that Ericsson could even consider it if they'd wanted to. In the intervening years there hasn't been any movement (that I'm aware of) to create a clean/pluggable interface to ETS for 3rd-party libraries. So the Judy stuff has languished. I haven't had the spare cycles to create a driver- or NIF-based interface for the Judy/JudyL/JudySL APIs. I can't recall if anyone else has done so, but it's still Monday morning, and things are still a bit hazy. :-) -Scott From fritchie@REDACTED Mon Aug 16 17:24:34 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Mon, 16 Aug 2010 10:24:34 -0500 Subject: [erlang-questions] Faster ets:lookups In-Reply-To: Message of "Fri, 13 Aug 2010 11:20:09 BST." Message-ID: <93458.1281972274@snookles.snookles.com> Oops, I forgot to add a comment about Tom's speed question. By design, ETS storage is separate from a process's private heap. That's generally considered a good thing, because you can put a zillion bytes into ETS and not worry about ETS data interfering with the garbage collection of your process's heap (which is presumably now much much smaller). But if you do a lot of lookups, and if the data is "big", then the VM will be spending a lot of time copying that data out of ETS and into your process's heap. Your problem sounds like most of the time, ets:lookup/2 is probably returning [] most of the time (?), so that data copying overhead shouldn't be a problem for you. That said, if ets:member/2 works for you, you should use member instead of ets:lookup/2 when possible. I haven't explored R13's ETS implementation, so I don't know if there's any extra overhead in public vs. protected vs. private tables ... but it should be easy enough to measure. -Scott From mjtruog@REDACTED Mon Aug 16 18:37:56 2010 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 16 Aug 2010 09:37:56 -0700 Subject: [erlang-questions] Module For Enforcing Password Strength In-Reply-To: References: Message-ID: <4C696964.2070706@gmail.com> I have a trie data structure that might help for the dictionary attack: http://github.com/okeuday/CloudI/blob/master/src/lib/cloud_stdlib/src/trie.erl The rest should be straight-forward calculations, it seems. On 08/16/2010 01:34 AM, Gordon Guthrie wrote: > Folks > > I was sitting fingers poised on the keyboard to write a module to test > password strength and I though, "haud on, somebody musta done this"... > > Does anyone know of a library module that checks passwords for: > * length > * dictionary attack > * contains numbers/punctuation > * etc, etc > > Or does anyone have the bones of one that could be turned into an open > source library (I would take on the maintenance if that was too much of a > chore). > > Gordon > > From magnus@REDACTED Mon Aug 16 18:49:44 2010 From: magnus@REDACTED (Magnus Henoch) Date: Mon, 16 Aug 2010 18:49:44 +0200 Subject: Module For Enforcing Password Strength In-Reply-To: (Gordon Guthrie's message of "Mon, 16 Aug 2010 09:34:45 +0100") References: Message-ID: Gordon Guthrie writes: > I was sitting fingers poised on the keyboard to write a module to test > password strength and I though, "haud on, somebody musta done this"... > > Does anyone know of a library module that checks passwords for: > * length > * dictionary attack > * contains numbers/punctuation > * etc, etc It sounds like you're looking for cracklib. I've never used it myself, but it seems to be living at http://sourceforge.net/projects/cracklib/ nowadays. It's also packaged for many Linux distributions. Magnus From gordon@REDACTED Mon Aug 16 18:52:48 2010 From: gordon@REDACTED (Gordon Guthrie) Date: Mon, 16 Aug 2010 17:52:48 +0100 Subject: [erlang-questions] Module For Enforcing Password Strength In-Reply-To: References: Message-ID: Richard > cracklib. C bindings. Write your own wrapper. Kinda what I was hoping NOT to have to do :( Had a go at writing a C port driver a couple of years ago and didn't really get anywhere... Gordon On 16 August 2010 11:20, Richard Andrews wrote: > cracklib. C bindings. Write your own wrapper. > > On Mon, Aug 16, 2010 at 6:34 PM, Gordon Guthrie > wrote: > > Folks > > > > I was sitting fingers poised on the keyboard to write a module to test > > password strength and I though, "haud on, somebody musta done this"... > > > > Does anyone know of a library module that checks passwords for: > > * length > > * dictionary attack > > * contains numbers/punctuation > > * etc, etc > > > > Or does anyone have the bones of one that could be turned into an open > > source library (I would take on the maintenance if that was too much of a > > chore). > > > > Gordon > > > > -- > > Gordon Guthrie > > CEO hypernumbers > > > > http://hypernumbers.com > > t: hypernumbers > > +44 7776 251669 > > > -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From gordon@REDACTED Mon Aug 16 18:54:48 2010 From: gordon@REDACTED (Gordon Guthrie) Date: Mon, 16 Aug 2010 17:54:48 +0100 Subject: [erlang-questions] Module For Enforcing Password Strength In-Reply-To: References: Message-ID: Jordan That was kind of my initial thoughts, but I had a poke about a bit... cracklib does a bit more than just look for patterns - it does case normalisation, takes your username and looks for you reusing that, looks for repeating patterns and all sorts of stuff... hmmm - but I can't help thinking someone must have done it... Gordon On 16 August 2010 14:10, Jordan Wilberding wrote: > You can do most of these except the dictionary lookup with a regular > expression. For dictionary lookup, I'd just take whatever dictionary > cracklib uses, load it into a table or dict, and you are good to go for > lookups. > > Thanks! > Jordan Wilberding > > On Mon, Aug 16, 2010 at 3:34 AM, Gordon Guthrie wrote: > >> Folks >> >> I was sitting fingers poised on the keyboard to write a module to test >> password strength and I though, "haud on, somebody musta done this"... >> >> Does anyone know of a library module that checks passwords for: >> * length >> * dictionary attack >> * contains numbers/punctuation >> * etc, etc >> >> Or does anyone have the bones of one that could be turned into an open >> source library (I would take on the maintenance if that was too much of a >> chore). >> >> Gordon >> >> -- >> Gordon Guthrie >> CEO hypernumbers >> >> http://hypernumbers.com >> t: hypernumbers >> +44 7776 251669 >> > > -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From michael.santos@REDACTED Mon Aug 16 20:33:21 2010 From: michael.santos@REDACTED (Michael Santos) Date: Mon, 16 Aug 2010 14:33:21 -0400 Subject: [erlang-questions] Module For Enforcing Password Strength In-Reply-To: References: Message-ID: <20100816183321.GA25135@ecn.lan> On Mon, Aug 16, 2010 at 05:52:48PM +0100, Gordon Guthrie wrote: > Richard > > > cracklib. C bindings. Write your own wrapper. > > Kinda what I was hoping NOT to have to do :( > > Had a go at writing a C port driver a couple of years ago and didn't really > get anywhere... Just for fun, I wrote an interface to libcrack and put it here: http://github.com/msantos/cerck Adding support for checking the length and the presence of characters besides those in the alphabet should be simple. I'll add them later but, it'd probably be better to write all of this in pure Erlang. One thing to watch out for: libcrack calls exit() if the dictionnary files don't exist, which will cause the vm to exit. > On 16 August 2010 11:20, Richard Andrews wrote: > > > cracklib. C bindings. Write your own wrapper. > > > > On Mon, Aug 16, 2010 at 6:34 PM, Gordon Guthrie > > wrote: > > > Folks > > > > > > I was sitting fingers poised on the keyboard to write a module to test > > > password strength and I though, "haud on, somebody musta done this"... > > > > > > Does anyone know of a library module that checks passwords for: > > > * length > > > * dictionary attack > > > * contains numbers/punctuation > > > * etc, etc > > > > > > Or does anyone have the bones of one that could be turned into an open > > > source library (I would take on the maintenance if that was too much of a > > > chore). > > > > > > Gordon > > > > > > -- > > > Gordon Guthrie > > > CEO hypernumbers > > > > > > http://hypernumbers.com > > > t: hypernumbers > > > +44 7776 251669 > > > > > > > > > -- > Gordon Guthrie > CEO hypernumbers > > http://hypernumbers.com > t: hypernumbers > +44 7776 251669 From ngreco@REDACTED Mon Aug 16 21:52:14 2010 From: ngreco@REDACTED (Nahuel Greco) Date: Mon, 16 Aug 2010 16:52:14 -0300 Subject: [erlang-questions] Closures across module reloads, undocumented behavior? In-Reply-To: <20100816084107.GA2694@corelatus.se> References: <20100812082114.GA3078@corelatus.se> <20100816084107.GA2694@corelatus.se> Message-ID: On Mon, Aug 16, 2010 at 5:41 AM, Matthias Lang wrote: > On Thursday, August 12, Nahuel Greco wrote: >> >> But the desired behaviour is at least undocumented, it needs to be >> defined and/or documented officially. > > It's not exactly official, but it's certainly useful: the "Barklund > Draft" defines hot code loading quite carefully, starting on p.125. > > ?http://www.erlang.org/download/erl_spec47.ps.gz > > I think the behaviour you get from a patched system matches what the > Barklund draft says will happen. Or have I misunderstood? There is no mention of what happens with function references stored in heap across modules reloads. The bug effect of an apparent automatic "migration" of closures to the new module code made me search for that explicit note. The R14A documentation mentions the case of a process containing a fun referencing the old module in the check_process_code/2 section, but it doesn't explicitly states the closures will continue to reference the old module code when a new version is loaded. In constrast, the Barklund Draft seems more incomplete, because in his section 9.7 check_process_code/2 is documented as only scanning the call stack, but the implementation also scans the process heap, leaving undefined the closures related purging behaviour. I think the intuitive and intended behaviour, in the spirit of the Barklund Draft, is the one your patch provides; no weird automatic closure "migrations" and check_process_code/2 detecting a process as using the old version when a fun in his heap points to it. Probably closures continuing to point to the old version is taken as obvious and with no need of clarification, but maybe an explicit note about what happens with the function references in heap when reloading a module would have made easier to classify the R14A behaviour as a bug. OTP Team: will you apply the Matthias patch to trunk or there is a reason to not use that patch? Saludos, Nahuel Greco. From gordon@REDACTED Mon Aug 16 22:05:23 2010 From: gordon@REDACTED (Gordon Guthrie) Date: Mon, 16 Aug 2010 21:05:23 +0100 Subject: [erlang-questions] Module For Enforcing Password Strength In-Reply-To: <20100816183321.GA25135@ecn.lan> References: <20100816183321.GA25135@ecn.lan> Message-ID: Michael Aye Caramba! I will give it a go! Would you expect cracklib to be pretty NIF-tastic - a stable code base, a quick turn-around so the VM doesn't stall? Try it and see I suppose... Gordon On 16 August 2010 19:33, Michael Santos wrote: > On Mon, Aug 16, 2010 at 05:52:48PM +0100, Gordon Guthrie wrote: > > Richard > > > > > cracklib. C bindings. Write your own wrapper. > > > > Kinda what I was hoping NOT to have to do :( > > > > Had a go at writing a C port driver a couple of years ago and didn't > really > > get anywhere... > > Just for fun, I wrote an interface to libcrack and put it here: > > http://github.com/msantos/cerck > > Adding support for checking the length and the presence of characters > besides those in the alphabet should be simple. I'll add them later > but, it'd probably be better to write all of this in pure Erlang. > > One thing to watch out for: libcrack calls exit() if the dictionnary > files don't exist, which will cause the vm to exit. > > > On 16 August 2010 11:20, Richard Andrews wrote: > > > > > cracklib. C bindings. Write your own wrapper. > > > > > > On Mon, Aug 16, 2010 at 6:34 PM, Gordon Guthrie < > gordon@REDACTED> > > > wrote: > > > > Folks > > > > > > > > I was sitting fingers poised on the keyboard to write a module to > test > > > > password strength and I though, "haud on, somebody musta done > this"... > > > > > > > > Does anyone know of a library module that checks passwords for: > > > > * length > > > > * dictionary attack > > > > * contains numbers/punctuation > > > > * etc, etc > > > > > > > > Or does anyone have the bones of one that could be turned into an > open > > > > source library (I would take on the maintenance if that was too much > of a > > > > chore). > > > > > > > > Gordon > > > > > > > > -- > > > > Gordon Guthrie > > > > CEO hypernumbers > > > > > > > > http://hypernumbers.com > > > > t: hypernumbers > > > > +44 7776 251669 > > > > > > > > > > > > > > > -- > > Gordon Guthrie > > CEO hypernumbers > > > > http://hypernumbers.com > > t: hypernumbers > > +44 7776 251669 > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From mfidelman@REDACTED Mon Aug 16 23:15:51 2010 From: mfidelman@REDACTED (Miles Fidelman) Date: Mon, 16 Aug 2010 17:15:51 -0400 Subject: very large distributed implementation? Message-ID: <4C69AA87.9000605@meetinghouse.net> Our of somewhat less than academic curiousity, are there any good examples of very large distributed systems written in Erlang - specifically systems consisting of 100s to 1000s of computing nodes? I'm looking at several applications along the lines of SETI@REDACTED and distributed.net, that take the background cycles, from very large numbers of machines, and turn them into a gridded processing environent. It strikes me that Erlang should make a very nice platform for such an environment. I wonder if anyone has actually done anything on this scale. Thanks, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From scottfinneran@REDACTED Tue Aug 17 00:37:58 2010 From: scottfinneran@REDACTED (Scott Finneran) Date: Tue, 17 Aug 2010 08:37:58 +1000 Subject: [erlang-questions] very large distributed implementation? In-Reply-To: <4C69AA87.9000605@meetinghouse.net> References: <4C69AA87.9000605@meetinghouse.net> Message-ID: <4C69BDC6.9090406@yahoo.com.au> Miles Fidelman wrote: > Our of somewhat less than academic curiousity, are there any good > examples of very large distributed systems written in Erlang - > specifically systems consisting of 100s to 1000s of computing nodes? Hi Miles, I can't remember the specifics, but some fairly large systems were described in the FLOSS weekly episode about Riak. It's worth a listen. http://twit.tv/floss129 Cheers, Scott Finneran From mjtruog@REDACTED Tue Aug 17 03:05:43 2010 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 16 Aug 2010 18:05:43 -0700 Subject: [erlang-questions] very large distributed implementation? In-Reply-To: <4C69AA87.9000605@meetinghouse.net> References: <4C69AA87.9000605@meetinghouse.net> Message-ID: <4C69E067.4010808@gmail.com> Scalaris probably has been tested with many nodes but you would probably need to check their IEEE paper, or the presentations (http://www.onscale.de/scalarix.html). Cloudi exists but is not meant for 100s of computing nodes (http://cloudi.org). The reason is that private clouds with new hardware would have many cores, so the need for many nodes is reduced. Any Erlang code that works on 100s or 1000s of nodes is unlikely to use the distributed Erlang mode because of the heartbeat message that is used every minute (i.e., so local network backbones that are not expected to go down). That means that such Erlang code is likely to utilize tcp, udp, etc. with custom code that manages consistency. On 08/16/2010 02:15 PM, Miles Fidelman wrote: > Our of somewhat less than academic curiousity, are there any good > examples of very large distributed systems written in Erlang - > specifically systems consisting of 100s to 1000s of computing nodes? > > I'm looking at several applications along the lines of SETI@REDACTED and > distributed.net, that take the background cycles, from very large > numbers of machines, and turn them into a gridded processing environent. > > It strikes me that Erlang should make a very nice platform for such an > environment. I wonder if anyone has actually done anything on this > scale. > > Thanks, > > Miles Fidelman > From michael.santos@REDACTED Tue Aug 17 03:24:28 2010 From: michael.santos@REDACTED (Michael Santos) Date: Mon, 16 Aug 2010 21:24:28 -0400 Subject: [erlang-questions] Module For Enforcing Password Strength In-Reply-To: References: <20100816183321.GA25135@ecn.lan> Message-ID: <20100817012428.GA25497@ecn.lan> On Mon, Aug 16, 2010 at 09:05:23PM +0100, Gordon Guthrie wrote: > Michael > > Aye Caramba! > > I will give it a go! > > Would you expect cracklib to be pretty NIF-tastic - a stable code base, a > quick turn-around so the VM doesn't stall? Stable enough to have the same bugs for many years it seems ;) I just discovered that cracklib isn't thread safe and can crash the VM. Apparently this was a problem in the python lib as well. I've put a lock around calls to cracklib for now. Password checks should be quick but it's possible that if you are doing a large number of sequential checks in parallel other processes may be starved. > Try it and see I suppose... > > Gordon > > On 16 August 2010 19:33, Michael Santos wrote: > > > On Mon, Aug 16, 2010 at 05:52:48PM +0100, Gordon Guthrie wrote: > > > Richard > > > > > > > cracklib. C bindings. Write your own wrapper. > > > > > > Kinda what I was hoping NOT to have to do :( > > > > > > Had a go at writing a C port driver a couple of years ago and didn't > > really > > > get anywhere... > > > > Just for fun, I wrote an interface to libcrack and put it here: > > > > http://github.com/msantos/cerck > > > > Adding support for checking the length and the presence of characters > > besides those in the alphabet should be simple. I'll add them later > > but, it'd probably be better to write all of this in pure Erlang. > > > > One thing to watch out for: libcrack calls exit() if the dictionnary > > files don't exist, which will cause the vm to exit. > > > > > On 16 August 2010 11:20, Richard Andrews wrote: > > > > > > > cracklib. C bindings. Write your own wrapper. > > > > > > > > On Mon, Aug 16, 2010 at 6:34 PM, Gordon Guthrie < > > gordon@REDACTED> > > > > wrote: > > > > > Folks > > > > > > > > > > I was sitting fingers poised on the keyboard to write a module to > > test > > > > > password strength and I though, "haud on, somebody musta done > > this"... > > > > > > > > > > Does anyone know of a library module that checks passwords for: > > > > > * length > > > > > * dictionary attack > > > > > * contains numbers/punctuation > > > > > * etc, etc > > > > > > > > > > Or does anyone have the bones of one that could be turned into an > > open > > > > > source library (I would take on the maintenance if that was too much > > of a > > > > > chore). > > > > > > > > > > Gordon > > > > > > > > > > -- > > > > > Gordon Guthrie > > > > > CEO hypernumbers > > > > > > > > > > http://hypernumbers.com > > > > > t: hypernumbers > > > > > +44 7776 251669 > > > > > > > > > > > > > > > > > > > > > -- > > > Gordon Guthrie > > > CEO hypernumbers > > > > > > http://hypernumbers.com > > > t: hypernumbers > > > +44 7776 251669 > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > -- > Gordon Guthrie > CEO hypernumbers > > http://hypernumbers.com > t: hypernumbers > +44 7776 251669 From mojitotech@REDACTED Tue Aug 17 04:06:48 2010 From: mojitotech@REDACTED (Mojito Sorbet) Date: Mon, 16 Aug 2010 22:06:48 -0400 Subject: [erlang-questions] very large distributed implementation? In-Reply-To: <4C69E067.4010808@gmail.com> References: <4C69AA87.9000605@meetinghouse.net> <4C69E067.4010808@gmail.com> Message-ID: <1282010808.2887.4.camel@sapphire> Another issue is the preference for Erlang to connect all nodes in a fully connected mesh of TCP links. That is N-1 links per node. This can be disabled, and result in tree connections instead of mesh connections, but depending on the problem being solved maybe only occasional connectivity is required. On Mon, 2010-08-16 at 18:05 -0700, Michael Truog wrote: > Any Erlang code that works on 100s or 1000s of nodes is unlikely to use > the distributed Erlang mode because of the heartbeat message that is > used every minute (i.e., so local network backbones that are not > expected to go down). That means that such Erlang code is likely to > utilize tcp, udp, etc. with custom code that manages consistency. From mythicalbox@REDACTED Tue Aug 17 10:02:38 2010 From: mythicalbox@REDACTED (Mitchel Constantin) Date: Tue, 17 Aug 2010 08:02:38 +0000 Subject: [erlang-questions] very large distributed implementation? In-Reply-To: <4C69AA87.9000605@meetinghouse.net> References: <4C69AA87.9000605@meetinghouse.net> Message-ID: Facebook's Chat system was developed using Erlang. http://www.process-one.net/en/blogs/article/facebook_chat_is_developed_in_erlang ________________________________________ From: erlang-questions@REDACTED [erlang-questions@REDACTED] on behalf of Miles Fidelman [mfidelman@REDACTED] Sent: Monday, August 16, 2010 2:15 PM To: erlang-questions@REDACTED Subject: [erlang-questions] very large distributed implementation? Our of somewhat less than academic curiousity, are there any good examples of very large distributed systems written in Erlang - specifically systems consisting of 100s to 1000s of computing nodes? I'm looking at several applications along the lines of SETI@REDACTED and distributed.net, that take the background cycles, from very large numbers of machines, and turn them into a gridded processing environent. It strikes me that Erlang should make a very nice platform for such an environment. I wonder if anyone has actually done anything on this scale. Thanks, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From mpalmer@REDACTED Tue Aug 17 10:42:19 2010 From: mpalmer@REDACTED (Matthew Palmer) Date: Tue, 17 Aug 2010 18:42:19 +1000 Subject: very large distributed implementation? In-Reply-To: <4C69AA87.9000605@meetinghouse.net> References: <4C69AA87.9000605@meetinghouse.net> Message-ID: <20100817084219.GJ11895@hezmatt.org> On Mon, Aug 16, 2010 at 05:15:51PM -0400, Miles Fidelman wrote: > Our of somewhat less than academic curiousity, are there any good > examples of very large distributed systems written in Erlang - > specifically systems consisting of 100s to 1000s of computing nodes? > > I'm looking at several applications along the lines of SETI@REDACTED and > distributed.net, that take the background cycles, from very large > numbers of machines, and turn them into a gridded processing environent. > > It strikes me that Erlang should make a very nice platform for such an > environment. I wonder if anyone has actually done anything on this > scale. I haven't deployed any such thing, but from what I've read and experimented with, Erlang would make an excellent language for writing the coordination and management layer of the whole thing, but I'd be inclined to write the actual number crunching parts in something a bit more suited to high efficiency. - Matt From masklinn@REDACTED Tue Aug 17 10:50:35 2010 From: masklinn@REDACTED (Masklinn) Date: Tue, 17 Aug 2010 10:50:35 +0200 Subject: [erlang-questions] very large distributed implementation? In-Reply-To: <4C69E067.4010808@gmail.com> References: <4C69AA87.9000605@meetinghouse.net> <4C69E067.4010808@gmail.com> Message-ID: <87F1FC52-5060-4A0B-99AC-3C5D176C6A03@masklinn.net> On 2010-08-17, at 03:05 , Michael Truog wrote: > Any Erlang code that works on 100s or 1000s of nodes is unlikely to use > the distributed Erlang mode because of the heartbeat message that is > used every minute (i.e., so local network backbones that are not > expected to go down). That means that such Erlang code is likely to > utilize tcp, udp, etc. with custom code that manages consistency. Also, I believe distributed erlang works on the basis of nodes being trusted, which probably wouldn't be the case for a distributed computing projects, and probably LAN nodes as well, I don't know how it'd cope with internet latency. From gnoblin@REDACTED Tue Aug 17 11:56:23 2010 From: gnoblin@REDACTED (gnoblin) Date: Tue, 17 Aug 2010 02:56:23 -0700 (PDT) Subject: a question on misultin Message-ID: Hello, I've decided to use misultin as a webserver for my client-server application, and I am trying to do the following: Case 1: % process1 handle_http(Req) -> % get params Args = Req:parse_qs(), case proplists:get_value("value", Args) of %% we got an empty request? undefined -> ok; Input -> io:format("got a request~p~n", [Input]), Req:ok([?CONTENT_TYPE], ""++"Ok"++"") Result: I see the response in my browser. Case 2: If instead of Req:ok([?CONTENT_TYPE], ""++"Ok"++"") I send a message to process2, which processes the request and sends the response via a message to process1, and it is received in process1 here: loop() -> receive {respond, Req, Json} -> io:format("Responding back! ~n"), %% falls somewhere here %% it seems i can't do it like this... Req:ok([?CONTENT_TYPE], ""++Json++""), loop(); Other -> io:format("says: I've got some strange message here: ~p~n", [Other]) end. Result: I don't see the response. --- The question is: if I can't do it like that - what is the correct way to do it? I'd like to get a string via http from client, send it to other processes, get the response back to process with misultin and send it back to the user. Thanks From roberto@REDACTED Tue Aug 17 12:04:28 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 17 Aug 2010 12:04:28 +0200 Subject: [erlang-questions] a question on misultin In-Reply-To: References: Message-ID: hello goblin, for every request a process is spawned to manage its response [aka the response controlling process], and it's this process that calls the handle function, which MUST return a response. this is a precise server design, since every single request/response entity has a precise lifetime which needs to be controlled server-wise, and this lifetime corresponds to the lifetime of the controlling process. if for whatever reason you need to build a response from an external process, you may do so by 'locking' the response controlling process and sending to it the response so that it can return it. a working example follows. -module(misultin_hello_world). -export([start/1, stop/0]). -export([external_proc/1]). % start misultin http server start(Port) -> misultin:start_link([{port, Port}, {loop, fun(Req) -> handle_http(Req) end}]). % stop misultin stop() -> misultin:stop(). % callback on request received handle_http(Req) -> % spawn external process Pid = spawn(?MODULE, external_proc, [self()]), % send req Pid ! Req, % wait receive Response -> Response end. external_proc(ControllerPid) -> receive Req -> % build response Response = Req:ok("Hello World."), % send it to controller ControllerPid ! Response end. cheers, r. 2010/8/17 gnoblin : > Hello, > > I've decided to use misultin as a webserver for my client-server > application, and I am trying to do the following: > > Case 1: > > % process1 > handle_http(Req) -> > ? ?% get params > ? ?Args = Req:parse_qs(), > > ? ?case proplists:get_value("value", Args) of > ? ? ? ?%% we got an empty request? > ? ? ? ?undefined -> > ? ? ? ? ? ?ok; > > ? ? ? ?Input -> > ? ? ? ? ? ?io:format("got a request~p~n", > [Input]), > ? ? ? ? ? ?Req:ok([?CONTENT_TYPE], ""++"Ok"++"") > > Result: I see the response in my browser. > > Case 2: > If instead of Req:ok([?CONTENT_TYPE], ""++"Ok"++"") I > send a message to process2, which processes the request and sends the > response via a message to process1, > > and it is received in process1 here: > > loop() -> > ? ?receive > ? ? ? ?{respond, Req, Json} -> > ? ? ? ? ? ?io:format("Responding back! ~n"), > ? ? ? ? ? ?%% falls somewhere here > ? ? ? ? ? ?%% it seems i can't do it like this... > ? ? ? ? ? ?Req:ok([?CONTENT_TYPE], ""++Json++""), > ? ? ? ? ? ?loop(); > ? ? ? ?Other -> > ? ? ? ? ? ?io:format("says: I've got some strange message here: ~p~n", > [Other]) > ? ?end. > > Result: I don't see the response. > > --- > The question is: if I can't do it like that - what is the correct way > to do it? I'd like to get a string via http from client, send it to > other processes, get the response back to process with misultin and > send it back to the user. > > Thanks From roberto@REDACTED Tue Aug 17 12:06:05 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 17 Aug 2010 12:06:05 +0200 Subject: [erlang-questions] very large distributed implementation? In-Reply-To: References: <4C69AA87.9000605@meetinghouse.net> Message-ID: 2010/8/17 Mitchel Constantin : > Facebook's Chat system was developed using Erlang. > > http://www.process-one.net/en/blogs/article/facebook_chat_is_developed_in_erlang yes, but that's using a federation of XMPP servers, which i believe is not what the requester is looking for.. From roberto@REDACTED Tue Aug 17 12:10:38 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 17 Aug 2010 12:10:38 +0200 Subject: [erlang-questions] very large distributed implementation? In-Reply-To: <4C69AA87.9000605@meetinghouse.net> References: <4C69AA87.9000605@meetinghouse.net> Message-ID: 2010/8/16 Miles Fidelman : > Our of somewhat less than academic curiousity, are there any good examples > of very large distributed systems written in Erlang - specifically systems > consisting of 100s to 1000s of computing nodes? > > I'm looking at several applications along the lines of SETI@REDACTED and > distributed.net, that take the background cycles, from very large numbers of > machines, and turn them into a gridded processing environent. > > It strikes me that Erlang should make a very nice platform for such an > environment. ?I wonder if anyone has actually done anything on this scale. > > Thanks, > > Miles Fidelman > > -- > In theory, there is no difference between theory and practice. > In ?practice, there is. ? .... Yogi Berra > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > hi miles, you may find some clues on this topic i opened some time ago: http://www.erlang.org/cgi-bin/ezmlm-cgi?4:msp:47731 glad to see that this comes up again, hopefully we'll have additional insights :) r. From ali.yakout@REDACTED Tue Aug 17 14:14:49 2010 From: ali.yakout@REDACTED (Ali Yakout) Date: Tue, 17 Aug 2010 14:14:49 +0200 Subject: [erlang-questions] Corba Help In-Reply-To: References: <591FA96352AC9946A403D860A60166520105941A@ESESSCMS0357.eemea.ericsson.se> <9b1b921f-5ee0-4e6d-a52c-b0f5c0c6b2a4@q35g2000yqn.googlegroups.com> Message-ID: <591FA96352AC9946A403D860A60166520112C1F5@ESESSCMS0357.eemea.ericsson.se> Hi, I faced a problem with the orber while decoding a 4 bytes wchar... > =ERROR REPORT==== 10-Aug-2010::08:17:16 === =================== Orber > ================= [723] cdr_decode:dec_type(4); unsupported wchar > =========================================== > ** exception throw: {'EXCEPTION', > {'DATA_CONVERSION',[],1163001856,'COMPLETED_NO'}} > in function corba:raise/1 Any workaround for this problem? /Ali -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Niclas Eklund Sent: Monday, August 09, 2010 10:31 AM To: Zvi Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Re: Corba Help Hello! Chapter 6 should take you through the process of creating a service rather easy, but here comes a very slimmed version. Run the example in an empty directory. %% Compile the attached file (my.idl) using the light_ifr option (reduces the footprints regarding memory usage): shell> erlc +"{light_ifr,true}" my.idl %% Create the call-back module (equivalent to *Impl.java): shell> erlc +"{be,erl_template}" my.idl %% Edit the module my_server_impl.erl echo(State, Str) -> io:format("~s\n", [Str]), %% Add this line. {reply, ok, State}. %% Compile shell> erlc *.erl %% "Jump-start" Orber. erl> orber:jump_start([{flags,16#80},{iiop_port,2809}]). %% Register interface in the IFR. erl> oe_my:oe_register(). %% Create an object instance. erl> Obj = my_server:oe_create([]). %% Invoke the echo operation. erl> my_server:echo(Obj, "It is easy to use CORBA"). It is easy to use CORBA To gain access to a server instance, simply stor the object reference in the name service and use the "corbaname" schema from the peer to get hold of it. I.e. the same way you'd do using, for example, a Java server. Best regards, Niclas On Fri, 6 Aug 2010, Zvi wrote: > Hi Niclas, > > I need a minimal example of using CORBA from Erlang. I.e. simple > "Hello World" stuff using orber application. > Can you give me any pointers / examples? > > Thanks in advance. > Zvi > > On Aug 5, 3:31?pm, Niclas Eklund wrote: >> Hello! >> >> Check out the chapter "6.8 Typecode, Identity and Name Access Functions" >> in Orber's User's Guide: >> >> http://www.erlang.org/doc/apps/orber/ch_idl_to_erlang_mapping.html#id... >> >> The record is defined in a generated hrl-file after you've compiled >> the IDL-file. Remember to include that file. >> >> In fact, you should read all in "6 OMG IDL to Erlang Mapping", since >> it describe step by step how one should do. >> >> The debugging chapter should also be helpfull >> -http://www.erlang.org/doc/apps/orber/ch_debugging.html >> >> You will probably find the IC backend erl_template rather usefull >> (will generate a complete call-back module). Note, it will overwrite >> any existing version of the *impl.erl file. Read more about it in chapter 6. >> >> If possible you should use Light IFR when compiling the IDL files and >> starting Orber. >> >> Best regards, >> >> Niclas @ Erlang/OTP >> >> >> >> On Thu, 5 Aug 2010, Ali Yakout wrote: >>> Hi, >> >>> I'm new to Corba and I can't find my way through... >>> I want to do the same operation done with the following Java code: >> >>> //----------------------------------------- >>> org.omg.CORBA.Any any = _orb.create_any(); >>> any.insert_wstring("1.1"); com.lhs.ccb.soi.types.NvElementI >>> namedValue = new com.lhs.ccb.soi.types.NvElementI("CS_CODE", any, 0, >>> (short)0); NvElementI[] input = new NvElementI[] {namedValue}; >>> NvElementI[] result = execute("CUSTOMERS.SEARCH", input); >>> //----------------------------------------- >> >>> My problem is that I don't know how to create an object of NvElement with the above values. >> >>> I have the following types: >> >>> %%------------------------------------------ >>> 254> com_lhs_ccb_soi_types_NvElementI:tc(). >>> {tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", >>> ? ? ? ? ? "NvElementI", >>> ? ? ? ? ? [{"name",{tk_string,0}}, >>> ? ? ? ? ? ?{"value",tk_any}, >>> ? ? ? ? ? ?{"flags",tk_long}, >>> ? ? ? ? ? ?{"length",tk_short}]} >>> 255> com_lhs_ccb_soi_types_NvElementListI:tc(). >>> {tk_sequence,{tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", >>> ? ? ? ? ? ? ? ? ? ? ? ?"NvElementI", >>> ? ? ? ? ? ? ? ? ? ? ? ?[{"name",{tk_string,0}}, >>> ? ? ? ? ? ? ? ? ? ? ? ? {"value",tk_any}, >>> ? ? ? ? ? ? ? ? ? ? ? ? {"flags",tk_long}, >>> ? ? ? ? ? ? ? ? ? ? ? ? {"length",tk_short}]}, >>> ? ? ? ? ? ? 0} >>> %%------------------------------------------ >> >>> I tried the following but I got 'MARSHAL' exception. >> >>> 244> AnyStr=any:create(orber_tc:wstring(3),"1.1"). >>> {any,{tk_wstring,3},"1.1"} >>> 245> NV=any:create(com_lhs_ccb_soi_types_NvElementI:tc(),["CS_CODE",AnyStr,0,0]). >>> {any,{tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", >>> ? ? ? ? ? ? ? ?"NvElementI", >>> ? ? ? ? ? ? ? ?[{"name",{tk_string,0}}, >>> ? ? ? ? ? ? ? ? {"value",tk_any}, >>> ? ? ? ? ? ? ? ? {"flags",tk_long}, >>> ? ? ? ? ? ? ? ? {"length",tk_short}]}, >>> ? ? ["CS_CODE",{any,{tk_wstring,3},"1.1"},0,0]} >>> 246> com_lhs_ccb_soi_ServiceObjectI:executeI(CMIObj,"CUSTOMERS.SEARCH",[NV]). >> >>> ** exception throw: >>> {'EXCEPTION',{'MARSHAL',[],1163001856,'COMPLETED_NO'}} >>> ? ? in function ?corba:raise/1 >>> ? ? in call from orber_iiop:request/8 >> >>> So my question is, how to create Corba types in Erlang? >> >>> Thanks, >>> Ali >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> Seehttp://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> Seehttp://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From michael.eugene.turner@REDACTED Tue Aug 17 15:27:20 2010 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 17 Aug 2010 22:27:20 +0900 Subject: [erlang-questions] shared data areas (was Re: [erlang-questions] OOP in Erlang) In-Reply-To: <4C66AC25.8030309@erlang-solutions.com> References: <4C627A5F.9060802@erlang-solutions.com> <4C63AA58.3040301@erlang-solutions.com> <4C66AC25.8030309@erlang-solutions.com> Message-ID: "... in my experience, from complex telephony and multimedia apps, it is in fact pretty common to have thousands of small 'clusters' of state machines, each cluster sharing a whopping big state record and taking turns processing different parts of the control flow." But what's behind the decision to structure things that way? I'm not sure what you mean by "processing different parts of the control flow"? Do you mean you had a kind of pipeline process structure? Taking Nicholas' view, but with due concern for performance: what about finding some way to privilege processes that manage big data structures? At the cost of a certain amount of global flow analysis, you might be able to automatically identify processes that were basically the system's Whopping Big State Record Managers (i.e., infinitely tail-recursive, passing Big State to itself each time, usually taking messages only in order to immediately send back fragments of Big State Record). Maybe you could then generate code that would do something more like a synchronous context switch (and back again) than a full message-send (with corresponding reply), when communicating with these processes. These record-access pseudo-messages might be made particularly fast for operations that do little more than read Big State data that never gets modified -- these probably being the overwhelming majority of accesses anyway. This could break hot-loading of modules, of course, depending on how you structure things. The interprocess coupling would go from being almost as loose as possible to almost as tight as a subroutine call. I doubt this idea would be practical if it required Whopping Big State Record Managers to be entirely local to a module. I don't see how you'd solve that problem in general. It would definitely be problematic across node boundaries, but perhaps this is less of an issue than how to permit hot loading. If your goal was to get higher speed for accesses of the Whopping Big State Record for a bunch of FSMs, you wouldn't be making those FSMs reach across node boundaries anyway. (These thoughts are somewhat inspired by an admittedly dilettantish interest in microkernel performance. For similar reasons, a number of similar issues about sharing, copying and performance arose in microkernel research. These issues seem to have been resolved pretty successfully in the L3/L4 microkernel family.) -michael turner On Sat, Aug 14, 2010 at 11:45 PM, Ulf Wiger wrote: > > Note: these are all hand-waving comments from me. > > > Nicholas Frechette wrote: > >> Co-locating processes on a shared heap on 1 scheduler might be dangerous. >> What if a process migrates to another core/scheduler? Should that be >> allowed? Blocked? >> > > An implementation issue, but yes, I imagine it could be forbidden, > or rather, the group would be considered as a whole for migration. > > > > What if one of those processes forks another one? Should it belong to the >> flock and be bound to that scheduler? >> > > This could be at the parent's discretion, using a new spawn_opt > option, e.g. 'same_heap', dictating that the child should use the > parent's heap. > > > > Should the message it is receiving dictate the scheduler it is going to >> process it on? >> > > No, I think this would be complicated, and can't see the need > for it. > > > > I think it will be quite hard to do transparently (sharing msg data >> between processes without copying) mainly because it might be hard to tell >> data that is a msg from data that isn't. >> > > But a lot of thinking already went into this. > Binaries are often off-heap, although in that case, you only need > to look at the type tag to know. > > In the case I suggested, the only check needed is for whether the > receiver is sharing the sender's heap. > > > > If a process sends a message then dies, you'll have to either: allocate >> the msg from the shared heap, somehow knowing or copying, or copy it in case >> the process heap gets garbage collected/freed (or prevent that from >> happening, which could have a detrimental effect on the system if the >> receiving process has a very large queue and takes a very long time before >> consuming said message) >> > > To my uneducated eye, this seems fairly similar to normal process > death + normal GC. If a process (not the last on that heap) dies, > the extra problem is that some data can be "freed" that is still > referenced by other processes. Something similar should have been > necessary for the 'shared heap' emulator to work in the first place. > > > This could be mitigated by adding a special syntax for message creation >> where they would be created on a shared heap (or a special syntax for shared >> heap allocation). Perhaps something like {Foo} = Whatever (or Foo }= >> Whatever.). Where 'Foo' would be copied on the shared heap. Binaries are >> already reference counted so it might not be too bad to implement. >> > > IMHO - no language changes! An option to spawn_opt is the right place > and scope. That's where you can peek under the covers and affect things > like GC parameters, which are otherwise fully transparent. > > > IMO though, the whole sending a large datastructure as a message is >> largely a non-issue. You can easily wrap it in a process and allocate it >> from that processe's heap and just pass the PID around. >> > > But in my experience, from complex telephony and multimedia apps, > it is in fact pretty common to have thousands of small 'clusters' > of state machines, each cluster sharing a whopping big state > record and taking turns processing different parts of the control flow. > > The suggestion is that this would work as an optimization for the > tuning stage of such an application, much like it is now possible > to tune heap size, GC thresholds, etc. > > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > http://www.erlang-solutions.com > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ulf.wiger@REDACTED Tue Aug 17 15:35:02 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 17 Aug 2010 15:35:02 +0200 Subject: [erlang-questions] shared data areas (was Re: [erlang-questions] OOP in Erlang) In-Reply-To: References: <4C627A5F.9060802@erlang-solutions.com> <4C63AA58.3040301@erlang-solutions.com> <4C66AC25.8030309@erlang-solutions.com> Message-ID: <4C6A9006.4090206@erlang-solutions.com> Michael Turner wrote: > "... in my experience, from complex telephony and multimedia apps, it is > in fact pretty common to have thousands of small 'clusters' of state > machines, each cluster sharing a whopping big state record and taking > turns processing different parts of the control flow." > > But what's behind the decision to structure things that way? The fact that the signaling carries all information as a single session, but when you break it apart, there are lots of different state machines in there. You split them into different processes in order to avoid interference between state machines. Still, the signaling protocol forces them all to pack the data into a single state. Your proposed alternative seems quite complicated compared to the (optimization) option of letting the FSMs share a common heap, but that is from a *very* cursory perusal. I will try to read it more closely later. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From zeno490@REDACTED Tue Aug 17 16:28:24 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Tue, 17 Aug 2010 10:28:24 -0400 Subject: [erlang-questions] shared data areas (was Re: [erlang-questions] OOP in Erlang) In-Reply-To: <4C6A9006.4090206@erlang-solutions.com> References: <4C627A5F.9060802@erlang-solutions.com> <4C63AA58.3040301@erlang-solutions.com> <4C66AC25.8030309@erlang-solutions.com> <4C6A9006.4090206@erlang-solutions.com> Message-ID: Perhaps erlang would benefit from a system similar to C# appdomains: Functions calls intra appdomain are simple calls, function calls extra appdomain are rpc calls where everything is copied. Obviously, this would translate to message passing for erlang and not necessarily function calls. Perhaps this could be implicit (ie: 1 domain per erlang application) or perhaps explicit (ie: as groups using spawn_opt as you suggested). Maybe it would make sense to create an EPP for a feature like the one you are requesting Ulf, since as you've shown, there are cases where the simplest design for a class of applications would benefit greatly from it (and thus prevent complicating the design to work around performance bottlenecks). Nicholas On Tue, Aug 17, 2010 at 9:35 AM, Ulf Wiger wrote: > Michael Turner wrote: > >> "... in my experience, from complex telephony and multimedia apps, it is >> in fact pretty common to have thousands of small 'clusters' of state >> machines, each cluster sharing a whopping big state record and taking turns >> processing different parts of the control flow." >> >> >> But what's behind the decision to structure things that way? >> > > The fact that the signaling carries all information as a single > session, but when you break it apart, there are lots of different > state machines in there. You split them into different processes > in order to avoid interference between state machines. Still, > the signaling protocol forces them all to pack the data into > a single state. > > Your proposed alternative seems quite complicated compared to > the (optimization) option of letting the FSMs share a common heap, > but that is from a *very* cursory perusal. I will try to read it > more closely later. > > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > http://www.erlang-solutions.com > From ingela@REDACTED Tue Aug 17 16:36:49 2010 From: ingela@REDACTED (Ingela Andin) Date: Tue, 17 Aug 2010 16:36:49 +0200 Subject: Active mode and packet http Message-ID: Hi! I would like to make a quick survey to find out if there is anyone out there that is using active once or active true socket option together with the inet option {packet, http | http_bin} . If you are please drop me a line Yaws and Mochiweb seems to use passive mode. Regards Ingela Erlang/OTP team - Ericsson AB From nick@REDACTED Tue Aug 17 16:38:49 2010 From: nick@REDACTED (Niclas Eklund) Date: Tue, 17 Aug 2010 16:38:49 +0200 (CEST) Subject: [erlang-questions] Corba Help In-Reply-To: <591FA96352AC9946A403D860A60166520112C1F5@ESESSCMS0357.eemea.ericsson.se> References: <591FA96352AC9946A403D860A60166520105941A@ESESSCMS0357.eemea.ericsson.se> <9b1b921f-5ee0-4e6d-a52c-b0f5c0c6b2a4@q35g2000yqn.googlegroups.com> <591FA96352AC9946A403D860A60166520112C1F5@ESESSCMS0357.eemea.ericsson.se> Message-ID: Hello! Depends on if the client you're using complies with wchar encoding information in the IOR. But changing the GIOP version to 1.1 should do the trick. See the Orber option 'giop_version' and make sure you use 1.1 in the corbaloc and/or corbaname string(s). Best regards, Niclas @ Erlang/OTP On Tue, 17 Aug 2010, Ali Yakout wrote: > Hi, > > I faced a problem with the orber while decoding a 4 bytes wchar... > >> =ERROR REPORT==== 10-Aug-2010::08:17:16 === =================== Orber >> ================= [723] cdr_decode:dec_type(4); unsupported wchar >> =========================================== >> ** exception throw: {'EXCEPTION', >> {'DATA_CONVERSION',[],1163001856,'COMPLETED_NO'}} >> in function corba:raise/1 > > Any workaround for this problem? > > /Ali > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Niclas Eklund > Sent: Monday, August 09, 2010 10:31 AM > To: Zvi > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Re: Corba Help > > > Hello! > > Chapter 6 should take you through the process of creating a service rather easy, but here comes a very slimmed version. Run the example in an empty directory. > > %% Compile the attached file (my.idl) using the light_ifr option (reduces the footprints regarding memory usage): > > shell> erlc +"{light_ifr,true}" my.idl > > %% Create the call-back module (equivalent to *Impl.java): > > shell> erlc +"{be,erl_template}" my.idl > > %% Edit the module my_server_impl.erl > > echo(State, Str) -> > io:format("~s\n", [Str]), %% Add this line. > {reply, ok, State}. > > %% Compile > > shell> erlc *.erl > > %% "Jump-start" Orber. > > erl> orber:jump_start([{flags,16#80},{iiop_port,2809}]). > > %% Register interface in the IFR. > > erl> oe_my:oe_register(). > > %% Create an object instance. > > erl> Obj = my_server:oe_create([]). > > %% Invoke the echo operation. > > erl> my_server:echo(Obj, "It is easy to use CORBA"). > It is easy to use CORBA > > To gain access to a server instance, simply stor the object reference in the name service and use the "corbaname" schema from the peer to get hold of it. I.e. the same way you'd do using, for example, a Java server. > > Best regards, > > Niclas > > On Fri, 6 Aug 2010, Zvi wrote: > >> Hi Niclas, >> >> I need a minimal example of using CORBA from Erlang. I.e. simple >> "Hello World" stuff using orber application. >> Can you give me any pointers / examples? >> >> Thanks in advance. >> Zvi >> >> On Aug 5, 3:31?pm, Niclas Eklund wrote: >>> Hello! >>> >>> Check out the chapter "6.8 Typecode, Identity and Name Access Functions" >>> in Orber's User's Guide: >>> >>> http://www.erlang.org/doc/apps/orber/ch_idl_to_erlang_mapping.html#id... >>> >>> The record is defined in a generated hrl-file after you've compiled >>> the IDL-file. Remember to include that file. >>> >>> In fact, you should read all in "6 OMG IDL to Erlang Mapping", since >>> it describe step by step how one should do. >>> >>> The debugging chapter should also be helpfull >>> -http://www.erlang.org/doc/apps/orber/ch_debugging.html >>> >>> You will probably find the IC backend erl_template rather usefull >>> (will generate a complete call-back module). Note, it will overwrite >>> any existing version of the *impl.erl file. Read more about it in chapter 6. >>> >>> If possible you should use Light IFR when compiling the IDL files and >>> starting Orber. >>> >>> Best regards, >>> >>> Niclas @ Erlang/OTP >>> >>> >>> >>> On Thu, 5 Aug 2010, Ali Yakout wrote: >>>> Hi, >>> >>>> I'm new to Corba and I can't find my way through... >>>> I want to do the same operation done with the following Java code: >>> >>>> //----------------------------------------- >>>> org.omg.CORBA.Any any = _orb.create_any(); >>>> any.insert_wstring("1.1"); com.lhs.ccb.soi.types.NvElementI >>>> namedValue = new com.lhs.ccb.soi.types.NvElementI("CS_CODE", any, 0, >>>> (short)0); NvElementI[] input = new NvElementI[] {namedValue}; >>>> NvElementI[] result = execute("CUSTOMERS.SEARCH", input); >>>> //----------------------------------------- >>> >>>> My problem is that I don't know how to create an object of NvElement with the above values. >>> >>>> I have the following types: >>> >>>> %%------------------------------------------ >>>> 254> com_lhs_ccb_soi_types_NvElementI:tc(). >>>> {tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", >>>> ? ? ? ? ? "NvElementI", >>>> ? ? ? ? ? [{"name",{tk_string,0}}, >>>> ? ? ? ? ? ?{"value",tk_any}, >>>> ? ? ? ? ? ?{"flags",tk_long}, >>>> ? ? ? ? ? ?{"length",tk_short}]} >>>> 255> com_lhs_ccb_soi_types_NvElementListI:tc(). >>>> {tk_sequence,{tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", >>>> ? ? ? ? ? ? ? ? ? ? ? ?"NvElementI", >>>> ? ? ? ? ? ? ? ? ? ? ? ?[{"name",{tk_string,0}}, >>>> ? ? ? ? ? ? ? ? ? ? ? ? {"value",tk_any}, >>>> ? ? ? ? ? ? ? ? ? ? ? ? {"flags",tk_long}, >>>> ? ? ? ? ? ? ? ? ? ? ? ? {"length",tk_short}]}, >>>> ? ? ? ? ? ? 0} >>>> %%------------------------------------------ >>> >>>> I tried the following but I got 'MARSHAL' exception. >>> >>>> 244> AnyStr=any:create(orber_tc:wstring(3),"1.1"). >>>> {any,{tk_wstring,3},"1.1"} >>>> 245> NV=any:create(com_lhs_ccb_soi_types_NvElementI:tc(),["CS_CODE",AnyStr,0,0]). >>>> {any,{tk_struct,"IDL:com/lhs/ccb/soi/types/NvElementI:1.0", >>>> ? ? ? ? ? ? ? ?"NvElementI", >>>> ? ? ? ? ? ? ? ?[{"name",{tk_string,0}}, >>>> ? ? ? ? ? ? ? ? {"value",tk_any}, >>>> ? ? ? ? ? ? ? ? {"flags",tk_long}, >>>> ? ? ? ? ? ? ? ? {"length",tk_short}]}, >>>> ? ? ["CS_CODE",{any,{tk_wstring,3},"1.1"},0,0]} >>>> 246> com_lhs_ccb_soi_ServiceObjectI:executeI(CMIObj,"CUSTOMERS.SEARCH",[NV]). >>> >>>> ** exception throw: >>>> {'EXCEPTION',{'MARSHAL',[],1163001856,'COMPLETED_NO'}} >>>> ? ? in function ?corba:raise/1 >>>> ? ? in call from orber_iiop:request/8 >>> >>>> So my question is, how to create Corba types in Erlang? >>> >>>> Thanks, >>>> Ali >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> Seehttp://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> Seehttp://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From roberto@REDACTED Tue Aug 17 16:43:34 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 17 Aug 2010 16:43:34 +0200 Subject: [erlang-questions] Active mode and packet http In-Reply-To: References: Message-ID: misultin uses {active, once} for headers, passive for html body, and {active, true} for websockets. http://code.google.com/p/misultin/ cheers, r. 2010/8/17 Ingela Andin : > Hi! > > I would like to make a quick survey to find out if there is anyone out > there that is > using active once or active true socket option together with the ?inet > option {packet, http | http_bin} . > If you are please drop me a line > > Yaws and Mochiweb seems to use passive mode. > > Regards Ingela Erlang/OTP team - Ericsson AB > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- ------------------------------------------------------------------- Roberto Ostinelli CTO, WideTag Inc. - Realtime, Social, Green widetag.com skype: rostinelli twitter: ostinelli mobile: +39 335 6 100 22 6 From bob@REDACTED Tue Aug 17 16:53:46 2010 From: bob@REDACTED (Bob Ippolito) Date: Tue, 17 Aug 2010 22:53:46 +0800 Subject: [erlang-questions] Active mode and packet http In-Reply-To: References: Message-ID: On Tue, Aug 17, 2010 at 10:36 PM, Ingela Andin wrote: > Hi! > > I would like to make a quick survey to find out if there is anyone out > there that is > using active once or active true socket option together with the ?inet > option {packet, http | http_bin} . > If you are please drop me a line > > Yaws and Mochiweb seems to use passive mode. I've played with {active, once}, it seems like it might be a little bit faster than passive. A future version of mochiweb might use it, at least for the request line and headers, similar to what misultin does. -bob From vinoski@REDACTED Tue Aug 17 17:35:00 2010 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 17 Aug 2010 11:35:00 -0400 Subject: [erlang-questions] Active mode and packet http In-Reply-To: References: Message-ID: On Tue, Aug 17, 2010 at 10:53 AM, Bob Ippolito wrote: > On Tue, Aug 17, 2010 at 10:36 PM, Ingela Andin wrote: >> Hi! >> >> I would like to make a quick survey to find out if there is anyone out >> there that is >> using active once or active true socket option together with the ?inet >> option {packet, http | http_bin} . >> If you are please drop me a line >> >> Yaws and Mochiweb seems to use passive mode. > > I've played with {active, once}, it seems like it might be a little > bit faster than passive. A future version of mochiweb might use it, at > least for the request line and headers, similar to what misultin does. Bob, I think you're right -- we just did some measurements last week that appeared to show that {active, once} is indeed a bit faster than passive mode. Ingela, I use {active, once} for HTTP headers in the system I work on. --steve From rick@REDACTED Tue Aug 17 17:43:51 2010 From: rick@REDACTED (Rick Moynihan) Date: Tue, 17 Aug 2010 16:43:51 +0100 Subject: Posting HTTP Form Data with Erlang Message-ID: Does anyone know if there is an API that I can use to post HTTP form data? I've looked at using inets:httpc, and it seems to have everything I need, except that it doesn't appear to allow you to pass it a list of key/value tuples and have them be encoded into the HTTP request as application/x-www-form-urlencoded data. Does anyone know if inets:httpc can do this, and if not whether there are any libraries I can easily use? Cheers, R. From roberto@REDACTED Tue Aug 17 18:00:47 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 17 Aug 2010 18:00:47 +0200 Subject: [erlang-questions] Active mode and packet http In-Reply-To: References: Message-ID: > Bob, I think you're right -- we just did some measurements last week > that appeared to show that {active, once} is indeed a bit faster than > passive mode. definitely. according to my benches, it was around 8-12% faster on R13B01. r. From rtrlists@REDACTED Tue Aug 17 18:46:35 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 17 Aug 2010 17:46:35 +0100 Subject: [erlang-questions] Posting HTTP Form Data with Erlang In-Reply-To: References: Message-ID: On Tue, Aug 17, 2010 at 4:43 PM, Rick Moynihan wrote: > Does anyone know if there is an API that I can use to post HTTP form data? > > I've looked at using inets:httpc, and it seems to have everything I > need, except that it doesn't appear to allow you to pass it a list of > key/value tuples and have them be encoded into the HTTP request as > application/x-www-form-urlencoded data. > > Does anyone know if inets:httpc can do this, and if not whether there > are any libraries I can easily use? > > Mochiweb provides a Request "object" to your callback function, and that has a parse_post/0 function, which'll give you back a proplist of key/value pairs from your POST body (and you can use parse_qs/0 for GET options after the ? on the url). Haven't used inets in a while. Robby From mfidelman@REDACTED Tue Aug 17 20:17:43 2010 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 17 Aug 2010 14:17:43 -0400 Subject: [erlang-questions] very large distributed implementation? In-Reply-To: References: <4C69AA87.9000605@meetinghouse.net> Message-ID: <4C6AD247.6030308@meetinghouse.net> Robert, et. al., Thanks for all the great pointers and comments. Miles Roberto Ostinelli wrote: > you may find some clues on this topic i opened some time ago: > http://www.erlang.org/cgi-bin/ezmlm-cgi?4:msp:47731 > > glad to see that this comes up again, hopefully we'll have additional > insights :) > > r. > -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From tony@REDACTED Wed Aug 18 00:34:05 2010 From: tony@REDACTED (Tony Rogvall) Date: Wed, 18 Aug 2010 00:34:05 +0200 Subject: [erlang-questions] Active mode and packet http In-Reply-To: References: Message-ID: <85E885E0-C2A8-49BC-AC49-785963E73DE9@rogvall.se> Of course I use the active once when ever possible (including http) This is how you can multiplex in one process. Why do you ask? /Tony On 17 aug 2010, at 16.36, Ingela Andin wrote: > Hi! > > I would like to make a quick survey to find out if there is anyone out > there that is > using active once or active true socket option together with the inet > option {packet, http | http_bin} . > If you are please drop me a line > > Yaws and Mochiweb seems to use passive mode. > > Regards Ingela Erlang/OTP team - Ericsson AB > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From steven.charles.davis@REDACTED Wed Aug 18 03:16:58 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 17 Aug 2010 18:16:58 -0700 (PDT) Subject: Active mode and packet http In-Reply-To: <85E885E0-C2A8-49BC-AC49-785963E73DE9@rogvall.se> References: <85E885E0-C2A8-49BC-AC49-785963E73DE9@rogvall.se> Message-ID: <30b62f20-1a0b-4f1f-8e17-9aff859ae371@y11g2000yqm.googlegroups.com> I'm experimenting with {active, true}, because I perceive it to be more "otp", and perhaps just to see whether the predicted doom/fear of flooding actually happens in practice on an http server connection... ...No load/profiling stats as yet. Probably a month or two before I'll get to those. /s From steven.charles.davis@REDACTED Wed Aug 18 03:19:08 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 17 Aug 2010 18:19:08 -0700 (PDT) Subject: Active mode and packet http In-Reply-To: <30b62f20-1a0b-4f1f-8e17-9aff859ae371@y11g2000yqm.googlegroups.com> References: <85E885E0-C2A8-49BC-AC49-785963E73DE9@rogvall.se> <30b62f20-1a0b-4f1f-8e17-9aff859ae371@y11g2000yqm.googlegroups.com> Message-ID: <9489db0b-2502-4a60-8517-8f60f1007f99@i13g2000yqd.googlegroups.com> However, I'm using binary/raw mode and parsing the http requests - so may not be of interest. On Aug 17, 8:16?pm, Steve Davis wrote: > I'm experimenting with {active, true}.... From michael.eugene.turner@REDACTED Wed Aug 18 05:26:36 2010 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 18 Aug 2010 12:26:36 +0900 Subject: [erlang-questions] shared data areas (was Re: [erlang-questions] OOP in Erlang) In-Reply-To: <4C6A9006.4090206@erlang-solutions.com> References: <4C627A5F.9060802@erlang-solutions.com> <4C63AA58.3040301@erlang-solutions.com> <4C66AC25.8030309@erlang-solutions.com> <4C6A9006.4090206@erlang-solutions.com> Message-ID: "Your proposed alternative seems quite complicated compared to the (optimization) option of letting the FSMs share a common heap, ..." It does complicate compilation, but at least it requires no changes to source code, and encourages loosely coupled encapsulation of large state records. Garbage collection issues seem to haunt this whole arena. I wonder if more compiler smarts would help here as well. I've read that almost all reference counting overhead (95%?) can be eliminated by optimizing compilers. Since using the memory hierarchy inefficiently is an increasing source of performance problems, even the remaining overhead of reference counting (i.e., in cases where the compiler can't resolve the liveness status of data at compile time) might be worthwhile. After all, heap use tends to be roughly stack-like. You'd probably get much better locality of reference in most cases if you collected stuff as soon as possible, or at least very soon after it had been orphaned. I don't see how SMP *necessarily* breaks reference-counting, though of course it complicates things. Processes that build but then "forget" complex data structures only to die not long after might run more slowly than before, since they'd be doing a lot of GC that wouldn't be necessary under an always-copy/infrequently-collect regime. But I wouldn't be surprised if everything else sped up, by reaping the advantages of (1) using short-term memory more efficiently, (2) using less memory overall for longer-term data by sharing large data structures, and (3) avoiding copy operations regardless of the size of the data. -michael turner On Tue, Aug 17, 2010 at 10:35 PM, Ulf Wiger wrote: > Michael Turner wrote: > >> "... in my experience, from complex telephony and multimedia apps, it is >> in fact pretty common to have thousands of small 'clusters' of state >> machines, each cluster sharing a whopping big state record and taking turns >> processing different parts of the control flow." >> >> >> But what's behind the decision to structure things that way? >> > > The fact that the signaling carries all information as a single > session, but when you break it apart, there are lots of different > state machines in there. You split them into different processes > in order to avoid interference between state machines. Still, > the signaling protocol forces them all to pack the data into > a single state. > > Your proposed alternative seems quite complicated compared to > the (optimization) option of letting the FSMs share a common heap, > but that is from a *very* cursory perusal. I will try to read it > more closely later. > > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > http://www.erlang-solutions.com > From tino.breddin@REDACTED Wed Aug 18 08:11:07 2010 From: tino.breddin@REDACTED (Tino Breddin) Date: Wed, 18 Aug 2010 08:11:07 +0200 Subject: [erlang-questions] Posting HTTP Form Data with Erlang In-Reply-To: References: Message-ID: <107C927F-AB60-4128-A2F8-9BD7E16491A8@gmail.com> Something like the following might work for you as well: httpc:request(post, {Url, [], "application/x-www-form-urlencoded, "key=value&key2=value2}, [], []). If required you could use edoc:escape_uri(String) for the individual values. Be sure to start inets before using httpc. Tino On Aug 17, 2010, at 6:46 PM, Robert Raschke wrote: > On Tue, Aug 17, 2010 at 4:43 PM, Rick Moynihan wrote: > >> Does anyone know if there is an API that I can use to post HTTP form data? >> >> I've looked at using inets:httpc, and it seems to have everything I >> need, except that it doesn't appear to allow you to pass it a list of >> key/value tuples and have them be encoded into the HTTP request as >> application/x-www-form-urlencoded data. >> >> Does anyone know if inets:httpc can do this, and if not whether there >> are any libraries I can easily use? >> >> > Mochiweb provides a Request "object" to your callback function, and that has > a parse_post/0 function, which'll give you back a proplist of key/value > pairs from your POST body (and you can use parse_qs/0 for GET options after > the ? on the url). > > Haven't used inets in a while. > > Robby From rick@REDACTED Wed Aug 18 10:44:40 2010 From: rick@REDACTED (Rick Moynihan) Date: Wed, 18 Aug 2010 09:44:40 +0100 Subject: [erlang-questions] Posting HTTP Form Data with Erlang In-Reply-To: <107C927F-AB60-4128-A2F8-9BD7E16491A8@gmail.com> References: <107C927F-AB60-4128-A2F8-9BD7E16491A8@gmail.com> Message-ID: On Wed, Aug 18, 2010 at 7:11 AM, Tino Breddin wrote: > Something like the following might work for you as well: > > httpc:request(post, {Url, [], "application/x-www-form-urlencoded, "key=value&key2=value2}, [], []). > > If required you could use edoc:escape_uri(String) for the individual values. Yes, I discovered that shortly after sending the email; prompting me to write the following function, rather than take on an external dependency, which I'm a little loathed to do for something so small: url_encode(Data) -> url_encode(Data,""). url_encode([],Acc) -> Acc; url_encode([{Key,Value}|R],"") -> url_encode(R, edoc_lib:escape_uri(Key) ++ "=" ++ edoc_lib:escape_uri(Value)); url_encode([{Key,Value}|R],Acc) -> url_encode(R, Acc ++ "&" ++ edoc_lib:escape_uri(Key) ++ "=" ++ edoc_lib:escape_uri(Value)). Thanks to everyone for your help, R. From mail@REDACTED Wed Aug 18 14:06:42 2010 From: mail@REDACTED (Tim Fletcher) Date: Wed, 18 Aug 2010 05:06:42 -0700 (PDT) Subject: Posting HTTP Form Data with Erlang In-Reply-To: <107C927F-AB60-4128-A2F8-9BD7E16491A8@gmail.com> References: <107C927F-AB60-4128-A2F8-9BD7E16491A8@gmail.com> Message-ID: <1cc988a2-b154-48e1-a427-67b5f61f6404@c10g2000yqi.googlegroups.com> > httpc:request(post, {Url, [], "application/x-www-form-urlencoded, "key=value&key2=value2}, [], []). > > If required you could use edoc:escape_uri(String) for the individual values. URI escaping and application/x-www-form-urlencoded escaping are not exactly the same thing though. There's an implementation of both here: http://github.com/tim/erlang-percent-encoding I'd appreciate any feedback on the correctness of the code/tests. > Yes, I discovered that shortly after sending the email; prompting me > to write the following function, rather than take on an external > dependency, which I'm a little loathed to do for something so small: Agreed. I usually copy/paste bits into other projects, for example: http://github.com/tim/erlang-webfinger/blob/master/src/webfinger.erl#L65-82 Cheers, Tim From rick@REDACTED Wed Aug 18 14:06:53 2010 From: rick@REDACTED (Rick Moynihan) Date: Wed, 18 Aug 2010 13:06:53 +0100 Subject: Eunit error messages Message-ID: Hi all, I've been using EUnit to do some unit testing; and it seems to work quite well. However, I'm having a couple of problems, mostly down to the legibility of error reports. In unit testing frameworks I've used in the past the assertion functions all took a string to provide a human readable description of what the error was, am I right in thinking EUnit doesn't support this? If so, what's the best way to replicate this behaviour, I've been manually adding debugMsg's but that's a bit messy as I also need to throw an exception to indicate failure. Any tips gratefully received, R. From magnus@REDACTED Wed Aug 18 14:39:07 2010 From: magnus@REDACTED (Magnus Henoch) Date: Wed, 18 Aug 2010 14:39:07 +0200 Subject: Eunit error messages In-Reply-To: (Rick Moynihan's message of "Wed, 18 Aug 2010 13:06:53 +0100") References: Message-ID: Rick Moynihan writes: > I've been using EUnit to do some unit testing; and it seems to work quite well. > > However, I'm having a couple of problems, mostly down to the > legibility of error reports. In unit testing frameworks I've used in > the past the assertion functions all took a string to provide a human > readable description of what the error was, am I right in thinking > EUnit doesn't support this? If so, what's the best way to replicate > this behaviour, I've been manually adding debugMsg's but that's a bit > messy as I also need to throw an exception to indicate failure. You can label your tests, as long as they are in EUnit's test representation: foo_test_() -> {"Testing whether foo is bar", ?_test(foo = bar)}. This gives: 1> foo:test(). foo:7: foo_test_ (Testing whether foo is bar)...*failed* ::error:{badmatch,bar} in function foo:'-foo_test_/0-fun-0-'/0 ======================================================= Failed: 1. Skipped: 0. Passed: 0. error Is that what you're looking for? Magnus From anthonym@REDACTED Wed Aug 18 23:40:14 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Wed, 18 Aug 2010 14:40:14 -0700 Subject: Supervisor with better restart strategies? Message-ID: <20100818214014.GD12371@alumni.caltech.edu> Hi, I've got a system which connects to another system. I'd like a pool of connections to that other system. I have something which works using a supervisor and pg2, however, the only restart strategy available in the supervisor doesn't play well with the back end service going down. I'd like some sort of supervisor which allows you to specify the number of attempted restarts with some backoff between tries (in other words instead of specifying {one_for_one, 10, 10} (which basically happens instantaneously when the backend is offline) I'd prefer something like {one_for_one, 10, 10, 100} (which would mean restart 10 times with 10 second sleep up to 100 seconds total, or something like that). Is there something like this already out there somewhere? Thanks, -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From mevans@REDACTED Wed Aug 18 23:56:39 2010 From: mevans@REDACTED (Evans, Matthew) Date: Wed, 18 Aug 2010 17:56:39 -0400 Subject: ibrowse question Message-ID: Hi, I have a simple ibrowse question. We are sending a POST out with the HTTP keep-alive header set. Assuming there are no active requests on that connection, and assuming no one has closed the socket on the remote end, would ibrowse reuse that current connection or open a new one? Thanks Matt From ulrich.moritz@REDACTED Thu Aug 19 00:12:53 2010 From: ulrich.moritz@REDACTED (Moritz Ulrich) Date: Thu, 19 Aug 2010 00:12:53 +0200 Subject: [erlang-questions] Supervisor with better restart strategies? In-Reply-To: <20100818214014.GD12371@alumni.caltech.edu> References: <20100818214014.GD12371@alumni.caltech.edu> Message-ID: Seconded. I was searching for a restart strategy like this too. On Wed, Aug 18, 2010 at 11:40 PM, Anthony Molinaro wrote: > Hi, > > ?I've got a system which connects to another system. ?I'd like a pool > of connections to that other system. ?I have something which works using > a supervisor and pg2, however, the only restart strategy available in > the supervisor doesn't play well with the back end service going down. > I'd like some sort of supervisor which allows you to specify the number > of attempted restarts with some backoff between tries (in other words > instead of specifying {one_for_one, 10, 10} (which basically happens > instantaneously when the backend is offline) I'd prefer something like > {one_for_one, 10, 10, 100} (which would mean restart 10 times with 10 > second sleep up to 100 seconds total, or something like that). > Is there something like this already out there somewhere? > > Thanks, > > -Anthony > > -- > ------------------------------------------------------------------------ > Anthony Molinaro ? ? ? ? ? ? ? ? ? ? ? ? ? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Moritz Ulrich Programmer, Student, Almost normal Guy http://www.google.com/profiles/ulrich.moritz BB5F086F-C798-41D5-B742-494C1E9677E8 From ro4tub@REDACTED Thu Aug 19 09:53:01 2010 From: ro4tub@REDACTED (Oscar) Date: Thu, 19 Aug 2010 15:53:01 +0800 Subject: how to design a scalable 'scheduler' app in erlang way Message-ID: hi all, Here are the scheduler's requirements: 1. Scheduler should receive a lot of requests from other apps. 2. Scheduler should send the received requests to 'worker' apps. 3. Scheduler should determine when to send the received requests to 'worker' apps. The strategy is: a. A request depends on something, named *resource*. b. When *resource* is idle, scheduler sends the request to one worker. c. When *resource* is busy, scheduler stores the request. 4. Scheduler should be scalable, supporting about 1million tps. I've implemented it in the shared-state way. TPS is about 100,000, and 'lock' consumes the most CPU time. Now, I want to implement an erlang version, but have no idea about it. Shall I store the resource status, pending request to ETS? Will ETS be a bottleneck? Thanks, -Oscar From roberto@REDACTED Thu Aug 19 13:24:50 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 19 Aug 2010 13:24:50 +0200 Subject: using a socket to send and receive Message-ID: dear list, i need to use a socket both to receive and send data. the socket must be listening for incoming data from TCP, but it needs also to receive data to be sent from external processes when events occur. the socket is in raw mode. i'm using a binary protocol which specifies that the length of the incoming data varies continuously. my current solution is to use 2 processes, one to listen for incoming data in passive mode, the other to listen for incoming erlang messages from other processes, in a simple way as: recv(Sock, Len) -> case gen_tpc:recv(Sock, Len) of [...] end. send(Sock) -> receive {send, DataToSend} -> gen_tpc:send(Sock, DataToSend) end. i use passive mode since it is very simple to set the length of the incoming data to wait for with gen_tpc:recv/2. what i would like to do is to use a single process in {active, once} mode, so that both messages received from the socket AND from the external processes are treated within a same loop. the solution would look something like: sock_loop(Sock, Len) -> inet:setopts(Sock, [{active, once}, {recv_size, Len}]), receive {http, Sock, Data} -> % data received from TCP socket, do something with it [...]; {send, DataToSend} -> % data received from external processes, send it to socket gen_tpc:send(Sock, DataToSend) end. problem is that the option 'recv_size' that i've specified here above obviously does not exist, and i can't find how to set the length of the packet to receive in the way that i can easily do with passive mode and gen_tpc:recv/2. the sndbuf inet option does not seem to be it, since it's a buffer size. i've read the docs but i guess i have missed a point somewhere. can anyone point me out on how to do this? thank you, r. From essiene@REDACTED Thu Aug 19 14:15:59 2010 From: essiene@REDACTED (Essien Essien) Date: Thu, 19 Aug 2010 13:15:59 +0100 Subject: [erlang-questions] Supervisor with better restart strategies? In-Reply-To: <20100818214014.GD12371@alumni.caltech.edu> References: <20100818214014.GD12371@alumni.caltech.edu> Message-ID: Hi Anthony, On Wed, Aug 18, 2010 at 10:40 PM, Anthony Molinaro wrote: > Hi, > > ?I've got a system which connects to another system. ?I'd like a pool > of connections to that other system. ?I have something which works using > a supervisor and pg2, however, the only restart strategy available in > the supervisor doesn't play well with the back end service going down. > I'd like some sort of supervisor which allows you to specify the number > of attempted restarts with some backoff between tries (in other words > instead of specifying {one_for_one, 10, 10} (which basically happens > instantaneously when the backend is offline) I'd prefer something like > {one_for_one, 10, 10, 100} (which would mean restart 10 times with 10 > second sleep up to 100 seconds total, or something like that). > Is there something like this already out there somewhere? I've been working on an SMPP gateway implementation and ran into this same problem when for connecting with the SMSC over a link that could go down. What I did was first to implement a generic "backoff" module. The algorithm is really really basic. It exposes: backoff:register/4 - register a process with the backoff module giving it a SPEC (pid, min, max, delta, MFA). Internally, timer:apply is used to call the MFA after 'min' time has passed. If the caller calls, increment/0, the wait time is updated to min+delta and this continues untill it gets to 'max' and basically stops there, untill the caller once again calls regular/0 then it starts again from min. backoff:deregister/0 - delete the previously added spec and stop calling MFA backoff:regular/0 - resets the wait counter backoff:increment/0 - increments the internal wait time by delta With this generic backoff module, I now created a generic "nanny" module. The nanny module does three things: start children - when starting, the nanny will attempt to start all children using a specified start MFA. After this, it registers with the backoff module sending it the babysit MFA. babysit children - when this is fired, the nanny walks through its children list and starts up the non-alive children wake children - the nanny can be asked by an external process to 'wake' its children. I use this to prompt some of the children that fetch tasks from a queue to proceed. I've not put much thought in making this useful any more generally than how I currently use these. This can most probably be done cleaner and better, or a mechanism could be built as a patch to supervisor according to your original suggestion (I initially) toyed with this idea, but eventually went for this disconnected approach. The entire project which makes use of this is not yet publicly released, but will be soon, so I have attached specific files. Hopefully, these shed some light on how they're used. I hope this comes in helpful, even slightly :) cheers, Essien > > Thanks, > > -Anthony > > -- > ------------------------------------------------------------------------ > Anthony Molinaro ? ? ? ? ? ? ? ? ? ? ? ? ? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -------------- next part -------------- A non-text attachment was scrubbed... Name: backoff.erl Type: text/x-erlang Size: 4667 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: nanny.erl Type: text/x-erlang Size: 6089 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: rx_sup.erl Type: text/x-erlang Size: 1401 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: tx_sup.erl Type: text/x-erlang Size: 1401 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mmayen.erl Type: text/x-erlang Size: 2468 bytes Desc: not available URL: From essiene@REDACTED Thu Aug 19 14:53:03 2010 From: essiene@REDACTED (Essien Essien) Date: Thu, 19 Aug 2010 13:53:03 +0100 Subject: [erlang-questions] using a socket to send and receive In-Reply-To: References: Message-ID: HI Roberto, On Thu, Aug 19, 2010 at 12:24 PM, Roberto Ostinelli wrote: > dear list, > > i need to use a socket both to receive and send data. the socket must > be listening for incoming data from TCP, but it needs also to receive > data to be sent from external processes when events occur. the socket > is in raw mode. > > i'm using a binary protocol which specifies that the length of the > incoming data varies continuously. > > my current solution is to use 2 processes, one to listen for incoming > data in passive mode, the other to listen for incoming erlang messages > from other processes, in a simple way as: > > recv(Sock, Len) -> > ? ? ? ?case gen_tpc:recv(Sock, Len) of > ? ? ? ? ? ? ? ?[...] > ? ? ? ?end. > > send(Sock) -> > ? ? ? ?receive > ? ? ? ? ? ? ? ?{send, DataToSend} -> gen_tpc:send(Sock, DataToSend) > ? ? ? ?end. > > i use passive mode since it is very simple to set the length of the > incoming data to wait for with gen_tpc:recv/2. > > what i would like to do is to use a single process in {active, once} > mode, so that both messages received from the socket AND from the > external processes are treated within a same loop. > > the solution would look something like: > > sock_loop(Sock, Len) -> > ? ? ? ?inet:setopts(Sock, [{active, once}, {recv_size, Len}]), > ? ? ? ?receive > ? ? ? ? ? ? ? ?{http, Sock, Data} -> > ? ? ? ? ? ? ? ? ? ? ? ?% data received from TCP socket, do something with it > ? ? ? ? ? ? ? ? ? ? ? ?[...]; > ? ? ? ? ? ? ? ?{send, DataToSend} -> > ? ? ? ? ? ? ? ? ? ? ? ?% data received from external processes, send it to socket > ? ? ? ? ? ? ? ? ? ? ? ?gen_tpc:send(Sock, DataToSend) > ? ? ? ?end. > > problem is that the option 'recv_size' that i've specified here above > obviously does not exist, and i can't find how to set the length of > the packet to receive in the way that i can easily do with passive > mode and gen_tpc:recv/2. the sndbuf inet option does not seem to be > it, since it's a buffer size. > > i've read the docs but i guess i have missed a point somewhere. > > can anyone point me out on how to do this? I've encountered this before where my process recieves a binary PDU stream from a server, but needs to break them up and unpack them properly before delivering to another process to deal with them. My approach is still to use {active, once}, thus recieving _all_ available data. When I process though, I only take the lenght of data that I need and leave the Rest in a loop variable. The next time around, I prepend the Rest from the previous run to the new coming data, then recurse. So, something like this: % I'm assuming that the socket is delivering % binary not list data. sock_loop(Sock, Len, Rest0) -> ? ? ? ?inet:setopts(Sock, [{active, once}]), ? ? ? ?receive ? ? ? ? ? ? ? ?{http, Sock, Data0} -> % data received from TCP socket, do something with it % first prepend pending data in Rest0 before continuing Data1 = list_to_binary([Rest0, Data0]), <> = Data1, % Now you can do what you want with Data worker_proc ! <>, sock_loop(Sock, Len, Rest1); ? ? ? ? ? ? ? ?{send, DataToSend} -> ? ? ? ? ? ? ? ? ? ? ? ?% data received from external processes, send it to socket ? ? ? ? ? ? ? ? ? ? ? ?gen_tpc:send(Sock, DataToSend) ? ? ? ?end. This does not exactly give you a way to modify the receive buffer, but still allows you to achieve what you need to. cheers, Essien > > thank you, > > r. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From roberto@REDACTED Thu Aug 19 15:22:55 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 19 Aug 2010 15:22:55 +0200 Subject: [erlang-questions] using a socket to send and receive In-Reply-To: References: Message-ID: 2010/8/19 Essien Essien : > > My approach is still to use {active, once}, thus recieving _all_ > available data. When I process though, I only take the lenght of data > that I need and leave the Rest in a loop variable. The next time > around, I prepend the Rest from the previous run to the new coming > data, then recurse. So, something like this: > > % I'm assuming that the socket is delivering > % binary not list data. > sock_loop(Sock, Len, Rest0) -> > ? ? ? ?inet:setopts(Sock, [{active, once}]), > ? ? ? ?receive > ? ? ? ? ? ? ? ?{http, Sock, Data0} -> > ? ? ? ? ? ? ? ? ? ? ? ?% data received from TCP socket, do something with it > ? ? ? ? ? ? ? ? ? ? ? ?% first prepend pending data in Rest0 before continuing > ? ? ? ? ? ? ? ? ? ? ? ?Data1 = list_to_binary([Rest0, Data0]), > ? ? ? ? ? ? ? ? ? ? ? ?<> = Data1, > > ? ? ? ? ? ? ? ? ? ? ? ? % Now you can do what you want with Data > ? ? ? ? ? ? ? ? ? ? ? ? ?worker_proc ! <>, > ? ? ? ? ? ? ? ? ? ? ? ? ? sock_loop(Sock, Len, Rest1); > ? ? ? ? ? ? ? ?{send, DataToSend} -> > ? ? ? ? ? ? ? ? ? ? ? ?% data received from external processes, send > it to socket > ? ? ? ? ? ? ? ? ? ? ? ?gen_tpc:send(Sock, DataToSend) > ? ? ? ?end. hi essien, first of all thank you for your reply. unfortunately this is not enough: you are assuming that you can match <> = Data1, which is not true if Len > what you have received. therefore, you would need additional checks. that apart, i don't want to go this way because i find it overcomplicated in respect to using passive mode. there really are no other alternatives? thank you, r. From kenrobinsonster@REDACTED Thu Aug 19 16:29:53 2010 From: kenrobinsonster@REDACTED (Ken Robinson) Date: Fri, 20 Aug 2010 00:29:53 +1000 Subject: [erlang-questions] Stopping of Erlang app hangs when mnesia stopped from within the program In-Reply-To: References: <4C657387.1020801@erlang-solutions.com> Message-ID: Whoops, forget to send it to the erlang list. Boot file worked. Ken. On Fri, Aug 20, 2010 at 12:27 AM, Ken Robinson wrote: > Hi Mazen, > I used a boot file as you suggested and it worked fine. many thanks > Ken. > > On Sat, Aug 14, 2010 at 2:32 AM, Mazen Harake > wrote: >> ?The application controller is already busy trying to stop your app, when >> you do mnesia:stop() I suspect it is trying to call the application >> controller to stop it but the application controller is already busy trying >> to stop your app, when you do mnesia:stop() I suspect it is trying to call >> the application controller to stop it but the application controller is >> already busy trying to stop your app. >> >> :) >> >> Use a boot file to handle startup of your applications or a function which >> starts mnesia and then starts your app (and the reverse in a stop function). >> >>> Should I not use these prototype functions within a fully fledged erlang >>> app? >> No you shouldn't >> >> /Mazen >> >> On 13/08/2010 17:02, Ken Robinson wrote: >>> >>> Hi all, >>> >>> I've run into a problem where I cannot stop mnesia within my program >>> without causing the app to hang. >>> >>> I'm presently doing prototyping of mnesia within my erlang app. >>> >>> In my jaus_app.erl file the start() calls: >>> >>> {atomic, ok} = mnesia:load_textfile("priv/mnesia_prototype.txt") >>> >>> My stop() function calls: >>> >>> mnesia:dump_to_textfile("priv/mnesia_prototype_res.txt"), >>> mnesia:stop(), >>> >>> When I comment out these lines and start and stop mnesia from the >>> erlang prompt, I am able to stop my application cleanly. >>> >>> Should I not use these prototype functions within a fully fledged erlang >>> app? >>> >>> Ken. >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> > From essiene@REDACTED Thu Aug 19 17:17:36 2010 From: essiene@REDACTED (Essien Essien) Date: Thu, 19 Aug 2010 16:17:36 +0100 Subject: [erlang-questions] using a socket to send and receive In-Reply-To: References: Message-ID: Hi Roberto, On Thu, Aug 19, 2010 at 2:22 PM, Roberto Ostinelli wrote: > 2010/8/19 Essien Essien : >> >> My approach is still to use {active, once}, thus recieving _all_ >> available data. When I process though, I only take the lenght of data >> that I need and leave the Rest in a loop variable. The next time >> around, I prepend the Rest from the previous run to the new coming >> data, then recurse. So, something like this: >> >> % I'm assuming that the socket is delivering >> % binary not list data. >> sock_loop(Sock, Len, Rest0) -> >> ? ? ? ?inet:setopts(Sock, [{active, once}]), >> ? ? ? ?receive >> ? ? ? ? ? ? ? ?{http, Sock, Data0} -> >> ? ? ? ? ? ? ? ? ? ? ? ?% data received from TCP socket, do something with it >> ? ? ? ? ? ? ? ? ? ? ? ?% first prepend pending data in Rest0 before continuing >> ? ? ? ? ? ? ? ? ? ? ? ?Data1 = list_to_binary([Rest0, Data0]), >> ? ? ? ? ? ? ? ? ? ? ? ?<> = Data1, >> >> ? ? ? ? ? ? ? ? ? ? ? ? % Now you can do what you want with Data >> ? ? ? ? ? ? ? ? ? ? ? ? ?worker_proc ! <>, >> ? ? ? ? ? ? ? ? ? ? ? ? ? sock_loop(Sock, Len, Rest1); >> ? ? ? ? ? ? ? ?{send, DataToSend} -> >> ? ? ? ? ? ? ? ? ? ? ? ?% data received from external processes, send >> it to socket >> ? ? ? ? ? ? ? ? ? ? ? ?gen_tpc:send(Sock, DataToSend) >> ? ? ? ?end. > > hi essien, > > first of all thank you for your reply. > > unfortunately this is not enough: you are assuming that you can match > <> = Data1, which is not true if Len > what you > have received. therefore, you would need additional checks. I hear your concerns. In my case, I got away lucky since parsing the PDU always meant I had to do these checks anyways. > > that apart, i don't want to go this way because i find it > overcomplicated in respect to using passive mode. true. > > there really are no other alternatives? Not that I know of unfortunately. cheers, Essien > > thank you, > > r. > From hynek@REDACTED Thu Aug 19 17:26:01 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Thu, 19 Aug 2010 17:26:01 +0200 Subject: [erlang-questions] using a socket to send and receive In-Reply-To: References: Message-ID: sock_loop(Sock, Len, Rest0) when size(Rest0) >= Len -> % We have enough data <> = Rest0, worker_proc ! {data, Data}, sock_loop(Sock, Len, Rest1); sock_loop(Sock, Len, Rest) -> inet:setopts(Sock, [{active, once}]), receive {http, Sock, Data} -> sock_loop(Sock, Len, <>) {send, DataToSend} -> % data received from external processes, send it to socket gen_tpc:send(Sock, DataToSend) sock_loop(Sock, Len, Rest) end. It is still too complicated? On Thu, Aug 19, 2010 at 3:22 PM, Roberto Ostinelli wrote: > 2010/8/19 Essien Essien : >> >> My approach is still to use {active, once}, thus recieving _all_ >> available data. When I process though, I only take the lenght of data >> that I need and leave the Rest in a loop variable. The next time >> around, I prepend the Rest from the previous run to the new coming >> data, then recurse. So, something like this: >> >> % I'm assuming that the socket is delivering >> % binary not list data. >> sock_loop(Sock, Len, Rest0) -> >> ? ? ? ?inet:setopts(Sock, [{active, once}]), >> ? ? ? ?receive >> ? ? ? ? ? ? ? ?{http, Sock, Data0} -> >> ? ? ? ? ? ? ? ? ? ? ? ?% data received from TCP socket, do something with it >> ? ? ? ? ? ? ? ? ? ? ? ?% first prepend pending data in Rest0 before continuing >> ? ? ? ? ? ? ? ? ? ? ? ?Data1 = list_to_binary([Rest0, Data0]), >> ? ? ? ? ? ? ? ? ? ? ? ?<> = Data1, >> >> ? ? ? ? ? ? ? ? ? ? ? ? % Now you can do what you want with Data >> ? ? ? ? ? ? ? ? ? ? ? ? ?worker_proc ! <>, >> ? ? ? ? ? ? ? ? ? ? ? ? ? sock_loop(Sock, Len, Rest1); >> ? ? ? ? ? ? ? ?{send, DataToSend} -> >> ? ? ? ? ? ? ? ? ? ? ? ?% data received from external processes, send >> it to socket >> ? ? ? ? ? ? ? ? ? ? ? ?gen_tpc:send(Sock, DataToSend) >> ? ? ? ?end. > > hi essien, > > first of all thank you for your reply. > > unfortunately this is not enough: you are assuming that you can match > <> = Data1, which is not true if Len > what you > have received. therefore, you would need additional checks. > > that apart, i don't want to go this way because i find it > overcomplicated in respect to using passive mode. > > there really are no other alternatives? > > thank you, > > r. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From roberto@REDACTED Thu Aug 19 18:47:29 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 19 Aug 2010 18:47:29 +0200 Subject: [erlang-questions] using a socket to send and receive In-Reply-To: References: Message-ID: 2010/8/19 Hynek Vychodil : > sock_loop(Sock, Len, Rest0) when size(Rest0) >= Len -> > ? ? ? % We have enough data > ? ? ? <> = Rest0, > ? ? ? worker_proc ! {data, Data}, > ? ? ? sock_loop(Sock, Len, Rest1); > sock_loop(Sock, Len, Rest) -> > ? ? ? inet:setopts(Sock, [{active, once}]), > ? ? ? receive > ? ? ? ? ? ? ? {http, Sock, Data} -> > ? ? ? ? ? ? ? ? ? ? ? sock_loop(Sock, Len, <>) > ? ? ? ? ? ? ? {send, DataToSend} -> > ? ? ? ? ? ? ? ? ? ? ? % data received from external processes, send > it to socket > ? ? ? ? ? ? ? ? ? ? ? gen_tpc:send(Sock, DataToSend) > ? ? ? ? ? ? ? ? ? ? ? sock_loop(Sock, Len, Rest) > ? ? ? end. > > It is still too complicated? no, definitely a considerable improvement on readability. thank you, r. From zabrane3@REDACTED Thu Aug 19 18:48:07 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Thu, 19 Aug 2010 18:48:07 +0200 Subject: [erlang-questions] using a socket to send and receive In-Reply-To: References: Message-ID: A lazy one, in the same spirit of Hynek solution: sock_loop_lazy(Sock, Len) -> sock_loop_lazy(Sock, Len, [], 0). sock_loop_lazy(Sock, Len, BuffList, BuffLen) when Len > BuffLen -> inet:setopts(Sock, [{active, once}]), receive {http, Sock, Data} -> sock_loop_lazy(Sock, Len, [Data | BuffList], BuffLen + size(Data)); {send, DataToSend} -> % data received from external processes, send it to socket gen_tpc:send(Sock, DataToSend), sock_loop_lazy(Sock, Len, BuffList, BuffLen) end; sock_loop_lazy(Sock, Len, BuffList, BuffLen) -> % We have enough data <> = list_to_binary(lists:reverse(BuffList)), worker_proc ! {data, Data}, sock_loop_lazy(Sock, Len, Rest1, BuffLen - Len). /Zab From zabrane3@REDACTED Thu Aug 19 18:51:15 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Thu, 19 Aug 2010 18:51:15 +0200 Subject: [erlang-questions] using a socket to send and receive In-Reply-To: References: Message-ID: A small typo in the last call: [...] sock_loop_lazy(Sock, Len, [Rest1], BuffLen - Len). 2010/8/19 zabrane Mikael : > A lazy one, in the same spirit of Hynek solution: > > sock_loop_lazy(Sock, Len) -> > ? ?sock_loop_lazy(Sock, Len, [], 0). > > sock_loop_lazy(Sock, Len, BuffList, BuffLen) when Len > BuffLen -> > ? inet:setopts(Sock, [{active, once}]), > ? ?receive > ? ? ? ?{http, Sock, Data} -> > ? ? ? ? ? ?sock_loop_lazy(Sock, Len, [Data | BuffList], BuffLen + size(Data)); > ? ? ? ?{send, DataToSend} -> > ? ? ? ? ? ?% data received from external processes, send it to socket > ? ? ? ? ? ?gen_tpc:send(Sock, DataToSend), > ? ? ? ? ? ?sock_loop_lazy(Sock, Len, BuffList, BuffLen) > ? ?end; > > sock_loop_lazy(Sock, Len, BuffList, BuffLen) ?-> > ? ?% We have enough data > ? ?<> = list_to_binary(lists:reverse(BuffList)), > ? ?worker_proc ! {data, Data}, > ? ?sock_loop_lazy(Sock, Len, Rest1, BuffLen - Len). > > > /Zab > From brian@REDACTED Thu Aug 19 19:04:58 2010 From: brian@REDACTED (Brian Lonsdorf) Date: Thu, 19 Aug 2010 10:04:58 -0700 Subject: Generic xml traversal. Message-ID: <86D3A83D-7F02-4A27-8EAC-03DB0E2F71A1@trnsfr.com> I apologize if this is a repost for some, I think I posted it in some old buried thread by accident yesterday. Hi everyone! I'm very excited to be starting my erlang journey, but I've been stuck on this for a few days now. I'm trying to create key, value pair tuples out of xml. I'd like to make a list out of nested xml too. For instance: The Name! WE 92 WS 90 Should turn out like: [{"name", "The Name!"}, {"reviews", [{"review-by", "WE"}, {"review-points", 92}], {"review-by", "WS"}, {"review-points", 90}]} ] Where Item is my main node that i want to get lists of. I've admittedly cargo culted and tweaked the code below. It only returns a list of the first Item's elements. And I'm not sure how to begin the nested ones. Thanks so much! -Brian ----------------------------- -module(reader). -compile(export_all). -include_lib("xmerl/include/xmerl.hrl"). parse(FileName) -> {Records,_} = xmerl_scan:file(FileName), extract(Records, []). extract(Record, Acc) when is_record(Record, xmlElement) -> case Record#xmlElement.name of 'Item' -> ItemData = lists:foldl(fun extract/2, [], Record#xmlElement.content), [ {item, ItemData} | Acc ]; _ -> lists:foldl(fun extract/2, Acc, Record#xmlElement.content) end; extract({xmlText, [{Attribute, _}, {'Item', 2}, _], _, _, Value, text}, Acc) -> [{Attribute, Value}|Acc]; extract(_, Acc) -> Acc. ------------------------------- I've also seen this done with list comprehension. It's beautiful, but i can't get it to nest: parse(FileName) -> {#xmlElement{content=Item}, _} = xmerl_scan:file(FileName), [{Tag, Val} || #xmlElement{content=Query} <- Item, #xmlElement{content=Fields} <- Query, #xmlText{parents=[{Tag, _} | _], value=Val} <- Fields]. From hpjcon@REDACTED Thu Aug 19 20:28:55 2010 From: hpjcon@REDACTED (Jan Jacobs) Date: Thu, 19 Aug 2010 20:28:55 +0200 Subject: Erlang runtime handle leak? Message-ID: <4C6D77E7.8030006@mweb.co.za> Hi, I noticed an increase of handles been used by the runtime (erl). Using handle.exe utility on the erl runtime I get the following: Start: Handle v3.42 Copyright (C) 1997-2008 Mark Russinovich Sysinternals - www.sysinternals.com Handle type summary: ALPC Port : 4 Desktop : 1 Directory : 8 EtwRegistration : 25 Event : 2075 File : 78 IoCompletion : 5 Key : 18 KeyedEvent : 2 Mutant : 2 Process : 15 Semaphore : 16 Thread : 1046 Timer : 6 TpWorkerFactory : 16 WindowStation : 2 Total handles: 3342 Hour Later: Handle v3.42 Copyright (C) 1997-2008 Mark Russinovich Sysinternals - www.sysinternals.com Handle type summary: ALPC Port : 4 Desktop : 1 Directory : 8 EtwRegistration : 25 Event : 2152 File : 78 IoCompletion : 5 Key : 18 KeyedEvent : 2 Mutant : 2 Process : 15 Semaphore : 16 Thread : 1046 Timer : 6 TpWorkerFactory : 16 WindowStation : 2 Total handles: 3390 The Event handles increased by 77. I am not sure if it is a result of me not closing socket/files correctly? I do not know the internals of the VM, which actions in the code can cause this? Platform and runtime: Windows 2008 64Bit R14A Basic summary of application operation: Read from socket and write to file. (No SSL) Read from file and write to socket. (No SSL) Web/SOAP operations with SSL. ODBC Database interactions. Running as a service. Any help or suggestions will be appreciated. Cheers Jan From jesper.louis.andersen@REDACTED Thu Aug 19 21:24:12 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 19 Aug 2010 21:24:12 +0200 Subject: A dialyzer question on improper list detection of list appends Message-ID: I have some code which goes like this (in gproc): qlc_next(_, '$end_of_table') -> []; qlc_next(Scope, K) -> case ets:lookup(?TAB, K) of [{{Key,_}, Pid, V}] -> [{Key,Pid,V} | fun() -> qlc_next(Scope, next(Scope, K)) end]; [] -> qlc_next(Scope, next(Scope, K)) end. The dialyzer does not like it, it reports: gproc.erl:1149: Cons will produce an improper list since its 2nd argument is fun(() -> maybe_improper_list({_,_,_},fun(() -> maybe_improper_list(any(),fun(() -> any()) | [])) | [])) This is entirely true and the warning is in addition good. Improper lists is somewhat of a mistake :) Since this is a problem in general with QLCs, and qlc_next/1 I wondered how it was fixed for, say, ETS tables in Erlang. In lib/stdlib/src/ets.erl we find: qlc_next(_Tab, '$end_of_table') -> []; qlc_next(Tab, Key) -> ets:lookup(Tab, Key) ++ fun() -> qlc_next(Tab, ets:next(Tab, Key)) end. Hmm, clever! So I change the code from gproc to: @@ -1157,7 +1146,7 @@ qlc_next(_, '$end_of_table') -> []; qlc_next(Scope, K) -> case ets:lookup(?TAB, K) of [{{Key,_}, Pid, V}] -> - [{Key,Pid,V} | fun() -> qlc_next(Scope, next(Scope, K)) end]; + [{Key,Pid,V}] ++ fun() -> qlc_next(Scope, next(Scope, K)) end; [] -> qlc_next(Scope, next(Scope, K)) end. @@ -1166,7 +1155,7 @@ qlc_prev(_, '$end_of_table') -> []; qlc_prev(Scope, K) -> case ets:lookup(?TAB, K) of [{{Key,_},Pid,V}] -> - [{Key,Pid,V} | fun() -> qlc_prev(Scope, prev(Scope, K)) end]; + [{Key,Pid,V}] ++ fun() -> qlc_prev(Scope, prev(Scope, K)) end; [] -> qlc_prev(Scope, prev(Scope, K)) end. and magically, the Dialyzer warning disappears. Now the million dollar question is: Why? Both are equivalent improper lists, right? Maybe Kostis or someone else with dialyzer-fu can shed some light on this? -- J. From kostis@REDACTED Thu Aug 19 21:46:04 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 19 Aug 2010 22:46:04 +0300 Subject: [erlang-questions] A dialyzer question on improper list detection of list appends In-Reply-To: References: Message-ID: <4C6D89FC.5030202@cs.ntua.gr> Jesper Louis Andersen wrote: > I have some code which goes like this (in gproc): > > qlc_next(_, '$end_of_table') -> []; > qlc_next(Scope, K) -> > case ets:lookup(?TAB, K) of > [{{Key,_}, Pid, V}] -> > [{Key,Pid,V} | fun() -> qlc_next(Scope, next(Scope, K)) end]; > [] -> > qlc_next(Scope, next(Scope, K)) > end. > > The dialyzer does not like it, it reports: > > gproc.erl:1149: Cons will produce an improper list since its 2nd > argument is fun(() -> maybe_improper_list({_,_,_},fun(() -> > maybe_improper_list(any(),fun(() -> any()) | [])) | [])) > > This is entirely true and the warning is in addition good. Improper > lists is somewhat of a mistake :) Since this is a problem in general > with QLCs, and qlc_next/1 I wondered how it was fixed for, say, ETS > tables in Erlang. In lib/stdlib/src/ets.erl we find: > > qlc_next(_Tab, '$end_of_table') -> > []; > qlc_next(Tab, Key) -> > ets:lookup(Tab, Key) ++ fun() -> qlc_next(Tab, ets:next(Tab, Key)) end. > > Hmm, clever! So I change the code from gproc to: > > @@ -1157,7 +1146,7 @@ qlc_next(_, '$end_of_table') -> []; > qlc_next(Scope, K) -> > case ets:lookup(?TAB, K) of > [{{Key,_}, Pid, V}] -> > - [{Key,Pid,V} | fun() -> qlc_next(Scope, next(Scope, K)) end]; > + [{Key,Pid,V}] ++ fun() -> qlc_next(Scope, next(Scope, K)) end; > [] -> > qlc_next(Scope, next(Scope, K)) > end. > @@ -1166,7 +1155,7 @@ qlc_prev(_, '$end_of_table') -> []; > qlc_prev(Scope, K) -> > case ets:lookup(?TAB, K) of > [{{Key,_},Pid,V}] -> > - [{Key,Pid,V} | fun() -> qlc_prev(Scope, prev(Scope, K)) end]; > + [{Key,Pid,V}] ++ fun() -> qlc_prev(Scope, prev(Scope, K)) end; > [] -> > qlc_prev(Scope, prev(Scope, K)) > end. > > and magically, the Dialyzer warning disappears. Now the million dollar > question is: Why? Both are equivalent improper lists, right? Maybe > Kostis or someone else with dialyzer-fu can shed some light on this? It's really very very simple. Pay attention to the fine line of the warning: "Cons will produce ..." In the ++ case there is no explicit cons involved ;-) Now, can I have my million dollars? :) Seriously though, the core of the problem is that you are making an assumption which does not hold. You are implicitly assuming that dialyzer is a tool which is *sound for correctness*: i.e., capable of detecting a complete set of warnings (of a certain kind). It is NOT. It's explicitly designed to be a tool which is *sound for defect detection*: i.e., it will never give you a false warning but you have no guarantee that it will give you all warnings. (Granted of course that in this particular case, it's damn easy to special-case ++/2 also, and indeed we might do this when and if we feel like it... not likely to happen soon though, so you are safe for the time being.) Cheers, Kostis From jesper.louis.andersen@REDACTED Thu Aug 19 22:07:43 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 19 Aug 2010 22:07:43 +0200 Subject: [erlang-questions] A dialyzer question on improper list detection of list appends In-Reply-To: <4C6D89FC.5030202@cs.ntua.gr> References: <4C6D89FC.5030202@cs.ntua.gr> Message-ID: On Thu, Aug 19, 2010 at 9:46 PM, Kostis Sagonas wrote: > > It's really very very simple. > > Pay attention to the fine line of the warning: "Cons will produce ..." > In the ++ case there is no explicit cons involved ;-) > > Now, can I have my million dollars? :) Oh, hehe. That makes sense. > Seriously though, the core of the problem is that you are making an > assumption which does not hold. You are implicitly assuming that dialyzer is > a tool which is *sound for correctness*: i.e., capable of detecting a > complete set of warnings (of a certain kind). It is NOT. Ah, right. There is a strong similarity to Type I and Type II errors in statistics here. You can have: The tool says Yes (no warnings present) or No The code may have a defect or not. So there are 4 outcomes: Yes/Defect, No/Defect, Yes/Correct, No/Correct. It is interesting how the dialyzer is different from a type checker in that respect, because it puts the emphasis differently and tries to eliminate outcomes differently. In particular the *sound for defect detection* means that the No/Correct should be 0. Contrast with a type checker which may fail to type check a program which has no defects at all, due to the type system being too weak. On the contrary the type checker seem to have the upper hand in the Yes/Defect case, where it makes a mistake rarer than a tool like the dialyzer. -- J. From anthonym@REDACTED Fri Aug 20 06:50:38 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Thu, 19 Aug 2010 21:50:38 -0700 Subject: [erlang-questions] ibrowse question In-Reply-To: References: Message-ID: <20100820045038.GB16042@alumni.caltech.edu> Hi, The default sessions for ibrowse is 10 and the max pipeline size is 10, so I believe it will spawn 10 connections and queue up to 10 requests per connection. You can see the current state by opening a remote shell and running ibrowse:get_dest_status() which will show you active connections. There's also a get_dest_status/2 which takes the host and port and shows you more information. You can also configure these numbers based on the host and port as well (although I've not done that before). -Anthony On Wed, Aug 18, 2010 at 05:56:39PM -0400, Evans, Matthew wrote: > Hi, > > I have a simple ibrowse question. We are sending a POST out with the HTTP keep-alive header set. Assuming there are no active requests on that connection, and assuming no one has closed the socket on the remote end, would ibrowse reuse that current connection or open a new one? > > Thanks > > Matt -- ------------------------------------------------------------------------ Anthony Molinaro From chandrashekhar.mullaparthi@REDACTED Fri Aug 20 08:11:33 2010 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 20 Aug 2010 07:11:33 +0100 Subject: [erlang-questions] ibrowse question In-Reply-To: References: Message-ID: On 18 August 2010 22:56, Evans, Matthew wrote: > Hi, > > I have a simple ibrowse question. We are sending a POST out with the HTTP > keep-alive header set. Assuming there are no active requests on that > connection, and assuming no one has closed the socket on the remote end, > would ibrowse reuse that current connection or open a new one? > > When you make a new request, if the number of connections is less than the maximum specified (10 by default), ibrowse will spawn a new connection. Only if it has reached that maximum, it will start pipelining requests on to existing connections. cheers Chandru From gordon@REDACTED Fri Aug 20 09:30:43 2010 From: gordon@REDACTED (Gordon Guthrie) Date: Fri, 20 Aug 2010 08:30:43 +0100 Subject: Leex And Character Encodings Message-ID: I'm hitting some problems with the character encoding for Leex. I have a front end which is submitting proper unicode in utf-8 format and the utf-8 is round-tripping correctly - I submit it from the webpage in a Jquery post, it is processed on the back end and then returned to the front end in utf-8 where it displays correctly... During the back end processing I need to feed it through leex to generate user actions - certain posts contain a domain specific language. The DSL is fairly well specified and strings in it are quoted so they just pass through the lexer in utf-8 and are fine and dandy and we so some processing on them in unicode - by running language parsers over the lexical token stream. The utf-8 just streams through the parser as single character stream and we don't care... The problem is that the white space elements of the DSL get knocked about, so two spaces are turned into ? It seems to me that I can't expect lex to work with utf-8 natively and I just have to suck it up and create a whitespace lexical token that matches ? Or am I just being a fool? Gordon -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From roberto@REDACTED Fri Aug 20 16:17:37 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 20 Aug 2010 16:17:37 +0200 Subject: [erlang-questions] using a socket to send and receive In-Reply-To: References: Message-ID: 2010/8/19 zabrane Mikael : > A small typo in the last call: > ?[...] > ? ?sock_loop_lazy(Sock, Len, [Rest1], BuffLen - Len). thank you all. still, i wish you could set the amount of bytes to receive from tcp when setting the {active, once} option in inet, in a way similar to: gen_tcp:setopts(Sock, [{active, once}, {recv_len, 16}]). hopefully the otp team will consider this a nice-to-have little addition. cheers, r. From gnoblin@REDACTED Fri Aug 20 18:15:49 2010 From: gnoblin@REDACTED (gnoblin) Date: Fri, 20 Aug 2010 09:15:49 -0700 (PDT) Subject: random lookup in ets Message-ID: <77f9f4ca-7fa7-4425-8f56-7b101955e40c@h19g2000yqb.googlegroups.com> hello how can I randomly pick an element from ets and what is the most efficient way of doing it? thank you From rapsey@REDACTED Fri Aug 20 19:14:53 2010 From: rapsey@REDACTED (Rapsey) Date: Fri, 20 Aug 2010 19:14:53 +0200 Subject: cdata xmerl export Message-ID: How do I produce an xml with values inside <[CDATA[....]]>? This isn't working: Xml = [{blabla,[#xmlText{type = cdata, value = "sometext"}]}], lists:flatten(xmerl:export_simple(L, xmerl_xml, [{prolog, [""]}])) Sergej From gnoblin@REDACTED Fri Aug 20 19:36:57 2010 From: gnoblin@REDACTED (gnoblin) Date: Fri, 20 Aug 2010 10:36:57 -0700 (PDT) Subject: json serialization of nested records Message-ID: <03732fa5-aff6-4fc1-97f6-7ea676e07874@f42g2000yqn.googlegroups.com> hello! is it possible to serialize a record to json (I use json.erl) like this: http://gist.github.com/538245 ? I have an error in my console when I try this. thanks From g9414002.pccu.edu.tw@REDACTED Fri Aug 20 20:39:42 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Sat, 21 Aug 2010 02:39:42 +0800 Subject: [erlang-questions] json serialization of nested records In-Reply-To: <03732fa5-aff6-4fc1-97f6-7ea676e07874@f42g2000yqn.googlegroups.com> References: <03732fa5-aff6-4fc1-97f6-7ea676e07874@f42g2000yqn.googlegroups.com> Message-ID: T1 = #topscore{place = <<"1">>, user_id = <<"Us">>, score = <<"200">>}, T2 = #topscore{place = <<"2">>, user_id = <<"Us2">>, score = <<"400">>}, Scores = #topscores{scores = [T1|T2]}, I think that you used [_|_] instead of [T1,T2], and it made a list with it rest as a record. Maybe #topscores{scores=[T1,T2]} would work. On Sat, Aug 21, 2010 at 1:36 AM, gnoblin wrote: > hello! > > is it possible to serialize a record to json (I use json.erl) like > this: > > http://gist.github.com/538245 ? > > I have an error in my console when I try this. > > thanks > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ngreco@REDACTED Fri Aug 20 21:34:41 2010 From: ngreco@REDACTED (Nahuel Greco) Date: Fri, 20 Aug 2010 16:34:41 -0300 Subject: [erlang-questions] cdata xmerl export In-Reply-To: References: Message-ID: The 'type' attribute seems to be ignored in xmerl.erl, probably there is no support for emit it right now (missing feature or bug ? :) ). Saludos, Nahuel Greco. On Fri, Aug 20, 2010 at 2:14 PM, Rapsey wrote: > How do I produce an xml with values inside <[CDATA[....]]>? This isn't > working: > > Xml = [{blabla,[#xmlText{type = cdata, value = "sometext"}]}], > lists:flatten(xmerl:export_simple(L, xmerl_xml, [{prolog, [" version=\"1.0\" encoding=\"utf-8\"?>"]}])) > > > > Sergej > From armando@REDACTED Fri Aug 20 21:39:43 2010 From: armando@REDACTED (Armando Di Cianno) Date: Fri, 20 Aug 2010 15:39:43 -0400 Subject: How to stop a detached OTP app? Message-ID: I'm most certainly doing something braindead, which is stopping me from being able to nicely stop an OTP app I've written. I've written a script to help me start, stop, and connect to the app for debugging purposes. However, no variation of 'stop' I've tried seems to work. Here's the script: ### start #!/usr/bin/env bash USER="myuser" APP="myapp" HOSTNAME=`hostname -f` NODE="${APP}@${HOSTNAME}" RELEASE_PATH="/usr/local/lib/${APP}" HOME="/var/lib/${APP}" LOG_PATH="/var/log/${APP}" case "$1" in "start") su ${USER} -p -c \ "erl -pa ${RELEASE_PATH}/ebin -name ${NODE} -boot ${APP} -mnesia dir '\"${HOME}\"' -sasl sasl_error_logger '{file, \"${LOG_PATH}/erlang.log\"}' -detached" ;; "stop") su ${USER} -p -c \ "erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE} -eval \"init:stop()\" -noshell" ;; "debug") echo "*** Use Ctrl-C Ctrl-C to exit. ***" su ${USER} -p -c \ "erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE}" ;; *) echo "Please use 'start', 'stop', or 'debug." exit 1 ;; esac exit 0 ### end For stopping, I've tried: * erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE} -eval "init:stop()" * erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE} -s init stop * erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE} -s ${APP} -extra stop #where I've added some command arg processing to the app ... and some other variantions. If I connect to the app using the debug mode of my script, and issue init:stop(), everything shuts down nicely. Thanks for any insights into this. Cheers, __armando From luismarianoguerra@REDACTED Fri Aug 20 22:21:55 2010 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Fri, 20 Aug 2010 17:21:55 -0300 Subject: calling a local function with an atom stored in a variable. Message-ID: hi, a question appeared in the spanish mailing list and we couldn't find the answer. from the erlang reference: http://www.erlang.org/doc/reference_manual/expressions.html#id2273323 """ ExprF(Expr1,...,ExprN) ExprM:ExprF(Expr1,...,ExprN) In the first form of function calls, ExprM:ExprF(Expr1,...,ExprN), each of ExprM and ExprF must be an atom or an expression that evaluates to an atom. The function is said to be called by using the fully qualified function name. This is often referred to as a remote or external function call. Example: lists:keysearch(Name, 1, List) In the second form of function calls, ExprF(Expr1,...,ExprN), ExprF must be an atom or evaluate to a fun. """ the last line " ExprF must be an atom or evaluate to a fun", why can't ExprF evaluate to an atom? why ExprF = foo, ExprF() doesn't work but ?MODULE:ExprF() does? From ulf.wiger@REDACTED Fri Aug 20 22:25:02 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 20 Aug 2010 22:25:02 +0200 Subject: [erlang-questions] How to stop a detached OTP app? In-Reply-To: References: Message-ID: <4C6EE49E.3090108@erlang-solutions.com> How about this? #!/usr/bin/env escript %% -*- erlang -*- %%! -name ctl main([N]) -> Res = rpc:call(list_to_atom(N), init, stop, []), io:fwrite("==> ~p~n", [Res]). It worked for me. Call it with e.g. ./stop thenode@REDACTED BR, Ulf W Armando Di Cianno wrote: > I'm most certainly doing something braindead, which is stopping me > from being able to nicely stop an OTP app I've written. > > I've written a script to help me start, stop, and connect to the app > for debugging purposes. However, no variation of 'stop' I've tried > seems to work. > > Here's the script: > ### start > #!/usr/bin/env bash > > USER="myuser" > APP="myapp" > HOSTNAME=`hostname -f` > NODE="${APP}@${HOSTNAME}" > RELEASE_PATH="/usr/local/lib/${APP}" > HOME="/var/lib/${APP}" > LOG_PATH="/var/log/${APP}" > > case "$1" in > "start") > su ${USER} -p -c \ > "erl -pa ${RELEASE_PATH}/ebin -name ${NODE} -boot ${APP} -mnesia > dir '\"${HOME}\"' -sasl sasl_error_logger '{file, > \"${LOG_PATH}/erlang.log\"}' -detached" > ;; > "stop") > su ${USER} -p -c \ > "erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE} > -eval \"init:stop()\" -noshell" > ;; > "debug") > echo "*** Use Ctrl-C Ctrl-C to exit. ***" > su ${USER} -p -c \ > "erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE}" > ;; > *) > echo "Please use 'start', 'stop', or 'debug." > exit 1 > ;; > esac > > exit 0 > ### end > > For stopping, I've tried: > * erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE} -eval > "init:stop()" > * erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE} -s init stop > * erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE} -s > ${APP} -extra stop #where I've added some command arg processing to > the app > ... and some other variantions. > > If I connect to the app using the debug mode of my script, and issue > init:stop(), everything shuts down nicely. > > Thanks for any insights into this. Cheers, > __armando > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From mazen.harake@REDACTED Fri Aug 20 21:35:36 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Fri, 20 Aug 2010 22:35:36 +0300 Subject: [erlang-questions] calling a local function with an atom stored in a variable. In-Reply-To: References: Message-ID: <4C6ED908.4030606@erlang-solutions.com> On 20/08/2010 23:21, Mariano Guerra wrote: > the last line " ExprF must be an atom or evaluate to a fun", why can't > ExprF evaluate to an atom? why ExprF = foo, ExprF() doesn't work but > ?MODULE:ExprF() does? Try: -module(m). -compile(export_all). foo() -> (bar()):(baz())(). bar() -> m. baz() -> biz. biz() -> io:format("Hello World"). Should work. Hope it sheds some light. From ulf.wiger@REDACTED Fri Aug 20 22:36:07 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 20 Aug 2010 22:36:07 +0200 Subject: [erlang-questions] How to stop a detached OTP app? In-Reply-To: <4C6EE49E.3090108@erlang-solutions.com> References: <4C6EE49E.3090108@erlang-solutions.com> Message-ID: <4C6EE737.6060600@erlang-solutions.com> Or, if you really want to set the controller node name automatically: #!/usr/bin/env escript %% -*- erlang -*- main([Me, N]) -> {ok, _} = net_kernel:start([list_to_atom(Me), longnames]), Res = rpc:call(list_to_atom(N), init, stop, []), io:fwrite("==> ~p~n", [Res]). You could complement it with net tick time and cookie settings. http://www.erlang.org/pipermail/erlang-questions/2008-April/034357.html BR, Ulf W Ulf Wiger wrote: > > How about this? > > #!/usr/bin/env escript > %% -*- erlang -*- > %%! -name ctl > > main([N]) -> > Res = rpc:call(list_to_atom(N), init, stop, []), > io:fwrite("==> ~p~n", [Res]). > > It worked for me. > > Call it with e.g. ./stop thenode@REDACTED > > BR, > Ulf W > > Armando Di Cianno wrote: >> I'm most certainly doing something braindead, which is stopping me >> from being able to nicely stop an OTP app I've written. >> >> I've written a script to help me start, stop, and connect to the app >> for debugging purposes. However, no variation of 'stop' I've tried >> seems to work. >> >> Here's the script: >> ### start >> #!/usr/bin/env bash >> >> USER="myuser" >> APP="myapp" >> HOSTNAME=`hostname -f` >> NODE="${APP}@${HOSTNAME}" >> RELEASE_PATH="/usr/local/lib/${APP}" >> HOME="/var/lib/${APP}" >> LOG_PATH="/var/log/${APP}" >> >> case "$1" in >> "start") >> su ${USER} -p -c \ >> "erl -pa ${RELEASE_PATH}/ebin -name ${NODE} -boot ${APP} >> -mnesia >> dir '\"${HOME}\"' -sasl sasl_error_logger '{file, >> \"${LOG_PATH}/erlang.log\"}' -detached" >> ;; >> "stop") >> su ${USER} -p -c \ >> "erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh >> ${NODE} >> -eval \"init:stop()\" -noshell" >> ;; >> "debug") >> echo "*** Use Ctrl-C Ctrl-C to exit. ***" >> su ${USER} -p -c \ >> "erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh >> ${NODE}" >> ;; >> *) >> echo "Please use 'start', 'stop', or 'debug." >> exit 1 >> ;; >> esac >> >> exit 0 >> ### end >> >> For stopping, I've tried: >> * erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE} -eval >> "init:stop()" >> * erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE} -s >> init stop >> * erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE} -s >> ${APP} -extra stop #where I've added some command arg processing to >> the app >> ... and some other variantions. >> >> If I connect to the app using the debug mode of my script, and issue >> init:stop(), everything shuts down nicely. >> >> Thanks for any insights into this. Cheers, >> __armando >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From mazen.harake@REDACTED Fri Aug 20 21:43:29 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Fri, 20 Aug 2010 22:43:29 +0300 Subject: [erlang-questions] How to stop a detached OTP app? In-Reply-To: <4C6EE737.6060600@erlang-solutions.com> References: <4C6EE49E.3090108@erlang-solutions.com> <4C6EE737.6060600@erlang-solutions.com> Message-ID: <4C6EDAE1.6070300@erlang-solutions.com> I would also recommend adding the -hidden flag for the temporary node in case the node you are connecting to is part of a cluster otherwise funny things could happen. /Mazen On 20/08/2010 23:36, Ulf Wiger wrote: > Or, if you really want to set the controller node name > automatically: > > #!/usr/bin/env escript > %% -*- erlang -*- > > main([Me, N]) -> > {ok, _} = net_kernel:start([list_to_atom(Me), longnames]), > Res = rpc:call(list_to_atom(N), init, stop, []), > io:fwrite("==> ~p~n", [Res]). > > > You could complement it with net tick time and cookie settings. > > http://www.erlang.org/pipermail/erlang-questions/2008-April/034357.html > > BR, > Ulf W > > Ulf Wiger wrote: >> >> How about this? >> >> #!/usr/bin/env escript >> %% -*- erlang -*- >> %%! -name ctl >> >> main([N]) -> >> Res = rpc:call(list_to_atom(N), init, stop, []), >> io:fwrite("==> ~p~n", [Res]). >> >> It worked for me. >> >> Call it with e.g. ./stop thenode@REDACTED >> >> BR, >> Ulf W >> >> Armando Di Cianno wrote: >>> I'm most certainly doing something braindead, which is stopping me >>> from being able to nicely stop an OTP app I've written. >>> >>> I've written a script to help me start, stop, and connect to the app >>> for debugging purposes. However, no variation of 'stop' I've tried >>> seems to work. >>> >>> Here's the script: >>> ### start >>> #!/usr/bin/env bash >>> >>> USER="myuser" >>> APP="myapp" >>> HOSTNAME=`hostname -f` >>> NODE="${APP}@${HOSTNAME}" >>> RELEASE_PATH="/usr/local/lib/${APP}" >>> HOME="/var/lib/${APP}" >>> LOG_PATH="/var/log/${APP}" >>> >>> case "$1" in >>> "start") >>> su ${USER} -p -c \ >>> "erl -pa ${RELEASE_PATH}/ebin -name ${NODE} -boot ${APP} >>> -mnesia >>> dir '\"${HOME}\"' -sasl sasl_error_logger '{file, >>> \"${LOG_PATH}/erlang.log\"}' -detached" >>> ;; >>> "stop") >>> su ${USER} -p -c \ >>> "erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh >>> ${NODE} >>> -eval \"init:stop()\" -noshell" >>> ;; >>> "debug") >>> echo "*** Use Ctrl-C Ctrl-C to exit. ***" >>> su ${USER} -p -c \ >>> "erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh >>> ${NODE}" >>> ;; >>> *) >>> echo "Please use 'start', 'stop', or 'debug." >>> exit 1 >>> ;; >>> esac >>> >>> exit 0 >>> ### end >>> >>> For stopping, I've tried: >>> * erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE} -eval >>> "init:stop()" >>> * erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE} -s >>> init stop >>> * erl -pa ${RELEASE_PATH}/ebin -name ctl_${NODE} -remsh ${NODE} -s >>> ${APP} -extra stop #where I've added some command arg processing to >>> the app >>> ... and some other variantions. >>> >>> If I connect to the app using the debug mode of my script, and issue >>> init:stop(), everything shuts down nicely. >>> >>> Thanks for any insights into this. Cheers, >>> __armando >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> > > From chandrashekhar.mullaparthi@REDACTED Sat Aug 21 10:24:37 2010 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Sat, 21 Aug 2010 09:24:37 +0100 Subject: [erlang-questions] random lookup in ets In-Reply-To: <77f9f4ca-7fa7-4425-8f56-7b101955e40c@h19g2000yqb.googlegroups.com> References: <77f9f4ca-7fa7-4425-8f56-7b101955e40c@h19g2000yqb.googlegroups.com> Message-ID: On 20 August 2010 17:15, gnoblin wrote: > hello > > how can I randomly pick an element from ets and what is the most > efficient way of doing it? > > Is ets:lookup_element/3 what you need? cheers Chandru From gordon@REDACTED Sat Aug 21 10:37:20 2010 From: gordon@REDACTED (Gordon Guthrie) Date: Sat, 21 Aug 2010 09:37:20 +0100 Subject: [erlang-questions] Leex And Character Encodings In-Reply-To: References: Message-ID: Robby The front end supports full unicode and the inputs are expressions like in Excel. The valid expressions consist of: * functions (hundreds with unicode names) * operators (+ - * / &) * types ("quoted unicode strings" 1 true #ERRVAL! {date, time}) - the dates are matched out of strings * cell addresses and paths (a1 ./a1 ../../some/path/bob33 /a/path/a2 !a!path!a3) The paths are lower-cased unicode and can't contain spaces and certain characters - the cell addresses are latin alphabet based... The front end sends everything utf-8 so some characters are encoded with 2, 3 or 4 bytes... Best to just do a round trip set of examples. Stick at expression in the front-end, store it in the db and it returns both the original expression and the result of evaluating it to the front-end. This table Front-End Back-End Expr Returned Val Returned ?????? ??????????? ?????? ?????? ="????? " " =\"?????????? \"" ="????? " ?????? ="?? " & "???" =\"???? \" & \"??????\"" ="?? " & "???" ?????? =1+2 =1+2 =1+2 3 = 1 + 2 = 1 + 2 = 1 + 2 3 The problem comes when I put spaces in the white space: = 1 + 2 "=? 1 +? ? ? ? 2" = 1 + 2 #ERROR! The expression round trips fine but (unlike the previous examples) the server-side expression returns an error for the value because the expression doesn't match any valid syntax. Tabs are expanded to white spaces so the only problem (I think) is with multiple white spaces - which is why I think just adding a lexical token to make ? the same as 2 spaces would work. The problem is that it is just fugly :( Gordon On 20 August 2010 10:12, Robert Raschke wrote: > Hi Gordon, > > > On Fri, Aug 20, 2010 at 8:30 AM, Gordon Guthrie wrote: > >> I'm hitting some problems with the character encoding for Leex. >> >> I have a front end which is submitting proper unicode in utf-8 format and >> the utf-8 is round-tripping correctly - I submit it from the webpage in a >> Jquery post, it is processed on the back end and then returned to the >> front >> end in utf-8 where it displays correctly... >> >> During the back end processing I need to feed it through leex to generate >> user actions - certain posts contain a domain specific language. >> >> The DSL is fairly well specified and strings in it are quoted so they just >> pass through the lexer in utf-8 and are fine and dandy and we so some >> processing on them in unicode - by running language parsers over the >> lexical >> token stream. The utf-8 just streams through the parser as single >> character >> stream and we don't care... >> >> The problem is that the white space elements of the DSL get knocked about, >> so two spaces are turned into ? >> >> It seems to me that I can't expect lex to work with utf-8 natively and I >> just have to suck it up and create a whitespace lexical token that matches >> ? >> >> Or am I just being a fool? >> >> Gordon >> >> -- >> Gordon Guthrie >> CEO hypernumbers >> >> http://hypernumbers.com >> t: hypernumbers >> +44 7776 251669 >> > > That doesn't sound right. Which whitespaces are getting trashed? Ones > between your DSL elements, or ones inside your quoted strings? > > I'm assuming that just your quoted strings contain non-7bit-ascii. Or does > your DSL have elements outside that? > > If only the strings have utf-8, then what are you doing to "process" them? > > Question, questions, > Robby > > PS You can buy me a pint some evening and bring your code, if you like :-)) > > -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From mail@REDACTED Sat Aug 21 13:48:37 2010 From: mail@REDACTED (Tim Fletcher) Date: Sat, 21 Aug 2010 04:48:37 -0700 (PDT) Subject: Generic xml traversal. In-Reply-To: <86D3A83D-7F02-4A27-8EAC-03DB0E2F71A1@trnsfr.com> References: <86D3A83D-7F02-4A27-8EAC-03DB0E2F71A1@trnsfr.com> Message-ID: <45297fb9-1056-471c-8153-40a73c58575d@h19g2000yqb.googlegroups.com> > Where Item is my main node that i want to get lists of. In that case you could use XPath to select the list of 'Item' elements, and then traverse each of those. For example: parse_file(Name) -> {XML, []} = xmerl_scan:file(Name), [parse(Item#xmlElement.content) || Item <- xmerl_xpath:string("// something/Item", XML)]. parse(Content) -> lists:foldl(fun (I, A) -> parse(I, A) end, [], Content). parse(#xmlElement{name=Name, content=[#xmlText{value=Value}]}, Acc) -> [{Name, Value} | Acc]; parse(#xmlElement{name=Name, content=Content}, Acc) -> [{Name, parse(Content)} | Acc]; parse(_, Acc) -> Acc. That doesn't produce the exact output you posted, but it's reasonably generic. Your example output uses strings for the tag names, but xmerl uses atoms. Sometimes you might want atoms. Your example output doesn't include the 'review' tag names; not sure whether that's a mistake or whether you deliberately want to filter those out? Really depends on what you're doing with the data. Hope that helps anyway. Tim From romanshestakov@REDACTED Sat Aug 21 16:51:24 2010 From: romanshestakov@REDACTED (Roman Shestakov) Date: Sat, 21 Aug 2010 14:51:24 +0000 (GMT) Subject: what is the best way to test Erlang FSM processes? Message-ID: <809001.55126.qm@web25402.mail.ukl.yahoo.com> hello, I have a dependency tree of connected fsm processes, the state of children fsm' depends on state of parents and in some cases, timers attached to fsm can also trigger state change. I would like to be able to test the state of entire system with Common Test but can't figure out a good way to use CT for testing fsm states. does anybody have some ideas if it is possible to use CT in such scenario? any examples? other ideas? Regards, Roman From g9414002.pccu.edu.tw@REDACTED Sat Aug 21 22:49:41 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Sun, 22 Aug 2010 04:49:41 +0800 Subject: On Starting A Remote Shell Programmatically Message-ID: Hi, all. I want to write something like MapReduce on a set of nodes, based on Hewlett-Packard High Performance Computer. SSH is allowed that I can jump from one node to another without password, but RSH is not permitted to use. I have a question: give one another hostname, can I write Erlang program to start a remote shell or a remote job on it? Best Regards, Y.H.H. From g9414002.pccu.edu.tw@REDACTED Sat Aug 21 23:37:23 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Sun, 22 Aug 2010 05:37:23 +0800 Subject: [erlang-questions] On Starting A Remote Shell Programmatically In-Reply-To: <20100821205254.20867.qmail@ps03.myhostcenter.com> References: <20100821205254.20867.qmail@ps03.myhostcenter.com> Message-ID: I found it by myself. To start an Erlang shell on SSH, use erl -rsh ssh command. To start a remote shell, use ssh Target-host "erl -rsh ssh -sname Node-name-you-want" & and prey no other problem that causes the remote shell to fail. Cheers, Y.H.H. From watson.timothy@REDACTED Sat Aug 21 23:38:58 2010 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 21 Aug 2010 22:38:58 +0100 Subject: [erlang-questions] On Starting A Remote Shell Programmatically In-Reply-To: References: Message-ID: On 21 Aug 2010, at 21:49, ??? (Yau-Hsien Huang) wrote: > Hi, all. I want to write something like MapReduce on a set of nodes, > based on Hewlett-Packard High Performance Computer. SSH is > allowed that I can jump from one node to another without password, > but RSH is not permitted to use. > > I have a question: give one another hostname, can I write Erlang > program to start a remote shell or a remote job on it? > > Best Regards, > Y.H.H. There is an ssh application distributed with OTP which should meet your needs. It'll allow you to open an SSH connection and send commands over it, so assuming you have installed the emulator on the remote machine, you should be able to spin up another node. If you're planning on writing a MapReduce framework, may I suggest that you look at riak_core (the underpinning framework behind riak), and luke (http://github.com/basho/luke) as these things have been done before. There is also some excellent stuff in scalaris (http://code.google.com/p/scalaris/) for coordinating work (paxos based IIRC) and the disco project (http://discoproject.org/) has implemented a MapReduce framework too, although jobs are written in python rather than Erlang. From watson.timothy@REDACTED Sat Aug 21 23:41:23 2010 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 21 Aug 2010 22:41:23 +0100 Subject: [erlang-questions] On Starting A Remote Shell Programmatically In-Reply-To: References: <20100821205254.20867.qmail@ps03.myhostcenter.com> Message-ID: There is an ssh application distributed with OTP which should meet your needs. It'll allow you to open an SSH connection and send commands over it, so assuming you have installed the emulator on the remote machine, you should be able to spin up another node. If you're planning on writing a MapReduce framework, may I suggest that you look at riak_core (the underpinning framework behind riak), and luke (http://github.com/basho/luke) as these things have been done before. There is also some excellent stuff in scalaris (http://code.google.com/p/scalaris/) for coordinating work (paxos based IIRC) and the disco project (http://discoproject.org/) has implemented a MapReduce framework too, although jobs are written in python rather than Erlang. On 21 Aug 2010, at 22:37, ??? (Yau-Hsien Huang) wrote: > I found it by myself. To start an Erlang shell on SSH, use > > erl -rsh ssh > > command. > > To start a remote shell, use > > ssh Target-host "erl -rsh ssh -sname Node-name-you-want" & > > and prey no other problem that causes the remote shell to fail. > > Cheers, > Y.H.H. From g9414002.pccu.edu.tw@REDACTED Sun Aug 22 18:46:13 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Mon, 23 Aug 2010 00:46:13 +0800 Subject: [erlang-questions] On Starting A Remote Shell Programmatically In-Reply-To: References: Message-ID: Thank you. Now I can use SSH to execute a remote command as "erl -rsh ssh -sname gandolf" , and eventually it sometimes show only the starting message of Erlang Shell and sometimes the starting message accompanied with the exit message. Then, How to write an Erlang program to robustly keep an alive remote shell? Related information is as following: % Commander test(Host, NodeName) -> erlang:open_port({spawn,"ssh -f " ++ Host ++ " \"erl -rsh ssh -sname " ++ NodeName ++ "\" &", [{env,[{"PATH",os:getenv("PATH")}]}]), receive AnyThing -> io:format("Receive:" ~w~n.",[AnyThing]), AnyThing after 5000 -> {error,no_response} end. ----------------------------- [xxxxxxx@REDACTED ~]$ erl -rsh ssh -sname me 1> c(test). {ok,test} 2> test:test("HPC9", "gandolf"). Receive: {#Port<0.126>,{data,[69,115,... ]}} {#Port<0.126>,{data,"Eshell V5.6.1 (abort with ^G)\n"}} 3> net_adm:ping(gandolf@REDACTED). pang Or, 2> {_,{S,D}} = test:test("HPC11", "mary"), io:format("~w:~s~n",[S,D]). Receive: {#Port<0.132>,{data,[69,115,... ]}} data:(gandolf@REDACTED) 1> *** Terminating erlang (gandolf@REDACTED) ok 3>halt(). 2010/8/22 Tim Watson > > There is an ssh application distributed with OTP which should meet your > needs. It'll allow you to open an SSH connection and send commands over it, > so assuming you have installed the emulator on the remote machine, you > should be able to spin up another node. If you're planning on writing a > MapReduce framework, may I suggest that you look at riak_core (the > underpinning framework behind riak), and luke ( > http://github.com/basho/luke) as these things have been done before. There > is also some excellent stuff in scalaris ( > http://code.google.com/p/scalaris/) for coordinating work (paxos based > IIRC) and the disco project (http://discoproject.org/) has implemented a > MapReduce framework too, although jobs are written in python rather than > Erlang. From watson.timothy@REDACTED Sun Aug 22 20:29:30 2010 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 22 Aug 2010 19:29:30 +0100 Subject: [erlang-questions] On Starting A Remote Shell Programmatically In-Reply-To: References: Message-ID: There is also an ssh application/module that wraps up the underlying details (like open_port). In terms of keeping track of nodes, there are a couple of things to consider. Firstly, if you're comfortable using distributed Erlang to keep track of the membership of your nodes then you can use net_kernel:monitor_nodes/2 to receive messages when they come up/down. There are some idiosyncrasies to watch out for with monitor_nodes/2, so you might find yourself fiddling around with erlang:start_timer/3 in conjunction with this approach. IMO you'd also be better looking at the slave module if you want to simplify managing these nodes. If you can't (or don't want to) rely on distributed Erlang for whatever reason, then you'll need a custom protocol. This is where something like riak_core would serve you very well. There are also some very useful modules in the scalaris codebase, although you'll need to copy (and comprehend) gen_component to do so. Cheers On 22 Aug 2010, at 17:46, ??? (Yau-Hsien Huang) wrote: > Thank you. Now I can use SSH to execute a remote command as > "erl -rsh ssh -sname gandolf" , and eventually it sometimes show > only the starting message of Erlang Shell and sometimes the starting > message accompanied with the exit message. Then, How to write > an Erlang program to robustly keep an alive remote shell? > > Related information is as following: > > % Commander > test(Host, NodeName) -> > erlang:open_port({spawn,"ssh -f " ++ Host ++ " \"erl -rsh ssh -sname " > ++ NodeName ++ "\" &", > [{env,[{"PATH",os:getenv("PATH")}]}]), > receive > AnyThing -> io:format("Receive:" ~w~n.",[AnyThing]), AnyThing > after > 5000 -> {error,no_response} > end. > ----------------------------- > [xxxxxxx@REDACTED ~]$ erl -rsh ssh -sname me > 1> c(test). > {ok,test} > 2> test:test("HPC9", "gandolf"). > Receive: {#Port<0.126>,{data,[69,115,... ]}} > {#Port<0.126>,{data,"Eshell V5.6.1 (abort with ^G)\n"}} > 3> net_adm:ping(gandolf@REDACTED). > pang > > Or, > > 2> {_,{S,D}} = test:test("HPC11", "mary"), io:format("~w:~s~n",[S,D]). > Receive: {#Port<0.132>,{data,[69,115,... ]}} > data:(gandolf@REDACTED) 1> *** Terminating erlang (gandolf@REDACTED) > > ok > 3>halt(). > > > 2010/8/22 Tim Watson > >> >> There is an ssh application distributed with OTP which should meet your >> needs. It'll allow you to open an SSH connection and send commands over it, >> so assuming you have installed the emulator on the remote machine, you >> should be able to spin up another node. If you're planning on writing a >> MapReduce framework, may I suggest that you look at riak_core (the >> underpinning framework behind riak), and luke ( >> http://github.com/basho/luke) as these things have been done before. There >> is also some excellent stuff in scalaris ( >> http://code.google.com/p/scalaris/) for coordinating work (paxos based >> IIRC) and the disco project (http://discoproject.org/) has implemented a >> MapReduce framework too, although jobs are written in python rather than >> Erlang. From ok@REDACTED Mon Aug 23 00:41:11 2010 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 23 Aug 2010 10:41:11 +1200 Subject: [erlang-questions] Leex And Character Encodings In-Reply-To: References: Message-ID: <23333213-113C-4EA4-92C2-6A68F43541E4@cs.otago.ac.nz> On Aug 21, 2010, at 8:37 PM, Gordon Guthrie wrote: > The problem comes when I put spaces in the white space: > = 1 + 2 "=? 1 +? ? ? ? 2" = 1 + > 2 #ERROR! > > The expression round trips fine but (unlike the previous examples) the > server-side expression returns an error for the value because the expression > doesn't match any valid syntax. > > Tabs are expanded to white spaces so the only problem (I think) is with > multiple white spaces - which is why I think just adding a lexical token to > make ? the same as 2 spaces would work. It's not clear to me what precisely is mangling the spaces. What _is_ clear is that "? " is precisely what you see when the Latin-1 No-Break-Space is first converted to UTF-8 and then displayed by something expecting Latin-1. 1. How do no-break-space   characters turn up? 2. What is it that is rendering them as if they were encoded in Latin-1 rather than UTF-8? 3. In any case, if you are going to hack it, you should make the 16#C2,16#20 sequence equivalent to ONE space, not two. From vinoski@REDACTED Mon Aug 23 06:20:25 2010 From: vinoski@REDACTED (Steve Vinoski) Date: Mon, 23 Aug 2010 00:20:25 -0400 Subject: [erlang-questions] VM silently exits In-Reply-To: References: Message-ID: 2010/7/29 Bj?rn Gustavsson : > On Wed, Jul 28, 2010 at 10:29 PM, Steve Vinoski wrote: >> I've found a few places in the VM C source code where exit() is called >> without logging anything. Some of these are normal exits, like when >> you exit an Erlang shell, where no logging is needed. But others seem >> to be error conditions, and there should be logging for those. I think >> I'll probably have to patch my system to add logging to those cases to >> try to track down this problem -- is there still time to get a patch >> like this into R14B? > > Yes, there should still be time. The issue seems important enough > to fix in R14B. Just to close off this thread: 1. I went through the exit() calls again, more carefully this time, and it looks like the ones that need logging already have it. So, I was wrong, no patch needed. 2. I found that the VM was dying due to running out of memory, and that nothing was reported because heart is aggressive in restarting the VM (as it should be). It uses SIGKILL, so the VM often has no chance to log anything, emit a crash dump, etc. To track down my issue I temporarily modified heart to first issue a SIGUSR1, thereby initiating a crash dump, then waiting 10 seconds before issuing a SIGQUIT if needed to cause a core dump, and after a 1 second sleep finally issuing a SIGKILL if needed to kill off any non-responsive VMs. Normally you don't want heart waiting 10-15 seconds like this before restarting, since you just want the VM to go back into action as quickly as possible, but the temporary change was handy for debug and allowed me to successfully track down the source of the problem (which turned out to just be unchecked growth on an ets table, doh). --steve From pascalchapier@REDACTED Mon Aug 23 07:05:40 2010 From: pascalchapier@REDACTED (Pascal Chapier) Date: Mon, 23 Aug 2010 07:05:40 +0200 Subject: [erlang-questions] random lookup in ets Message-ID: Hello, I guess that your ets key is not an integer list :o). I imagine that you can do 2 things: first, if your application manage the ets, you can add an new table as index, with an integer list as key, and the main ets key as value. then you will have to select a random integer between min and max index key, retreive the main ets key and then the value. The problem with this is that you will have to keep the index keys continous, it is not so easy and may takes some time when you make a delete operation. On the other hand it should be fast for random read. second, you can use ets:tab2list(Ets), and ets:info(Ets,size), generate X with random:uniform(Size): a random value between 1 and Size, and finally take the Xth element of the list with lists:nth(X).. hope this will help you. Best regards From gordon@REDACTED Mon Aug 23 08:23:24 2010 From: gordon@REDACTED (Gordon Guthrie) Date: Mon, 23 Aug 2010 07:23:24 +0100 Subject: [erlang-questions] Leex And Character Encodings In-Reply-To: <23333213-113C-4EA4-92C2-6A68F43541E4@cs.otago.ac.nz> References: <23333213-113C-4EA4-92C2-6A68F43541E4@cs.otago.ac.nz> Message-ID: Richard This all makes more sense now... 1. How do no-break-space   characters turn up? The input is coming from a browser and there will be a library in there that is saying "hey many spaces, you will need to   them to keep them 2. What is it that is rendering them as if they were encoded in Latin-1 rather than UTF-8? Erlang io:format - we just store utf-8 in the database... 3. In any case, if you are going to hack it, you should make the 16#C2,16#20 sequence equivalent to ONE space, not two. I just need to recognise 16#C2, 16#20 as white space - it has no significance in the lexer... Cheers Gordon On 22 August 2010 23:41, Richard O'Keefe wrote: > > On Aug 21, 2010, at 8:37 PM, Gordon Guthrie wrote: > > The problem comes when I put spaces in the white space: > > = 1 + 2 "=? 1 +? ? ? ? 2" = 1 + > > 2 #ERROR! > > > > The expression round trips fine but (unlike the previous examples) the > > server-side expression returns an error for the value because the > expression > > doesn't match any valid syntax. > > > > Tabs are expanded to white spaces so the only problem (I think) is with > > multiple white spaces - which is why I think just adding a lexical token > to > > make ? the same as 2 spaces would work. > > It's not clear to me what precisely is mangling the spaces. > What _is_ clear is that "? " is precisely what you see when > the Latin-1 No-Break-Space is first converted to UTF-8 and > then displayed by something expecting Latin-1. > > 1. How do no-break-space   characters turn up? > 2. What is it that is rendering them as if they were encoded > in Latin-1 rather than UTF-8? > 3. In any case, if you are going to hack it, you should make > the 16#C2,16#20 sequence equivalent to ONE space, not two. > > -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From ulf.wiger@REDACTED Mon Aug 23 08:49:55 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 23 Aug 2010 08:49:55 +0200 Subject: [erlang-questions] VM silently exits In-Reply-To: References: Message-ID: <4C721A13.4080004@erlang-solutions.com> Steve Vinoski wrote: > > To track down my issue I temporarily modified heart to first issue a > SIGUSR1, thereby initiating a crash dump, then waiting 10 seconds > before issuing a SIGQUIT if needed to cause a core dump, and after a 1 > second sleep finally issuing a SIGKILL if needed to kill off any > non-responsive VMs. Normally you don't want heart waiting 10-15 > seconds like this before restarting, since you just want the VM to go > back into action as quickly as possible, [...] Steve, I think that depends. If you have a redundant setup, you would probably fail over to the standby(s) rather than waiting for the node to restart, in which case you most likely also prefer to wait for it to log some useful diagnostics first. Perhaps the appropriate thing is to be able to instruct heart whether it should be nice or brutal when restarting the VM? BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From andrew@REDACTED Mon Aug 23 09:21:15 2010 From: andrew@REDACTED (Andrew Thompson) Date: Mon, 23 Aug 2010 03:21:15 -0400 Subject: [erlang-questions] VM silently exits In-Reply-To: <4C721A13.4080004@erlang-solutions.com> References: <4C721A13.4080004@erlang-solutions.com> Message-ID: <20100823072115.GA14928@hijacked.us> On Mon, Aug 23, 2010 at 08:49:55AM +0200, Ulf Wiger wrote: > Steve, I think that depends. If you have a redundant setup, you > would probably fail over to the standby(s) rather than waiting > for the node to restart, in which case you most likely also prefer > to wait for it to log some useful diagnostics first. > > Perhaps the appropriate thing is to be able to instruct heart > whether it should be nice or brutal when restarting the VM? > It'd also be helpful for other people troubleshooting this problem. I agree that this functionality might be generally useful. Andrew From gordon@REDACTED Mon Aug 23 10:13:11 2010 From: gordon@REDACTED (Gordon Guthrie) Date: Mon, 23 Aug 2010 09:13:11 +0100 Subject: [erlang-questions] Leex And Character Encodings In-Reply-To: <23333213-113C-4EA4-92C2-6A68F43541E4@cs.otago.ac.nz> References: <23333213-113C-4EA4-92C2-6A68F43541E4@cs.otago.ac.nz> Message-ID: Richard > 3. In any case, if you are going to hack it, you should make the 16#C2,16#20 sequence equivalent to ONE space, not two. I am getting 16#C2, 16#A0 and not 16#C2, 16#20 which I think is right for non-breaking spaces... Gordon On 22 August 2010 23:41, Richard O'Keefe wrote: > > On Aug 21, 2010, at 8:37 PM, Gordon Guthrie wrote: > > The problem comes when I put spaces in the white space: > > = 1 + 2 "=? 1 +? ? ? ? 2" = 1 + > > 2 #ERROR! > > > > The expression round trips fine but (unlike the previous examples) the > > server-side expression returns an error for the value because the > expression > > doesn't match any valid syntax. > > > > Tabs are expanded to white spaces so the only problem (I think) is with > > multiple white spaces - which is why I think just adding a lexical token > to > > make ? the same as 2 spaces would work. > > It's not clear to me what precisely is mangling the spaces. > What _is_ clear is that "? " is precisely what you see when > the Latin-1 No-Break-Space is first converted to UTF-8 and > then displayed by something expecting Latin-1. > > 1. How do no-break-space   characters turn up? > 2. What is it that is rendering them as if they were encoded > in Latin-1 rather than UTF-8? > 3. In any case, if you are going to hack it, you should make > the 16#C2,16#20 sequence equivalent to ONE space, not two. > > -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From icfp.publicity@REDACTED Mon Aug 23 10:30:55 2010 From: icfp.publicity@REDACTED (Wouter Swierstra) Date: Mon, 23 Aug 2010 10:30:55 +0200 Subject: ICFP '10: Final call for participation Message-ID: ===================================================================== Final Call for Participation The 15th ACM SIGPLAN International Conference on Functional Programming (ICFP 2010) http://www.icfpconference.org/icfp2010/ Baltimore, Maryland September 25 ? October 2 ===================================================================== ICFP 2010 provides a forum for researchers and developers to hear about the latest work on the design, implementations, principles, and uses of functional programming. The conference covers the entire spectrum of work, from practice to theory, including its peripheries. ** Not that the early registration deadline and discount hotel rates expire next week. ** * Program: http://www.icfpconference.org/icfp2010/program.html * Invited speakers: - Mike Gordon ML: Metalanguage or Object Language? - Matthias Felleisen TeachScheme!: A Checkpoint - Guy Blelloch Functional Parallel Algorithms Schedule including related events: * September 25: Workshop on Mechanizing Metatheory (WMM) Workshop on Mathematically Structured Functional Programming (MSFP) Workshop on High-Level Parallel Programming and Applications (HLPP) * September 26: Workshop on ML Workshop on Generic Programming (WGP) * September 27-29: ICFP 2010 * September 30: Haskell Symposium Erlang Workshop * October 1: Commercial Users of Functional Programming ? Day 1 (CUFP Tutorials) Haskell Implementors' Workshop * October 2: Commercial Users of Functional Programming ? Day 2 (CUFP Talks) This year there will also be a special series of Birds-of-a-Feather sessions associated with CUFP. More information can be found at: http://cufp.org/bofs-2010 Registration information: * Registration link: https://regmaster3.com/2010conf/ICFP10/register.php Local arrangements (including travel and accommodation): * http://www.icfpconference.org/icfp2010/local.html * Conference reservation/rate deadline: September 1st Conference organizers: * General Chair: Paul Hudak, Yale University * Program Chair: Stephanie Weirich, University of Pennsylvania * Local Arrangements Chair: Michael Hicks, University of Maryland * Workshop Co-Chairs: Derek Dreyer, Max Planck Institute for Software Systems Christopher Stone, Harvey Mudd College * Programming Contest Chair: Johannes Waldmann, Hochschule f?r Technik, Wirtschaft und Kultur, Leipzig * Video Chair: Scott Smith, Johns Hopkins University * Publicity Chair: Wouter Swierstra, Vector Fabrics ===================================================================== From emile@REDACTED Mon Aug 23 10:37:55 2010 From: emile@REDACTED (Emile Joubert) Date: Mon, 23 Aug 2010 09:37:55 +0100 Subject: [erlang-questions] enforcing ssl trust chain In-Reply-To: References: <4C62D6FD.4010208@rabbitmq.com> Message-ID: <4C723363.60609@rabbitmq.com> On 16/08/10 13:30, Jesper Pettersson wrote: >> My personal preference for default value was not to accept any >> path-validation errors as default, but the motivation was that it >> should be as easy as possible to get an ssl connection up and >> running. I am just back from vacation and I do not remember >> all the details of the discussion. We are of course interested in all >> user feedback we can get. >> So if you have any arguments for or against please let us know. > > In my opinion the default behavior should be very strict with regards to > certificate validation. I agree with Jesper - the default should be strict and a relaxed mode should be available as a configuration option. Thanks for confirming that overriding the verify_fun is currently the best way of achieving a configuration more suitable for a production environment. Thanks Emile From hynek@REDACTED Mon Aug 23 11:31:11 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Mon, 23 Aug 2010 11:31:11 +0200 Subject: [erlang-questions] enforcing ssl trust chain In-Reply-To: <4C723363.60609@rabbitmq.com> References: <4C62D6FD.4010208@rabbitmq.com> <4C723363.60609@rabbitmq.com> Message-ID: http://www.eff.org/observatory It's not fully on topic but big overlap. On Mon, Aug 23, 2010 at 10:37 AM, Emile Joubert wrote: > On 16/08/10 13:30, Jesper Pettersson wrote: >>> My personal preference for default value was not to accept any >>> path-validation errors as default, but the motivation was that it >>> should be as easy as possible to get an ssl connection up and >>> running. I am just back from vacation and I do not remember >>> all the details of the discussion. ?We are of course interested in all >>> user feedback we can get. >>> So if you have any arguments for or against please let us know. >> >> In my opinion the default behavior should be very strict with regards to >> certificate validation. > > I agree with Jesper - the default should be strict and a relaxed mode > should be available as a configuration option. > > Thanks for confirming that overriding the verify_fun is currently the > best way of achieving a configuration more suitable for a production > environment. > > Thanks > > Emile > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From erlang@REDACTED Mon Aug 23 11:32:04 2010 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 23 Aug 2010 11:32:04 +0200 Subject: [erlang-questions] Let's start a project for a (MMO-)Game in Erlang... (inspired by "Erlang: Who uses it for games?") In-Reply-To: References: Message-ID: I like the idea of an MMO-game infrastructure in Erlang - I think to do this we need to make a highly modular architecture with pluggable components. If these talk through pure socket based message passing it should be relatively easy to scale the thing. This seems to be a question of packaging and identifying the components. The key components for a game seem to be graphics/sound/windows part of the problem this is an area where Erlang is traditionally weak. The massive/communication/database/security part is an area where Erlang is traditionally strong. Graphics in games seems to be exclusively written in C++ which make Erlang programmers recoil in horror. But C++ graphics components can be controlled from scripting languages, lua has been pretty successful here. I was looking at http://love2d.org/ it looks pretty neat. It's a combination of lua + SDL + OpenGL at a first glance controlling this through lua sockets from an Erlang node seems possible. Another possibility would be to rip out the SDL/OpenGL part of wings (http://www.wings3d.com/) and make something like love with erlang replacing lua. I think to get anywhere we really need to package things so they run cross-platform "out of the box". The love2d packing is excellent. Basically a file X.love ( a renamed zip archive) must contain a top-level file main.lua. That's it. The command "love X" runs the application on all platforms I've so far tested. I think to attract sufficient interest we need to package Erlang games applications in a similar manner. The only application I've seen that does this is wings. Wings has two beautiful sub-components: - the SDL/OpenGL stuff - the way the wings application is built into a stand-alone application Unfortunately these are hard-wired into wings in such a way that that graphics and bundling cannot be used outside the wings framework. It would be really nice to refactor this code into three components, ie the SDL/OpenGL part, the bundle part and the application itself. Adding communication to wings (or to love) would enable all kind of fun things ... If there is sufficient interest this might be the way to go. Alternatively scrap everything and start with SDL and a minimal OpenGL interface and do as much as possible in erlang. /Joe On Fri, Jul 9, 2010 at 7:55 PM, Boris M?hmer wrote: > It looks like some people are interested in Erlang game programming. > On the other hand getting started in Erlang and OTP isn't too easy. > Especially for people with a strong C/C++/C#/Java/etc background. > > Inspired by the "Erlang: Who uses it for games?" thread I thought about > starting a community project to implement a MMO game in Erlang/OTP. > The goal would be to collect some "best-practices" for different tasks. > > Actually I am more interested in the server side than how to implement > a top-notch-state-of-the-art client. Also I think Erlang is more suited > for the server side. But I am also interested in how to interface the > "server-side erlang" using a (C/C++, Java, Python) cross-plattform > client (using wxWidget or Qt as a base and OpenGL for graphics). > > What I would like to see are reponses from other people who may also > be interested in such a project. Not only beginners with Erlang/OTP, > but also (or especially) experienced people to guide and support the > others. And most of all, to have some fun practising Erlang. > > Well, what do You think about it? > > Best regards, > Boris > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From rtrlists@REDACTED Mon Aug 23 12:14:57 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 23 Aug 2010 11:14:57 +0100 Subject: [erlang-questions] Let's start a project for a (MMO-)Game in Erlang... (inspired by "Erlang: Who uses it for games?") In-Reply-To: References: Message-ID: On Mon, Aug 23, 2010 at 10:32 AM, Joe Armstrong wrote: > I like the idea of an MMO-game infrastructure in Erlang - I think to > do this we need to make > a highly modular architecture with pluggable components. If these talk > through pure socket > based message passing it should be relatively easy to scale the thing. > > This seems to be a question of packaging and identifying the components. > > The key components for a game seem to be graphics/sound/windows part > of the problem > this is an area where Erlang is traditionally weak. The > massive/communication/database/security part is an area where Erlang > is traditionally strong. > > On the communication and synchronisation front, it may be useful to have a look and the Croquet Tea Time concept of keeping a distributed world in synch ( http://www.opencroquet.org/index.php/TeaTime_Architecture , and for a bigger architectural view http://www.opencroquet.org/index.php/System_Overview ). There's a fabulous short summary of the concept here: http://inventing-the-future.wetmachine.com/content/teatime-in-a-nutshell-by-my-daughter This looks like an easy framework to adopt in an Erlang setting. Robby From watson.timothy@REDACTED Mon Aug 23 13:28:26 2010 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 23 Aug 2010 12:28:26 +0100 Subject: [erlang-questions] Let's start a project for a (MMO-)Game in Erlang... (inspired by "Erlang: Who uses it for games?") In-Reply-To: References: Message-ID: <004E5DE8-B408-4723-8D16-E4FB68D761A7@gmail.com> On 23 Aug 2010, at 10:32, Joe Armstrong wrote: > Graphics in games seems to be exclusively written in C++ which make > Erlang programmers recoil > in horror. But C++ graphics components can be controlled from > scripting languages, lua has been pretty successful here. There are some existing integrations between Lua and Erlang available as a linked in driver (http://github.com/raycmorgan/erl-lua) or port program (erlua - one "L" - don't have the link to hand). There appear to be a few others popping up on github as well. From vinoski@REDACTED Mon Aug 23 15:59:58 2010 From: vinoski@REDACTED (Steve Vinoski) Date: Mon, 23 Aug 2010 09:59:58 -0400 Subject: [erlang-questions] VM silently exits In-Reply-To: <4C721A13.4080004@erlang-solutions.com> References: <4C721A13.4080004@erlang-solutions.com> Message-ID: On Mon, Aug 23, 2010 at 2:49 AM, Ulf Wiger wrote: > Steve Vinoski wrote: >> >> To track down my issue I temporarily modified heart to first issue a >> SIGUSR1, thereby initiating a crash dump, then waiting 10 seconds >> before issuing a SIGQUIT if needed to cause a core dump, and after a 1 >> second sleep finally issuing a SIGKILL if needed to kill off any >> non-responsive VMs. Normally you don't want heart waiting 10-15 >> seconds like this before restarting, since you just want the VM to go >> back into action as quickly as possible, [...] > > Steve, I think that depends. If you have a redundant setup, you > would probably fail over to the standby(s) rather than waiting > for the node to restart, in which case you most likely also prefer > to wait for it to log some useful diagnostics first. > > Perhaps the appropriate thing is to be able to instruct heart > whether it should be nice or brutal when restarting the VM? Hi Ulf, that's an excellent point. Perhaps I can devise a patch providing this capability. --steve From per@REDACTED Mon Aug 23 17:52:51 2010 From: per@REDACTED (Per Hedeland) Date: Mon, 23 Aug 2010 17:52:51 +0200 (CEST) Subject: Auto-recognizing IPv6-tuple Message-ID: <201008231552.o7NFqpkB073201@pluto.hedeland.org> Hi, Way back in the thread starting with http://www.erlang.org/cgi-bin/ezmlm-cgi/4/42722, I complained about "bad behaviour" when some gen_tcp and gen_udp functions were passed an IPv6 8-tuple without the 'inet6' (or 'inet') option, suggesting (with patches:-) that it would be more reasonable if IPv6 and IPv4 worked the same - i.e. an 8-tuple would imply 'inet6'. Raimo made me happy with the reply (in part): > Not really. But as you see for gen_tcp:connect below, the inet6 > option is currently required there even for an IPv6 address tuple. > These are corners we have not smoothed yet. However the "bad behaviour" was fixed in R13B, not by auto-recognizing the 8-tuple but by exit(badarg), and this remains the case in R14A. Should I take this to mean that the auto-recognition won't happen? FWIW, the second version of the patches in that thread has been working fine in production for over a year - initially in R10B-10, later in R13B03 - and seems to apply cleanly and work fine with R14A too. --Per Hedeland From comptekki@REDACTED Mon Aug 23 18:03:11 2010 From: comptekki@REDACTED (Wes James) Date: Mon, 23 Aug 2010 10:03:11 -0600 Subject: [erlang-questions] application:start crash, bug? In-Reply-To: <42D87025-59D9-482D-BC7C-326B19B9FC27@yahoo.ca> References: <42D87025-59D9-482D-BC7C-326B19B9FC27@yahoo.ca> Message-ID: I see you still have had no response on this. Maybe the start is happening to fast for the stop to complete? Does the application:stop have its own thread to complete or does it complete before the next line, in this case, application:start get executed? How does erlang handle each line of code when they are executed? -wes On Sun, Aug 15, 2010 at 10:29 PM, Alisdair Sullivan wrote: > the sequence: > > application:start({application, crypto, [{mod, {crypto_app, []}}]}), > application:stop(crypto), > application:start({application, crypto, [{mod, {crypto_app, []}}]}). > > results in the following error: > > {"Kernel pid terminated",application_controller,{{badmatch,false},[{application_controller,handle_application_started,3},{gen_server,handle_msg,5}]}} > > and a hard crash. It doesn't matter which application I attempt to start, I encounter this error with all of them. > > I realize using application:start in this manner is undocumented, but from looking over application_controller.erl, I can't locate the source of the error. Any insight? > > Alisdair > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From emile@REDACTED Mon Aug 23 19:04:10 2010 From: emile@REDACTED (Emile Joubert) Date: Mon, 23 Aug 2010 18:04:10 +0100 Subject: [erlang-questions] enforcing ssl trust chain In-Reply-To: References: <4C62D6FD.4010208@rabbitmq.com> Message-ID: <4C72AA0A.6030500@rabbitmq.com> On 16/08/10 13:18, Ingela Andin wrote: > Hi! > 2010/8/11 Emile Joubert : [...] >> In a production environment I want to prevent clients without >> certificates signed by a known CA from connecting. Is there any way of >> getting this behaviour by using configuration files? The only way I can >> find is to set verify_fun to an appropriate function, but this is >> unappealing because I want to change my mind without needing to recompile. > > At the moment defining a verify fun would be your option to accomplish this. > We might add some other configuration option if we find that it seems to be > a good thing from a general point of view. I've tried that, but verify_fun gets called regardless of whether verify is set to verify_none or verify_peer. My reading of the documentation is that certificate path validation errors should be ignored if verify_none is set, regardless of verify_fun. Can you please confirm? Thanks Emile From per@REDACTED Mon Aug 23 19:11:42 2010 From: per@REDACTED (Per Hedeland) Date: Mon, 23 Aug 2010 19:11:42 +0200 (CEST) Subject: IPV6_V6ONLY Message-ID: <201008231711.o7NHBgvY074811@pluto.hedeland.org> Hi, On Linux (and maybe some others), by default, when binding a socket to the IPv6 wildcard address, the socket accepts both IPv6 and IPv4, the latter in the form of IPv4-mapped IPv6. Trying to bind another socket with the same port and the IPv4 wildcard results in EADDRINUSE. On *BSD, by default (and always on Windows I believe), the situation is the opposite - the IPv6-wildcard socket accepts only IPv6, and you need to bind another socket for IPv4 if you want to support both. I believe there is some consensus in the networking community that the first behaviour is "bad" for various reasons. Be that as it may, it is unlikely to change, and it seems applications tend to ensure the second behavior by means of unconditionally using the IPV6_V6ONLY socket option in order to get consistent behavior across OSes (and avoid the "badness"). Since one strong point of Erlang/OTP is to as far as possible isolate the Erlang programmer from these annoying differences between OSes, could it be considered to make inet_drv always apply IPV6_V6ONLY (if it exists) for IPv6 sockets? --Per Hedeland From tony@REDACTED Mon Aug 23 20:02:00 2010 From: tony@REDACTED (Tony Rogvall) Date: Mon, 23 Aug 2010 20:02:00 +0200 Subject: [erlang-questions] IPV6_V6ONLY In-Reply-To: <201008231711.o7NHBgvY074811@pluto.hedeland.org> References: <201008231711.o7NHBgvY074811@pluto.hedeland.org> Message-ID: Hi! I listened to what apple had to say about this and they tend to prefer the first option, and let the kernel handle the IPv4 mapping. The reason is (future) backwards compatibility. In other words they suggest that every one should start using IPv6 only now. I am not sure about this, but it sound like a good thing for end point application hacks. If you are somewhere in the middle you may want to distinguish between the to cases, and take different action depending on situation. /Tony On 23 aug 2010, at 19.11, Per Hedeland wrote: > Hi, > > On Linux (and maybe some others), by default, when binding a socket to > the IPv6 wildcard address, the socket accepts both IPv6 and IPv4, the > latter in the form of IPv4-mapped IPv6. Trying to bind another socket > with the same port and the IPv4 wildcard results in EADDRINUSE. > > On *BSD, by default (and always on Windows I believe), the situation is > the opposite - the IPv6-wildcard socket accepts only IPv6, and you need > to bind another socket for IPv4 if you want to support both. > > I believe there is some consensus in the networking community that the > first behaviour is "bad" for various reasons. Be that as it may, it is > unlikely to change, and it seems applications tend to ensure the second > behavior by means of unconditionally using the IPV6_V6ONLY socket option > in order to get consistent behavior across OSes (and avoid the "badness"). > > Since one strong point of Erlang/OTP is to as far as possible isolate > the Erlang programmer from these annoying differences between OSes, > could it be considered to make inet_drv always apply IPV6_V6ONLY (if it > exists) for IPv6 sockets? > > --Per Hedeland > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From Antonio.Musumeci@REDACTED Mon Aug 23 19:37:45 2010 From: Antonio.Musumeci@REDACTED (Musumeci, Antonio S) Date: Mon, 23 Aug 2010 13:37:45 -0400 Subject: io_lib and unicode post processing? Message-ID: <01489E237C2C8F45AF7898E135F48E8019A3B1CD47@NYWEXMBX2129.msad.ms.com> I've got a webmachine/mochiweb setup and it seems much slower/higher CPU than I expected. I'm creating JSON for 10k records using jsonerl and sending on to webmachine. After the serializing of the data the CPU is still pegged and I see in etop io_lib and unicode being called. Does anyone know why those would be called? It takes several seconds. From watson.timothy@REDACTED Mon Aug 23 20:30:46 2010 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 23 Aug 2010 19:30:46 +0100 Subject: [erlang-questions] io_lib and unicode post processing? In-Reply-To: <01489E237C2C8F45AF7898E135F48E8019A3B1CD47@NYWEXMBX2129.msad.ms.com> References: <01489E237C2C8F45AF7898E135F48E8019A3B1CD47@NYWEXMBX2129.msad.ms.com> Message-ID: <69AB868D-0BFD-4DD9-A565-2D6D705D88CB@gmail.com> On 23 Aug 2010, at 18:37, Musumeci, Antonio S wrote: > I've got a webmachine/mochiweb setup and it seems much slower/higher CPU than I expected. I'm creating JSON for 10k records using jsonerl and sending on to webmachine. After the serializing of the data the CPU is still pegged and I see in etop io_lib and unicode being called. Does anyone know why those would be called? It takes several seconds. Surely they're getting called a lot during serialisation. Haven't used jsonerl before but what kind of output is getting produced - strings, iolists, binaries? Does it not offer a streaming API so you can forward the data as multipart/chunked? There are some useful profiling tools available with OTP and there's a good library that'll allow you use them on production systems without incurring dangerous overhead - eper. You'll find it on google code IIRC. I've been using a newer json library called jsx recently (http://github.com/talentdeficit/jsx) and find it performs really well, although I haven't load tested it so don't know if it's production worthy. From francesco@REDACTED Mon Aug 23 20:42:12 2010 From: francesco@REDACTED (Francesco Cesarini) Date: Mon, 23 Aug 2010 19:42:12 +0100 Subject: Erlang Factory Lite in LA Nov. 13th: Looking for Conference room Message-ID: <4C72C104.2050201@erlang-solutions.com> Hi Everyone, Together with Matt Sacks from thebitsource.com we are in the early planning stages for an Erlang Factory Lite in Los Angeles. We hope to turn this in to a reoccurring half day event which will be free of charge (or very cheap) for attendees. We are looking to attract not only members of the Erlang community, but also other communities interested in learning more about Erlang and networking with Erlangers. To keep it free, we hope someone can help with a venue that accommodates between 50 - 100 people. If your company has a meeting room we can use free of charge or you have a friend who has a spare conference room in a bar, let us know. We have options, but if possible, would prefer not to charge, no matter how small the sum. We are also looking for speakers. Quite a few speakers in LA have already expressed interest, but I am sure there are many more out there. If interested, please contact Jodie. Once we have the premises, we will be launching a page on the Erlang Factory website. Date wise, we have set the tentative date to Saturday November the 13th. Our choice of date coincides with Ulf Wiger being in town. We are aware it is close to the Erlang User Conference, but we do not expect the attendees to overlap; the events will complement more than compete with each other. For those hard core delegates who want to attend both events, catching a flight on Sunday will get you to Stockholm on time, including a few hours to recover from Jet Lag before the Stockholm walk and pub crawl Monday afternoon. Looking forward to your feedback, but most importantly, help with a venue. Francesco -- Erlang Solutions Ltd. http://www.erlang-solutions.com From Antonio.Musumeci@REDACTED Mon Aug 23 20:46:52 2010 From: Antonio.Musumeci@REDACTED (Musumeci, Antonio S) Date: Mon, 23 Aug 2010 14:46:52 -0400 Subject: [erlang-questions] io_lib and unicode post processing? In-Reply-To: <69AB868D-0BFD-4DD9-A565-2D6D705D88CB@gmail.com> References: <01489E237C2C8F45AF7898E135F48E8019A3B1CD47@NYWEXMBX2129.msad.ms.com> <69AB868D-0BFD-4DD9-A565-2D6D705D88CB@gmail.com> Message-ID: <01489E237C2C8F45AF7898E135F48E8019A3B1CD5F@NYWEXMBX2129.msad.ms.com> It's not the generation of the JSON. I've tested that and it makes up only a small portion of the overhead. It's most of it is after I hand it off to webmachine. Even if I create a large binary and pass it along it does the same. -----Original Message----- From: Tim Watson [mailto:watson.timothy@REDACTED] Sent: Monday, August 23, 2010 2:31 PM To: Musumeci, Antonio S (Enterprise Infrastructure) Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] io_lib and unicode post processing? On 23 Aug 2010, at 18:37, Musumeci, Antonio S wrote: > I've got a webmachine/mochiweb setup and it seems much slower/higher CPU than I expected. I'm creating JSON for 10k records using jsonerl and sending on to webmachine. After the serializing of the data the CPU is still pegged and I see in etop io_lib and unicode being called. Does anyone know why those would be called? It takes several seconds. Surely they're getting called a lot during serialisation. Haven't used jsonerl before but what kind of output is getting produced - strings, iolists, binaries? Does it not offer a streaming API so you can forward the data as multipart/chunked? There are some useful profiling tools available with OTP and there's a good library that'll allow you use them on production systems without incurring dangerous overhead - eper. You'll find it on google code IIRC. I've been using a newer json library called jsx recently (http://github.com/talentdeficit/jsx) and find it performs really well, although I haven't load tested it so don't know if it's production worthy. From watson.timothy@REDACTED Mon Aug 23 20:52:50 2010 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 23 Aug 2010 19:52:50 +0100 Subject: [erlang-questions] io_lib and unicode post processing? In-Reply-To: <01489E237C2C8F45AF7898E135F48E8019A3B1CD5F@NYWEXMBX2129.msad.ms.com> References: <01489E237C2C8F45AF7898E135F48E8019A3B1CD47@NYWEXMBX2129.msad.ms.com> <69AB868D-0BFD-4DD9-A565-2D6D705D88CB@gmail.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CD5F@NYWEXMBX2129.msad.ms.com> Message-ID: How do you "pass it on" to webmachine - REST call over HTTP, rpc (distributed Erlang), raw socket call? On 23 Aug 2010, at 19:46, Musumeci, Antonio S wrote: > It's not the generation of the JSON. I've tested that and it makes up only a small portion of the overhead. It's most of it is after I hand it off to webmachine. Even if I create a large binary and pass it along it does the same. > > -----Original Message----- > From: Tim Watson [mailto:watson.timothy@REDACTED] > Sent: Monday, August 23, 2010 2:31 PM > To: Musumeci, Antonio S (Enterprise Infrastructure) > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] io_lib and unicode post processing? > > On 23 Aug 2010, at 18:37, Musumeci, Antonio S wrote: > >> I've got a webmachine/mochiweb setup and it seems much slower/higher CPU than I expected. I'm creating JSON for 10k records using jsonerl and sending on to webmachine. After the serializing of the data the CPU is still pegged and I see in etop io_lib and unicode being called. Does anyone know why those would be called? It takes several seconds. > > Surely they're getting called a lot during serialisation. Haven't used jsonerl before but what kind of output is getting produced - strings, iolists, binaries? Does it not offer a streaming API so you can forward the data as multipart/chunked? There are some useful profiling tools available with OTP and there's a good library that'll allow you use them on production systems without incurring dangerous overhead - eper. You'll find it on google code IIRC. I've been using a newer json library called jsx recently (http://github.com/talentdeficit/jsx) and find it performs really well, although I haven't load tested it so don't know if it's production worthy. > From Antonio.Musumeci@REDACTED Mon Aug 23 21:02:09 2010 From: Antonio.Musumeci@REDACTED (Musumeci, Antonio S) Date: Mon, 23 Aug 2010 15:02:09 -0400 Subject: [erlang-questions] io_lib and unicode post processing? In-Reply-To: References: <01489E237C2C8F45AF7898E135F48E8019A3B1CD47@NYWEXMBX2129.msad.ms.com> <69AB868D-0BFD-4DD9-A565-2D6D705D88CB@gmail.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CD5F@NYWEXMBX2129.msad.ms.com> Message-ID: <01489E237C2C8F45AF7898E135F48E8019A3B1CD66@NYWEXMBX2129.msad.ms.com> The API for webmachine expects a callback for a particular content type. What I mean is that my function is called and I return a list or binary of the data and then it's no longer in my control. Whatever is happening is behind the scenes. -----Original Message----- From: Tim Watson [mailto:watson.timothy@REDACTED] Sent: Monday, August 23, 2010 2:53 PM To: Musumeci, Antonio S (Enterprise Infrastructure) Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] io_lib and unicode post processing? How do you "pass it on" to webmachine - REST call over HTTP, rpc (distributed Erlang), raw socket call? On 23 Aug 2010, at 19:46, Musumeci, Antonio S wrote: > It's not the generation of the JSON. I've tested that and it makes up only a small portion of the overhead. It's most of it is after I hand it off to webmachine. Even if I create a large binary and pass it along it does the same. > > -----Original Message----- > From: Tim Watson [mailto:watson.timothy@REDACTED] > Sent: Monday, August 23, 2010 2:31 PM > To: Musumeci, Antonio S (Enterprise Infrastructure) > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] io_lib and unicode post processing? > > On 23 Aug 2010, at 18:37, Musumeci, Antonio S wrote: > >> I've got a webmachine/mochiweb setup and it seems much slower/higher CPU than I expected. I'm creating JSON for 10k records using jsonerl and sending on to webmachine. After the serializing of the data the CPU is still pegged and I see in etop io_lib and unicode being called. Does anyone know why those would be called? It takes several seconds. > > Surely they're getting called a lot during serialisation. Haven't used jsonerl before but what kind of output is getting produced - strings, iolists, binaries? Does it not offer a streaming API so you can forward the data as multipart/chunked? There are some useful profiling tools available with OTP and there's a good library that'll allow you use them on production systems without incurring dangerous overhead - eper. You'll find it on google code IIRC. I've been using a newer json library called jsx recently (http://github.com/talentdeficit/jsx) and find it performs really well, although I haven't load tested it so don't know if it's production worthy. From anthonym@REDACTED Mon Aug 23 22:30:51 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Mon, 23 Aug 2010 13:30:51 -0700 Subject: [erlang-questions] random lookup in ets In-Reply-To: References: Message-ID: <20100823203051.GA8406@alumni.caltech.edu> On Mon, Aug 23, 2010 at 07:05:40AM +0200, Pascal Chapier wrote: > first, if your application manage the ets, you can add an new table as > index, with an integer list as key, and the main ets key as value. then > you will have to select a random integer between min and max index key, > retreive the main ets key and then the value. The problem with this is > that you will have to keep the index keys continous, it is not so easy > and may takes some time when you make a delete operation. On the other > hand it should be fast for random read. If you are keeping a secondary index you could do what Paul Mineiro suggested I do with a similar problem. In my problem I have non-overlapping ranges of 32-bit integers. For each range I have data associated with the range. Given an integer I wanted to look up the value associated with the range. I use an ordered_set ets table which contains tuples of the form { StartOfRange, SecondaryIndexKey } Then given an integer I do { StartOfRange, SecondaryIndexKey } = case ets:lookup (Index, IntIp) of [] -> case ets:lookup (Index, ets:prev (Index, IntIp)) of [] -> {0, -1}; V -> hd (V) end; V -> hd (V) end, To get back the SecondaryIndexKey (ets:prev will return the previous entry, so using it when I don't exact match will give me the entry responsible for the range). This is extremely fast (I haven't tested it but something like [ lookup (random:uniform (trunc (math:pow (2,32)))) || V <- lists:seq (1, 10000) ] where lookup/1 actually looks up the SecondaryIndexKey, gets the ets value out of the secondary table (which is a large binary object, my SecondaryIndexKey are actually offsets into a file), turns the returned binary into a record where integers are turned into string via tuple lookup tables, and returned. So doing lookup 10000 times with random input takes about ~950 milliseconds on my laptop, so very, very fast :) You could do the same thing to avoid keeping the keys continuous, as you just want a random entry, so the previous entry is as good at the random entry you went looking for. Deletes are still hard because you have to find the SecondaryIndexKey of the entry you are deleting from your primary table, but maybe not too bad if you keep both a forward {Integer, Key} and a backward {Key, Integer}. Then when you delete the Key in your real table you get the Key from the backward table, and then remove it from that table, then remove from the forward table. > second, you can use ets:tab2list(Ets), and ets:info(Ets,size), generate > X with random:uniform(Size): a random value between 1 and Size, and > finally take the Xth element of the list with lists:nth(X).. This seems really slow with any sizable data, I wouldn't even attempt it with the table I do the above on (that table has 10 million entries). Hope that helps, -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From ge@REDACTED Tue Aug 24 01:15:37 2010 From: ge@REDACTED (=?ISO-8859-15?Q?G=E9_Weijers?=) Date: Mon, 23 Aug 2010 16:15:37 -0700 (PDT) Subject: [erlang-questions] IPV6_V6ONLY In-Reply-To: <201008231711.o7NHBgvY074811@pluto.hedeland.org> References: <201008231711.o7NHBgvY074811@pluto.hedeland.org> Message-ID: On Mon, 23 Aug 2010, Per Hedeland wrote: > Since one strong point of Erlang/OTP is to as far as possible isolate > the Erlang programmer from these annoying differences between OSes, > could it be considered to make inet_drv always apply IPV6_V6ONLY (if it > exists) for IPv6 sockets? One way to implement a protocol-agnostic client and server in C on a Posix system is to use the getaddrinfo() standard library routine, which produces a list of parameters that can be fed into socket() + bind() for servers, and socket() + connect() for clients. On the server size you may end up with multiple sockets to listen to. getaddrinfo() and getnameinfo() or simular routines seem to be missing from Erlang's resolver implementation. Any particular reason why? Ge' (utter Erlang newbie) > > --Per Hedeland > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From setori88@REDACTED Tue Aug 24 05:11:29 2010 From: setori88@REDACTED (stewart) Date: Mon, 23 Aug 2010 20:11:29 -0700 (PDT) Subject: Spawn an otp application In-Reply-To: References: Message-ID: Please forget this question, I have gone mad On Aug 24, 1:33?am, stewart mackenzie wrote: > Hi chaps, > > I would like to know if it is possible to dynamically spawn an otp app. > > If I use Wordpress as an example. It is an application which can be > downloaded and hosted at your expense, or Wordpress.com can do the dirty > work and host it for you at a fee. > > Presume I create a FOSS Erlang otp blog and an otp blog management > application and used it to setup a Wordpress.com clone. When a new user > creates an account this otp blog management program will ?dynamically spawn > an new otp blog for them. > > How do I do this? > > Thanks in advance. > > Rgds Stewart From fritchie@REDACTED Tue Aug 24 06:12:01 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Mon, 23 Aug 2010 23:12:01 -0500 Subject: ACM Erlang Workshop: accepted papers & schedule Message-ID: <1589.1282623121@snookles.snookles.com> Hi, all. While I work on getting the http://www.erlang.org/workshop/2010/* documents updated, here's a list of the accepted papers and schedule for the ACM Erlang Workshop 2010. I need to apologize to at least one erlang-questions reader who emailed me directly to ask about the schedule. Sorry! Hopefully the workshop Web-a-thingie will be updated in the next day or so. -Scott --- snip --- snip --- snip --- snip --- snip --- snip --- snip --- Ninth ACM SIGPLAN Erlang Workshop Baltimore, Maryland, USA, September 30, 2010 http://www.erlang.org/workshop/2010/ Satellite event of the 15th ACM SIGPLAN International Conference on Functional Programming, September 27-29, 2010 Workshop Program ---------------- Keynote Address Session Chair: Kostis Sagonas, National Technical University of Athens * Invited Talk: David Smith, Basho Technologies: "Concurrent Life, the Universe, and Everything Rebar" Testing and Programming Session Chair: Ulf Wiger, Erlang Solutions, London, UK. * From Test Cases to FSMs: Augmented Test Driven Development and Property Inference. Thomas Arts and Simon Thompson * Coordinating and Visualizing Independent Behaviors in Erlang. Guy Wiener, Gera Weiss, and Assaf Marron Theory and Practice Session Chair: Simon Thompson, University of Kent, Canterbury, Kent, UK. * A Unified Semantics for Future Erlang. Hans Svensson, Lars-?ke Fredlund, and Clara Benac Earle * Chain Replication In Theory and in Practice. Scott Lystig Fritchie Language Issues and Extensions Session Chair: Erik Stenman, Klarna AB, Stockholm, Sweden. * Analysis of Preprocessor Constructs in Erlang. R?bert Kitlei, Istv?n Boz?, Tam?s Kozsik, M?t? Tejfel, and Melinda T?th * Generic Load Regulation Framework for Erlang. Ulf Wiger Implementations, Tools, and Demos Session Chair: Scott Lystig Fritchie, Basho Technologies, Cambridge, Massachusetts, USA * Implementing a Multiagent Negotiation Protocol in Erlang. Alvaro Fernandez, Clara Benac Earle, and Lars-?ke Fredlund * QuickChecking Refactoring Tools. D?niel Drienyovszky, D?niel Horp?csi, and Simon Thompson * Using Erlang to Implement an Autonomous Build and Distribution System for Software Projects. Tino Breddin Latest news from the Erlang/OTP team at Ericsson From ingela@REDACTED Tue Aug 24 09:39:33 2010 From: ingela@REDACTED (Ingela Anderton Andin) Date: Tue, 24 Aug 2010 09:39:33 +0200 Subject: [erlang-questions] enforcing ssl trust chain In-Reply-To: <4C72AA0A.6030500@rabbitmq.com> References: <4C62D6FD.4010208@rabbitmq.com> <4C72AA0A.6030500@rabbitmq.com> Message-ID: <4C737735.6040502@erix.ericsson.se> Hi! Emile Joubert wrote: > >>> In a production environment I want to prevent clients without >>> certificates signed by a known CA from connecting. Is there any way of >>> getting this behaviour by using configuration files? The only way I can >>> find is to set verify_fun to an appropriate function, but this is >>> unappealing because I want to change my mind without needing to recompile. >>> >> At the moment defining a verify fun would be your option to accomplish this. >> We might add some other configuration option if we find that it seems to be >> a good thing from a general point of view. >> > > I've tried that, but verify_fun gets called regardless of whether verify > is set to verify_none or verify_peer. My reading of the documentation is > that certificate path validation errors should be ignored if verify_none > is set, regardless of verify_fun. Can you please confirm? > > Humm ... that might be a bug, proably in the public_key path validation. I am looking into that now. It should work as: If verify_none all possible path validation errors will be passed to the the verify fun and it will be up to the verify fun if the connection should fail or succeed. At the moment this option is only relevant for the client side as the server will never send a certificate request in verify_none mode. ( It is possible that we want this to be possible and we are looking in to that now). If verify_peer any path validation error should make the connection fail we should never get to the branch using the verify fun, the public_key path validation should return {error, Reason}. Regards Ingela Erlang/OTP team - Ericsson AB From raimo+erlang-questions@REDACTED Tue Aug 24 09:56:31 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 24 Aug 2010 09:56:31 +0200 Subject: [erlang-questions] IPV6_V6ONLY In-Reply-To: <201008231711.o7NHBgvY074811@pluto.hedeland.org> References: <201008231711.o7NHBgvY074811@pluto.hedeland.org> Message-ID: <20100824075631.GA854@erix.ericsson.se> On Mon, Aug 23, 2010 at 07:11:42PM +0200, Per Hedeland wrote: > Hi, > > On Linux (and maybe some others), by default, when binding a socket to > the IPv6 wildcard address, the socket accepts both IPv6 and IPv4, the > latter in the form of IPv4-mapped IPv6. Trying to bind another socket > with the same port and the IPv4 wildcard results in EADDRINUSE. > > On *BSD, by default (and always on Windows I believe), the situation is > the opposite - the IPv6-wildcard socket accepts only IPv6, and you need > to bind another socket for IPv4 if you want to support both. > > I believe there is some consensus in the networking community that the > first behaviour is "bad" for various reasons. Be that as it may, it is > unlikely to change, and it seems applications tend to ensure the second > behavior by means of unconditionally using the IPV6_V6ONLY socket option > in order to get consistent behavior across OSes (and avoid the "badness"). > It seems the situation has changed since Stevens' Unix Network Programming - The Sockets Networking API, third edition. There it seems one of the strong points of a IPv6 socket is that it accepts IPv4 connections _unless_ the IPV6_V6ONLY socket option is set. Where can these network community discussions be followed where can one find a motivation to why FreeBSD has diverged from Stevens' book (if that is the case)... > Since one strong point of Erlang/OTP is to as far as possible isolate > the Erlang programmer from these annoying differences between OSes, > could it be considered to make inet_drv always apply IPV6_V6ONLY (if it > exists) for IPv6 sockets? Since Linux (and previously Solaris) are the chief platforms for Erlang/OTP I would prefer trying to always get the Linux behaviour, if possible. > > --Per Hedeland > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Tue Aug 24 10:28:20 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 24 Aug 2010 10:28:20 +0200 Subject: [erlang-questions] IPV6_V6ONLY In-Reply-To: References: <201008231711.o7NHBgvY074811@pluto.hedeland.org> Message-ID: <20100824082820.GB854@erix.ericsson.se> On Mon, Aug 23, 2010 at 04:15:37PM -0700, G? Weijers wrote: > > > On Mon, 23 Aug 2010, Per Hedeland wrote: > > >Since one strong point of Erlang/OTP is to as far as possible isolate > >the Erlang programmer from these annoying differences between OSes, > >could it be considered to make inet_drv always apply IPV6_V6ONLY (if it > >exists) for IPv6 sockets? > > One way to implement a protocol-agnostic client and server in C on a Posix > system is to use the getaddrinfo() standard library routine, which > produces a list of parameters that can be fed into socket() + bind() for > servers, and socket() + connect() for clients. On the server size you may > end up with multiple sockets to listen to. They probably do not solve the problem at hand since when calling getaddrinfo("", "www", {AI_PASSIVE,AF_UNSPEC,SOCK_STREAM,0,...} you might get separate addrinfo structures for IPv6 and IPv4 back, even though creating only a listening socket for IPv6 (that can handle IPv4) is the "right" thing to do on the OS. This problem is actually mentioned in Stevens' Unix Network Programming 3:rd edition, section 11.6, towards the end in a fineprint note. > > getaddrinfo() and getnameinfo() or simular routines seem to be missing > from Erlang's resolver implementation. Any particular reason why? We have focused on making the old functions work good enough. In fact the native resolver in Erlang/OTP use getaddrinfo() and getnameinfo() internally when calling the OS resolver. Implementing getaddrinfo() and getnameinfo() would also demand a rework of all Erlang/OTP resolver parts to mimic their behaviour even when they do not exist in the underlying OS. > > Ge' (utter Erlang newbie) > > > > >--Per Hedeland > > > >________________________________________________________________ > >erlang-questions (at) erlang.org mailing list. > >See http://www.erlang.org/faq.html > >To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From v@REDACTED Tue Aug 24 10:38:04 2010 From: v@REDACTED (Valentin Micic) Date: Tue, 24 Aug 2010 10:38:04 +0200 Subject: [erlang-questions] IPV6_V6ONLY In-Reply-To: <201008231711.o7NHBgvY074811@pluto.hedeland.org> Message-ID: <20100824083814.E2AE33D0D58@mail.pharos-avantgard.com> Unless one is trying to bind to a raw socket, I'd say that the behavior implemented by Linux is the "proper" one, for socket is bound to the transport end point, thus, regardless which network protocol you're referring to, it should report EADDRINUSE. In other words, using different networking protocol should not create another address space at the transport layer. Having said this, I am not quite sure if the same holds for multi-homed host. For example, even if we're talking IPV4 only, but have a multiple interfaces, if you bind to port P1 on the interface I1, should another application be allowed to access P1 from another interface, say I2? IMO -- not at all, for transport access point should be the same regardless of which networking protocol is utilized. V/ -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Per Hedeland Sent: 23 August 2010 07:12 PM To: erlang-questions@REDACTED Subject: [erlang-questions] IPV6_V6ONLY Hi, On Linux (and maybe some others), by default, when binding a socket to the IPv6 wildcard address, the socket accepts both IPv6 and IPv4, the latter in the form of IPv4-mapped IPv6. Trying to bind another socket with the same port and the IPv4 wildcard results in EADDRINUSE. On *BSD, by default (and always on Windows I believe), the situation is the opposite - the IPv6-wildcard socket accepts only IPv6, and you need to bind another socket for IPv4 if you want to support both. I believe there is some consensus in the networking community that the first behaviour is "bad" for various reasons. Be that as it may, it is unlikely to change, and it seems applications tend to ensure the second behavior by means of unconditionally using the IPV6_V6ONLY socket option in order to get consistent behavior across OSes (and avoid the "badness"). Since one strong point of Erlang/OTP is to as far as possible isolate the Erlang programmer from these annoying differences between OSes, could it be considered to make inet_drv always apply IPV6_V6ONLY (if it exists) for IPv6 sockets? --Per Hedeland ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From raimo+erlang-questions@REDACTED Tue Aug 24 10:42:38 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 24 Aug 2010 10:42:38 +0200 Subject: [erlang-questions] Auto-recognizing IPv6-tuple In-Reply-To: <201008231552.o7NFqpkB073201@pluto.hedeland.org> References: <201008231552.o7NFqpkB073201@pluto.hedeland.org> Message-ID: <20100824084238.GC854@erix.ericsson.se> On Mon, Aug 23, 2010 at 05:52:51PM +0200, Per Hedeland wrote: > Hi, > > Way back in the thread starting with > http://www.erlang.org/cgi-bin/ezmlm-cgi/4/42722, I complained about "bad > behaviour" when some gen_tcp and gen_udp functions were passed an IPv6 > 8-tuple without the 'inet6' (or 'inet') option, suggesting (with > patches:-) that it would be more reasonable if IPv6 and IPv4 worked the > same - i.e. an 8-tuple would imply 'inet6'. Raimo made me happy with the > reply (in part): > > > Not really. But as you see for gen_tcp:connect below, the inet6 > > option is currently required there even for an IPv6 address tuple. > > These are corners we have not smoothed yet. > > However the "bad behaviour" was fixed in R13B, not by auto-recognizing > the 8-tuple but by exit(badarg), and this remains the case in R14A. > Should I take this to mean that the auto-recognition won't happen? FWIW, > the second version of the patches in that thread has been working fine > in production for over a year - initially in R10B-10, later in R13B03 - > and seems to apply cleanly and work fine with R14A too. No, the remaining problem just fell into oblivion when contemplating about what to do for the 4-tuple with option inet6... If I remember correctly your second version of patches only fixed the case of feeding an 8-tuple without using the inet6 option, and that should be a safe feature to add. (A git fetch patch would be easier to apply ;-) > > --Per Hedeland > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From roberto@REDACTED Tue Aug 24 12:15:37 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 24 Aug 2010 12:15:37 +0200 Subject: perform a function on a fixed time interval Message-ID: dear list, i have a loop process which needs to perform a computation every 1000 ms. this process needs also to receive messages from external processes. the issue arises from the fact that in every normal erlang loop, the 'after' statement is called only if no messages are receive by the process for a specific timeout. this is more or less how i could write a process which executes a computation every 1000 ms. the send_loop/2 emulates the external sending of messages, main_loop/1 is the process which must receive external processes messages and ensure to perform a computation every 1000 ms. you may run the code from the console with test:start/0. ----------------------------------- -module(test). -compile(export_all). start() -> % start main loop Pid = spawn(?MODULE, main_loop, [now()]), % start sender loop send_loop(Pid, 20), Pid ! shutdown. % send 20 messages every 500 ms send_loop(_Pid, 0) -> ok; send_loop(Pid, Count) -> receive after 500 -> Pid ! test, send_loop(Pid, Count - 1) end. main_loop(LastUpdate) -> receive shutdown -> shutdown; test -> io:format("received test message~n"), Now = now(), case timer:now_diff(Now, LastUpdate) / 1000 >= 1000 of true -> compute(), main_loop(Now); false -> main_loop(LastUpdate) end after 1000 -> compute(), main_loop(now()) end. compute() -> io:format("ping~n"). ----------------------------------- another solution would imply running a timer process which would signal the main_loop when to perform the computation. you may run the code from the console with test:start/0. ----------------------------------- -module(test). -compile(export_all). start() -> % start main loop Pid = spawn(?MODULE, main_loop, []), % start sender loop send_loop(Pid, 20), Pid ! shutdown. % send 20 messages every 500 ms send_loop(_Pid, 0) -> ok; send_loop(Pid, Count) -> receive after 500 -> Pid ! test, send_loop(Pid, Count - 1) end. main_loop() -> % start timer loop TimerPid = spawn(?MODULE, timer, [self()]), main_loop(TimerPid). main_loop(TimerPid) -> receive shutdown -> TimerPid ! shutdown; test -> io:format("received test message~n"), main_loop(TimerPid); compute -> compute(), main_loop(TimerPid) end. % send a ping every 1000 ms to the main_loop timer(MainPid) -> receive shutdown -> shutdown after 1000 -> MainPid ! compute, timer(MainPid) end. compute() -> io:format("ping~n"). ----------------------------------- which one would you choose, or is there any other way to perform this better? thank you, r. From hynek@REDACTED Tue Aug 24 12:34:32 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Tue, 24 Aug 2010 12:34:32 +0200 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: Message-ID: use timer:send_interval/2,3 On Tue, Aug 24, 2010 at 12:15 PM, Roberto Ostinelli wrote: > dear list, > > i have a loop process which needs to perform a computation every 1000 > ms. this process needs also to receive messages from external > processes. > > the issue arises from the fact that in every normal erlang loop, the > 'after' statement is called only if no messages are receive by the > process for a specific timeout. > > this is more or less how i could write a process which executes a > computation every 1000 ms. > > the send_loop/2 emulates the external sending of messages, main_loop/1 > is the process which must receive external processes messages and > ensure to perform a computation every 1000 ms. you may run the code > from the console with test:start/0. > > ----------------------------------- > > -module(test). > -compile(export_all). > > start() -> > ? ? ? ?% start main loop > ? ? ? ?Pid = spawn(?MODULE, main_loop, [now()]), > ? ? ? ?% start sender loop > ? ? ? ?send_loop(Pid, 20), > ? ? ? ?Pid ! shutdown. > > % send 20 messages every 500 ms > send_loop(_Pid, 0) -> ok; > send_loop(Pid, Count) -> > ? ? ? ?receive > ? ? ? ?after 500 -> > ? ? ? ? ? ? ? ?Pid ! test, > ? ? ? ? ? ? ? ?send_loop(Pid, Count - 1) > ? ? ? ?end. > > main_loop(LastUpdate) -> > ? ? ? ?receive > ? ? ? ? ? ? ? ?shutdown -> > ? ? ? ? ? ? ? ? ? ? ? ?shutdown; > ? ? ? ? ? ? ? ?test -> > ? ? ? ? ? ? ? ? ? ? ? ?io:format("received test message~n"), > ? ? ? ? ? ? ? ? ? ? ? ?Now = now(), > ? ? ? ? ? ? ? ? ? ? ? ?case timer:now_diff(Now, LastUpdate) / 1000 >= 1000 of > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?true -> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?compute(), > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?main_loop(Now); > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?false -> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?main_loop(LastUpdate) > ? ? ? ? ? ? ? ? ? ? ? ?end > ? ? ? ?after 1000 -> > ? ? ? ? ? ? ? ?compute(), > ? ? ? ? ? ? ? ?main_loop(now()) > ? ? ? ?end. > > compute() -> > ? ? ? ?io:format("ping~n"). > > ----------------------------------- > > another solution would imply running a timer process which would > signal the main_loop when to perform the computation. you may run the > code from the console with test:start/0. > > ----------------------------------- > > -module(test). > -compile(export_all). > > start() -> > ? ? ? ?% start main loop > ? ? ? ?Pid = spawn(?MODULE, main_loop, []), > ? ? ? ?% start sender loop > ? ? ? ?send_loop(Pid, 20), > ? ? ? ?Pid ! shutdown. > > % send 20 messages every 500 ms > send_loop(_Pid, 0) -> ok; > send_loop(Pid, Count) -> > ? ? ? ?receive > ? ? ? ?after 500 -> > ? ? ? ? ? ? ? ?Pid ! test, > ? ? ? ? ? ? ? ?send_loop(Pid, Count - 1) > ? ? ? ?end. > > main_loop() -> > ? ? ? ?% start timer loop > ? ? ? ?TimerPid = spawn(?MODULE, timer, [self()]), > ? ? ? ?main_loop(TimerPid). > main_loop(TimerPid) -> > ? ? ? ?receive > ? ? ? ? ? ? ? ?shutdown -> > ? ? ? ? ? ? ? ? ? ? ? ?TimerPid ! shutdown; > ? ? ? ? ? ? ? ?test -> > ? ? ? ? ? ? ? ? ? ? ? ?io:format("received test message~n"), > ? ? ? ? ? ? ? ? ? ? ? ?main_loop(TimerPid); > ? ? ? ? ? ? ? ?compute -> > ? ? ? ? ? ? ? ? ? ? ? ?compute(), > ? ? ? ? ? ? ? ? ? ? ? ?main_loop(TimerPid) > ? ? ? ?end. > > % send a ping every 1000 ms to the main_loop > timer(MainPid) -> > ? ? ? ?receive > ? ? ? ? ? ? ? ?shutdown -> > ? ? ? ? ? ? ? ? ? ? ? ?shutdown > ? ? ? ?after 1000 -> > ? ? ? ? ? ? ? ?MainPid ! compute, > ? ? ? ? ? ? ? ?timer(MainPid) > ? ? ? ?end. > > compute() -> > ? ? ? ?io:format("ping~n"). > > ----------------------------------- > > which one would you choose, or is there any other way to perform this better? > > thank you, > > r. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From roberto@REDACTED Tue Aug 24 12:40:52 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 24 Aug 2010 12:40:52 +0200 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: Message-ID: 2010/8/24 Hynek Vychodil : > use timer:send_interval/2,3 sure, have this version too: ---------------------------------------------- -module(test). -compile(export_all). start() -> % start main loop Pid = spawn(?MODULE, main_loop, []), % start timer timer:send_interval(1000, Pid, compute), % start sender loop send_loop(Pid, 20), Pid ! shutdown. % send 20 messages every 500 ms send_loop(_Pid, 0) -> ok; send_loop(Pid, Count) -> receive after 500 -> Pid ! test, send_loop(Pid, Count - 1) end. main_loop() -> receive shutdown -> shutdown; test -> io:format("received test message~n"), main_loop(); compute -> compute(), main_loop() end. compute() -> io:format("ping~n"). ---------------------------------------------- doesn't change much. fact is: is this really the only way? cheers, r. From roberto@REDACTED Tue Aug 24 12:54:00 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 24 Aug 2010 12:54:00 +0200 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: Message-ID: 2010/8/24 Hynek Vychodil : > use timer:send_interval/2,3 also: can't seem to find in the docs anywhere: what happens if the Pid the timer is sending a message to crashes? is this detected by the timer, or will it keep on sending messages to it anyway? From mazen.harake@REDACTED Tue Aug 24 11:54:24 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 24 Aug 2010 12:54:24 +0300 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: Message-ID: <4C7396D0.8000301@erlang-solutions.com> Be cautious, Using send messages in an interval is dangerous if compute() can take more then the interval. It can build up message queues on a congested system; especially if what compute() does has side effects. Just a word of advise. On 24/08/2010 13:40, Roberto Ostinelli wrote: > 2010/8/24 Hynek Vychodil: >> use timer:send_interval/2,3 > sure, have this version too: > > ---------------------------------------------- > > -module(test). > -compile(export_all). > > > start() -> > % start main loop > Pid = spawn(?MODULE, main_loop, []), > % start timer > timer:send_interval(1000, Pid, compute), > % start sender loop > send_loop(Pid, 20), > Pid ! shutdown. > > % send 20 messages every 500 ms > send_loop(_Pid, 0) -> ok; > send_loop(Pid, Count) -> > receive > after 500 -> > Pid ! test, > send_loop(Pid, Count - 1) > end. > > main_loop() -> > receive > shutdown -> > shutdown; > test -> > io:format("received test message~n"), > main_loop(); > compute -> > compute(), > main_loop() > end. > > compute() -> > io:format("ping~n"). > > ---------------------------------------------- > > doesn't change much. > > fact is: is this really the only way? > > cheers, > > r. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From roberto@REDACTED Tue Aug 24 13:06:12 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 24 Aug 2010 13:06:12 +0200 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: <4C7396D0.8000301@erlang-solutions.com> References: <4C7396D0.8000301@erlang-solutions.com> Message-ID: 2010/8/24 Mazen Harake : > ?Be cautious, > > Using send messages in an interval is dangerous if compute() can take more > then the interval. It can build up message queues on a congested system; > especially if what compute() does has side effects. > > Just a word of advise. > you are absolutely right, and i'm aware of the issue. it's a game engine which needs to detect collision between particles, and thus it needs to periodically compute the position of all elements and see if these collide or not. what would you recommend to ensure avoiding this issue? From max.lapshin@REDACTED Tue Aug 24 13:11:27 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 24 Aug 2010 15:11:27 +0400 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: <4C7396D0.8000301@erlang-solutions.com> Message-ID: This task is very important for erlyvideo, because video frames must arrive according to real time. http://github.com/erlyvideo/erlyvideo/blob/master/src/media/media_ticker.erl#L183 This is how I've solved it. We take wall clock on start and then on each cycle we calculate how much should we sleep. From therevoltingx@REDACTED Tue Aug 24 13:13:05 2010 From: therevoltingx@REDACTED (Miguel Morales) Date: Tue, 24 Aug 2010 04:13:05 -0700 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: <4C7396D0.8000301@erlang-solutions.com> Message-ID: Well, you'll want to learn about keeping a steady frame rate when doing calculations. Some of my bookmarks: http://gafferongames.com/game-physics/fix-your-timestep/ http://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking What you'll want to do is something like this: -define(FRAMES_PER_SECOND, 17). get_timestamp() -> {Mega,Sec,Micro} = erlang:now(), (Mega*1000000+Sec)*1000000+Micro. loop() -> Before = trunc(get_timestamp() / 1000000), update_physics(), TimeTakenMS = (get_timestamp() - Before), SleepTime = trunc(?FRAMES_PER_SECOND - (TimeTakenMS / 1000)), if SleepTime > 1 -> {ok, TRef} = timer:send_after(SleepTime, self(), loop); true -> %%we're running behind... loop() ! tick end, Pretty bad cut & paste from my code. But like Max said, just calculate how much you need to sleep before the next loop. On Tue, Aug 24, 2010 at 4:06 AM, Roberto Ostinelli wrote: > 2010/8/24 Mazen Harake : >> ?Be cautious, >> >> Using send messages in an interval is dangerous if compute() can take more >> then the interval. It can build up message queues on a congested system; >> especially if what compute() does has side effects. >> >> Just a word of advise. >> > > you are absolutely right, and i'm aware of the issue. > > it's a game engine which needs to detect collision between particles, > and thus it needs to periodically compute the position of all elements > and see if these collide or not. > > what would you recommend to ensure avoiding this issue? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- http://developingthedream.blogspot.com/, http://diastrofunk.com, http://www.youtube.com/user/revoltingx, ~Isaiah 55:8-9 From max.lapshin@REDACTED Tue Aug 24 13:16:05 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 24 Aug 2010 15:16:05 +0400 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: <4C7396D0.8000301@erlang-solutions.com> Message-ID: On Tue, Aug 24, 2010 at 3:13 PM, Miguel Morales wrote: > > get_timestamp() -> > ? ?{Mega,Sec,Micro} = erlang:now(), > ? ?(Mega*1000000+Sec)*1000000+Micro. > erlang:now may be very expensive, if you use if often From therevoltingx@REDACTED Tue Aug 24 13:26:37 2010 From: therevoltingx@REDACTED (Miguel Morales) Date: Tue, 24 Aug 2010 04:26:37 -0700 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: <4C7396D0.8000301@erlang-solutions.com> Message-ID: I'll keep that in mind. Didn't know it was expensive. Thanks. On Tue, Aug 24, 2010 at 4:16 AM, Max Lapshin wrote: > On Tue, Aug 24, 2010 at 3:13 PM, Miguel Morales wrote: >> >> get_timestamp() -> >> ? ?{Mega,Sec,Micro} = erlang:now(), >> ? ?(Mega*1000000+Sec)*1000000+Micro. >> > > erlang:now may be very expensive, if you use if often > -- http://developingthedream.blogspot.com/, http://diastrofunk.com, http://www.youtube.com/user/revoltingx, ~Isaiah 55:8-9 From max.lapshin@REDACTED Tue Aug 24 13:31:37 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 24 Aug 2010 15:31:37 +0400 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: <4C7396D0.8000301@erlang-solutions.com> Message-ID: On Tue, Aug 24, 2010 at 3:26 PM, Miguel Morales wrote: > I'll keep that in mind. ?Didn't know it was expensive. ?Thanks. > erlang:statistics(wallclock) may be lighter, if you use it 1000 times per second From roberto@REDACTED Tue Aug 24 13:35:41 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 24 Aug 2010 13:35:41 +0200 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: <4C7396D0.8000301@erlang-solutions.com> Message-ID: 2010/8/24 Miguel Morales : > Well, you'll want to learn about keeping a steady frame rate when > doing calculations. > Some of my bookmarks: http://gafferongames.com/game-physics/fix-your-timestep/ > http://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking > > What you'll want to do is something like this: > > -define(FRAMES_PER_SECOND, 17). > > get_timestamp() -> > ? ?{Mega,Sec,Micro} = erlang:now(), > ? ?(Mega*1000000+Sec)*1000000+Micro. > > loop() -> > ? ?Before = trunc(get_timestamp() / 1000000), > ? ?update_physics(), > ? ?TimeTakenMS = (get_timestamp() - Before), > ? ?SleepTime = trunc(?FRAMES_PER_SECOND - (TimeTakenMS / 1000)), > > ? if > ? ? ? ?SleepTime > 1 -> > ? ? ? ? ? ?{ok, TRef} = timer:send_after(SleepTime, self(), loop); > ? ? ? ?true -> ?%%we're running behind... > ? ? ? ? ? ?loop() ! tick > ? ?end, > > Pretty bad cut & paste from my code. ?But like Max said, just > calculate how much you need to sleep before the next loop. max, miguel, thank you i get your point. since the movements of my particles are defined in meters/second, i will therefore need to use the effective time passed since last computation, instead of a fixed variable, to determine current particle positions. since the top speed of a particle is 150m/s, and since the smallest object is of 10m in diameter, this means that i need to have a framerate less of 10/150 i.e. around 67ms, otherwise i could get the famous 'ghosts' [i.e. collisions not being detected, so it looks like if particles can get past one another]. 67 seconds should be enough of a time for these computations?.. From lucas.robsahm@REDACTED Tue Aug 24 14:44:48 2010 From: lucas.robsahm@REDACTED (Lucas Robsahm) Date: Tue, 24 Aug 2010 14:44:48 +0200 Subject: Receive UDP Broadcast when opening UDP port on specific IP Message-ID: I can't receive UDP Broadcast when I open UDP port on a specific IP. If I open it without specific IP or open the x.x.x.255 IP I do receive the UDP broadcast. If I open the x.x.x.255 IP or without specific IP, only one erlang-instance per computer can listen to the UDP broadcast so that's no solution. Can't receive UDP broadcast: gen_udp:open(123, [{ip, {123.123.123.123}}]). Can receive UDP broadcast: gen_udp:open(123, []). gen_udp_open(123, [{ip, {123.123.123.255}]). LUCAS ROBSAHM Software Developer Ericsson AB FJL/NT Datalinjen 4 Link?ping, Sweden Phone +46107114134 Mobile +46761498595 lucas.robsahm@REDACTED www.ericsson.com [cid:image002.gif@REDACTED] This Communication is Confidential. We only send and receive email on the basis of the terms set out at www.ericsson.com/email_disclaimer -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Tue Aug 24 15:21:17 2010 From: rvirding@REDACTED (Robert Virding) Date: Tue, 24 Aug 2010 15:21:17 +0200 Subject: [erlang-questions] random lookup in ets In-Reply-To: <20100823203051.GA8406@alumni.caltech.edu> References: <20100823203051.GA8406@alumni.caltech.edu> Message-ID: On 23 August 2010 22:30, Anthony Molinaro wrote: > On Mon, Aug 23, 2010 at 07:05:40AM +0200, Pascal Chapier wrote: >> first, if your application manage the ets, you can add an new table as >> index, with an integer list as key, and the main ets key as value. then >> you will have to select a random integer between min and max index key, >> retreive the main ets key and then the value. The problem with this is >> that you will have to keep the index keys continous, it is not so easy >> and may takes some time when you make a delete operation. On the other >> hand it should be fast for random read. > > If you are keeping a secondary index you could do what Paul Mineiro > suggested I do with a similar problem. ?In my problem I have non-overlapping > ranges of 32-bit integers. ?For each range I have data associated with > the range. ?Given an integer I wanted to look up the value associated > with the range. ?I use an ordered_set ets table which contains tuples > of the form > > { StartOfRange, SecondaryIndexKey } > > Then given an integer I do > > { StartOfRange, SecondaryIndexKey } = > ?case ets:lookup (Index, IntIp) of > ? ?[] -> > ? ? ?case ets:lookup (Index, ets:prev (Index, IntIp)) of > ? ? ? ?[] -> {0, -1}; > ? ? ? ?V -> hd (V) > ? ? ?end; > ? ?V -> hd (V) > ?end, > > To get back the SecondaryIndexKey (ets:prev will return the previous entry, > so using it when I don't exact match will give me the entry responsible > for the range). ?This is extremely fast (I haven't tested it but something > like > [ lookup (random:uniform (trunc (math:pow (2,32)))) > ?|| V <- lists:seq (1, 10000) ] > > where lookup/1 actually looks up the SecondaryIndexKey, gets the ets value > out of the secondary table (which is a large binary object, my > SecondaryIndexKey are actually offsets into a file), turns the returned > binary into a record where integers are turned into string via tuple > lookup tables, and returned. ?So doing lookup 10000 times with random > input takes about ~950 milliseconds on my laptop, so very, very fast :) This solutions assumes that ranges are adjacent to one another and that all possible indexes are a member of a range. Was this specified in the problem? Robert From mixolyde@REDACTED Tue Aug 24 15:27:44 2010 From: mixolyde@REDACTED (Brian Williams) Date: Tue, 24 Aug 2010 09:27:44 -0400 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: <4C7396D0.8000301@erlang-solutions.com> Message-ID: On Tue, Aug 24, 2010 at 7:06 AM, Roberto Ostinelli wrote: > 2010/8/24 Mazen Harake : >> ?Be cautious, >> >> Using send messages in an interval is dangerous if compute() can take more >> then the interval. It can build up message queues on a congested system; >> especially if what compute() does has side effects. >> >> Just a word of advise. >> > > you are absolutely right, and i'm aware of the issue. > > it's a game engine which needs to detect collision between particles, > and thus it needs to periodically compute the position of all elements > and see if these collide or not. > > what would you recommend to ensure avoiding this issue? > Another option is to spawn off another process to run the computation. If you receive another request to compute() before this process is finished, you can either ignore the first compute's return, send it a stop signal of some kind or handle its partial computation somehow. Then spawn off the new computation. This also gives you the opportunity to log an event of some kind whenever the computation runs too long and you can track it down. Personally I prefer the idea of a computation process that receives a tick message, that way it can handle its own state, and keep that updated by other messages in between ticks. -- Brian E. Williams mixolyde@REDACTED http://www.techhouse.us/wordpress-mu/brianw "Never attribute to malice that which can be adequately explained by stupidity." - Hanlon's Razor From max.lapshin@REDACTED Tue Aug 24 15:30:55 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 24 Aug 2010 17:30:55 +0400 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: <4C7396D0.8000301@erlang-solutions.com> Message-ID: On Tue, Aug 24, 2010 at 5:27 PM, Brian Williams wrote: > Personally I prefer the idea of a computation process that receives a > tick message, that way it can handle its own state, and keep that > updated by other messages in between ticks. It cannot guarantee you monotonic in time. You will have 1.0001 coefficient from wall clock and it may become a real problem in some cases. Only comparing to wall clock may give you synchronization From hynek@REDACTED Tue Aug 24 16:45:00 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Tue, 24 Aug 2010 16:45:00 +0200 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: Message-ID: I'm not checked if docs is silent about it but source code is pretty straightforward (it's why I love Erlang so much), timer links to receiver and also removes timer when receiver dies. On Tue, Aug 24, 2010 at 12:54 PM, Roberto Ostinelli wrote: > 2010/8/24 Hynek Vychodil : >> use timer:send_interval/2,3 > > also: can't seem to find in the docs anywhere: what happens if the Pid > the timer is sending a message to crashes? is this detected by the > timer, or will it keep on sending messages to it anyway? > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From per@REDACTED Tue Aug 24 17:20:00 2010 From: per@REDACTED (Per Hedeland) Date: Tue, 24 Aug 2010 17:20:00 +0200 (CEST) Subject: [erlang-questions] IPV6_V6ONLY In-Reply-To: <20100824075631.GA854@erix.ericsson.se> Message-ID: <201008241520.o7OFK0CB003180@pluto.hedeland.org> Raimo Niskanen wrote: > >It seems the situation has changed since Stevens' Unix Network Programming >- The Sockets Networking API, third edition. There it seems one of the strong >points of a IPv6 socket is that it accepts IPv4 connections _unless_ the >IPV6_V6ONLY socket option is set. Yes, it sounds nice in theory, but practice seems to be another thing. >Where can these network community discussions be followed where >can one find a motivation to why FreeBSD has diverged from Stevens' book >(if that is the case)... The *BSD family did indeed start out with the "map by default" logic, but reverted their default some years back (for FreeBSD I believe it was with 5.0). I haven't found any significant discussion *preceding* the FreeBSD change (nor even explicit mention of it in release notes), but from references after-the-fact in mailing lists etc it seems to have been primarily security-motivated, boiling down to the fact that the application cannot differentiate between "real" IPv4 connections that are mapped by the stack, and connections that use mapped addresses on the wire. Frequently referenced are: http://tools.ietf.org/html/draft-cmetz-v6ops-v4mapped-api-harmful-00 http://tools.ietf.org/html/draft-itojun-v6ops-v4mapped-harmful-02 (arguing for opposite fixes to the problem:-). Google also turned up http://books.google.se/books?id=JYhA5uqOxIAC&printsec=frontcover#v=snippet&q=ipv4-mapped&f=false Personally I also see usability issues, e.g.: - Having "real" IPv4 connections show up in the mapped form in status info and logs is confusing/misleading. - Server applications typically allow for configuration of (multiple) addresses to listen on - if you want to configure specific addresses, the stack-mapping can't come into play, and you always need to configure IPv4 and IPv6 separately (and the application needs to use separate sockets). But if you want to configure a wildcard, you should configure *only* IPv6 (and the appearance of IPv4 connections changes per above). And it's not even possible (without extra config options) to configure "wildcard but only IPv6". Inconsistent/confusing. >> Since one strong point of Erlang/OTP is to as far as possible isolate >> the Erlang programmer from these annoying differences between OSes, >> could it be considered to make inet_drv always apply IPV6_V6ONLY (if it >> exists) for IPv6 sockets? > >Since Linux (and previously Solaris) are the chief platforms for Erlang/OTP >I would prefer trying to always get the Linux behaviour, if possible. I guess you have a point, even though I think it's the wrong choice.:-) --Per From jon@REDACTED Tue Aug 24 16:49:41 2010 From: jon@REDACTED (Jon Meredith) Date: Tue, 24 Aug 2010 08:49:41 -0600 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: <4C7396D0.8000301@erlang-solutions.com> Message-ID: <4C73DC05.6080707@jonmeredith.com> On 8/24/10 5:31 AM, Max Lapshin wrote: > On Tue, Aug 24, 2010 at 3:26 PM, Miguel Morales wrote: >> I'll keep that in mind. Didn't know it was expensive. Thanks. >> >> erlang:statistics(wallclock) may be lighter, if you use it 1000 times per second >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED I did a little microbenchmarking on OSX to look at the three date/timestamp options I know about. It turns out that statistics(wallclock) takes a very similar amount of time to run as erlang:now(), but os:timestamp() is almost twice as fast (without any of the guarantees of now/0). now()/statistics(wallclock) ~ 6.5 million calls/sec os:timestamp() ~ 11.7 million calls/sec *NB* I didn't measure concurrent performance of the three calls so there are no effects from locks measured. Also be careful of using timer:tc() when measuring code with many calls to now(). Each call to now() increments the nanosecs to make sure you get a unique value per node. If you call now() fast enough it affects your timings. I wrote equivalents of timer:tc for the other two time measuring methods and they align fairly well. %% Erlang time mechanism micro-benchmarks (8-core 3Ghz Macpro2,1, R13B04 64-bit) %% %% Conclusion: now()/statistics(wallclock) take about the same amount of time %% os:timestamp() runs almost twice as fast as the other two %% using timer:tc on code that calls now() a lot is unreliable %% %% 35> bench:tstc(bench, do_now, [10000000]). %% {1537955,ok} %% 36> bench:wctc(bench, do_now, [10000000]). %% {1542000,ok} %% 37> bench:tstc(bench, do_timestamp, [10000000]). %% {853846,ok} %% 38> bench:wctc(bench, do_timestamp, [10000000]). %% {857000,ok} %% 39> bench:tstc(bench, do_wallclock, [10000000]). %% {1515972,ok} %% 40> bench:wctc(bench, do_wallclock, [10000000]). %% {1525000,ok} -module(bench). -compile([export_all]). do_now(0) -> ok; do_now(N) -> now(), do_now(N-1). do_wallclock(0) -> ok; do_wallclock(N) -> erlang:statistics(wall_clock), do_wallclock(N-1). do_timestamp(0) -> ok; do_timestamp(N) -> os:timestamp(), do_timestamp(N-1). wctc(M, F, A) -> {S,_} = erlang:statistics(wall_clock), R = apply(M,F,A), {E,_} = erlang:statistics(wall_clock), {(E - S) * 1000, R}. tstc(M, F, A) -> S = os:timestamp(), R = apply(M,F,A), E = os:timestamp(), {timer:now_diff(E, S), R}. From anthonym@REDACTED Tue Aug 24 19:45:16 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Tue, 24 Aug 2010 10:45:16 -0700 Subject: [erlang-questions] random lookup in ets In-Reply-To: References: <20100823203051.GA8406@alumni.caltech.edu> Message-ID: <20100824174516.GA11673@alumni.caltech.edu> On Tue, Aug 24, 2010 at 03:21:17PM +0200, Robert Virding wrote: > On 23 August 2010 22:30, Anthony Molinaro wrote: > > On Mon, Aug 23, 2010 at 07:05:40AM +0200, Pascal Chapier wrote: > >> first, if your application manage the ets, you can add an new table as > >> index, with an integer list as key, and the main ets key as value. then > >> you will have to select a random integer between min and max index key, > >> retreive the main ets key and then the value. The problem with this is > >> that you will have to keep the index keys continous, it is not so easy > >> and may takes some time when you make a delete operation. On the other > >> hand it should be fast for random read. > > > > If you are keeping a secondary index you could do what Paul Mineiro > > suggested I do with a similar problem. ?In my problem I have non-overlapping > > ranges of 32-bit integers. ?For each range I have data associated with > > the range. ?Given an integer I wanted to look up the value associated > > with the range. ?I use an ordered_set ets table which contains tuples > > of the form > > > > { StartOfRange, SecondaryIndexKey } > > > > Then given an integer I do > > > > { StartOfRange, SecondaryIndexKey } = > > ?case ets:lookup (Index, IntIp) of > > ? ?[] -> > > ? ? ?case ets:lookup (Index, ets:prev (Index, IntIp)) of > > ? ? ? ?[] -> {0, -1}; > > ? ? ? ?V -> hd (V) > > ? ? ?end; > > ? ?V -> hd (V) > > ?end, > > > > To get back the SecondaryIndexKey (ets:prev will return the previous entry, > > so using it when I don't exact match will give me the entry responsible > > for the range). ?This is extremely fast (I haven't tested it but something > > like > > [ lookup (random:uniform (trunc (math:pow (2,32)))) > > ?|| V <- lists:seq (1, 10000) ] > > > > where lookup/1 actually looks up the SecondaryIndexKey, gets the ets value > > out of the secondary table (which is a large binary object, my > > SecondaryIndexKey are actually offsets into a file), turns the returned > > binary into a record where integers are turned into string via tuple > > lookup tables, and returned. ?So doing lookup 10000 times with random > > input takes about ~950 milliseconds on my laptop, so very, very fast :) > > This solutions assumes that ranges are adjacent to one another and > that all possible indexes are a member of a range. Was this specified > in the problem? In my problem the ranges were adjacent and all ranges were represented in the ets lookup table, I made sure of that when I created the ets table. For the problem here I don't think it matters, since you don't care what data you get back, just that its randomly choosen. -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From Antonio.Musumeci@REDACTED Tue Aug 24 22:08:23 2010 From: Antonio.Musumeci@REDACTED (Musumeci, Antonio S) Date: Tue, 24 Aug 2010 16:08:23 -0400 Subject: Bug, feature or limitation of erl_parse? Message-ID: <01489E237C2C8F45AF7898E135F48E8019A3B1CE74@NYWEXMBX2129.msad.ms.com> It doesn't appear to understand the "test" ++ Else shortcut for [$t,$e,$s,$t|Else]. Reason I noticed was I'm reading in terms as strings and passing them to mnesia:select. erl_parse works fine with [$t|'_'] but not "t" ++ '_'. > S = "[$t,$e,$s,$t|'_']". "[$t,$e,$s,$t|'_']" > T = "\"test\" ++ '_'". "\"test\" ++ '_'" > [$t,$e,$s,$t|'_'] = "test" ++ '_'. [116,101,115,116|'_'] > {ok,S2,_} = erl_scan:string(S). {ok,[{'[',1}, {char,1,116}, {',',1}, {char,1,101}, {',',1}, {char,1,115}, {',',1}, {char,1,116}, {'|',1}, {atom,1,'_'}, {']',1}], 1} > {ok,T2,_} = erl_scan:string(T). {ok,[{string,1,"test"},{'++',1},{atom,1,'_'}],1} > erl_parse:parse_term(S2++[{dot,1}]). {ok,[116,101,115,116|'_']} > erl_parse:parse_term(T2++[{dot,1}]). {error,{1,erl_parse,"bad term"}} From rvirding@REDACTED Tue Aug 24 23:01:16 2010 From: rvirding@REDACTED (Robert Virding) Date: Tue, 24 Aug 2010 23:01:16 +0200 Subject: [erlang-questions] Bug, feature or limitation of erl_parse? In-Reply-To: <01489E237C2C8F45AF7898E135F48E8019A3B1CE74@NYWEXMBX2129.msad.ms.com> References: <01489E237C2C8F45AF7898E135F48E8019A3B1CE74@NYWEXMBX2129.msad.ms.com> Message-ID: Erl_parse does recognise '"test" ++ Else', but it is not a term it is an expression. It is later inside the compiler that the expression is converted into it's shortcut form. The parser just parses the expression. When you did it by hand, so to speak, you got a term which erl_parse could parse as a term. So: 17> f(S), {ok,S,_} = erl_scan:string("\"test\" ++ Else. "). {ok,[{string,1,"test"},{'++',1},{var,1,'Else'},{dot,1}],1} 18> erl_parse:parse_exprs(S). {ok,[{op,1,'++',{string,1,"test"},{var,1,'Else'}}]} N.B. it it not the string "term" which causes it to be an expression but the operator '++'. Robert On 24 August 2010 22:08, Musumeci, Antonio S wrote: > It doesn't appear to understand the "test" ++ Else shortcut for [$t,$e,$s,$t|Else]. Reason I noticed was I'm reading in terms as strings and passing them to mnesia:select. erl_parse works fine with [$t|'_'] but not "t" ++ '_'. > >> S = "[$t,$e,$s,$t|'_']". > "[$t,$e,$s,$t|'_']" >> T = "\"test\" ++ '_'". > "\"test\" ++ '_'" >> [$t,$e,$s,$t|'_'] = "test" ++ '_'. > [116,101,115,116|'_'] >> {ok,S2,_} = erl_scan:string(S). > {ok,[{'[',1}, > ? ? {char,1,116}, > ? ? {',',1}, > ? ? {char,1,101}, > ? ? {',',1}, > ? ? {char,1,115}, > ? ? {',',1}, > ? ? {char,1,116}, > ? ? {'|',1}, > ? ? {atom,1,'_'}, > ? ? {']',1}], > ? ?1} >> {ok,T2,_} = erl_scan:string(T). > {ok,[{string,1,"test"},{'++',1},{atom,1,'_'}],1} >> erl_parse:parse_term(S2++[{dot,1}]). > {ok,[116,101,115,116|'_']} >> erl_parse:parse_term(T2++[{dot,1}]). > {error,{1,erl_parse,"bad term"}} > > From norton@REDACTED Tue Aug 24 13:20:02 2010 From: norton@REDACTED (Joseph Wayne Norton) Date: Tue, 24 Aug 2010 20:20:02 +0900 Subject: [erlang-questions] perform a function on a fixed time interval In-Reply-To: References: <4C7396D0.8000301@erlang-solutions.com> Message-ID: One technique is to flush the erlang process's mailbox after returning from compute(). You can find an example inside this erlang module: http://github.com/norton/gdss/blob/master/src/brick_clientmon.erl Look for flush_mailbox_extra_check_status(). The flush is needed because net_adm:ping/1 can take several seconds if the target host is offline. thanks, On Tue, 24 Aug 2010 20:06:12 +0900, Roberto Ostinelli wrote: > 2010/8/24 Mazen Harake : >> Be cautious, >> >> Using send messages in an interval is dangerous if compute() can take >> more >> then the interval. It can build up message queues on a congested system; >> especially if what compute() does has side effects. >> >> Just a word of advise. >> > > you are absolutely right, and i'm aware of the issue. > > it's a game engine which needs to detect collision between particles, > and thus it needs to periodically compute the position of all elements > and see if these collide or not. > > what would you recommend to ensure avoiding this issue? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- norton@REDACTED From watson.timothy@REDACTED Wed Aug 25 00:16:05 2010 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 24 Aug 2010 23:16:05 +0100 Subject: [erlang-questions] ACM Erlang Workshop: accepted papers & schedule In-Reply-To: <1589.1282623121@snookles.snookles.com> References: <1589.1282623121@snookles.snookles.com> Message-ID: > ? ? * Using Erlang to Implement an Autonomous Build and Distribution > ? ? ? System for Software Projects. Tino Breddin > I was thinking of starting an OSS project to do exactly that (plus Continuous Integration). Think maybe I'll wait now and see what's been done already! From fritchie@REDACTED Wed Aug 25 04:37:51 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 24 Aug 2010 21:37:51 -0500 Subject: [erlang-questions] How to stop a detached OTP app? In-Reply-To: Message of "Fri, 20 Aug 2010 22:43:29 +0300." <4C6EDAE1.6070300@erlang-solutions.com> Message-ID: <83201.1282703871@snookles.snookles.com> Mazen Harake wrote: mh> I would also recommend adding the -hidden flag for the temporary mh> node in case the node you are connecting to is part of a cluster mh> otherwise funny things could happen. I second this recommendation. If your app uses 'global', then a very short-lived administrative task/control script like this can cause the global service's replication protocol to generate error messages that are harmless but look scary(*). Ditto for other applications that subscribe to the net_kernel's event notifications to react when remote nodes connect & disconnect. -Scott (*) And cause customers to open trouble tickets. From mojitotech@REDACTED Wed Aug 25 13:56:56 2010 From: mojitotech@REDACTED (Mojito Sorbet) Date: Wed, 25 Aug 2010 07:56:56 -0400 Subject: [erlang-questions] Let's start a project for a (MMO-)Game in Erlang... (inspired by "Erlang: Who uses it for games?") In-Reply-To: References: Message-ID: <1282737416.2612.13.camel@crescendo> I have been working on some ideas in this area, not using the croquet model, but something more centralized like OpenSim and SecondLife. The implementations of those virtual world systems have serious scalability problems because they tie inworld "geography" to fixed compute resources. My project investigates ways to get around that. So far I have the basic world simulator application, a network-access application, and a viewer, all written in Erlang. The viewer uses OpenGL and wxWidgets. You can read about it at http://github.com/msorbet/ConWorld I am currently trying to get the documentation to catch up to the code. :) My focus is not on making any particular game, but on looking at scalability issues, including dynamic use of additional compute resources. > > On Fri, Jul 9, 2010 at 7:55 PM, Boris M?hmer > wrote: > > It looks like some people are interested in Erlang game programming. > > On the other hand getting started in Erlang and OTP isn't too easy. > > Especially for people with a strong C/C++/C#/Java/etc background. > > > > Inspired by the "Erlang: Who uses it for games?" thread I thought about > > starting a community project to implement a MMO game in Erlang/OTP. > > The goal would be to collect some "best-practices" for different tasks. > > > > Actually I am more interested in the server side than how to implement > > a top-notch-state-of-the-art client. Also I think Erlang is more suited > > for the server side. But I am also interested in how to interface the > > "server-side erlang" using a (C/C++, Java, Python) cross-plattform > > client (using wxWidget or Qt as a base and OpenGL for graphics). > > > > What I would like to see are reponses from other people who may also > > be interested in such a project. Not only beginners with Erlang/OTP, > > but also (or especially) experienced people to guide and support the > > others. And most of all, to have some fun practising Erlang. > > > > Well, what do You think about it? > > > > Best regards, > > Boris > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From Antonio.Musumeci@REDACTED Wed Aug 25 14:17:27 2010 From: Antonio.Musumeci@REDACTED (Musumeci, Antonio S) Date: Wed, 25 Aug 2010 08:17:27 -0400 Subject: [erlang-questions] Bug, feature or limitation of erl_parse? In-Reply-To: References: <01489E237C2C8F45AF7898E135F48E8019A3B1CE74@NYWEXMBX2129.msad.ms.com> Message-ID: <01489E237C2C8F45AF7898E135F48E8019A3B1CEC3@NYWEXMBX2129.msad.ms.com> Understood. Is there no way to turn "test" ++ '_' into [$t,$e,$s,$t|'_'] from a string for use in a matchspec? -----Original Message----- From: Robert Virding [mailto:rvirding@REDACTED] Sent: Tuesday, August 24, 2010 5:01 PM To: Musumeci, Antonio S (Enterprise Infrastructure) Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Bug, feature or limitation of erl_parse? Erl_parse does recognise '"test" ++ Else', but it is not a term it is an expression. It is later inside the compiler that the expression is converted into it's shortcut form. The parser just parses the expression. When you did it by hand, so to speak, you got a term which erl_parse could parse as a term. So: 17> f(S), {ok,S,_} = erl_scan:string("\"test\" ++ Else. "). {ok,[{string,1,"test"},{'++',1},{var,1,'Else'},{dot,1}],1} 18> erl_parse:parse_exprs(S). {ok,[{op,1,'++',{string,1,"test"},{var,1,'Else'}}]} N.B. it it not the string "term" which causes it to be an expression but the operator '++'. Robert On 24 August 2010 22:08, Musumeci, Antonio S wrote: > It doesn't appear to understand the "test" ++ Else shortcut for [$t,$e,$s,$t|Else]. Reason I noticed was I'm reading in terms as strings and passing them to mnesia:select. erl_parse works fine with [$t|'_'] but not "t" ++ '_'. > >> S = "[$t,$e,$s,$t|'_']". > "[$t,$e,$s,$t|'_']" >> T = "\"test\" ++ '_'". > "\"test\" ++ '_'" >> [$t,$e,$s,$t|'_'] = "test" ++ '_'. > [116,101,115,116|'_'] >> {ok,S2,_} = erl_scan:string(S). > {ok,[{'[',1}, > {char,1,116}, > {',',1}, > {char,1,101}, > {',',1}, > {char,1,115}, > {',',1}, > {char,1,116}, > {'|',1}, > {atom,1,'_'}, > {']',1}], > 1} >> {ok,T2,_} = erl_scan:string(T). > {ok,[{string,1,"test"},{'++',1},{atom,1,'_'}],1} >> erl_parse:parse_term(S2++[{dot,1}]). > {ok,[116,101,115,116|'_']} >> erl_parse:parse_term(T2++[{dot,1}]). > {error,{1,erl_parse,"bad term"}} From rtrlists@REDACTED Wed Aug 25 14:52:10 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 25 Aug 2010 13:52:10 +0100 Subject: [erlang-questions] Bug, feature or limitation of erl_parse? In-Reply-To: <01489E237C2C8F45AF7898E135F48E8019A3B1CEC3@NYWEXMBX2129.msad.ms.com> References: <01489E237C2C8F45AF7898E135F48E8019A3B1CE74@NYWEXMBX2129.msad.ms.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CEC3@NYWEXMBX2129.msad.ms.com> Message-ID: Hmm, [$t,$e,$s,$t|'_'] is not a proper list. Do you mean [$t,$e,$s,$t,'_'] (or the equivalent [$t,$e,$s,$t|['_']])? If yes, then "test"++['_'] will do the match: Eshell V5.6.5 (abort with ^G) 1> L=[$t,$e,$s,$t|['_']]. [116,101,115,116,'_'] 2> "test"++['_']=L. [116,101,115,116,'_'] But I'm pretty sure I am not understanding your question exactly. Robby PS Interesting, why does this work: 3> "test"++'_'. [116,101,115,116|'_'] ??? On Wed, Aug 25, 2010 at 1:17 PM, Musumeci, Antonio S < Antonio.Musumeci@REDACTED> wrote: > Understood. Is there no way to turn "test" ++ '_' into [$t,$e,$s,$t|'_'] > from a string for use in a matchspec? > > -----Original Message----- > From: Robert Virding [mailto:rvirding@REDACTED] > Sent: Tuesday, August 24, 2010 5:01 PM > To: Musumeci, Antonio S (Enterprise Infrastructure) > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Bug, feature or limitation of erl_parse? > > Erl_parse does recognise '"test" ++ Else', but it is not a term it is an > expression. It is later inside the compiler that the expression is converted > into it's shortcut form. The parser just parses the expression. When you did > it by hand, so to speak, you got a term which erl_parse could parse as a > term. So: > > 17> f(S), {ok,S,_} = erl_scan:string("\"test\" ++ Else. "). > {ok,[{string,1,"test"},{'++',1},{var,1,'Else'},{dot,1}],1} > 18> erl_parse:parse_exprs(S). > {ok,[{op,1,'++',{string,1,"test"},{var,1,'Else'}}]} > > N.B. it it not the string "term" which causes it to be an expression but > the operator '++'. > > Robert > > On 24 August 2010 22:08, Musumeci, Antonio S < > Antonio.Musumeci@REDACTED> wrote: > > It doesn't appear to understand the "test" ++ Else shortcut for > [$t,$e,$s,$t|Else]. Reason I noticed was I'm reading in terms as strings and > passing them to mnesia:select. erl_parse works fine with [$t|'_'] but not > "t" ++ '_'. > > > >> S = "[$t,$e,$s,$t|'_']". > > "[$t,$e,$s,$t|'_']" > >> T = "\"test\" ++ '_'". > > "\"test\" ++ '_'" > >> [$t,$e,$s,$t|'_'] = "test" ++ '_'. > > [116,101,115,116|'_'] > >> {ok,S2,_} = erl_scan:string(S). > > {ok,[{'[',1}, > > {char,1,116}, > > {',',1}, > > {char,1,101}, > > {',',1}, > > {char,1,115}, > > {',',1}, > > {char,1,116}, > > {'|',1}, > > {atom,1,'_'}, > > {']',1}], > > 1} > >> {ok,T2,_} = erl_scan:string(T). > > {ok,[{string,1,"test"},{'++',1},{atom,1,'_'}],1} > >> erl_parse:parse_term(S2++[{dot,1}]). > > {ok,[116,101,115,116|'_']} > >> erl_parse:parse_term(T2++[{dot,1}]). > > {error,{1,erl_parse,"bad term"}} > > > From Antonio.Musumeci@REDACTED Wed Aug 25 15:22:23 2010 From: Antonio.Musumeci@REDACTED (Musumeci, Antonio S) Date: Wed, 25 Aug 2010 09:22:23 -0400 Subject: [erlang-questions] Bug, feature or limitation of erl_parse? In-Reply-To: References: <01489E237C2C8F45AF7898E135F48E8019A3B1CE74@NYWEXMBX2129.msad.ms.com><01489E237C2C8F45AF7898E135F48E8019A3B1CEC3@NYWEXMBX2129.msad.ms.com> Message-ID: <01489E237C2C8F45AF7898E135F48E8019A3B1CEDF@NYWEXMBX2129.msad.ms.com> http://www.erlang.org/doc/apps/erts/match_spec.html It shows that [$t|'_'] is a proper match pattern... And it does work. Since otherwise "t" ++ Else is a shortcut for [$t|Else] I was trying to find out why I appear unable to parse a string of the former to use in a matchspec as the latter one does work. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Robert Raschke Sent: Wednesday, August 25, 2010 8:52 AM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] Bug, feature or limitation of erl_parse? Hmm, [$t,$e,$s,$t|'_'] is not a proper list. Do you mean [$t,$e,$s,$t,'_'] (or the equivalent [$t,$e,$s,$t|['_']])? If yes, then "test"++['_'] will do the match: Eshell V5.6.5 (abort with ^G) 1> L=[$t,$e,$s,$t|['_']]. [116,101,115,116,'_'] 2> "test"++['_']=L. [116,101,115,116,'_'] But I'm pretty sure I am not understanding your question exactly. Robby PS Interesting, why does this work: 3> "test"++'_'. [116,101,115,116|'_'] ??? On Wed, Aug 25, 2010 at 1:17 PM, Musumeci, Antonio S < Antonio.Musumeci@REDACTED> wrote: > Understood. Is there no way to turn "test" ++ '_' into > [$t,$e,$s,$t|'_'] from a string for use in a matchspec? > > -----Original Message----- > From: Robert Virding [mailto:rvirding@REDACTED] > Sent: Tuesday, August 24, 2010 5:01 PM > To: Musumeci, Antonio S (Enterprise Infrastructure) > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Bug, feature or limitation of erl_parse? > > Erl_parse does recognise '"test" ++ Else', but it is not a term it is > an expression. It is later inside the compiler that the expression is > converted into it's shortcut form. The parser just parses the > expression. When you did it by hand, so to speak, you got a term which > erl_parse could parse as a term. So: > > 17> f(S), {ok,S,_} = erl_scan:string("\"test\" ++ Else. "). > {ok,[{string,1,"test"},{'++',1},{var,1,'Else'},{dot,1}],1} > 18> erl_parse:parse_exprs(S). > {ok,[{op,1,'++',{string,1,"test"},{var,1,'Else'}}]} > > N.B. it it not the string "term" which causes it to be an expression > but the operator '++'. > > Robert > > On 24 August 2010 22:08, Musumeci, Antonio S < > Antonio.Musumeci@REDACTED> wrote: > > It doesn't appear to understand the "test" ++ Else shortcut for > [$t,$e,$s,$t|Else]. Reason I noticed was I'm reading in terms as > strings and passing them to mnesia:select. erl_parse works fine with > [$t|'_'] but not "t" ++ '_'. > > > >> S = "[$t,$e,$s,$t|'_']". > > "[$t,$e,$s,$t|'_']" > >> T = "\"test\" ++ '_'". > > "\"test\" ++ '_'" > >> [$t,$e,$s,$t|'_'] = "test" ++ '_'. > > [116,101,115,116|'_'] > >> {ok,S2,_} = erl_scan:string(S). > > {ok,[{'[',1}, > > {char,1,116}, > > {',',1}, > > {char,1,101}, > > {',',1}, > > {char,1,115}, > > {',',1}, > > {char,1,116}, > > {'|',1}, > > {atom,1,'_'}, > > {']',1}], > > 1} > >> {ok,T2,_} = erl_scan:string(T). > > {ok,[{string,1,"test"},{'++',1},{atom,1,'_'}],1} > >> erl_parse:parse_term(S2++[{dot,1}]). > > {ok,[116,101,115,116|'_']} > >> erl_parse:parse_term(T2++[{dot,1}]). > > {error,{1,erl_parse,"bad term"}} > > > From hynek@REDACTED Wed Aug 25 15:38:21 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Wed, 25 Aug 2010 15:38:21 +0200 Subject: [erlang-questions] Bug, feature or limitation of erl_parse? In-Reply-To: <01489E237C2C8F45AF7898E135F48E8019A3B1CEDF@NYWEXMBX2129.msad.ms.com> References: <01489E237C2C8F45AF7898E135F48E8019A3B1CE74@NYWEXMBX2129.msad.ms.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CEC3@NYWEXMBX2129.msad.ms.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CEDF@NYWEXMBX2129.msad.ms.com> Message-ID: What is problem? 1> L = "test". "test" 2> Match = L ++ '_'. [116,101,115,116|'_'] On Wed, Aug 25, 2010 at 3:22 PM, Musumeci, Antonio S wrote: > http://www.erlang.org/doc/apps/erts/match_spec.html > > It shows that [$t|'_'] is a proper match pattern... And it does work. Since otherwise "t" ++ Else is a shortcut for [$t|Else] I was trying to find out why I appear unable to parse a string of the former to use in a matchspec as the latter one does work. > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Robert Raschke > Sent: Wednesday, August 25, 2010 8:52 AM > To: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Bug, feature or limitation of erl_parse? > > Hmm, [$t,$e,$s,$t|'_'] is not a proper list. > Do you mean [$t,$e,$s,$t,'_'] (or the equivalent [$t,$e,$s,$t|['_']])? > > If yes, then "test"++['_'] will do the match: > > Eshell V5.6.5 ?(abort with ^G) > 1> L=[$t,$e,$s,$t|['_']]. > [116,101,115,116,'_'] > 2> "test"++['_']=L. > [116,101,115,116,'_'] > > But I'm pretty sure I am not understanding your question exactly. > > Robby > > PS Interesting, why does this work: > 3> "test"++'_'. > [116,101,115,116|'_'] > ??? > > On Wed, Aug 25, 2010 at 1:17 PM, Musumeci, Antonio S < Antonio.Musumeci@REDACTED> wrote: > >> Understood. Is there no way to turn "test" ++ '_' into >> [$t,$e,$s,$t|'_'] from a string for use in a matchspec? >> >> -----Original Message----- >> From: Robert Virding [mailto:rvirding@REDACTED] >> Sent: Tuesday, August 24, 2010 5:01 PM >> To: Musumeci, Antonio S (Enterprise Infrastructure) >> Cc: erlang-questions@REDACTED >> Subject: Re: [erlang-questions] Bug, feature or limitation of erl_parse? >> >> Erl_parse does recognise '"test" ++ Else', but it is not a term it is >> an expression. It is later inside the compiler that the expression is >> converted into it's shortcut form. The parser just parses the >> expression. When you did it by hand, so to speak, you got a term which >> erl_parse could parse as a term. So: >> >> 17> f(S), {ok,S,_} = erl_scan:string("\"test\" ++ Else. "). >> {ok,[{string,1,"test"},{'++',1},{var,1,'Else'},{dot,1}],1} >> 18> erl_parse:parse_exprs(S). >> {ok,[{op,1,'++',{string,1,"test"},{var,1,'Else'}}]} >> >> N.B. it it not the string "term" which causes it to be an expression >> but the operator '++'. >> >> Robert >> >> On 24 August 2010 22:08, Musumeci, Antonio S < >> Antonio.Musumeci@REDACTED> wrote: >> > It doesn't appear to understand the "test" ++ Else shortcut for >> [$t,$e,$s,$t|Else]. Reason I noticed was I'm reading in terms as >> strings and passing them to mnesia:select. erl_parse works fine with >> [$t|'_'] but not "t" ++ '_'. >> > >> >> S = "[$t,$e,$s,$t|'_']". >> > "[$t,$e,$s,$t|'_']" >> >> T = "\"test\" ++ '_'". >> > "\"test\" ++ '_'" >> >> [$t,$e,$s,$t|'_'] = "test" ++ '_'. >> > [116,101,115,116|'_'] >> >> {ok,S2,_} = erl_scan:string(S). >> > {ok,[{'[',1}, >> > ? ? {char,1,116}, >> > ? ? {',',1}, >> > ? ? {char,1,101}, >> > ? ? {',',1}, >> > ? ? {char,1,115}, >> > ? ? {',',1}, >> > ? ? {char,1,116}, >> > ? ? {'|',1}, >> > ? ? {atom,1,'_'}, >> > ? ? {']',1}], >> > ? ?1} >> >> {ok,T2,_} = erl_scan:string(T). >> > {ok,[{string,1,"test"},{'++',1},{atom,1,'_'}],1} >> >> erl_parse:parse_term(S2++[{dot,1}]). >> > {ok,[116,101,115,116|'_']} >> >> erl_parse:parse_term(T2++[{dot,1}]). >> > {error,{1,erl_parse,"bad term"}} >> > >> > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From rtrlists@REDACTED Wed Aug 25 16:11:14 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 25 Aug 2010 15:11:14 +0100 Subject: [erlang-questions] Bug, feature or limitation of erl_parse? In-Reply-To: References: <01489E237C2C8F45AF7898E135F48E8019A3B1CE74@NYWEXMBX2129.msad.ms.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CEC3@NYWEXMBX2129.msad.ms.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CEDF@NYWEXMBX2129.msad.ms.com> Message-ID: I wonder if the problem is simply that [$t|'_'] is not actually a properly formed list! These are proper: [$t,'_'] [$t|['_']] [$t,'_'|[]] And "t"++'_' creates a improper list instead of an exception, and that seems to confuse things a wee bit. This is most likely due to the fact that ++ will be implemented by simply stuffing whatever it finds on its right hand side into the tail of the left hand side; and if that ain't no list, your result is an improper list. Robby On Wed, Aug 25, 2010 at 2:38 PM, Hynek Vychodil wrote: > What is problem? > > 1> L = "test". > "test" > 2> Match = L ++ '_'. > [116,101,115,116|'_'] > > > On Wed, Aug 25, 2010 at 3:22 PM, Musumeci, Antonio S > wrote: > > http://www.erlang.org/doc/apps/erts/match_spec.html > > > > It shows that [$t|'_'] is a proper match pattern... And it does work. > Since otherwise "t" ++ Else is a shortcut for [$t|Else] I was trying to find > out why I appear unable to parse a string of the former to use in a > matchspec as the latter one does work. > > > > -----Original Message----- > > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] > On Behalf Of Robert Raschke > > Sent: Wednesday, August 25, 2010 8:52 AM > > To: erlang-questions@REDACTED > > Subject: Re: [erlang-questions] Bug, feature or limitation of erl_parse? > > > > Hmm, [$t,$e,$s,$t|'_'] is not a proper list. > > Do you mean [$t,$e,$s,$t,'_'] (or the equivalent [$t,$e,$s,$t|['_']])? > > > > If yes, then "test"++['_'] will do the match: > > > > Eshell V5.6.5 (abort with ^G) > > 1> L=[$t,$e,$s,$t|['_']]. > > [116,101,115,116,'_'] > > 2> "test"++['_']=L. > > [116,101,115,116,'_'] > > > > But I'm pretty sure I am not understanding your question exactly. > > > > Robby > > > > PS Interesting, why does this work: > > 3> "test"++'_'. > > [116,101,115,116|'_'] > > ??? > > > > On Wed, Aug 25, 2010 at 1:17 PM, Musumeci, Antonio S < > Antonio.Musumeci@REDACTED> wrote: > > > >> Understood. Is there no way to turn "test" ++ '_' into > >> [$t,$e,$s,$t|'_'] from a string for use in a matchspec? > >> > >> -----Original Message----- > >> From: Robert Virding [mailto:rvirding@REDACTED] > >> Sent: Tuesday, August 24, 2010 5:01 PM > >> To: Musumeci, Antonio S (Enterprise Infrastructure) > >> Cc: erlang-questions@REDACTED > >> Subject: Re: [erlang-questions] Bug, feature or limitation of erl_parse? > >> > >> Erl_parse does recognise '"test" ++ Else', but it is not a term it is > >> an expression. It is later inside the compiler that the expression is > >> converted into it's shortcut form. The parser just parses the > >> expression. When you did it by hand, so to speak, you got a term which > >> erl_parse could parse as a term. So: > >> > >> 17> f(S), {ok,S,_} = erl_scan:string("\"test\" ++ Else. "). > >> {ok,[{string,1,"test"},{'++',1},{var,1,'Else'},{dot,1}],1} > >> 18> erl_parse:parse_exprs(S). > >> {ok,[{op,1,'++',{string,1,"test"},{var,1,'Else'}}]} > >> > >> N.B. it it not the string "term" which causes it to be an expression > >> but the operator '++'. > >> > >> Robert > >> > >> On 24 August 2010 22:08, Musumeci, Antonio S < > >> Antonio.Musumeci@REDACTED> wrote: > >> > It doesn't appear to understand the "test" ++ Else shortcut for > >> [$t,$e,$s,$t|Else]. Reason I noticed was I'm reading in terms as > >> strings and passing them to mnesia:select. erl_parse works fine with > >> [$t|'_'] but not "t" ++ '_'. > >> > > >> >> S = "[$t,$e,$s,$t|'_']". > >> > "[$t,$e,$s,$t|'_']" > >> >> T = "\"test\" ++ '_'". > >> > "\"test\" ++ '_'" > >> >> [$t,$e,$s,$t|'_'] = "test" ++ '_'. > >> > [116,101,115,116|'_'] > >> >> {ok,S2,_} = erl_scan:string(S). > >> > {ok,[{'[',1}, > >> > {char,1,116}, > >> > {',',1}, > >> > {char,1,101}, > >> > {',',1}, > >> > {char,1,115}, > >> > {',',1}, > >> > {char,1,116}, > >> > {'|',1}, > >> > {atom,1,'_'}, > >> > {']',1}], > >> > 1} > >> >> {ok,T2,_} = erl_scan:string(T). > >> > {ok,[{string,1,"test"},{'++',1},{atom,1,'_'}],1} > >> >> erl_parse:parse_term(S2++[{dot,1}]). > >> > {ok,[116,101,115,116|'_']} > >> >> erl_parse:parse_term(T2++[{dot,1}]). > >> > {error,{1,erl_parse,"bad term"}} > >> > > >> > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try GoodData now for free: www.gooddata.com > From hynek@REDACTED Wed Aug 25 17:10:51 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Wed, 25 Aug 2010 17:10:51 +0200 Subject: [erlang-questions] Bug, feature or limitation of erl_parse? In-Reply-To: <20100825104754.0ef62600@otis> References: <01489E237C2C8F45AF7898E135F48E8019A3B1CE74@NYWEXMBX2129.msad.ms.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CEC3@NYWEXMBX2129.msad.ms.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CEDF@NYWEXMBX2129.msad.ms.com> <20100825104754.0ef62600@otis> Message-ID: I still don't understand. 1> T = ets:new(foo,[private]). 20496 2> ets:insert(T, [{"foo", 1}, {"foobar", 2}, {"bar", 3}, {"foobaz", 4}]). true 3> Prefix = "foo". "foo" 4> catch ets:match(T,{Prefix++'$1', '$2'}). [["bar",2],[[],1],["baz",4]] 5> catch ets:select(T,[{{Prefix++'_', '_'}, [], ['$_']}]). [{"foobar",2},{"foo",1},{"foobaz",4}] On Wed, Aug 25, 2010 at 4:47 PM, wrote: > Try that using erl_scan and erl_parse. That's what doesn't work and > that's what I'm trying to solve. I am reading in strings of erlang > terms for running selects on mnesia. "prefix" ++ '_' is easier for my > customer to understand and deal with than [$p,$r,$e,$f,$i,$x|'_']. I > can make a preprocessor which does the substitution but I wanted to > know if I was doing something wrong. > > On Wed, 25 Aug 2010 15:38:21 +0200 > Hynek Vychodil wrote: > >> What is problem? >> >> 1> L = "test". >> "test" >> 2> Match = L ++ '_'. >> [116,101,115,116|'_'] >> >> >> On Wed, Aug 25, 2010 at 3:22 PM, Musumeci, Antonio S >> wrote: >> > http://www.erlang.org/doc/apps/erts/match_spec.html >> > >> > It shows that [$t|'_'] is a proper match pattern... And it does >> > work. Since otherwise "t" ++ Else is a shortcut for [$t|Else] I was >> > trying to find out why I appear unable to parse a string of the >> > former to use in a matchspec as the latter one does work. >> > >> > -----Original Message----- >> > From: erlang-questions@REDACTED >> > [mailto:erlang-questions@REDACTED] On Behalf Of Robert Raschke >> > Sent: Wednesday, August 25, 2010 8:52 AM To: >> > erlang-questions@REDACTED Subject: Re: [erlang-questions] Bug, >> > feature or limitation of erl_parse? >> > >> > Hmm, [$t,$e,$s,$t|'_'] is not a proper list. >> > Do you mean [$t,$e,$s,$t,'_'] (or the equivalent >> > [$t,$e,$s,$t|['_']])? >> > >> > If yes, then "test"++['_'] will do the match: >> > >> > Eshell V5.6.5 ?(abort with ^G) >> > 1> L=[$t,$e,$s,$t|['_']]. >> > [116,101,115,116,'_'] >> > 2> "test"++['_']=L. >> > [116,101,115,116,'_'] >> > >> > But I'm pretty sure I am not understanding your question exactly. >> > >> > Robby >> > >> > PS Interesting, why does this work: >> > 3> "test"++'_'. >> > [116,101,115,116|'_'] >> > ??? >> > >> > On Wed, Aug 25, 2010 at 1:17 PM, Musumeci, Antonio S < >> > Antonio.Musumeci@REDACTED> wrote: >> > >> >> Understood. Is there no way to turn "test" ++ '_' into >> >> [$t,$e,$s,$t|'_'] from a string for use in a matchspec? >> >> >> >> -----Original Message----- >> >> From: Robert Virding [mailto:rvirding@REDACTED] >> >> Sent: Tuesday, August 24, 2010 5:01 PM >> >> To: Musumeci, Antonio S (Enterprise Infrastructure) >> >> Cc: erlang-questions@REDACTED >> >> Subject: Re: [erlang-questions] Bug, feature or limitation of >> >> erl_parse? >> >> >> >> Erl_parse does recognise '"test" ++ Else', but it is not a term it >> >> is an expression. It is later inside the compiler that the >> >> expression is converted into it's shortcut form. The parser just >> >> parses the expression. When you did it by hand, so to speak, you >> >> got a term which erl_parse could parse as a term. So: >> >> >> >> 17> f(S), {ok,S,_} = erl_scan:string("\"test\" ++ Else. "). >> >> {ok,[{string,1,"test"},{'++',1},{var,1,'Else'},{dot,1}],1} >> >> 18> erl_parse:parse_exprs(S). >> >> {ok,[{op,1,'++',{string,1,"test"},{var,1,'Else'}}]} >> >> >> >> N.B. it it not the string "term" which causes it to be an >> >> expression but the operator '++'. >> >> >> >> Robert >> >> >> >> On 24 August 2010 22:08, Musumeci, Antonio S < >> >> Antonio.Musumeci@REDACTED> wrote: >> >> > It doesn't appear to understand the "test" ++ Else shortcut for >> >> [$t,$e,$s,$t|Else]. Reason I noticed was I'm reading in terms as >> >> strings and passing them to mnesia:select. erl_parse works fine >> >> with [$t|'_'] but not "t" ++ '_'. >> >> > >> >> >> S = "[$t,$e,$s,$t|'_']". >> >> > "[$t,$e,$s,$t|'_']" >> >> >> T = "\"test\" ++ '_'". >> >> > "\"test\" ++ '_'" >> >> >> [$t,$e,$s,$t|'_'] = "test" ++ '_'. >> >> > [116,101,115,116|'_'] >> >> >> {ok,S2,_} = erl_scan:string(S). >> >> > {ok,[{'[',1}, >> >> > ? ? {char,1,116}, >> >> > ? ? {',',1}, >> >> > ? ? {char,1,101}, >> >> > ? ? {',',1}, >> >> > ? ? {char,1,115}, >> >> > ? ? {',',1}, >> >> > ? ? {char,1,116}, >> >> > ? ? {'|',1}, >> >> > ? ? {atom,1,'_'}, >> >> > ? ? {']',1}], >> >> > ? ?1} >> >> >> {ok,T2,_} = erl_scan:string(T). >> >> > {ok,[{string,1,"test"},{'++',1},{atom,1,'_'}],1} >> >> >> erl_parse:parse_term(S2++[{dot,1}]). >> >> > {ok,[116,101,115,116|'_']} >> >> >> erl_parse:parse_term(T2++[{dot,1}]). >> >> > {error,{1,erl_parse,"bad term"}} >> >> > >> >> >> > ________________________________________________________________ >> > erlang-questions (at) erlang.org mailing list. >> > See http://www.erlang.org/faq.html >> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > >> > >> >> >> > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From watson.timothy@REDACTED Wed Aug 25 17:12:13 2010 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 25 Aug 2010 16:12:13 +0100 Subject: [erlang-questions] io_lib and unicode post processing? In-Reply-To: <01489E237C2C8F45AF7898E135F48E8019A3B1CD66@NYWEXMBX2129.msad.ms.com> References: <01489E237C2C8F45AF7898E135F48E8019A3B1CD47@NYWEXMBX2129.msad.ms.com> <69AB868D-0BFD-4DD9-A565-2D6D705D88CB@gmail.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CD5F@NYWEXMBX2129.msad.ms.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CD66@NYWEXMBX2129.msad.ms.com> Message-ID: On 23 August 2010 20:02, Musumeci, Antonio S wrote: > The API for webmachine expects a callback for a particular content type. What I mean is that my function is called and I return a list or binary of the data and then it's no longer in my control. Whatever is happening is behind the scenes. > Did you get any further with this? Poking around in the webmachine source, there aren't any obvious uses, but mochiweb_response flushes data with `send([io_lib:format("~.16b\r\n", [Length]), Data, <<"\r\n">>])` in write_chunk/1 and of course mochijson uses io_lib and unicode heavily. The only place webmachine seems to be using mochijson internally is in the wmtrace resource. From rvirding@REDACTED Wed Aug 25 17:35:48 2010 From: rvirding@REDACTED (Robert Virding) Date: Wed, 25 Aug 2010 17:35:48 +0200 Subject: [erlang-questions] Bug, feature or limitation of erl_parse? In-Reply-To: References: <01489E237C2C8F45AF7898E135F48E8019A3B1CE74@NYWEXMBX2129.msad.ms.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CEC3@NYWEXMBX2129.msad.ms.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CEDF@NYWEXMBX2129.msad.ms.com> <20100825104754.0ef62600@otis> Message-ID: It is a problem of *where* the conversion of "test" ++ Else to [$t,$e,$s,$t|Else] is done. It has nothing to do with whether the list is a proper list or not. The parser just parses the input and doesn't not attempt any optimisation or transformation of the input. This I think is a Good Thing as if were to attempt something like this it would be trying to guess how its results were going to be used, and this never works. The parser should return the AST formed from the input, no more no less. That it works in shell is a completely different matter. After the input has been parsed it is passed to an erlang interpreter, erl_eval, which then evaluates the input. It is here where *expressions* like "foo"++'_' below are evaluated to [$f,$o,$o|'_'] and passed in to ets:match. It is also here where similar constructs in *patterns* are handled in the pattern matching so as if they were converted and the result is the same as with the compiler. It means that if you want to be able to read in terms from strings, parse them and send them to mnesia:select you will have to do it yourself. Unfortunately. It is not really difficult if you are restricted to patterns. Robert On 25 August 2010 17:10, Hynek Vychodil wrote: > I still don't understand. > > 1> T = ets:new(foo,[private]). > 20496 > 2> ets:insert(T, [{"foo", 1}, {"foobar", 2}, {"bar", 3}, {"foobaz", 4}]). > true > 3> Prefix = "foo". > "foo" > 4> catch ets:match(T,{Prefix++'$1', '$2'}). > [["bar",2],[[],1],["baz",4]] Here the expression Prefix++'$1' is evaluated to the select spec [$f,$o,$o|'$1'] by the shell evaluator so it works. Writing the same thing in compiled code basically results in the same evaluation though the compiler specially recognises this and can do some of the work at compile time. > 5> catch ets:select(T,[{{Prefix++'_', '_'}, [], ['$_']}]). > [{"foobar",2},{"foo",1},{"foobaz",4}] Same as previous expression. > On Wed, Aug 25, 2010 at 4:47 PM, ? wrote: >> Try that using erl_scan and erl_parse. That's what doesn't work and >> that's what I'm trying to solve. I am reading in strings of erlang >> terms for running selects on mnesia. "prefix" ++ '_' is easier for my >> customer to understand and deal with than [$p,$r,$e,$f,$i,$x|'_']. I >> can make a preprocessor which does the substitution but I wanted to >> know if I was doing something wrong. >> >> On Wed, 25 Aug 2010 15:38:21 +0200 >> Hynek Vychodil wrote: >> >>> What is problem? >>> >>> 1> L = "test". >>> "test" >>> 2> Match = L ++ '_'. >>> [116,101,115,116|'_'] >>> >>> >>> On Wed, Aug 25, 2010 at 3:22 PM, Musumeci, Antonio S >>> wrote: >>> > http://www.erlang.org/doc/apps/erts/match_spec.html >>> > >>> > It shows that [$t|'_'] is a proper match pattern... And it does >>> > work. Since otherwise "t" ++ Else is a shortcut for [$t|Else] I was >>> > trying to find out why I appear unable to parse a string of the >>> > former to use in a matchspec as the latter one does work. >>> > >>> > -----Original Message----- >>> > From: erlang-questions@REDACTED >>> > [mailto:erlang-questions@REDACTED] On Behalf Of Robert Raschke >>> > Sent: Wednesday, August 25, 2010 8:52 AM To: >>> > erlang-questions@REDACTED Subject: Re: [erlang-questions] Bug, >>> > feature or limitation of erl_parse? >>> > >>> > Hmm, [$t,$e,$s,$t|'_'] is not a proper list. >>> > Do you mean [$t,$e,$s,$t,'_'] (or the equivalent >>> > [$t,$e,$s,$t|['_']])? >>> > >>> > If yes, then "test"++['_'] will do the match: >>> > >>> > Eshell V5.6.5 ?(abort with ^G) >>> > 1> L=[$t,$e,$s,$t|['_']]. >>> > [116,101,115,116,'_'] >>> > 2> "test"++['_']=L. >>> > [116,101,115,116,'_'] >>> > >>> > But I'm pretty sure I am not understanding your question exactly. >>> > >>> > Robby >>> > >>> > PS Interesting, why does this work: >>> > 3> "test"++'_'. >>> > [116,101,115,116|'_'] >>> > ??? >>> > >>> > On Wed, Aug 25, 2010 at 1:17 PM, Musumeci, Antonio S < >>> > Antonio.Musumeci@REDACTED> wrote: >>> > >>> >> Understood. Is there no way to turn "test" ++ '_' into >>> >> [$t,$e,$s,$t|'_'] from a string for use in a matchspec? >>> >> >>> >> -----Original Message----- >>> >> From: Robert Virding [mailto:rvirding@REDACTED] >>> >> Sent: Tuesday, August 24, 2010 5:01 PM >>> >> To: Musumeci, Antonio S (Enterprise Infrastructure) >>> >> Cc: erlang-questions@REDACTED >>> >> Subject: Re: [erlang-questions] Bug, feature or limitation of >>> >> erl_parse? >>> >> >>> >> Erl_parse does recognise '"test" ++ Else', but it is not a term it >>> >> is an expression. It is later inside the compiler that the >>> >> expression is converted into it's shortcut form. The parser just >>> >> parses the expression. When you did it by hand, so to speak, you >>> >> got a term which erl_parse could parse as a term. So: >>> >> >>> >> 17> f(S), {ok,S,_} = erl_scan:string("\"test\" ++ Else. "). >>> >> {ok,[{string,1,"test"},{'++',1},{var,1,'Else'},{dot,1}],1} >>> >> 18> erl_parse:parse_exprs(S). >>> >> {ok,[{op,1,'++',{string,1,"test"},{var,1,'Else'}}]} >>> >> >>> >> N.B. it it not the string "term" which causes it to be an >>> >> expression but the operator '++'. >>> >> >>> >> Robert >>> >> >>> >> On 24 August 2010 22:08, Musumeci, Antonio S < >>> >> Antonio.Musumeci@REDACTED> wrote: >>> >> > It doesn't appear to understand the "test" ++ Else shortcut for >>> >> [$t,$e,$s,$t|Else]. Reason I noticed was I'm reading in terms as >>> >> strings and passing them to mnesia:select. erl_parse works fine >>> >> with [$t|'_'] but not "t" ++ '_'. >>> >> > >>> >> >> S = "[$t,$e,$s,$t|'_']". >>> >> > "[$t,$e,$s,$t|'_']" >>> >> >> T = "\"test\" ++ '_'". >>> >> > "\"test\" ++ '_'" >>> >> >> [$t,$e,$s,$t|'_'] = "test" ++ '_'. >>> >> > [116,101,115,116|'_'] >>> >> >> {ok,S2,_} = erl_scan:string(S). >>> >> > {ok,[{'[',1}, >>> >> > ? ? {char,1,116}, >>> >> > ? ? {',',1}, >>> >> > ? ? {char,1,101}, >>> >> > ? ? {',',1}, >>> >> > ? ? {char,1,115}, >>> >> > ? ? {',',1}, >>> >> > ? ? {char,1,116}, >>> >> > ? ? {'|',1}, >>> >> > ? ? {atom,1,'_'}, >>> >> > ? ? {']',1}], >>> >> > ? ?1} >>> >> >> {ok,T2,_} = erl_scan:string(T). >>> >> > {ok,[{string,1,"test"},{'++',1},{atom,1,'_'}],1} >>> >> >> erl_parse:parse_term(S2++[{dot,1}]). >>> >> > {ok,[116,101,115,116|'_']} >>> >> >> erl_parse:parse_term(T2++[{dot,1}]). >>> >> > {error,{1,erl_parse,"bad term"}} >>> >> > >>> >> >>> > ________________________________________________________________ >>> > erlang-questions (at) erlang.org mailing list. >>> > See http://www.erlang.org/faq.html >>> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> > >>> > >>> >>> >>> >> >> > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss.? Be a data hero! > Try GoodData now for free: www.gooddata.com > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From bile@REDACTED Wed Aug 25 16:47:54 2010 From: bile@REDACTED (bile@REDACTED) Date: Wed, 25 Aug 2010 10:47:54 -0400 Subject: [erlang-questions] Bug, feature or limitation of erl_parse? In-Reply-To: References: <01489E237C2C8F45AF7898E135F48E8019A3B1CE74@NYWEXMBX2129.msad.ms.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CEC3@NYWEXMBX2129.msad.ms.com> <01489E237C2C8F45AF7898E135F48E8019A3B1CEDF@NYWEXMBX2129.msad.ms.com> Message-ID: <20100825104754.0ef62600@otis> Try that using erl_scan and erl_parse. That's what doesn't work and that's what I'm trying to solve. I am reading in strings of erlang terms for running selects on mnesia. "prefix" ++ '_' is easier for my customer to understand and deal with than [$p,$r,$e,$f,$i,$x|'_']. I can make a preprocessor which does the substitution but I wanted to know if I was doing something wrong. On Wed, 25 Aug 2010 15:38:21 +0200 Hynek Vychodil wrote: > What is problem? > > 1> L = "test". > "test" > 2> Match = L ++ '_'. > [116,101,115,116|'_'] > > > On Wed, Aug 25, 2010 at 3:22 PM, Musumeci, Antonio S > wrote: > > http://www.erlang.org/doc/apps/erts/match_spec.html > > > > It shows that [$t|'_'] is a proper match pattern... And it does > > work. Since otherwise "t" ++ Else is a shortcut for [$t|Else] I was > > trying to find out why I appear unable to parse a string of the > > former to use in a matchspec as the latter one does work. > > > > -----Original Message----- > > From: erlang-questions@REDACTED > > [mailto:erlang-questions@REDACTED] On Behalf Of Robert Raschke > > Sent: Wednesday, August 25, 2010 8:52 AM To: > > erlang-questions@REDACTED Subject: Re: [erlang-questions] Bug, > > feature or limitation of erl_parse? > > > > Hmm, [$t,$e,$s,$t|'_'] is not a proper list. > > Do you mean [$t,$e,$s,$t,'_'] (or the equivalent > > [$t,$e,$s,$t|['_']])? > > > > If yes, then "test"++['_'] will do the match: > > > > Eshell V5.6.5 ?(abort with ^G) > > 1> L=[$t,$e,$s,$t|['_']]. > > [116,101,115,116,'_'] > > 2> "test"++['_']=L. > > [116,101,115,116,'_'] > > > > But I'm pretty sure I am not understanding your question exactly. > > > > Robby > > > > PS Interesting, why does this work: > > 3> "test"++'_'. > > [116,101,115,116|'_'] > > ??? > > > > On Wed, Aug 25, 2010 at 1:17 PM, Musumeci, Antonio S < > > Antonio.Musumeci@REDACTED> wrote: > > > >> Understood. Is there no way to turn "test" ++ '_' into > >> [$t,$e,$s,$t|'_'] from a string for use in a matchspec? > >> > >> -----Original Message----- > >> From: Robert Virding [mailto:rvirding@REDACTED] > >> Sent: Tuesday, August 24, 2010 5:01 PM > >> To: Musumeci, Antonio S (Enterprise Infrastructure) > >> Cc: erlang-questions@REDACTED > >> Subject: Re: [erlang-questions] Bug, feature or limitation of > >> erl_parse? > >> > >> Erl_parse does recognise '"test" ++ Else', but it is not a term it > >> is an expression. It is later inside the compiler that the > >> expression is converted into it's shortcut form. The parser just > >> parses the expression. When you did it by hand, so to speak, you > >> got a term which erl_parse could parse as a term. So: > >> > >> 17> f(S), {ok,S,_} = erl_scan:string("\"test\" ++ Else. "). > >> {ok,[{string,1,"test"},{'++',1},{var,1,'Else'},{dot,1}],1} > >> 18> erl_parse:parse_exprs(S). > >> {ok,[{op,1,'++',{string,1,"test"},{var,1,'Else'}}]} > >> > >> N.B. it it not the string "term" which causes it to be an > >> expression but the operator '++'. > >> > >> Robert > >> > >> On 24 August 2010 22:08, Musumeci, Antonio S < > >> Antonio.Musumeci@REDACTED> wrote: > >> > It doesn't appear to understand the "test" ++ Else shortcut for > >> [$t,$e,$s,$t|Else]. Reason I noticed was I'm reading in terms as > >> strings and passing them to mnesia:select. erl_parse works fine > >> with [$t|'_'] but not "t" ++ '_'. > >> > > >> >> S = "[$t,$e,$s,$t|'_']". > >> > "[$t,$e,$s,$t|'_']" > >> >> T = "\"test\" ++ '_'". > >> > "\"test\" ++ '_'" > >> >> [$t,$e,$s,$t|'_'] = "test" ++ '_'. > >> > [116,101,115,116|'_'] > >> >> {ok,S2,_} = erl_scan:string(S). > >> > {ok,[{'[',1}, > >> > ? ? {char,1,116}, > >> > ? ? {',',1}, > >> > ? ? {char,1,101}, > >> > ? ? {',',1}, > >> > ? ? {char,1,115}, > >> > ? ? {',',1}, > >> > ? ? {char,1,116}, > >> > ? ? {'|',1}, > >> > ? ? {atom,1,'_'}, > >> > ? ? {']',1}], > >> > ? ?1} > >> >> {ok,T2,_} = erl_scan:string(T). > >> > {ok,[{string,1,"test"},{'++',1},{atom,1,'_'}],1} > >> >> erl_parse:parse_term(S2++[{dot,1}]). > >> > {ok,[116,101,115,116|'_']} > >> >> erl_parse:parse_term(T2++[{dot,1}]). > >> > {error,{1,erl_parse,"bad term"}} > >> > > >> > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > From sg2342@REDACTED Wed Aug 25 21:23:43 2010 From: sg2342@REDACTED (Stefan Grundmann) Date: Wed, 25 Aug 2010 19:23:43 +0000 Subject: ssh application leaks processes(?) Message-ID: <201008251923.43899.sg2342@googlemail.com> Hi, while using the ssh-2.0 application i encountered the following problem: The ssh_system_sup supervisor (created via ssh:daemon/1,2,3) has as children the ssh_acceptor_sup supervisor and for every connected client an ssh_subsystem_sup supervisor. When a client connection is closed one would expect that all processes in the supervision tree of this connection die - which is not the case: the ssh_subsystem_sup process and one child (ssh_channel_sup) stay alive. The reason for this behavior is that the child specs of the ssh_channel_sup and ssh_connection_sup in lib/ssh-2.0/src/ssh_subsystem_sup,erl (line 75 and line 88) define a 'transient' Restart and while the subtree supervised by ssh_connection_sup as well as the ssh_connection_sup process itself terminates on connection close, the ssh_channel_sup process does not. Which leaves 2 processes alive that won't be of any further use. If the Restart is changed to 'permanent' the problem goes away. Since lib/ssh-2.0/src/ssh_subsystem_sup,erl contains (line 76 and line 89) commented out code that sets Restart to permanent; i would like to know what the reason for a transient Restart might be. best regards Stefan Grundmann From attila.r.nohl@REDACTED Wed Aug 25 23:48:49 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 25 Aug 2010 23:48:49 +0200 Subject: [erlang-questions] ssh application leaks processes(?) In-Reply-To: <201008251923.43899.sg2342@googlemail.com> References: <201008251923.43899.sg2342@googlemail.com> Message-ID: I've patched our local installation about a year ago to use 'permanent'. It works fine. 2010/8/25, Stefan Grundmann : > Hi, > while using the ssh-2.0 application i encountered the following problem: > > The ssh_system_sup supervisor (created via ssh:daemon/1,2,3) has as > children > the ssh_acceptor_sup supervisor and for every connected client an > ssh_subsystem_sup supervisor. When a client connection is closed one would > expect that all processes in the supervision tree of this connection die - > which is not the case: the ssh_subsystem_sup process and one child > (ssh_channel_sup) stay alive. > > The reason for this behavior is that the child specs of the ssh_channel_sup > and ssh_connection_sup in lib/ssh-2.0/src/ssh_subsystem_sup,erl (line 75 and > line 88) define a 'transient' Restart and while the subtree supervised by > ssh_connection_sup as well as the ssh_connection_sup process itself > terminates > on connection close, the ssh_channel_sup process does not. Which leaves 2 > processes alive that won't be of any further use. > > If the Restart is changed to 'permanent' the problem goes away. > > Since lib/ssh-2.0/src/ssh_subsystem_sup,erl contains (line 76 and line 89) > commented out code that sets Restart to permanent; i would like to know what > the reason for a transient Restart might be. > > best regards > Stefan Grundmann > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From roberto@REDACTED Thu Aug 26 13:03:33 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 26 Aug 2010 13:03:33 +0200 Subject: sine and cosine with degree angles Message-ID: dear list, sorry if this does sound stupid but i need to compute sine and cosine of angles expressed in degrees. the math:sin/1 and math:cos/1 functions compute respectively the sine and cosine of radiant angles, which can be converted to degrees simply like this: radians = degrees * ? / 180 however, due to approximations there seem to be an error in the computation of the cosine of 90? [which should be 0]: 1> math:sin(90 * math:pi() / 180). 1.0 2> math:sin(0 * math:pi() / 180). 0.0 3> math:cos(90 * math:pi() / 180). 6.123233995736766e-17 4> math:cos(0 * math:pi() / 180). 1.0 any suggestions on how to proceed? am i missing some erlang functions on this? thank you, r. From olopierpa@REDACTED Thu Aug 26 13:24:30 2010 From: olopierpa@REDACTED (Pierpaolo Bernardi) Date: Thu, 26 Aug 2010 13:24:30 +0200 Subject: [erlang-questions] sine and cosine with degree angles In-Reply-To: References: Message-ID: On Thu, Aug 26, 2010 at 13:03, Roberto Ostinelli wrote: > any suggestions on how to proceed? Just go on. Why is this a problem? P. From roberto@REDACTED Thu Aug 26 13:32:07 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 26 Aug 2010 13:32:07 +0200 Subject: [erlang-questions] sine and cosine with degree angles In-Reply-To: References: Message-ID: 2010/8/26 Pierpaolo Bernardi : >> any suggestions on how to proceed? > > Just go on. ?Why is this a problem? ..because 0 =/= 6.123233995736766e-17 ? i use sine/cosine to compute distances of particles based on x,y,z angles. r. From rumata-estor@REDACTED Thu Aug 26 13:36:49 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Thu, 26 Aug 2010 15:36:49 +0400 Subject: [erlang-questions] sine and cosine with degree angles In-Reply-To: References: Message-ID: <4C7651D1.6020401@nm.ru> Suddenly 1> math:sin(math:pi()). 1.2246063538223773e-16 It's floating point, it's always not completely accurate. Check for epsilon range: -define(EPS, 1.0e-10). if -?EPS < Value andalso Value < ?EPS -> 0; _ -> Value end. Dmitry Belyaev On 08/26/2010 03:03 PM, Roberto Ostinelli wrote: > dear list, > > sorry if this does sound stupid but i need to compute sine and cosine > of angles expressed in degrees. > > the math:sin/1 and math:cos/1 functions compute respectively the sine > and cosine of radiant angles, which can be converted to degrees simply > like this: > > radians = degrees * ? / 180 > > however, due to approximations there seem to be an error in the > computation of the cosine of 90? [which should be 0]: > > 1> math:sin(90 * math:pi() / 180). > 1.0 > 2> math:sin(0 * math:pi() / 180). > 0.0 > 3> math:cos(90 * math:pi() / 180). > 6.123233995736766e-17 > 4> math:cos(0 * math:pi() / 180). > 1.0 > > any suggestions on how to proceed? am i missing some erlang functions on this? > > thank you, > > r. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From olopierpa@REDACTED Thu Aug 26 13:42:03 2010 From: olopierpa@REDACTED (Pierpaolo Bernardi) Date: Thu, 26 Aug 2010 13:42:03 +0200 Subject: [erlang-questions] sine and cosine with degree angles In-Reply-To: References: Message-ID: On Thu, Aug 26, 2010 at 13:32, Roberto Ostinelli wrote: > 2010/8/26 Pierpaolo Bernardi : >>> any suggestions on how to proceed? >> >> Just go on. ?Why is this a problem? > > ..because 0 =/= 6.123233995736766e-17 ? > > i use sine/cosine to compute distances of particles based on x,y,z angles. the answer is close enough for any practical purpose, and the same number will be computed by any programming language using ieee754 double precision, it's not a problem of erlang. From roberto@REDACTED Thu Aug 26 13:57:25 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 26 Aug 2010 13:57:25 +0200 Subject: [erlang-questions] sine and cosine with degree angles In-Reply-To: <4C7651D1.6020401@nm.ru> References: <4C7651D1.6020401@nm.ru> Message-ID: 2010/8/26 Dmitry Belyaev : > Suddenly > 1> math:sin(math:pi()). > 1.2246063538223773e-16 > > It's floating point, it's always not completely accurate. > > Check for epsilon range: > -define(EPS, 1.0e-10). > ?if -?EPS < Value andalso Value < ?EPS -> 0; _ -> Value end. > > Dmitry Belyaev i know it's about approximations. this is why i was asking if i'm missing an erlang function which takes degrees instead of radiants as input. thank you for your suggestion, i will use the EPS. cheers :) r. From rumata-estor@REDACTED Thu Aug 26 14:09:41 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Thu, 26 Aug 2010 16:09:41 +0400 Subject: [erlang-questions] sine and cosine with degree angles In-Reply-To: References: <4C7651D1.6020401@nm.ru> Message-ID: <4C765985.8060808@nm.ru> I just wanted to show that the problem is not with degrees but with real numbers. However, you can make your own sin/cos for _integer_ degrees: cos(0) -> 1; cos(90) -> 0; cos(-90) -> 0; cos(180) -> -1; cos(-180) -> -1; cos(Deg) when Deg >= 180 orelse Deg <= -180 -> cos(Deg rem 360); cos(Deg) -> math:cos(90 * math:pi() / 180). Dmitry Belyaev On 08/26/2010 03:57 PM, Roberto Ostinelli wrote: > 2010/8/26 Dmitry Belyaev: > >> Suddenly >> 1> math:sin(math:pi()). >> 1.2246063538223773e-16 >> >> It's floating point, it's always not completely accurate. >> >> Check for epsilon range: >> -define(EPS, 1.0e-10). >> if -?EPS< Value andalso Value< ?EPS -> 0; _ -> Value end. >> >> Dmitry Belyaev >> > i know it's about approximations. this is why i was asking if i'm > missing an erlang function which takes degrees instead of radiants as > input. > > thank you for your suggestion, i will use the EPS. > > cheers :) > > r. > > From dmitrii@REDACTED Thu Aug 26 14:10:48 2010 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Thu, 26 Aug 2010 15:10:48 +0300 Subject: =?windows-1252?Q?ji_=97_simplify_mapping_custom_Java_objects_to_?= =?windows-1252?Q?Erlang_tuples/lists/etc.?= Message-ID: If anyon'es interested, I've put my half-baked somewhat-working library up on GitHub: http://github.com/dmitriid/ji/ This is a simple and some what dumb library to facliltate conversion between custom Java types and Erlang. [----------------code start ----------------] CoversionManager cm = new ConversionManager(); // loads basic convertors for integers, booleans, // strings and the like cm.registerBasic(); ArrayList arr = new ArrayList(); arr.add("hello"); arr.add(10); OtpErlangList list = cm.convert(arr); // if you send this list over to Erlang, you'll get // ["hello", 10] [----------------code end ----------------] A basic usage would be something like this: [----------------code start ----------------] ConversionManager cm = new ConversionManager(); cm.registerBasic(); cm.register( new MyCustomConverter1(), new MyCustomConverter2(), new MyCustomConverter3() ); OtpNode self = new OtpNode("node@REDACTED"); OtpMbox mbox = self.createMbox("mbox"); while(true) { Object o = o = mbox.receive(); Object result = do_something_horrible(o); OtpErlangObject out = cm.convert(result); mbox.send(from, out); } [----------------code start ----------------] Code is extremely simple, so writing a converter ? any type of converter ? shouldn't be a problem. It might not be as useful as it sounds, but I don't have the necessary Java skills to make it an uber-library :) From rumata-estor@REDACTED Thu Aug 26 14:16:05 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Thu, 26 Aug 2010 16:16:05 +0400 Subject: [erlang-questions] sine and cosine with degree angles In-Reply-To: <4C765985.8060808@nm.ru> References: <4C7651D1.6020401@nm.ru> <4C765985.8060808@nm.ru> Message-ID: <4C765B05.5070701@nm.ru> Shame on me, wrong guards. Hope, this one is better: cos(0) -> 1; cos(90) -> 0; cos(180) -> -1; cod(270) -> 0; cos(Deg) when Deg < 0 -> cos(-Deg); cos(Deg) when Deg >= 360 -> cos(Deg rem 360); cos(Deg) -> math:cos(90 * math:pi() / 180). Dmitry Belyaev On 08/26/2010 04:09 PM, Dmitry Belyaev wrote: > > I just wanted to show that the problem is not with degrees but with > real numbers. > However, you can make your own sin/cos for _integer_ degrees: > > cos(0) -> 1; > cos(90) -> 0; > cos(-90) -> 0; > cos(180) -> -1; > cos(-180) -> -1; > cos(Deg) when Deg >= 180 orelse Deg <= -180 -> cos(Deg rem 360); > cos(Deg) -> math:cos(90 * math:pi() / 180). > > Dmitry Belyaev > > > On 08/26/2010 03:57 PM, Roberto Ostinelli wrote: >> 2010/8/26 Dmitry Belyaev: >>> Suddenly >>> 1> math:sin(math:pi()). >>> 1.2246063538223773e-16 >>> >>> It's floating point, it's always not completely accurate. >>> >>> Check for epsilon range: >>> -define(EPS, 1.0e-10). >>> if -?EPS< Value andalso Value< ?EPS -> 0; _ -> Value end. >>> >>> Dmitry Belyaev >> i know it's about approximations. this is why i was asking if i'm >> missing an erlang function which takes degrees instead of radiants as >> input. >> >> thank you for your suggestion, i will use the EPS. >> >> cheers :) >> >> r. >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From peppe@REDACTED Thu Aug 26 16:08:01 2010 From: peppe@REDACTED (Peter Andersson) Date: Thu, 26 Aug 2010 16:08:01 +0200 Subject: [erlang-questions] what is the best way to test Erlang FSM processes? In-Reply-To: <809001.55126.qm@web25402.mail.ukl.yahoo.com> References: <809001.55126.qm@web25402.mail.ukl.yahoo.com> Message-ID: <4C767541.5060504@erix.ericsson.se> Roman Shestakov wrote: > hello, > > I have a dependency tree of connected fsm processes, the state of children fsm' > depends on state of parents and in some cases, timers attached to fsm can also > trigger state change. > > I would like to be able to test the state of entire system with Common Test but > can't figure out a good way to use CT for testing fsm states. > > does anybody have some ideas if it is possible to use CT in such scenario? any > examples? other ideas? > > Regards, Roman > Hi Roman, This is an interesting question. Actually, a while back we had some discussions about this with some people using Common Test in a project at Ericsson. They were testing event driven state machines and they were forced to implement difficult recursive test case functions with nested receive and case expressions to test these FSMs. The idea these guys had was to implement test suites as gen_fsm behaviours to test their FSMs instead. This way, the suites become much better structured, more intuitive to read, and easier to modify and extend. We've decided to implement support for this type of suite in Common Test, so that's in the pipeline. We haven't decided exactly what this support should look like, but we have ideas along the lines of: - let Common Test recognize an fsm suite and handle start/stop of the test fsm process automatically (per suite or test case) - a test is executed in the scope of a test case, as usual, and the test case function name will be a parameter in the state of the test fsm process - print info about state transitions, events and messages during a test to the test case log - indicate test case pass or fail by means of state transitions and/or reports to the test case process - provide i/f to modify the state of the test fsm from the init/end config functions We don't really add any new functionality to what the gen_fsm already provides. What we want you to be able to do, you can already do with a gen_fsm behaviour test suite today! We will let Common Test take care of common operations though, to make your test fsm implementations simpler and more compact. Also we want the users to "have to" structure the fsm suites the same way, and use predefined names for common functions/states/events. It'll be easier to share and re-use test code this way. If you'd already like to use a Common Test suite with a gen_fsm behaviour to test your FSMs, here's a very simple (and very incomplete!) example: -module(fsm_SUITE). -behaviour(gen_fsm). -export([code_change/4, handle_event/3, handle_info/3, handle_sync_event/4, init/1, terminate/3]). -include_lib("common_test/include/ct.hrl"). -define(FSM, ?MODULE). -define(TO, 5000). -record(fsm_state, {testcase, result}). %% --- Test suite callback functions --- all() -> [open_connection, ...]. suite() -> [{require,connection_data}]. init_per_testcase(TC, Config) -> {ok,_} = gen_fsm:start({local, ?FSM}, ?MODULE, [TC], []), Config. end_per_testcase(_, _Config) -> gen_fsm:sync_send_all_state_event(?FSM, stop, ?TO). open_connection(_Config) -> ok = gen_fsm:send_event(?FSM, open_connection), verify_result(open, ok). verify_result(State, Result) -> {State,Result} = gen_fsm:sync_send_all_state_event(?FSM, report, ?TO). %% --- FSM behaviour callback functions --- init([TestCase]) -> ct:log("-> Idle", []), {ok, idle, #fsm_state{testcase = TestCase}}. idle(open_connection, State) -> Result = server_utils:open(ct:get_config(connection_data)), ct:log("-> Open", []), {next_state, open, State#fsm_state{result = Result}}. open(Event, State) -> ... handle_event(_Event, StateName, State) -> {next_state, StateName, State}. handle_sync_event(report, From, StateName, State) -> {reply, {StateName,State#fsm_state.result}, StateName, State}; handle_sync_event(stop, _From, _StateName, State) -> {stop, normal, ok, State}. handle_info(_Info, StateName, State) -> {next_state, StateName, State}. terminate(_Reason, _StateName, _State) -> ok. If you have any comments or suggestions on this topic, let us know! /Peter Erlang/OTP, Ericsson AB From max.lapshin@REDACTED Thu Aug 26 16:16:51 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 26 Aug 2010 18:16:51 +0400 Subject: [erlang-questions] what is the best way to test Erlang FSM processes? In-Reply-To: <4C767541.5060504@erix.ericsson.se> References: <809001.55126.qm@web25402.mail.ukl.yahoo.com> <4C767541.5060504@erix.ericsson.se> Message-ID: If you have really clean state machine, it's state is always described by initial state and command words sequence. You may try several techniques: 1) most simple and hard to support: test each transition from any state by each possible command word. 2) fuzzy testing. Log down initial state, log stream of random input command words and perform some state cheks. If failed, write you an email with sequence of command words. Well known technique in gamedev. From dang@REDACTED Thu Aug 26 18:20:04 2010 From: dang@REDACTED (Daniel Goertzen) Date: Thu, 26 Aug 2010 11:20:04 -0500 Subject: [erlang-questions] how to design a scalable 'scheduler' app in erlang way In-Reply-To: References: Message-ID: Your question is a little vague, so I'll just share an Erlang idiom that I've learned: - Spawn a new worker process for each request and have it handle the whole request from beginning to end. Passing a partially completed request from scheduler process to worker process is slow[er]. - Use a "server" process to serialize access to the resource. - The scheduler sounds like a bottleneck. Remove if you can. Consider a worker process that runs like this: 1. Receive request from file, socket or whatever. 2. Spawn a new process, identical to this one, to handle the next request. 3. Process request, using resource as needed. 4. Store/send result. 5. Terminate. (heap gets dropped, no garbage collection needed) Does that give you some direction? Cheers, Dan. On Thu, Aug 19, 2010 at 2:53 AM, Oscar wrote: > hi all, > > Here are the scheduler's requirements: > 1. Scheduler should receive a lot of requests from other apps. > 2. Scheduler should send the received requests to 'worker' apps. > 3. Scheduler should determine when to send the received requests to > 'worker' apps. > > The strategy is: > a. A request depends on something, named *resource*. > b. When *resource* is idle, scheduler sends the request to one worker. > c. When *resource* is busy, scheduler stores the request. > > 4. Scheduler should be scalable, supporting about 1million tps. > > > I've implemented it in the shared-state way. TPS is about 100,000, and > 'lock' consumes the most CPU time. > Now, I want to implement an erlang version, but have no idea about it. > > Shall I store the resource status, pending request to ETS? Will ETS be > a bottleneck? > > > Thanks, > -Oscar > -- Daniel Goertzen ----------------- dang@REDACTED (work) daniel.goertzen@REDACTED (home) ----------------- 1 204 272 6149 (home/office) 1 204 470 8360 (mobile) ----------------- From pascalchapier@REDACTED Thu Aug 26 18:57:04 2010 From: pascalchapier@REDACTED (Pascal Chapier) Date: Thu, 26 Aug 2010 18:57:04 +0200 Subject: [erlang-questions] random lookup in ets Message-ID: Hello It is a funny usage of the previous function in ordered set! I would have bet that it produces an exception in case of non existing key (like it does with simple set). I tried it in a shell, and it works (almost as in your example). ############################## 1> T = ets:new(table,[ordered_set,public,named_table]). table 2> ets:tab2list(table). [] 3> ets:insert(table,[{1,"one"},{10,"ten"},{5,"five"},{50,"fifty"}]). true 4> ets:tab2list(table). [{1,"one"},{5,"five"},{10,"ten"},{50,"fifty"}] 5> ets:prev(table,10). 5 6> ets:prev(table,9). 5 7> ets:prev(table,1). '$end_of_table' 8> ets:prev(table,1000000). 50 9> T1 = ets:new(table1,[set,public,named_table]). table1 10> ets:insert(table1,[{1,"one"},{10,"ten"},{5,"five"},{50,"fifty"}]). true 11> ets:tab2list(table1). [{1,"one"},{10,"ten"},{50,"fifty"},{5,"five"}] 12> ets:prev(table1,10). 1 13> ets:prev(table1,1). '$end_of_table' 14> ets:prev(table1,9). ** exception error: bad argument in function ets:prev/2 called as ets:prev(table1,9) 15> ############################## This method is quite easy to implement and should be fast, but it does not guarantee that all keys have the same probability to be selected, because the interval between keys is not a constant. In my example, if you select a random value between 1 and 60 (included) you will get: '$end_of_table' -> 1.67% 1 -> 6.67% 5 -> 8.33% 10 -> 66.67% 50 -> 16.67% ?????????????????????????????? I have though about the solution with a complementary index table. I could be interesting, if the initial table is formed by {PrimKey,Value} to build 2 tables: primTable contents {PrimKey,Index,Value}, index contents {Index,PrimKey} (and is an ordered_set). then: add(NewPrimKey,NewValue) -> NewIndex = ets:info(index,size) + 1, ets:insert(index,{NewIndex,NewPrimKey}), ets:insert(primTable,{NewPrimKey,NewIndex,NewValue}). delete(PrimKey) -> [{PrimKey,Index,Value}] = ets:lookup(primTable,PrimKey), % warning iterate in the list if primTable is a bag delete(primTable,Primkey), LastIndex = ets:info(index,size), case (LastIndex == Index) of false -> [{LastIndex,LastKey}] = ets:lookup(index,LastIndex), ets:insert(index,{Index,LastKey}), [{LastKey,LastIndex,Value}] = ets:lookup(primTable,LastKey), ets:insert(primTable,{LastKey,Index,Value}); true -> ok end, delete(index,LastIndex). The order of elements in the tables changes, but it should have no impact on probabilities. In case of parallel accesses to the table, the function must be serialized, and (may be) use ets:safe_fixtable (and the code should be debbugged, I didn't try it... :o) ?????????????????????????????? The simplest solution >> "second, you can use ets:tab2list(Ets), and ets:info(Ets,size), generate >> X with random:uniform(Size): a random value between 1 and Size, and >> finally take the Xth element of the list with lists:nth(X).." can be accelerated, even if it will still be slow with big tables: Size = ets:info(Ets,size), X = random:uniform(Size), ets:lookup(get(Ets,X)). get(Ets,N) -> get(Ets,N-1,ets:first(Ets)). get(_,0,K) -> K; get(Ets,N,K) -> get(Ets,N-1,ets:next(Ets,K)). CU. From pascalchapier@REDACTED Thu Aug 26 19:07:29 2010 From: pascalchapier@REDACTED (Pascal Chapier) Date: Thu, 26 Aug 2010 19:07:29 +0200 Subject: [erlang-questions] random lookup in ets Message-ID: Offtopic ... I use firefox/hotmail/windows to send messages, I try to have a nice typo in my message, but it is a mess arriving in the forum, while it is still correct if I send the mail to myself. Any clue? Best regards Pascal From ulf.wiger@REDACTED Thu Aug 26 19:14:20 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 26 Aug 2010 19:14:20 +0200 Subject: [erlang-questions] random lookup in ets In-Reply-To: References: Message-ID: <4C76A0EC.9040300@erlang-solutions.com> On 26/08/2010 18:57, Pascal Chapier wrote: > > It is a > funny usage of the previous function in ordered set! > > I would > have bet that it produces an exception in case of > > non existing > key (like it does with simple set). But next(T,K) and prev(T,K) on an ordered set can be well defined, as "next key in T greater than K" and "next key in T smaller than K". This way, K doesn't actually have to be part of the set. In a hash table, there is no defined ordering, so the next/prev functions simply step in "slot order". What is the previous slot of a non-existing object? An option would be to return '$end_of_table', which would make that return value ambiguous - it could either mean "I've stepped to the end of the table", or "I don't know where to go". Of course, one could invent yet another return value, but that would hardly be an improvement. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From comptekki@REDACTED Thu Aug 26 19:14:44 2010 From: comptekki@REDACTED (Wes James) Date: Thu, 26 Aug 2010 11:14:44 -0600 Subject: [erlang-questions] random lookup in ets In-Reply-To: References: Message-ID: Can you try gmail or some other web email and see if it is any different? In gmail you can send email as plain text or rich text (allows some html formatting). Are you using plain text or rich text in your hotmail settings? -wes On Thu, Aug 26, 2010 at 11:07 AM, Pascal Chapier wrote: > > Offtopic ... > > I use firefox/hotmail/windows to send messages, > > I try to have a nice typo in my message, > but it is a mess arriving in the forum, > while it is still correct if I send the mail to myself. > > Any clue? > > Best regards > > Pascal > From jodie.burch@REDACTED Thu Aug 26 19:16:35 2010 From: jodie.burch@REDACTED (Jodie Burch) Date: Thu, 26 Aug 2010 18:16:35 +0100 Subject: Erlang User Conference 2010 Message-ID: Hi all, A note to let you all know that we have now opened the registration for the Erlang User Conference 2010, 16th November 2010 in Stockholm, Sweden. https://www.erlang-factory.com/conference/ErlangUserConference2010/register We have released 100 places at the Very-Early bird rate of SEK 750 for the day. When they are gone, they are gone, and the price will revert to the Early Bird rate. Don?t delay, as last year?s conference sold out! There are also tutorials on the 15th November which are free to all to attend. With an Erlang University taking place on the 17th-19th November. We expect to publish the full program around the 20th of September. We try to keep emails of this nature on this mailing list to a minimum. To keep up to date with the latest tutorials, talks and speakers, subscribe to our RSS feeds, our mailing list, or follow us on twitter or facebook: http://twitter.com/erlangfactory http://www.facebook.com/home.php?#!/pages/London-United-Kingdom/Erlang-Solut ions/170711784976?ref=ts http://www.erlang-factory.com/ Thanks Jodie From romanshestakov@REDACTED Thu Aug 26 21:06:52 2010 From: romanshestakov@REDACTED (Roman Shestakov) Date: Thu, 26 Aug 2010 12:06:52 -0700 (PDT) Subject: [erlang-questions] what is the best way to test Erlang FSM processes? In-Reply-To: <4C767541.5060504@erix.ericsson.se> References: <809001.55126.qm@web25402.mail.ukl.yahoo.com> <4C767541.5060504@erix.ericsson.se> Message-ID: <822125.27739.qm@web25406.mail.ukl.yahoo.com> thanks Peter for suggestion - this sounds like a good idea, will give a try Regards, Roman ________________________________ From: Peter Andersson To: Roman Shestakov Cc: erlang-questions@REDACTED Sent: Thu, 26 August, 2010 15:08:01 Subject: Re: [erlang-questions] what is the best way to test Erlang FSM processes? Roman Shestakov wrote: > hello, > > I have a dependency tree of connected fsm processes, the state of children fsm' > > depends on state of parents and in some cases, timers attached to fsm can also > trigger state change. > > I would like to be able to test the state of entire system with Common Test but > > can't figure out a good way to use CT for testing fsm states. > > does anybody have some ideas if it is possible to use CT in such scenario? any > examples? other ideas? > > Regards, Roman > Hi Roman, This is an interesting question. Actually, a while back we had some discussions about this with some people using Common Test in a project at Ericsson. They were testing event driven state machines and they were forced to implement difficult recursive test case functions with nested receive and case expressions to test these FSMs. The idea these guys had was to implement test suites as gen_fsm behaviours to test their FSMs instead. This way, the suites become much better structured, more intuitive to read, and easier to modify and extend. We've decided to implement support for this type of suite in Common Test, so that's in the pipeline. We haven't decided exactly what this support should look like, but we have ideas along the lines of: - let Common Test recognize an fsm suite and handle start/stop of the test fsm process automatically (per suite or test case) - a test is executed in the scope of a test case, as usual, and the test case function name will be a parameter in the state of the test fsm process - print info about state transitions, events and messages during a test to the test case log - indicate test case pass or fail by means of state transitions and/or reports to the test case process - provide i/f to modify the state of the test fsm from the init/end config functions We don't really add any new functionality to what the gen_fsm already provides. What we want you to be able to do, you can already do with a gen_fsm behaviour test suite today! We will let Common Test take care of common operations though, to make your test fsm implementations simpler and more compact. Also we want the users to "have to" structure the fsm suites the same way, and use predefined names for common functions/states/events. It'll be easier to share and re-use test code this way. If you'd already like to use a Common Test suite with a gen_fsm behaviour to test your FSMs, here's a very simple (and very incomplete!) example: -module(fsm_SUITE). -behaviour(gen_fsm). -export([code_change/4, handle_event/3, handle_info/3, handle_sync_event/4, init/1, terminate/3]). -include_lib("common_test/include/ct.hrl"). -define(FSM, ?MODULE). -define(TO, 5000). -record(fsm_state, {testcase, result}). %% --- Test suite callback functions --- all() -> [open_connection, ...]. suite() -> [{require,connection_data}]. init_per_testcase(TC, Config) -> {ok,_} = gen_fsm:start({local, ?FSM}, ?MODULE, [TC], []), Config. end_per_testcase(_, _Config) -> gen_fsm:sync_send_all_state_event(?FSM, stop, ?TO). open_connection(_Config) -> ok = gen_fsm:send_event(?FSM, open_connection), verify_result(open, ok). verify_result(State, Result) -> {State,Result} = gen_fsm:sync_send_all_state_event(?FSM, report, ?TO). %% --- FSM behaviour callback functions --- init([TestCase]) -> ct:log("-> Idle", []), {ok, idle, #fsm_state{testcase = TestCase}}. idle(open_connection, State) -> Result = server_utils:open(ct:get_config(connection_data)), ct:log("-> Open", []), {next_state, open, State#fsm_state{result = Result}}. open(Event, State) -> ... handle_event(_Event, StateName, State) -> {next_state, StateName, State}. handle_sync_event(report, From, StateName, State) -> {reply, {StateName,State#fsm_state.result}, StateName, State}; handle_sync_event(stop, _From, _StateName, State) -> {stop, normal, ok, State}. handle_info(_Info, StateName, State) -> {next_state, StateName, State}. terminate(_Reason, _StateName, _State) -> ok. If you have any comments or suggestions on this topic, let us know! /Peter Erlang/OTP, Ericsson AB From dizzyd@REDACTED Thu Aug 26 22:41:18 2010 From: dizzyd@REDACTED (Dave Smith) Date: Thu, 26 Aug 2010 14:41:18 -0600 Subject: Denver/Boulder Erlang User's Group Message-ID: Hi all, I'd like to start an Erlang User's Group in the Denver/Boulder area. I know there's at least 2 people here that use Erlang regularly. :) If someone is willing to host (or sponsor it) I'm willing to put together the agenda and presentations. I've been working in Erlang full-time for the past 5 years or so and would be happy to share what I know about about using it. Anyone else share my interest in getting together with fellow local Erlangers? D. From jh@REDACTED Thu Aug 26 23:40:34 2010 From: jh@REDACTED (Jan Huwald) Date: Thu, 26 Aug 2010 23:40:34 +0200 Subject: {active, once} for generic file descriptors Message-ID: <201008262340.36878.jh@sotun.de> Is there a way to read from a (UNIX) file descriptor with the {active, once} semantics of gen_tcp? I repeatedly used NIFs to obtain a handle (inotify, UNIX domain socket, ...), which I then plugged into the VM poll mechanism using open_port( ... {fd, Handle, _}]). But this way I introduced possible message queue overflows. To prevent this I'd like to imitate the convenient {active, once} behaviour. So I'm up to writing a port driver for generic file descriptor handling, mimicking the gen_tcp interface. But given the need to simply do fail-safe I/O on a FD obtained elsewhere is a reoccuring one, I hope that someone already did that job, or found a better way around. Regards, Jan From rvirding@REDACTED Fri Aug 27 01:26:09 2010 From: rvirding@REDACTED (Robert Virding) Date: Fri, 27 Aug 2010 01:26:09 +0200 Subject: [erlang-questions] calling a local function with an atom stored in a variable. In-Reply-To: <4C6ED908.4030606@erlang-solutions.com> References: <4C6ED908.4030606@erlang-solutions.com> Message-ID: On 20 August 2010 21:35, Mazen Harake wrote: > ?On 20/08/2010 23:21, Mariano Guerra wrote: >> >> the last line " ExprF must be an atom or evaluate to a fun", why can't >> ExprF evaluate to an atom? why ExprF = foo, ExprF() doesn't work but >> ?MODULE:ExprF() does? > > Try: > > -module(m). > -compile(export_all). > foo() -> (bar()):(baz())(). > bar() -> m. > baz() -> biz. > biz() -> io:format("Hello World"). > > Should work. Hope it sheds some light. This works because you are making an external call to the function, even if is the same module. The main reason why you cannot evaluate to a local function at run-time is because local calls to functions in the same module are resolved at compile-time and not at run-time. This for speed. Calls to exported functions in other module, however, external calls, are resolved at run-time which allows evaluating the module name and function name at run-time. Robert From kenji.rikitake@REDACTED Fri Aug 27 03:07:42 2010 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Fri, 27 Aug 2010 10:07:42 +0900 Subject: Tokyo Erlang Workshop #5 walk-in available / reservation required Message-ID: <20100827010742.GA43149@k2r.org> Tokyo Erlang Workshop #5 at Oracle Aoyama Center, Gaienmae, Tokyo, Japan Today 27-AUG-2010 8pm-10pm JST (Please come by 7:45pm due to the entrance security clearance) walk-in seats still available Reservation required at: http://atnd.org/events/6270 FYI Kenji Rikitake From ok@REDACTED Fri Aug 27 05:38:43 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 27 Aug 2010 15:38:43 +1200 Subject: [erlang-questions] sine and cosine with degree angles In-Reply-To: References: Message-ID: <3D209EC5-5490-4018-BA59-434CF100987E@cs.otago.ac.nz> On Aug 26, 2010, at 11:03 PM, Roberto Ostinelli wrote: > > however, due to approximations there seem to be an error in the > computation of the cosine of 90? [which should be 0]: > > 1> math:sin(90 * math:pi() / 180). > 1.0 > 2> math:sin(0 * math:pi() / 180). > 0.0 > 3> math:cos(90 * math:pi() / 180). > 6.123233995736766e-17 > 4> math:cos(0 * math:pi() / 180). > 1.0 As a matter of fact cos(90) looks about as good as it can get. You can't get *precisely* close to pi/2; the closest you can get is pi/2 +/- epsilon, and the cosine of that will be -/+ epsilon. > cos(1.57079632679489661923132169163975144) [1] 6.123234e-17 Now Ada lets you write cos(90, cycle => 360) and Sun's math library lets you write cospi(90.0/180.0) where cospi(x) = cos(pi * x) as if computed with an infinitely precise pi. We would expect either of those to return an exact 0. With the traditional C interface, 6.123233995736766e-17 is as close as you are going to get. From pascalchapier@REDACTED Fri Aug 27 06:42:01 2010 From: pascalchapier@REDACTED (Pascal Chapier) Date: Fri, 27 Aug 2010 06:42:01 +0200 Subject: [erlang-questions] random lookup in ets Message-ID: Hello Ulf, I agree with you, the code is sensible. The funny (strange) things are the barriers I can have in my mind. I take the opportunity of this reply to re-post my previous message using thunderbird, and check if the result is better. Sorry for the annoyance. ######### repost ############# Hello It is a funny usage of the previous function in ordered set! I would have bet that it produces an exception in case of non existing key (like it does with simple set). I tried it in a shell, and it works (almost as in your example). ############################## 1> T = ets:new(table,[ordered_set,public,named_table]). table 2> ets:tab2list(table). [] 3> ets:insert(table,[{1,"one"},{10,"ten"},{5,"five"},{50,"fifty"}]). true 4> ets:tab2list(table). [{1,"one"},{5,"five"},{10,"ten"},{50,"fifty"}] 5> ets:prev(table,10). 5 6> ets:prev(table,9). 5 7> ets:prev(table,1). '$end_of_table' 8> ets:prev(table,1000000). 50 9> T1 = ets:new(table1,[set,public,named_table]). table1 10> ets:insert(table1,[{1,"one"},{10,"ten"},{5,"five"},{50,"fifty"}]). true 11> ets:tab2list(table1). [{1,"one"},{10,"ten"},{50,"fifty"},{5,"five"}] 12> ets:prev(table1,10). 1 13> ets:prev(table1,1). '$end_of_table' 14> ets:prev(table1,9). ** exception error: bad argument in function ets:prev/2 called as ets:prev(table1,9) 15> ############################## This method is quite easy to implement and should be fast, but it does not guarantee that all keys have the same probability to be selected, because the interval between keys is not a constant. In my example, if you select a random value between 1 and 60 (included) you will get: '$end_of_table' -> 1.67% 1 -> 6.67% 5 -> 8.33% 10 -> 66.67% 50 -> 16.67% ?????????????????????????????? I have though about the solution with a complementary index table. I could be interesting, if the initial table is formed by {PrimKey,Value} to build 2 tables: primTable contents {PrimKey,Index,Value}, index contents {Index,PrimKey} (and is an ordered_set). then: add(NewPrimKey,NewValue) -> NewIndex = ets:info(index,size) + 1, ets:insert(index,{NewIndex,NewPrimKey}), ets:insert(primTable,{NewPrimKey,NewIndex,NewValue}). delete(PrimKey) -> [{PrimKey,Index,Value}] = ets:lookup(primTable,PrimKey), % warning iterate in the list if primTable is a bag delete(primTable,Primkey), LastIndex = ets:info(index,size), case (LastIndex == Index) of false -> [{LastIndex,LastKey}] = ets:lookup(index,LastIndex), ets:insert(index,{Index,LastKey}), [{LastKey,LastIndex,Value}] = ets:lookup(primTable,LastKey), ets:insert(primTable,{LastKey,Index,Value}); true -> ok end, delete(index,LastIndex). The order of elements in the tables changes, but it should have no impact on probabilities. In case of parallel accesses to the table, the function must be serialized, and (may be) use ets:safe_fixtable (and the code should be debbugged, I didn't try it... :o) ?????????????????????????????? The simplest solution > > "second, you can use ets:tab2list(Ets), and ets:info(Ets,size), generate > > X with random:uniform(Size): a random value between 1 and Size, and > > finally take the Xth element of the list with lists:nth(X).." can be accelerated, even if it will still be slow with big tables: Size = ets:info(Ets,size), X = random:uniform(Size), ets:lookup(get(Ets,X)). get(Ets,N) -> get(Ets,N-1,ets:first(Ets)). get(_,0,K) -> K; get(Ets,N,K) -> get(Ets,N-1,ets:next(Ets,K)). CU. From matthias@REDACTED Fri Aug 27 15:53:42 2010 From: matthias@REDACTED (Matthias Lang) Date: Fri, 27 Aug 2010 15:53:42 +0200 Subject: how hard is it to rewrite code in a .beam file? Message-ID: <20100827135342.GA14752@corelatus.se> Hi, I'd appreciate some wisdom about writing a .beam -> .beam transform which lets me load a few known (i.e. not every possible) .beam files compiled with R10B into an R13B04 VM. I've looked at beam_load.c and beam_disasm.erl, they seem like reasonable starting points. Anything else I should look at? Am I doomed to fail, or do I have a sporting chance? Kostis? Bj?rn? Tony? --- I want to do this because upgrades on my embedded systems work roughly like this: 1. Unpack a release file 2. Run gth_install.beam on the freshly unpacked release 3. Reboot This works fine until someone tries to downgrade from a modern release, which uses R13B04, to an old release, which uses R10B: =ERROR REPORT==== 27-Aug-2010::15:07:03 === beam/beam_load.c(1730): Error loading function gth_install:make_nand_symlinks/1: op m_plus p y i r: please re-compile this module with an R13B04 compiler I should have heeded warnings about efforts being made to preserve .beam compatibility for two major OTP versions only. But I didn't. Duh. I have a few ideas about how to extricate myself from this, e.g. running multiple VMs at the same time or providing new versions of the old .beams, but the .beam->.beam seems fun and neat, given that I only have to make it work for a handful of versions of one module. Matt From jerometruong@REDACTED Fri Aug 27 16:49:02 2010 From: jerometruong@REDACTED (Jerome Truong) Date: Fri, 27 Aug 2010 09:49:02 -0500 Subject: erlang gs demo and wx Message-ID: I'm trying to test some demos with gs on erlang but not sure how to view the graphics. I cd to /lib/gs-1.5.9/examples and run erl -pa ebin 1>color_demo:start(). and i just get <0.33.0>, i'm guessing it's an object id. But how do i see what was created by the demo? Thanks, Jerome From matthias@REDACTED Sat Aug 28 00:03:17 2010 From: matthias@REDACTED (Matthias Lang) Date: Sat, 28 Aug 2010 00:03:17 +0200 Subject: how hard is it to rewrite code in a .beam file? In-Reply-To: <20100827135342.GA14752@corelatus.se> References: <20100827135342.GA14752@corelatus.se> Message-ID: <20100827220317.GA16945@corelatus.se> Mostly answering my own question: Hacking .beam files isn't very hard, getting into it takes a few hours. beam_disasm.erl, beam_asm.erl, compile.erl and beam_lib.erl are useful references. For me, the spanner in the works was ordinary additions, i.e. the .beam made by R10B contains: {m_plus,[{f,0},{x,0},{i,2},{x,0}]}, % erlang code: X + 2 but m_plus is long gone, R13B generates this instead: {gc_bif2,[{f,0},{u,1},{u,0},{x,0},{i,2},{x,0}]}, I don't completely understand the arguments lists. In the first case, the arguments are {m_plus,[{f,0}, %% what's this? {x,0}, %% source register {i,2}, %% source immediate integer value 2 {x,0}, %% destination register So I don't know exactly what {f,0} and {u,1} and {u,0} are. I think they tell beam which function I want and supply some liveness information, whatever exactly that is. Clues, please? Matt ---------------------------------------------------------------------- On Friday, August 27, Matthias Lang wrote: > Hi, > > I'd appreciate some wisdom about writing a .beam -> .beam transform > which lets me load a few known (i.e. not every possible) .beam files > compiled with R10B into an R13B04 VM. > > I've looked at beam_load.c and beam_disasm.erl, they seem like > reasonable starting points. Anything else I should look at? Am > I doomed to fail, or do I have a sporting chance? Kostis? Bj?rn? Tony? > > --- > > I want to do this because upgrades on my embedded systems work roughly > like this: > > 1. Unpack a release file > 2. Run gth_install.beam on the freshly unpacked release > 3. Reboot > > This works fine until someone tries to downgrade from a modern release, > which uses R13B04, to an old release, which uses R10B: > > =ERROR REPORT==== 27-Aug-2010::15:07:03 === > beam/beam_load.c(1730): Error loading function gth_install:make_nand_symlinks/1: op m_plus p y i r: > please re-compile this module with an R13B04 compiler > > I should have heeded warnings about efforts being made to preserve .beam > compatibility for two major OTP versions only. But I didn't. Duh. > > I have a few ideas about how to extricate myself from this, > e.g. running multiple VMs at the same time or providing new versions > of the old .beams, but the .beam->.beam seems fun and neat, given > that I only have to make it work for a handful of versions of one module. > > Matt From kostis@REDACTED Sat Aug 28 00:15:37 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 28 Aug 2010 01:15:37 +0300 Subject: [erlang-questions] Re: how hard is it to rewrite code in a .beam file? In-Reply-To: <20100827220317.GA16945@corelatus.se> References: <20100827135342.GA14752@corelatus.se> <20100827220317.GA16945@corelatus.se> Message-ID: <4C783909.7030904@cs.ntua.gr> Matthias Lang wrote: > Mostly answering my own question: > > Hacking .beam files isn't very hard, getting into it takes a few > hours. beam_disasm.erl, beam_asm.erl, compile.erl and beam_lib.erl > are useful references. > > For me, the spanner in the works was ordinary additions, i.e. the > .beam made by R10B contains: > > {m_plus,[{f,0},{x,0},{i,2},{x,0}]}, % erlang code: X + 2 > > but m_plus is long gone, R13B generates this instead: > > {gc_bif2,[{f,0},{u,1},{u,0},{x,0},{i,2},{x,0}]}, > > I don't completely understand the arguments lists. In the first case, > the arguments are > > {m_plus,[{f,0}, %% what's this? > {x,0}, %% source register > {i,2}, %% source immediate integer value 2 > {x,0}, %% destination register > > So I don't know exactly what {f,0} and {u,1} and {u,0} are. I think > they tell beam which function I want and supply some liveness > information, whatever exactly that is. Clues, please? You are mostly right. The {f,L} args denote labels -- possibly where to go upon failure. The {u,N} args denote unsigned integers N. If you look into beam_disasm, the relevant entry reads: resolve_inst({gc_bif2,Args},Imports,_,_) -> [F,Live,Bif,A1,A2,Reg] = resolve_args(Args), {extfunc,_Mod,BifName,_Arity} = lookup(Bif+1,Imports), {gc_bif,BifName,F,Live,[A1,A2],Reg}; and the names give you clues about what the two u args are. For example, the Bif arg ({u,0}) corresponds to binary +. Kostis From james.aimonetti@REDACTED Sat Aug 28 01:11:13 2010 From: james.aimonetti@REDACTED (James Aimonetti) Date: Fri, 27 Aug 2010 16:11:13 -0700 Subject: Question about using case statements Message-ID: Hey list, Wondered what the list thought of this usage of case statements: I have a file path and a proplist with some values, one of which tells me what to do with the file. I want to be sure the file exists before I do something with the file but didn't want to get too deep into nested case statements, so I have something like this -> ... Prop = get_proplist_from_somwhere(), File = get_file_from_somewhere(), case filelib:is_regular(File) andalso get_value(key, Prop) of false -> handle_missing_file_error(File); specific_value_1 -> do_specific_value_1_action(File, Prop); ... _Val -> io:format("Unhandled file action value ~p~n", [_Val]) % to be removed after testing end, ... Is this common / acceptable use of the andalso short-circuit boolean? Thanks, James From jay@REDACTED Sat Aug 28 08:41:41 2010 From: jay@REDACTED (jay@REDACTED) Date: Fri, 27 Aug 2010 23:41:41 -0700 (PDT) Subject: [erlang-questions] Using ETS for large amounts of data? Message-ID: <50021.64.81.32.110.1282977701.squirrel@duomark.com> You have two issues here the initial loading of the data, and the actions which happen on lookup. If you use and ETS table, you have to use integer index offsets as the return value or the binary data will be copied into the ETS on insert and out again on every lookup match. The index offset should be fast, but it is a repeated binary match step that is really unnecessary. Try the following, you will probably get better overall performance with other things running and putting pressure on memory: 1) Read in the whole file as a single binary 2) Never append or modify this binary so it doesn't get copied 3) Iterate over it using bit syntax to match 26-byte segments - [ Rec || <> <= FileBinary ] - You will get sub-binaries which are pointers into #1 3) Store each sub-binary as the value for a key in a tree or dict You will probably do better in the long run by storing the intervals in a tree, dictionary, tuple or array. Lookups might be slower (although depending on the key you might be surprised at how little difference there is), but there should be no memory copying and thus no future garbage collection pressure. The pre-fetched sub-binaries will never incur a binary match penalty, nor copying and should avoid all garbage collection. The underlying large binary will always be present, but it will be stored in the shared binary heap rather than internal to your process so message passing the sub-binaries should be small and fast as well (all receivers on the same node will point to the same underlying large binary in the single binary heap). You could keep the lookup tree in a process and send messages out on requests for lookup. You just can't distribute it off a single erlang node, but you should get good benefit on a multi-core processor. jay From kostis@REDACTED Sat Aug 28 09:24:54 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 28 Aug 2010 10:24:54 +0300 Subject: [erlang-questions] Question about using case statements In-Reply-To: References: Message-ID: <4C78B9C6.6010806@cs.ntua.gr> James Aimonetti wrote: > Hey list, > > Wondered what the list thought of this usage of case statements: I > have a file path and a proplist with some values, one of which tells > me what to do with the file. I want to be sure the file exists before > I do something with the file but didn't want to get too deep into > nested case statements, so I have something like this -> > > ... > Prop = get_proplist_from_somwhere(), > File = get_file_from_somewhere(), > case filelib:is_regular(File) andalso get_value(key, Prop) of > false -> > handle_missing_file_error(File); > specific_value_1 -> > do_specific_value_1_action(File, Prop); > ... > _Val -> > io:format("Unhandled file action value ~p~n", [_Val]) % to be > removed after testing > end, > ... > > Is this common / acceptable use of the andalso short-circuit boolean? It's something you can do alright, but it's not common. I personally think that in this particular case it is unnecessary. The following code expresses the code's intention much clearer, IMO: ... File = get_file_from_somewhere(), case filelib:is_regular(File) of false -> handle_missing_file_error(File); true -> Prop = get_proplist_from_somwhere(), case get_value(key, Prop) of specific_value_1 -> do_specific_value_1_action(File, Prop); ... _Val -> io:format("Unhandled file action value ~p~n", [_Val]) % to be removed after testing end end, ... and it is not much more verbose. Kostis From matthias@REDACTED Sat Aug 28 09:58:16 2010 From: matthias@REDACTED (Matthias Lang) Date: Sat, 28 Aug 2010 09:58:16 +0200 Subject: [erlang-questions] Re: how hard is it to rewrite code in a .beam file? In-Reply-To: <4C783909.7030904@cs.ntua.gr> References: <20100827135342.GA14752@corelatus.se> <20100827220317.GA16945@corelatus.se> <4C783909.7030904@cs.ntua.gr> Message-ID: <20100828075816.GA3418@corelatus.se> Kostis wrote: > The {f,L} args denote labels -- possibly where to go upon failure. > The {u,N} args denote unsigned integers N. Ah, ok. I had a working transform yesterday, and that gets me closer to understanding why it works: case beam_opcodes:opname(Opcode) of {m_plus, Arity = 4} -> io:fwrite("rewriting mplus instruction\n"), {[Function, S1, S2, Dest], Leftover} = decode_n_args(Arity, [], Rest), New_opcode = beam_opcodes:opcode(gc_bif2, 6), Live = {{u,1}, 16}, Bif = {{u,0}, 0}, New_args = [Function, Live, Bif, S1, S2, Dest], New_bytes = lists:flatten([New_opcode|args_to_ops(New_args)]), io:fwrite("rewritten mplus assembles to ~p\n", [New_bytes]), [{New_bytes, gc_bif2, args_to_syms(New_args)}|disasm(Leftover)]; There's another avenue I should take a look at: R11 (and, probably, R12) can load those R10 files, I should check what those releases do with the m_plus instructions. Thanks for the tips. Matt From zabrane3@REDACTED Sat Aug 28 12:11:34 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Sat, 28 Aug 2010 12:11:34 +0200 Subject: Parsing custom HTTP headers with inet, [{active, false},{packet, http_bin}, ...] Message-ID: Hi, I'm facing a strange problem to parse custom HTTP headers in my server code which looks like: {ok, ListenSocket} = gen_tcp:listen(9876, [binary, inet, {active, false}, {packet, http_bin}, {reuseaddr, true}]), ... headers(Socket) -> ok = inet:setopts(Socket, [{active, once}]), receive ... {http, _, {http_header, _, Header, _, Val}} -> io:format("---> The header is: ~p: ~p~n", [Header, Val]), Trying to send the following custom header "X-ARCHWEB-RESOURCE-UUID" works fine, and the custom header appears correctly in my server code: $ curl -x localhost:9876 -H 'X-ARCHWEB-RESOURCE-UUID: 123456789' --url http://www.foo.com 1> ... ---> The header is: <<"X-ARCHWEB-RESOURCE-UUID">>: <<"123456789">> But changing a bit the custom header name to "X-WEBARCH-RESOURCE-UUID" doesn't work and the server don't even print it out (i.e nothing is received). Now, trying with "X-WEBARCH-RESOURC-UUID" (notice the missing "E" at the end of "RESOURC") works fine: 2> ... ---> The header is: <<"X-WEBARCH-RESOURC-UUID">>: <<"123456789">> Even bad, trying with "X-WEBARCH-RESOURCE" completely alter the letters cases (first letter is capital letter and the rest is lowercase): 3> ---> The header is: <<"X-Webarch-Resource">>: <<"123456789">> Could someone tell me what am I doing wrong? Is there any option to tell "inet" to not alter the headers at all? -- Regards Zabrane From tony@REDACTED Sat Aug 28 12:32:38 2010 From: tony@REDACTED (Tony Rogvall) Date: Sat, 28 Aug 2010 12:32:38 +0200 Subject: [erlang-questions] Parsing custom HTTP headers with inet, [{active, false},{packet, http_bin}, ...] In-Reply-To: References: Message-ID: <73826F72-3D7E-4797-8B8B-078EBD7C300E@rogvall.se> What version of OTP are you using? Try: > erlang:decode_packet(httph, <<"X-WEBARCH-RESOURCE-UUID: 123456789\r\n\r\n">>, []). Return value should be: {ok,{http_header,0,"X-WEBARCH-RESOURCE-UUID",undefined, "123456789"}, <<"\r\n">>} If this works, then the problem is probably somewhere else. tcp-dump? /Tony On 28 aug 2010, at 12.11, zabrane Mikael wrote: > Hi, > > I'm facing a strange problem to parse custom HTTP headers in my server > code which looks like: > > {ok, ListenSocket} = gen_tcp:listen(9876, [binary, inet, {active, > false}, {packet, http_bin}, {reuseaddr, true}]), > ... > headers(Socket) -> > ok = inet:setopts(Socket, [{active, once}]), > receive > ... > {http, _, {http_header, _, Header, _, Val}} -> > io:format("---> The header is: ~p: ~p~n", [Header, Val]), > > Trying to send the following custom header "X-ARCHWEB-RESOURCE-UUID" > works fine, and the custom header appears > correctly in my server code: > $ curl -x localhost:9876 -H 'X-ARCHWEB-RESOURCE-UUID: 123456789' --url > http://www.foo.com > > 1> ... > ---> The header is: <<"X-ARCHWEB-RESOURCE-UUID">>: <<"123456789">> > > But changing a bit the custom header name to "X-WEBARCH-RESOURCE-UUID" > doesn't work > and the server don't even print it out (i.e nothing is received). > > Now, trying with "X-WEBARCH-RESOURC-UUID" (notice the missing "E" at > the end of "RESOURC") works fine: > > 2> ... > ---> The header is: <<"X-WEBARCH-RESOURC-UUID">>: <<"123456789">> > > Even bad, trying with "X-WEBARCH-RESOURCE" completely alter the > letters cases (first letter is capital letter and the rest is > lowercase): > > 3> > ---> The header is: <<"X-Webarch-Resource">>: <<"123456789">> > > Could someone tell me what am I doing wrong? > Is there any option to tell "inet" to not alter the headers at all? > > -- > Regards > Zabrane > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From zabrane3@REDACTED Sat Aug 28 12:46:04 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Sat, 28 Aug 2010 12:46:04 +0200 Subject: [erlang-questions] Parsing custom HTTP headers with inet, [{active, false},{packet, http_bin}, ...] In-Reply-To: <73826F72-3D7E-4797-8B8B-078EBD7C300E@rogvall.se> References: <73826F72-3D7E-4797-8B8B-078EBD7C300E@rogvall.se> Message-ID: Hi Tony, 2010/8/28 Tony Rogvall : > What ?version of OTP are you using? I'm under OSX Snow Leopard: $ erl Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> > Try: >> erlang:decode_packet(httph, <<"X-WEBARCH-RESOURCE-UUID: 123456789\r\n\r\n">>, []). > > Return value should be: > > {ok,{http_header,0,"X-WEBARCH-RESOURCE-UUID",undefined, > ? ? ? ? ? ? ? ? "123456789"}, > ? ?<<"\r\n">>} Got the same result as you: 1> erlang:decode_packet(httph, <<"X-WEBARCH-RESOURCE-UUID: 123456789\r\n\r\n">>, []). {ok,{http_header,0,"X-WEBARCH-RESOURCE-UUID",undefined, "123456789"}, <<"\r\n">>} "httph_bin" works fine too: 2> erlang:decode_packet(httph_bin, <<"X-WEBARCH-RESOURCE-UUID: 123456789\r\n\r\n">>, []). {ok,{http_header,0,<<"X-WEBARCH-RESOURCE-UUID">>,undefined, <<"123456789">>}, > If this works, ?then the problem is probably somewhere else. Do you think it's related to my socket options? > tcp-dump? Hmm ... I'm pretty sure that "curl" sent the headers correctly. I can see them with debug "on": $ curl -v -D - -x localhost:9876 -H 'X-ARCHWEB-RESOURCE-UUID: 123456789' --url http://www.foo.com * About to connect() to proxy 127.0.0.1 port 8080 (#0) * Trying 127.0.0.1... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed ^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0connected * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) > GET http://www.lemonde.fr HTTP/1.0 > User-Agent: curl/7.20.0 (i386-apple-darwin10.3.0) libcurl/7.20.0 OpenSSL/0.9.8o zlib/1.2.5 libidn/1.19 > Host: www.foo.com > Accept: */* > Proxy-Connection: Keep-Alive > X-WEBARCH-RESOURCE-UUID: 123456789 ... -- Regards Zabrane > > /Tony > > > On 28 aug 2010, at 12.11, zabrane Mikael wrote: > >> Hi, >> >> I'm facing a strange problem to parse custom HTTP headers in my server >> code which looks like: >> >> {ok, ListenSocket} = gen_tcp:listen(9876, [binary, inet, {active, >> false}, ?{packet, http_bin}, {reuseaddr, true}]), >> ... >> headers(Socket) -> >> ? ?ok = inet:setopts(Socket, [{active, once}]), >> ? ?receive >> ? ? ? ? ... >> ? ? ? ?{http, _, {http_header, _, Header, _, Val}} -> >> ? ? ? ? ? ?io:format("---> The header is: ~p: ~p~n", [Header, Val]), >> >> Trying to send the following custom header "X-ARCHWEB-RESOURCE-UUID" >> works fine, and the custom header appears >> correctly in my server code: >> $ curl -x localhost:9876 -H 'X-ARCHWEB-RESOURCE-UUID: 123456789' --url >> http://www.foo.com >> >> 1> ... >> ---> The header is: <<"X-ARCHWEB-RESOURCE-UUID">>: <<"123456789">> >> >> But changing a bit the custom header name to "X-WEBARCH-RESOURCE-UUID" >> doesn't work >> and the server don't even print it out (i.e nothing is received). >> >> Now, trying with "X-WEBARCH-RESOURC-UUID" (notice the missing "E" at >> the end of "RESOURC") works fine: >> >> 2> ... >> ---> The header is: <<"X-WEBARCH-RESOURC-UUID">>: <<"123456789">> >> >> Even bad, trying with "X-WEBARCH-RESOURCE" completely alter the >> letters cases (first letter is capital letter and the rest is >> lowercase): >> >> 3> >> ---> The header is: <<"X-Webarch-Resource">>: <<"123456789">> >> >> Could someone tell me what am I doing wrong? >> Is there any option to tell "inet" to not alter the headers at all? >> >> -- >> Regards >> Zabrane >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > From jerometruong@REDACTED Sat Aug 28 15:57:13 2010 From: jerometruong@REDACTED (Jerome Truong) Date: Sat, 28 Aug 2010 08:57:13 -0500 Subject: [erlang-questions] erlang gs demo and wx In-Reply-To: References: Message-ID: <743641CD-1831-4F2A-B6E7-FD43E67AFDDC@gmail.com> Thanks, pointing out wx directed me to install the erlang git repository with the latest changes (including the wx library.) I did get a lot of warning during compiling, below are a couple: 1) g++ -c -I/usr/lib/wx/include/mac-unicode-debug-2.8 -I/usr/include/wx-2.8 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -D__WXDEBUG__ -D__WXMAC__ -g -Wall -fPIC -DDEBUG -no-cpp-precomp -m32 -g -O2 -D_MACOSX -D_THREAD_SAFE -D_REENTRANT -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DSIZEOF_VOID_P=4 -DHAVE_OPENGL_GL_H=1 -DHAVE_GL_SUPPORT=1 -DHAVE_GLINTPTR=1 -DHAVE_GLINTPTRARB=1 -DHAVE_GLCHAR=1 -DHAVE_GLCHARARB=1 -DHAVE_GLHALFARB=1 -DHAVE_WX_STC_STC_H=1 -I/Users/jqtruong/Programs/erlang/otp/erts/emulator/beam -I/Users/jqtruong/Programs/erlang/otp/erts/include -I/Users/jqtruong/Programs/erlang/otp/erts/include/i386-apple-darwin10.4.0 -I/Users/jqtruong/Programs/erlang/otp/erts/include/internal -I/Users/jqtruong/Programs/erlang/otp/erts/include/internal/i386-apple-darwin10.4.0 -I/Users/jqtruong/Programs/erlang/otp/erts/emulator/sys/unix wxe_return.cpp -o i386-apple-darwin10.4.0/wxe_return.o In file included from /usr/include/wx-2.8/wx/mac/glcanvas.h:4, from /usr/include/wx-2.8/wx/glcanvas.h:60, from wxe_impl.h:23, from wxe_return.h:30, from wxe_return.cpp:20: /usr/include/wx-2.8/wx/mac/carbon/glcanvas.h:49: warning: ?AGLDrawable? is deprecated (declared at /System/Library/Frameworks/AGL.framework/Headers/agl.h:48) /usr/include/wx-2.8/wx/mac/carbon/glcanvas.h:53: warning: ?AGLDrawable? is deprecated (declared at /System/Library/Frameworks/AGL.framework/Headers/agl.h:48) mkdir -p i386-apple-darwin10.4.0 2) /Users/jqtruong/Programs/erlang/otp/bootstrap/lib/wx/include/wx.hrl:26: Warning: record wx has field(s) without type information Excuse the long strings...As you can probably guess, i'm installing on mac os x, but the 10.6 64-bit version... so i'm a little confused as to why it's pointing out i386-apple-darwin10.4.0 I tested the wx demo and it worked, but should i be worried about the warnings? Thanks, Jerome On Aug 27, 2010, at 10:46 AM, Michael Turner wrote: > > The gs app isn't really being supported anymore. They should say this in the docs, but they don't. > > Try the wx demos, and if they don't work, you're really in trouble. Version? Platform? > > -michael turner > > On Fri, Aug 27, 2010 at 11:49 PM, Jerome Truong wrote: > I'm trying to test some demos with gs on erlang but not sure how to view the graphics. > > I cd to /lib/gs-1.5.9/examples and run erl -pa ebin > > 1>color_demo:start(). > > and i just get <0.33.0>, i'm guessing it's an object id. > > But how do i see what was created by the demo? > > Thanks, > Jerome > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tony@REDACTED Sat Aug 28 22:26:58 2010 From: tony@REDACTED (Tony Rogvall) Date: Sat, 28 Aug 2010 22:26:58 +0200 Subject: [erlang-questions] Parsing custom HTTP headers with inet, [{active, false},{packet, http_bin}, ...] In-Reply-To: References: <73826F72-3D7E-4797-8B8B-078EBD7C300E@rogvall.se> Message-ID: <04AA60E9-BFA1-4B41-877A-80EDCB1671C8@rogvall.se> On 28 aug 2010, at 12.46, zabrane Mikael wrote: > Hi Tony, > > 2010/8/28 Tony Rogvall : >> What version of OTP are you using? > > I'm under OSX Snow Leopard: > $ erl > Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] > [kernel-poll:false] > > Eshell V5.7.5 (abort with ^G) > 1> > Same as me. > >> > > Hmm ... I'm pretty sure that "curl" sent the headers correctly. I can > see them with debug "on": > > $ curl -v -D - -x localhost:9876 -H 'X-ARCHWEB-RESOURCE-UUID: > 123456789' --url http://www.foo.com > * About to connect() to proxy 127.0.0.1 port 8080 (#0) > * Trying 127.0.0.1... % Total % Received % Xferd Average Speed Hmm why is it saying 8080 ? you wrote 9876? My line: curl -v -D - -x localhost:8888 -H 'X-ARCHWEB-RESOURCE-UUID:123456789' --url http://www.foo.com * About to connect() to proxy localhost port 8888 (#0) * Trying ::1... Connection refused * Trying fe80::1... Connection refused * Trying 127.0.0.1... connected * Connected to localhost (127.0.0.1) port 8888 (#0) > GET http://www.foo.com HTTP/1.1 > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3 > Host: www.foo.com > Accept: */* > Proxy-Connection: Keep-Alive > X-ARCHWEB-RESOURCE-UUID:123456789 > The small test program I used worked fine: BTW The reason you get mixed cases on some (size <= 20) header names is because the packet parser normalize the input to the "normal" case usage (as in Content-Length). The header names are case insensitive anyway, so you must handle it. /Tony -module(http_test). -compile(export_all). start() -> {ok,L} = gen_tcp:listen(8888, [{active,false},{reuseaddr,true}, {packet,http}]), accept(L). accept(L) -> case gen_tcp:accept(L) of {ok, S} -> loop(S), gen_tcp:close(S), accept(L); Error -> io:format("accept: ~p\n", [Error]), accept(L) end. loop(S) -> inet:setopts(S, [{active,once}]), receive {tcp_closed, S} -> io:format("closed\n", []), ok; Msg -> io:format("Http: ~p\n", [Msg]), loop(S) end. > Time Time Time Current > Dload Upload Total Spent Left Speed > ^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- > --:--:-- 0connected > * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) >> GET http://www.lemonde.fr HTTP/1.0 >> User-Agent: curl/7.20.0 (i386-apple-darwin10.3.0) libcurl/7.20.0 OpenSSL/0.9.8o zlib/1.2.5 libidn/1.19 >> Host: www.foo.com >> Accept: */* >> Proxy-Connection: Keep-Alive >> X-WEBARCH-RESOURCE-UUID: 123456789 > ... > > -- > Regards > Zabrane > > >> >> /Tony >> >> >> On 28 aug 2010, at 12.11, zabrane Mikael wrote: >> >>> Hi, >>> >>> I'm facing a strange problem to parse custom HTTP headers in my server >>> code which looks like: >>> >>> {ok, ListenSocket} = gen_tcp:listen(9876, [binary, inet, {active, >>> false}, {packet, http_bin}, {reuseaddr, true}]), >>> ... >>> headers(Socket) -> >>> ok = inet:setopts(Socket, [{active, once}]), >>> receive >>> ... >>> {http, _, {http_header, _, Header, _, Val}} -> >>> io:format("---> The header is: ~p: ~p~n", [Header, Val]), >>> >>> Trying to send the following custom header "X-ARCHWEB-RESOURCE-UUID" >>> works fine, and the custom header appears >>> correctly in my server code: >>> $ curl -x localhost:9876 -H 'X-ARCHWEB-RESOURCE-UUID: 123456789' --url >>> http://www.foo.com >>> >>> 1> ... >>> ---> The header is: <<"X-ARCHWEB-RESOURCE-UUID">>: <<"123456789">> >>> >>> But changing a bit the custom header name to "X-WEBARCH-RESOURCE-UUID" >>> doesn't work >>> and the server don't even print it out (i.e nothing is received). >>> >>> Now, trying with "X-WEBARCH-RESOURC-UUID" (notice the missing "E" at >>> the end of "RESOURC") works fine: >>> >>> 2> ... >>> ---> The header is: <<"X-WEBARCH-RESOURC-UUID">>: <<"123456789">> >>> >>> Even bad, trying with "X-WEBARCH-RESOURCE" completely alter the >>> letters cases (first letter is capital letter and the rest is >>> lowercase): >>> >>> 3> >>> ---> The header is: <<"X-Webarch-Resource">>: <<"123456789">> >>> >>> Could someone tell me what am I doing wrong? >>> Is there any option to tell "inet" to not alter the headers at all? >>> >>> -- >>> Regards >>> Zabrane >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From zabrane3@REDACTED Sun Aug 29 02:02:59 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Sun, 29 Aug 2010 02:02:59 +0200 Subject: [erlang-questions] Parsing custom HTTP headers with inet, [{active, false},{packet, http_bin}, ...] In-Reply-To: <04AA60E9-BFA1-4B41-877A-80EDCB1671C8@rogvall.se> References: <73826F72-3D7E-4797-8B8B-078EBD7C300E@rogvall.se> <04AA60E9-BFA1-4B41-877A-80EDCB1671C8@rogvall.se> Message-ID: Hi Tony, >> $ curl -v -D - -x localhost:9876 -H 'X-ARCHWEB-RESOURCE-UUID: >> 123456789' --url http://www.foo.com >> * About to connect() to proxy 127.0.0.1 port 8080 (#0) >> * ? Trying 127.0.0.1... ? % Total ? ?% Received % Xferd ?Average Speed > > Hmm why is it saying 8080 ? you wrote 9876? Correct. It was a typo ;-) > curl -v -D - -x localhost:8888 -H 'X-ARCHWEB-RESOURCE-UUID:123456789' --url http://www.foo.com > > * About to connect() to proxy localhost port 8888 (#0) > * ? Trying ::1... Connection refused > * ? Trying fe80::1... Connection refused > * ? Trying 127.0.0.1... connected > * Connected to localhost (127.0.0.1) port 8888 (#0) >> GET http://www.foo.com HTTP/1.1 >> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3 >> Host: www.foo.com >> Accept: */* >> Proxy-Connection: Keep-Alive >> X-ARCHWEB-RESOURCE-UUID:123456789 >> > > The small test program I used worked fine: Could you share it please? > BTW The reason you get mixed cases on some (size <= 20) header names is because the packet parser > normalize the input to the "normal" case usage (as in Content-Length). The header names > are case insensitive anyway, so you must handle it. Is there a way to tell the packet parser to not normalize anything? -- Regards Zabrane From jvliwanag@REDACTED Sun Aug 29 03:41:50 2010 From: jvliwanag@REDACTED (Jan Vincent) Date: Sun, 29 Aug 2010 09:41:50 +0800 Subject: Erlang post-production Message-ID: Hi, Upon production of an erlang based system, how are maintenance and upgrades done. I found the tools that come with erlang - from debugging to monitoring to live code updates to be really powerful. However, these seem to be best used not by sysadmins, but by developers. I would imagine that an erlang based system would have to introduce a new workflow compared to projects say, when done in Java or PHP. Any tips would be greatly appreciated! Jan Vincent Liwanag jvliwanag@REDACTED From rzezeski@REDACTED Sun Aug 29 05:41:55 2010 From: rzezeski@REDACTED (Ryan Zezeski) Date: Sat, 28 Aug 2010 23:41:55 -0400 Subject: [erlang-questions] Using ETS for large amounts of data? In-Reply-To: <50021.64.81.32.110.1282977701.squirrel@duomark.com> References: <50021.64.81.32.110.1282977701.squirrel@duomark.com> Message-ID: On Sat, Aug 28, 2010 at 2:41 AM, wrote: > > If you use and ETS table, you have to use integer index offsets as the > return value or the binary data will be copied into the ETS on insert and > out again on every lookup match. I think the documentation needs to be updated because I don't witness this behavior in R14A. For example, I can read a 436M CSV file, store it in ETS and not see my memory usage go up. Furthermore, I can then use lookup and match 400MB of the binary and also not see the memory go up. My erl session looked something like: 1> {ok,Data} = file:read_file("436MB_file"). 2> ets:new(test,[named_table]). 3> ets:insert(test,{data,Data}). 4> [{data,<>}] = ets:lookup(test,data). After line 1, the OS process was using ~445MB of RAM. After line 4 it was still using only 445MB and I never saw it move. -Ryan From boris.muehmer@REDACTED Sun Aug 29 08:21:47 2010 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Sun, 29 Aug 2010 08:21:47 +0200 Subject: [erlang-questions] Let's start a project for a (MMO-)Game in Erlang... (inspired by "Erlang: Who uses it for games?") In-Reply-To: References: Message-ID: First of all, thank You very much for Your response! In my (first) answer I will only address a few of Your suggestions. 2010/8/23 Joe Armstrong : > I like the idea of an MMO-game infrastructure in Erlang - I think to > do this we need to make > a highly modular architecture with pluggable components. If these talk Basically this is what I wanted to do, but (currently) I don't know how to implement "pluggable components" using Erlang/OTP. Somehow I still lack the necessary Erlang/OTP "thinking". Using C#/.Net I would define some interfaces, implement them in various classes/assemblies, and (ab-)use reflection to find and load them... not to mention some AppDomain magic (to succesfully get rid of dynamically loaded stuff). > [...] > Another possibility would be to rip out the SDL/OpenGL part of wings > (http://www.wings3d.com/) and make something like love with erlang > replacing lua. I was thinking about a "pure" OpenGL Interface like "freeglut" first, but it would be not enough for a game. After that I had a brief look at wings, because of the SDL interface. Than I thought OGRE (http://www.ogre3d.org/) or Open Scene Graph (http://www.openscenegraph.org/) would be more suitable environments for a game client. But when I started this fork of the thread, I saw from the responses that the client would be the real hard part to implement, because every single person hat a least one other idea what would be needed for the client. I am looking at wings (and yaws) from time to time now to see how "things" can be done in Erlang. I think I will also look at ejabberd and CouchDB. This is not just to learn from those projects but also to reuse them. Still my primary goal is to learn more about Erlang and OTP first... so PEMMOX might be stalled a bit... - boris From tony@REDACTED Sun Aug 29 11:06:53 2010 From: tony@REDACTED (Tony Rogvall) Date: Sun, 29 Aug 2010 11:06:53 +0200 Subject: [erlang-questions] Parsing custom HTTP headers with inet, [{active, false},{packet, http_bin}, ...] In-Reply-To: References: <73826F72-3D7E-4797-8B8B-078EBD7C300E@rogvall.se> <04AA60E9-BFA1-4B41-877A-80EDCB1671C8@rogvall.se> Message-ID: <31D07A88-3A7A-4DB5-9A46-C948C9E0C382@rogvall.se> On 29 aug 2010, at 02.02, zabrane Mikael wrote: >> >> The small test program I used worked fine: > > Could you share it please? > It was in the message a bit below ;-) Check again. >> BTW The reason you get mixed cases on some (size <= 20) header names is because the packet parser >> normalize the input to the "normal" case usage (as in Content-Length). The header names >> are case insensitive anyway, so you must handle it. > > Is there a way to tell the packet parser to not normalize anything? > Nope. But as I said you must (normally) handle it. So there is really no point in adding that options (I think). Regards /Tony > -- > Regards > Zabrane > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From zvi.avraham@REDACTED Sun Aug 29 11:20:56 2010 From: zvi.avraham@REDACTED (Zvi) Date: Sun, 29 Aug 2010 02:20:56 -0700 (PDT) Subject: using guards in gen_server Message-ID: <7ea62c4b-ddbe-43cc-8f97-abe507b19d33@l20g2000yqm.googlegroups.com> Hi, When writing gen_server, do you put validation guards in API functions or in handle_call / cast or in both ? What's the convention? For example: send(Pid) when is_pid(Pid) -> gen_server:call(?SERVER, {send, Pid}). handle_call({send, Pid}, _From, _State) -> Resp = do_send(Pid), {reply, Resp, State}. OR: send(Pid) -> gen_server:call(?SERVER, {send, Pid}). handle_call({send, Pid}, _From, _State) when is_pid(Pid) -> Resp = do_send(Pid), {reply, Resp, State}. OR: send(Pid) when is_pid(Pid) -> gen_server:call(?SERVER, {send, Pid}). handle_call({send, Pid}, _From, _State) when is_pid(Pid) -> Resp = do_send(Pid), {reply, Resp, State}. From norton@REDACTED Sun Aug 29 11:49:17 2010 From: norton@REDACTED (Joseph Wayne Norton) Date: Sun, 29 Aug 2010 18:49:17 +0900 Subject: [erlang-questions] using guards in gen_server In-Reply-To: <7ea62c4b-ddbe-43cc-8f97-abe507b19d33@l20g2000yqm.googlegroups.com> References: <7ea62c4b-ddbe-43cc-8f97-abe507b19d33@l20g2000yqm.googlegroups.com> Message-ID: I'd place the guards in the API functions. Let the caller crash rather than the server. On Sun, 29 Aug 2010 18:20:56 +0900, Zvi wrote: > Hi, > > When writing gen_server, do you put validation guards in API functions > or in handle_call / cast or in both ? What's the convention? > > For example: > > send(Pid) when is_pid(Pid) -> > gen_server:call(?SERVER, {send, Pid}). > > handle_call({send, Pid}, _From, _State) -> > Resp = do_send(Pid), > {reply, Resp, State}. > > OR: > > send(Pid) -> > gen_server:call(?SERVER, {send, Pid}). > > handle_call({send, Pid}, _From, _State) when is_pid(Pid) -> > Resp = do_send(Pid), > {reply, Resp, State}. > > OR: > > send(Pid) when is_pid(Pid) -> > gen_server:call(?SERVER, {send, Pid}). > > handle_call({send, Pid}, _From, _State) when is_pid(Pid) -> > Resp = do_send(Pid), > {reply, Resp, State}. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- norton@REDACTED From zabrane3@REDACTED Sun Aug 29 15:19:49 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Sun, 29 Aug 2010 15:19:49 +0200 Subject: [erlang-questions] Parsing custom HTTP headers with inet, [{active, false},{packet, http_bin}, ...] In-Reply-To: <31D07A88-3A7A-4DB5-9A46-C948C9E0C382@rogvall.se> References: <73826F72-3D7E-4797-8B8B-078EBD7C300E@rogvall.se> <04AA60E9-BFA1-4B41-877A-80EDCB1671C8@rogvall.se> <31D07A88-3A7A-4DB5-9A46-C948C9E0C382@rogvall.se> Message-ID: Fixed. Thousand thanks Tony! -- Regards Zabrane 2010/8/29 Tony Rogvall : > > On 29 aug 2010, at 02.02, zabrane Mikael wrote: > >>> >>> The small test program I used worked fine: >> >> Could you share it please? >> > > It was in the message a bit below ;-) > Check again. > >>> BTW The reason you get mixed cases on some (size <= 20) header names is because the packet parser >>> normalize the input to the "normal" case usage (as in Content-Length). The header names >>> are case insensitive anyway, so you must handle it. >> >> Is there a way to tell the packet parser to not normalize anything? >> > Nope. But as I said you must (normally) handle it. So there is really no point in adding > that options (I think). > > Regards > > /Tony > >> -- >> Regards >> Zabrane >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > From rvirding@REDACTED Sun Aug 29 17:29:02 2010 From: rvirding@REDACTED (Robert Virding) Date: Sun, 29 Aug 2010 17:29:02 +0200 Subject: [erlang-questions] using guards in gen_server In-Reply-To: <7ea62c4b-ddbe-43cc-8f97-abe507b19d33@l20g2000yqm.googlegroups.com> References: <7ea62c4b-ddbe-43cc-8f97-abe507b19d33@l20g2000yqm.googlegroups.com> Message-ID: It depends on where you want the error caught and what is to be done. If you put the guard in the API then the caller will crash, or if you add an extra clause return some for of error. I would most definitely let it crash as it a clear programming error if it happens. If you put the guard in the server then the server will crash or the error will have to be handled there. This doesn't seem right in this case but it does depend on what is right for the application. It all comes back to the central problem with errors: where are they to be detected and "handled". And this is very application dependent. I personally feel that type errors, like this one, should be caught as early as possible and generate and exception as they are usually caused by programming errors. Robert On 29 August 2010 11:20, Zvi wrote: > Hi, > > When writing gen_server, do you put validation guards in API functions > or in handle_call / cast or in both ? What's the convention? > > For example: > > send(Pid) when is_pid(Pid) -> > ? gen_server:call(?SERVER, {send, Pid}). > > handle_call({send, Pid}, _From, _State) -> > ? Resp = do_send(Pid), > ? {reply, ?Resp, State}. > > OR: > > send(Pid) -> > ? gen_server:call(?SERVER, {send, Pid}). > > handle_call({send, Pid}, _From, _State) when is_pid(Pid) -> > ? Resp = do_send(Pid), > ? {reply, ?Resp, State}. > > OR: > > send(Pid) when is_pid(Pid) -> > ? gen_server:call(?SERVER, {send, Pid}). > > handle_call({send, Pid}, _From, _State) when is_pid(Pid) -> > ? Resp = do_send(Pid), > ? {reply, ?Resp, State}. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From attila.r.nohl@REDACTED Sun Aug 29 18:46:16 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Sun, 29 Aug 2010 18:46:16 +0200 Subject: [erlang-questions] using guards in gen_server In-Reply-To: References: <7ea62c4b-ddbe-43cc-8f97-abe507b19d33@l20g2000yqm.googlegroups.com> Message-ID: I'd like to add one note: a badarg or function_clause error in a handle_call function might be hard to understand, because there might be many handle_call function clauses. So either crash in the API call or put the application logic into a do_something() function called from a handle_call and put the guard there (if possible). 2010/8/29, Robert Virding : > It depends on where you want the error caught and what is to be done. > > If you put the guard in the API then the caller will crash, or if you > add an extra clause return some for of error. I would most definitely > let it crash as it a clear programming error if it happens. > > If you put the guard in the server then the server will crash or the > error will have to be handled there. This doesn't seem right in this > case but it does depend on what is right for the application. > > It all comes back to the central problem with errors: where are they > to be detected and "handled". And this is very application dependent. > I personally feel that type errors, like this one, should be caught as > early as possible and generate and exception as they are usually > caused by programming errors. > > Robert > > On 29 August 2010 11:20, Zvi wrote: >> Hi, >> >> When writing gen_server, do you put validation guards in API functions >> or in handle_call / cast or in both ? What's the convention? >> >> For example: >> >> send(Pid) when is_pid(Pid) -> >> gen_server:call(?SERVER, {send, Pid}). >> >> handle_call({send, Pid}, _From, _State) -> >> Resp = do_send(Pid), >> {reply, Resp, State}. >> >> OR: >> >> send(Pid) -> >> gen_server:call(?SERVER, {send, Pid}). >> >> handle_call({send, Pid}, _From, _State) when is_pid(Pid) -> >> Resp = do_send(Pid), >> {reply, Resp, State}. >> >> OR: >> >> send(Pid) when is_pid(Pid) -> >> gen_server:call(?SERVER, {send, Pid}). >> >> handle_call({send, Pid}, _From, _State) when is_pid(Pid) -> >> Resp = do_send(Pid), >> {reply, Resp, State}. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ulf.wiger@REDACTED Sun Aug 29 21:33:56 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 29 Aug 2010 21:33:56 +0200 Subject: [erlang-questions] using guards in gen_server In-Reply-To: <7ea62c4b-ddbe-43cc-8f97-abe507b19d33@l20g2000yqm.googlegroups.com> References: <7ea62c4b-ddbe-43cc-8f97-abe507b19d33@l20g2000yqm.googlegroups.com> Message-ID: <4C7AB624.1070102@erlang-solutions.com> On 29/08/2010 11:20, Zvi wrote: > Hi, > > When writing gen_server, do you put validation guards in API functions > or in handle_call / cast or in both ? What's the convention? A convention I use, for cases where I really can't know until in the handle_call whether the input is ok, is to e.g. reply 'badarg' and handle it in a wrapper around gen_server:call(): http://github.com/esl/gproc/blob/master/src/gproc.erl#L849 call(Req) -> call(Req, l). call(Req, l) -> chk_reply(gen_server:call(?MODULE, Req), Req); call(Req, g) -> chk_reply(gproc_dist:leader_call(Req), Req). chk_reply(Reply, Req) -> case Reply of badarg -> erlang:error(badarg, Req); Reply -> Reply end. A slight disadvantage is that the error is reported as happening in chk_reply/2 rather than in the API function. This could be fixed e.g. by adding some ugly macros. The server side might then look like: http://github.com/esl/gproc/blob/master/src/gproc.erl#L764 handle_call({reg, {_T,l,_} = Key, Val}, {Pid,_}, S) -> case try_insert_reg(Key, Val, Pid) of true -> gproc_lib:ensure_monitor(Pid,l), {reply, true, S}; false -> {reply, badarg, S} end; ... BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From steven.charles.davis@REDACTED Mon Aug 30 00:06:11 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 29 Aug 2010 15:06:11 -0700 (PDT) Subject: using guards in gen_server In-Reply-To: <7ea62c4b-ddbe-43cc-8f97-abe507b19d33@l20g2000yqm.googlegroups.com> References: <7ea62c4b-ddbe-43cc-8f97-abe507b19d33@l20g2000yqm.googlegroups.com> Message-ID: I think it was Joe A who said something similar to: "crash early, crash often". I think it was Robert V who said: "check at the boundary". Both are excellent insights, and together this means to me: Ensure the type input is right at the boundary of the system, and let it crash inside if anything gets into an unexpected state. - so the right place to check the type would the API function, I believe. /s On Aug 29, 4:20?am, Zvi wrote: > Hi, > > When writing gen_server, do you put validation guards in API functions > or in handle_call / cast or in both ? What's the convention? > > For example: > > send(Pid) when is_pid(Pid) -> > ? ?gen_server:call(?SERVER, {send, Pid}). > > handle_call({send, Pid}, _From, _State) -> > ? ?Resp = do_send(Pid), > ? ?{reply, ?Resp, State}. > > OR: > > send(Pid) -> > ? ?gen_server:call(?SERVER, {send, Pid}). > > handle_call({send, Pid}, _From, _State) when is_pid(Pid) -> > ? ?Resp = do_send(Pid), > ? ?{reply, ?Resp, State}. > > OR: > > send(Pid) when is_pid(Pid) -> > ? ?gen_server:call(?SERVER, {send, Pid}). > > handle_call({send, Pid}, _From, _State) when is_pid(Pid) -> > ? ?Resp = do_send(Pid), > ? ?{reply, ?Resp, State}. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED From anthonym@REDACTED Mon Aug 30 00:27:00 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Sun, 29 Aug 2010 15:27:00 -0700 Subject: [erlang-questions] Using ETS for large amounts of data? In-Reply-To: <50021.64.81.32.110.1282977701.squirrel@duomark.com> References: <50021.64.81.32.110.1282977701.squirrel@duomark.com> Message-ID: <20100829222700.GF23023@alumni.caltech.edu> I'm not sure if I ever posted what I did, I basically created two files using perl, one was a binary containing ~10 million 26 byte records, the other was a sorted list containing set of 2 integers. The first the start of a range, the second an offset into the data file. I then had an escript which transformed the index into an ets ordered_set. I then have a gen_server which loads up the ets table using ets:file2tab/1 and the binary using file:read_file/1, and store them both in the gen_server state. I then have a call which looks up the offset by {Start, Offset} = case ets:lookup (Index, Val) of [] -> case ets:lookup (Index, ets:prev (Index, Val)) of [] -> {0, -1}; V -> hd (V) end; V -> hd (V) end, I then lookup the record in the binary via <<_:Offset/binary, Record:26/binary, _/binary>> = Binary Thus I only have one copy of the binary, as well as an ets table. Memory wise the largest part is the ets table, but it's only about 700-800M. Then with the binary at about 250M I actually see something like 1.1G or so of memory. I didn't try to find out if just storing the 26 bytes as the values in the ets table was more efficient memory wise, but what I have now seems to work well. -Anthony On Fri, Aug 27, 2010 at 11:41:41PM -0700, jay@REDACTED wrote: > You have two issues here the initial loading of the data, and the actions > which happen on lookup. > > If you use and ETS table, you have to use integer index offsets as the > return value or the binary data will be copied into the ETS on insert and > out again on every lookup match. The index offset should be fast, but it > is a repeated binary match step that is really unnecessary. > > Try the following, you will probably get better overall performance with > other things running and putting pressure on memory: > > 1) Read in the whole file as a single binary > 2) Never append or modify this binary so it doesn't get copied > 3) Iterate over it using bit syntax to match 26-byte segments > - [ Rec || <> <= FileBinary ] > - You will get sub-binaries which are pointers into #1 > 3) Store each sub-binary as the value for a key in a tree or dict > > You will probably do better in the long run by storing the intervals in a > tree, dictionary, tuple or array. Lookups might be slower (although > depending on the key you might be surprised at how little difference there > is), but there should be no memory copying and thus no future garbage > collection pressure. The pre-fetched sub-binaries will never incur a > binary match penalty, nor copying and should avoid all garbage collection. > > The underlying large binary will always be present, but it will be stored > in the shared binary heap rather than internal to your process so message > passing the sub-binaries should be small and fast as well (all receivers > on the same node will point to the same underlying large binary in the > single binary heap). You could keep the lookup tree in a process and send > messages out on requests for lookup. You just can't distribute it off a > single erlang node, but you should get good benefit on a multi-core > processor. > > jay > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- ------------------------------------------------------------------------ Anthony Molinaro From wiener.guy@REDACTED Mon Aug 30 08:49:17 2010 From: wiener.guy@REDACTED (Guy Wiener) Date: Mon, 30 Aug 2010 09:49:17 +0300 Subject: [erlang-questions] ACM Erlang Workshop: accepted papers & schedule In-Reply-To: <1589.1282623121@snookles.snookles.com> References: <1589.1282623121@snookles.snookles.com> Message-ID: Is there a timetable for this program, or at least an approximated end time? Thanks, Guy On Tue, Aug 24, 2010 at 7:12 AM, Scott Lystig Fritchie < fritchie@REDACTED> wrote: > Hi, all. While I work on getting the > http://www.erlang.org/workshop/2010/* documents updated, here's a list > of the accepted papers and schedule for the ACM Erlang Workshop 2010. I > need to apologize to at least one erlang-questions reader who emailed me > directly to ask about the schedule. Sorry! Hopefully the workshop > Web-a-thingie will be updated in the next day or so. > > -Scott > > --- snip --- snip --- snip --- snip --- snip --- snip --- snip --- > > Ninth ACM SIGPLAN > Erlang Workshop > Baltimore, Maryland, USA, September 30, 2010 > http://www.erlang.org/workshop/2010/ > > Satellite event of the 15th ACM SIGPLAN > International Conference on Functional Programming, > September 27-29, 2010 > > Workshop Program > ---------------- > > Keynote Address > Session Chair: Kostis Sagonas, National Technical University of > Athens > * Invited Talk: David Smith, Basho Technologies: "Concurrent Life, > the Universe, and Everything Rebar" > > Testing and Programming > Session Chair: Ulf Wiger, Erlang Solutions, London, UK. > * From Test Cases to FSMs: Augmented Test Driven Development and > Property Inference. Thomas Arts and Simon Thompson > * Coordinating and Visualizing Independent Behaviors in Erlang. Guy > Wiener, Gera Weiss, and Assaf Marron > > Theory and Practice > Session Chair: Simon Thompson, University of Kent, Canterbury, Kent, UK. > * A Unified Semantics for Future Erlang. Hans Svensson, Lars-?ke > Fredlund, and Clara Benac Earle > * Chain Replication In Theory and in Practice. Scott Lystig Fritchie > > Language Issues and Extensions > Session Chair: Erik Stenman, Klarna AB, Stockholm, Sweden. > * Analysis of Preprocessor Constructs in Erlang. R?bert Kitlei, > Istv?n Boz?, Tam?s Kozsik, M?t? Tejfel, and Melinda T?th > * Generic Load Regulation Framework for Erlang. Ulf Wiger > > Implementations, Tools, and Demos > Session Chair: Scott Lystig Fritchie, Basho Technologies, Cambridge, > Massachusetts, USA > * Implementing a Multiagent Negotiation Protocol in Erlang. Alvaro > Fernandez, Clara Benac Earle, and Lars-?ke Fredlund > * QuickChecking Refactoring Tools. D?niel Drienyovszky, D?niel > Horp?csi, and Simon Thompson > * Using Erlang to Implement an Autonomous Build and Distribution > System for Software Projects. Tino Breddin > > Latest news from the Erlang/OTP team at Ericsson > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From magda.mansour@REDACTED Mon Aug 30 09:50:55 2010 From: magda.mansour@REDACTED (Magda Mansour) Date: Mon, 30 Aug 2010 09:50:55 +0200 Subject: rpc:call has a delay in response when calling a nodedown Message-ID: <4C7B62DF.4030709@myriadgroup.com> Hello, We are encountering the following problem : The command below responds after a few seconds (~7 seconds) rpc:call(dummy_node@REDACTED, erlang, whereis, [dummy_process]). {badrpc,nodedown} where dummy_host is defined in /etc/hosts but is unreachable Adding a timeout in the fourth argument does not fasten the response of this command : rpc:call(dummy_node@REDACTED, erlang, whereis, [dummy_process], 1000). {badrpc,nodedown} We are using OTP-R11B-5 on Red Hat Enterprise Linux ES release 4 (Nahant Update 8) or Ubuntu As a work around, we were obliged to check if dummy_node@REDACTED is part of [node()|nodes()] before calling rpc:call. Is this delay considered as the normal behaviour of rpc:call ? Is there another work around or an alternative function to use ? This example seems to be dummy, but on a live production cluster, where one node has a problem, this delay can lead to real problems. Thank you in advance, Magda Mansour This message, including attachments, is intended solely for the addressee indicated in this message and is strictly confidential or otherwise privileged. If you are not the intended recipient (or responsible for delivery of the message to such person) : - (1) please immediately (i) notify the sender by reply email and (ii) delete this message and attachments, - (2) any use, copy or dissemination of this transmission is strictly prohibited. If you or your employer does not consent to Internet email messages of this kind, please advise Myriad Group AG by reply e-mail immediately. Opinions, conclusions and other information expressed in this message are not given or endorsed by Myriad Group AG unless otherwise indicated by an authorized representative independent of this message. From mazen.harake@REDACTED Mon Aug 30 09:27:45 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 30 Aug 2010 10:27:45 +0300 Subject: [erlang-questions] rpc:call has a delay in response when calling a nodedown In-Reply-To: <4C7B62DF.4030709@myriadgroup.com> References: <4C7B62DF.4030709@myriadgroup.com> Message-ID: <4C7B5D71.5020403@erlang-solutions.com> If the system is down it will take a few seconds (sometimes much more) to know that if you use rpc:call/4,5 straight away. Adding a timeout is only useful when the call has gone through to the other side, it does not take the connection time into account (e.g. rpc:call(Node, timer, sleep, [3000], 1000).). A tip could be to have a connection pool which monitor nodes and abstract the rpc call so that you have one place where you keep track of available nodes and always (try) to connect to nodes that are available (there are corner cases of course). If you are load balancing you probably already have this; you are getting the nodename from somewhere so you could have that somewhere keep track of which node to supply. erlang:monitor_node/2 etc could be used for this. Another thing to consider is that it doesn't matter how careful you are; networks are always hard to predict and can fluctuate A LOT. If you create the system in a way that 7 seconds (in opposed to 1-2 seconds) "lead to real problems" then I would argue that you are doing it wrong. The system has to be resilient enough to fall back on alternative nodes in case one node is unresponsive even if it takes 120 seconds (TCP/IP timeout). I don't know how hard your requirements are on these things but making assumptions on network ping and availability is extremely difficult and is best countered by letting them be part of the design. /Mazen On 30/08/2010 10:50, Magda Mansour wrote: > Hello, > > We are encountering the following problem : > > The command below responds after a few seconds (~7 seconds) > rpc:call(dummy_node@REDACTED, erlang, whereis, [dummy_process]). > {badrpc,nodedown} > where dummy_host is defined in /etc/hosts but is unreachable > > Adding a timeout in the fourth argument does not fasten the response > of this command : > rpc:call(dummy_node@REDACTED, erlang, whereis, [dummy_process], > 1000). > {badrpc,nodedown} > > We are using OTP-R11B-5 on Red Hat Enterprise Linux ES release 4 (Nahant > Update 8) or Ubuntu > > As a work around, we were obliged to check if dummy_node@REDACTED is > part of [node()|nodes()] before calling rpc:call. > > Is this delay considered as the normal behaviour of rpc:call ? Is there > another work around or an alternative function to use ? > > This example seems to be dummy, but on a live production cluster, where > one node has a problem, this delay can lead to real problems. > > Thank you in advance, > Magda Mansour > > > This message, including attachments, is intended solely for the > addressee indicated in this message and is strictly confidential or > otherwise privileged. If you are not the intended recipient (or > responsible for delivery of the message to such person) : - (1) please > immediately (i) notify the sender by reply email and (ii) delete this > message and attachments, - (2) any use, copy or dissemination of this > transmission is strictly prohibited. If you or your employer does not > consent to Internet email messages of this kind, please advise Myriad > Group AG by reply e-mail immediately. Opinions, conclusions and other > information expressed in this message are not given or endorsed by > Myriad Group AG unless otherwise indicated by an authorized > representative independent of this message. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From mazen.harake@REDACTED Mon Aug 30 09:38:08 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 30 Aug 2010 10:38:08 +0300 Subject: [erlang-questions] using guards in gen_server In-Reply-To: References: <7ea62c4b-ddbe-43cc-8f97-abe507b19d33@l20g2000yqm.googlegroups.com> Message-ID: <4C7B5FE0.3010308@erlang-solutions.com> I have as a design principal to never crash inside a gen_server if it is a central process. I.e. If the process that is handling these calls is some how serving many other processes or is somehow in a "central" position then it should be avoided to crash (in general). The reason for this is that you shouldn't let one process bring down a central one because it will (potentially) propagate to other processes (by calling a server that is crashing and/or restarting etc). Note, however, that this does not mean that I put useless checks everywhere in the gen_server to keep it from crashing but rather treat all data as "safe" as long as I have allowed it inside. I agree that it doesn't seem right to crash inside the gen_server in this case and I believe that in general it is best to crash the caller. If you have checked the caller at the API function you can (should!) assume that it is safe to continue without checking your data. Sometimes it is the case (as Ulf mentions) that you can't know if the call is correct until you are "in" the gen_server but I would argue that these tend to be side-effect-based (I'm not sure that is correct English :P) and in general should be treated differently then type errors (as Robert mentions). In the case of the OP I would suggest the first variant. /Mazen On 29/08/2010 18:29, Robert Virding wrote: > It depends on where you want the error caught and what is to be done. > > If you put the guard in the API then the caller will crash, or if you > add an extra clause return some for of error. I would most definitely > let it crash as it a clear programming error if it happens. > > If you put the guard in the server then the server will crash or the > error will have to be handled there. This doesn't seem right in this > case but it does depend on what is right for the application. > > It all comes back to the central problem with errors: where are they > to be detected and "handled". And this is very application dependent. > I personally feel that type errors, like this one, should be caught as > early as possible and generate and exception as they are usually > caused by programming errors. > > Robert > > On 29 August 2010 11:20, Zvi wrote: >> Hi, >> >> When writing gen_server, do you put validation guards in API functions >> or in handle_call / cast or in both ? What's the convention? >> >> For example: >> >> send(Pid) when is_pid(Pid) -> >> gen_server:call(?SERVER, {send, Pid}). >> >> handle_call({send, Pid}, _From, _State) -> >> Resp = do_send(Pid), >> {reply, Resp, State}. >> >> OR: >> >> send(Pid) -> >> gen_server:call(?SERVER, {send, Pid}). >> >> handle_call({send, Pid}, _From, _State) when is_pid(Pid) -> >> Resp = do_send(Pid), >> {reply, Resp, State}. >> >> OR: >> >> send(Pid) when is_pid(Pid) -> >> gen_server:call(?SERVER, {send, Pid}). >> >> handle_call({send, Pid}, _From, _State) when is_pid(Pid) -> >> Resp = do_send(Pid), >> {reply, Resp, State}. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From hynek@REDACTED Mon Aug 30 11:56:42 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Mon, 30 Aug 2010 11:56:42 +0200 Subject: [erlang-questions] Using ETS for large amounts of data? In-Reply-To: <20100829222700.GF23023@alumni.caltech.edu> References: <50021.64.81.32.110.1282977701.squirrel@duomark.com> <20100829222700.GF23023@alumni.caltech.edu> Message-ID: I have very similar experience. 1> T = ets:new(foo,[private]). 16400 2> [ets:insert(T, [{X,<<"0123456789ABCDEF">>} || X<-lists:seq(1000*(Y-1)+1, 1000*Y)]) || Y <-lists:seq(1,10000)],ok. ok 3> ets:info(T). [{memory,161554561}, {owner,<0.32.0>}, {heir,none}, {name,foo}, {size,10000000}, {node,nonode@REDACTED}, {named_table,false}, {type,set}, {keypos,1}, {protection,private}] 4> 161554561*4/1024/1024. 616.2817420959473 PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 11598 hynek 20 0 1198m 1.1g 2196 S 0 28.6 1:12.34 beam.smp Where those 500MB (1.1G - 600M) comes from? It is 50B per one 16B value. If I put same amount of data in perl hash it take same 1.1GB which is well known for wasting of memory. Anyway: 1> T = ets:new(foo,[private]). 16400 2> [ets:insert(T, [{X,[]} || X<-lists:seq(1000*(Y-1)+1, 1000*Y)]) || Y <-lists:seq(1,10000)],ok. ok 3> ets:info(T). [{memory,101554561}, {owner,<0.32.0>}, {heir,none}, {name,foo}, {size,10000000}, {node,nonode@REDACTED}, {named_table,false}, {type,set}, {keypos,1}, {protection,private}] 4> 101554561*4/1024/1024. 387.39990615844727 RES from top is 471MB so it seems better. 1> T = ets:new(foo,[private]). 16400 2> [ets:insert(T, [{X,X*10,(X+1)*10} || X<-lists:seq(1000*(Y-1)+1, 1000*Y)]) || Y <-lists:seq(1,10000)],ok. ok 3> ets:info(T). [{memory,111554561}, {owner,<0.32.0>}, {heir,none}, {name,foo}, {size,10000001}, {node,nonode@REDACTED}, {named_table,false}, {type,set}, {keypos,1}, {protection,private}] 4> 111554561*4/1024/1024. 425.54687881469727 RES: 547MB So keeping it in one big binary and store only pointers will save you 300 - 400 MB of data depending of length of item (16-26B). Anyway using better tuned k/v storage would be better. On Mon, Aug 30, 2010 at 12:27 AM, Anthony Molinaro wrote: > I'm not sure if I ever posted what I did, I basically created two files > using perl, one was a binary containing ~10 million 26 byte records, the > other was a sorted list containing set of 2 integers. ?The first the start > of a range, the second an offset into the data file. ?I then had an escript > which transformed the index into an ets ordered_set. > > I then have a gen_server which loads up the ets table using ets:file2tab/1 > and the binary using file:read_file/1, and store them both in the gen_server > state. ?I then have a call which looks up the offset by > > {Start, Offset} = > ?case ets:lookup (Index, Val) of > ? ?[] -> > ? ? ?case ets:lookup (Index, ets:prev (Index, Val)) of > ? ? ? ?[] -> {0, -1}; > ? ? ? ?V -> hd (V) > ? ? ?end; > ? ?V -> hd (V) > ?end, > > I then lookup the record in the binary via > > <<_:Offset/binary, Record:26/binary, _/binary>> = Binary > > Thus I only have one copy of the binary, as well as an ets table. ?Memory > wise the largest part is the ets table, but it's only about 700-800M. > Then with the binary at about 250M I actually see something like 1.1G or > so of memory. ?I didn't try to find out if just storing the 26 bytes as > the values in the ets table was more efficient memory wise, but what > I have now seems to work well. > > -Anthony > > On Fri, Aug 27, 2010 at 11:41:41PM -0700, jay@REDACTED wrote: >> You have two issues here the initial loading of the data, and the actions >> which happen on lookup. >> >> If you use and ETS table, you have to use integer index offsets as the >> return value or the binary data will be copied into the ETS on insert and >> out again on every lookup match. ?The index offset should be fast, but it >> is a repeated binary match step that is really unnecessary. >> >> Try the following, you will probably get better overall performance with >> other things running and putting pressure on memory: >> >> 1) Read in the whole file as a single binary >> 2) Never append or modify this binary so it doesn't get copied >> 3) Iterate over it using bit syntax to match 26-byte segments >> ? ?- [ Rec || <> <= FileBinary ] >> ? ?- You will get sub-binaries which are pointers into #1 >> 3) Store each sub-binary as the value for a key in a tree or dict >> >> You will probably do better in the long run by storing the intervals in a >> tree, dictionary, tuple or array. ?Lookups might be slower (although >> depending on the key you might be surprised at how little difference there >> is), but there should be no memory copying and thus no future garbage >> collection pressure. The pre-fetched sub-binaries will never incur a >> binary match penalty, nor copying and should avoid all garbage collection. >> >> The underlying large binary will always be present, but it will be stored >> in the shared binary heap rather than internal to your process so message >> passing the sub-binaries should be small and fast as well (all receivers >> on the same node will point to the same underlying large binary in the >> single binary heap). ?You could keep the lookup tree in a process and send >> messages out on requests for lookup. ?You just can't distribute it off a >> single erlang node, but you should get good benefit on a multi-core >> processor. >> >> jay >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > -- > ------------------------------------------------------------------------ > Anthony Molinaro ? ? ? ? ? ? ? ? ? ? ? ? ? > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From kagato@REDACTED Mon Aug 30 12:01:18 2010 From: kagato@REDACTED (Jayson Vantuyl) Date: Mon, 30 Aug 2010 03:01:18 -0700 Subject: Why Distributed Is So Hard Message-ID: <60497EC4-0B3D-40DB-BA27-B2955A114ADF@souja.net> Hey everybody, While it's not Erlang-specific, I know a lot of people here care a lot about distributed programming. I had a really good question from someone the other day and it spawned a blog post that might interest you. In particular, most distributed programming in Erlang is usually based on the assumption of a highly-available network layer (Mnesia, I'm looking at you). This post talks about why that isn't good enough for distributed, in general, and gives a way to think about the problem specifically. http://is.gd/eLhKF Thanks, -- Jayson Vantuyl 417-207-6962 (mobile) kagato@REDACTED From xramtsov@REDACTED Mon Aug 30 12:27:56 2010 From: xramtsov@REDACTED (Evgeniy Khramtsov) Date: Mon, 30 Aug 2010 20:27:56 +1000 Subject: [erlang-questions] Why Distributed Is So Hard In-Reply-To: <60497EC4-0B3D-40DB-BA27-B2955A114ADF@souja.net> References: <60497EC4-0B3D-40DB-BA27-B2955A114ADF@souja.net> Message-ID: <4C7B87AC.7020304@gmail.com> 30.08.2010 20:01, Jayson Vantuyl wrote > In particular, most distributed programming in Erlang is usually based on the assumption of a highly-available network layer (Mnesia, I'm looking at you). Well, the main problem is network split. But I agree, distributed mnesia tables are not a good way to scale, especially if you have lots of nodes. > This post talks about why that isn't good enough for distributed, in general, and gives a way to think about the problem specifically. > Everybody knows that this approach has several drawbacks, but, due to CAP-theorem, we don't have lots of choices - there is no silver bullet for this subject. -- Regards, Evgeniy Khramtsov, ProcessOne. xmpp:xram@REDACTED From kagato@REDACTED Mon Aug 30 12:53:30 2010 From: kagato@REDACTED (Jayson Vantuyl) Date: Mon, 30 Aug 2010 03:53:30 -0700 Subject: [erlang-questions] Why Distributed Is So Hard In-Reply-To: <4C7B87AC.7020304@gmail.com> References: <60497EC4-0B3D-40DB-BA27-B2955A114ADF@souja.net> <4C7B87AC.7020304@gmail.com> Message-ID: <7C66AA70-58D0-4546-9E57-E1095E360FBA@souja.net> Well, I didn't mean to single out Mnesia specifically as being bad. Rather, most people make assumptions when discussing things. For example, when Mnesia has issues during a netsplit, it's often painful to even try to talk about the tradeoffs because people don't have a frame of reference. In my post, I related that a coworker had asked me why distributed programming was any different than threaded programming. His attitude was that it was just concurrency, right? Well, we both know there's more, but I found that this question was a really good way to expose the differences between the two, and then blogged about it. On Aug 30, 2010, at 3:27 AM, Evgeniy Khramtsov wrote: > 30.08.2010 20:01, Jayson Vantuyl wrote >> In particular, most distributed programming in Erlang is usually based on the assumption of a highly-available network layer (Mnesia, I'm looking at you). > > Well, the main problem is network split. But I agree, distributed mnesia tables are not a good way to scale, especially if you have lots of nodes. > >> This post talks about why that isn't good enough for distributed, in general, and gives a way to think about the problem specifically. >> > > Everybody knows that this approach has several drawbacks, but, due to CAP-theorem, we don't have lots of choices - there is no silver bullet for this subject. > > -- > Regards, > Evgeniy Khramtsov, ProcessOne. > xmpp:xram@REDACTED > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Jayson Vantuyl 417-207-6962 (mobile) kagato@REDACTED From toby@REDACTED Mon Aug 30 16:09:36 2010 From: toby@REDACTED (Toby Thain) Date: Mon, 30 Aug 2010 10:09:36 -0400 Subject: [erlang-questions] Why Distributed Is So Hard In-Reply-To: <60497EC4-0B3D-40DB-BA27-B2955A114ADF@souja.net> References: <60497EC4-0B3D-40DB-BA27-B2955A114ADF@souja.net> Message-ID: <5CFC2628-A788-475E-9A45-7F11948BE6C7@telegraphics.com.au> On 30-Aug-10, at 6:01 AM, Jayson Vantuyl wrote: > Hey everybody, > > While it's not Erlang-specific, I know a lot of people here care a > lot about distributed programming. I had a really good question > from someone the other day and it spawned a blog post that might > interest you. In particular, most distributed programming in Erlang > is usually based on the assumption of ... you might remember, or wish to cite, this classic: http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Computing --Toby > a highly-available network layer (Mnesia, I'm looking at you). This > post talks about why that isn't good enough for distributed, in > general, and gives a way to think about the problem specifically. > > http://is.gd/eLhKF > > Thanks, > > -- > Jayson Vantuyl > 417-207-6962 (mobile) > kagato@REDACTED > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From raould@REDACTED Mon Aug 30 19:40:29 2010 From: raould@REDACTED (Raoul Duke) Date: Mon, 30 Aug 2010 10:40:29 -0700 Subject: [erlang-questions] Why Distributed Is So Hard In-Reply-To: <5CFC2628-A788-475E-9A45-7F11948BE6C7@telegraphics.com.au> References: <60497EC4-0B3D-40DB-BA27-B2955A114ADF@souja.net> <5CFC2628-A788-475E-9A45-7F11948BE6C7@telegraphics.com.au> Message-ID: On Mon, Aug 30, 2010 at 7:09 AM, Toby Thain wrote: > ... you might remember, or wish to cite, this classic: > http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Computing which has since been expanded upon! distributed computing is even harder than they thought! http://blogs.sun.com/hjfphd/entry/the_9th_fallacy_of_distributed From michael.eugene.turner@REDACTED Tue Aug 31 09:24:21 2010 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 31 Aug 2010 16:24:21 +0900 Subject: [erlang-questions] Why Distributed Is So Hard In-Reply-To: <60497EC4-0B3D-40DB-BA27-B2955A114ADF@souja.net> References: <60497EC4-0B3D-40DB-BA27-B2955A114ADF@souja.net> Message-ID: Hey Jason, nice essay, but I just wanted to say: before DOS there was this OS concept called "time sharing." But don't worry, nothing ever came of it. ;-) It would be interesting to extend Erlang to evaluate its current "boolean expressions" in terms of Relevance Logic -- yielding something like "no information" and "conflicting information" values alongside "true" and "false". There might be interesting implications here, for retrofitting Erlang to selectively provide lazy evaluation, speculative execution and certain pleasing variations on vanilla exception handling. But perhaps the most salutary effect could be "philosophical" in the best (i.e., most practical) way. In what ordinarily seems to be the very simplest form of computation (e.g., "=:="), the problems and potential of distributed computation would always be at least latent. -michael turner On Mon, Aug 30, 2010 at 7:01 PM, Jayson Vantuyl wrote: > Hey everybody, > > While it's not Erlang-specific, I know a lot of people here care a lot > about distributed programming. I had a really good question from someone > the other day and it spawned a blog post that might interest you. In > particular, most distributed programming in Erlang is usually based on the > assumption of a highly-available network layer (Mnesia, I'm looking at you). > This post talks about why that isn't good enough for distributed, in > general, and gives a way to think about the problem specifically. > > http://is.gd/eLhKF > > Thanks, > > -- > Jayson Vantuyl > 417-207-6962 (mobile) > kagato@REDACTED > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From cris.kiev@REDACTED Tue Aug 31 11:07:03 2010 From: cris.kiev@REDACTED (Sergii Boiko) Date: Tue, 31 Aug 2010 02:07:03 -0700 (PDT) Subject: Missing features in Erlang type system and several additional question related to it Message-ID: <472dce18-3969-486a-b42a-e72f0652c517@q22g2000yqm.googlegroups.com> Hi all, I'm working with json using mochiwebjson2 library and need to specialize next type: -type packet_encoding() :: 1 | 0. -type packet() :: [integer(), packet_encoding(), binary()]. But it's turned out, Erlang doesn't have support for lists with fixed data amount and type packet doesn't compiling. If you wondered why I choose list over tuple, it's because in json there is no tuple and i serialize packets from erlang to json via mochiwebjson2, which also doesn't work with tuples. But in any case i guess this is missed feature of Erlang type system and it should be addressed. Another issue is with parametrized erlang modules. I use mochiweb which uses this feature. And for things like Req:method(), Dialyzer thinks that Req is atom. Maybe it even doesn't know about parametrized modules. Maybe, there is some type or additional hint for such cases? I also have issue with Dialyzer and dirty check on dict'ness of value. Maybe there is a point to add some 'true' is_dict guard? because dirty is_record(Term, dict, 8) also made it complain when i lately use some functions from dict module. Looking for examples of type declaration i investigated Erlang sources and found two unknown directives: -opaque and -file. Can someone explain, what are they for? I've also heard some mention about runtime type checking, which can be addition to Dialyzer static checking. But have no luck to find out how to enable it. Maybe someone can point me out which tool is responsible for this functionality? About char() type. In documentation it's described as: char() 0..16#10ffff. Maybe this is some typo? Or what is mean? Here http://schemecookbook.org/Erlang/StringBasics chars described as 4- bytes value. From dmitrii@REDACTED Tue Aug 31 14:29:34 2010 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Tue, 31 Aug 2010 15:29:34 +0300 Subject: worker_pool ... extended Message-ID: I've somewhat extended RabbitMQ's excellent implementation of worker pools (see full description at http://www.lshift.net/blog/2010/03/29/on-the-limits-of-concurrency-worker-pools-in-erlang) Limitations of RabbitMQ's worker pool are as follows: ? only one can be created within the running application ? you have to supply the pool with the function you want to run each time you invoke it My fork allows you to: ? Run several worker pools (each independently named, if needed) ? Make a worker pool execute a specified function every time it's invoked ? If such a function requires initialization of certain parameters or somesuch, you can supply an init function that will be called once the worker pool has been started You can get it at http://github.com/dmitriid/worker_pool There are examples in the README, and the code should be working (despite the fact that I pulled it from a different project) From kostis@REDACTED Tue Aug 31 15:11:29 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 31 Aug 2010 16:11:29 +0300 Subject: [erlang-questions] Missing features in Erlang type system and several additional question related to it In-Reply-To: <472dce18-3969-486a-b42a-e72f0652c517@q22g2000yqm.googlegroups.com> References: <472dce18-3969-486a-b42a-e72f0652c517@q22g2000yqm.googlegroups.com> Message-ID: <4C7CFF81.1010400@cs.ntua.gr> Sergii Boiko wrote: > Hi all, > > I'm working with json using mochiwebjson2 library and need to > specialize next type: > > -type packet_encoding() :: 1 | 0. > -type packet() :: [integer(), packet_encoding(), binary()]. > > But it's turned out, Erlang doesn't have support for lists with fixed > data amount and type packet doesn't compiling. If you wondered why I > choose list over tuple, it's because in json there is no tuple and i > serialize packets from erlang to json via mochiwebjson2, which also > doesn't work with tuples. Your arguments are very weak, but you probably know that already. In Erlang it's tuples and not lists which are supposed to be used for collections of items of known number. If json or mochiwebjson2 or whatever has limitations, then the answer to what you are trying to do is most probably a call to tuple_to_list/1 away... If you want to stick with lists, the closest you currently can get with the current type language is to use the following definition: -type packet() :: [integer() | packet_encoding() | binary()]. which specifies a list which can contain integers, packet_encodings and/or binaries (of any size). This is a type which is more general than the one you want to specify, but it should work OK. > But in any case i guess this is missed feature of Erlang type system > and it should be addressed. Well, that's your opinion. I beg to differ here, but perhaps this is irrelevant. One nice thing about Erlang/OTP is that its implementation is open source and nowadays there is an easy way for users to submit patches for features they feel they should be included into it. > Another issue is with parametrized erlang modules. I use mochiweb > which uses this feature. And for things like Req:method(), Dialyzer > thinks that Req is atom. Maybe it even doesn't know about parametrized > modules. Maybe, there is some type or additional hint for such cases? Which version of dialyzer are you using, by the way? Dialyzer supports parameterized modules since Erlang/OTP R13B04. > I also have issue with Dialyzer and dirty check on dict'ness of value. > Maybe there is a point to add some 'true' is_dict guard? because dirty > is_record(Term, dict, 8) also made it complain when i lately use some > functions from dict module. Yes, here I agree with you. It would be a good idea for the dict module to export an is_dict/1 function (similarly for other modules with opaque data structures). Perhaps the OTP team can add these. Allowing them to be used as guards is an extra bonus, which might happen some day. Kostis From cris.kiev@REDACTED Tue Aug 31 16:41:27 2010 From: cris.kiev@REDACTED (Sergii Boiko) Date: Tue, 31 Aug 2010 07:41:27 -0700 (PDT) Subject: Missing features in Erlang type system and several additional question related to it In-Reply-To: <4C7CFF81.1010400@cs.ntua.gr> References: <472dce18-3969-486a-b42a-e72f0652c517@q22g2000yqm.googlegroups.com> <4C7CFF81.1010400@cs.ntua.gr> Message-ID: <5dd50f3e-51c3-4e95-b530-74211d80e2c9@v41g2000yqv.googlegroups.com> > Your arguments are very weak, but you probably know that already. In > Erlang it's tuples and not lists which are supposed to be used for > collections of items of known number. You've missed the point. I don't argue with anything. I've pointed out in the leak of type system. There is a structure in Erlang, which is correct. You can create it, you can use it, but can't describe it precisely. And at the same time there is much less 'idiomatic' and 'correct' types like 'maybe_improper_list' and 'nonempty_maybe_improper_list' which you can. > If json or mochiwebjson2 or > whatever has limitations, then the answer to what you are trying to do > is most probably a call to tuple_to_list/1 away... It's not a question of library. It's a question of integration with other languages. In Ruby, JavaScript and many others there are no tuples. And instead of tuple arrays is used. So we have tuple=>array=>list conversion as result. And i don't wont to pay performance penalty with call to tuple_to_list when i can avoid it. > If you want to stick with lists, the closest you currently can get with > the current type language is to use the following definition: > > ? -type packet() :: [integer() | packet_encoding() | binary()]. I did it like this. > Which version of dialyzer are you using, by the way? ?Dialyzer supports > parameterized modules since Erlang/OTP R13B04. Thanks for the tip. I'm using R13B03. Will check it out on R13B04. From djk121@REDACTED Tue Aug 31 17:14:38 2010 From: djk121@REDACTED (Dan Kelley) Date: Tue, 31 Aug 2010 11:14:38 -0400 Subject: strptime? Message-ID: HI, I'm an erlang noob. I'm trying write a program in erlang that will replace a program that's currently implemented in python. The program needs to parse XML documents, perform some calculations on the contents, and populate a write-through cache that's backed by a database. Some of the elements that I'm parsing out of the XML documents are date strings, which look like "20091215015928". In the python program, I'm parsing these into datetime.datetime objects, which I then use as dictionary keys in the cache. While working on the erlang version of the program, I tried to find the erlang stdlib date parsing routines. To my surprise, I couldn't find any helper functions for parsing datetime representations out of strings. This seems unlikely enough that there's a chance I'm missing something obvious. Am I? Thanks, Dan From kostis@REDACTED Tue Aug 31 17:45:29 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 31 Aug 2010 18:45:29 +0300 Subject: [erlang-questions] Re: Missing features in Erlang type system and several additional question related to it In-Reply-To: <5dd50f3e-51c3-4e95-b530-74211d80e2c9@v41g2000yqv.googlegroups.com> References: <472dce18-3969-486a-b42a-e72f0652c517@q22g2000yqm.googlegroups.com> <4C7CFF81.1010400@cs.ntua.gr> <5dd50f3e-51c3-4e95-b530-74211d80e2c9@v41g2000yqv.googlegroups.com> Message-ID: <4C7D2399.2020809@cs.ntua.gr> Sergii Boiko wrote: >> Your arguments are very weak, but you probably know that already. In >> Erlang it's tuples and not lists which are supposed to be used for >> collections of items of known number. > > You've missed the point. I don't argue with anything. I've pointed > out in the leak of type system. There is a structure in Erlang, which > is correct. You can create it, you can use it, but can't describe it > precisely. Please realize that this is a property of all type languages. In Erlang, one can for example write a function that creates lists whose length is a prime number. Should the type language cater for expressing this? Similarly, some application may need to create binaries whose middle element is 42. How about binaries where segments in positions which are prime numbers are integers divisible by 42? You probably see what I mean... No matter how expressive we make the language of types there will always be some set of Erlang terms that one can create, can use, but cannot describe precisely. It's true that the current type language does not allow you to specify that lists are of some particular length. But I would argue that in *Erlang* these are programs where tuples rather than lists should better be used. >> If json or mochiwebjson2 or >> whatever has limitations, then the answer to what you are trying to do >> is most probably a call to tuple_to_list/1 away... > > It's not a question of library. It's a question of integration with > other > languages. In Ruby, JavaScript and many others there are no tuples. > And instead of tuple arrays is used. So we have tuple=>array=>list > conversion > as result. And i don't wont to pay performance penalty with call to > tuple_to_list when i can avoid it. I do not think the type language of Erlang has to cater for performance concerns of integration with other languages. This is of very low priority to its design. Best, Kostis From gustav.simonsson@REDACTED Tue Aug 31 21:52:58 2010 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Tue, 31 Aug 2010 19:52:58 +0000 (GMT) Subject: [erlang-questions] strptime? In-Reply-To: <1414769282.752901283284298059.JavaMail.root@zimbra> Message-ID: <567307978.752921283284378703.JavaMail.root@zimbra> http://www.erlang.org/doc/man/calendar.html has a bunch of date and time functions. http://www.erlang.org/doc/man/string.html#tokens-2 is useful for parsing strings in general, but as your datetime string doesn't have separator characters you have to parse it "manually". Best Regards, Gustav Simonsson. ----- Original Message ----- From: "Dan Kelley" To: erlang-questions@REDACTED Sent: Tuesday, 31 August, 2010 17:14:38 GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: [erlang-questions] strptime? HI, I'm an erlang noob. I'm trying write a program in erlang that will replace a program that's currently implemented in python. The program needs to parse XML documents, perform some calculations on the contents, and populate a write-through cache that's backed by a database. Some of the elements that I'm parsing out of the XML documents are date strings, which look like "20091215015928". In the python program, I'm parsing these into datetime.datetime objects, which I then use as dictionary keys in the cache. While working on the erlang version of the program, I tried to find the erlang stdlib date parsing routines. To my surprise, I couldn't find any helper functions for parsing datetime representations out of strings. This seems unlikely enough that there's a chance I'm missing something obvious. Am I? Thanks, Dan From gordon@REDACTED Tue Aug 31 22:20:05 2010 From: gordon@REDACTED (Gordon Guthrie) Date: Tue, 31 Aug 2010 21:20:05 +0100 Subject: [erlang-questions] strptime? In-Reply-To: <567307978.752921283284378703.JavaMail.root@zimbra> References: <1414769282.752901283284298059.JavaMail.root@zimbra> <567307978.752921283284378703.JavaMail.root@zimbra> Message-ID: Dale Harvey has published a date/time parser on github as well: http://github.com/daleharvey/erlang_util/blob/master/dh_date.erl Gordon On 31 August 2010 20:52, Gustav Simonsson < gustav.simonsson@REDACTED> wrote: > > http://www.erlang.org/doc/man/calendar.html > has a bunch of date and time functions. > > http://www.erlang.org/doc/man/string.html#tokens-2 > is useful for parsing strings in general, but as > your datetime string doesn't have separator characters > you have to parse it "manually". > > Best Regards, Gustav Simonsson. > > ----- Original Message ----- > From: "Dan Kelley" > To: erlang-questions@REDACTED > Sent: Tuesday, 31 August, 2010 17:14:38 GMT +01:00 Amsterdam / Berlin / > Bern / Rome / Stockholm / Vienna > Subject: [erlang-questions] strptime? > > HI, > > I'm an erlang noob. I'm trying write a program in erlang that will replace > a program that's currently implemented in python. The program needs to > parse XML documents, perform some calculations on the contents, and > populate > a write-through cache that's backed by a database. > > Some of the elements that I'm parsing out of the XML documents are date > strings, which look like "20091215015928". In the python program, I'm > parsing these into datetime.datetime objects, which I then use as > dictionary > keys in the cache. > > While working on the erlang version of the program, I tried to find the > erlang stdlib date parsing routines. To my surprise, I couldn't find any > helper functions for parsing datetime representations out of strings. This > seems unlikely enough that there's a chance I'm missing something obvious. > Am I? > > Thanks, > > Dan > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669