From fritchie@REDACTED Tue Feb 1 00:06:03 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Mon, 31 Jan 2011 17:06:03 -0600 Subject: Update to the "cluster_info" app Message-ID: <33781.1296515163@snookles.snookles.com> Hi, all. In response to a customer problem, I've pushed a bugfix(*) to the cluster_info app over at GitHub. I also added a long-overdue README file and also added a small sample report to the repository (in case you haven't looked at the code but have wondered what all the buzz is about). https://github.com/basho/cluster_info -Scott (*) Note that the bugfix also requires the "riak_err" application to perform its string truncation magic. If anyone would like to rewrite the patch to be able to use trunc_io.erl directly (by Matthias Lang) or some other formatting+truncation library, feel free to send a patch & pull request to us. :-) Both apps are covered by Apache Public License 2.0. From pmorreale@REDACTED Tue Feb 1 00:17:28 2011 From: pmorreale@REDACTED (Peter W. Morreale) Date: Mon, 31 Jan 2011 16:17:28 -0700 Subject: Questions about OTP debugging... Message-ID: <1296515848.11676.89.camel@hermosa.morreale.net> I'm new to Erlang and OTP and have some general questions about debugging. I'm learning erland and otp with the books: Programming Erlang" and "Erlang and OTP". Currently I'm working on the "tcp_rpc" example in the OTP book. I have the tr_server module written and working according to the example in the book. Now I'm trying to add a supervisor and application to create a complete OTP application. At the moment I have all the source created and compiled, as referenced by the code in the book. When I attempt to start the application, I get the following: pmorreale@REDACTED:~/erlang> cd rpc pmorreale@REDACTED:~/erlang/rpc> erl -pa ebin Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) 1> application:start(tcp_rpc). {error,{not_loaded,tcp_rpc}} 2> The question is how to debug this? Please note that I'm less interested in getting this app to work, than to learn debugging strategies related to erlang and OTP. Is there some flag/etc available to turn on logging or tracing? Thanks, -PWM From taylor@REDACTED Tue Feb 1 02:46:26 2011 From: taylor@REDACTED (Taylor Venable) Date: Mon, 31 Jan 2011 20:46:26 -0500 Subject: edoc: infinite loop In-Reply-To: References: Message-ID: On Sat, Jan 29, 2011 at 22:01, Taylor Venable wrote: > Hi there, the following seems to produce an infinite loop in edoc > (version 0.7.6.8): > > %% START FILE > %% @doc > %% ``` > > %% text > %% ''' > > -module(test). > -export([foo/0]). > > foo() -> ok. > %% END FILE The problem comes from edoc_wiki:strip_empty_lines/2 - edoc_wiki:expand/3 passes it an empty list in this case, and that causes strip_empty_spaces to infinitely recurse. -- Taylor C. Venable http://metasyntax.net/ From bengt.kleberg@REDACTED Tue Feb 1 07:44:07 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 1 Feb 2011 07:44:07 +0100 Subject: [erlang-questions] Questions about OTP debugging... In-Reply-To: <1296515848.11676.89.camel@hermosa.morreale.net> References: <1296515848.11676.89.camel@hermosa.morreale.net> Message-ID: <1296542647.5230.5.camel@seasc1137> Greetings, Your error message seems to indicate that the tcp_rpc module is not loaded. Since Erlang loads beam files automatically if they are in the path this could be two problems. 1 You do not have the right path 2 You do not have the beam file Can you locate the file tcp_rpc.beam on your system? When you have started Erlang, can you find the directory where the bream file resides in your path (see code:get_path/0)? bengt On Tue, 2011-02-01 at 00:17 +0100, Peter W. Morreale wrote: > I'm new to Erlang and OTP and have some general questions about > debugging. I'm learning erland and otp with the books: Programming > Erlang" and "Erlang and OTP". > > Currently I'm working on the "tcp_rpc" example in the OTP book. I have > the tr_server module written and working according to the example in the > book. Now I'm trying to add a supervisor and application to create a > complete OTP application. > > At the moment I have all the source created and compiled, as referenced > by the code in the book. When I attempt to start the application, I get > the following: > > > pmorreale@REDACTED:~/erlang> cd rpc > pmorreale@REDACTED:~/erlang/rpc> erl -pa ebin > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.1 (abort with ^G) > 1> application:start(tcp_rpc). > {error,{not_loaded,tcp_rpc}} > 2> > > The question is how to debug this? Please note that I'm less interested > in getting this app to work, than to learn debugging strategies related > to erlang and OTP. > > Is there some flag/etc available to turn on logging or tracing? > > Thanks, > -PWM > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From g9414002.pccu.edu.tw@REDACTED Tue Feb 1 08:00:52 2011 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Tue, 1 Feb 2011 15:00:52 +0800 Subject: Beepbeep web framework on Windows Message-ID: Hi, all. I tried an Erlang web framework named beepbeep on Windows by using MinGW32/MSYS, and there's a problem after building beepbeep when it issues a message of lacking a command `mklink'. And then new beepbeep apps won't be built without links to some header files. I know there is mklink command in Windows Vista or 7 but Windows XP. Do you know how to solve the problem? -- Best Regards. --- Y-H. H. From mazen.harake@REDACTED Tue Feb 1 10:32:04 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 01 Feb 2011 10:32:04 +0100 Subject: [erlang-questions] Questions about OTP debugging... In-Reply-To: <1296515848.11676.89.camel@hermosa.morreale.net> References: <1296515848.11676.89.camel@hermosa.morreale.net> Message-ID: <4D47D314.5010504@erlang-solutions.com> Hi Peter, Perhaps my tutorial from the Erlang User Conference will help you out: http://www.erlang-factory.com/conference/testingtutorialworkshop2010/speakers/MazenHarake Good luck. BR /M On 01/02/2011 00:17, Peter W. Morreale wrote: > I'm new to Erlang and OTP and have some general questions about > debugging. I'm learning erland and otp with the books: Programming > Erlang" and "Erlang and OTP". > > Currently I'm working on the "tcp_rpc" example in the OTP book. I have > the tr_server module written and working according to the example in the > book. Now I'm trying to add a supervisor and application to create a > complete OTP application. > > At the moment I have all the source created and compiled, as referenced > by the code in the book. When I attempt to start the application, I get > the following: > > > pmorreale@REDACTED:~/erlang> cd rpc > pmorreale@REDACTED:~/erlang/rpc> erl -pa ebin > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.1 (abort with ^G) > 1> application:start(tcp_rpc). > {error,{not_loaded,tcp_rpc}} > 2> > > The question is how to debug this? Please note that I'm less interested > in getting this app to work, than to learn debugging strategies related > to erlang and OTP. > > Is there some flag/etc available to turn on logging or tracing? > > Thanks, > -PWM > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ingela.andin@REDACTED Tue Feb 1 12:31:00 2011 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 1 Feb 2011 12:31:00 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> Message-ID: Hi! We are getting there thank you for your good work, and with a few more adjustments we will include it. I would like you to change the function pem_entry_encode in the following way pem_entry_encode('SubjectPublicKeyInfo', Entity=#'RSAPublicKey'{}) -> %% The function clause that you implemented for RSA pem_entry_encode('SubjectPublicKeyInfo', {DsaInt, Params=#'Dss-Parms'{}}) when is_integer(DsaInt) -> %% The function clause that you implemented for DSA pem_entry_encode('RSAPublicKey', Entity=#'RSAPublicKey'{}) -> %% This clause I think should not need to be special but should be able to go as default %% and create an "BEGIN RSA PUBLIC KEY" entry. And finally include the trivial patch from Jocke. RSA-keys does not have any parameters hence the value NULL value "<<5,0>>" you could make a constante for it calling it DER_NULL. And also a test case for the "BEGIN RSA PUBLIC KEY" maybe you could use Jokes example. Regards Ingela Erlang/OTP team - Ericsson AB 2011/1/31 Seth Falcon : > Hi Ingela, > > I've updated my patch: > > ? ?https://github.com/seth/otp/tree/sf/rsa_pub_key > > it now includes basic tests for decoding and encoding both RSA and DSA > public keys in the SubjectPublicKeyInfo format. ?After moving all > lib/FOO/test directories with references to wx, I was able to run the > tests for public_key on my OS X laptop :-) > > Here's the summary of where I ended up: > > ? ?{ok, RSAPubPem} = file:read_file(filename:join(Datadir, "rsa_pub.pem")), > ? ?[{'SubjectPublicKeyInfo', _, _} = PubEntry1] = > ? ? ? ?public_key:pem_decode(RSAPubPem), > ? ?RSAPubKey = public_key:pem_entry_decode(PubEntry1), > ? ?true = check_entry_type(RSAPubKey, 'RSAPublicKey'), > ? ?PubEntry1 = public_key:pem_entry_encode('RSAPublicKey', RSAPubKey), > > Notes: > > - I haven't merged in the trivial patch from Joakim to allow reading > ?RSA public keys in "unwrapped" form where the PEM header has "RSA > ?Public Key". > > - RSA public key records get encoded into the SubjectPublicKeyInfo > ?wrapper form. ?So to create the form that Joakim is dealing with, > ?one would have to use der_encode directly I believe. > > - When encoding RSAPublic keys, there is a parameter spot where it > ?seems the right thing to put is <<5, 0>>, but I wanted to mention > ?this in case I'm wrong about this being the right value for all RSA > ?public keys in SPKI form. > > Let me know what else I can do to help get this patch into the > public_key module :-) > > Best, > > + seth > > -- > Seth Falcon | @sfalcon | http://userprimary.net/ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From alexey.v.romanov@REDACTED Tue Feb 1 13:36:21 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Tue, 1 Feb 2011 15:36:21 +0300 Subject: econnrefused on gen_tcp:connect Message-ID: I am getting {error,econnrefused} on any call to gen_tcp:connect even though 1) epmd is running and epmd -names returns the correct result; 2) ping works fine. # HOME=/mware/cookie erts-5.8.1/bin/erl -sname aname Eshell V5.8.1 (abort with ^G) (aname@REDACTED)1> gen_tcp:connect("localhost", 1234, []). {error,econnrefused} Same result for {127,0,0,1}, other computers on the network, etc. # /mware/erts-5.8.1/bin/epmd -names epmd: up and running on port 4369 with data: name aname at port 34272 # ifconfig eth0 Link encap:Ethernet HWaddr 00:90:A9:94:BD:BD inet addr:192.168.1.202 Bcast:192.168.1.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:10257 errors:1 dropped:0 overruns:0 frame:1 TX packets:4480 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:256 RX bytes:9303332 (8.8 MiB) TX bytes:671790 (656.0 KiB) Interrupt:46 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:4900 errors:0 dropped:0 overruns:0 frame:0 TX packets:4900 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:331668 (323.8 KiB) TX bytes:331668 (323.8 KiB) # ping 192.168.1.9 PING 192.168.1.9 (192.168.1.9): 56 data bytes 64 bytes from 192.168.1.9: seq=0 ttl=64 time=0.342 ms 64 bytes from 192.168.1.9: seq=1 ttl=64 time=0.254 ms What could be the reason? Yours, Alexey Romanov From rtrlists@REDACTED Tue Feb 1 13:49:54 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 1 Feb 2011 12:49:54 +0000 Subject: [erlang-questions] econnrefused on gen_tcp:connect In-Reply-To: References: Message-ID: On Tue, Feb 1, 2011 at 12:36 PM, Alexey Romanov wrote: > I am getting {error,econnrefused} on any call to gen_tcp:connect even > though 1) epmd is running and epmd -names returns the correct result; > 2) ping works fine. > > # HOME=/mware/cookie erts-5.8.1/bin/erl -sname aname > Eshell V5.8.1 (abort with ^G) > (aname@REDACTED)1> gen_tcp:connect("localhost", 1234, []). > {error,econnrefused} > > Same result for {127,0,0,1}, other computers on the network, etc. > > Do you have a server listening on port 1234? Robby From alexey.v.romanov@REDACTED Tue Feb 1 13:56:55 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Tue, 1 Feb 2011 15:56:55 +0300 Subject: [erlang-questions] econnrefused on gen_tcp:connect In-Reply-To: References: Message-ID: Argh. Apparently I don't. Should have, but don't (the actual server I tested it on was turned off, and I forgot to start one when testing the local connection). Sorry for the confusion. Yours, Alexey Romanov On Tue, Feb 1, 2011 at 3:49 PM, Robert Raschke wrote: > > On Tue, Feb 1, 2011 at 12:36 PM, Alexey Romanov > wrote: >> >> I am getting {error,econnrefused} on any call to gen_tcp:connect even >> though 1) epmd is running and epmd -names returns the correct result; >> 2) ping works fine. >> >> # HOME=/mware/cookie erts-5.8.1/bin/erl -sname aname >> Eshell V5.8.1 ?(abort with ^G) >> (aname@REDACTED)1> gen_tcp:connect("localhost", 1234, []). >> {error,econnrefused} >> >> Same result for {127,0,0,1}, other computers on the network, etc. >> > > Do you have a server listening on port 1234? > > Robby > > From yann.secq@REDACTED Tue Feb 1 14:20:44 2011 From: yann.secq@REDACTED (Yann SECQ) Date: Tue, 01 Feb 2011 14:20:44 +0100 Subject: if and fun with erl Message-ID: <4D4808AC.4060904@univ-lille1.fr> Dear erlang users, I'm beginnning my journey with this language and I'm a bit surprised by a strange behaviour concerning if within fun. I'm probably missing something obvious but I've not been able to find any clue on the official documentation or the web. I wanted to write within the erlang shell something like: 13> Compare = fun(X, Y) -> if X>Y -> greater; X==Y -> equal; X less end. * 1: syntax error before: '.' So I tried to add another 'end' to match the 'if' declaration 13> Compare = fun(X, Y) -> if X>Y -> greater; X==Y -> equal; X less end, end. * 1: syntax error before: 'end' With no more success. What surprises me is that if I catch the conditional return value it works: 14> Compare = fun(X, Y) -> Res = if X>Y -> greater; X==Y -> equal; X less end, Res end. #Fun What bugs me is that when doing this in a module: compare(X, Y) -> if X>Y -> greater; X==Y -> equal; true -> less end. It works as expected ! Can someone explain me what I'm missing to be able to return directly the conditional result without having to go through the Res temporary variable when using erl ? Thank you for your help, yann. -- Yann.Secq{at}univ-lille1.fr | http://www.lifl.fr/~secq Universite de Lille 1 | http://www.univ-lille1.fr Ouverture & Independance | http://www.ouverture-independance.fr Appel des Appels | http://lille.appeldesappels.fr "Ne d?sesp?rez jamais. Faites infuser davantage.", Henri Michaux From steve@REDACTED Tue Feb 1 14:26:13 2011 From: steve@REDACTED (Steve Strong) Date: Tue, 1 Feb 2011 14:26:13 +0100 Subject: if and fun with erl In-Reply-To: <4D4808AC.4060904@univ-lille1.fr> References: <4D4808AC.4060904@univ-lille1.fr> Message-ID: You were close, tried *almost* every combination :) Try this: 13> Compare = fun(X, Y) -> if X>Y -> greater; X==Y -> equal; X less end end. You need an "end" to the "if" and an "end" to the "fun". The first end doesn't need a comma after it since it is the last statement in the fun. -- Steve Strong twitter.com/srstrong On Tuesday, February 1, 2011 at 2:20 PM, Yann SECQ wrote: > 13> Compare = fun(X, Y) -> > if X>Y -> greater; > X==Y -> equal; > X less > end, > end. From bengt.kleberg@REDACTED Tue Feb 1 14:27:55 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 1 Feb 2011 14:27:55 +0100 Subject: [erlang-questions] if and fun with erl In-Reply-To: <4D4808AC.4060904@univ-lille1.fr> References: <4D4808AC.4060904@univ-lille1.fr> Message-ID: <1296566875.31061.2.camel@seasc0694> Greetings, The problem is due to a "," too many. Try: Compare = fun(X, Y) -> if X>Y -> greater; X==Y -> equal; X less end end. Also, consider using =:=, to get into the habit. bengt On Tue, 2011-02-01 at 14:20 +0100, Yann SECQ wrote: > Dear erlang users, > > I'm beginnning my journey with this language and I'm a bit > surprised by a strange behaviour concerning if within fun. > I'm probably missing something obvious but I've not been > able to find any clue on the official documentation or the web. > > I wanted to write within the erlang shell something like: > > 13> Compare = fun(X, Y) -> > if X>Y -> greater; > X==Y -> equal; > X less > end. > * 1: syntax error before: '.' > > So I tried to add another 'end' to match the 'if' declaration > > 13> Compare = fun(X, Y) -> > if X>Y -> greater; > X==Y -> equal; > X less > end, > end. > * 1: syntax error before: 'end' > > With no more success. What surprises me is that if I catch > the conditional return value it works: > > 14> Compare = fun(X, Y) -> > Res = if X>Y -> greater; > X==Y -> equal; > X less > end, > Res > end. > #Fun > > What bugs me is that when doing this in a module: > > compare(X, Y) -> > if X>Y -> greater; > X==Y -> equal; > true -> less > end. > > It works as expected ! > > Can someone explain me what I'm missing to be able to return > directly the conditional result without having to go through > the Res temporary variable when using erl ? > > Thank you for your help, yann. > From fernando.benavides@REDACTED Tue Feb 1 14:28:30 2011 From: fernando.benavides@REDACTED (Fernando Benavides) Date: Tue, 01 Feb 2011 10:28:30 -0300 Subject: [erlang-questions] if and fun with erl In-Reply-To: <4D4808AC.4060904@univ-lille1.fr> References: <4D4808AC.4060904@univ-lille1.fr> Message-ID: <1296566910.2874.3.camel@army.local> If statements require a true clause. On your module you have one, on the console you don't. Try with: 13> Compare = fun(X, Y) -> > if X>Y -> greater; > X==Y -> equal; > true -> less > end > end. Cheers! ________________________________________________________________________ Fernando Benavides fernando@REDACTED On Tue, 2011-02-01 at 14:20 +0100, Yann SECQ wrote: > Dear erlang users, > > I'm beginnning my journey with this language and I'm a bit > surprised by a strange behaviour concerning if within fun. > I'm probably missing something obvious but I've not been > able to find any clue on the official documentation or the web. > > I wanted to write within the erlang shell something like: > > 13> Compare = fun(X, Y) -> > if X>Y -> greater; > X==Y -> equal; > X less > end. > * 1: syntax error before: '.' > > So I tried to add another 'end' to match the 'if' declaration > > 13> Compare = fun(X, Y) -> > if X>Y -> greater; > X==Y -> equal; > X less > end, > end. > * 1: syntax error before: 'end' > > With no more success. What surprises me is that if I catch > the conditional return value it works: > > 14> Compare = fun(X, Y) -> > Res = if X>Y -> greater; > X==Y -> equal; > X less > end, > Res > end. > #Fun > > What bugs me is that when doing this in a module: > > compare(X, Y) -> > if X>Y -> greater; > X==Y -> equal; > true -> less > end. > > It works as expected ! > > Can someone explain me what I'm missing to be able to return > directly the conditional result without having to go through > the Res temporary variable when using erl ? > > Thank you for your help, yann. From yann.secq@REDACTED Tue Feb 1 14:31:37 2011 From: yann.secq@REDACTED (Yann SECQ) Date: Tue, 01 Feb 2011 14:31:37 +0100 Subject: if and fun with erl In-Reply-To: References: <4D4808AC.4060904@univ-lille1.fr> Message-ID: <4D480B39.60208@univ-lille1.fr> On 01/02/2011 14:26, Steve Strong wrote: > You were close, tried *almost* every combination :) Try this: > > 13> Compare = fun(X, Y) -> > if X>Y -> greater; > X==Y -> equal; > X less > end > end. > > You need an "end" to the "if" and an "end" to the "fun". The first end > doesn't need a comma after it since it is the last statement in the fun. Thank you ! I tried several combinations but I miss this one as I thought that I needed some punctuation at the end of each expression :) Thank again for your help (and reactivity !). -- Yann.Secq{at}univ-lille1.fr | http://www.lifl.fr/~secq Universite de Lille 1 | http://www.univ-lille1.fr Ouverture & Independance | http://www.ouverture-independance.fr Appel des Appels | http://lille.appeldesappels.fr "Ne d?sesp?rez jamais. Faites infuser davantage.", Henri Michaux From fred.hebert@REDACTED Tue Feb 1 14:29:01 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Tue, 1 Feb 2011 08:29:01 -0500 Subject: [erlang-questions] if and fun with erl In-Reply-To: <4D4808AC.4060904@univ-lille1.fr> References: <4D4808AC.4060904@univ-lille1.fr> Message-ID: <4F950A40-D38D-4A8C-B3AC-7806715FB099@erlang-solutions.com> On 2011-02-01, at 08:20 AM, Yann SECQ wrote: > Dear erlang users, > > I'm beginnning my journey with this language and I'm a bit > surprised by a strange behaviour concerning if within fun. > I'm probably missing something obvious but I've not been > able to find any clue on the official documentation or the web. > > I wanted to write within the erlang shell something like: > > 13> Compare = fun(X, Y) -> > if X>Y -> greater; > X==Y -> equal; > X less > end. > * 1: syntax error before: '.' This is missing an 'end'. the syntax is 'fun([Args]) -> ... end' (where the [] around Args mean they're optional) and 'if ... end'. > > So I tried to add another 'end' to match the 'if' declaration > > 13> Compare = fun(X, Y) -> > if X>Y -> greater; > X==Y -> equal; > X less > end, > end. > * 1: syntax error before: 'end' > The ',' is an expression, not a line terminator; you do not need it in this case. > With no more success. What surprises me is that if I catch > the conditional return value it works: > > 14> Compare = fun(X, Y) -> > Res = if X>Y -> greater; > X==Y -> equal; > X less > end, > Res > end. > #Fun This is because now you add a ',' and the Res variable, which are two separate expressions. 'Res' must not be followed by a comma (and isn't) because no expression comes after. This is normal and expected behaviour. > > What bugs me is that when doing this in a module: > > compare(X, Y) -> > if X>Y -> greater; > X==Y -> equal; > true -> less > end. > > It works as expected ! Of course. The syntax is fine this time! > > Can someone explain me what I'm missing to be able to return > directly the conditional result without having to go through > the Res temporary variable when using erl ? if Cond1 -> Ret1; Cond2 -> Ret2; ... CondN -> RetN end. You might want to give a read to one of my blog posts, 'On Erlang's Syntax' (http://ferd.ca/on-erlang-s-syntax.html) which attempts to make this kind of syntactic issues easier to understand. Hopefully this will help. -- Fred H?bert http://www.erlang-solutions.com From taylor@REDACTED Tue Feb 1 14:37:23 2011 From: taylor@REDACTED (Taylor Venable) Date: Tue, 1 Feb 2011 08:37:23 -0500 Subject: [erlang-questions] if and fun with erl In-Reply-To: <4D4808AC.4060904@univ-lille1.fr> References: <4D4808AC.4060904@univ-lille1.fr> Message-ID: On Tue, Feb 1, 2011 at 08:20, Yann SECQ wrote: > So I tried to add another 'end' to match the 'if' declaration > > 13> Compare = fun(X, Y) -> > ? ? ?if X>Y -> greater; > ? ? ? ? X==Y -> equal; > ? ? ? ? X less > ? ? ?end, > ? ?end. > * 1: syntax error before: 'end' Think of ";" and "," as statement *separators* not statement *terminators* like in C/C++/Java. Since the only statement directly inside your fun is the if, there's no need to put a "," after it. 1> Compare = fun (X, Y) -> 1> if X > Y -> greater; 1> X == Y -> equal; 1> X < Y -> less 1> end 1> end. #Fun > With no more success. What surprises me is that if I catch > the conditional return value it works: > > 14> Compare = fun(X, Y) -> > ? ? ?Res = if X>Y -> greater; > ? ? ? ? ? ? ? X==Y -> equal; > ? ? ? ? ? ? ? X less > ? ? ? ? ? ?end, > ? ? ?Res > ? ? ?end. > #Fun This works because now you've got two expressions inside your fun, so you need the "," after the variable assignment to separate the assignment (which contains the if in the RHS) from the "Res". > What bugs me is that when doing this in a module: > > compare(X, Y) -> > ?if X>Y -> greater; > ? ? X==Y -> equal; > ? ? true -> less > end. > > It works as expected ! This is because when defining a function inside a module, not using an explicit fun expression, you don't need to provide an "end" for the function itself; it ends at the full stop (period). So in this case, you only need one "end" for the if expression. Hope this explanation helps. -- Taylor C. Venable http://metasyntax.net/ From ingela.andin@REDACTED Tue Feb 1 14:43:37 2011 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 1 Feb 2011 14:43:37 +0100 Subject: [erlang-questions] Broken error reporting with ODBC app, w/ patch In-Reply-To: <82788BBB-A34D-4993-BCCD-0D92A3438054@gmail.com> References: <82788BBB-A34D-4993-BCCD-0D92A3438054@gmail.com> Message-ID: Hi Ken! Yes please follow the patch instruction: https://github.com/erlang/otp/wiki/submitting-patches we will be happy to accept this patch, good catch :) Regards Ingela Erlang/OTP team - Ericsson AB 2011/1/31 Alain O'Dea : > On 2011-01-30, at 23:18, Ken Ellis wrote: > >> The ODBC app does not report informative error messages for connection >> issues. ?It always reports "No SQL-driver information available", >> which is misleading since something as simple as attempting to connect >> to a database that does not exist will return this error. ?This is due >> to a bad handle being passed in db_connect in odbcserver.c -- >> incorrectly as a statement handle instead of a db connection handle. >> See patch below. >> >> With the patch you get: >> {error,"[unixODBC]Sending the authentication packet failed;\nCould not >> connect to remote server. Connection to database failed."} >> or >> {error,"[unixODBC][Driver Manager]Data source name not found, and no >> default driver specified ?Connection to database failed."} >> >> In the current version both of the above errors would result in the >> same message: >> {error,"No SQL-driver information available. Connection to database failed."} >> >> A google search suggests a reasonable amount of time could be saved >> for those trying to use this package: >> http://www.google.com/search?q=erlang+odbc+%22No+SQL-driver+information+available%22&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a >> >> -Ken >> >> >> diff otp_src_R14B01/lib/odbc/c_src/odbcserver.c >> otp_src_R14B01_working/lib/odbc/c_src/odbcserver.c >> 475c475 >> < ? ? ? diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); >> --- >>> ? ? ?diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state)); >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > Better error messages for the win :) > > Thank you for posting this Ken! > > You should send your patch on to erlang-patches@REDACTED for integration into OTP. ?For more information on contributing to OTP see: > https://github.com/erlang/otp/wiki/submitting-patches > > Thanks again, > Alain > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From taylor@REDACTED Tue Feb 1 14:43:59 2011 From: taylor@REDACTED (Taylor Venable) Date: Tue, 1 Feb 2011 08:43:59 -0500 Subject: [erlang-questions] if and fun with erl In-Reply-To: <1296566910.2874.3.camel@army.local> References: <4D4808AC.4060904@univ-lille1.fr> <1296566910.2874.3.camel@army.local> Message-ID: On Tue, Feb 1, 2011 at 08:28, Fernando Benavides wrote: > If statements require a true clause. On your module you have one, on the > console you don't. A true clause is not strictly required; when evaluated, if no matching branch of an "if" expression is found, an error occurs just like with case: 2> X = 3. 3 3> if X =:= 2 -> ok end. ** exception error: no true branch found when evaluating an if expression In some (perhaps many) cases you may not even want a true branch, so that invalid uses of the if expression cause an error, rather than proceeding (for which you must then code defensively). That's part of the "let it crash" philosophy. -- Taylor C. Venable http://metasyntax.net/ From fernando.benavides@REDACTED Tue Feb 1 15:02:01 2011 From: fernando.benavides@REDACTED (Fernando Benavides) Date: Tue, 01 Feb 2011 11:02:01 -0300 Subject: [erlang-questions] if and fun with erl In-Reply-To: References: <4D4808AC.4060904@univ-lille1.fr> <1296566910.2874.3.camel@army.local> Message-ID: <1296568921.2874.15.camel@army.local> You're right. Sorry for the confusion. On Tue, 2011-02-01 at 08:43 -0500, Taylor Venable wrote: > On Tue, Feb 1, 2011 at 08:28, Fernando Benavides > wrote: > > If statements require a true clause. On your module you have one, on the > > console you don't. > > A true clause is not strictly required; when evaluated, if no matching > branch of an "if" expression is found, an error occurs just like with > case: > > 2> X = 3. > 3 > 3> if X =:= 2 -> ok end. > ** exception error: no true branch found when evaluating an if expression > > In some (perhaps many) cases you may not even want a true branch, so > that invalid uses of the if expression cause an error, rather than > proceeding (for which you must then code defensively). That's part of > the "let it crash" philosophy. > From thatpythonguy@REDACTED Tue Feb 1 15:28:34 2011 From: thatpythonguy@REDACTED (Ahmed Al-Saadi) Date: Tue, 01 Feb 2011 09:28:34 -0500 Subject: Command-line Debugger? Message-ID: <4D481892.3090708@gmail.com> Sorry if this was double-submitted, but I think that it didn't go to the original group on the first attempt: Hello: I'm a newcomer to Erlang (from Python). I've had great difficulties in using the wx-based debugger and was wondering if there is a command-line alternative a la python's pdb?! It seems that every time I use the default debugger, my Wish app hangs or crashes. Even with the latest release of wxErlang (0.97.1212), I was not even able to set a breakpoint! The use of a good debugger to trace program execution is invaluable in learning a new language--a pedagogical role, in addition to its normal troubleshooting role. Any ideas of how I can improve the situation of my development environment? Is there another command-line debugger? My platform is MacOS X 10.6.6 with Erlang R14B... and, no, changing platforms is not an option ;) Also, if I knew how to write a debugger at this point in time, I would try to do so! Some guidance from a veteran could help. TPG/TEG From steve@REDACTED Tue Feb 1 15:33:01 2011 From: steve@REDACTED (Steve Strong) Date: Tue, 1 Feb 2011 15:33:01 +0100 Subject: Command-line Debugger? In-Reply-To: <4D481892.3090708@gmail.com> References: <4D481892.3090708@gmail.com> Message-ID: Take a loog at the builtin dbg module - for tracing, it probably does pretty much everything you need. -- Steve Strong twitter.com/srstrong On Tuesday, February 1, 2011 at 3:28 PM, Ahmed Al-Saadi wrote: > Sorry if this was double-submitted, but I think that it didn't go to the > original group on the first attempt: > > > Hello: > > I'm a newcomer to Erlang (from Python). I've had great difficulties in > using the wx-based debugger and was wondering if there is a command-line > alternative a la python's pdb?! > > It seems that every time I use the default debugger, my Wish app hangs > or crashes. Even with the latest release of wxErlang (0.97.1212), I was > not even able to set a breakpoint! > > The use of a good debugger to trace program execution is invaluable in > learning a new language--a pedagogical role, in addition to its normal > troubleshooting role. > > Any ideas of how I can improve the situation of my development > environment? Is there another command-line debugger? My platform is > MacOS X 10.6.6 with Erlang R14B... and, no, changing platforms is not an > option ;) Also, if I knew how to write a debugger at this point in > time, I would try to do so! Some guidance from a veteran could help. > > TPG/TEG > > > From norton@REDACTED Tue Feb 1 15:39:09 2011 From: norton@REDACTED (Joseph Wayne Norton) Date: Tue, 1 Feb 2011 23:39:09 +0900 Subject: [erlang-questions] Command-line Debugger? In-Reply-To: <4D481892.3090708@gmail.com> References: <4D481892.3090708@gmail.com> Message-ID: Ahmed - It may sound like a crazy idea but I'd recommend not using the debugger. Erlang's trace facilities are really great for troubleshooting code during development and production. The redbug tool is any easy way to get started with Erlang's tracer (https://github.com/massemanet/eper). On 2011/02/01, at 23:28, Ahmed Al-Saadi wrote: > Sorry if this was double-submitted, but I think that it didn't go to the original group on the first attempt: > > > Hello: > > I'm a newcomer to Erlang (from Python). I've had great difficulties in using the wx-based debugger and was wondering if there is a command-line alternative a la python's pdb?! > > It seems that every time I use the default debugger, my Wish app hangs or crashes. Even with the latest release of wxErlang (0.97.1212), I was not even able to set a breakpoint! > > The use of a good debugger to trace program execution is invaluable in learning a new language--a pedagogical role, in addition to its normal troubleshooting role. > > Any ideas of how I can improve the situation of my development environment? Is there another command-line debugger? My platform is MacOS X 10.6.6 with Erlang R14B... and, no, changing platforms is not an option ;) Also, if I knew how to write a debugger at this point in time, I would try to do so! Some guidance from a veteran could help. > > TPG/TEG Joseph Wayne Norton norton@REDACTED From pmorreale@REDACTED Tue Feb 1 15:58:25 2011 From: pmorreale@REDACTED (Peter W. Morreale) Date: Tue, 01 Feb 2011 07:58:25 -0700 Subject: [erlang-questions] Questions about OTP debugging... In-Reply-To: <1296542647.5230.5.camel@seasc1137> References: <1296515848.11676.89.camel@hermosa.morreale.net> <1296542647.5230.5.camel@seasc1137> Message-ID: <1296572305.11676.107.camel@hermosa.morreale.net> On Tue, 2011-02-01 at 07:44 +0100, Bengt Kleberg wrote: > Greetings, > > Your error message seems to indicate that the tcp_rpc module is not > loaded. Since Erlang loads beam files automatically if they are in the > path this could be two problems. > 1 You do not have the right path > 2 You do not have the beam file > > Can you locate the file tcp_rpc.beam on your system? Thanks for the response. There is no tcp_rpc.beam in this example from the book. I needed to create a tcp_rpc.app file in the ebin dir, which defines the modules for the application. (This is chapter 4 of the "Erlang and OTP in Action" book.) > When you have started Erlang, can you find the directory where the bream > file resides in your path (see code:get_path/0)? > > > bengt > I *think* I have everything in the right place. Here is what I have for a directory layout and compile/run (including a code:get_path()): pmorreale@REDACTED:~/erlang/tcp_rpc> ls -R : doc ebin include priv src /doc: /ebin: tcp_rpc.app tr_app.beam tr_server.beam tr_sup.beam /include: /priv: /src: tr_app.erl tr_server.erl tr_sup.erl pmorreale@REDACTED:~/erlang/tcp_rpc> more ebin/tcp_rpc.app %% -*- mode: Erlang; fill-column: 75; comment-column: 50; -*- {application, tcp_rcp, [ {description, "RPC server for Erlang testing"}, {vsn, "0.1.0"}, {modules, [tr_app, tr_sup, tr_server]}, {registered, [tr_sup]}, {applications, [kernel, stdlib]}, {mod, {tr_app, []}} ]}. pmorreale@REDACTED:~/erlang/tcp_rpc> erlc -o ebin src/*.erl pmorreale@REDACTED:~/erlang/tcp_rpc> erl -pa ebin Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) 1> application:start(tcp_rpc). {error,{not_loaded,tcp_rpc}} 2> BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution a pmorreale@REDACTED:~/erlang/tcp_rpc> erl -pa ebin Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) 1> code:get_path(). ["ebin",".","/usr/lib64/erlang/lib/kernel-2.14.1/ebin", "/usr/lib64/erlang/lib/stdlib-1.17.1/ebin", "/usr/lib64/erlang/lib/xmerl-1.2.6/ebin", "/usr/lib64/erlang/lib/wx-0.98.7/ebin", "/usr/lib64/erlang/lib/webtool-0.8.7/ebin", "/usr/lib64/erlang/lib/typer-0.1.7.5/ebin", "/usr/lib64/erlang/lib/tv-2.1.4.6/ebin", "/usr/lib64/erlang/lib/tools-2.6.6.1/ebin", "/usr/lib64/erlang/lib/toolbar-1.4.1/ebin", "/usr/lib64/erlang/lib/test_server-3.4.1/ebin", "/usr/lib64/erlang/lib/syntax_tools-1.6.6/ebin", "/usr/lib64/erlang/lib/ssl-4.0.1/ebin", "/usr/lib64/erlang/lib/ssh-2.0.1/ebin", "/usr/lib64/erlang/lib/snmp-4.18/ebin", "/usr/lib64/erlang/lib/sasl-2.1.9.2/ebin", "/usr/lib64/erlang/lib/runtime_tools-1.8.4.1/ebin", "/usr/lib64/erlang/lib/reltool-0.5.4/ebin", "/usr/lib64/erlang/lib/public_key-0.8/ebin", "/usr/lib64/erlang/lib/pman-2.7.1/ebin", "/usr/lib64/erlang/lib/percept-0.8.4/ebin", "/usr/lib64/erlang/lib/parsetools-2.0.4/ebin", "/usr/lib64/erlang/lib/otp_mibs-1.0.6/ebin", "/usr/lib64/erlang/lib/os_mon-2.2.5/ebin", "/usr/lib64/erlang/lib/orber-3.6.17/ebin", "/usr/lib64/erlang/lib/odbc-2.10.8/ebin", "/usr/lib64/erlang/lib/observer-0.9.8.3/ebin", [...]|...] 2> BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution a Thanks, -PWM From pmorreale@REDACTED Tue Feb 1 16:43:56 2011 From: pmorreale@REDACTED (Peter W. Morreale) Date: Tue, 01 Feb 2011 08:43:56 -0700 Subject: [erlang-questions] Questions about OTP debugging... In-Reply-To: <1296572305.11676.107.camel@hermosa.morreale.net> References: <1296515848.11676.89.camel@hermosa.morreale.net> <1296542647.5230.5.camel@seasc1137> <1296572305.11676.107.camel@hermosa.morreale.net> Message-ID: <1296575036.11676.110.camel@hermosa.morreale.net> Arrggghhh... I found the issue... Typo in the .app file: > {application, tcp_rcp, [ Should be: {application, tcp_rpc, { ^^^^ My fingers need to go on a diet. ;-) Thanks all -PWM From dgud@REDACTED Tue Feb 1 16:45:06 2011 From: dgud@REDACTED (Dan Gudmundsson) Date: Tue, 1 Feb 2011 16:45:06 +0100 Subject: [erlang-questions] Command-line Debugger? In-Reply-To: References: <4D481892.3090708@gmail.com> Message-ID: Just for the information, the latest wx is 0.98.8 and is included with erlang. The old sourceforge site is dead... except some occasional support mail. But on MacOsX it only works for 32bits erlang currently, and also requires the user to install and build wxWidgets, the one included in snow leopard is buggy. /Dan On Tue, Feb 1, 2011 at 3:39 PM, Joseph Wayne Norton wrote: > > Ahmed - > > It may sound like a crazy idea but I'd recommend not using the debugger. > > Erlang's trace facilities are really great for troubleshooting code during development and production. > > The redbug tool is any easy way to get started with Erlang's tracer (https://github.com/massemanet/eper). > > > On 2011/02/01, at 23:28, Ahmed Al-Saadi wrote: > >> Sorry if this was double-submitted, but I think that it didn't go to the original group on the first attempt: >> >> >> Hello: >> >> I'm a newcomer to Erlang (from Python). I've had great difficulties in using the wx-based debugger and was wondering if there is a command-line alternative a la python's pdb?! >> >> It seems that every time I use the default debugger, my Wish app hangs or crashes. Even with the latest release of wxErlang (0.97.1212), I was not even able to set a breakpoint! >> >> The use of a good debugger to trace program execution is invaluable in learning a new language--a pedagogical role, in addition to its normal troubleshooting role. >> >> Any ideas of how I can improve the situation of my development environment? Is there another command-line debugger? My platform is MacOS X 10.6.6 with Erlang R14B... and, no, changing platforms is not an option ;) ?Also, if I knew how to write a debugger at this point in time, I would try to do so! Some guidance from a veteran could help. >> >> TPG/TEG > > Joseph Wayne Norton > norton@REDACTED > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jared.nance@REDACTED Tue Feb 1 17:12:47 2011 From: jared.nance@REDACTED (Jared Kofron) Date: Tue, 1 Feb 2011 08:12:47 -0800 Subject: [erlang-questions] Questions about OTP debugging... In-Reply-To: <1296575036.11676.110.camel@hermosa.morreale.net> References: <1296515848.11676.89.camel@hermosa.morreale.net> <1296542647.5230.5.camel@seasc1137> <1296572305.11676.107.camel@hermosa.morreale.net> <1296575036.11676.110.camel@hermosa.morreale.net> Message-ID: Peter - For future reference, I find SASL to be a huge help in debugging things. To turn it on, just application:start(sasl) Be warned, it will increase the verbosity of your terminal dramatically! On Feb 1, 2011, at 7:43 AM, Peter W. Morreale wrote: > Arrggghhh... I found the issue... > > Typo in the .app file: > >> {application, tcp_rcp, [ > > Should be: > > {application, tcp_rpc, { > ^^^^ > My fingers need to go on a diet. ;-) > > Thanks all > -PWM > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ferenc.holzhauser@REDACTED Tue Feb 1 17:55:31 2011 From: ferenc.holzhauser@REDACTED (Ferenc Holzhauser) Date: Tue, 1 Feb 2011 17:55:31 +0100 Subject: SSL number of connections Message-ID: Hi, I'd like to use a lot (I was aiming at 20-30k) of active SSL connections from an Erlang app. I expected it to be no problem (assuming I have done all OS limit/kernel and Erlang max port settings correctly). I've come across 2 things though that have ruined my expectations. 1. Apparently ssl_esock opens a pair of local TCP connections to itself (according to lsof) on port 53813 for each SSL connection I open from the app. I intended to use multiple source/destination addresses to get around the TCP source port limitation and have more sessions. That trick won't work then. I must confess I don't know the purpose of this local connection but I'd really like to get rid of it if possible, is it? 2. There seems to be only 1 ssl_esock process running on the machine eventually eating up one CPU core (and not more) way before I'd reach any reasonable number of connections or the beam process would become very busy. It doesn't look very scalable. I have enough CPU power left and I'd really want to convince SSL to use it. Am I overlooking something? Is it really not possible to have more ssl_esock processes using more cores? I'm running R13B03 but I haven't found anything in R14 either addressing these issues. Thank you for your help in advance, Ferenc From knut.nesheim@REDACTED Tue Feb 1 18:25:00 2011 From: knut.nesheim@REDACTED (Knut Nesheim) Date: Tue, 1 Feb 2011 18:25:00 +0100 Subject: gen_server clustering strategy Message-ID: Hello list, We are interested in understanding how the Erlang distribution mechanism behaves in a fully connected cluster. Our concern is that as we add nodes, the overhead will become a problem. Our application is a bunch of gen_servers containing state, which we need to run on multiple nodes due to memory usage. A request will come to our application and it needs to be handled by the correct gen_server. Every request includes a user id which we can use to map to the process. We have two specific use cases in mind. 1) Proxy to workers In this scenario we have a proxy (or multiple proxies) accepting requests at the edge of our cluster. The proxy will ask the correct server for a response. Either through using the 'global' module or gproc or something similar, the proxy node will keep track of the mapping of user ids to process and nodes. The proxy will call the node using the Erlang distribution protocol. 2) Mesh In this scenario a request can be handled by any node in the cluster. If the request cannot be handled by the local node, the correct node is called on instead. Every node in the cluster needs to keep track of which id belongs to which process. The numbers: * We must be able to run 100,000 processes, each may peak at 300kb of memory * We expect 5000 requests coming in to our application per second at peak, generating the same number of messages * Out of those 5000 messages, 1000 has a content that may peak at 300kb * All the other messages are well under 1kb Our concerns: * In the mesh, will the traffic between the nodes become a problem? Lets say we have 4 nodes, if the requests are evenly divided between the nodes the probability of hitting the correct node is 25%. With 100 nodes this is 1%. As we add nodes, there will be more chatter. May this chatter "drown-out" the important messages, like pid down, etc. * Will it be more efficient to use the proxy to message every node? Will the message then always go to the node directly, or may it go through some other node in the cluster? * For the mesh, we need to keep the 'global' or gproc state replicated. With 'global' we will run into the default limit of atoms. If we were to increase this limit, what kind of memory usage could we expect? Are there any other nasty side-effects that we should be aware of? * In the "Erlang/OTP in Action" book it is stated that you may have a "couple of dozen, but probably not hundreds of nodes." Our understanding is that this is because of traffic like "this pid just died", "this node is no longer available", etc. * If we were to eliminate messaging between nodes, how many nodes could we actually run in a fully connected cluster? Will a high-latency network like ec2 affect this number or just the latency of messages? What else may affect the size? Regards Knut From thatpythonguy@REDACTED Tue Feb 1 18:40:56 2011 From: thatpythonguy@REDACTED (Ahmed Al-Saadi) Date: Tue, 01 Feb 2011 12:40:56 -0500 Subject: [erlang-questions] Command-line Debugger? In-Reply-To: References: <4D481892.3090708@gmail.com> Message-ID: <4D4845A8.9010206@gmail.com> Thanks, Joseph and Steve for the responses. I suppose I did not look at the original Programming Erlang coverage of debugging and instead assumed that OTP in Action's was comprehensive! Cheers, Ahmed On 2/1/11 9:39 AM, Joseph Wayne Norton wrote: > Ahmed - > > It may sound like a crazy idea but I'd recommend not using the debugger. > > Erlang's trace facilities are really great for troubleshooting code during development and production. > > The redbug tool is any easy way to get started with Erlang's tracer (https://github.com/massemanet/eper). > > > On 2011/02/01, at 23:28, Ahmed Al-Saadi wrote: > >> Sorry if this was double-submitted, but I think that it didn't go to the original group on the first attempt: >> >> >> Hello: >> >> I'm a newcomer to Erlang (from Python). I've had great difficulties in using the wx-based debugger and was wondering if there is a command-line alternative a la python's pdb?! >> >> It seems that every time I use the default debugger, my Wish app hangs or crashes. Even with the latest release of wxErlang (0.97.1212), I was not even able to set a breakpoint! >> >> The use of a good debugger to trace program execution is invaluable in learning a new language--a pedagogical role, in addition to its normal troubleshooting role. >> >> Any ideas of how I can improve the situation of my development environment? Is there another command-line debugger? My platform is MacOS X 10.6.6 with Erlang R14B... and, no, changing platforms is not an option ;) Also, if I knew how to write a debugger at this point in time, I would try to do so! Some guidance from a veteran could help. >> >> TPG/TEG > Joseph Wayne Norton > norton@REDACTED > > > From pmorreale@REDACTED Tue Feb 1 19:00:49 2011 From: pmorreale@REDACTED (Peter W. Morreale) Date: Tue, 01 Feb 2011 11:00:49 -0700 Subject: [erlang-questions] Questions about OTP debugging... In-Reply-To: References: <1296515848.11676.89.camel@hermosa.morreale.net> <1296542647.5230.5.camel@seasc1137> <1296572305.11676.107.camel@hermosa.morreale.net> <1296575036.11676.110.camel@hermosa.morreale.net> Message-ID: <1296583249.11676.115.camel@hermosa.morreale.net> On Tue, 2011-02-01 at 08:12 -0800, Jared Kofron wrote: > Peter - > For future reference, I find SASL to be a huge help in debugging things. To turn it on, > just > > application:start(sasl) > > Be warned, it will increase the verbosity of your terminal dramatically! > Thank you. I actually did find that and tried it, however I didn't see anything that suggested what the problem was. Now that I know, I'll try adding the typo and reread the output again. Its a brave new world... ;-) Best, -PWM > On Feb 1, 2011, at 7:43 AM, Peter W. Morreale wrote: > > > Arrggghhh... I found the issue... > > > > Typo in the .app file: > > > >> {application, tcp_rcp, [ > > > > Should be: > > > > {application, tcp_rpc, { > > ^^^^ > > My fingers need to go on a diet. ;-) > > > > Thanks all > > -PWM > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From kaiduanx@REDACTED Tue Feb 1 19:15:12 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Tue, 1 Feb 2011 13:15:12 -0500 Subject: [erlang-questions] SSL number of connections In-Reply-To: References: Message-ID: Ferenc, I have been working a project with Erlang SSL for a while. For better performance, please use R14B01. In Erlang, old SSL uses openssl, and ssl_esock talks to Erlang in a port. Starting from R14, new ssl is introduced where all SSL negotiation is done in Erlang instead of openssl, only libcrypto is used in new ssl for encryption/decryption. So far, I am very happy with the performance of new SSL. I managed to server 100K SSL concurrent connections without any problem. Best regards, /Kaiduan On Tue, Feb 1, 2011 at 11:55 AM, Ferenc Holzhauser wrote: > Hi, > > I'd like to use a lot (I was aiming at 20-30k) of active SSL connections > from an Erlang app. > I expected it to be no problem (assuming I have done all OS limit/kernel and > Erlang max port settings correctly). > > I've come across 2 things though that have ruined my expectations. > > 1. > Apparently ssl_esock opens a pair of local TCP connections to itself > (according to lsof) on port 53813 for each SSL connection I open from the > app. > I intended to use multiple source/destination addresses to get around the > TCP source port limitation and have more sessions. That trick won't work > then. > > I must confess I don't know the purpose of this local connection but I'd > really like to get rid of it if possible, is it? > > 2. > There seems to be only 1 ssl_esock process running on the machine eventually > eating up one CPU core (and not more) way before I'd reach any reasonable > number of connections or the beam process would become very busy. > It doesn't look very scalable. I have enough CPU power left and I'd really > want to convince SSL to use it. > > Am I overlooking something? Is it really not possible to have more ssl_esock > processes using more cores? > > I'm running R13B03 but I haven't found anything in R14 either addressing > these issues. > > Thank you for your help in advance, > Ferenc > From mazen.harake@REDACTED Tue Feb 1 19:05:37 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 01 Feb 2011 19:05:37 +0100 Subject: [erlang-questions] Command-line Debugger? In-Reply-To: <4D481892.3090708@gmail.com> References: <4D481892.3090708@gmail.com> Message-ID: <4D484B71.6050300@erlang-solutions.com> Hi Ahmed, Perhaps my tutorial from the Erlang User Conference will help you out: http://www.erlang-factory.com/conference/testingtutorialworkshop2010/speakers/MazenHarake Good luck. BR /M On 01/02/2011 15:28, Ahmed Al-Saadi wrote: > Sorry if this was double-submitted, but I think that it didn't go to > the original group on the first attempt: > > > Hello: > > I'm a newcomer to Erlang (from Python). I've had great difficulties in > using the wx-based debugger and was wondering if there is a > command-line alternative a la python's pdb?! > > It seems that every time I use the default debugger, my Wish app hangs > or crashes. Even with the latest release of wxErlang (0.97.1212), I > was not even able to set a breakpoint! > > The use of a good debugger to trace program execution is invaluable in > learning a new language--a pedagogical role, in addition to its normal > troubleshooting role. > > Any ideas of how I can improve the situation of my development > environment? Is there another command-line debugger? My platform is > MacOS X 10.6.6 with Erlang R14B... and, no, changing platforms is not > an option ;) Also, if I knew how to write a debugger at this point in > time, I would try to do so! Some guidance from a veteran could help. > > TPG/TEG > From mevans@REDACTED Tue Feb 1 20:10:05 2011 From: mevans@REDACTED (Evans, Matthew) Date: Tue, 1 Feb 2011 14:10:05 -0500 Subject: [erlang-questions] gen_server clustering strategy In-Reply-To: References: Message-ID: Without knowing much more another model would be to use your own ets/mnesia table to map the workers. I am assuming the creation of a worker is relatively infrequent (compared to the times you need to find a worker). When you create a worker use gen_server:start(?MODULE, [], []). This will return {ok,Pid}. You can save the Pid in an mnesia table along with whatever reference you need (doesn't need to be an atom). All nodes in the mesh will get the pid and the reference. Starting it as shown above (with an arity of 3) means you don't need to use the global service and ensure the names are unique atoms for each server. You can even do start via some application that will spread the gen_servers over different nodes. When a request comes in it only need do a lookup to find the correct pid. I can't recall as to how bad the process crash messaging is. But what you could do is to monitor a gen_server locally, and monitor each node globally. When a node fails all other nodes can cleanup processes that were registered on that node. Matt ________________________________________ From: erlang-questions@REDACTED [erlang-questions@REDACTED] On Behalf Of Knut Nesheim [knut.nesheim@REDACTED] Sent: Tuesday, February 01, 2011 12:25 PM To: erlang-questions@REDACTED Subject: [erlang-questions] gen_server clustering strategy Hello list, We are interested in understanding how the Erlang distribution mechanism behaves in a fully connected cluster. Our concern is that as we add nodes, the overhead will become a problem. Our application is a bunch of gen_servers containing state, which we need to run on multiple nodes due to memory usage. A request will come to our application and it needs to be handled by the correct gen_server. Every request includes a user id which we can use to map to the process. We have two specific use cases in mind. 1) Proxy to workers In this scenario we have a proxy (or multiple proxies) accepting requests at the edge of our cluster. The proxy will ask the correct server for a response. Either through using the 'global' module or gproc or something similar, the proxy node will keep track of the mapping of user ids to process and nodes. The proxy will call the node using the Erlang distribution protocol. 2) Mesh In this scenario a request can be handled by any node in the cluster. If the request cannot be handled by the local node, the correct node is called on instead. Every node in the cluster needs to keep track of which id belongs to which process. The numbers: * We must be able to run 100,000 processes, each may peak at 300kb of memory * We expect 5000 requests coming in to our application per second at peak, generating the same number of messages * Out of those 5000 messages, 1000 has a content that may peak at 300kb * All the other messages are well under 1kb Our concerns: * In the mesh, will the traffic between the nodes become a problem? Lets say we have 4 nodes, if the requests are evenly divided between the nodes the probability of hitting the correct node is 25%. With 100 nodes this is 1%. As we add nodes, there will be more chatter. May this chatter "drown-out" the important messages, like pid down, etc. * Will it be more efficient to use the proxy to message every node? Will the message then always go to the node directly, or may it go through some other node in the cluster? * For the mesh, we need to keep the 'global' or gproc state replicated. With 'global' we will run into the default limit of atoms. If we were to increase this limit, what kind of memory usage could we expect? Are there any other nasty side-effects that we should be aware of? * In the "Erlang/OTP in Action" book it is stated that you may have a "couple of dozen, but probably not hundreds of nodes." Our understanding is that this is because of traffic like "this pid just died", "this node is no longer available", etc. * If we were to eliminate messaging between nodes, how many nodes could we actually run in a fully connected cluster? Will a high-latency network like ec2 affect this number or just the latency of messages? What else may affect the size? Regards Knut ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From thatpythonguy@REDACTED Tue Feb 1 20:13:34 2011 From: thatpythonguy@REDACTED (Ahmed Al-Saadi) Date: Tue, 01 Feb 2011 14:13:34 -0500 Subject: [erlang-questions] Command-line Debugger? In-Reply-To: <4D484B71.6050300@erlang-solutions.com> References: <4D481892.3090708@gmail.com> <4D484B71.6050300@erlang-solutions.com> Message-ID: <4D485B5E.8070301@gmail.com> Hi Mazen: I already started reading it and it does seem quite useful. Thanks! Cheers, Ahmed On 2/1/11 1:05 PM, Mazen Harake wrote: > Hi Ahmed, > > Perhaps my tutorial from the Erlang User Conference will help you out: > http://www.erlang-factory.com/conference/testingtutorialworkshop2010/speakers/MazenHarake > > > Good luck. > > BR > > /M > > On 01/02/2011 15:28, Ahmed Al-Saadi wrote: >> Sorry if this was double-submitted, but I think that it didn't go to >> the original group on the first attempt: >> >> >> Hello: >> >> I'm a newcomer to Erlang (from Python). I've had great difficulties >> in using the wx-based debugger and was wondering if there is a >> command-line alternative a la python's pdb?! >> >> It seems that every time I use the default debugger, my Wish app >> hangs or crashes. Even with the latest release of wxErlang >> (0.97.1212), I was not even able to set a breakpoint! >> >> The use of a good debugger to trace program execution is invaluable >> in learning a new language--a pedagogical role, in addition to its >> normal troubleshooting role. >> >> Any ideas of how I can improve the situation of my development >> environment? Is there another command-line debugger? My platform is >> MacOS X 10.6.6 with Erlang R14B... and, no, changing platforms is not >> an option ;) Also, if I knew how to write a debugger at this point >> in time, I would try to do so! Some guidance from a veteran could help. >> >> TPG/TEG >> > From garret.smith@REDACTED Tue Feb 1 20:20:14 2011 From: garret.smith@REDACTED (Garret Smith) Date: Tue, 1 Feb 2011 11:20:14 -0800 Subject: [erlang-questions] Android Tablet and Erlang (+Java) In-Reply-To: <34072B4B-57BD-4C55-9CCD-2E565013CA50@gmail.com> References: <1296395849734-3247153.post@n4.nabble.com> <01F08CDE-681F-4C02-A8AB-9470435B7F33@gmail.com> <89EA73CE-11FC-4D23-88C9-20EC52820632@gmail.com> <34072B4B-57BD-4C55-9CCD-2E565013CA50@gmail.com> Message-ID: You can also take a look at Erlang for Android at http://www.burbas.se/artiklar/erlang-for-the-android-plattform/ I have gotten as far as installing it and running a few commands in the interactive interpreter on my HTC Evo running the popular Cyanogenmod ROM. -Garret Smith On Mon, Jan 31, 2011 at 11:05 AM, Zabrane Mickael wrote: > Le 31 janv. 2011 ? 18:58, Felix Hamilton a ?crit : > >> Recently got Ubuntu running on an Archos 10.1 'android tablet'. Havent >> actually tried to get erlang compiled natively on an android device, >> however ... > > > Cool. Thanks Felix! > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From brum76@REDACTED Tue Feb 1 21:23:00 2011 From: brum76@REDACTED (Radu Brumariu) Date: Tue, 1 Feb 2011 12:23:00 -0800 Subject: escript -- undefined function main/1 if using relative path Message-ID: Hi, I have this escript archive and if I execute it from a relative path I get the following error : ebin/myscript -p . escript: exception error: undefined function myscript:main/1 in function escript:run/2 in call from escript:start/1 in call from init:start_it/1 in call from init:start_em/1 by if I cd in its directory of if I supply the full path to the script it works. Any suggestions what I might be doing wrong ? Thanks, Radu From watson.timothy@REDACTED Tue Feb 1 21:40:48 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 1 Feb 2011 20:40:48 +0000 Subject: [erlang-questions] escript -- undefined function main/1 if using relative path In-Reply-To: References: Message-ID: On 1 February 2011 20:23, Radu Brumariu wrote: > Hi, > I have this escript archive and if I execute it from a relative path I > get the following error : > Can you post the error (or a link/gist/etc) - it's a bit hard to diagnose without more detail. From zabrane3@REDACTED Tue Feb 1 20:53:52 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 1 Feb 2011 20:53:52 +0100 Subject: [erlang-questions] Android Tablet and Erlang (+Java) In-Reply-To: References: <1296395849734-3247153.post@n4.nabble.com> <01F08CDE-681F-4C02-A8AB-9470435B7F33@gmail.com> <89EA73CE-11FC-4D23-88C9-20EC52820632@gmail.com> <34072B4B-57BD-4C55-9CCD-2E565013CA50@gmail.com> Message-ID: <7C99E311-7866-43AF-97AC-FE3C14C59028@gmail.com> Thanks for the link Garret. Le 1 f?vr. 2011 ? 20:20, Garret Smith a ?crit : > You can also take a look at Erlang for Android at > http://www.burbas.se/artiklar/erlang-for-the-android-plattform/ > > I have gotten as far as installing it and running a few commands in > the interactive interpreter on my HTC Evo running the popular > Cyanogenmod ROM. > > -Garret Smith > > On Mon, Jan 31, 2011 at 11:05 AM, Zabrane Mickael wrote: >> Le 31 janv. 2011 ? 18:58, Felix Hamilton a ?crit : >> >>> Recently got Ubuntu running on an Archos 10.1 'android tablet'. Havent >>> actually tried to get erlang compiled natively on an android device, >>> however ... >> >> >> Cool. Thanks Felix! >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> Regards, Zabrane From ferenc.holzhauser@REDACTED Wed Feb 2 02:01:24 2011 From: ferenc.holzhauser@REDACTED (Ferenc Holzhauser) Date: Wed, 2 Feb 2011 02:01:24 +0100 Subject: [erlang-questions] SSL number of connections In-Reply-To: References: Message-ID: Hi, Thanks a lot for the hint! It looks lot better indeed with R14. Best Regards, Ferenc On 1 February 2011 19:15, Kaiduan Xie wrote: > Ferenc, > > I have been working a project with Erlang SSL for a while. For better > performance, please use R14B01. In Erlang, old SSL uses openssl, and > ssl_esock talks to Erlang in a port. Starting from R14, new ssl is > introduced where all SSL negotiation is done in Erlang instead of > openssl, only libcrypto is used in new ssl for encryption/decryption. > > So far, I am very happy with the performance of new SSL. I managed to > server 100K SSL concurrent connections without any problem. > > Best regards, > > /Kaiduan > > On Tue, Feb 1, 2011 at 11:55 AM, Ferenc Holzhauser > wrote: > > Hi, > > > > I'd like to use a lot (I was aiming at 20-30k) of active SSL connections > > from an Erlang app. > > I expected it to be no problem (assuming I have done all OS limit/kernel > and > > Erlang max port settings correctly). > > > > I've come across 2 things though that have ruined my expectations. > > > > 1. > > Apparently ssl_esock opens a pair of local TCP connections to itself > > (according to lsof) on port 53813 for each SSL connection I open from the > > app. > > I intended to use multiple source/destination addresses to get around the > > TCP source port limitation and have more sessions. That trick won't work > > then. > > > > I must confess I don't know the purpose of this local connection but I'd > > really like to get rid of it if possible, is it? > > > > 2. > > There seems to be only 1 ssl_esock process running on the machine > eventually > > eating up one CPU core (and not more) way before I'd reach any reasonable > > number of connections or the beam process would become very busy. > > It doesn't look very scalable. I have enough CPU power left and I'd > really > > want to convince SSL to use it. > > > > Am I overlooking something? Is it really not possible to have more > ssl_esock > > processes using more cores? > > > > I'm running R13B03 but I haven't found anything in R14 either addressing > > these issues. > > > > Thank you for your help in advance, > > Ferenc > > > From watson.timothy@REDACTED Wed Feb 2 04:42:56 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 2 Feb 2011 03:42:56 +0000 Subject: [erlang-questions] escript -- undefined function main/1 if using relative path In-Reply-To: References: Message-ID: On 1 February 2011 20:40, Tim Watson wrote: > On 1 February 2011 20:23, Radu Brumariu wrote: >> Hi, >> I have this escript archive and if I execute it from a relative path I >> get the following error : >> > > Can you post the error (or a link/gist/etc) - it's a bit hard to > diagnose without more detail. > Sorry, I meant to say can you post the code (e.g., your main/1 function). It's probably that you're passing "." somewhere that wants a fully qualified path rather than an absolute, or perhaps depending on the current working directory in some way. It's not possible to diagnose without knowing what your script is doing. From marcel.meyer@REDACTED Wed Feb 2 04:59:47 2011 From: marcel.meyer@REDACTED (Marcel Meyer) Date: Tue, 1 Feb 2011 22:59:47 -0500 Subject: Erlang and MSMQ Message-ID: Hi there, I was wondering if anyone was ever successful in publishing to a MSMQ from Erlang and how they did it. Regards, Marcel From thatpythonguy@REDACTED Wed Feb 2 06:19:48 2011 From: thatpythonguy@REDACTED (Ahmed Al-Saadi) Date: Wed, 02 Feb 2011 00:19:48 -0500 Subject: [erlang-questions] Command-line Debugger? In-Reply-To: References: <4D481892.3090708@gmail.com> Message-ID: <4D48E974.1020109@gmail.com> Dan: It seems that the Homebrew Erlang installation on my Mac did not include the wx package! I downloaded the latest tarball from erlang.org and, so far, the previous problems with the debugger haven't manifested themselves. I do have a first-generation macbook which has a 32-bit processor and I did install wxWidgets myself, so that must have avoided the pitfalls you mentioned. Thanks for your input... Cheers, Ahmed On 2/1/11 10:45 AM, Dan Gudmundsson wrote: > Just for the information, the latest wx is 0.98.8 and is included with erlang. > The old sourceforge site is dead... except some occasional support mail. > > But on MacOsX it only works for 32bits erlang currently, and also > requires the user to install > and build wxWidgets, the one included in snow leopard is buggy. > > /Dan > > On Tue, Feb 1, 2011 at 3:39 PM, Joseph Wayne Norton > wrote: >> Ahmed - >> >> It may sound like a crazy idea but I'd recommend not using the debugger. >> >> Erlang's trace facilities are really great for troubleshooting code during development and production. >> >> The redbug tool is any easy way to get started with Erlang's tracer (https://github.com/massemanet/eper). >> >> >> On 2011/02/01, at 23:28, Ahmed Al-Saadi wrote: >> >>> Sorry if this was double-submitted, but I think that it didn't go to the original group on the first attempt: >>> >>> >>> Hello: >>> >>> I'm a newcomer to Erlang (from Python). I've had great difficulties in using the wx-based debugger and was wondering if there is a command-line alternative a la python's pdb?! >>> >>> It seems that every time I use the default debugger, my Wish app hangs or crashes. Even with the latest release of wxErlang (0.97.1212), I was not even able to set a breakpoint! >>> >>> The use of a good debugger to trace program execution is invaluable in learning a new language--a pedagogical role, in addition to its normal troubleshooting role. >>> >>> Any ideas of how I can improve the situation of my development environment? Is there another command-line debugger? My platform is MacOS X 10.6.6 with Erlang R14B... and, no, changing platforms is not an option ;) Also, if I knew how to write a debugger at this point in time, I would try to do so! Some guidance from a veteran could help. >>> >>> TPG/TEG >> Joseph Wayne Norton >> norton@REDACTED >> >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From ttmrichter@REDACTED Wed Feb 2 10:07:19 2011 From: ttmrichter@REDACTED (Michael Richter) Date: Wed, 2 Feb 2011 17:07:19 +0800 Subject: Jungerl anonymous CVS Message-ID: The CVS command line specified at http://sourceforge.net/projects/jungerl/develop doesn't seem to work. The commands shown are: cvs -d:pserver:anonymous@REDACTED:/cvsroot/jungerl login cvs -z3 -d:pserver:anonymous@REDACTED:/cvsroot/jungerl co -P *modulename* When trying to log in, however, I get asked for a password and it won't accept a blank password, it won't accept "anonymous" and it won't accept anything else reasonable (or unreasonable) I tried. Has the CVS repository been misconfigured or have I don't something wrong? -- "Perhaps people don't believe this, but throughout all of the discussions of entering China our focus has really been what's best for the Chinese people. It's not been about our revenue or profit or whatnot." --Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra. From vladdu55@REDACTED Wed Feb 2 10:12:49 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 2 Feb 2011 10:12:49 +0100 Subject: [erlang-questions] Jungerl anonymous CVS In-Reply-To: References: Message-ID: Hi, On Wed, Feb 2, 2011 at 10:07, Michael Richter wrote: > The CVS command line specified at > http://sourceforge.net/projects/jungerl/develop doesn't seem to work. The > commands shown are: > > cvs -d:pserver:anonymous@REDACTED:/cvsroot/jungerl > login > cvs -z3 -d:pserver:anonymous@REDACTED:/cvsroot/jungerl > co > -P *modulename* > > When trying to log in, however, I get asked for a password and it won't > accept a blank password, it won't accept "anonymous" and it won't accept > anything else reasonable (or unreasonable) I tried. Has the CVS repository > been misconfigured or have I don't something wrong? > I think this is an effect of the recent attack on SourceForge, at the moment many services are blocked. regards, Vlad From billiob@REDACTED Wed Feb 2 10:19:11 2011 From: billiob@REDACTED (Boris 'billiob' Faure) Date: Wed, 2 Feb 2011 10:19:11 +0100 Subject: [erlang-questions] Jungerl anonymous CVS In-Reply-To: References: Message-ID: On Wed, Feb 2, 2011 at 10:12, Vlad Dumitrescu wrote: > I think this is an effect of the recent attack on SourceForge, at the moment > many services are blocked. > > regards, > Vlad > You are right. From http://sourceforge.net/blog/sourceforge-attack-full-report/ : ?CVS service is one of SourceForge.net?s oldest services and, due to limitations in CVS itself, cannot readily live on our scalable network storage solution. Validation of this data is going to require several days and we anticipate that this service will be restored sometime in the later part of week. We are also considering the end-of-life of the CVS service and hope to have user support in migrating CVS users to Subversion in coming months. Subversion generally provides parity to CVS commands, and many of our users have made this transition successfully in the past.? Regards, -- Boris 'billiob' Faure From watson.timothy@REDACTED Wed Feb 2 11:58:45 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 2 Feb 2011 10:58:45 +0000 Subject: [erlang-questions] Erlang and MSMQ In-Reply-To: References: Message-ID: On 2 February 2011 03:59, Marcel Meyer wrote: > Hi there, > > I was wondering if anyone was ever successful in publishing to a MSMQ from > Erlang and how they did it. > > Regards, > Marcel > You might want to look at comet (the erl_com erlang <=> COM bridge component). I'm guessing you can access MSMQ using COM. It'll be a pretty low level interface, but there won't be much overhead as erl_com can run either as an external port program (using stdio to talk to the driver) or a linked-in driver running in the same process space as the emulator. Running a linked-in driver is going to be the lowest overhead, but there are risks of course. Also if you're .NET inclined, then a guy on this list has been doing recent work on .NET/Erlang interoperability, so you could write a wrapper around MSMQ in C# (or whatever CLI language takes your fancy) and call it from Erlang. There will be some overhead with this because there will need to be I/O between the Erlang node(s) and the .NET *node* - but it might make for a clean interface and on the .NET side you should be able to expose the MSMQ endpoint very easily with WCF. A third option might be to write/configure the MQ client part in a .NET language, export a COM Callable Wrapper and consume that via erl_com, thereby getting your cake (i.e., to write the bulk of your code in something reasonable like F# or C#) and eat it - minimising hops and getting good performance. Doing that in linked-in driver mode is probably a very bad idea though - who knows how two runtime would interact in this scenario. Once you've gone to a port program (for the reason I mentioned), moving to a .NET node(s) might be worth the performance overhead. All my suggests are based on pure guesswork anyway - I've never tried to do this before. I have no idea the status of erl_com, although the other .NET integration I mentioned looks fairly active - it is called ExtendedVisualOtp IIRC. Anyway, good luck, and please do post back when you figure out what works! I've been thinking about the same issues for bridging to JMS providers (where currently I use AMQP and convert to JMS using apache camel) so it'll be interesting to see how the MS route turns out. From mitja.thomas1@REDACTED Wed Feb 2 13:08:52 2011 From: mitja.thomas1@REDACTED (Mitja Thomas) Date: Wed, 02 Feb 2011 13:08:52 +0100 Subject: Problem detecting interrupted TCP Connection Message-ID: <4D494954.8010103@ewetel.de> Hello Folks, I'm new to this Mailing List, so my question might already have been answered at some point. Still I got a strange behaviour in a TCP Session which I just cant figur out. Beckground: I have a Server who listens on a Port for incomming TCP Connections and handles them in a User List. When something happens for this User (on the other side, which doesnt matter for the problem), the user is informed and can send commands. When the TCP Connection ist closed, the Socket is closed and the User will be erased from the User list. Problem: When the user somehow disconnects his TCP/IP Connection and closes the client application, my server dont get the tcp_closed Event and the user is still present in the List. Well I know this is the way TCP should behave, thus the server tries to send a packet to the Socket once in while and should receive and Error of some kind. But it does not, it just keeps "sending", without actually sending TCP Packets. The Problem is well explained in gen_tcp manual page and an example how it should be handled in such a way with an application ping is given. But I just dont the {error, timeout} while trying to send to the broken TCP Socket. I broke it down to a simple Server, who just pings the client and tried various ways to do it. Im pretty sure, Im doing something wrong, but I just dont know what. Any Help is appreciated. Regards Mitja sample Code (Client is normally a simple Telnet): -module(tcp). -compile(export_all). start() -> {ok, Listener} = gen_tcp:listen(8999, [binary, {ip, {xxx,xxx,xxx,xxx}}, {active, true}, {packet, 0}, {keepalive, true}, {reuseaddr, true}, {send_timeout_close, true}, {send_timeout, 100}]), io:format("Listener started~n"), acceptor(Listener). acceptor(Listener) -> case gen_tcp:accept(Listener, 1000) of {ok, Socket} -> io:format("Starting client (~p)\n", [Socket]), Pid=spawn_link(fun()->loop(Socket) end), gen_tcp:controlling_process(Socket, Pid), acceptor(Listener); {error, timeout} -> io:format("still running\n"), acceptor(Listener); X -> io:format("SocketError:~p~n", [X]) end. loop(Socket) -> receive {tcp, Socket, Data} -> io:format("Client: Received '~p'\n", [Data]), loop(Socket); {tcp_closed, Socket} -> % This is the desired close Event io:format("Client: Socket closed by remote\n"); {tcp_error, Socket, Reason} -> io:format("Client: Socket error '~p'\n", [Reason]) after 1000 -> case gen_tcp:send(Socket, "Ping\r\n") of Ret -> % If Ping fails the output should be =/= ok io:format("Send ret value:~p~n", [Ret]), loop(Socket) end end. From erlang@REDACTED Wed Feb 2 13:45:49 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 2 Feb 2011 13:45:49 +0100 Subject: [erlang-questions] Problem detecting interrupted TCP Connection In-Reply-To: <4D494954.8010103@ewetel.de> References: <4D494954.8010103@ewetel.de> Message-ID: I tried your code and it worked beautifully - but I had to remove the {ip, {xxx,...}} term - if you omit the IP then you this will default to the local interface then you can do > telnet localhost 8999, or > telnet 127.0.0.1 8999, or > telnet 8999 all worked as expected /Joe On Wed, Feb 2, 2011 at 1:08 PM, Mitja Thomas wrote: > Hello Folks, > > I'm new to this Mailing List, so my question might already have been > answered at some point. Still I got a strange behaviour in a TCP Session > which I just cant figur out. > > Beckground: > I have a Server who listens on a Port for incomming TCP Connections and > handles them in a User List. When something happens for this User (on the > other side, which doesnt matter for the problem), the user is informed and > can send commands. When the TCP Connection ist closed, the Socket is closed > and the User will be erased from the User list. > > Problem: > When the user somehow disconnects his TCP/IP Connection and closes the > client application, my server dont get the tcp_closed Event and the user is > still present in the List. Well I know this is the way TCP should behave, > thus the server tries to send a packet to the Socket once in while and > should receive and Error of some kind. But it does not, it just keeps > "sending", without actually sending TCP Packets. > The Problem is well explained in gen_tcp manual page and an example how it > should be handled in such a way with an application ping is given. But I > just dont the {error, timeout} while trying to send to the broken TCP > Socket. > > I broke it down to a simple Server, who just pings the client and tried > various ways to do it. Im pretty sure, Im doing something wrong, but I just > dont know what. Any Help is appreciated. > > Regards > Mitja > > sample Code (Client is normally a simple Telnet): > -module(tcp). > -compile(export_all). > > start() -> > {ok, Listener} = gen_tcp:listen(8999, [binary, {ip, {xxx,xxx,xxx,xxx}}, > {active, true}, {packet, 0}, {keepalive, true}, {reuseaddr, true}, > {send_timeout_close, true}, {send_timeout, 100}]), > io:format("Listener started~n"), > acceptor(Listener). > > acceptor(Listener) -> > case gen_tcp:accept(Listener, 1000) of > {ok, Socket} -> > io:format("Starting client (~p)\n", [Socket]), > Pid=spawn_link(fun()->loop(Socket) end), > gen_tcp:controlling_process(Socket, Pid), > acceptor(Listener); > > {error, timeout} -> io:format("still running\n"), > acceptor(Listener); > > X -> io:format("SocketError:~p~n", [X]) > end. > > loop(Socket) -> > receive > {tcp, Socket, Data} -> > io:format("Client: Received '~p'\n", [Data]), > loop(Socket); > > {tcp_closed, Socket} -> > % This is the desired close Event > io:format("Client: Socket closed by remote\n"); > > {tcp_error, Socket, Reason} -> > io:format("Client: Socket error '~p'\n", [Reason]) > after > 1000 -> > case gen_tcp:send(Socket, "Ping\r\n") of > Ret -> > % If Ping fails the output should be > =/= ok > io:format("Send ret value:~p~n", > [Ret]), > loop(Socket) > end > end. > > From mitja.thomas1@REDACTED Wed Feb 2 14:21:37 2011 From: mitja.thomas1@REDACTED (Mitja Thomas) Date: Wed, 02 Feb 2011 14:21:37 +0100 Subject: [erlang-questions] Problem detecting interrupted TCP Connection In-Reply-To: References: <4D494954.8010103@ewetel.de> Message-ID: <4D495A61.7050209@ewetel.de> When I reread my mail, I think I should clearify the events that produced the problematic behaviour. - I start the server (I tried it without the {ip, ...} option). - I connect from remote via Telnet. - I then take of the ethernet connection of the client. Quit telnet and wait a minute, so no resendings would come into my way. - I reestablish the Ethernet Conenction from client (actually that wouldnt even matter because the server should recognise the lost connection anyway). The output I get is: 1> tcp:start(). Listener started still running still running Starting client (#Port<0.521>) still running still running still running still running still running %%% Somewhere here I pulled the plug still running still running still running still running Send ret value:ok still running still running still running still running still running still running still running still running still running still running Send ret value:ok still running still running still running still running still running still running still running still running still running still running Send ret value:ok still running ... and so forth. Normally I would have suspected a some recognition of the lost socket at least when the first ping message is sent after the connection is interrupted. > I tried your code and it worked beautifully - but I had to remove the > {ip, {xxx,...}} term - if you omit the IP then you this will default > to the local interface > > then you can do > > telnet localhost 8999, or > > telnet 127.0.0.1 8999, or > > telnet 8999 > > all worked as expected > > /Joe > > > On Wed, Feb 2, 2011 at 1:08 PM, Mitja Thomas > wrote: > > Hello Folks, > > I'm new to this Mailing List, so my question might already have > been answered at some point. Still I got a strange behaviour in a > TCP Session which I just cant figur out. > > Beckground: > I have a Server who listens on a Port for incomming TCP > Connections and handles them in a User List. When something > happens for this User (on the other side, which doesnt matter for > the problem), the user is informed and can send commands. When the > TCP Connection ist closed, the Socket is closed and the User will > be erased from the User list. > > Problem: > When the user somehow disconnects his TCP/IP Connection and closes > the client application, my server dont get the tcp_closed Event > and the user is still present in the List. Well I know this is the > way TCP should behave, thus the server tries to send a packet to > the Socket once in while and should receive and Error of some > kind. But it does not, it just keeps "sending", without actually > sending TCP Packets. > The Problem is well explained in gen_tcp manual page and an > example how it should be handled in such a way with an application > ping is given. But I just dont the {error, timeout} while trying > to send to the broken TCP Socket. > > I broke it down to a simple Server, who just pings the client and > tried various ways to do it. Im pretty sure, Im doing something > wrong, but I just dont know what. Any Help is appreciated. > > Regards > Mitja > > sample Code (Client is normally a simple Telnet): > -module(tcp). > -compile(export_all). > > start() -> > {ok, Listener} = gen_tcp:listen(8999, [binary, {ip, > {xxx,xxx,xxx,xxx}}, {active, true}, {packet, 0}, {keepalive, > true}, {reuseaddr, true}, {send_timeout_close, true}, > {send_timeout, 100}]), > io:format("Listener started~n"), > acceptor(Listener). > > acceptor(Listener) -> > case gen_tcp:accept(Listener, 1000) of > {ok, Socket} -> > io:format("Starting client (~p)\n", [Socket]), > Pid=spawn_link(fun()->loop(Socket) end), > gen_tcp:controlling_process(Socket, Pid), > acceptor(Listener); > > {error, timeout} -> io:format("still running\n"), > acceptor(Listener); > > X -> io:format("SocketError:~p~n", [X]) > end. > > loop(Socket) -> > receive > {tcp, Socket, Data} -> > io:format("Client: Received '~p'\n", [Data]), > loop(Socket); > > {tcp_closed, Socket} -> > % This is the desired close Event > io:format("Client: Socket closed by remote\n"); > > {tcp_error, Socket, Reason} -> > io:format("Client: Socket error '~p'\n", [Reason]) > after > 1000 -> > case gen_tcp:send(Socket, "Ping\r\n") of > Ret -> > % If Ping fails the output > should be =/= ok > io:format("Send ret > value:~p~n", [Ret]), > loop(Socket) > end > end. > > From lenartlad@REDACTED Wed Feb 2 14:20:34 2011 From: lenartlad@REDACTED (Ladislav Lenart) Date: Wed, 02 Feb 2011 14:20:34 +0100 Subject: [erlang-questions] Problem detecting interrupted TCP Connection In-Reply-To: References: <4D494954.8010103@ewetel.de> Message-ID: <4D495A22.3070007@volny.cz> Hello, I might be wrong, but I think that opening a socket in the active mode introduces a race condition: the socket can deliver TCP messages to the acceptor process instead of the loop process even before the call to set a new controlling process. As a general recommendation from discussions on this list I'd say that the socket should be opened in {active, false} mode, then the control should be transferred to the appropriate process which should set mode to {active, once} whenever it is ready to handle another TCP message and thus implement a form of flow control. Please correct me if this is not so. HTH, Ladislav Lenart Joe Armstrong wrote: > I tried your code and it worked beautifully - but I had to remove the > {ip, {xxx,...}} term - if you omit the IP then you this will default to the > local interface > > then you can do > > telnet localhost 8999, or > > telnet 127.0.0.1 8999, or > > telnet 8999 > > all worked as expected > > /Joe > > > On Wed, Feb 2, 2011 at 1:08 PM, Mitja Thomas wrote: > >> Hello Folks, >> >> I'm new to this Mailing List, so my question might already have been >> answered at some point. Still I got a strange behaviour in a TCP Session >> which I just cant figur out. >> >> Beckground: >> I have a Server who listens on a Port for incomming TCP Connections and >> handles them in a User List. When something happens for this User (on the >> other side, which doesnt matter for the problem), the user is informed and >> can send commands. When the TCP Connection ist closed, the Socket is closed >> and the User will be erased from the User list. >> >> Problem: >> When the user somehow disconnects his TCP/IP Connection and closes the >> client application, my server dont get the tcp_closed Event and the user is >> still present in the List. Well I know this is the way TCP should behave, >> thus the server tries to send a packet to the Socket once in while and >> should receive and Error of some kind. But it does not, it just keeps >> "sending", without actually sending TCP Packets. >> The Problem is well explained in gen_tcp manual page and an example how it >> should be handled in such a way with an application ping is given. But I >> just dont the {error, timeout} while trying to send to the broken TCP >> Socket. >> >> I broke it down to a simple Server, who just pings the client and tried >> various ways to do it. Im pretty sure, Im doing something wrong, but I just >> dont know what. Any Help is appreciated. >> >> Regards >> Mitja >> >> sample Code (Client is normally a simple Telnet): >> -module(tcp). >> -compile(export_all). >> >> start() -> >> {ok, Listener} = gen_tcp:listen(8999, [binary, {ip, {xxx,xxx,xxx,xxx}}, >> {active, true}, {packet, 0}, {keepalive, true}, {reuseaddr, true}, >> {send_timeout_close, true}, {send_timeout, 100}]), >> io:format("Listener started~n"), >> acceptor(Listener). >> >> acceptor(Listener) -> >> case gen_tcp:accept(Listener, 1000) of >> {ok, Socket} -> >> io:format("Starting client (~p)\n", [Socket]), >> Pid=spawn_link(fun()->loop(Socket) end), >> gen_tcp:controlling_process(Socket, Pid), >> acceptor(Listener); >> >> {error, timeout} -> io:format("still running\n"), >> acceptor(Listener); >> >> X -> io:format("SocketError:~p~n", [X]) >> end. >> >> loop(Socket) -> >> receive >> {tcp, Socket, Data} -> >> io:format("Client: Received '~p'\n", [Data]), >> loop(Socket); >> >> {tcp_closed, Socket} -> >> % This is the desired close Event >> io:format("Client: Socket closed by remote\n"); >> >> {tcp_error, Socket, Reason} -> >> io:format("Client: Socket error '~p'\n", [Reason]) >> after >> 1000 -> >> case gen_tcp:send(Socket, "Ping\r\n") of >> Ret -> >> % If Ping fails the output should be >> =/= ok >> io:format("Send ret value:~p~n", >> [Ret]), >> loop(Socket) >> end >> end. >> >> > From jameschurchman@REDACTED Wed Feb 2 16:01:42 2011 From: jameschurchman@REDACTED (James Churchman) Date: Wed, 2 Feb 2011 15:01:42 +0000 Subject: [erlang-questions] Command-line Debugger? In-Reply-To: <4D48E974.1020109@gmail.com> References: <4D481892.3090708@gmail.com> <4D48E974.1020109@gmail.com> Message-ID: try erlide, a plugin for eclipse, http://erlide.org/ :-) On 2 Feb 2011, at 05:19, Ahmed Al-Saadi wrote: > Dan: > > It seems that the Homebrew Erlang installation on my Mac did not include the wx package! I downloaded the latest tarball from erlang.org and, so far, the previous problems with the debugger haven't manifested themselves. > > I do have a first-generation macbook which has a 32-bit processor and I did install wxWidgets myself, so that must have avoided the pitfalls you mentioned. > > Thanks for your input... > > Cheers, > Ahmed > > > On 2/1/11 10:45 AM, Dan Gudmundsson wrote: >> Just for the information, the latest wx is 0.98.8 and is included with erlang. >> The old sourceforge site is dead... except some occasional support mail. >> >> But on MacOsX it only works for 32bits erlang currently, and also >> requires the user to install >> and build wxWidgets, the one included in snow leopard is buggy. >> >> /Dan >> >> On Tue, Feb 1, 2011 at 3:39 PM, Joseph Wayne Norton >> wrote: >>> Ahmed - >>> >>> It may sound like a crazy idea but I'd recommend not using the debugger. >>> >>> Erlang's trace facilities are really great for troubleshooting code during development and production. >>> >>> The redbug tool is any easy way to get started with Erlang's tracer (https://github.com/massemanet/eper). >>> >>> >>> On 2011/02/01, at 23:28, Ahmed Al-Saadi wrote: >>> >>>> Sorry if this was double-submitted, but I think that it didn't go to the original group on the first attempt: >>>> >>>> >>>> Hello: >>>> >>>> I'm a newcomer to Erlang (from Python). I've had great difficulties in using the wx-based debugger and was wondering if there is a command-line alternative a la python's pdb?! >>>> >>>> It seems that every time I use the default debugger, my Wish app hangs or crashes. Even with the latest release of wxErlang (0.97.1212), I was not even able to set a breakpoint! >>>> >>>> The use of a good debugger to trace program execution is invaluable in learning a new language--a pedagogical role, in addition to its normal troubleshooting role. >>>> >>>> Any ideas of how I can improve the situation of my development environment? Is there another command-line debugger? My platform is MacOS X 10.6.6 with Erlang R14B... and, no, changing platforms is not an option ;) Also, if I knew how to write a debugger at this point in time, I would try to do so! Some guidance from a veteran could help. >>>> >>>> TPG/TEG >>> Joseph Wayne Norton >>> norton@REDACTED >>> >>> >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From dmitriy.kargapolov@REDACTED Wed Feb 2 16:04:45 2011 From: dmitriy.kargapolov@REDACTED (Dmitriy Kargapolov) Date: Wed, 2 Feb 2011 10:04:45 -0500 Subject: process trace triggered by sequential trace token Message-ID: Hi All, I'd like to trace gen_server body execution, not for all calls, but for particular one. Is there a way to enable trace for the process by received (and matched) message "contaminated" by trace token? (And disable it at the end of message processing). I know there is a way to turn sequential trace by matching regular function call, but I need things to go in reverse order... Thanks. From marcel.meyer@REDACTED Wed Feb 2 16:21:40 2011 From: marcel.meyer@REDACTED (Marcel Meyer) Date: Wed, 2 Feb 2011 10:21:40 -0500 Subject: [erlang-questions] Erlang and MSMQ In-Reply-To: References: Message-ID: Thank you for the pointers Tim, and the Erl->.Net->MSMQ path sound the shortest. I'll let you know when I've done it. We have parts of our app running on NServiceBus and it would be nice to interface to it. Kind regards, Marcel On Wed, Feb 2, 2011 at 5:58 AM, Tim Watson wrote: > On 2 February 2011 03:59, Marcel Meyer wrote: > > Hi there, > > > > I was wondering if anyone was ever successful in publishing to a MSMQ > from > > Erlang and how they did it. > > > > Regards, > > Marcel > > > > You might want to look at comet (the erl_com erlang <=> COM bridge > component). I'm guessing you can access MSMQ using COM. It'll be a > pretty low level interface, but there won't be much overhead as > erl_com can run either as an external port program (using stdio to > talk to the driver) or a linked-in driver running in the same process > space as the emulator. Running a linked-in driver is going to be the > lowest overhead, but there are risks of course. > > Also if you're .NET inclined, then a guy on this list has been doing > recent work on .NET/Erlang interoperability, so you could write a > wrapper around MSMQ in C# (or whatever CLI language takes your fancy) > and call it from Erlang. There will be some overhead with this because > there will need to be I/O between the Erlang node(s) and the .NET > *node* - but it might make for a clean interface and on the .NET side > you should be able to expose the MSMQ endpoint very easily with WCF. > > A third option might be to write/configure the MQ client part in a > .NET language, export a COM Callable Wrapper and consume that via > erl_com, thereby getting your cake (i.e., to write the bulk of your > code in something reasonable like F# or C#) and eat it - minimising > hops and getting good performance. Doing that in linked-in driver mode > is probably a very bad idea though - who knows how two runtime would > interact in this scenario. Once you've gone to a port program (for the > reason I mentioned), moving to a .NET node(s) might be worth the > performance overhead. > > All my suggests are based on pure guesswork anyway - I've never tried > to do this before. I have no idea the status of erl_com, although the > other .NET integration I mentioned looks fairly active - it is called > ExtendedVisualOtp IIRC. > > Anyway, good luck, and please do post back when you figure out what > works! I've been thinking about the same issues for bridging to JMS > providers (where currently I use AMQP and convert to JMS using apache > camel) so it'll be interesting to see how the MS route turns out. > From jameschurchman@REDACTED Wed Feb 2 16:37:36 2011 From: jameschurchman@REDACTED (James Churchman) Date: Wed, 2 Feb 2011 15:37:36 +0000 Subject: [erlang-questions] gen_server clustering strategy In-Reply-To: References: Message-ID: all great questions, most of which i don't know the answers to, but queuing systems can help if a bit of extra latency is ok, and more so if the order does not have to be exact i think at very large scale hashing becomes the only answer, and handling failed nodes becomes tricky, but a basic distributed mnesia table that contains the number of alive nodes and maybe reports on their repeated failures, and then hashes the input (on userid in your case) to the correct node would get you half way there. Also i don't think that 100,000 processes that peak at 300kb is a problem for erlang. Thats only 30GB max. If its stored as binaries, and possibly after a really large message you invoke the garbage collector for that process ( i have no idea if this is a bad idea or not, but can be done easily and should reduce memory consumption) then a single box should be able to handle your needs easily, and 3 no probs at all. My mac pro should be ok :-) i think dedicated hardware is always better than virtualised As for 1000 messages at 300kb, that seems like quite a lot, bordering on what standard gigabit ethernet can handle, but just benchmark erlang, again one box might be enough... you can get a 32 core amd server for not much these days. Are these requirements realistic tho, or just "if 10 million people use my service on day one that i have put together on a shoe string etc... wishful thinking" On 1 Feb 2011, at 19:10, Evans, Matthew wrote: > Without knowing much more another model would be to use your own ets/mnesia table to map the workers. > > I am assuming the creation of a worker is relatively infrequent (compared to the times you need to find a worker). > > When you create a worker use gen_server:start(?MODULE, [], []). > > This will return {ok,Pid}. You can save the Pid in an mnesia table along with whatever reference you need (doesn't need to be an atom). All nodes in the mesh will get the pid and the reference. Starting it as shown above (with an arity of 3) means you don't need to use the global service and ensure the names are unique atoms for each server. You can even do start via some application that will spread the gen_servers over different nodes. > > When a request comes in it only need do a lookup to find the correct pid. > > I can't recall as to how bad the process crash messaging is. But what you could do is to monitor a gen_server locally, and monitor each node globally. When a node fails all other nodes can cleanup processes that were registered on that node. > > Matt > > ________________________________________ > From: erlang-questions@REDACTED [erlang-questions@REDACTED] On Behalf Of Knut Nesheim [knut.nesheim@REDACTED] > Sent: Tuesday, February 01, 2011 12:25 PM > To: erlang-questions@REDACTED > Subject: [erlang-questions] gen_server clustering strategy > > Hello list, > > We are interested in understanding how the Erlang distribution > mechanism behaves in a fully connected cluster. Our concern is that as > we add nodes, the overhead will become a problem. > > Our application is a bunch of gen_servers containing state, which we > need to run on multiple nodes due to memory usage. A request will come > to our application and it needs to be handled by the correct > gen_server. Every request includes a user id which we can use to map > to the process. > > We have two specific use cases in mind. > > 1) Proxy to workers > > In this scenario we have a proxy (or multiple proxies) accepting > requests at the edge of our cluster. The proxy will ask the correct > server for a response. Either through using the 'global' module or > gproc or something similar, the proxy node will keep track of the > mapping of user ids to process and nodes. The proxy will call the node > using the Erlang distribution protocol. > > 2) Mesh > > In this scenario a request can be handled by any node in the cluster. > If the request cannot be handled by the local node, the correct node > is called on instead. Every node in the cluster needs to keep track of > which id belongs to which process. > > > The numbers: > * We must be able to run 100,000 processes, each may peak at 300kb of memory > * We expect 5000 requests coming in to our application per second at > peak, generating the same number of messages > * Out of those 5000 messages, 1000 has a content that may peak at 300kb > * All the other messages are well under 1kb > > Our concerns: > > * In the mesh, will the traffic between the nodes become a problem? > Lets say we have 4 nodes, if the requests are evenly divided between > the nodes the probability of hitting the correct node is 25%. With 100 > nodes this is 1%. As we add nodes, there will be more chatter. May > this chatter "drown-out" the important messages, like pid down, etc. > > * Will it be more efficient to use the proxy to message every node? > Will the message then always go to the node directly, or may it go > through some other node in the cluster? > > * For the mesh, we need to keep the 'global' or gproc state > replicated. With 'global' we will run into the default limit of atoms. > If we were to increase this limit, what kind of memory usage could we > expect? Are there any other nasty side-effects that we should be aware > of? > > * In the "Erlang/OTP in Action" book it is stated that you may have a > "couple of dozen, but probably not hundreds of nodes." Our > understanding is that this is because of traffic like "this pid just > died", "this node is no longer available", etc. > > * If we were to eliminate messaging between nodes, how many nodes > could we actually run in a fully connected cluster? Will a > high-latency network like ec2 affect this number or just the latency > of messages? What else may affect the size? > > > Regards > Knut > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From kaiduanx@REDACTED Wed Feb 2 17:04:03 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 2 Feb 2011 11:04:03 -0500 Subject: [erlang-questions] process trace triggered by sequential trace token In-Reply-To: References: Message-ID: Dimitriy, I On Wed, Feb 2, 2011 at 10:04 AM, Dmitriy Kargapolov wrote: > Hi All, > I'd like to trace gen_server body execution, not for all calls, but for > particular one. > Is there a way to enable trace for the process by received (and matched) > message "contaminated" by trace token? (And disable it at the end of message > processing). I know there is a way to turn sequential trace by matching > regular function call, but I need things to go in reverse order... > Thanks. > From kaiduanx@REDACTED Wed Feb 2 17:07:04 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 2 Feb 2011 11:07:04 -0500 Subject: [erlang-questions] process trace triggered by sequential trace token In-Reply-To: References: Message-ID: Sorry for hitting wrong key. I have successfully traced a particular call from user A to user B based on the headers in SIP message in a system that has many concurrent calls using sequential trace. I do not know if this is what you want. Best regards, /Kaiduan On Wed, Feb 2, 2011 at 11:04 AM, Kaiduan Xie wrote: > Dimitriy, > > I > > On Wed, Feb 2, 2011 at 10:04 AM, Dmitriy Kargapolov > wrote: >> Hi All, >> I'd like to trace gen_server body execution, not for all calls, but for >> particular one. >> Is there a way to enable trace for the process by received (and matched) >> message "contaminated" by trace token? (And disable it at the end of message >> processing). I know there is a way to turn sequential trace by matching >> regular function call, but I need things to go in reverse order... >> Thanks. >> > From mitja.thomas1@REDACTED Wed Feb 2 17:27:08 2011 From: mitja.thomas1@REDACTED (Mitja Thomas) Date: Wed, 02 Feb 2011 17:27:08 +0100 Subject: [erlang-questions] Problem detecting interrupted TCP Connection In-Reply-To: <4D495A22.3070007@volny.cz> References: <4D494954.8010103@ewetel.de> <4D495A22.3070007@volny.cz> Message-ID: <4D4985DC.1080608@ewetel.de> Hi, I wont even argue there. Flow Control via {active, false} is really relevant especially at connection establishment. But I tried it either way. This is just a very simplified version. The connection works fine, the server recognises that a client has connected. The problem occurs later, which I guess you can see in my server Output. Mitja > Hello, > > I might be wrong, but I think that opening a socket in the active mode > introduces a race condition: the socket can deliver TCP messages to the > acceptor process instead of the loop process even before the call to set > a new controlling process. > > As a general recommendation from discussions on this list I'd say that > the socket should be opened in {active, false} mode, then the control > should be transferred to the appropriate process which should set mode > to {active, once} whenever it is ready to handle another TCP message > and thus implement a form of flow control. > > > Please correct me if this is not so. > > > HTH, > > Ladislav Lenart > > > Joe Armstrong wrote: >> I tried your code and it worked beautifully - but I had to remove the >> {ip, {xxx,...}} term - if you omit the IP then you this will default >> to the >> local interface >> >> then you can do >> > telnet localhost 8999, or >> > telnet 127.0.0.1 8999, or >> > telnet 8999 >> >> all worked as expected >> >> /Joe >> >> >> On Wed, Feb 2, 2011 at 1:08 PM, Mitja Thomas >> wrote: >> >>> Hello Folks, >>> >>> I'm new to this Mailing List, so my question might already have been >>> answered at some point. Still I got a strange behaviour in a TCP >>> Session >>> which I just cant figur out. >>> >>> Beckground: >>> I have a Server who listens on a Port for incomming TCP Connections and >>> handles them in a User List. When something happens for this User >>> (on the >>> other side, which doesnt matter for the problem), the user is >>> informed and >>> can send commands. When the TCP Connection ist closed, the Socket is >>> closed >>> and the User will be erased from the User list. >>> >>> Problem: >>> When the user somehow disconnects his TCP/IP Connection and closes the >>> client application, my server dont get the tcp_closed Event and the >>> user is >>> still present in the List. Well I know this is the way TCP should >>> behave, >>> thus the server tries to send a packet to the Socket once in while and >>> should receive and Error of some kind. But it does not, it just keeps >>> "sending", without actually sending TCP Packets. >>> The Problem is well explained in gen_tcp manual page and an example >>> how it >>> should be handled in such a way with an application ping is given. >>> But I >>> just dont the {error, timeout} while trying to send to the broken TCP >>> Socket. >>> >>> I broke it down to a simple Server, who just pings the client and tried >>> various ways to do it. Im pretty sure, Im doing something wrong, but >>> I just >>> dont know what. Any Help is appreciated. >>> >>> Regards >>> Mitja >>> >>> sample Code (Client is normally a simple Telnet): >>> -module(tcp). >>> -compile(export_all). >>> >>> start() -> >>> {ok, Listener} = gen_tcp:listen(8999, [binary, {ip, >>> {xxx,xxx,xxx,xxx}}, >>> {active, true}, {packet, 0}, {keepalive, true}, {reuseaddr, true}, >>> {send_timeout_close, true}, {send_timeout, 100}]), >>> io:format("Listener started~n"), >>> acceptor(Listener). >>> >>> acceptor(Listener) -> >>> case gen_tcp:accept(Listener, 1000) of >>> {ok, Socket} -> >>> io:format("Starting client (~p)\n", [Socket]), >>> Pid=spawn_link(fun()->loop(Socket) end), >>> gen_tcp:controlling_process(Socket, Pid), >>> acceptor(Listener); >>> >>> {error, timeout} -> io:format("still running\n"), >>> acceptor(Listener); >>> >>> X -> io:format("SocketError:~p~n", [X]) >>> end. >>> >>> loop(Socket) -> >>> receive >>> {tcp, Socket, Data} -> >>> io:format("Client: Received '~p'\n", [Data]), >>> loop(Socket); >>> >>> {tcp_closed, Socket} -> >>> % This is the desired close Event >>> io:format("Client: Socket closed by remote\n"); >>> >>> {tcp_error, Socket, Reason} -> >>> io:format("Client: Socket error '~p'\n", [Reason]) >>> after >>> 1000 -> >>> case gen_tcp:send(Socket, "Ping\r\n") of >>> Ret -> >>> % If Ping fails the output >>> should be >>> =/= ok >>> io:format("Send ret value:~p~n", >>> [Ret]), >>> loop(Socket) >>> end >>> end. >>> >>> >> > > > -- Mitja Thomas GE FD L?sungsmanagement Telefon: 0441 8000 - 4916 E-Mail: mitja.thomas@REDACTED ___________________________________ EWE TEL GmbH Cloppenburger Stra?e 310 26133 Oldenburg Handelsregister Amtsgericht Oldenburg HRB 3723 Vorsitzender des Aufsichtsrates: Dr. Werner Brinker Gesch?ftsf?hrung: Konrad Meier (Vorsitzender), Dirk Brameier, Ulf Heggenberger, Norbert Westfal Homepage: http://www.ewetel.de Wir stehen f?r starken Service. Versprochen! +++ Besuchen Sie uns auf der Hausmesse der Finanz Informatik vom 16. bis zum 18.11.2010 in Frankfurt a.M. Sie finden uns auf dem Stand P-04.13 +++ From dmitriy.kargapolov@REDACTED Wed Feb 2 17:40:22 2011 From: dmitriy.kargapolov@REDACTED (Dmitriy Kargapolov) Date: Wed, 2 Feb 2011 11:40:22 -0500 Subject: [erlang-questions] process trace triggered by sequential trace token In-Reply-To: References: Message-ID: Kaiduan, I'd like to trace function calls (per match specs) in the body (callback) of gen_server for particular call to this gen_server. I'd like to do this on existing code, meaning no code alteration should be done to this gen_server process. If this is what you were able to do, would you please share your experience? Thank you. On Wed, Feb 2, 2011 at 11:07 AM, Kaiduan Xie wrote: > Sorry for hitting wrong key. > > I have successfully traced a particular call from user A to user B > based on the headers in SIP message in a system that has many > concurrent calls using sequential trace. I do not know if this is what > you want. > > Best regards, > > /Kaiduan > > On Wed, Feb 2, 2011 at 11:04 AM, Kaiduan Xie wrote: > > Dimitriy, > > > > I > > > > On Wed, Feb 2, 2011 at 10:04 AM, Dmitriy Kargapolov > > wrote: > >> Hi All, > >> I'd like to trace gen_server body execution, not for all calls, but for > >> particular one. > >> Is there a way to enable trace for the process by received (and matched) > >> message "contaminated" by trace token? (And disable it at the end of > message > >> processing). I know there is a way to turn sequential trace by matching > >> regular function call, but I need things to go in reverse order... > >> Thanks. > >> > > > From mevans@REDACTED Wed Feb 2 17:40:25 2011 From: mevans@REDACTED (Evans, Matthew) Date: Wed, 2 Feb 2011 11:40:25 -0500 Subject: [erlang-questions] gen_server clustering strategy In-Reply-To: References: , Message-ID: My worry is the OP is suggesting using the global name server. This has a few problems 1) That he will soon run out of the atom table space (I think he's making the assumption that gen_servers need to be registered, which is not the case). 2) Sharing process state information between all nodes 3) All all lookups need to go to a central gen_server (the global name-server) running on a single logical core. (I might be wrong about point 3). But actually a hashing function could be a good idea, at least to find the correct node. This could negate the need to share process availability information between host/node boundaries. One could maintain a list of current nodes in the mesh, and then use a hash to find the correct node. Identifier to pid still needs to be mapped, but this can be done by a local ets table (optimized and fast for lookups). For example: process_inbound_request(Message,Identifier) -> Nodes = nodes()++[node()], Id = erlang:phash2(Identifier, length(Nodes)), case lists:nth(Id+1,Nodes) of node() -> % On same host, lookup locally to find the correct instance dispatch_locally(Message.Identifier); Node -> dispatch_remotely(Message,Identifier) end. dispatch_locally(Message,Identifier) -> case ets:lookup(local_pids, Identifier) -> [{_,Pid}] -> gen_server:cast(Pid,{inbound,Message}); _ -> ok; end. dispatch_remotely(Message,Identifier) -> gen_server:cast({local_proxy,Node},{request,Message,Identifier}). local_proxy is a locally registered gen_server on each node that implements the equivalent of the dispatch_locally function. Matt ________________________________________ From: James Churchman [jameschurchman@REDACTED] Sent: Wednesday, February 02, 2011 10:37 AM To: Evans, Matthew Cc: Knut Nesheim; erlang-questions@REDACTED Subject: Re: [erlang-questions] gen_server clustering strategy all great questions, most of which i don't know the answers to, but queuing systems can help if a bit of extra latency is ok, and more so if the order does not have to be exact i think at very large scale hashing becomes the only answer, and handling failed nodes becomes tricky, but a basic distributed mnesia table that contains the number of alive nodes and maybe reports on their repeated failures, and then hashes the input (on userid in your case) to the correct node would get you half way there. Also i don't think that 100,000 processes that peak at 300kb is a problem for erlang. Thats only 30GB max. If its stored as binaries, and possibly after a really large message you invoke the garbage collector for that process ( i have no idea if this is a bad idea or not, but can be done easily and should reduce memory consumption) then a single box should be able to handle your needs easily, and 3 no probs at all. My mac pro should be ok :-) i think dedicated hardware is always better than virtualised As for 1000 messages at 300kb, that seems like quite a lot, bordering on what standard gigabit ethernet can handle, but just benchmark erlang, again one box might be enough... you can get a 32 core amd server for not much these days. Are these requirements realistic tho, or just "if 10 million people use my service on day one that i have put together on a shoe string etc... wishful thinking" On 1 Feb 2011, at 19:10, Evans, Matthew wrote: > Without knowing much more another model would be to use your own ets/mnesia table to map the workers. > > I am assuming the creation of a worker is relatively infrequent (compared to the times you need to find a worker). > > When you create a worker use gen_server:start(?MODULE, [], []). > > This will return {ok,Pid}. You can save the Pid in an mnesia table along with whatever reference you need (doesn't need to be an atom). All nodes in the mesh will get the pid and the reference. Starting it as shown above (with an arity of 3) means you don't need to use the global service and ensure the names are unique atoms for each server. You can even do start via some application that will spread the gen_servers over different nodes. > > When a request comes in it only need do a lookup to find the correct pid. > > I can't recall as to how bad the process crash messaging is. But what you could do is to monitor a gen_server locally, and monitor each node globally. When a node fails all other nodes can cleanup processes that were registered on that node. > > Matt > > ________________________________________ > From: erlang-questions@REDACTED [erlang-questions@REDACTED] On Behalf Of Knut Nesheim [knut.nesheim@REDACTED] > Sent: Tuesday, February 01, 2011 12:25 PM > To: erlang-questions@REDACTED > Subject: [erlang-questions] gen_server clustering strategy > > Hello list, > > We are interested in understanding how the Erlang distribution > mechanism behaves in a fully connected cluster. Our concern is that as > we add nodes, the overhead will become a problem. > > Our application is a bunch of gen_servers containing state, which we > need to run on multiple nodes due to memory usage. A request will come > to our application and it needs to be handled by the correct > gen_server. Every request includes a user id which we can use to map > to the process. > > We have two specific use cases in mind. > > 1) Proxy to workers > > In this scenario we have a proxy (or multiple proxies) accepting > requests at the edge of our cluster. The proxy will ask the correct > server for a response. Either through using the 'global' module or > gproc or something similar, the proxy node will keep track of the > mapping of user ids to process and nodes. The proxy will call the node > using the Erlang distribution protocol. > > 2) Mesh > > In this scenario a request can be handled by any node in the cluster. > If the request cannot be handled by the local node, the correct node > is called on instead. Every node in the cluster needs to keep track of > which id belongs to which process. > > > The numbers: > * We must be able to run 100,000 processes, each may peak at 300kb of memory > * We expect 5000 requests coming in to our application per second at > peak, generating the same number of messages > * Out of those 5000 messages, 1000 has a content that may peak at 300kb > * All the other messages are well under 1kb > > Our concerns: > > * In the mesh, will the traffic between the nodes become a problem? > Lets say we have 4 nodes, if the requests are evenly divided between > the nodes the probability of hitting the correct node is 25%. With 100 > nodes this is 1%. As we add nodes, there will be more chatter. May > this chatter "drown-out" the important messages, like pid down, etc. > > * Will it be more efficient to use the proxy to message every node? > Will the message then always go to the node directly, or may it go > through some other node in the cluster? > > * For the mesh, we need to keep the 'global' or gproc state > replicated. With 'global' we will run into the default limit of atoms. > If we were to increase this limit, what kind of memory usage could we > expect? Are there any other nasty side-effects that we should be aware > of? > > * In the "Erlang/OTP in Action" book it is stated that you may have a > "couple of dozen, but probably not hundreds of nodes." Our > understanding is that this is because of traffic like "this pid just > died", "this node is no longer available", etc. > > * If we were to eliminate messaging between nodes, how many nodes > could we actually run in a fully connected cluster? Will a > high-latency network like ec2 affect this number or just the latency > of messages? What else may affect the size? > > > Regards > Knut > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From kaiduanx@REDACTED Wed Feb 2 18:38:06 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 2 Feb 2011 12:38:06 -0500 Subject: [erlang-questions] process trace triggered by sequential trace token In-Reply-To: References: Message-ID: Dmitriy, Please correct me if my understanding is wrong. You would like to trace a particular function of a particular gen_server without changing the code. Do you know the pid of the gen_server? How you know the pid of the interested gen_server? Best regards, /Kaiduan On Wed, Feb 2, 2011 at 11:40 AM, Dmitriy Kargapolov wrote: > Kaiduan, > I'd like to trace function calls (per match specs) in the body (callback) of > gen_server for particular call to this gen_server. I'd like to do this on > existing code, meaning no code alteration should be done to this gen_server > process. If this is what you were able to do, would you please share your > experience? > Thank you. > > On Wed, Feb 2, 2011 at 11:07 AM, Kaiduan Xie wrote: >> >> Sorry for hitting wrong key. >> >> I have successfully traced a particular call from user A to user B >> based on the headers in SIP message in a system that has many >> concurrent calls using sequential trace. I do not know if this is what >> you want. >> >> Best regards, >> >> /Kaiduan >> >> On Wed, Feb 2, 2011 at 11:04 AM, Kaiduan Xie wrote: >> > Dimitriy, >> > >> > I >> > >> > On Wed, Feb 2, 2011 at 10:04 AM, Dmitriy Kargapolov >> > wrote: >> >> Hi All, >> >> I'd like to trace gen_server body execution, not for all calls, but for >> >> particular one. >> >> Is there a way to enable trace for the process by received (and >> >> matched) >> >> message "contaminated" by trace token? (And disable it at the end of >> >> message >> >> processing). I know there is a way to turn sequential trace by matching >> >> regular function call, but I need things to go in reverse order... >> >> Thanks. >> >> >> > > > From dmitriy.kargapolov@REDACTED Wed Feb 2 19:14:36 2011 From: dmitriy.kargapolov@REDACTED (Dmitriy Kargapolov) Date: Wed, 2 Feb 2011 13:14:36 -0500 Subject: [erlang-questions] process trace triggered by sequential trace token In-Reply-To: References: Message-ID: Yes, I do know the pid. Its easy to turn tracing on for the particular pid and set of match specs, but the problem is that the gen_server is serving real calls and these calls must not interfere with tracing of one particular call which I would like to trace. Its easy to achieve that adding small code into callback (which for example would check the presence of token and trigger process trace), but the question is if this is possible to do the same without patching the code. I didn't find the way. On Wed, Feb 2, 2011 at 12:38 PM, Kaiduan Xie wrote: > Dmitriy, > > Please correct me if my understanding is wrong. > > You would like to trace a particular function of a particular > gen_server without changing the code. Do you know the pid of the > gen_server? How you know the pid of the interested gen_server? > > Best regards, > > /Kaiduan > > On Wed, Feb 2, 2011 at 11:40 AM, Dmitriy Kargapolov > wrote: > > Kaiduan, > > I'd like to trace function calls (per match specs) in the body (callback) > of > > gen_server for particular call to this gen_server. I'd like to do this on > > existing code, meaning no code alteration should be done to this > gen_server > > process. If this is what you were able to do, would you please share your > > experience? > > Thank you. > > > > On Wed, Feb 2, 2011 at 11:07 AM, Kaiduan Xie wrote: > >> > >> Sorry for hitting wrong key. > >> > >> I have successfully traced a particular call from user A to user B > >> based on the headers in SIP message in a system that has many > >> concurrent calls using sequential trace. I do not know if this is what > >> you want. > >> > >> Best regards, > >> > >> /Kaiduan > >> > >> On Wed, Feb 2, 2011 at 11:04 AM, Kaiduan Xie > wrote: > >> > Dimitriy, > >> > > >> > I > >> > > >> > On Wed, Feb 2, 2011 at 10:04 AM, Dmitriy Kargapolov > >> > wrote: > >> >> Hi All, > >> >> I'd like to trace gen_server body execution, not for all calls, but > for > >> >> particular one. > >> >> Is there a way to enable trace for the process by received (and > >> >> matched) > >> >> message "contaminated" by trace token? (And disable it at the end of > >> >> message > >> >> processing). I know there is a way to turn sequential trace by > matching > >> >> regular function call, but I need things to go in reverse order... > >> >> Thanks. > >> >> > >> > > > > > > From ixmatus@REDACTED Wed Feb 2 19:26:07 2011 From: ixmatus@REDACTED (Parnell Springmeyer) Date: Wed, 2 Feb 2011 10:26:07 -0800 Subject: Consultant Recommendations? Message-ID: Hello, I'm writing this on behalf of a friend who came to me for consulting work to be done in Erlang. I'm too busy ATM (working on my own startup) and told him I would ask around for some recommendations. Anybody have favorites or recommendations for *individuals* that do excellent Erlang consulting/contract work? The project (details require an NDA) is a RESTful API service that would be handling a large number of requests. No UI is needed (that's being handled by their PHP team or something). Thanks in advance. -- Parnell "ixmatus" Springmeyer (http://ixmat.us) From anthonym@REDACTED Wed Feb 2 20:17:02 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Wed, 2 Feb 2011 11:17:02 -0800 Subject: [erlang-questions] gen_server clustering strategy In-Reply-To: References: Message-ID: <20110202191702.GA34126@alumni.caltech.edu> Riak Core actually does a bunch of this stuff for you http://blog.basho.com/category/riak-core/ https://github.com/basho/riak_core It will manage a partitioned ring sending reads and writes to some number of nodes, and is great a library. We use it as a bridge between a thrift server and a custom storage module, and it allows us to scale out as necessary. Also according to riak FAQ, the largest cluster they've run is about 60 nodes http://wiki.basho.com/FAQ.html#What-is-the-largest-cluster-currently-running? so that's at least one example of scaling into double digits. -Anthony On Wed, Feb 02, 2011 at 11:40:25AM -0500, Evans, Matthew wrote: > My worry is the OP is suggesting using the global name server. This has a few problems > > 1) That he will soon run out of the atom table space (I think he's making the assumption that gen_servers need to be registered, which is not the case). > 2) Sharing process state information between all nodes > 3) All all lookups need to go to a central gen_server (the global name-server) running on a single logical core. > > (I might be wrong about point 3). > > But actually a hashing function could be a good idea, at least to find the correct node. This could negate the need to share process availability information between host/node boundaries. > > One could maintain a list of current nodes in the mesh, and then use a hash to find the correct node. Identifier to pid still needs to be mapped, but this can be done by a local ets table (optimized and fast for lookups). > > For example: > > process_inbound_request(Message,Identifier) -> > Nodes = nodes()++[node()], > Id = erlang:phash2(Identifier, length(Nodes)), > case lists:nth(Id+1,Nodes) of > node() -> > % On same host, lookup locally to find the correct instance > dispatch_locally(Message.Identifier); > Node -> > dispatch_remotely(Message,Identifier) > end. > > dispatch_locally(Message,Identifier) -> > case ets:lookup(local_pids, Identifier) -> > [{_,Pid}] -> > gen_server:cast(Pid,{inbound,Message}); > _ -> > ok; > end. > > dispatch_remotely(Message,Identifier) -> > gen_server:cast({local_proxy,Node},{request,Message,Identifier}). > > local_proxy is a locally registered gen_server on each node that implements the equivalent of the dispatch_locally function. > > Matt > > ________________________________________ > From: James Churchman [jameschurchman@REDACTED] > Sent: Wednesday, February 02, 2011 10:37 AM > To: Evans, Matthew > Cc: Knut Nesheim; erlang-questions@REDACTED > Subject: Re: [erlang-questions] gen_server clustering strategy > > all great questions, most of which i don't know the answers to, but queuing systems can help if a bit of extra latency is ok, and more so if the order does not have to be exact > > i think at very large scale hashing becomes the only answer, and handling failed nodes becomes tricky, but a basic distributed mnesia table that contains the number of alive nodes and maybe reports on their repeated failures, and then hashes the input (on userid in your case) to the correct node would get you half way there. > > Also i don't think that 100,000 processes that peak at 300kb is a problem for erlang. Thats only 30GB max. If its stored as binaries, and possibly after a really large message you invoke the garbage collector for that process ( i have no idea if this is a bad idea or not, but can be done easily and should reduce memory consumption) then a single box should be able to handle your needs easily, and 3 no probs at all. My mac pro should be ok :-) > > i think dedicated hardware is always better than virtualised > > As for 1000 messages at 300kb, that seems like quite a lot, bordering on what standard gigabit ethernet can handle, but just benchmark erlang, again one box might be enough... you can get a 32 core amd server for not much these days. Are these requirements realistic tho, or just "if 10 million people use my service on day one that i have put together on a shoe string etc... wishful thinking" > > > On 1 Feb 2011, at 19:10, Evans, Matthew wrote: > > > Without knowing much more another model would be to use your own ets/mnesia table to map the workers. > > > > I am assuming the creation of a worker is relatively infrequent (compared to the times you need to find a worker). > > > > When you create a worker use gen_server:start(?MODULE, [], []). > > > > This will return {ok,Pid}. You can save the Pid in an mnesia table along with whatever reference you need (doesn't need to be an atom). All nodes in the mesh will get the pid and the reference. Starting it as shown above (with an arity of 3) means you don't need to use the global service and ensure the names are unique atoms for each server. You can even do start via some application that will spread the gen_servers over different nodes. > > > > When a request comes in it only need do a lookup to find the correct pid. > > > > I can't recall as to how bad the process crash messaging is. But what you could do is to monitor a gen_server locally, and monitor each node globally. When a node fails all other nodes can cleanup processes that were registered on that node. > > > > Matt > > > > ________________________________________ > > From: erlang-questions@REDACTED [erlang-questions@REDACTED] On Behalf Of Knut Nesheim [knut.nesheim@REDACTED] > > Sent: Tuesday, February 01, 2011 12:25 PM > > To: erlang-questions@REDACTED > > Subject: [erlang-questions] gen_server clustering strategy > > > > Hello list, > > > > We are interested in understanding how the Erlang distribution > > mechanism behaves in a fully connected cluster. Our concern is that as > > we add nodes, the overhead will become a problem. > > > > Our application is a bunch of gen_servers containing state, which we > > need to run on multiple nodes due to memory usage. A request will come > > to our application and it needs to be handled by the correct > > gen_server. Every request includes a user id which we can use to map > > to the process. > > > > We have two specific use cases in mind. > > > > 1) Proxy to workers > > > > In this scenario we have a proxy (or multiple proxies) accepting > > requests at the edge of our cluster. The proxy will ask the correct > > server for a response. Either through using the 'global' module or > > gproc or something similar, the proxy node will keep track of the > > mapping of user ids to process and nodes. The proxy will call the node > > using the Erlang distribution protocol. > > > > 2) Mesh > > > > In this scenario a request can be handled by any node in the cluster. > > If the request cannot be handled by the local node, the correct node > > is called on instead. Every node in the cluster needs to keep track of > > which id belongs to which process. > > > > > > The numbers: > > * We must be able to run 100,000 processes, each may peak at 300kb of memory > > * We expect 5000 requests coming in to our application per second at > > peak, generating the same number of messages > > * Out of those 5000 messages, 1000 has a content that may peak at 300kb > > * All the other messages are well under 1kb > > > > Our concerns: > > > > * In the mesh, will the traffic between the nodes become a problem? > > Lets say we have 4 nodes, if the requests are evenly divided between > > the nodes the probability of hitting the correct node is 25%. With 100 > > nodes this is 1%. As we add nodes, there will be more chatter. May > > this chatter "drown-out" the important messages, like pid down, etc. > > > > * Will it be more efficient to use the proxy to message every node? > > Will the message then always go to the node directly, or may it go > > through some other node in the cluster? > > > > * For the mesh, we need to keep the 'global' or gproc state > > replicated. With 'global' we will run into the default limit of atoms. > > If we were to increase this limit, what kind of memory usage could we > > expect? Are there any other nasty side-effects that we should be aware > > of? > > > > * In the "Erlang/OTP in Action" book it is stated that you may have a > > "couple of dozen, but probably not hundreds of nodes." Our > > understanding is that this is because of traffic like "this pid just > > died", "this node is no longer available", etc. > > > > * If we were to eliminate messaging between nodes, how many nodes > > could we actually run in a fully connected cluster? Will a > > high-latency network like ec2 affect this number or just the latency > > of messages? What else may affect the size? > > > > > > Regards > > Knut > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- ------------------------------------------------------------------------ Anthony Molinaro From knutin@REDACTED Wed Feb 2 21:27:37 2011 From: knutin@REDACTED (Knut Ivar Nesheim) Date: Wed, 2 Feb 2011 21:27:37 +0100 Subject: [erlang-questions] gen_server clustering strategy In-Reply-To: <20110202191702.GA34126@alumni.caltech.edu> References: <20110202191702.GA34126@alumni.caltech.edu> Message-ID: Thanks for all the great feedback! To clarify some points: * We need to be able to send messages from every process to any other process, so we could use some form of registry. Using the global module would get us going, but not take us all the way (which is something we can worry about later). * We have considered writing our own process registry, based on ets or mnesia or redis or whatever to perfectly support our use case. We are however interested in standing on the shoulders of giants, especially if we can use a library like gproc, which has been extensively tested and verified(at least without distribution). * We won't hit 10 million users anytime soon, however 2 million daily users four months after marketing starts marketing is realistic. Also, our system cannot break when doubling that number. This means we need to be able to handle at least 5000 requests per second. The numbers are from one of our other products, so we know pretty well what to expect. * We would like to add nodes without disturbing the processes already running, so consistent hashing must be used if hashing Our worry with the cluster of identical nodes, the "mesh" described earlier, is that there will be too many messages passing around. As we add nodes, it will only get worse and we might start hitting the limits of the Erlang distribution itself. Our worry with the cluster that includes "proxy" nodes, is that they must maintain the state of the cluster in order to route accordingly. At least they must know where to send the requests using some form of hashing. To this end, building on top of riak_core might be a very good option. However, we feel that riak_core is pretty complex to understand fully and would prefer a simpler solution. As Matt suggested in his second post, we can make the system simpler by not keeping a global registry of userid to process mapping, but only keep a mapping of userid to the node the process is running on. The node can then itself store the userid to process mapping using gproc or something similar locally. If riak is running on 60 nodes, then the question for us is how much message passing is going on. Our concern is that when we reach 10 or 20 nodes, our architecture will just not cut it anymore as we will increase the message passing. Also, what kind of overhead can we expect if we monitor processes on other nodes? Our understanding is that the information regarding pids that die, will be sent to any node in the cluster anyway, so actually monitoring and receiving the info won't make that much of a difference as it is already there. Thanks for all the great feedback! Regards Knut On Wed, Feb 2, 2011 at 8:17 PM, Anthony Molinaro wrote: > Riak Core actually does a bunch of this stuff for you > > http://blog.basho.com/category/riak-core/ > https://github.com/basho/riak_core > > It will manage a partitioned ring sending reads and writes to some number > of nodes, and is great a library. ?We use it as a bridge between a thrift > server and a custom storage module, and it allows us to scale out as > necessary. > > Also according to riak FAQ, the largest cluster they've run is about 60 > nodes > > http://wiki.basho.com/FAQ.html#What-is-the-largest-cluster-currently-running? > > so that's at least one example of scaling into double digits. > > -Anthony > > On Wed, Feb 02, 2011 at 11:40:25AM -0500, Evans, Matthew wrote: >> My worry is the OP is suggesting using the global name server. This has a few problems >> >> 1) That he will soon run out of the atom table space (I think he's making the assumption that gen_servers need to be registered, which is not the case). >> 2) Sharing process state information between all nodes >> 3) All all lookups need to go to a central gen_server (the global name-server) running on a single logical core. >> >> (I might be wrong about point 3). >> >> But actually a hashing function could be a good idea, at least to find the correct node. This could negate the need to share process availability information between host/node boundaries. >> >> One could maintain a list of current nodes in the mesh, and then use a hash to find the correct node. Identifier to pid still needs to be mapped, but this can be done by a local ets table (optimized and fast for lookups). >> >> For example: >> >> process_inbound_request(Message,Identifier) -> >> ? ? Nodes = nodes()++[node()], >> ? ? Id = erlang:phash2(Identifier, length(Nodes)), >> ? ? case lists:nth(Id+1,Nodes) of >> ? ? ? ? node() -> >> ? ? ? ? ? ? ?% On same host, lookup locally to find the correct instance >> ? ? ? ? ? ? ?dispatch_locally(Message.Identifier); >> ? ? ? ? ?Node -> >> ? ? ? ? ? ? ?dispatch_remotely(Message,Identifier) >> ? ? end. >> >> dispatch_locally(Message,Identifier) -> >> ? ? case ets:lookup(local_pids, Identifier) -> >> ? ? ? ? [{_,Pid}] -> >> ? ? ? ? ? ? gen_server:cast(Pid,{inbound,Message}); >> ? ? ? ?_ -> >> ? ? ? ? ? ? ok; >> ? ? ?end. >> >> dispatch_remotely(Message,Identifier) -> >> ? ? gen_server:cast({local_proxy,Node},{request,Message,Identifier}). >> >> local_proxy is a locally registered gen_server on each node that implements the equivalent of the dispatch_locally function. >> >> Matt >> >> ________________________________________ >> From: James Churchman [jameschurchman@REDACTED] >> Sent: Wednesday, February 02, 2011 10:37 AM >> To: Evans, Matthew >> Cc: Knut Nesheim; erlang-questions@REDACTED >> Subject: Re: [erlang-questions] gen_server clustering strategy >> >> all great questions, most of which i don't know the answers to, but queuing systems can help if a bit of extra latency is ok, and more so if the order does not have to be exact >> >> i think at very large scale hashing becomes the only answer, and handling failed nodes becomes tricky, but a basic distributed mnesia table that contains the number of alive nodes and maybe reports on their repeated failures, and then hashes the input (on userid in your case) to the correct node would get you half way there. >> >> Also i don't think that 100,000 processes that peak at 300kb is a problem for erlang. Thats only 30GB max. If its stored as binaries, and possibly after a really large message you invoke the garbage collector for that process ( i have no idea if this is a bad idea or not, but can be done easily and should reduce memory consumption) then a single box should be able to handle your needs easily, and 3 no probs at all. My mac pro should be ok :-) >> >> i think dedicated hardware is always better than virtualised >> >> As for 1000 messages at 300kb, that seems like quite a lot, bordering on what standard gigabit ethernet can handle, ?but just benchmark erlang, again one box might be enough... you can get a 32 core amd server for not much these days. Are these requirements realistic tho, or just "if 10 million people use my service on day one that i have put together on a shoe string etc... wishful thinking" >> >> >> On 1 Feb 2011, at 19:10, Evans, Matthew wrote: >> >> > Without knowing much more another model would be to use your own ets/mnesia table to map the workers. >> > >> > I am assuming the creation of a worker is relatively infrequent (compared to the times you need to find a worker). >> > >> > When you create a worker use gen_server:start(?MODULE, [], []). >> > >> > This will return {ok,Pid}. You can save the Pid in an mnesia table along with whatever reference you need (doesn't need to be an atom). All nodes in the mesh will get the pid and the reference. Starting it as shown above (with an arity of 3) ?means you don't need to use the global service and ensure the names are unique atoms for each server. You can even do start via some application that will spread the gen_servers over different nodes. >> > >> > When a request comes in it only need do a lookup to find the correct pid. >> > >> > I can't recall as to how bad the process crash messaging is. But what you could do is to monitor a gen_server locally, and monitor each node globally. When a node fails all other nodes can cleanup processes that were registered on that node. >> > >> > Matt >> > >> > ________________________________________ >> > From: erlang-questions@REDACTED [erlang-questions@REDACTED] On Behalf Of Knut Nesheim [knut.nesheim@REDACTED] >> > Sent: Tuesday, February 01, 2011 12:25 PM >> > To: erlang-questions@REDACTED >> > Subject: [erlang-questions] gen_server clustering strategy >> > >> > Hello list, >> > >> > We are interested in understanding how the Erlang distribution >> > mechanism behaves in a fully connected cluster. Our concern is that as >> > we add nodes, the overhead will become a problem. >> > >> > Our application is a bunch of gen_servers containing state, which we >> > need to run on multiple nodes due to memory usage. A request will come >> > to our application and it needs to be handled by the correct >> > gen_server. Every request includes a user id which we can use to map >> > to the process. >> > >> > We have two specific use cases in mind. >> > >> > 1) Proxy to workers >> > >> > In this scenario we have a proxy (or multiple proxies) accepting >> > requests at the edge of our cluster. The proxy will ask the correct >> > server for a response. Either through using the 'global' module or >> > gproc or something similar, the proxy node will keep track of the >> > mapping of user ids to process and nodes. The proxy will call the node >> > using the Erlang distribution protocol. >> > >> > 2) Mesh >> > >> > In this scenario a request can be handled by any node in the cluster. >> > If the request cannot be handled by the local node, the correct node >> > is called on instead. Every node in the cluster needs to keep track of >> > which id belongs to which process. >> > >> > >> > The numbers: >> > * We must be able to run 100,000 processes, each may peak at 300kb of memory >> > * We expect 5000 requests coming in to our application per second at >> > peak, generating the same number of messages >> > * Out of those 5000 messages, 1000 has a content that may peak at 300kb >> > * All the other messages are well under 1kb >> > >> > Our concerns: >> > >> > * In the mesh, will the traffic between the nodes become a problem? >> > Lets say we have 4 nodes, if the requests are evenly divided between >> > the nodes the probability of hitting the correct node is 25%. With 100 >> > nodes this is 1%. As we add nodes, there will be more chatter. May >> > this chatter "drown-out" the important messages, like pid down, etc. >> > >> > * Will it be more efficient to use the proxy to message every node? >> > Will the message then always go to the node directly, or may it go >> > through some other node in the cluster? >> > >> > * For the mesh, we need to keep the 'global' or gproc state >> > replicated. With 'global' we will run into the default limit of atoms. >> > If we were to increase this limit, what kind of memory usage could we >> > expect? Are there any other nasty side-effects that we should be aware >> > of? >> > >> > * In the "Erlang/OTP in Action" book it is stated that you may have a >> > "couple of dozen, but probably not hundreds of nodes." Our >> > understanding is that this is because of traffic like "this pid just >> > died", "this node is no longer available", etc. >> > >> > * If we were to eliminate messaging between nodes, how many nodes >> > could we actually run in a fully connected cluster? Will a >> > high-latency network like ec2 affect this number or just the latency >> > of messages? What else may affect the size? >> > >> > >> > Regards >> > Knut >> > >> > ________________________________________________________________ >> > erlang-questions (at) erlang.org mailing list. >> > See http://www.erlang.org/faq.html >> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > >> > >> > ________________________________________________________________ >> > erlang-questions (at) erlang.org mailing list. >> > See http://www.erlang.org/faq.html >> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > -- > ------------------------------------------------------------------------ > Anthony Molinaro ? ? ? ? ? ? ? ? ? ? ? ? ? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From yrashk@REDACTED Thu Feb 3 05:33:22 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Wed, 2 Feb 2011 20:33:22 -0800 Subject: gen_fsm:handle_state/2 Message-ID: Hi OTP team & everybody, I've added a handle_state/2 callback to gen_fsm (strictly speaking, to my fork called gen_fsm2) and was wondering if anybody else is interested in it. It's a fairly simple patch that will call handle_state(StateName, State) callback (if exported) on every gen_fsm's loop/X cycle so that your FSM can put some logic in state transitions and what not. I constantly run into situations when I need something like this, and I finally decided to go and implement it, luckily it was quite trivial. The diff is available https://github.com/yrashk/gen_fsm2/compare/bcc9b5e2adc0c0fdb5d5f83a121b94fd0fc40a86...master I am also interested to learn whether OTP would be interested to consider it or further modification of it as a patch for proper gen_fsm; if so, I will prepare a proper pull request on GitHub with tests and updated documentation. I'd be nice to have it merged to prevent gen_server2 situation. Questions, concerns, suggestions? Cheers, Yurii. From maruthavanan_s@REDACTED Thu Feb 3 10:56:14 2011 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Thu, 3 Feb 2011 04:56:14 -0500 Subject: lists:takewhile and lists:splitwith Message-ID: Hi, The document says lists:takewhile(Pred,List) would return tuple of lists which would split the List when Pred returns true. I tried the below. lists:takewhile(fun(A)-> A==3 end,[1,2,3,4,5]). it gives me an empty result.. What should I do to get {[1,2],[3,4,5]} Regards, Marutha From torben.lehoff@REDACTED Thu Feb 3 11:01:14 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Thu, 3 Feb 2011 11:01:14 +0100 Subject: [erlang-questions] lists:takewhile and lists:splitwith In-Reply-To: References: Message-ID: use fun(A) -> A <3 end instead. On Thu, Feb 3, 2011 at 10:56, maruthavanan s wrote: > > Hi, > > The document says lists:takewhile(Pred,List) would return tuple of lists > which would split the List when Pred returns true. > > I tried the below. > > lists:takewhile(fun(A)-> A==3 end,[1,2,3,4,5]). > > it gives me an empty result.. What should I do to get {[1,2],[3,4,5]} > > Regards, > Marutha > -- http://www.linkedin.com/in/torbenhoffmann From masklinn@REDACTED Thu Feb 3 11:02:25 2011 From: masklinn@REDACTED (Masklinn) Date: Thu, 3 Feb 2011 11:02:25 +0100 Subject: [erlang-questions] lists:takewhile and lists:splitwith In-Reply-To: References: Message-ID: On 2011-02-03, at 10:56 , maruthavanan s wrote: > Hi, > > The document says lists:takewhile(Pred,List) would return tuple of lists which would split the List when Pred returns true. > > I tried the below. > > lists:takewhile(fun(A)-> A==3 end,[1,2,3,4,5]). > > it gives me an empty result.. What should I do to get {[1,2],[3,4,5]} I don't know which documentation you read, mine says the following: takewhile(Pred, List1) -> List2 Takes elements Elem from List1 while Pred(Elem) returns true, that is, the function returns the longest prefix of the list for which all elements satisfy the predicate. Here, the first element of `List1` is `1`, for which `Pred` is `false`, so `takewhile` returns an empty list. Sounds sensible and works as I would expect. What you seem to describe is lists:splitwith[0], and note that in any case your predicate is incorrect. [0] http://www.erlang.org/doc/man/lists.html#splitwith-2 From masklinn@REDACTED Thu Feb 3 11:02:51 2011 From: masklinn@REDACTED (Masklinn) Date: Thu, 3 Feb 2011 11:02:51 +0100 Subject: [erlang-questions] lists:takewhile and lists:splitwith In-Reply-To: References: Message-ID: On 2011-02-03, at 11:01 , Torben Hoffmann wrote: > use > fun(A) -> A <3 end > instead. This will return [1, 2] not the pair he expects. From bengt.kleberg@REDACTED Thu Feb 3 11:04:35 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 3 Feb 2011 11:04:35 +0100 Subject: [erlang-questions] lists:takewhile and lists:splitwith In-Reply-To: References: Message-ID: <1296727475.5443.5.camel@seasc0694> Greetings, Are you sure about this? I think that takewhile/2 returns a single list. The function splitwith/2 returns a tuple. ebngt On Thu, 2011-02-03 at 10:56 +0100, maruthavanan s wrote: > Hi, > > The document says lists:takewhile(Pred,List) would return tuple of lists which would split the List when Pred returns true. > > I tried the below. > > lists:takewhile(fun(A)-> A==3 end,[1,2,3,4,5]). > > it gives me an empty result.. What should I do to get {[1,2],[3,4,5]} > > Regards, > Marutha > From norton@REDACTED Thu Feb 3 11:06:30 2011 From: norton@REDACTED (Joseph Wayne Norton) Date: Thu, 03 Feb 2011 19:06:30 +0900 Subject: [erlang-questions] lists:takewhile and lists:splitwith In-Reply-To: References: Message-ID: Try the following ... 5> lists:partition(fun(A)-> A<3 end,[1,2,3,4,5]). {[1,2],[3,4,5]} On Thu, 03 Feb 2011 18:56:14 +0900, maruthavanan s wrote: > > Hi, > > The document says lists:takewhile(Pred,List) would return tuple of lists > which would split the List when Pred returns true. > > I tried the below. > > lists:takewhile(fun(A)-> A==3 end,[1,2,3,4,5]). > > it gives me an empty result.. What should I do to get {[1,2],[3,4,5]} > > Regards, > Marutha > -- norton@REDACTED From maruthavanan_s@REDACTED Thu Feb 3 11:09:46 2011 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Thu, 3 Feb 2011 05:09:46 -0500 Subject: [erlang-questions] lists:takewhile and lists:splitwith In-Reply-To: References: , Message-ID: Hi, Sorry for the confusion. It was my mistake, I was reading the list document and got confused with "splitwith" /"takewhil"/ "dropwhile". Got it. Thanks, Marutha > From: masklinn@REDACTED > Date: Thu, 3 Feb 2011 11:02:25 +0100 > CC: erlang-questions@REDACTED > To: maruthavanan_s@REDACTED > Subject: Re: [erlang-questions] lists:takewhile and lists:splitwith > > On 2011-02-03, at 10:56 , maruthavanan s wrote: > > Hi, > > > > The document says lists:takewhile(Pred,List) would return tuple of lists which would split the List when Pred returns true. > > > > I tried the below. > > > > lists:takewhile(fun(A)-> A==3 end,[1,2,3,4,5]). > > > > it gives me an empty result.. What should I do to get {[1,2],[3,4,5]} > I don't know which documentation you read, mine says the following: > > takewhile(Pred, List1) -> List2 > Takes elements Elem from List1 while Pred(Elem) returns true, that is, the function returns the longest prefix of the list for which all elements satisfy the predicate. > > Here, the first element of `List1` is `1`, for which `Pred` is `false`, so `takewhile` returns an empty list. Sounds sensible and works as I would expect. > > What you seem to describe is lists:splitwith[0], and note that in any case your predicate is incorrect. > > [0] http://www.erlang.org/doc/man/lists.html#splitwith-2 > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From maruthavanan_s@REDACTED Thu Feb 3 11:10:54 2011 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Thu, 3 Feb 2011 05:10:54 -0500 Subject: [erlang-questions] lists:takewhile and lists:splitwith In-Reply-To: <1296727475.5443.5.camel@seasc0694> References: ,<1296727475.5443.5.camel@seasc0694> Message-ID: Sorry to all for the confusions. > To: > From: bengt.kleberg@REDACTED > CC: erlang-questions@REDACTED > Date: Thu, 3 Feb 2011 11:04:35 +0100 > Subject: Re: [erlang-questions] lists:takewhile and lists:splitwith > > Greetings, > > Are you sure about this? I think that takewhile/2 returns a single list. > > The function splitwith/2 returns a tuple. > > > ebngt > > On Thu, 2011-02-03 at 10:56 +0100, maruthavanan s wrote: > > Hi, > > > > The document says lists:takewhile(Pred,List) would return tuple of lists which would split the List when Pred returns true. > > > > I tried the below. > > > > lists:takewhile(fun(A)-> A==3 end,[1,2,3,4,5]). > > > > it gives me an empty result.. What should I do to get {[1,2],[3,4,5]} > > > > Regards, > > Marutha > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From matthias@REDACTED Thu Feb 3 11:43:25 2011 From: matthias@REDACTED (Matthias Lang) Date: Thu, 3 Feb 2011 11:43:25 +0100 Subject: [erlang-questions] Erlang crossplatform In-Reply-To: <1296494566167-3249229.post@n4.nabble.com> References: <1296494566167-3249229.post@n4.nabble.com> Message-ID: <20110203104325.GA4279@corelatus.se> On Monday, January 31, shk wrote: > In future i'll plan to port my erlang application in MS Windows. And i want > to ask question: Erlang is corssplatform in full? Is anything in Erlang > which not supporting Windows? I think basically all of Erlang works on Windows. Scraping for something non-trivial which might not work, I came up with SCTP: http://www.erlang.org/doc/man/gen_sctp.html the manpage mentions the environments its been tested on, and Windows isn't one of them. But I haven't actually tried, because I don't use Windows. A softer problem is that windows-specific questions to the mailing list often go without an answer for a long time. Maybe there aren't many windows users who are able or inclined. Matt From masklinn@REDACTED Thu Feb 3 11:50:50 2011 From: masklinn@REDACTED (Masklinn) Date: Thu, 3 Feb 2011 11:50:50 +0100 Subject: [erlang-questions] lists:takewhile and lists:splitwith In-Reply-To: References: Message-ID: On 2011-02-03, at 11:06 , Joseph Wayne Norton wrote: > Try the following ... > > 5> lists:partition(fun(A)-> A<3 end,[1,2,3,4,5]). > {[1,2],[3,4,5]} > But remember that lists:partition's semantics are based on those of lists:filter, not lists:takewhile and lists:dropwhile From jesper.louis.andersen@REDACTED Thu Feb 3 12:20:06 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 3 Feb 2011 12:20:06 +0100 Subject: [erlang-questions] Erlang crossplatform In-Reply-To: <20110203104325.GA4279@corelatus.se> References: <1296494566167-3249229.post@n4.nabble.com> <20110203104325.GA4279@corelatus.se> Message-ID: On Thu, Feb 3, 2011 at 11:43, Matthias Lang wrote: > I think basically all of Erlang works on Windows. Scraping for > something non-trivial which might not work, I came up with SCTP: > > ?http://www.erlang.org/doc/man/gen_sctp.html > > the manpage mentions the environments its been tested on, and Windows > isn't one of them. But I haven't actually tried, because I don't use > Windows. Windows has notoriously bad SCTP support. This is unfortunate as it hurts the adoption of a pretty nice protocol. BitTorrent would have been happy with that protocol for instance. -- J. From thatpythonguy@REDACTED Thu Feb 3 20:10:59 2011 From: thatpythonguy@REDACTED (Ahmed Al-Saadi) Date: Thu, 03 Feb 2011 14:10:59 -0500 Subject: [erlang-questions] Command-line Debugger? In-Reply-To: References: <4D481892.3090708@gmail.com> <4D48E974.1020109@gmail.com> Message-ID: <4D4AFDC3.8030804@gmail.com> James: I hesitantly tried erlide before posting to this mailing list, but I generally don't like IDEs and prefer command-line tools and editors (I use MacVim myself). One would have to learn a bunch of new non-core concepts (e.g., Run Configurations) just to be able to run a project when I only want to set a brake point to trace the code! I suppose erlide would be a good choice for someone who already uses Eclipse. Cheers, Ahmed On 2/2/11 10:01 AM, James Churchman wrote: > try erlide, a plugin for eclipse, http://erlide.org/ :-) > > On 2 Feb 2011, at 05:19, Ahmed Al-Saadi wrote: > >> Dan: >> >> It seems that the Homebrew Erlang installation on my Mac did not >> include the wx package! I downloaded the latest tarball from >> erlang.org and, so far, the previous problems >> with the debugger haven't manifested themselves. >> >> I do have a first-generation macbook which has a 32-bit processor and >> I did install wxWidgets myself, so that must have avoided the >> pitfalls you mentioned. >> >> Thanks for your input... >> >> Cheers, >> Ahmed >> >> >> On 2/1/11 10:45 AM, Dan Gudmundsson wrote: >>> Just for the information, the latest wx is 0.98.8 and is included >>> with erlang. >>> The old sourceforge site is dead... except some occasional support mail. >>> >>> But on MacOsX it only works for 32bits erlang currently, and also >>> requires the user to install >>> and build wxWidgets, the one included in snow leopard is buggy. >>> >>> /Dan >>> >>> On Tue, Feb 1, 2011 at 3:39 PM, Joseph Wayne Norton >>> > wrote: >>>> Ahmed - >>>> >>>> It may sound like a crazy idea but I'd recommend not using the >>>> debugger. >>>> >>>> Erlang's trace facilities are really great for troubleshooting code >>>> during development and production. >>>> >>>> The redbug tool is any easy way to get started with Erlang's tracer >>>> (https://github.com/massemanet/eper). >>>> >>>> >>>> On 2011/02/01, at 23:28, Ahmed Al-Saadi wrote: >>>> >>>>> Sorry if this was double-submitted, but I think that it didn't go >>>>> to the original group on the first attempt: >>>>> >>>>> >>>>> Hello: >>>>> >>>>> I'm a newcomer to Erlang (from Python). I've had great >>>>> difficulties in using the wx-based debugger and was wondering if >>>>> there is a command-line alternative a la python's pdb?! >>>>> >>>>> It seems that every time I use the default debugger, my Wish app >>>>> hangs or crashes. Even with the latest release of wxErlang >>>>> (0.97.1212), I was not even able to set a breakpoint! >>>>> >>>>> The use of a good debugger to trace program execution is >>>>> invaluable in learning a new language--a pedagogical role, in >>>>> addition to its normal troubleshooting role. >>>>> >>>>> Any ideas of how I can improve the situation of my development >>>>> environment? Is there another command-line debugger? My platform >>>>> is MacOS X 10.6.6 with Erlang R14B... and, no, changing platforms >>>>> is not an option ;) Also, if I knew how to write a debugger at >>>>> this point in time, I would try to do so! Some guidance from a >>>>> veteran could help. >>>>> >>>>> TPG/TEG >>>> Joseph Wayne Norton >>>> norton@REDACTED >>>> >>>> >>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > From ok@REDACTED Thu Feb 3 22:29:56 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 4 Feb 2011 10:29:56 +1300 Subject: [erlang-questions] lists:takewhile and lists:splitwith In-Reply-To: References: Message-ID: <6F4594F3-47C2-42C7-BB6C-9142D8DF7050@cs.otago.ac.nz> On 3/02/2011, at 10:56 PM, maruthavanan s wrote: > > Hi, > > The document says lists:takewhile(Pred,List) would return tuple of lists which would split the List when Pred returns true. No it doesn't. This is what the documentation says: takewhile(Pred, List1) -> List2 Types: Pred = fun(Elem) -> bool() Elem = term() List1 = List2 = [term()] Takes elements Elem from List1 while Pred(Elem) returns true, that is, the function returns the longest prefix of the list for which all elements satisfy the predicate. > > I tried the below. > > lists:takewhile(fun(A)-> A==3 end,[1,2,3,4,5]). > > it gives me an empty result.. And indeed the longest prefix of [1,2,3,4,5] such that all elements are the number 3 really and truly is []. > What should I do to get {[1,2],[3,4,5]} Write your own function. What you want is Pred, [X1,...,Xk] => {[Y1,...,Ym],[Z1,...,Zn]} such that [Y1,...,Ym]++[Z1,...,Zn] is [X1,...,Xk] and all(Pred, [Y1,...,Ym] and n = 0 or not Pred(Z1). split_at_first(Pred, List) -> split_at_first_loop(Pred, List, []). split_at_first_loop(Pred, [], Reversed_Prefix) -> {lists:reverse(Reversed_Prefix), []}; split_at_first_loop(Pred, [X|Xs], Reversed_Prefix) -> case Pred(X) of true -> {lists:reverse(Reversed_Prefix), [X|Xs]} ; false -> split_at_first_loop(Pred, Xs, [X|Reversed_Prefix]) end. You should probably ask yourself whether [Ym,...,Y1] will do just as well as [Y1,...,Ym]. From ok@REDACTED Thu Feb 3 22:34:59 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 4 Feb 2011 10:34:59 +1300 Subject: [erlang-questions] lists:takewhile and lists:splitwith In-Reply-To: References: Message-ID: On 3/02/2011, at 11:06 PM, Joseph Wayne Norton wrote: > > Try the following ... > > 5> lists:partition(fun(A)-> A<3 end,[1,2,3,4,5]). > {[1,2],[3,4,5]} For this example, all of lists:partition(fun (A) -> A < 3 end, [1,2,3,4,5]) lists:splitwith(fun (A) -> A < 3 end, [1,2,3,4,5]) lists:splitwith(fun (A) -> A/= 3 end, [1,2,3,4,5]) give the same answer. It is only when the list is sorted that this is so. None of them uses the fun that the OP wished to use. From dmitriy.kargapolov@REDACTED Fri Feb 4 03:46:50 2011 From: dmitriy.kargapolov@REDACTED (Dmitriy Kargapolov) Date: Thu, 3 Feb 2011 21:46:50 -0500 Subject: process trace triggered by sequential trace token In-Reply-To: References: Message-ID: On Wed, Feb 2, 2011 at 10:04 AM, Dmitriy Kargapolov < dmitriy.kargapolov@REDACTED> wrote: > Hi All, > I'd like to trace gen_server body execution, not for all calls, but for > particular one. > Is there a way to enable trace for the process by received (and matched) > message "contaminated" by trace token? (And disable it at the end of message > processing). I know there is a way to turn sequential trace by matching > regular function call, but I need things to go in reverse order... > Thanks. > Just want to let everybody know that this can be achieved by using is_seq_trace as a guard in match specification. It could be used to allow function trace only when trace token is set, or, to trigger silent mode to allow/disable further trace messages. See http://www.erlang.org/doc/apps/erts/match_spec.html for details. From kaiduanx@REDACTED Fri Feb 4 04:03:18 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Thu, 3 Feb 2011 22:03:18 -0500 Subject: Can't call ssl:transport_accept/ssl:accept from multiple processes Message-ID: Hi, all, I am having trouble to accept SSL connection from multiple processes, two processes are spawned on server side with the following code, tls_acceptor(LSock) -> {ok, Socket} = ssl:transport_accept(LSock), ok = ssl:ssl_accept(Socket), {ok, {PeerAddr, PeerPort}} = ssl:peername(Socket), PA = erlsip_util:tuple_ip4_to_str_ip4(PeerAddr), ?DEBUG("Accept from: ~p:~p", [self(), PA]), tls_acceptor(LSock). And the log shows that server alternatively accepts from two processes and the it stops accepting more, =INFO REPORT==== 03-02-2011::21:35:49:378 === DEBUG|<0.119.0>|erlsip_transport_tls|117| Accept from: <0.119.0>:"192.168.1.3" =INFO REPORT==== 03-02-2011::21:35:49:418 === DEBUG|<0.120.0>|erlsip_transport_tls|117| Accept from: <0.120.0>:"192.168.1.3" =INFO REPORT==== 03-02-2011::21:35:49:418 === DEBUG|<0.119.0>|erlsip_transport_tls|117| Accept from: <0.119.0>:"192.168.1.3" =INFO REPORT==== 03-02-2011::21:35:49:419 === DEBUG|<0.120.0>|erlsip_transport_tls|117| Accept from: <0.120.0>:"192.168.1.3" =INFO REPORT==== 03-02-2011::21:35:49:457 === DEBUG|<0.119.0>|erlsip_transport_tls|117| Accept from: <0.119.0>:"192.168.1.3" The wireshark trace attached shows that server does not send out "Server Hello" for the sixth connection. If one process is spawned instead of one, the problem is gone. I would like to scale up the SSL connection establishment speed on multi-core box. For sake of completeness, I have tried on R14B and R14B01 with same result. Thanks for help, /Kaiduan -------------- next part -------------- A non-text attachment was scrubbed... Name: ssl-accept.pcap Type: application/cap Size: 24243 bytes Desc: not available URL: From fritchie@REDACTED Fri Feb 4 05:09:27 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 03 Feb 2011 22:09:27 -0600 Subject: OTP app for sharing system_monitor messages: riak_sysmon Message-ID: <82602.1296792567@snookles.snookles.com> Hi, all. Basho's got a new OTP app available for review. We hope y'all find it useful. Suggestions, feedback, bugfixes (especially in the form of GitHub pull requests), rotten tomatoes, etc. are welcome. https://github.com/basho/riak_sysmon The Erlang VM only allows a single process to be a "system monitor" process and receive system monitor events. This app allows many OTP apps to share system monitor events while also protecting the system from being overrun by too many events. See the README for more details. -Scott From ingela@REDACTED Fri Feb 4 09:37:35 2011 From: ingela@REDACTED (Ingela Andin) Date: Fri, 4 Feb 2011 09:37:35 +0100 Subject: [erlang-questions] Can't call ssl:transport_accept/ssl:accept from multiple processes In-Reply-To: References: Message-ID: Hi ! It sounds like it might be a socket ownership problem. The best would be to have the socket in active false mode and then transfering socket ownership (ssl:controlling_process/2) to the new process before it gets to do ssl_accept. Regards Ingela Erlang/OTP team - Ericsson AB 2011/2/4 Kaiduan Xie : > Hi, all, > > I am having trouble to accept SSL connection from multiple processes, > two processes are spawned on server side with the following code, > > tls_acceptor(LSock) -> > ? ?{ok, Socket} = ssl:transport_accept(LSock), > ? ?ok = ssl:ssl_accept(Socket), > ? ?{ok, {PeerAddr, PeerPort}} = ssl:peername(Socket), > ? ?PA = erlsip_util:tuple_ip4_to_str_ip4(PeerAddr), > ? ??DEBUG("Accept from: ~p:~p", [self(), PA]), > ? ?tls_acceptor(LSock). > > And the log shows that server alternatively accepts from two processes > and the it stops accepting more, > > =INFO REPORT==== 03-02-2011::21:35:49:378 === > DEBUG|<0.119.0>|erlsip_transport_tls|117| Accept from: <0.119.0>:"192.168.1.3" > > =INFO REPORT==== 03-02-2011::21:35:49:418 === > DEBUG|<0.120.0>|erlsip_transport_tls|117| Accept from: <0.120.0>:"192.168.1.3" > > =INFO REPORT==== 03-02-2011::21:35:49:418 === > DEBUG|<0.119.0>|erlsip_transport_tls|117| Accept from: <0.119.0>:"192.168.1.3" > > =INFO REPORT==== 03-02-2011::21:35:49:419 === > DEBUG|<0.120.0>|erlsip_transport_tls|117| Accept from: <0.120.0>:"192.168.1.3" > > =INFO REPORT==== 03-02-2011::21:35:49:457 === > DEBUG|<0.119.0>|erlsip_transport_tls|117| Accept from: <0.119.0>:"192.168.1.3" > > The wireshark trace attached shows that server does not send out > "Server Hello" for the sixth connection. > > If one process is spawned instead of one, the problem is gone. > > I would like to scale up the SSL connection establishment speed on > multi-core box. For sake of completeness, I have tried on R14B and > R14B01 with same result. > > Thanks for help, > > /Kaiduan > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ferenc.holzhauser@REDACTED Fri Feb 4 09:57:11 2011 From: ferenc.holzhauser@REDACTED (Ferenc Holzhauser) Date: Fri, 4 Feb 2011 09:57:11 +0100 Subject: ssl_connection process hibernation option Message-ID: Hi, I'm using R14B01 new SSL to set up a lot of (not very active) client connections. I have noticed that ssl_connection processes use a lot of memory between GC's. A forced GC makes them small again until the next piece of received data. This adds up to a significant memory usage (around 350-400 kB/session on a x64 system in my case) limiting the number of possible sessions. An option to hibernate this gen_fsm would decrease the average memory usage in this scenario. Would it be possible to introduce this option? Thanks in advance, Ferenc From adam@REDACTED Fri Feb 4 11:44:13 2011 From: adam@REDACTED (Adam Lindberg) Date: Fri, 04 Feb 2011 11:44:13 +0100 Subject: Slave node restart not working Message-ID: <4D4BD87D.7050409@erlang-solutions.com> Hi! I'm running a slave node in a project, and sometimes we want to restart the slave node. However, it seems that init:restart() shuts down the node instead of restarting in. The documentation for init:restart() says that it restarts the node in the current emulator, which makes me wonder why it goes down instead of restarts. The node should have everything it needs to restart based on the boot flags etc. (hest@REDACTED)5> slave:start(yama, apa). {ok,apa@REDACTED} (hest@REDACTED)6> nodes(). [apa@REDACTED] (hest@REDACTED)7> rpc:call('apa@REDACTED', init, restart, []). ok (hest@REDACTED)8> nodes(). [] There is no crash dump generated as well. Why doesn't the slave node restart? Cheers, Adam From andrey.pampukha@REDACTED Fri Feb 4 11:48:12 2011 From: andrey.pampukha@REDACTED (Andrey Pampukha) Date: Fri, 4 Feb 2011 13:48:12 +0300 Subject: [erlang-questions] Slave node restart not working In-Reply-To: <4D4BD87D.7050409@erlang-solutions.com> References: <4D4BD87D.7050409@erlang-solutions.com> Message-ID: Hi, Looks like nobody reconnects the slave node back to the master after restart. Try to ping it, I believe it should work. Andrey. 2011/2/4, Adam Lindberg : > Hi! > > I'm running a slave node in a project, and sometimes we want to restart > the slave node. However, it seems that init:restart() shuts down the > node instead of restarting in. > > The documentation for init:restart() says that it restarts the node in > the current emulator, which makes me wonder why it goes down instead of > restarts. The node should have everything it needs to restart based on > the boot flags etc. > > > (hest@REDACTED)5> slave:start(yama, apa). > {ok,apa@REDACTED} > (hest@REDACTED)6> nodes(). > [apa@REDACTED] > (hest@REDACTED)7> rpc:call('apa@REDACTED', init, restart, []). > ok > (hest@REDACTED)8> nodes(). > [] > > There is no crash dump generated as well. Why doesn't the slave node > restart? > > Cheers, > Adam > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From andrey.pampukha@REDACTED Fri Feb 4 11:52:09 2011 From: andrey.pampukha@REDACTED (Andrey Pampukha) Date: Fri, 4 Feb 2011 13:52:09 +0300 Subject: [erlang-questions] Slave node restart not working In-Reply-To: References: <4D4BD87D.7050409@erlang-solutions.com> Message-ID: I've checked, slave doesn't want to work at my machine because of absence of ssh-server, but ct_slave works: ma@REDACTED)1> ct_slave:start(te). {ok,te@REDACTED} (ma@REDACTED)2> nodes(). [te@REDACTED] (ma@REDACTED)3> rpc:call(te@REDACTED,init,restart,[]). ok (ma@REDACTED)4> nodes(). [] (ma@REDACTED)5> net_adm:ping(te@REDACTED). pong (ma@REDACTED)6> nodes(). [te@REDACTED] 2011/2/4, Andrey Pampukha : > Hi, > > Looks like nobody reconnects the slave node back to the master after > restart. > Try to ping it, I believe it should work. > > Andrey. > > 2011/2/4, Adam Lindberg : >> Hi! >> >> I'm running a slave node in a project, and sometimes we want to restart >> the slave node. However, it seems that init:restart() shuts down the >> node instead of restarting in. >> >> The documentation for init:restart() says that it restarts the node in >> the current emulator, which makes me wonder why it goes down instead of >> restarts. The node should have everything it needs to restart based on >> the boot flags etc. >> >> >> (hest@REDACTED)5> slave:start(yama, apa). >> {ok,apa@REDACTED} >> (hest@REDACTED)6> nodes(). >> [apa@REDACTED] >> (hest@REDACTED)7> rpc:call('apa@REDACTED', init, restart, []). >> ok >> (hest@REDACTED)8> nodes(). >> [] >> >> There is no crash dump generated as well. Why doesn't the slave node >> restart? >> >> Cheers, >> Adam >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From adam@REDACTED Fri Feb 4 14:20:54 2011 From: adam@REDACTED (Adam Lindberg) Date: Fri, 04 Feb 2011 14:20:54 +0100 Subject: [erlang-questions] Slave node restart not working In-Reply-To: References: <4D4BD87D.7050409@erlang-solutions.com> Message-ID: <4D4BFD36.4040707@erlang-solutions.com> I can see that the new beam process for the slave node actually dissappears, and it and it is not possible to ping it. I have verified this behavior on Windows as well. ? ps aux | grep beam | wc -l 1 (hest@REDACTED)24> net_adm:ping(apa@REDACTED). pang (hest@REDACTED)25> net_adm:ping(apa@REDACTED). pang (hest@REDACTED)26> slave:start(yama, apa). {ok,apa@REDACTED} (hest@REDACTED)27> net_adm:ping(apa@REDACTED). pong ? ps aux | grep beam | wc -l 2 (hest@REDACTED)28> rpc:call(apa@REDACTED, init, restart, []). ok (hest@REDACTED)29> net_adm:ping(apa@REDACTED). pang (hest@REDACTED)30> nodes(). [] ? ps aux | grep beam | wc -l 1 Cheers, Adam Andrey Pampukha wrote: > I've checked, slave doesn't want to work at my machine because of > absence of ssh-server, but ct_slave works: > ma@REDACTED)1> ct_slave:start(te). > {ok,te@REDACTED} > (ma@REDACTED)2> nodes(). > [te@REDACTED] > (ma@REDACTED)3> rpc:call(te@REDACTED,init,restart,[]). > ok > (ma@REDACTED)4> nodes(). > [] > (ma@REDACTED)5> net_adm:ping(te@REDACTED). > pong > (ma@REDACTED)6> nodes(). > [te@REDACTED] > > > 2011/2/4, Andrey Pampukha: >> Hi, >> >> Looks like nobody reconnects the slave node back to the master after >> restart. >> Try to ping it, I believe it should work. >> >> Andrey. >> >> 2011/2/4, Adam Lindberg: >>> Hi! >>> >>> I'm running a slave node in a project, and sometimes we want to restart >>> the slave node. However, it seems that init:restart() shuts down the >>> node instead of restarting in. >>> >>> The documentation for init:restart() says that it restarts the node in >>> the current emulator, which makes me wonder why it goes down instead of >>> restarts. The node should have everything it needs to restart based on >>> the boot flags etc. >>> >>> >>> (hest@REDACTED)5> slave:start(yama, apa). >>> {ok,apa@REDACTED} >>> (hest@REDACTED)6> nodes(). >>> [apa@REDACTED] >>> (hest@REDACTED)7> rpc:call('apa@REDACTED', init, restart, []). >>> ok >>> (hest@REDACTED)8> nodes(). >>> [] >>> >>> There is no crash dump generated as well. Why doesn't the slave node >>> restart? >>> >>> Cheers, >>> Adam >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> From adam@REDACTED Fri Feb 4 14:27:47 2011 From: adam@REDACTED (Adam Lindberg) Date: Fri, 04 Feb 2011 14:27:47 +0100 Subject: [erlang-questions] Slave node restart not working In-Reply-To: <4D4BFD36.4040707@erlang-solutions.com> References: <4D4BD87D.7050409@erlang-solutions.com> <4D4BFD36.4040707@erlang-solutions.com> Message-ID: <4D4BFED3.3020001@erlang-solutions.com> Yes, ct_slave from R14B01 is working. It can handle an init:restart(). However, ct_slave is not in R13B04 which we're using. Cheers, Adam Adam Lindberg wrote: > I can see that the new beam process for the slave node actually > dissappears, and it and it is not possible to ping it. > > I have verified this behavior on Windows as well. > > > ? ps aux | grep beam | wc -l > 1 > > (hest@REDACTED)24> net_adm:ping(apa@REDACTED). > pang > (hest@REDACTED)25> net_adm:ping(apa@REDACTED). > pang > (hest@REDACTED)26> slave:start(yama, apa). > {ok,apa@REDACTED} > (hest@REDACTED)27> net_adm:ping(apa@REDACTED). > pong > > ? ps aux | grep beam | wc -l > 2 > > (hest@REDACTED)28> rpc:call(apa@REDACTED, init, restart, []). > ok > (hest@REDACTED)29> net_adm:ping(apa@REDACTED). > pang > (hest@REDACTED)30> nodes(). > [] > > ? ps aux | grep beam | wc -l > 1 > > > Cheers, > Adam > > > > Andrey Pampukha wrote: >> I've checked, slave doesn't want to work at my machine because of >> absence of ssh-server, but ct_slave works: >> ma@REDACTED)1> ct_slave:start(te). >> {ok,te@REDACTED} >> (ma@REDACTED)2> nodes(). >> [te@REDACTED] >> (ma@REDACTED)3> rpc:call(te@REDACTED,init,restart,[]). >> ok >> (ma@REDACTED)4> nodes(). >> [] >> (ma@REDACTED)5> net_adm:ping(te@REDACTED). >> pong >> (ma@REDACTED)6> nodes(). >> [te@REDACTED] >> >> >> 2011/2/4, Andrey Pampukha: >>> Hi, >>> >>> Looks like nobody reconnects the slave node back to the master after >>> restart. >>> Try to ping it, I believe it should work. >>> >>> Andrey. >>> >>> 2011/2/4, Adam Lindberg: >>>> Hi! >>>> >>>> I'm running a slave node in a project, and sometimes we want to restart >>>> the slave node. However, it seems that init:restart() shuts down the >>>> node instead of restarting in. >>>> >>>> The documentation for init:restart() says that it restarts the node in >>>> the current emulator, which makes me wonder why it goes down instead of >>>> restarts. The node should have everything it needs to restart based on >>>> the boot flags etc. >>>> >>>> >>>> (hest@REDACTED)5> slave:start(yama, apa). >>>> {ok,apa@REDACTED} >>>> (hest@REDACTED)6> nodes(). >>>> [apa@REDACTED] >>>> (hest@REDACTED)7> rpc:call('apa@REDACTED', init, restart, []). >>>> ok >>>> (hest@REDACTED)8> nodes(). >>>> [] >>>> >>>> There is no crash dump generated as well. Why doesn't the slave node >>>> restart? >>>> >>>> Cheers, >>>> Adam >>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> From koops.j@REDACTED Fri Feb 4 15:04:34 2011 From: koops.j@REDACTED (Jeroen Koops) Date: Fri, 4 Feb 2011 15:04:34 +0100 Subject: [erlang-questions] ssl_connection process hibernation option In-Reply-To: References: Message-ID: I would very much appreciate this too. As a test, I inserted a proxy-module between the gen_fsm and ssl_connection callback, that changes all responses from the ssl_connection callback to include the 'hibernate' option (it's not a hack -- it's AOP!) and the savings are dramatic: before the change an SSL connection used around 350 kB, after the change only around 600 bytes. If a system is maintaining thousands of SSL connections, this really adds up. If we could somehow get hibernation built in to the ssl application, that would be great. On Fri, Feb 4, 2011 at 9:57 AM, Ferenc Holzhauser < ferenc.holzhauser@REDACTED> wrote: > Hi, > > I'm using R14B01 new SSL to set up a lot of (not very active) client > connections. > > I have noticed that ssl_connection processes use a lot of memory between > GC's. > A forced GC makes them small again until the next piece of received data. > > This adds up to a significant memory usage (around 350-400 kB/session on a > x64 system in my case) limiting the number of possible sessions. > > An option to hibernate this gen_fsm would decrease the average memory usage > in this scenario. > > Would it be possible to introduce this option? > > Thanks in advance, > Ferenc > From ingela.andin@REDACTED Fri Feb 4 16:51:42 2011 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 4 Feb 2011 16:51:42 +0100 Subject: [erlang-questions] ssl_connection process hibernation option In-Reply-To: References: Message-ID: Hi! We already put some effort into cleaning the state of the gen_fsm processes holding the connection, but some form of hibernation could be interesting too we will take this into consideration. Regards Ingela Erlang/OTP team - Ericsson AB 2011/2/4 Jeroen Koops : > I would very much appreciate this too. > > As a test, I inserted a proxy-module between the gen_fsm and ssl_connection > callback, that changes all responses from the ssl_connection callback to > include the 'hibernate' option (it's not a hack -- it's AOP!) and the > savings are dramatic: before the change an SSL connection used around 350 > kB, after the change only around 600 bytes. If a system is maintaining > thousands of SSL connections, this really adds up. > > If we could somehow get hibernation built in to the ssl application, that > would be great. > > On Fri, Feb 4, 2011 at 9:57 AM, Ferenc Holzhauser < > ferenc.holzhauser@REDACTED> wrote: > >> Hi, >> >> I'm using R14B01 new SSL to set up a lot of (not very active) client >> connections. >> >> I have noticed that ssl_connection processes use a lot of memory between >> GC's. >> A forced GC makes them small again until the next piece of received data. >> >> This adds up to a significant memory usage (around 350-400 kB/session on a >> x64 system in my case) limiting the number of possible sessions. >> >> An option to hibernate this gen_fsm would decrease the average memory usage >> in this scenario. >> >> Would it be possible to introduce this option? >> >> Thanks in advance, >> Ferenc >> > From kaiduanx@REDACTED Fri Feb 4 16:57:03 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Fri, 4 Feb 2011 10:57:03 -0500 Subject: [erlang-questions] ssl_connection process hibernation option In-Reply-To: References: Message-ID: Ingela, This is a great news, Ingela. When can we have this improvement? Best regards, /Kaiduan On Fri, Feb 4, 2011 at 10:51 AM, Ingela Andin wrote: > Hi! > > We already put some effort into cleaning the state of the gen_fsm > processes holding the connection, > but some form of hibernation could be interesting too we will take > this into consideration. > > Regards Ingela Erlang/OTP team - Ericsson AB > > 2011/2/4 Jeroen Koops : >> I would very much appreciate this too. >> >> As a test, I inserted a proxy-module between the gen_fsm and ssl_connection >> callback, that changes all responses from the ssl_connection callback to >> include the 'hibernate' option (it's not a hack -- it's AOP!) and the >> savings are dramatic: before the change an SSL connection used around 350 >> kB, after the change only around 600 bytes. If a system is maintaining >> thousands of SSL connections, this really adds up. >> >> If we could somehow get hibernation built in to the ssl application, that >> would be great. >> >> On Fri, Feb 4, 2011 at 9:57 AM, Ferenc Holzhauser < >> ferenc.holzhauser@REDACTED> wrote: >> >>> Hi, >>> >>> I'm using R14B01 new SSL to set up a lot of (not very active) client >>> connections. >>> >>> I have noticed that ssl_connection processes use a lot of memory between >>> GC's. >>> A forced GC makes them small again until the next piece of received data. >>> >>> This adds up to a significant memory usage (around 350-400 kB/session on a >>> x64 system in my case) limiting the number of possible sessions. >>> >>> An option to hibernate this gen_fsm would decrease the average memory usage >>> in this scenario. >>> >>> Would it be possible to introduce this option? >>> >>> Thanks in advance, >>> Ferenc >>> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From fred.hebert@REDACTED Fri Feb 4 18:29:32 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Fri, 4 Feb 2011 12:29:32 -0500 Subject: OTP Application Controller Vs. Supervisors with simple_one_for_one workers Message-ID: <7AD4C90A-0CF3-47E3-A4B9-DA921FEFC9E1@erlang-solutions.com> A bit earlier, I was helping Yurii Rashkovskii (yrashk) with his code for Agner (https://github.com/agner/agner) and he tripped on a pretty interesting corner case. I'm writing this both to document what we found (and how we solved it) and to ask a few questions about the reasons for things to be this way (skip to the end if you don't want the explanation). His application is basically structured in a way where the application is started, starts a top-level supervisor, which starts a server and another supervisor, which in turn spawns the dynamic children (hoping this looks right in the email): App ---> TopSup ---> Sup ---> [SimpleOneForOneWorkers] | SomeWorker [url just in case: http://ideone.com/pr0vc] Now the thing is that the documentation says the following: "Important note on simple-one-for-one supervisors: The dynamically created child processes of a simple-one-for-one supervisor are not explicitly killed, regardless of shutdown strategy, but are expected to terminate when the supervisor does (that is, when an exit signal from the parent process is received)." And indeed they are not. The supervisor just kills its regular children and then disappears, leaving it to the simple-one-for-one children's behaviours to catch the exit message and leave. This, alone is fine. Next thing we have is the application itself. For each application, OTP spawns an Application Controller (AC) which acts as a group leader. As a reminder, the AC is linked both to its parent and its direct child and monitors them. When any of them fails, the AC terminates its own execution, using its status as a group leader to terminate all of the leftover children. Again, this alone is fine. However, if you mix in both features, and then decide to shut the application down with 'application:stop(agner)', you end up in a very troublesome situation: App --> TopSup (dead) --> Sup (dead) --> [SimpleOneForOneWorkers] | SomeWorker (dead) [url just in case: http://ideone.com/KklZ8] At this precise point in time, both supervisors are dead, as well as the regular worker in the app. The simple-one-for-one (SOFO) workers are currently dying, each catching the 'EXIT' signal sent by their direct ancestor. At the same time, though, The AC gets wind of its direct child dying (TopSup) and ends up killing every one of the SOFO workers that weren't dead yet. The result is a bunch of workers which managed to clean up after them, and a bunch of others that didn't manage to do it. This is highly timing dependent, hard to debug and easy to fix. Yurii and I basically found two fixes for that one. The first one is to simply make the SOFO workers transient and kill them beforehand (which is messy). The second one is to use the 'ApplicationCallback:prep_stop(State)' function to fetch a list of all the dynamic SOFO children, monitor them, and then wait for all of them to die in the 'stop(State)' callback function. This forces the application controller to stay alive until all of the dynamic children died. You can see the actual implementation here: https://github.com/agner/agner/blob/db2527cfa2133a0679d8da999d1c37567151b49f/src/agner_app.erl -- Now for the questions: - Is there any reason for simple_one_for_one supervisors to use this kind of asynchronous scheme when it comes to terminating? We wouldn't have had any bug at all if it were possible to call for a synchronous termination. Is this due to any legacy reason or because of general performances for larger SOFO supervisors? - Is there any way someone can think of to document this kind of error? We realise this really is a corner case, but it's a known one (at least now it is) and it could be useful to read about it somewhere without having to do the big logical leap between the application controller (which we needed to dive in the source) and the supervisor of simple_one_for_one workers terminating behaviours. - Can any of you think of a cleaner way to shut things down than what we used? Thanks for reading, -- Fred H?bert http://www.erlang-solutions.com From fred.hebert@REDACTED Fri Feb 4 18:32:26 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Fri, 4 Feb 2011 12:32:26 -0500 Subject: [erlang-questions] OTP Application Controller Vs. Supervisors with simple_one_for_one workers In-Reply-To: <7AD4C90A-0CF3-47E3-A4B9-DA921FEFC9E1@erlang-solutions.com> References: <7AD4C90A-0CF3-47E3-A4B9-DA921FEFC9E1@erlang-solutions.com> Message-ID: <5068F9D9-A0A4-44B6-893D-AD4182EEB891@erlang-solutions.com> Sorry, It seems I got it wrong -- it's the Application Master that actually does the killing, not the Application Controller. Just substitute one for the other in the post :) -- Fred H?bert http://www.erlang-solutions.com On 2011-02-04, at 12:29 PM, Fr?d?ric Trottier-H?bert wrote: > A bit earlier, I was helping Yurii Rashkovskii (yrashk) with his code for Agner (https://github.com/agner/agner) and he tripped on a pretty interesting corner case. I'm writing this both to document what we found (and how we solved it) and to ask a few questions about the reasons for things to be this way (skip to the end if you don't want the explanation). > > His application is basically structured in a way where the application is started, starts a top-level supervisor, which starts a server and another supervisor, which in turn spawns the dynamic children (hoping this looks right in the email): > > App ---> TopSup ---> Sup ---> [SimpleOneForOneWorkers] > | > SomeWorker > [url just in case: http://ideone.com/pr0vc] > > Now the thing is that the documentation says the following: "Important note on simple-one-for-one supervisors: The dynamically created child processes of a simple-one-for-one supervisor are not explicitly killed, regardless of shutdown strategy, but are expected to terminate when the supervisor does (that is, when an exit signal from the parent process is received)." > > And indeed they are not. The supervisor just kills its regular children and then disappears, leaving it to the simple-one-for-one children's behaviours to catch the exit message and leave. This, alone is fine. > > Next thing we have is the application itself. For each application, OTP spawns an Application Controller (AC) which acts as a group leader. As a reminder, the AC is linked both to its parent and its direct child and monitors them. When any of them fails, the AC terminates its own execution, using its status as a group leader to terminate all of the leftover children. Again, this alone is fine. > > However, if you mix in both features, and then decide to shut the application down with 'application:stop(agner)', you end up in a very troublesome situation: > > App --> TopSup (dead) --> Sup (dead) --> [SimpleOneForOneWorkers] > | > SomeWorker (dead) > [url just in case: http://ideone.com/KklZ8] > > At this precise point in time, both supervisors are dead, as well as the regular worker in the app. The simple-one-for-one (SOFO) workers are currently dying, each catching the 'EXIT' signal sent by their direct ancestor. > > At the same time, though, The AC gets wind of its direct child dying (TopSup) and ends up killing every one of the SOFO workers that weren't dead yet. > > The result is a bunch of workers which managed to clean up after them, and a bunch of others that didn't manage to do it. This is highly timing dependent, hard to debug and easy to fix. > > Yurii and I basically found two fixes for that one. The first one is to simply make the SOFO workers transient and kill them beforehand (which is messy). The second one is to use the 'ApplicationCallback:prep_stop(State)' function to fetch a list of all the dynamic SOFO children, monitor them, and then wait for all of them to die in the 'stop(State)' callback function. This forces the application controller to stay alive until all of the dynamic children died. > > You can see the actual implementation here: https://github.com/agner/agner/blob/db2527cfa2133a0679d8da999d1c37567151b49f/src/agner_app.erl > > -- > Now for the questions: > > - Is there any reason for simple_one_for_one supervisors to use this kind of asynchronous scheme when it comes to terminating? We wouldn't have had any bug at all if it were possible to call for a synchronous termination. Is this due to any legacy reason or because of general performances for larger SOFO supervisors? > > - Is there any way someone can think of to document this kind of error? We realise this really is a corner case, but it's a known one (at least now it is) and it could be useful to read about it somewhere without having to do the big logical leap between the application controller (which we needed to dive in the source) and the supervisor of simple_one_for_one workers terminating behaviours. > > - Can any of you think of a cleaner way to shut things down than what we used? > > Thanks for reading, > > -- > Fred H?bert > http://www.erlang-solutions.com > > > From maruthavanan_s@REDACTED Fri Feb 4 18:51:19 2011 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Fri, 4 Feb 2011 12:51:19 -0500 Subject: noproc error in starting gen_fsm Message-ID: Hi, I have gen_fsm M1 and gen_Event M2 module, I am starting the M2 module, where init on M2 will start M1 with start_link, But once I start it I receive the following error =ERROR REPORT==== 4-Feb-2011::23:00:16 === ** State machine dahdi_59 terminating ** Last message in was {'EXIT',<0.34.0>,noproc} ** When State == 'INITIALIZE' ** Data == {state} ** Reason for termination = ** noproc I thought there might be some problem with M1 and tried below in erlang shell gen_fsm:start_link({local,test},M1,[],[]). this works good below without any problem. What is wrong I am doing here? Kindly help. Thanks, Marutha From maruthavanan_s@REDACTED Fri Feb 4 19:34:31 2011 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Fri, 4 Feb 2011 13:34:31 -0500 Subject: [erlang-questions] noproc error in starting gen_fsm In-Reply-To: References: Message-ID: Hi All, Found the reason, I started as a supervisor's child and then it went away. Thanks, Marutha > From: maruthavanan_s@REDACTED > To: erlang-questions@REDACTED > Date: Fri, 4 Feb 2011 12:51:19 -0500 > Subject: [erlang-questions] noproc error in starting gen_fsm > > > Hi, > > I have gen_fsm M1 and gen_Event M2 module, > > I am starting the M2 module, where init on M2 will start M1 with start_link, > > But once I start it I receive the following error > =ERROR REPORT==== 4-Feb-2011::23:00:16 === > ** State machine dahdi_59 terminating > ** Last message in was {'EXIT',<0.34.0>,noproc} > ** When State == 'INITIALIZE' > ** Data == {state} > ** Reason for termination = > ** noproc > > I thought there might be some problem with M1 > > and tried below in erlang shell > > gen_fsm:start_link({local,test},M1,[],[]). > > this works good below without any problem. > > What is wrong I am doing here? > > Kindly help. > > Thanks, > Marutha > From ulf.wiger@REDACTED Fri Feb 4 19:41:27 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 4 Feb 2011 19:41:27 +0100 Subject: [erlang-questions] OTP Application Controller Vs. Supervisors with simple_one_for_one workers In-Reply-To: <7AD4C90A-0CF3-47E3-A4B9-DA921FEFC9E1@erlang-solutions.com> References: <7AD4C90A-0CF3-47E3-A4B9-DA921FEFC9E1@erlang-solutions.com> Message-ID: <1AC1D795-CB81-4A1A-B48A-79DECF140D59@erlang-solutions.com> I am not going to guess at the reason for the behaviour. :) The simplest workaround I can think of would be to set the group_leader of each SOFO worker to 'user' (i.e. whereis(user)). This has one unwanted side-effect, namely that application:get_application(SOFO) will always return {ok, kernel}, which is probably not the desired result. The application_master only forwards the IO requests to its original group_leader (which I believe is 'user'). There may be more unwanted side-effects that I haven't thought of. Dirty tricks are always? well, dirty. BR, Ulf On 4 Feb 2011, at 18:29, Fr?d?ric Trottier-H?bert wrote: > A bit earlier, I was helping Yurii Rashkovskii (yrashk) with his code for Agner (https://github.com/agner/agner) and he tripped on a pretty interesting corner case. I'm writing this both to document what we found (and how we solved it) and to ask a few questions about the reasons for things to be this way (skip to the end if you don't want the explanation). > > His application is basically structured in a way where the application is started, starts a top-level supervisor, which starts a server and another supervisor, which in turn spawns the dynamic children (hoping this looks right in the email): > > App ---> TopSup ---> Sup ---> [SimpleOneForOneWorkers] > | > SomeWorker > [url just in case: http://ideone.com/pr0vc] > > Now the thing is that the documentation says the following: "Important note on simple-one-for-one supervisors: The dynamically created child processes of a simple-one-for-one supervisor are not explicitly killed, regardless of shutdown strategy, but are expected to terminate when the supervisor does (that is, when an exit signal from the parent process is received)." > > And indeed they are not. The supervisor just kills its regular children and then disappears, leaving it to the simple-one-for-one children's behaviours to catch the exit message and leave. This, alone is fine. > > Next thing we have is the application itself. For each application, OTP spawns an Application Controller (AC) which acts as a group leader. As a reminder, the AC is linked both to its parent and its direct child and monitors them. When any of them fails, the AC terminates its own execution, using its status as a group leader to terminate all of the leftover children. Again, this alone is fine. > > However, if you mix in both features, and then decide to shut the application down with 'application:stop(agner)', you end up in a very troublesome situation: > > App --> TopSup (dead) --> Sup (dead) --> [SimpleOneForOneWorkers] > | > SomeWorker (dead) > [url just in case: http://ideone.com/KklZ8] > > At this precise point in time, both supervisors are dead, as well as the regular worker in the app. The simple-one-for-one (SOFO) workers are currently dying, each catching the 'EXIT' signal sent by their direct ancestor. > > At the same time, though, The AC gets wind of its direct child dying (TopSup) and ends up killing every one of the SOFO workers that weren't dead yet. > > The result is a bunch of workers which managed to clean up after them, and a bunch of others that didn't manage to do it. This is highly timing dependent, hard to debug and easy to fix. > > Yurii and I basically found two fixes for that one. The first one is to simply make the SOFO workers transient and kill them beforehand (which is messy). The second one is to use the 'ApplicationCallback:prep_stop(State)' function to fetch a list of all the dynamic SOFO children, monitor them, and then wait for all of them to die in the 'stop(State)' callback function. This forces the application controller to stay alive until all of the dynamic children died. > > You can see the actual implementation here: https://github.com/agner/agner/blob/db2527cfa2133a0679d8da999d1c37567151b49f/src/agner_app.erl > > -- > Now for the questions: > > - Is there any reason for simple_one_for_one supervisors to use this kind of asynchronous scheme when it comes to terminating? We wouldn't have had any bug at all if it were possible to call for a synchronous termination. Is this due to any legacy reason or because of general performances for larger SOFO supervisors? > > - Is there any way someone can think of to document this kind of error? We realise this really is a corner case, but it's a known one (at least now it is) and it could be useful to read about it somewhere without having to do the big logical leap between the application controller (which we needed to dive in the source) and the supervisor of simple_one_for_one workers terminating behaviours. > > - Can any of you think of a cleaner way to shut things down than what we used? > > Thanks for reading, > > -- > Fred H?bert > http://www.erlang-solutions.com > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From boris.muehmer@REDACTED Sat Feb 5 09:01:21 2011 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Sat, 5 Feb 2011 09:01:21 +0100 Subject: How to connect erlang nodes over IPv6 (on Ubuntu 10.04 server installations, Erlang R14B1)? Message-ID: I try to connect Erlang nodes using only IPv6. But somehow it doesn't work. The systems in questions are Ubuntu 10.04 based server setups. The DNS names of the participating hosts resolv only to IPv6 addresses. "ping6" and "traceroute6" work without any problems. After running "erl -name node-X", I try to "net_adm:ping/1" the other host, but I only get "pang"s. I also tried to add "-proto_dist inet6_tcp" (found this information with google) to the erl command line, but it doesn't improve the situation. Does anyone have any hints how to start an IPv6 based Erlang "cluster", please? - boris From alexey.v.romanov@REDACTED Sat Feb 5 10:45:37 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Sat, 5 Feb 2011 12:45:37 +0300 Subject: Passing emulator command line flags to escript Message-ID: According to http://www.erlang.org/doc/man/escript.html, it doesn't seem possible to start the same script with different emulator arguments without editing the script. Is this so? It would seem to make some sense for escript to support such flags in the same way as erl does... Yours, Alexey Romanov From tony@REDACTED Sat Feb 5 23:02:24 2011 From: tony@REDACTED (Tony Rogvall) Date: Sat, 5 Feb 2011 23:02:24 +0100 Subject: [erlang-questions] How to connect erlang nodes over IPv6 (on Ubuntu 10.04 server installations, Erlang R14B1)? In-Reply-To: References: Message-ID: <5ACC5034-A565-4634-AF64-BF3429E3C4B5@rogvall.se> Hi! Yes the -proto_dist inet6_tcp is required, since the default is inet_tcp. A (near) future release may even default to handle both ipv6 and ipv4. The transition to ipv6 will happen any day now ;-) Any how, at least on my mac I discovered that epmd did not accept IPv6 connection for looking up the port name. (ipv4) erl_epmd:port_please("b", {127,0,0,1}, 1000). {port,65347,5} (ipv6) erl_epmd:port_please("b", {0,0,0,0,0,0,0,1}, 1000). noport I have not been hacking on this for years ! Any one know the status of this ? /Tony On 5 feb 2011, at 09.01, Boris M?hmer wrote: > I try to connect Erlang nodes using only IPv6. But somehow it doesn't > work. The systems in questions are Ubuntu 10.04 based server setups. > > The DNS names of the participating hosts resolv only to IPv6 > addresses. "ping6" and "traceroute6" work without any problems. > > After running "erl -name node-X", I try to "net_adm:ping/1" the other > host, but I only get "pang"s. > > I also tried to add "-proto_dist inet6_tcp" (found this information > with google) to the erl command line, but it doesn't improve the > situation. > > Does anyone have any hints how to start an IPv6 based Erlang "cluster", please? > > > - boris > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > "Have run Make so many times I dunno what's installed anymore" From als@REDACTED Sun Feb 6 03:48:30 2011 From: als@REDACTED (Anthony Shipman) Date: Sun, 6 Feb 2011 13:48:30 +1100 Subject: [erlang-questions] Passing emulator command line flags to escript In-Reply-To: References: Message-ID: <201102061348.30228.als@iinet.net.au> On Sat, 5 Feb 2011 08:45:37 pm Alexey Romanov wrote: > According to http://www.erlang.org/doc/man/escript.html, it doesn't > seem possible to start the same script with different emulator > arguments without editing the script. Is this so? It would seem to > make some sense for escript to support such flags in the same way as > erl does... > > Yours, Alexey Romanov > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED You can use the ERL_AFLAGS environment variable. -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From alexey.v.romanov@REDACTED Sun Feb 6 09:09:52 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Sun, 6 Feb 2011 11:09:52 +0300 Subject: [erlang-questions] Passing emulator command line flags to escript In-Reply-To: <201102061348.30228.als@iinet.net.au> References: <201102061348.30228.als@iinet.net.au> Message-ID: On Sun, Feb 6, 2011 at 5:48 AM, Anthony Shipman wrote: > On Sat, 5 Feb 2011 08:45:37 pm Alexey Romanov wrote: >> According to http://www.erlang.org/doc/man/escript.html, it doesn't >> seem possible to start the same script with different emulator >> arguments without editing the script. Is this so? It would seem to >> make some sense for escript to support such flags in the same way as >> erl does... >> >> Yours, Alexey Romanov >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > You can use the ERL_AFLAGS environment variable. Is doesn't only work for erl? Thanks! Yours, Alexey Romanov > -- > Anthony Shipman ? ? ? ? ? ? ? ? ? ?Mamas don't let your babies > als@REDACTED ? ? ? ? ? ? ? ? ? grow up to be outsourced. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From zerthurd@REDACTED Sun Feb 6 12:45:12 2011 From: zerthurd@REDACTED (Maxim Treskin) Date: Sun, 6 Feb 2011 17:45:12 +0600 Subject: [erlang-questions] Passing emulator command line flags to escript In-Reply-To: References: <201102061348.30228.als@iinet.net.au> Message-ID: Hi Alexey You can use commented out with '%%!' line of parameters: #!/usr/bin/escript %%! +K true +A 20 On 6 February 2011 14:09, Alexey Romanov wrote: > On Sun, Feb 6, 2011 at 5:48 AM, Anthony Shipman wrote: > > On Sat, 5 Feb 2011 08:45:37 pm Alexey Romanov wrote: > >> According to http://www.erlang.org/doc/man/escript.html, it doesn't > >> seem possible to start the same script with different emulator > >> arguments without editing the script. Is this so? It would seem to > >> make some sense for escript to support such flags in the same way as > >> erl does... > >> > >> Yours, Alexey Romanov > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > You can use the ERL_AFLAGS environment variable. > > Is doesn't only work for erl? Thanks! > > Yours, Alexey Romanov > > > > > -- > > Anthony Shipman Mamas don't let your babies > > als@REDACTED grow up to be outsourced. > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Maxim Treskin From zerthurd@REDACTED Sun Feb 6 12:46:19 2011 From: zerthurd@REDACTED (Maxim Treskin) Date: Sun, 6 Feb 2011 17:46:19 +0600 Subject: [erlang-questions] Passing emulator command line flags to escript In-Reply-To: References: <201102061348.30228.als@iinet.net.au> Message-ID: Sorry for my answer, I missread your post :) On 6 February 2011 17:45, Maxim Treskin wrote: > Hi Alexey > > You can use commented out with '%%!' line of parameters: > > #!/usr/bin/escript > %%! +K true +A 20 > > > > On 6 February 2011 14:09, Alexey Romanov wrote: > >> On Sun, Feb 6, 2011 at 5:48 AM, Anthony Shipman wrote: >> > On Sat, 5 Feb 2011 08:45:37 pm Alexey Romanov wrote: >> >> According to http://www.erlang.org/doc/man/escript.html, it doesn't >> >> seem possible to start the same script with different emulator >> >> arguments without editing the script. Is this so? It would seem to >> >> make some sense for escript to support such flags in the same way as >> >> erl does... >> >> >> >> Yours, Alexey Romanov >> >> >> >> ________________________________________________________________ >> >> erlang-questions (at) erlang.org mailing list. >> >> See http://www.erlang.org/faq.html >> >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > >> > You can use the ERL_AFLAGS environment variable. >> >> Is doesn't only work for erl? Thanks! >> >> Yours, Alexey Romanov >> >> >> >> > -- >> > Anthony Shipman Mamas don't let your babies >> > als@REDACTED grow up to be outsourced. >> > >> > ________________________________________________________________ >> > erlang-questions (at) erlang.org mailing list. >> > See http://www.erlang.org/faq.html >> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > >> > >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > > -- > Maxim Treskin > -- Maxim Treskin From nirth.furzahad@REDACTED Sun Feb 6 15:58:10 2011 From: nirth.furzahad@REDACTED (David Sergey) Date: Sun, 6 Feb 2011 16:58:10 +0200 Subject: YAWS RESTful examples In-Reply-To: References: <13448848.82521290583867809.JavaMail.www@wsfrf1116> Message-ID: Hello List I have upcoming project where I can chose technology for building RESTful web services (html-less). I found some examples with YAWS, but they are all dated 2008-2009. I was wondering if YAWS is still best option for building web-services on Erlang platform? Also, does anyone know any turorials/examples/literature on a subject, documentation seems pretty helpful, but examples always welcome. Best Regards David From garazdawi@REDACTED Sun Feb 6 16:11:45 2011 From: garazdawi@REDACTED (Lukas Larsson) Date: Sun, 6 Feb 2011 16:11:45 +0100 Subject: [erlang-questions] YAWS RESTful examples In-Reply-To: References: <13448848.82521290583867809.JavaMail.www@wsfrf1116> Message-ID: If you are building RESTfull things in Erlang you want to have a look at webmachine. It takes care of alot of the details which come with RESTfull http without you having to worry about them. http://webmachine.basho.com/ Lukas On Sun, Feb 6, 2011 at 3:58 PM, David Sergey wrote: > Hello List > > I have upcoming project where I can chose technology for building RESTful > web services (html-less). I found some examples with YAWS, but they are all > dated 2008-2009. I was wondering if YAWS is still best option for building > web-services on Erlang platform? > > Also, does anyone know any turorials/examples/literature on a subject, > documentation seems pretty helpful, but examples always welcome. > > Best Regards > David > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From alain.odea@REDACTED Sun Feb 6 16:32:42 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sun, 6 Feb 2011 12:02:42 -0330 Subject: [erlang-questions] YAWS RESTful examples In-Reply-To: References: <13448848.82521290583867809.JavaMail.www@wsfrf1116> Message-ID: Hi David: I totally agree with Lukas. I could bot make heads or tails of REST on YAWS. It seemed like a square peg in a round hole. With Webmachine I built Scrumjet, a toy task tracker with a RESTful interface and Mnesia persistence: http://github.com/AlainODea/scrumjet I have released several Zotonic modules built on Webmachine: http://github.com/AlainODea/mod_paypal http://github.com/AlainODea/mod_download http://github.com/AlainODea/mod_term_feed Best regards, Alain On 2011-02-06, at 11:41, Lukas Larsson wrote: > If you are building RESTfull things in Erlang you want to have a look at > webmachine. It takes care of alot of the details which come with RESTfull > http without you having to worry about them. > > http://webmachine.basho.com/ > > Lukas > > On Sun, Feb 6, 2011 at 3:58 PM, David Sergey wrote: > >> Hello List >> >> I have upcoming project where I can chose technology for building RESTful >> web services (html-less). I found some examples with YAWS, but they are all >> dated 2008-2009. I was wondering if YAWS is still best option for building >> web-services on Erlang platform? >> >> Also, does anyone know any turorials/examples/literature on a subject, >> documentation seems pretty helpful, but examples always welcome. >> >> Best Regards >> David >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From john@REDACTED Sun Feb 6 16:11:18 2011 From: john@REDACTED (John Kemp) Date: Sun, 6 Feb 2011 10:11:18 -0500 Subject: [erlang-questions] YAWS RESTful examples In-Reply-To: References: <13448848.82521290583867809.JavaMail.www@wsfrf1116> Message-ID: <713EC026-DD47-43D3-B77E-A4C99BA76FA2@jkemp.net> Hi David, On Feb 6, 2011, at 9:58 AM, David Sergey wrote: > Hello List > > I have upcoming project where I can chose technology for building RESTful web services (html-less). I found some examples with YAWS, but they are all dated 2008-2009. I was wondering if YAWS is still best option for building web-services on Erlang platform? I've successfully used WebMachine: https://bitbucket.org/justin/webmachine/wiki/Home and Mochiweb: https://github.com/mochi/mochiweb for building Web services. Cheers, - John > > Also, does anyone know any turorials/examples/literature on a subject, documentation seems pretty helpful, but examples always welcome. > > Best Regards > David > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From fritchie@REDACTED Sun Feb 6 20:25:00 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Sun, 06 Feb 2011 13:25:00 -0600 Subject: [erlang-questions] gen_server clustering strategy In-Reply-To: Message of "Wed, 02 Feb 2011 21:27:37 +0100." Message-ID: <18395.1297020300@snookles.snookles.com> Following-up many days after the last posting, sorry. Knut Ivar Nesheim wrote: kin> * We need to be able to send messages from every process to any kin> other process, so we could use some form of registry. Using the kin> global module would get us going, but not take us all the way kin> (which is something we can worry about later). 'global' will allow prototype code to work in a small development system, but it won't scale to what you need. Previous messages have pointed to reasons why. kin> * We have considered writing our own process registry, based on ets kin> or mnesia or redis or whatever [...] Mnesia and Redis and Riak would work quite well for that, ... kin> * We would like to add nodes without disturbing the processes kin> already running, so consistent hashing must be used if hashing... ... though it isn't clear to me (haven't followed recent Redis developments) if Redis can cope with changing cluster sizes dynamically. Mnesia and Riak both can. kin> Our worry with the cluster of identical nodes, the "mesh" described kin> earlier, is that there will be too many messages passing around. kin> [...] kin> Our worry with the cluster that includes "proxy" nodes, is that kin> they must maintain the state of the cluster in order to route kin> accordingly. At least they must know where to send the requests kin> using some form of hashing. Er, you could build a system that routes messages like that, but neither Mnesia nor Riak use that method, so one need not worry about it. kin> To this end, building on top of riak_core might be a very good kin> option. However, we feel that riak_core is pretty complex to kin> understand fully and would prefer a simpler solution. Riak Core's callback functions aren't terribly complex to implement. Though for your name->pid process registry, using the as-is Riak KV application would be sufficient: KV is already using Core to figure out where your key to map name->pid would be. kin> If riak is running on 60 nodes, then the question for us is how kin> much message passing is going on. The hash calculation is O(1) with respect to CPU time, and fetching the value is O(1) with respect to number of messages required. kin> Also, what kind of overhead can we expect if we monitor processes kin> on other nodes? Our understanding is that the information regarding kin> pids that die, will be sent to any node in the cluster anyway, so kin> actually monitoring and receiving the info won't make that much of kin> a difference as it is already there. Er, if I'm understanding your understanding, no. Information on a monitored process P is only sent to interested parties, i.e. those procs that actually created a monitor relationship with P. One thing that you didn't mention is what happens to the application when one of your gen_server worker processes dies. Do those processes contain state that must be persistent? If that state must be persistent, then you have another data management task that you haven't asked about yet. -Scott From nirth.furzahad@REDACTED Sun Feb 6 20:43:21 2011 From: nirth.furzahad@REDACTED (David Sergey) Date: Sun, 6 Feb 2011 21:43:21 +0200 Subject: [erlang-questions] YAWS RESTful examples In-Reply-To: <1297020976.15438.40.camel@devil> References: <13448848.82521290583867809.JavaMail.www@wsfrf1116> <1297005194.15438.36.camel@devil> <1297020976.15438.40.camel@devil> Message-ID: Hi Klaus Thanks for pointing it out. Will check address next time. Regrds David On 6 Feb 2011, at 21:36, Klaus Trainer wrote: > Hi David, > > annoyingly, this mailing list doesn't set the "Reply-to" header. Hence, > my reply exclusively arrived at your address instead of the list's. > Also, it's the same case with your reply here... > > Cheers, > Klaus > > > On Sun, 2011-02-06 at 20:54 +0200, David Sergey wrote: >> Thank you guys. >> >> Webmachine seems to be a perfect match. >> >> Thanks >> >> On 6 Feb 2011, at 17:13, Klaus Trainer wrote: >> >>> I haven't used Yaws yet, so I can't say anything with regard to that. >>> >>> However, if you're building RESTful web services, you should probably >>> have a look at Webmachine (http://webmachine.basho.com), which is a >>> "REST toolkit" on top of Mochiweb. >>> >>> - Klaus >>> >>> >>> On Sun, 2011-02-06 at 16:58 +0200, David Sergey wrote: >>>> Hello List >>>> >>>> I have upcoming project where I can chose technology for building RESTful web services (html-less). I found some examples with YAWS, but they are all dated 2008-2009. I was wondering if YAWS is still best option for building web-services on Erlang platform? >>>> >>>> Also, does anyone know any turorials/examples/literature on a subject, documentation seems pretty helpful, but examples always welcome. >>>> >>>> Best Regards >>>> David >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>> >>> >> > > From brum76@REDACTED Mon Feb 7 00:08:57 2011 From: brum76@REDACTED (Radu Brumariu) Date: Sun, 6 Feb 2011 15:08:57 -0800 Subject: Is this code tail recursive ? Message-ID: gather_files([]) -> []; gather_files(F) -> gather_files(F,[]). gather_files([],[Acc]) -> Acc; gather_files([{path,F}|Files],Acc) -> io:format("Processing path ~p~n",[F]), {ok, T} = file:read_file_info(F), case T#file_info.type of regular -> gather_files(Files, [F | Acc]); directory -> gather_files(Files, [gather_directory_files(F) | Acc ]); _ -> gather_files(Files, Acc) end. gather_directory_files(D) -> filelib:fold_files( D, ".*", true, fun(X,Acc) -> [X|Acc] end, []). If not, how can it be rewritten ? Thanks, Radu From mihai@REDACTED Mon Feb 7 01:05:32 2011 From: mihai@REDACTED (Mihai Balea) Date: Sun, 6 Feb 2011 19:05:32 -0500 Subject: [erlang-questions] Is this code tail recursive ? In-Reply-To: References: Message-ID: On Feb 6, 2011, at 6:08 PM, Radu Brumariu wrote: > gather_files([]) -> []; > gather_files(F) -> > gather_files(F,[]). > gather_files([],[Acc]) -> Acc; > gather_files([{path,F}|Files],Acc) -> > io:format("Processing path ~p~n",[F]), > {ok, T} = file:read_file_info(F), > case T#file_info.type > of > regular -> > gather_files(Files, [F | Acc]); > directory -> > gather_files(Files, [gather_directory_files(F) | Acc ]); > _ -> > gather_files(Files, Acc) > end. > > gather_directory_files(D) -> > filelib:fold_files( D, ".*", true, fun(X,Acc) -> > [X|Acc] > end, > []). > Looks tail recursive to me, although looking at the way you build up your result, it might not be what you expect :) Not sure what you expect though, so I cannot comment on that. Hope this helps, Mihai From boris.muehmer@REDACTED Mon Feb 7 06:40:28 2011 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Mon, 7 Feb 2011 06:40:28 +0100 Subject: [erlang-questions] How to connect erlang nodes over IPv6 (on Ubuntu 10.04 server installations, Erlang R14B1)? In-Reply-To: <5ACC5034-A565-4634-AF64-BF3429E3C4B5@rogvall.se> References: <5ACC5034-A565-4634-AF64-BF3429E3C4B5@rogvall.se> Message-ID: [ Note 1: I am working with the source tarball R14B01 from erlang.org. ] [ Note 2: The systems I use are 64 Bit, using either Ubuntu 10.04 or 10.10. ] [ Note 3: The current test system is a Ubuntu 10.10 desktop, 64 Bit. ] I had a look at "erts/epmd/src/epmd_int.h". In there is the following line: #if defined(HAVE_IN6) && defined(AF_INET6) && defined(EPMD6) On my system "HAVE_IN6" is defined during "configure" (e.g. "erts/x86_64-unknown-linux-gnu/config.h"), but I can't find a definition for "AF_INET6" or "EPMD6". I checked one system with running "epmd -d" manually and calling "netstat -an" afterwards: I only could find an "0.0.0.0:4369" entry => tcp 0 0 0.0.0.0:4369 0.0.0.0:* LISTEN After setting "AF_INET6" or "EPMD6" manually in the "config.h" file, e.g. adding #define AF_INET6 1 #define EPMD6 1 to "erts/x86_64-unknown-linux-gnu/config.h" and doing the above test, netstat returned tcp6 0 0 :::4369 :::* LISTEN But calling "erl -name test -setcookie abc123 -proto_dist inet6_tcp" results in a crash dump being written: ---> snipp <--- $ /opt/Erlang/OTP/R14B01/20110206/1030/bin/erl -name test -setcookie abc123 -proto_dist inet6_tcp {error_logger,{{2011,2,6},{11,26,45}},"Protocol: ~p: register/listen error: ~p~n",["inet6_tcp",eprotonosupport]} {error_logger,{{2011,2,6},{11,26,45}},crash_report,[[{initial_call,{net_kernel,init,['Argument__1']}},{pid,<0.19.0>},{registered_name,[]},{error_info,{exit,{error,badarg},[{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]}},{ancestors,[net_sup,kernel_sup,<0.9.0>]},{messages,[]},{links,[<0.16.0>]},{dictionary,[{longnames,true}]},{trap_exit,true},{status,running},{heap_size,377},{stack_size,24},{reductions,246}],[]]} {error_logger,{{2011,2,6},{11,26,45}},supervisor_report,[{supervisor,{local,net_sup}},{errorContext,start_error},{reason,{'EXIT',nodistribution}},{offender,[{pid,undefined},{name,net_kernel},{mfargs,{net_kernel,start_link,[[test,longnames]]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]} {error_logger,{{2011,2,6},{11,26,45}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,shutdown},{offender,[{pid,undefined},{name,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]} {error_logger,{{2011,2,6},{11,26,45}},std_info,[{application,kernel},{exited,{shutdown,{kernel,start,[normal,[]]}}},{type,permanent}]} {"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}}"} Crash dump was written to: erl_crash.dump Kernel pid terminated (application_controller) ({application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}}) ---> snipp <--- - boris 2011/2/5 Tony Rogvall : > Hi! > Yes the -proto_dist inet6_tcp is required, since the default is inet_tcp. A > (near) future release may even > default to handle both ipv6 and ipv4. The transition to ipv6 will happen any > day now ;-) > Any how, at least on my mac I discovered that epmd did not accept IPv6 > connection for > looking up the port name. > (ipv4) > erl_epmd:port_please("b", {127,0,0,1}, 1000). > {port,65347,5} > (ipv6) > erl_epmd:port_please("b", {0,0,0,0,0,0,0,1}, 1000). > noport > > I have not been hacking on this for years ! Any one know the status of this > ? > /Tony > > > On 5 feb 2011, at 09.01, Boris M?hmer wrote: > > I try to connect Erlang nodes using only IPv6. But somehow it doesn't > work. The systems in questions are Ubuntu 10.04 based server setups. > > The DNS names of the participating hosts resolv only to IPv6 > addresses. "ping6" and "traceroute6" work without any problems. > > After running "erl -name node-X", I try to "net_adm:ping/1" the other > host, but I only get "pang"s. > > I also tried to add "-proto_dist inet6_tcp" (found this information > with google) to the erl command line, but it doesn't improve the > situation. > > Does anyone have any hints how to start an IPv6 based Erlang "cluster", > please? > > > ?- boris > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > "Have run Make so many times I dunno what's installed anymore" > From mitja.thomas1@REDACTED Mon Feb 7 09:22:35 2011 From: mitja.thomas1@REDACTED (Mitja Thomas) Date: Mon, 07 Feb 2011 09:22:35 +0100 Subject: [erlang-questions] Problem detecting interrupted TCP Connection In-Reply-To: <4D495A61.7050209@ewetel.de> References: <4D494954.8010103@ewetel.de> <4D495A61.7050209@ewetel.de> Message-ID: <4D4FABCB.60807@ewetel.de> Hello again, has anyone tried this the way I described? I repeated the process at home once again with R14B01 on a windows machines, to make sure its not due to some outdated erlang version (can't get all the versions on old centos) or due to some firewall restrictions. Any hint would be great. Thanks in advance Mitja > When I reread my mail, I think I should clearify the events that > produced the problematic behaviour. > - I start the server (I tried it without the {ip, ...} option). > - I connect from remote via Telnet. > - I then take of the ethernet connection of the client. Quit telnet > and wait a minute, so no resendings would come into my way. > - I reestablish the Ethernet Conenction from client (actually that > wouldnt even matter because the server should recognise the lost > connection anyway). > > The output I get is: > 1> tcp:start(). > Listener started > still running > still running > Starting client (#Port<0.521>) > still running > still running > still running > still running > still running %%% Somewhere here I pulled the plug > still running > still running > still running > still running > Send ret value:ok > still running > still running > still running > still running > still running > still running > still running > still running > still running > still running > Send ret value:ok > still running > still running > still running > still running > still running > still running > still running > still running > still running > still running > Send ret value:ok > still running > ... > > and so forth. Normally I would have suspected a some recognition of > the lost socket at least when the first ping message is sent after the > connection is interrupted. > >> I tried your code and it worked beautifully - but I had to remove the >> {ip, {xxx,...}} term - if you omit the IP then you this will default >> to the local interface >> >> then you can do >> > telnet localhost 8999, or >> > telnet 127.0.0.1 8999, or >> > telnet 8999 >> >> all worked as expected >> >> /Joe >> >> >> On Wed, Feb 2, 2011 at 1:08 PM, Mitja Thomas > > wrote: >> >> Hello Folks, >> >> I'm new to this Mailing List, so my question might already have >> been answered at some point. Still I got a strange behaviour in a >> TCP Session which I just cant figur out. >> >> Beckground: >> I have a Server who listens on a Port for incomming TCP >> Connections and handles them in a User List. When something >> happens for this User (on the other side, which doesnt matter for >> the problem), the user is informed and can send commands. When the >> TCP Connection ist closed, the Socket is closed and the User will >> be erased from the User list. >> >> Problem: >> When the user somehow disconnects his TCP/IP Connection and closes >> the client application, my server dont get the tcp_closed Event >> and the user is still present in the List. Well I know this is the >> way TCP should behave, thus the server tries to send a packet to >> the Socket once in while and should receive and Error of some >> kind. But it does not, it just keeps "sending", without actually >> sending TCP Packets. >> The Problem is well explained in gen_tcp manual page and an >> example how it should be handled in such a way with an application >> ping is given. But I just dont the {error, timeout} while trying >> to send to the broken TCP Socket. >> >> I broke it down to a simple Server, who just pings the client and >> tried various ways to do it. Im pretty sure, Im doing something >> wrong, but I just dont know what. Any Help is appreciated. >> >> Regards >> Mitja >> >> sample Code (Client is normally a simple Telnet): >> -module(tcp). >> -compile(export_all). >> >> start() -> >> {ok, Listener} = gen_tcp:listen(8999, [binary, {ip, >> {xxx,xxx,xxx,xxx}}, {active, true}, {packet, 0}, {keepalive, >> true}, {reuseaddr, true}, {send_timeout_close, true}, >> {send_timeout, 100}]), >> io:format("Listener started~n"), >> acceptor(Listener). >> >> acceptor(Listener) -> >> case gen_tcp:accept(Listener, 1000) of >> {ok, Socket} -> >> io:format("Starting client (~p)\n", [Socket]), >> Pid=spawn_link(fun()->loop(Socket) end), >> gen_tcp:controlling_process(Socket, Pid), >> acceptor(Listener); >> >> {error, timeout} -> io:format("still running\n"), >> acceptor(Listener); >> >> X -> io:format("SocketError:~p~n", [X]) >> end. >> >> loop(Socket) -> >> receive >> {tcp, Socket, Data} -> >> io:format("Client: Received '~p'\n", [Data]), >> loop(Socket); >> >> {tcp_closed, Socket} -> >> % This is the desired close Event >> io:format("Client: Socket closed by remote\n"); >> >> {tcp_error, Socket, Reason} -> >> io:format("Client: Socket error '~p'\n", [Reason]) >> after >> 1000 -> >> case gen_tcp:send(Socket, "Ping\r\n") of >> Ret -> >> % If Ping fails the output >> should be =/= ok >> io:format("Send ret >> value:~p~n", [Ret]), >> loop(Socket) >> end >> end. >> >> > From ingela.andin@REDACTED Mon Feb 7 09:28:44 2011 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 7 Feb 2011 09:28:44 +0100 Subject: [erlang-questions] ssl_connection process hibernation option In-Reply-To: References: Message-ID: Hi! It is already on github on our dev-branch and will be part of the next release. Regards Ingela Erlang/OTP team - Ericsson AB 2011/2/4 Kaiduan Xie : > Ingela, > > This is a great news, Ingela. When can we have this improvement? > > Best regards, > > /Kaiduan > > On Fri, Feb 4, 2011 at 10:51 AM, Ingela Andin wrote: >> Hi! >> >> We already put some effort into cleaning the state of the gen_fsm >> processes holding the connection, >> but some form of hibernation could be interesting too we will take >> this into consideration. >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> 2011/2/4 Jeroen Koops : >>> I would very much appreciate this too. >>> >>> As a test, I inserted a proxy-module between the gen_fsm and ssl_connection >>> callback, that changes all responses from the ssl_connection callback to >>> include the 'hibernate' option (it's not a hack -- it's AOP!) and the >>> savings are dramatic: before the change an SSL connection used around 350 >>> kB, after the change only around 600 bytes. If a system is maintaining >>> thousands of SSL connections, this really adds up. >>> >>> If we could somehow get hibernation built in to the ssl application, that >>> would be great. >>> >>> On Fri, Feb 4, 2011 at 9:57 AM, Ferenc Holzhauser < >>> ferenc.holzhauser@REDACTED> wrote: >>> >>>> Hi, >>>> >>>> I'm using R14B01 new SSL to set up a lot of (not very active) client >>>> connections. >>>> >>>> I have noticed that ssl_connection processes use a lot of memory between >>>> GC's. >>>> A forced GC makes them small again until the next piece of received data. >>>> >>>> This adds up to a significant memory usage (around 350-400 kB/session on a >>>> x64 system in my case) limiting the number of possible sessions. >>>> >>>> An option to hibernate this gen_fsm would decrease the average memory usage >>>> in this scenario. >>>> >>>> Would it be possible to introduce this option? >>>> >>>> Thanks in advance, >>>> Ferenc >>>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From ingela@REDACTED Mon Feb 7 09:32:02 2011 From: ingela@REDACTED (Ingela Andin) Date: Mon, 7 Feb 2011 09:32:02 +0100 Subject: [erlang-questions] ssl_connection process hibernation option In-Reply-To: References: Message-ID: Hi! We (the OTP team) would consider it. Regards Ingela Erlang/OTP team - Ericsson AB 2011/2/4 Jeroen Koops : > Hi Ingela, > Thanks for your response. Suppose I come up with a patch for this, do you > think the OTP team would consider it for inclusion in an upcoming release? > > On Fri, Feb 4, 2011 at 4:51 PM, Ingela Andin wrote: >> >> Hi! >> >> We already put some effort into cleaning the state of the gen_fsm >> processes holding the connection, >> but some form of hibernation could be interesting too we will take >> this into consideration. >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> 2011/2/4 Jeroen Koops : >> > I would very much appreciate this too. >> > >> > As a test, I inserted a proxy-module between the gen_fsm and >> > ssl_connection >> > callback, that changes all responses from the ssl_connection callback to >> > include the 'hibernate' option (it's not a hack -- it's AOP!) and the >> > savings are dramatic: before the change an SSL connection used around >> > 350 >> > kB, after the change only around 600 bytes. If a system is maintaining >> > thousands of SSL connections, this really adds up. >> > >> > If we could somehow get hibernation built in to the ssl application, >> > that >> > would be great. >> > >> > On Fri, Feb 4, 2011 at 9:57 AM, Ferenc Holzhauser < >> > ferenc.holzhauser@REDACTED> wrote: >> > >> >> Hi, >> >> >> >> I'm using R14B01 new SSL to set up a lot of (not very active) client >> >> connections. >> >> >> >> I have noticed that ssl_connection processes use a lot of memory >> >> between >> >> GC's. >> >> A forced GC makes them small again until the next piece of received >> >> data. >> >> >> >> This adds up to a significant memory usage (around 350-400 kB/session >> >> on a >> >> x64 system in my case) limiting the number of possible sessions. >> >> >> >> An option to hibernate this gen_fsm would decrease the average memory >> >> usage >> >> in this scenario. >> >> >> >> Would it be possible to introduce this option? >> >> >> >> Thanks in advance, >> >> Ferenc >> >> >> > > > From icarus.alive@REDACTED Mon Feb 7 15:01:58 2011 From: icarus.alive@REDACTED (Icarus Alive) Date: Mon, 7 Feb 2011 19:31:58 +0530 Subject: web framework, db, architecture query, with requirements Message-ID: Posted a question on Stackoverflow here: http://bit.ly/f3TIuo but this isn't exactly a cross-post, rather an Erlang specific extention of same :-) ... (to avoid any Erlang/Python religious wars, although Stackoverflow is pretty good in preventing such wars, from what I've seen). While my question there has a more generic question on which way I should be heading, i.e. Erlang or Python, I admit that my limited experience with Functional programming, and almost 12yrs history of doing OO & Procedural programming in imperative languages, my familiarity with Python is slightly higher, though love elegance of Erlang. The challenge is in understanding slightly more involved / complex constructs of Erlang. Anyhow, my query here to seek community's view on what might be the ideal route using Erlang/OTP and Erlang based Web-frameworks, given the following requirements. Here are my requirements -- 1. Web2.0 (i.e. interactive AJAXy) service, but with a parallel access via mobile web as well. 2. Application will need to do things like initiate outbound voice-calls, send/receive SMS (think Twilio) etc. 3. Application will make lot of short .flv / .3gp videos (think vimio mini), which are not mass-appeal videos (e.g. baby's 1st walk video shared with grandparents, so 1 uploader, at max 3-4 diff viewers and 3-4 views in all, so CDN is pointless -- I think). 4. There will be some video/audio transcoding (mostly ffmpeg type) 5. Application will need to show trendlines and some charts etc. (think popularity alerts / ranks) 6. Overall, should be easy to learn, easy to maintain, has good community support -- documentation / tutorials / screencasts & active-development 7. While performance is not the top-most item on my priority-list, it is definitely a parameter, including memory-footprint, as I plan to run this on cloud-infra, and every MB counts. 8. While many people tell me to think of scaling later, but I think if one can spend a little time thinking about it early one, what's the harm. Later changes are costly. The frameworks & key-components that I am considering after reading various comparisons are:- Option-1:: 1) Backend comprising of : a) RESTful App-engine: (i) Webmachine (communicating data w/ FE via JSON), or (ii) Nitrogen b) Database engine: (i) some distributed/scalable/highly-reliable (cloud-friendly) database engine (??), or (ii) VoltDB (though it's main memory requirement, 64-bit requirement and official support for CentOS5.4 only, is a bit worrying) 2) Frontend based on: jQuery (for regular internet thick-client users) / jQuery-mobile (for mobile-web users) That choice is driven by need to push rendering / non core-IP computation to the browser as much as possible & lighten the load on server, although I wonder how well it would work for the mobile-web users! PS> Completely understand that the above is probably a much-too-complex for an Erlang (& web-dev) newbie to target. But, that's the ultimate target, I am happy to take the right approach, right tools, right frameworks and keep taking baby-steps towards my goal. Some advice / guidance on what choices & why, I could make. TIA, ~i++ From alex.arnon@REDACTED Mon Feb 7 15:41:27 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Mon, 7 Feb 2011 16:41:27 +0200 Subject: [erlang-questions] web framework, db, architecture query, with requirements In-Reply-To: References: Message-ID: A possibly off-topic question: why VoltDB, and while we're there, why are its requirements a problem? On Mon, Feb 7, 2011 at 4:01 PM, Icarus Alive wrote: > Posted a question on Stackoverflow here: http://bit.ly/f3TIuo > but this isn't exactly a cross-post, rather an Erlang specific > extention of same :-) ... (to avoid any Erlang/Python religious wars, > although Stackoverflow is pretty good in preventing such wars, from > what I've seen). > > While my question there has a more generic question on which way I > should be heading, i.e. Erlang or Python, I admit that my limited > experience with Functional programming, and almost 12yrs history of > doing OO & Procedural programming in imperative languages, my > familiarity with Python is slightly higher, though love elegance of > Erlang. The challenge is in understanding slightly more involved / > complex constructs of Erlang. Anyhow, my query here to seek > community's view on what might be the ideal route using Erlang/OTP and > Erlang based Web-frameworks, given the following requirements. > > Here are my requirements -- > > 1. Web2.0 (i.e. interactive AJAXy) service, but with a parallel access > via mobile web as well. > 2. Application will need to do things like initiate outbound > voice-calls, send/receive SMS (think Twilio) etc. > 3. Application will make lot of short .flv / .3gp videos (think vimio > mini), which are not mass-appeal videos (e.g. baby's 1st walk video > shared with grandparents, so 1 uploader, at max 3-4 diff viewers and > 3-4 views in all, so CDN is pointless -- I think). > 4. There will be some video/audio transcoding (mostly ffmpeg type) > 5. Application will need to show trendlines and some charts etc. > (think popularity alerts / ranks) > 6. Overall, should be easy to learn, easy to maintain, has good > community support -- documentation / tutorials / screencasts & > active-development > 7. While performance is not the top-most item on my priority-list, it > is definitely a parameter, including memory-footprint, as I plan to > run this on cloud-infra, and every MB counts. > 8. While many people tell me to think of scaling later, but I think if > one can spend a little time thinking about it early one, what's the > harm. Later changes are costly. > > The frameworks & key-components that I am considering after reading > various comparisons are:- > > Option-1:: > 1) Backend comprising of : > a) RESTful App-engine: > (i) Webmachine (communicating data w/ FE via JSON), or > (ii) Nitrogen > b) Database engine: > (i) some distributed/scalable/highly-reliable (cloud-friendly) > database engine (??), or > (ii) VoltDB (though it's main memory requirement, 64-bit > requirement and official support for CentOS5.4 only, is a bit > worrying) > 2) Frontend based on: jQuery (for regular internet thick-client users) > / jQuery-mobile (for mobile-web users) > > That choice is driven by need to push rendering / non core-IP > computation to the browser as much as possible & lighten the load on > server, although I wonder how well it would work for the mobile-web > users! > > PS> Completely understand that the above is probably a > much-too-complex for an Erlang (& web-dev) newbie to target. But, > that's the ultimate target, I am happy to take the right approach, > right tools, right frameworks and keep taking baby-steps towards my > goal. > > Some advice / guidance on what choices & why, I could make. > > TIA, > ~i++ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From alain.odea@REDACTED Mon Feb 7 16:05:56 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Mon, 7 Feb 2011 11:35:56 -0330 Subject: [erlang-questions] web framework, db, architecture query, with requirements In-Reply-To: References: Message-ID: On Mon, Feb 7, 2011 at 10:31 AM, Icarus Alive wrote: > Posted a question on Stackoverflow here: http://bit.ly/f3TIuo > but this isn't exactly a cross-post, rather an Erlang specific > extention of same :-) ... (to avoid any Erlang/Python religious wars, > although Stackoverflow is pretty good in preventing such wars, from > what I've seen). > > While my question there has a more generic question on which way I > should be heading, i.e. Erlang or Python, I admit that my limited > experience with Functional programming, and almost 12yrs history of > doing OO & Procedural programming in imperative languages, my > familiarity with Python is slightly higher, though love elegance of > Erlang. The challenge is in understanding slightly more involved / > complex constructs of Erlang. Anyhow, my query here to seek > community's view on what might be the ideal route using Erlang/OTP and > Erlang based Web-frameworks, given the following requirements. > > Here are my requirements -- > > 1. Web2.0 (i.e. interactive AJAXy) service, but with a parallel access > via mobile web as well. > 2. Application will need to do things like initiate outbound > voice-calls, send/receive SMS (think Twilio) etc. > 3. Application will make lot of short .flv / .3gp videos (think vimio > mini), which are not mass-appeal videos (e.g. baby's 1st walk video > shared with grandparents, so 1 uploader, at max 3-4 diff viewers and > 3-4 views in all, so CDN is pointless -- I think). > 4. There will be some video/audio transcoding (mostly ffmpeg type) > 5. Application will need to show trendlines and some charts etc. > (think popularity alerts / ranks) > 6. Overall, should be easy to learn, easy to maintain, has good > community support -- documentation / tutorials / screencasts & > active-development > 7. While performance is not the top-most item on my priority-list, it > is definitely a parameter, including memory-footprint, as I plan to > run this on cloud-infra, and every MB counts. > 8. While many people tell me to think of scaling later, but I think if > one can spend a little time thinking about it early one, what's the > harm. Later changes are costly. > > The frameworks & key-components that I am considering after reading > various comparisons are:- > > Option-1:: > 1) Backend comprising of : > ? ?a) RESTful App-engine: > ? ? ? ?(i) Webmachine (communicating data w/ FE via JSON), or > ? ? ? ?(ii) Nitrogen > ? ?b) Database engine: > ? ? ? ?(i) some distributed/scalable/highly-reliable (cloud-friendly) > database engine (??), or > ? ? ? ?(ii) VoltDB (though it's main memory requirement, 64-bit > requirement and official support for CentOS5.4 only, is a bit > worrying) > 2) Frontend based on: jQuery (for regular internet thick-client users) > / jQuery-mobile (for mobile-web users) > > That choice is driven by need to push rendering / non core-IP > computation to the browser as much as possible & lighten the load on > server, although I wonder how well it would work for the mobile-web > users! > > PS> Completely understand that the above is probably a > much-too-complex for an Erlang (& web-dev) newbie to target. But, > that's the ultimate target, I am happy to take the right approach, > right tools, right frameworks and keep taking baby-steps towards my > goal. > > Some advice / guidance on what choices & why, I could make. > > TIA, > ~i++ Zotonic, the Erlang CMS at http://zotonic.com/ is a proven solution for building fast AJAX/comet-driven websites. A big plus for me is that the resulting sites are easy for non-technical people to maintain once you hand it off. It's lightning fast, easy to build on and easy to deeply extend. It's AJAX support is entirely JQuery-driven which fits your bill and makes it generally easier to work with and extend. It has excellent transparent Comet/Websockets support and Michael Connors has built a chat client using that: http://michaelconnors.net/article/367/mod-chat In terms of scalability the team is working on Elastic Zotonic which will support distribution of a site and content across multiple servers. Unless you have outrageous amounts of visitors you won't need this. Zotonic makes incredibly efficient use of system resources through in-memory caching of query results. See examples: http://zotonic.com/ http://totally-erlang.com/ http://www.timbenniks.nl/ http://scherpenisse.net/ http://pelleasselbergs.nl/ http://joreelsiegel.nl/ http://goudentonamsterdam.nl/ http://www.afhaal.nl/ http://miffy.whatwebwhat.com/ http://australie.timbenniks.nl/ http://verafin.com/ All of the above (except http://verafin.com/) are running on a single server. If you want an effective solution for building websites and web applications, then Zotonic is your secret weapon. From robert.virding@REDACTED Mon Feb 7 16:21:25 2011 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 7 Feb 2011 15:21:25 +0000 (GMT) Subject: [erlang-questions] Is this code tail recursive ? In-Reply-To: Message-ID: <552417436.406791297092078554.JavaMail.root@zimbra> Yes, it is tail-recursive in gather_files/1/2. Though it will probably not return what you want as the first clause of gather_files/2: gather_files([], [Acc]) -> Acc; will only succeed if there is ONE element (file) in the accumulator in which case it will return the file, otherwise it will crash. Did you intend gather_files([], Acc) -> Acc; ? Robert ----- "Mihai Balea" wrote: > On Feb 6, 2011, at 6:08 PM, Radu Brumariu wrote: > > > gather_files([]) -> []; > > gather_files(F) -> > > gather_files(F,[]). > > gather_files([],[Acc]) -> Acc; > > gather_files([{path,F}|Files],Acc) -> > > io:format("Processing path ~p~n",[F]), > > {ok, T} = file:read_file_info(F), > > case T#file_info.type > > of > > regular -> > > gather_files(Files, [F | Acc]); > > directory -> > > gather_files(Files, [gather_directory_files(F) | Acc ]); > > _ -> > > gather_files(Files, Acc) > > end. > > > > gather_directory_files(D) -> > > filelib:fold_files( D, ".*", true, fun(X,Acc) -> > > [X|Acc] > > end, > > []). > > > > Looks tail recursive to me, although looking at the way you build up > your result, it might not be what you expect :) > Not sure what you expect though, so I cannot comment on that. > > Hope this helps, > Mihai > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- Robert Virding, Erlang Solutions Ltd. From robert.virding@REDACTED Mon Feb 7 16:24:04 2011 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 7 Feb 2011 15:24:04 +0000 (GMT) Subject: [erlang-questions] Is this code tail recursive ? In-Reply-To: <1019214448.406881297092187077.JavaMail.root@zimbra> Message-ID: <417426161.406901297092244355.JavaMail.root@zimbra> Yes, it is tail-recursive in gather_files/1/2. Though it will probably not return what you want as the first clause of gather_files/2: gather_files([], [Acc]) -> Acc; will only succeed if there is ONE element (file) in the accumulator in which case it will return the file, otherwise it will crash. Did you intend gather_files([], Acc) -> Acc; ? Robert ----- "Mihai Balea" wrote: > On Feb 6, 2011, at 6:08 PM, Radu Brumariu wrote: > > > gather_files([]) -> []; > > gather_files(F) -> > > gather_files(F,[]). > > gather_files([],[Acc]) -> Acc; > > gather_files([{path,F}|Files],Acc) -> > > io:format("Processing path ~p~n",[F]), > > {ok, T} = file:read_file_info(F), > > case T#file_info.type > > of > > regular -> > > gather_files(Files, [F | Acc]); > > directory -> > > gather_files(Files, [gather_directory_files(F) | Acc ]); > > _ -> > > gather_files(Files, Acc) > > end. > > > > gather_directory_files(D) -> > > filelib:fold_files( D, ".*", true, fun(X,Acc) -> > > [X|Acc] > > end, > > []). > > > > Looks tail recursive to me, although looking at the way you build up > your result, it might not be what you expect :) > Not sure what you expect though, so I cannot comment on that. > > Hope this helps, > Mihai > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- Robert Virding, Erlang Solutions Ltd. From adam@REDACTED Mon Feb 7 16:26:19 2011 From: adam@REDACTED (Adam Lindberg) Date: Mon, 07 Feb 2011 16:26:19 +0100 Subject: [erlang-questions] Slave node restart not working In-Reply-To: References: <4D4BD87D.7050409@erlang-solutions.com> Message-ID: <4D500F1B.9080205@erlang-solutions.com> Is this to be reported as a bug in the slave module then? Cheers, Adam Andrey Pampukha wrote: > I've checked, slave doesn't want to work at my machine because of > absence of ssh-server, but ct_slave works: > ma@REDACTED)1> ct_slave:start(te). > {ok,te@REDACTED} > (ma@REDACTED)2> nodes(). > [te@REDACTED] > (ma@REDACTED)3> rpc:call(te@REDACTED,init,restart,[]). > ok > (ma@REDACTED)4> nodes(). > [] > (ma@REDACTED)5> net_adm:ping(te@REDACTED). > pong > (ma@REDACTED)6> nodes(). > [te@REDACTED] > > > 2011/2/4, Andrey Pampukha: >> Hi, >> >> Looks like nobody reconnects the slave node back to the master after >> restart. >> Try to ping it, I believe it should work. >> >> Andrey. >> >> 2011/2/4, Adam Lindberg: >>> Hi! >>> >>> I'm running a slave node in a project, and sometimes we want to restart >>> the slave node. However, it seems that init:restart() shuts down the >>> node instead of restarting in. >>> >>> The documentation for init:restart() says that it restarts the node in >>> the current emulator, which makes me wonder why it goes down instead of >>> restarts. The node should have everything it needs to restart based on >>> the boot flags etc. >>> >>> >>> (hest@REDACTED)5> slave:start(yama, apa). >>> {ok,apa@REDACTED} >>> (hest@REDACTED)6> nodes(). >>> [apa@REDACTED] >>> (hest@REDACTED)7> rpc:call('apa@REDACTED', init, restart, []). >>> ok >>> (hest@REDACTED)8> nodes(). >>> [] >>> >>> There is no crash dump generated as well. Why doesn't the slave node >>> restart? >>> >>> Cheers, >>> Adam >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> From erlang@REDACTED Mon Feb 7 16:27:36 2011 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 7 Feb 2011 16:27:36 +0100 Subject: [erlang-questions] Re: Erlang: Who uses it for games? In-Reply-To: <95f43ac3-f279-4990-b4eb-c90fef53192e@r27g2000yqb.googlegroups.com> References: <4C3490D3.1030108@gmail.com> <7DC4E578-DDDA-47BA-BE09-1733A209799E@cs.otago.ac.nz> <95f43ac3-f279-4990-b4eb-c90fef53192e@r27g2000yqb.googlegroups.com> Message-ID: On Thu, Jul 8, 2010 at 9:54 AM, ingo.schramm wrote: > Great! So many Erlang gamers out there! > > Somebody told me, a Battlestar Galactica browser game is backed up by > an Erlang engine. > > There's a article about this here: http://www.theregister.co.uk/2011/02/05/battlestar_galactica_mmp/ /Joe On Jul 8, 2:05 am, "Richard O'Keefe" wrote: > > On Jul 8, 2010, at 4:27 AM, Nicholas Frechette wrote: > > > > > hand optimize code which will require C++/assembly) and not so much > > > on the > > > PS3 where you may get a hard time using the SPUs from erlang (as I'm > > > guessing the VM won't run on them out of the box and you might have > > > to spend > > > considerable time to get erlang processes to run on SPUs (feasible I'm > > > sure)). > > > > It's not clear that it _is_ feasible in any interesting sense. > > The SPUs are essentially numeric vector engines. They have 128 > > registers, each 128 bits wide; loads are only 128 bits (16 bytes) > > and stores are only 128 bits (16 bytes). Memory is byte addressed > > and addresses are 32 bits, but the 2009 manual talks about 256 kB. > > > > One could imagine SPU _nodes_ doing vector crunching that Erlang > > processes running on the PPU communicated with as if Erlang. But > > Erlang would not be the language of choice for programming SPUs. > > (An APL dialect would be ideal, or Fortran 95.) > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > Seehttp://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From B.Candler@REDACTED Mon Feb 7 16:27:43 2011 From: B.Candler@REDACTED (Brian Candler) Date: Mon, 7 Feb 2011 15:27:43 +0000 Subject: web framework, db, architecture query, with requirements In-Reply-To: References: Message-ID: <20110207152743.GA3364@talktalkplc.com> On Mon, Feb 07, 2011 at 07:31:58PM +0530, Icarus Alive wrote: > (i) some distributed/scalable/highly-reliable (cloud-friendly) > database engine (??), or Have you had a look at CouchDB? It's written in erlang, which means you don't need another runtime environment. The API is HTTP, and you can serve static pages directly from it should you wish, and/or you can open it up for direct HTTP queries and updates using JSON, leaving the intelligence in the browser as you desire. It's the foundation for UbuntuOne's application synchronization capabilities too. People with a local CouchDB instance could even replicate their whole database onto their local machine, thus offloading from your servers entirely. Just a suggestion. Regards, Brian. From kaiduanx@REDACTED Mon Feb 7 16:33:45 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Mon, 7 Feb 2011 10:33:45 -0500 Subject: [erlang-questions] ssl_connection process hibernation option In-Reply-To: References: Message-ID: Ingela, May I have a suggestion to have the option to set the idle time for SSL connection so that after a mount of time of idle, the SSL connection will be put into hibernation? Thanks, /Kaiduan On Mon, Feb 7, 2011 at 3:32 AM, Ingela Andin wrote: > Hi! > > We (the OTP team) would consider it. > > Regards Ingela Erlang/OTP team - Ericsson AB > > > 2011/2/4 Jeroen Koops : >> Hi Ingela, >> Thanks for your response. Suppose I come up with a patch for this, do you >> think the OTP team would consider it for inclusion in an upcoming release? >> >> On Fri, Feb 4, 2011 at 4:51 PM, Ingela Andin wrote: >>> >>> Hi! >>> >>> We already put some effort into cleaning the state of the gen_fsm >>> processes holding the connection, >>> but some form of hibernation could be interesting too we will take >>> this into consideration. >>> >>> Regards Ingela Erlang/OTP team - Ericsson AB >>> >>> 2011/2/4 Jeroen Koops : >>> > I would very much appreciate this too. >>> > >>> > As a test, I inserted a proxy-module between the gen_fsm and >>> > ssl_connection >>> > callback, that changes all responses from the ssl_connection callback to >>> > include the 'hibernate' option (it's not a hack -- it's AOP!) and the >>> > savings are dramatic: before the change an SSL connection used around >>> > 350 >>> > kB, after the change only around 600 bytes. If a system is maintaining >>> > thousands of SSL connections, this really adds up. >>> > >>> > If we could somehow get hibernation built in to the ssl application, >>> > that >>> > would be great. >>> > >>> > On Fri, Feb 4, 2011 at 9:57 AM, Ferenc Holzhauser < >>> > ferenc.holzhauser@REDACTED> wrote: >>> > >>> >> Hi, >>> >> >>> >> I'm using R14B01 new SSL to set up a lot of (not very active) client >>> >> connections. >>> >> >>> >> I have noticed that ssl_connection processes use a lot of memory >>> >> between >>> >> GC's. >>> >> A forced GC makes them small again until the next piece of received >>> >> data. >>> >> >>> >> This adds up to a significant memory usage (around 350-400 kB/session >>> >> on a >>> >> x64 system in my case) limiting the number of possible sessions. >>> >> >>> >> An option to hibernate this gen_fsm would decrease the average memory >>> >> usage >>> >> in this scenario. >>> >> >>> >> Would it be possible to introduce this option? >>> >> >>> >> Thanks in advance, >>> >> Ferenc >>> >> >>> > >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From nirth.furzahad@REDACTED Mon Feb 7 16:57:44 2011 From: nirth.furzahad@REDACTED (David Sergey) Date: Mon, 7 Feb 2011 17:57:44 +0200 Subject: [erlang-questions] Re: Erlang: Who uses it for games? In-Reply-To: References: <4C3490D3.1030108@gmail.com> <7DC4E578-DDDA-47BA-BE09-1733A209799E@cs.otago.ac.nz> <95f43ac3-f279-4990-b4eb-c90fef53192e@r27g2000yqb.googlegroups.com> Message-ID: <273EE83D-04FC-4174-87CD-828915400616@gmail.com> I'm working to on Agent-Oriented AI for PS3 game. On 7 Feb 2011, at 17:27, Joe Armstrong wrote: > On Thu, Jul 8, 2010 at 9:54 AM, ingo.schramm wrote: > >> Great! So many Erlang gamers out there! >> >> Somebody told me, a Battlestar Galactica browser game is backed up by >> an Erlang engine. >> >> > There's a article about this here: > > http://www.theregister.co.uk/2011/02/05/battlestar_galactica_mmp/ > > > /Joe > > > On Jul 8, 2:05 am, "Richard O'Keefe" wrote: >>> On Jul 8, 2010, at 4:27 AM, Nicholas Frechette wrote: >>> >>>> hand optimize code which will require C++/assembly) and not so much >>>> on the >>>> PS3 where you may get a hard time using the SPUs from erlang (as I'm >>>> guessing the VM won't run on them out of the box and you might have >>>> to spend >>>> considerable time to get erlang processes to run on SPUs (feasible I'm >>>> sure)). >>> >>> It's not clear that it _is_ feasible in any interesting sense. >>> The SPUs are essentially numeric vector engines. They have 128 >>> registers, each 128 bits wide; loads are only 128 bits (16 bytes) >>> and stores are only 128 bits (16 bytes). Memory is byte addressed >>> and addresses are 32 bits, but the 2009 manual talks about 256 kB. >>> >>> One could imagine SPU _nodes_ doing vector crunching that Erlang >>> processes running on the PPU communicated with as if Erlang. But >>> Erlang would not be the language of choice for programming SPUs. >>> (An APL dialect would be ideal, or Fortran 95.) >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> Seehttp://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From icarus.alive@REDACTED Mon Feb 7 17:26:39 2011 From: icarus.alive@REDACTED (Icarus Alive) Date: Mon, 7 Feb 2011 21:56:39 +0530 Subject: [erlang-questions] web framework, db, architecture query, with requirements In-Reply-To: References: Message-ID: On Mon, Feb 7, 2011 at 8:11 PM, Alex Arnon wrote: > A possibly off-topic question: why VoltDB, and while we're there, why are > its requirements a problem? Good question. The worry (not so much of a problem) is completely non-technical :-). 64-bit requirement means that the AWS EC2 'Small Instance' can't be used as the starting point, which is a 32-bit instance. For how I intend to sell my ware eventually, this does impact my ability to offer a competitive / attractive price. > On Mon, Feb 7, 2011 at 4:01 PM, Icarus Alive wrote: >> >> Posted a question on Stackoverflow here: http://bit.ly/f3TIuo >> but this isn't exactly a cross-post, rather an Erlang specific >> extention of same :-) ... (to avoid any Erlang/Python religious wars, >> although Stackoverflow is pretty good in preventing such wars, from >> what I've seen). >> >> While my question there has a more generic question on which way I >> should be heading, i.e. Erlang or Python, I admit that my limited >> experience with Functional programming, and almost 12yrs history of >> doing OO & Procedural programming in imperative languages, my >> familiarity with Python is slightly higher, though love elegance of >> Erlang. The challenge is in understanding slightly more involved / >> complex constructs of Erlang. Anyhow, my query here to seek >> community's view on what might be the ideal route using Erlang/OTP and >> Erlang based Web-frameworks, given the following requirements. >> >> Here are my requirements -- >> >> 1. Web2.0 (i.e. interactive AJAXy) service, but with a parallel access >> via mobile web as well. >> 2. Application will need to do things like initiate outbound >> voice-calls, send/receive SMS (think Twilio) etc. >> 3. Application will make lot of short .flv / .3gp videos (think vimio >> mini), which are not mass-appeal videos (e.g. baby's 1st walk video >> shared with grandparents, so 1 uploader, at max 3-4 diff viewers and >> 3-4 views in all, so CDN is pointless -- I think). >> 4. There will be some video/audio transcoding (mostly ffmpeg type) >> 5. Application will need to show trendlines and some charts etc. >> (think popularity alerts / ranks) >> 6. Overall, should be easy to learn, easy to maintain, has good >> community support -- documentation / tutorials / screencasts & >> active-development >> 7. While performance is not the top-most item on my priority-list, it >> is definitely a parameter, including memory-footprint, as I plan to >> run this on cloud-infra, and every MB counts. >> 8. While many people tell me to think of scaling later, but I think if >> one can spend a little time thinking about it early one, what's the >> harm. Later changes are costly. >> >> The frameworks & key-components that I am considering after reading >> various comparisons are:- >> >> Option-1:: >> 1) Backend comprising of : >> ? ?a) RESTful App-engine: >> ? ? ? ?(i) Webmachine (communicating data w/ FE via JSON), or >> ? ? ? ?(ii) Nitrogen >> ? ?b) Database engine: >> ? ? ? ?(i) some distributed/scalable/highly-reliable (cloud-friendly) >> database engine (??), or >> ? ? ? ?(ii) VoltDB (though it's main memory requirement, 64-bit >> requirement and official support for CentOS5.4 only, is a bit >> worrying) >> 2) Frontend based on: jQuery (for regular internet thick-client users) >> / jQuery-mobile (for mobile-web users) >> >> That choice is driven by need to push rendering / non core-IP >> computation to the browser as much as possible & lighten the load on >> server, although I wonder how well it would work for the mobile-web >> users! >> >> PS> Completely understand that the above is probably a >> much-too-complex for an Erlang (& web-dev) newbie to target. But, >> that's the ultimate target, I am happy to take the right approach, >> right tools, right frameworks and keep taking baby-steps towards my >> goal. >> >> Some advice / guidance on what choices & why, I could make. >> >> TIA, >> ~i++ From vinoski@REDACTED Mon Feb 7 17:53:13 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Mon, 7 Feb 2011 11:53:13 -0500 Subject: [erlang-questions] YAWS RESTful examples In-Reply-To: References: <13448848.82521290583867809.JavaMail.www@wsfrf1116> Message-ID: On Sun, Feb 6, 2011 at 10:32 AM, Alain O'Dea wrote: > Hi David: > > I totally agree with Lukas. ?I could bot make heads or tails of REST on YAWS. ?It seemed like a square peg in a round hole. No offense, but I don't think this statement makes much sense because it implies that writing RESTful applications with Yaws is impossible, which most definitely is not the case: http://www.infoq.com/articles/vinoski-erlang-rest Yaws is an extensible web server with which you can write server-side web applications. It provides complete information about every HTTP request to your application, allowing your app to react to each request appropriately, and allows your app to completely control response headers and return any form of content it wishes to. Thus, writing a RESTful app on Yaws is entirely a matter of structuring and writing the app correctly. The means by which you access request details and provide responses with Yaws are fairly well documented at http://yaws.hyber.org/ and if you can't figure something out, send a message to the yaws mailing list: https://lists.sourceforge.net/lists/listinfo/erlyaws-list Last year Justin Sheehy and I wrote about Webmachine here (PDF): http://steve.vinoski.net/pdf/IC-Developing_RESTful_Web_Services_with_Webmachine.pdf Note that Webmachine is itself a framework that sits on Mochiweb. I'm sure it could run on Yaws equally well. The nice thing that Webmachine does for you is encode the rules of the HTTP application protocol, thereby making it hard for your app to get those wrong, but that by itself is still not enough to guarantee RESTfulness. Only the application itself, regardless of underlying framework or web server, can determine RESTfulness. For example, one of the most important constraints of the REST architectural style is "hypermedia as the engine of application state," which means for each request, the server uses hypermedia to direct each client application (user agent) to possible next steps given what it just requested. Many apps claiming to be RESTful fail miserably on this critical constraint, and thus are not actually RESTful, because they don't use a standard media type that supports links (or use Link headers) -- they instead use custom media types based on JSON or XML that require the client to be specifically coupled to the server so it can interpret the custom media type. In my InfoQ article linked above from 2008 I failed to give this critical constraint the focus it requires. --steve From paolo.negri@REDACTED Mon Feb 7 18:07:23 2011 From: paolo.negri@REDACTED (Paolo Negri) Date: Mon, 7 Feb 2011 18:07:23 +0100 Subject: global module limits Message-ID: Dear list I've been reading various opinions about the global module and I'd like some light to be shed on its limits and their underlying reasons. Particularly I'd like to understand what are its limit relative to register a big number of processes. How do the following factors affect the global module performance - number of registered processes at a given time, is this a problem if the registered processes are in the order of 1K 10K or 100K - registration rate, what if I register 10 100 or 1000 processes per seconds - lookup rate, this is how often the global name registry is queried - de-registration rate, symmetrical to the question above, 10 100 or 1000 registered processes terminate per second - overall number of registered processes over time: assuming that in a given second processes are registered at the same rate at which old registered processes terminate, the number of currently registered processes is constant (say 10K) but overtime I might have registered 1000K different process names, does the "register history" get cleaned or it'll stick around in the global environment? - cluster replication overhead, how does the number of nodes in a system affect the performance of the global registration facility? On a side note I would be very interested in people willing to share experiences in using in production the gproc[1] project using it's global facility to handle in the region of 100K to 1000K processes. [1] https://github.com/esl/gproc Thanks in advance for your answers. Paolo From alain.odea@REDACTED Mon Feb 7 18:14:26 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Mon, 7 Feb 2011 13:44:26 -0330 Subject: [erlang-questions] YAWS RESTful examples In-Reply-To: References: <13448848.82521290583867809.JavaMail.www@wsfrf1116> Message-ID: On 2011-02-07, at 13:23, Steve Vinoski wrote: > On Sun, Feb 6, 2011 at 10:32 AM, Alain O'Dea wrote: >> Hi David: >> >> I totally agree with Lukas. I could bot make heads or tails of REST on YAWS. It seemed like a square peg in a round hole. > > No offense, but I don't think this statement makes much sense because > it implies that writing RESTful applications with Yaws is impossible, > which most definitely is not the case: > > http://www.infoq.com/articles/vinoski-erlang-rest Wait a second, I was not claiming that nor did I intend to imply that YAWS is incapable of REST. I found YAWS required more conditional code for HTTP lifecycle issues that Webmachine encapsulates. I found YAWS to be awkward for building RESTful interfaces. > The nice thing that Webmachine does for you is encode the rules of the > HTTP application protocol, thereby making it hard for your app to get > those wrong, but that by itself is still not enough to guarantee > RESTfulness. Only the application itself, regardless of underlying > framework or web server, can determine RESTfulness. It does help a lot though :) > For example, one of the most important constraints of the REST > architectural style is "hypermedia as the engine of application > state," which means for each request, the server uses hypermedia to > direct each client application (user agent) to possible next steps > given what it just requested. Many apps claiming to be RESTful fail > miserably on this critical constraint, and thus are not actually > RESTful, because they don't use a standard media type that supports > links (or use Link headers) -- they instead use custom media types > based on JSON or XML that require the client to be specifically > coupled to the server so it can interpret the custom media type. In my > InfoQ article linked above from 2008 I failed to give this critical > constraint the focus it requires. The HATEOAS point is hugely important. I have failed at this in Scrumjet and it definitely leads to coupling. From marcel.meyer@REDACTED Mon Feb 7 19:29:23 2011 From: marcel.meyer@REDACTED (Marcel Meyer) Date: Mon, 7 Feb 2011 13:29:23 -0500 Subject: Concurrency Orientated Design question Message-ID: Hi there, Let's say I have a hierarchical data resource, like a directory structure. I have a server that expose typical functions on such a tree, like "get me the children of this node", and typical traffic constitutes of a lot more reads than updates. Given I have cheap processes, can I compose the processing like this? (Analysis of the kinds of operations I can do on these trees show that operations complete within microseconds) Scenario 1: Spawn a process per tree and route all requests to that process for processing. The good: Since this is essentially a synchronous processor, no race conditions. The bad: Its synchronous. Scenario 2: Spawn an "owner" tree processor, and on all "read-like" operations, spawn a new process to deal with that request. The good: Better scaling The bad: Spawning a new process with the existing state might be expensive (ito memory and time), as these trees could be large (4Mb max, usually < 200kb). Updating the tree will become hairy, unless updates are only done by the owner, like when you spawn an ets table as private. I hope the general idea of the problem is clear: do I spawn new processes for each request (and then tear it down afterwords)? Kind regards, Marcel From icarus.alive@REDACTED Mon Feb 7 19:43:55 2011 From: icarus.alive@REDACTED (Icarus Alive) Date: Tue, 8 Feb 2011 00:13:55 +0530 Subject: [erlang-questions] web framework, db, architecture query, with requirements In-Reply-To: References: Message-ID: On Mon, Feb 7, 2011 at 8:35 PM, Alain O'Dea wrote: > On Mon, Feb 7, 2011 at 10:31 AM, Icarus Alive wrote: >> Posted a question on Stackoverflow here: http://bit.ly/f3TIuo >> but this isn't exactly a cross-post, rather an Erlang specific >> extention of same :-) ... (to avoid any Erlang/Python religious wars, >> although Stackoverflow is pretty good in preventing such wars, from >> what I've seen). > Zotonic, the Erlang CMS at http://zotonic.com/ is a proven solution > for building fast AJAX/comet-driven websites. Thanks for the pointer to Zotonic. The feature set is quite impressive, however my understanding is that CMS is good where content needs to be seen by lots and lots of people, and you have lots of content to organize, make it searchable, classifiable etc. Just to clarify my requirement a bit further, this is for a rather niche service, where small groups of people have access to a fairly limited set of content, and that content changes quite frequently, i.e. content older than few days gets purged. And there are possibly hundreds of thousands of such small groups. Would you say a CMS is good for such application ? > ?A big plus for me is > that the resulting sites are easy for non-technical people to maintain > once you hand it off. That is indeed a good thing to have, though for my purpose I am still not quite sure how it will be used. > It's lightning fast, easy to build on and easy to deeply extend. ?It's > AJAX support is entirely JQuery-driven which fits your bill and makes > it generally easier to work with and extend. ?It has excellent > transparent Comet/Websockets support and Michael Connors has built a > chat client using that: > http://michaelconnors.net/article/367/mod-chat That is all very good, and looks quite promising, but for a newbie like me, the tutorials / samples section seems pretty sparse. > In terms of scalability the team is working on Elastic Zotonic which > will support distribution of a site and content across multiple > servers. ?Unless you have outrageous amounts of visitors you won't > need this. ?Zotonic makes incredibly efficient use of system resources > through in-memory caching of query results. > > See examples: > http://zotonic.com/ > http://totally-erlang.com/ > http://www.timbenniks.nl/ > http://scherpenisse.net/ > http://pelleasselbergs.nl/ > http://joreelsiegel.nl/ > http://goudentonamsterdam.nl/ > http://www.afhaal.nl/ > http://miffy.whatwebwhat.com/ > http://australie.timbenniks.nl/ > http://verafin.com/ > > All of the above (except http://verafin.com/) are running on a single server. > > If you want an effective solution for building websites and web > applications, then Zotonic is your secret weapon. > Once again, thank you for the response. From alain.odea@REDACTED Mon Feb 7 19:55:35 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Mon, 7 Feb 2011 15:25:35 -0330 Subject: [erlang-questions] web framework, db, architecture query, with requirements In-Reply-To: References: Message-ID: On Mon, Feb 7, 2011 at 3:13 PM, Icarus Alive wrote: > On Mon, Feb 7, 2011 at 8:35 PM, Alain O'Dea wrote: >> On Mon, Feb 7, 2011 at 10:31 AM, Icarus Alive wrote: >>> Posted a question on Stackoverflow here: http://bit.ly/f3TIuo >>> but this isn't exactly a cross-post, rather an Erlang specific >>> extention of same :-) ... (to avoid any Erlang/Python religious wars, >>> although Stackoverflow is pretty good in preventing such wars, from >>> what I've seen). > > > >> Zotonic, the Erlang CMS at http://zotonic.com/ is a proven solution >> for building fast AJAX/comet-driven websites. > > Thanks for the pointer to Zotonic. The feature set is quite > impressive, however my understanding is that CMS is good where content > needs to be seen by lots and lots of people, and you have lots of > content to organize, make it searchable, classifiable etc. Just to > clarify my requirement a bit further, this is for a rather niche > service, where small groups of people have access to a fairly limited > set of content, and that content changes quite frequently, i.e. > content older than few days gets purged. And there are possibly > hundreds of thousands of such small groups. Would you say a CMS is > good for such application ? > > >> ?A big plus for me is >> that the resulting sites are easy for non-technical people to maintain >> once you hand it off. > > That is indeed a good thing to have, though for my purpose I am still > not quite sure how it will be used. > >> It's lightning fast, easy to build on and easy to deeply extend. ?It's >> AJAX support is entirely JQuery-driven which fits your bill and makes >> it generally easier to work with and extend. ?It has excellent >> transparent Comet/Websockets support and Michael Connors has built a >> chat client using that: >> http://michaelconnors.net/article/367/mod-chat > > That is all very good, and looks quite promising, but for a newbie > like me, the tutorials / samples section seems pretty sparse. > >> In terms of scalability the team is working on Elastic Zotonic which >> will support distribution of a site and content across multiple >> servers. ?Unless you have outrageous amounts of visitors you won't >> need this. ?Zotonic makes incredibly efficient use of system resources >> through in-memory caching of query results. >> >> See examples: >> http://zotonic.com/ >> http://totally-erlang.com/ >> http://www.timbenniks.nl/ >> http://scherpenisse.net/ >> http://pelleasselbergs.nl/ >> http://joreelsiegel.nl/ >> http://goudentonamsterdam.nl/ >> http://www.afhaal.nl/ >> http://miffy.whatwebwhat.com/ >> http://australie.timbenniks.nl/ >> http://verafin.com/ >> >> All of the above (except http://verafin.com/) are running on a single server. >> >> If you want an effective solution for building websites and web >> applications, then Zotonic is your secret weapon. >> > > Once again, thank you for the response. > Zotonic has been used to manage hundreds of thousands of pages, but I'm not sure how you would handle that many users from a configuration perspective. I have copied this thread to Zotonic-Users on Google Groups to see what the Zotonic community thinks. Given your use case, you may be better building a lot directly with Nitrogen. Zotonic is built primarily on Nitrogen and Webmachine so in the worst case you can take advantage of its code base as an example of how to leverage them. Documentation is admittedly a big gap for Zotonic. Lloyd R. Prentice created a FLOSS manual for it with some shell content and I have been working with him to populate it with tutorials and guides. From alain.odea@REDACTED Mon Feb 7 19:57:40 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Mon, 7 Feb 2011 15:27:40 -0330 Subject: [erlang-questions] web framework, db, architecture query, with requirements In-Reply-To: References: Message-ID: On Mon, Feb 7, 2011 at 3:25 PM, Alain O'Dea wrote: > On Mon, Feb 7, 2011 at 3:13 PM, Icarus Alive wrote: >> On Mon, Feb 7, 2011 at 8:35 PM, Alain O'Dea wrote: >>> On Mon, Feb 7, 2011 at 10:31 AM, Icarus Alive wrote: >>>> Posted a question on Stackoverflow here: http://bit.ly/f3TIuo >>>> but this isn't exactly a cross-post, rather an Erlang specific >>>> extention of same :-) ... (to avoid any Erlang/Python religious wars, >>>> although Stackoverflow is pretty good in preventing such wars, from >>>> what I've seen). >> >> >> >>> Zotonic, the Erlang CMS at http://zotonic.com/ is a proven solution >>> for building fast AJAX/comet-driven websites. >> >> Thanks for the pointer to Zotonic. The feature set is quite >> impressive, however my understanding is that CMS is good where content >> needs to be seen by lots and lots of people, and you have lots of >> content to organize, make it searchable, classifiable etc. Just to >> clarify my requirement a bit further, this is for a rather niche >> service, where small groups of people have access to a fairly limited >> set of content, and that content changes quite frequently, i.e. >> content older than few days gets purged. And there are possibly >> hundreds of thousands of such small groups. Would you say a CMS is >> good for such application ? >> >> >>> ?A big plus for me is >>> that the resulting sites are easy for non-technical people to maintain >>> once you hand it off. >> >> That is indeed a good thing to have, though for my purpose I am still >> not quite sure how it will be used. >> >>> It's lightning fast, easy to build on and easy to deeply extend. ?It's >>> AJAX support is entirely JQuery-driven which fits your bill and makes >>> it generally easier to work with and extend. ?It has excellent >>> transparent Comet/Websockets support and Michael Connors has built a >>> chat client using that: >>> http://michaelconnors.net/article/367/mod-chat >> >> That is all very good, and looks quite promising, but for a newbie >> like me, the tutorials / samples section seems pretty sparse. >> >>> In terms of scalability the team is working on Elastic Zotonic which >>> will support distribution of a site and content across multiple >>> servers. ?Unless you have outrageous amounts of visitors you won't >>> need this. ?Zotonic makes incredibly efficient use of system resources >>> through in-memory caching of query results. >>> >>> See examples: >>> http://zotonic.com/ >>> http://totally-erlang.com/ >>> http://www.timbenniks.nl/ >>> http://scherpenisse.net/ >>> http://pelleasselbergs.nl/ >>> http://joreelsiegel.nl/ >>> http://goudentonamsterdam.nl/ >>> http://www.afhaal.nl/ >>> http://miffy.whatwebwhat.com/ >>> http://australie.timbenniks.nl/ >>> http://verafin.com/ >>> >>> All of the above (except http://verafin.com/) are running on a single server. >>> >>> If you want an effective solution for building websites and web >>> applications, then Zotonic is your secret weapon. >>> >> >> Once again, thank you for the response. >> > > Zotonic has been used to manage hundreds of thousands of pages, but > I'm not sure how you would handle that many users from a configuration > perspective. ?I have copied this thread to Zotonic-Users on Google > Groups to see what the Zotonic community thinks. CORRECTION: I did not intend to send the statement about hundreds of thousands of pages. I have no source for it yet. I remember reading it on zotonic-users but I cannot find that. Disregard that until I find the source :) Thanks. > > Given your use case, you may be better building a lot directly with > Nitrogen. ?Zotonic is built primarily on Nitrogen and Webmachine so in > the worst case you can take advantage of its code base as an example > of how to leverage them. > > Documentation is admittedly a big gap for Zotonic. ?Lloyd R. Prentice > created a FLOSS manual for it with some shell content and I have been > working with him to populate it with tutorials and guides. > From jesper.louis.andersen@REDACTED Mon Feb 7 20:13:49 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 7 Feb 2011 20:13:49 +0100 Subject: [erlang-questions] Concurrency Orientated Design question In-Reply-To: References: Message-ID: On Mon, Feb 7, 2011 at 19:29, Marcel Meyer wrote: > Scenario 2: > Spawn an "owner" tree processor, and on all "read-like" operations, spawn a > new process to deal with that request. > The good: > Better scaling > The bad: > Spawning a new process with the existing state might be expensive (ito > memory and time), as these trees could be large (4Mb max, usually < 200kb). > Updating the tree will become hairy, unless updates are only done by the > owner, like when you spawn an ets table as private. Scenario 3: The tree nodes are stored in ETS. Otherwise it is like scenario 2. Scenario 4: Each tree node is a process :) -- J. From manprog@REDACTED Mon Feb 7 20:41:07 2011 From: manprog@REDACTED (Bernardo Alvez) Date: Mon, 07 Feb 2011 17:41:07 -0200 Subject: [off-topic] MWC GSMA 2011 Message-ID: <4D504AD3.2010008@adinet.com.uy> Hi All, Are there any companies using Erlang attending MWC Barcelona 2011? Regards, Bernardo From marcel.meyer@REDACTED Mon Feb 7 22:20:25 2011 From: marcel.meyer@REDACTED (Marcel Meyer) Date: Mon, 7 Feb 2011 16:20:25 -0500 Subject: [erlang-questions] Concurrency Orientated Design question In-Reply-To: References: Message-ID: Haha, I was actually wondering if each node could be a process, but some trees have 100,000's of nodes, so that didn't seem right? By scenario 3 do you mean to store the hierarchical data as a flat list? I've done some analysis on this, and even with graph optimization like pre-ordered tree traversal, traversing the actual tree was still faster. There are some business scenarios for example where access has been removed for a user on a node, and access to lower down nodes have to be inferred, so having it as a proper graph is handy. On Mon, Feb 7, 2011 at 2:13 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Mon, Feb 7, 2011 at 19:29, Marcel Meyer wrote: > > > Scenario 2: > > Spawn an "owner" tree processor, and on all "read-like" operations, spawn > a > > new process to deal with that request. > > The good: > > Better scaling > > The bad: > > Spawning a new process with the existing state might be expensive (ito > > memory and time), as these trees could be large (4Mb max, usually < > 200kb). > > Updating the tree will become hairy, unless updates are only done by the > > owner, like when you spawn an ets table as private. > > Scenario 3: > > The tree nodes are stored in ETS. Otherwise it is like scenario 2. > > Scenario 4: > > Each tree node is a process :) > -- > J. > From ok@REDACTED Mon Feb 7 23:16:43 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 8 Feb 2011 11:16:43 +1300 Subject: [erlang-questions] Concurrency Orientated Design question In-Reply-To: References: Message-ID: <7A355D38-11DA-4A19-AC8F-F6723330272B@cs.otago.ac.nz> On 8/02/2011, at 10:20 AM, Marcel Meyer wrote: > Haha, I was actually wondering if each node could be a process, but some > trees have 100,000's of nodes, so that didn't seem right? What's the minimum memory for a process? 1> F = fun () -> receive ok -> ok end end. 2> {_,{memory,Bytes}} = process_info(spawn(F), memory). 3> Bytes div erlang:system_info(wordsize). -- From the efficiency guide, with a minor but important correction -- in line 2. The guide says 309; one of my machines says 317; another says 330. Whether a process per node makes sense depends on how much data you have for each node. If you have a kilobyte or more, then using a process makes sense. If you have just a few bytes, it probably doesn't. From jesper.louis.andersen@REDACTED Mon Feb 7 23:36:54 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 7 Feb 2011 23:36:54 +0100 Subject: [erlang-questions] Concurrency Orientated Design question In-Reply-To: References: Message-ID: On Mon, Feb 7, 2011 at 22:20, Marcel Meyer wrote: > Haha, I was actually wondering if each node could be a process, but some > trees have 100,000's of nodes, so that didn't seem right? Think in B-trees. Each node has a "page" of depth 8, say and up to 256 Pid links to the next nodes in the tree. This divides down the number of active processes you need but also increases the price of updating slightly. I've never tried to build something like this by the way. > By scenario 3 do you mean to store the?hierarchical?data as a flat list? Yes and no. I meant to store a node ini ETS as: {Identifier, NodeData} where Identifier is something make_ref() returns to me. And NodeData is a list [{file, FileMeta} | {deferred_directory, Identifier}]. The price is that you are copying these to the process doing lookups and this of course has a price, especially since we throw out data right afterwards. But the advantage is that this has really good parallel access properties. Again, you can toy with the idea of storing larger chunks of data as terms at the expense of more costly updates. The idea is that we have the deferred_directory as a continuation for our data, should we wish to walk further down that path in the hierarchy, but we don't pay up front for the price of loading it. Essentially it is a lazy load. > I've done some analysis on this, and even with graph optimization like > pre-ordered tree traversal, traversing the actual tree was still faster. > There are some business scenarios for example where access has been removed > for a user on a node, and access to lower down nodes have to be?inferred, so > having it as a?proper?graph is handy. I just wanted to make sure you had considered these options as well. Personally, I'd probably try the paging model. -- J. From oribrost@REDACTED Tue Feb 8 00:47:57 2011 From: oribrost@REDACTED (ori brost) Date: Mon, 7 Feb 2011 15:47:57 -0800 (PST) Subject: DNS is slow when run from many processes Message-ID: <1e70a214-84a6-4456-8f04-a90fb38a16a0@d2g2000yqn.googlegroups.com> I've written a small program to demonstrate this: ---BEGIN PROGRAM--- -module(ghbm_bug_test). -compile(export_all). run(PortCount,Url) -> run(2000,PortCount,Url). run(Port,PortCount,Url) -> case Port < PortCount of true -> spawn(fun() -> loop(Port,Url) end), run(Port + 1,PortCount,Url); false -> ok end. loop(Port,Url) -> case Port of 2003 -> io:format("Doing connect\n"); _ -> ok end, Sock = gen_tcp:connect(Url, 8080, [], 5000), case Port of 2003 -> io:format("Did connect\n"); _ -> ok end, case (catch gen_tcp:close(Sock)) of _ -> ok end, loop(Port,Url). ---END PROGRAM--- This program takes a lot of time to connect when connecting via run(50000,"127.0.0.1") and yet connects quickly when I do run(50000, {127,0,0,1}). I am running it with a max number of processes of 1000000 (set via +P). After checking the states of processes I see that many of them spend time in: 3> i(0,200,0). [{current_function,{inet_gethost_native,getit,2}}, What are my possibilities for better DNS? I now that I can use erlang dns instead of native dns, this solves the problem for 127.0.0.1, but when I try a real address (i.e. run(50000,"some.server.of.mine.com")) connections are very slow with both native and erlang DNS. Any advice on a solution? ` From oribrost@REDACTED Tue Feb 8 00:50:34 2011 From: oribrost@REDACTED (ori brost) Date: Tue, 8 Feb 2011 01:50:34 +0200 Subject: DNS is slow when run from many processes Message-ID: I've written a small program to demonstrate this: ---BEGIN PROGRAM--- -module(ghbm_bug_test). -compile(export_all). run(PortCount,Url) -> run(2000,PortCount,Url). run(Port,PortCount,Url) -> case Port < PortCount of true -> spawn(fun() -> loop(Port,Url) end), run(Port + 1,PortCount,Url); false -> ok end. loop(Port,Url) -> case Port of 2003 -> io:format("Doing connect\n"); _ -> ok end, Sock = gen_tcp:connect(Url, 8080, [], 5000), case Port of 2003 -> io:format("Did connect\n"); _ -> ok end, case (catch gen_tcp:close(Sock)) of _ -> ok end, loop(Port,Url). ---END PROGRAM--- This program takes a lot of time to connect when connecting via run(50000,"127.0.0.1") and yet connects quickly when I do run(50000, {127,0,0,1}). I am running it with a max number of processes of 1000000 (set via +P). After checking the states of processes I see that many of them spend time in: 3> i(0,200,0). [{current_function,{inet_gethost_native,getit,2}}, What are my possibilities for better DNS? I now that I can use erlang dns instead of native dns, this solves the problem for 127.0.0.1, but when I try a real address (i.e. run(50000,"some.server.of.mine.com")) connections are very slow with both native and erlang DNS. Any advice on a solution? From axling@REDACTED Tue Feb 8 00:54:27 2011 From: axling@REDACTED (Erik Axling) Date: Tue, 8 Feb 2011 00:54:27 +0100 Subject: [erlang-questions] Destructor called twice on NIF resource Message-ID: Hello! I've been creating NIFs for a C-library and when I'm starting to run EUnit tests on them it fails with a seg fault. It seems that the destructor for a resource is called twice which leads to the error as the pointer has already been freed. I do not unserdtand why this happens though If I run the code myself in a shell it doesn't fail but then I am not executing it inside of functions which, I guess, gets another behaviour for the GC. But if I compile the eunit test suite file and run the individual functions myself it also fails but with a different seg fault this time: fann_nif.c: https://gist.github.com/815506 fannerl_test.erl: https://gist.github.com/815543 First fault: $ ./rebar eunit GNU gdb (GDB) 7.2-ubuntu Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i686-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/local/lib/erlang/erts-5.8.1/bin/erlexec...done. (gdb) r Starting program: /usr/local/lib/erlang/erts-5.8.1/bin/erlexec +B -boot start_clean -noshell -noshell -noinput -run escript start -extra ./rebar eunit process 3188 is executing new program: /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp [Thread debugging using libthread_db enabled] [New Thread 0xb7d03b70 (LWP 3191)] [New Thread 0xb7ffdb70 (LWP 3192)] [New Thread 0xb7502b70 (LWP 3193)] [New Thread 0xb7ff4b70 (LWP 3194)] [New Thread 0xb6d01b70 (LWP 3195)] [New Thread 0xb6500b70 (LWP 3196)] ==> fannerl (eunit) test/fannerl_tests.erl:8: exiting create_test Destroy the fann pointer test/fannerl_tests.erl:13: exiting get_mse_test test/fannerl_tests.erl:19: before last get test/fannerl_tests.erl:21: after last get testing to see if we get here Destroy the fann pointer Destroy the fann pointer *** glibc detected *** /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp: free(): invalid pointer: 0x082da1d0 *** ======= Backtrace: ========= /lib/libc.so.6(+0x6c501)[0x225501] /lib/libc.so.6(+0x6dd70)[0x226d70] /lib/libc.so.6(cfree+0x6d)[0x229e5d] /usr/local/lib/libfann.so.2(fann_destroy+0x43)[0x493343] /home/dude/projects/fannerl/.eunit/../priv/fannerl_nif.so(+0x27cd)[0x4857cd] /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp[0x814397a] ======= Memory map: ======== 00110000-0012c000 r-xp 00000000 08:01 8920592 /lib/ld-2.12.1.so 0012c000-0012d000 r--p 0001b000 08:01 8920592 /lib/ld-2.12.1.so 0012d000-0012e000 rw-p 0001c000 08:01 8920592 /lib/ld-2.12.1.so 0012e000-0012f000 r-xp 00000000 00:00 0 [vdso] 0012f000-00131000 r-xp 00000000 08:01 8920620 /lib/libutil-2.12.1.so 00131000-00132000 r--p 00001000 08:01 8920620 /lib/libutil-2.12.1.so 00132000-00133000 rw-p 00002000 08:01 8920620 /lib/libutil-2.12.1.so 00133000-00135000 r-xp 00000000 08:01 8920604 /lib/libdl-2.12.1.so 00135000-00136000 r--p 00001000 08:01 8920604 /lib/libdl-2.12.1.so 00136000-00137000 rw-p 00002000 08:01 8920604 /lib/libdl-2.12.1.so 00137000-0015b000 r-xp 00000000 08:01 8920605 /lib/libm-2.12.1.so 0015b000-0015c000 r--p 00023000 08:01 8920605 /lib/libm-2.12.1.so 0015c000-0015d000 rw-p 00024000 08:01 8920605 /lib/libm-2.12.1.so 0015d000-00193000 r-xp 00000000 08:01 8912994 /lib/libncurses.so.5.7 00193000-00195000 r--p 00035000 08:01 8912994 /lib/libncurses.so.5.7 00195000-00196000 rw-p 00037000 08:01 8912994 /lib/libncurses.so.5.7 00196000-001ab000 r-xp 00000000 08:01 8920615 /lib/libpthread-2.12.1.so 001ab000-001ac000 ---p 00015000 08:01 8920615 /lib/libpthread-2.12.1.so 001ac000-001ad000 r--p 00015000 08:01 8920615 /lib/libpthread-2.12.1.so 001ad000-001ae000 rw-p 00016000 08:01 8920615 /lib/libpthread-2.12.1.so 001ae000-001b0000 rw-p 00000000 00:00 0 001b0000-001b7000 r-xp 00000000 08:01 8920617 /lib/librt-2.12.1.so 001b7000-001b8000 r--p 00006000 08:01 8920617 /lib/librt-2.12.1.so 001b8000-001b9000 rw-p 00007000 08:01 8920617 /lib/librt-2.12.1.so 001b9000-00310000 r-xp 00000000 08:01 8920601 /lib/libc-2.12.1.so 00310000-00312000 r--p 00157000 08:01 8920601 /lib/libc-2.12.1.so 00312000-00313000 rw-p 00159000 08:01 8920601 /lib/libc-2.12.1.so 00313000-00316000 rw-p 00000000 00:00 0 00316000-0031d000 r-xp 00000000 08:01 4987538 /usr/local/lib/erlang/lib/crypto-2.0.1/priv/lib/crypto.so 0031d000-0031e000 r--p 00006000 08:01 4987538 /usr/local/lib/erlang/lib/crypto-2.0.1/priv/lib/crypto.so 0031e000-0031f000 rw-p 00007000 08:01 4987538 /usr/local/lib/erlang/lib/crypto-2.0.1/priv/lib/crypto.so 0031f000-00454000 r-xp 00000000 08:01 8913384 /lib/libcrypto.so.0.9.8 00454000-0045c000 r--p 00134000 08:01 8913384 /lib/libcrypto.so.0.9.8 0045c000-0046b000 rw-p 0013c000 08:01 8913384 /lib/libcrypto.so.0.9.8 0046b000-0046e000 rw-p 00000000 00:00 0 0046e000-00481000 r-xp 00000000 08:01 8913090 /lib/libz.so.1.2.3.4 00481000-00482000 r--p 00012000 08:01 8913090 /lib/libz.so.1.2.3.4 00482000-00483000 rw-p 00013000 08:01 8913090 /lib/libz.so.1.2.3.4 00483000-0048b000 r-xp 00000000 08:01 6034321 /home/dude/projects/fannerl/priv/fannerl_nif.so 0048b000-0048c000 r--p 00007000 08:01 6034321 /home/dude/projects/fannerl/priv/fannerl_nif.so 0048c000-0048d000 rw-p 00008000 08:01 6034321 /home/dude/projects/fannerl/priv/fannerl_nif.so 0048d000-0049d000 r-xp 00000000 08:01 3814722 /usr/local/lib/libfann.so.2.0.1 0049d000-0049e000 r--p 0000f000 08:01 3814722 /usr/local/lib/libfann.so.2.0.1 0049e000-0049f000 rw-p 00010000 08:01 3814722 /usr/local/lib/libfann.so.2.0.1 0049f000-004b9000 r-xp 00000000 08:01 8912975 /lib/libgcc_s.so.1 004b9000-004ba000 r--p 00019000 08:01 8912975 /lib/libgcc_s.so.1 004ba000-004bb000 rw-p 0001a000 08:01 8912975 /lib/libgcc_s.so.1 08048000-081d3000 r-xp 00000000 08:01 3809922 /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp 081d3000-081d4000 r--p 0018a000 08:01 3809922 /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp 081d4000-08201000 rw-p 0018b000 08:01 3809922 /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp 08201000-082e2000 rw-p 00000000 00:00 0 [heap] b5700000-b5708000 rw-p 00000000 00:00 0 b5708000-b5800000 ---p 00000000 00:00 0 b588a000-b5d00000 rw-p 00000000 00:00 0 b5d00000-b5d01000 ---p 00000000 00:00 0 b5d01000-b6501000 rw-p 00000000 00:00 0 b6501000-b6502000 ---p 00000000 00:00 0 b6502000-b6d02000 rw-p 00000000 00:00 0 b6d02000-b6d03000 ---p 00000000 00:00 0 b6d03000-b7503000 rw-p 00000000 00:00 0 b7503000-b7504000 ---p 00000000 00:00 0 b7504000-b7fed000 rw-p 00000000 00:00 0 b7fef000-b7ff0000 rw-p 00000000 00:00 0 b7ff0000-b7ff1000 ---p 00000000 00:00 0 b7ff1000-b7ff5000 rw-p 00000000 00:00 0 b7ff5000-b7ff6000 ---p 00000000 00:00 0 b7ff6000-b8000000 rw-p 00000000 00:00 0 bffdf000-c0000000 rw-p 00000000 00:00 0 [stack] Program received signal SIGABRT, Aborted. [Switching to Thread 0xb6d01b70 (LWP 3195)] 0x0012e416 in ?? () (gdb) where #0 0x0012e416 in ?? () #1 0x001e3941 in raise () from /lib/libc.so.6 #2 0x001e6e42 in abort () from /lib/libc.so.6 #3 0x0021b305 in ?? () from /lib/libc.so.6 #4 0x00225501 in ?? () from /lib/libc.so.6 #5 0x00226d70 in ?? () from /lib/libc.so.6 #6 0x00229e5d in free () from /lib/libc.so.6 #7 0x00493343 in fann_destroy (ann=0x0) at fann.c:786 #8 0x004857cd in destroy_fann_pointer (env=0xb6d01144, resource=0x82849c0) at c_src/fann_nif.c:130 #9 0x0814397a in nif_resource_dtor (bin=0x82849a8) at beam/erl_nif.c:1176 #10 0x080c6946 in erts_bin_free (offheap=0xb5bc35f8) at beam/erl_binary.h:294 #11 erts_cleanup_offheap (offheap=0xb5bc35f8) at beam/erl_message.c:174 #12 0x080d8a8c in delete_process (p=0xb5bc34f4, pix_lock=) at beam/erl_process.c:8328 #13 continue_exit_process (p=0xb5bc34f4, pix_lock=) at beam/erl_process.c:9370 #14 0x0814d90e in process_main () at beam/beam_emu.c:2976 #15 0x080d165e in sched_thread_func (vesdp=0xb7dff000) at beam/erl_process.c:3635 #16 0x081b1485 in thr_wrapper (vtwd=0xbfffee10) at pthread/ethread.c:106 #17 0x0019bcc9 in start_thread () from /lib/libpthread.so.0 #18 0x0028969e in clone () from /lib/libc.so.6 (gdb) second error: Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) 1> l(fannerl). {module,fannerl} 2> c("test/fannerl_tests"). {ok,fannerl_tests} 3> fannerl fannerl fannerl_tests 3> fannerl_tests:create_test(). test/fannerl_tests.erl:8: exiting create_test ok 4> fannerl_tests:get_mse_test(). Destroy the fann pointer test/fannerl_tests.erl:13: exiting get_mse_test ok 5> fannerl_tests:activation_function_test(). Destroy the fann pointer test/fannerl_tests.erl:19: before last get test/fannerl_tests.erl:21: after last get testing to see if we get here Destroy the fann pointer Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xb6d01b70 (LWP 3214)] 0x002268ac in ?? () from /lib/libc.so.6 (gdb) where #0 0x002268ac in ?? () from /lib/libc.so.6 #1 0x00229e5d in free () from /lib/libc.so.6 #2 0x00326549 in fann_destroy (ann=0xa3c4517b) at fann.c:811 #3 0x003187cd in destroy_fann_pointer (env=0xb6d010c4, resource=0x82833f8) at c_src/fann_nif.c:130 #4 0x0814397a in nif_resource_dtor (bin=0x82833e0) at beam/erl_nif.c:1176 #5 0x080c6946 in erts_bin_free (offheap=0xb7eccba0) at beam/erl_binary.h:294 #6 erts_cleanup_offheap (offheap=0xb7eccba0) at beam/erl_message.c:174 #7 0x080d8a8c in delete_process (p=0xb7ecca9c, pix_lock=) at beam/erl_process.c:8328 #8 continue_exit_process (p=0xb7ecca9c, pix_lock=) at beam/erl_process.c:9370 #9 0x0814c14e in terminate_proc (c_p=0xb7ecca9c, pc=, reg=, bf=0x80af500) at beam/beam_emu.c:5363 #10 handle_error (c_p=0xb7ecca9c, pc=, reg=, bf=0x80af500) at beam/beam_emu.c:5238 #11 0x081539d5 in process_main () at beam/beam_emu.c:2260 #12 0x080d165e in sched_thread_func (vesdp=0xb7dff000) at beam/erl_process.c:3635 #13 0x081b1485 in thr_wrapper (vtwd=0xbfffee90) at pthread/ethread.c:106 #14 0x0019bcc9 in start_thread () from /lib/libpthread.so.0 #15 0x0028969e in clone () from /lib/libc.so.6 /Erik Axling From michael.eugene.turner@REDACTED Tue Feb 8 05:39:36 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 8 Feb 2011 13:39:36 +0900 Subject: [erlang-questions] Re: Erlang: Who uses it for games? In-Reply-To: References: <4C3490D3.1030108@gmail.com> <7DC4E578-DDDA-47BA-BE09-1733A209799E@cs.otago.ac.nz> <95f43ac3-f279-4990-b4eb-c90fef53192e@r27g2000yqb.googlegroups.com> Message-ID: Well, *that* definitely needed to be added to the Wikipedia article about Erlang. Done. Now all we need is a parse transform to make a new language called Cylon: Erlang with FORTH syntax. Just in case you ever wondered why you could never understood the Cylons. -michael turner On Tue, Feb 8, 2011 at 12:27 AM, Joe Armstrong wrote: > On Thu, Jul 8, 2010 at 9:54 AM, ingo.schramm >wrote: > > > Great! So many Erlang gamers out there! > > > > Somebody told me, a Battlestar Galactica browser game is backed up by > > an Erlang engine. > > > > > There's a article about this here: > > http://www.theregister.co.uk/2011/02/05/battlestar_galactica_mmp/ > > > /Joe > > > On Jul 8, 2:05 am, "Richard O'Keefe" wrote: > > > On Jul 8, 2010, at 4:27 AM, Nicholas Frechette wrote: > > > > > > > hand optimize code which will require C++/assembly) and not so much > > > > on the > > > > PS3 where you may get a hard time using the SPUs from erlang (as I'm > > > > guessing the VM won't run on them out of the box and you might have > > > > to spend > > > > considerable time to get erlang processes to run on SPUs (feasible > I'm > > > > sure)). > > > > > > It's not clear that it _is_ feasible in any interesting sense. > > > The SPUs are essentially numeric vector engines. They have 128 > > > registers, each 128 bits wide; loads are only 128 bits (16 bytes) > > > and stores are only 128 bits (16 bytes). Memory is byte addressed > > > and addresses are 32 bits, but the 2009 manual talks about 256 kB. > > > > > > One could imagine SPU _nodes_ doing vector crunching that Erlang > > > processes running on the PPU communicated with as if Erlang. But > > > Erlang would not be the language of choice for programming SPUs. > > > (An APL dialect would be ideal, or Fortran 95.) > > > > > > ________________________________________________________________ > > > erlang-questions (at) erlang.org mailing list. > > > Seehttp://www.erlang.org/faq.html > > > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > From ingela@REDACTED Tue Feb 8 08:19:00 2011 From: ingela@REDACTED (Ingela Andin) Date: Tue, 8 Feb 2011 08:19:00 +0100 Subject: [erlang-questions] ssl_connection process hibernation option In-Reply-To: References: Message-ID: Hi! I think hibernation should be optional and if you choose to use it, it could have configurtion parameters such as the one you suggest. Regards Ingela Erlang/OTP team - Ericsson AB 2011/2/7 Kaiduan Xie : > Ingela, > > May I have a suggestion to have the option to set the idle time for > SSL connection so that after a mount of time of idle, the SSL > connection will be put into hibernation? > > Thanks, > > /Kaiduan > > On Mon, Feb 7, 2011 at 3:32 AM, Ingela Andin wrote: >> Hi! >> >> We (the OTP team) would consider it. >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> >> 2011/2/4 Jeroen Koops : >>> Hi Ingela, >>> Thanks for your response. Suppose I come up with a patch for this, do you >>> think the OTP team would consider it for inclusion in an upcoming release? >>> >>> On Fri, Feb 4, 2011 at 4:51 PM, Ingela Andin wrote: >>>> >>>> Hi! >>>> >>>> We already put some effort into cleaning the state of the gen_fsm >>>> processes holding the connection, >>>> but some form of hibernation could be interesting too we will take >>>> this into consideration. >>>> >>>> Regards Ingela Erlang/OTP team - Ericsson AB >>>> >>>> 2011/2/4 Jeroen Koops : >>>> > I would very much appreciate this too. >>>> > >>>> > As a test, I inserted a proxy-module between the gen_fsm and >>>> > ssl_connection >>>> > callback, that changes all responses from the ssl_connection callback to >>>> > include the 'hibernate' option (it's not a hack -- it's AOP!) and the >>>> > savings are dramatic: before the change an SSL connection used around >>>> > 350 >>>> > kB, after the change only around 600 bytes. If a system is maintaining >>>> > thousands of SSL connections, this really adds up. >>>> > >>>> > If we could somehow get hibernation built in to the ssl application, >>>> > that >>>> > would be great. >>>> > >>>> > On Fri, Feb 4, 2011 at 9:57 AM, Ferenc Holzhauser < >>>> > ferenc.holzhauser@REDACTED> wrote: >>>> > >>>> >> Hi, >>>> >> >>>> >> I'm using R14B01 new SSL to set up a lot of (not very active) client >>>> >> connections. >>>> >> >>>> >> I have noticed that ssl_connection processes use a lot of memory >>>> >> between >>>> >> GC's. >>>> >> A forced GC makes them small again until the next piece of received >>>> >> data. >>>> >> >>>> >> This adds up to a significant memory usage (around 350-400 kB/session >>>> >> on a >>>> >> x64 system in my case) limiting the number of possible sessions. >>>> >> >>>> >> An option to hibernate this gen_fsm would decrease the average memory >>>> >> usage >>>> >> in this scenario. >>>> >> >>>> >> Would it be possible to introduce this option? >>>> >> >>>> >> Thanks in advance, >>>> >> Ferenc >>>> >> >>>> > >>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From icfp.publicity@REDACTED Tue Feb 8 09:32:03 2011 From: icfp.publicity@REDACTED (Wouter Swierstra) Date: Tue, 8 Feb 2011 09:32:03 +0100 Subject: ICFP 2011: Second Call for Papers Message-ID: ===================================================================== Second Call for Papers ICFP 2011: International Conference on Functional Programming Tokyo, Japan, Monday 19 -- Wednesday 21 September 2011 http://www.icfpconference.org/icfp2011 ===================================================================== Important Dates ~~~~~~~~~~~~~~~ Titles, abstracts & keywords due: Thursday 17 March 2011 at 14:00 UTC Submissions due: Thursday 24 March 2011 at 14:00 UTC Author response: Tuesday & Wednesday 17-18 May Notification: Monday 30 May 2011 Final copy due: Friday 01 July 2011 Conference: Monday-Wednesday 19-21 September 2011 Scope ~~~~~ ICFP 2011 seeks original papers on the art and science of functional programming. Submissions are invited on all topics from principles to practice, from foundations to features, and from abstraction to application. The scope includes all languages that encourage functional programming, including both purely applicative and imperative languages, as well as languages with objects, concurrency, or parallelism. Particular topics of interest include * Language Design: type systems; concurrency and distribution; modules; components and composition; metaprogramming; relations to imperative, object-oriented, or logic programming; interoperability * Implementation: abstract machines; virtual machines; interpretation; compilation; compile-time and run-time optimization; memory management; multi-threading; exploiting parallel hardware; interfaces to foreign functions, services, components, or low-level machine resources * Software-Development Techniques: algorithms and data structures; design patterns; specification; verification; validation; proof assistants; debugging; testing; tracing; profiling * Foundations: formal semantics; lambda calculus; rewriting; type theory; mathematical logic; monads; continuations; delimited continuations; global, delimited, or local effects * Transformation and Analysis: abstract interpretation; partial evaluation; program transformation; program calculation; program proofs; normalization by evaluation * Applications and Domain-Specific Languages: symbolic computing; formal-methods tools; artificial intelligence; systems programming; distributed-systems and web programming; hardware design; databases; XML processing; scientific and numerical computing; graphical user interfaces; multimedia programming; scripting; system administration; security; education * Functional Pearls: elegant, instructive, and fun essays on functional programming * Experience Reports: short papers that provide evidence that functional programming really works or describe obstacles that have kept it from working in a particular application Abbreviated instructions for authors ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * By 17 March 2011, 14:00 UTC, submit a title, an abstract of at most 300 words, and keywords. * By 24 March 2011, 14:00 UTC, submit a full paper of at most 12 pages (6 pages for a Functional Pearl and for an Experience Report), including bibliography and figures. The deadlines will be strictly enforced and papers exceeding the page limits will be summarily rejected. * Authors have the option to attach supplementary material to a submission, on the understanding that reviewers may choose not to look at it. * Each submission must adhere to SIGPLAN's republication policy, as explained on the web at http://www.acm.org/sigplan/republicationpolicy.htm In addition, authors of resubmitted (but previously rejected) papers have the option to attach an annotated copy of the reviews of their previous submission(s), explaining how they have addressed these previous reviews in the present submission. If a reviewer identifies him/herself as a reviewer of this previous submission and wishes to see how his/her comments have been addressed, the program chair will communicate to this reviewer the annotated copy of his/her previous review. Otherwise, no rewiewer will read the annotated copies of the previous reviews. Overall, a submission will be evaluated according to its relevance, correctness, significance, originality, and clarity. It should explain its contributions in both general and technical terms, clearly identifying what has been accomplished, explaining why it is significant, and comparing it with previous work. The technical content should be accessible to a broad audience. Functional Pearls and Experience Reports are separate categories of papers that need not report original research results and must be marked as such at the time of submission. Detailed guidelines on both categories are on the conference web site. Proceedings will be published by ACM Press. Authors of accepted submissions are expected to transfer the copyright to the ACM. Presentations will be videotaped and released online if the presenter consents by signing an additional permission form at the time of the presentation. Formatting: Submissions must be in PDF format printable in black and white on US Letter sized paper and interpretable by Ghostscript. If this requirement is a hardship, make contact with the program chair at least one week before the deadline. Papers must adhere to the standard ACM conference format: two columns, nine-point font on a ten-point baseline, with columns 20pc (3.33in) wide and 54pc (9in) tall, with a column gutter of 2pc (0.33in). A suitable document template for LaTeX is available from SIGPLAN at http://www.acm.org/sigs/sigplan/authorInformation.htm Submission: Submissions will be accepted electronically at a URL to be named later. Improved versions of a paper may be submitted at any point before the submission deadline using the same web interface. Author response: Authors will have a 48-hour period, starting at 14:00 UTC on Tuesday 17 May 2010, to read reviews and respond to them. Special Journal Issue: There will be a special issue of the Journal of Functional Programming with papers from ICFP 2011. The program committee will invite the authors of select accepted papers to submit a journal version to this issue. Conference Chairs: Manuel M T Chakravarty, University of New South Wales, Australia Zhenjiang Hu, National Institute of Informatics (NII), Japan Program Chair: Olivier Danvy, Aarhus University, Denmark Program Committee: Kenichi Asai, Ochanomizu University, Japan Josh Berdine, Microsoft Research, UK Adam Chlipala, Harvard University, USA William Cook, University of Texas at Austin, USA Maribel Fernandez, King's College London, UK Ronald Garcia, Carnegie Mellon University, USA Neal Glew, Intel Labs, USA Jacques Garrigue, Nagoya University, Japan Suresh Jagannathan, Purdue University, USA Sam Lindley, University of Edinburgh, UK Frank Pfenning, Carnegie Mellon University, USA Paola Quaglia, University of Trento, Italy Alexis Saurin, University of Paris VII, France Mike Spivey, Oxford University, UK Kristian Stoevring, University of Copenhagen, Denmark Doaitse Swierstra, Utrecht University, The Netherlands David Van Horn, Northeastern University, USA Rene Vestergaard, JAIST, Japan Edwin Westbrook, Rice University, USA From iostres@REDACTED Tue Feb 8 09:51:48 2011 From: iostres@REDACTED (Ivan Ostres) Date: Tue, 8 Feb 2011 09:51:48 +0100 Subject: [erlang-questions] DNS is slow when run from many processes In-Reply-To: References: Message-ID: On 8/2/11 12:50 AM, ori brost wrote: > After checking the states of processes I see that many of them spend > time in: > 3> i(0,200,0). > [{current_function,{inet_gethost_native,getit,2}}, > > > What are my possibilities for better DNS? I now that I can use erlang > dns instead of native dns, this solves the problem for 127.0.0.1, but > when I try a real address (i.e. run(50000,"some.server.of.mine.com")) > connections are very slow with both native and erlang DNS. > > Any advice on a solution? You are doing DNS lookup for every port connect even though destination host does not change for whole PortCount. Did you consider doing DNS lookup before entering loop and cache the result? This would get you O(1) time for hostname lookup instead of O(n). BR, Ivan Ostres From rtrlists@REDACTED Tue Feb 8 10:42:11 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 8 Feb 2011 09:42:11 +0000 Subject: Opportunity for working with Erlang and Lua in Edinburgh, Scotland Message-ID: Hi, if it is no longer OK to post job ads to the list, let me know and I'll desist in future. The company I work for (Axios Systems) is looking for someone to join our Integrations Team (http://www.axiossystems.com/en/company/jobdetail.php/206 ). We write software that connects our main product to all kinds of third-party systems. For this we use an Erlang part for comms and orchestration and Lua for scripting business logic. It is a job that requires direct end-customer communication (and sometimes travel) for requirements, testing, installation, and handover. If you want to have more info, feel free to get in touch with me directly, either on this address or my work one (robert.raschke@REDACTED). Thanks, Robby From sverker@REDACTED Tue Feb 8 11:56:57 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Tue, 8 Feb 2011 11:56:57 +0100 Subject: [erlang-questions] Destructor called twice on NIF resource In-Reply-To: References: Message-ID: <4D512179.70000@erix.ericsson.se> Hi Erik You have NIF resources that refer to each other. I think you need to use enif_keep_resource() to get that to play nice with the GC. If resource A have a pointer to resource B, then you need to do enif_keep_resource(B) to let the GC know that you have a reference of your own to B. If you don't, then the GC may destruct B while A is still alive with an invalid pointer to a dead B. If A stops referring to B (if A is destructed for example), then you need to call enif_release_resource(B) to let the GC know that you no longer have a reference of your own to B. A resource can only be destructed when the initial call to alloc_resource plus all calls to keep_resource have been balanced by an equal number of calls to release_resource. Tip of the day: Run debug emulator during development to (maybe) get better fault identification when you do something wrong in your NIFs. Read "How to Build a Debug Enabled Erlang RunTime System" in INSTALL.md. Calling enif_release_resource one time to many will for example be detected by the debug emulator. /Sverker, Erlang/OTP Erik Axling wrote: > Hello! > > I've been creating NIFs for a C-library and when I'm starting to run EUnit > tests on them it fails with a seg fault. It seems that the destructor for a > resource is called twice which leads to the error as the pointer has already > been freed. I do not unserdtand why this happens though > > If I run the code myself in a shell it doesn't fail but then I am not > executing it inside of functions which, I guess, gets another behaviour for > the GC. But if I compile the eunit test suite file and run the individual > functions myself it also fails but with a different seg fault this time: > > fann_nif.c: https://gist.github.com/815506 > fannerl_test.erl: https://gist.github.com/815543 > > First fault: > $ ./rebar eunit > GNU gdb (GDB) 7.2-ubuntu > Copyright (C) 2010 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show copying" > and "show warranty" for details. > This GDB was configured as "i686-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/local/lib/erlang/erts-5.8.1/bin/erlexec...done. > (gdb) r > Starting program: /usr/local/lib/erlang/erts-5.8.1/bin/erlexec +B -boot > start_clean -noshell -noshell -noinput -run escript start -extra ./rebar > eunit > process 3188 is executing new program: > /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp > [Thread debugging using libthread_db enabled] > [New Thread 0xb7d03b70 (LWP 3191)] > [New Thread 0xb7ffdb70 (LWP 3192)] > [New Thread 0xb7502b70 (LWP 3193)] > [New Thread 0xb7ff4b70 (LWP 3194)] > [New Thread 0xb6d01b70 (LWP 3195)] > [New Thread 0xb6500b70 (LWP 3196)] > ==> fannerl (eunit) > test/fannerl_tests.erl:8: exiting create_test > Destroy the fann pointer > test/fannerl_tests.erl:13: exiting get_mse_test > test/fannerl_tests.erl:19: before last get > test/fannerl_tests.erl:21: after last get > testing to see if we get here > Destroy the fann pointer > Destroy the fann pointer > *** glibc detected *** /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp: > free(): invalid pointer: 0x082da1d0 *** > ======= Backtrace: ========= > /lib/libc.so.6(+0x6c501)[0x225501] > /lib/libc.so.6(+0x6dd70)[0x226d70] > /lib/libc.so.6(cfree+0x6d)[0x229e5d] > /usr/local/lib/libfann.so.2(fann_destroy+0x43)[0x493343] > /home/dude/projects/fannerl/.eunit/../priv/fannerl_nif.so(+0x27cd)[0x4857cd] > /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp[0x814397a] > ======= Memory map: ======== > 00110000-0012c000 r-xp 00000000 08:01 8920592 /lib/ld-2.12.1.so > 0012c000-0012d000 r--p 0001b000 08:01 8920592 /lib/ld-2.12.1.so > 0012d000-0012e000 rw-p 0001c000 08:01 8920592 /lib/ld-2.12.1.so > 0012e000-0012f000 r-xp 00000000 00:00 0 [vdso] > 0012f000-00131000 r-xp 00000000 08:01 8920620 /lib/libutil-2.12.1.so > 00131000-00132000 r--p 00001000 08:01 8920620 /lib/libutil-2.12.1.so > 00132000-00133000 rw-p 00002000 08:01 8920620 /lib/libutil-2.12.1.so > 00133000-00135000 r-xp 00000000 08:01 8920604 /lib/libdl-2.12.1.so > 00135000-00136000 r--p 00001000 08:01 8920604 /lib/libdl-2.12.1.so > 00136000-00137000 rw-p 00002000 08:01 8920604 /lib/libdl-2.12.1.so > 00137000-0015b000 r-xp 00000000 08:01 8920605 /lib/libm-2.12.1.so > 0015b000-0015c000 r--p 00023000 08:01 8920605 /lib/libm-2.12.1.so > 0015c000-0015d000 rw-p 00024000 08:01 8920605 /lib/libm-2.12.1.so > 0015d000-00193000 r-xp 00000000 08:01 8912994 /lib/libncurses.so.5.7 > 00193000-00195000 r--p 00035000 08:01 8912994 /lib/libncurses.so.5.7 > 00195000-00196000 rw-p 00037000 08:01 8912994 /lib/libncurses.so.5.7 > 00196000-001ab000 r-xp 00000000 08:01 8920615 /lib/libpthread-2.12.1.so > 001ab000-001ac000 ---p 00015000 08:01 8920615 /lib/libpthread-2.12.1.so > 001ac000-001ad000 r--p 00015000 08:01 8920615 /lib/libpthread-2.12.1.so > 001ad000-001ae000 rw-p 00016000 08:01 8920615 /lib/libpthread-2.12.1.so > 001ae000-001b0000 rw-p 00000000 00:00 0 > 001b0000-001b7000 r-xp 00000000 08:01 8920617 /lib/librt-2.12.1.so > 001b7000-001b8000 r--p 00006000 08:01 8920617 /lib/librt-2.12.1.so > 001b8000-001b9000 rw-p 00007000 08:01 8920617 /lib/librt-2.12.1.so > 001b9000-00310000 r-xp 00000000 08:01 8920601 /lib/libc-2.12.1.so > 00310000-00312000 r--p 00157000 08:01 8920601 /lib/libc-2.12.1.so > 00312000-00313000 rw-p 00159000 08:01 8920601 /lib/libc-2.12.1.so > 00313000-00316000 rw-p 00000000 00:00 0 > 00316000-0031d000 r-xp 00000000 08:01 4987538 > /usr/local/lib/erlang/lib/crypto-2.0.1/priv/lib/crypto.so > 0031d000-0031e000 r--p 00006000 08:01 4987538 > /usr/local/lib/erlang/lib/crypto-2.0.1/priv/lib/crypto.so > 0031e000-0031f000 rw-p 00007000 08:01 4987538 > /usr/local/lib/erlang/lib/crypto-2.0.1/priv/lib/crypto.so > 0031f000-00454000 r-xp 00000000 08:01 8913384 /lib/libcrypto.so.0.9.8 > 00454000-0045c000 r--p 00134000 08:01 8913384 /lib/libcrypto.so.0.9.8 > 0045c000-0046b000 rw-p 0013c000 08:01 8913384 /lib/libcrypto.so.0.9.8 > 0046b000-0046e000 rw-p 00000000 00:00 0 > 0046e000-00481000 r-xp 00000000 08:01 8913090 /lib/libz.so.1.2.3.4 > 00481000-00482000 r--p 00012000 08:01 8913090 /lib/libz.so.1.2.3.4 > 00482000-00483000 rw-p 00013000 08:01 8913090 /lib/libz.so.1.2.3.4 > 00483000-0048b000 r-xp 00000000 08:01 6034321 > /home/dude/projects/fannerl/priv/fannerl_nif.so > 0048b000-0048c000 r--p 00007000 08:01 6034321 > /home/dude/projects/fannerl/priv/fannerl_nif.so > 0048c000-0048d000 rw-p 00008000 08:01 6034321 > /home/dude/projects/fannerl/priv/fannerl_nif.so > 0048d000-0049d000 r-xp 00000000 08:01 3814722 > /usr/local/lib/libfann.so.2.0.1 > 0049d000-0049e000 r--p 0000f000 08:01 3814722 > /usr/local/lib/libfann.so.2.0.1 > 0049e000-0049f000 rw-p 00010000 08:01 3814722 > /usr/local/lib/libfann.so.2.0.1 > 0049f000-004b9000 r-xp 00000000 08:01 8912975 /lib/libgcc_s.so.1 > 004b9000-004ba000 r--p 00019000 08:01 8912975 /lib/libgcc_s.so.1 > 004ba000-004bb000 rw-p 0001a000 08:01 8912975 /lib/libgcc_s.so.1 > 08048000-081d3000 r-xp 00000000 08:01 3809922 > /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp > 081d3000-081d4000 r--p 0018a000 08:01 3809922 > /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp > 081d4000-08201000 rw-p 0018b000 08:01 3809922 > /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp > 08201000-082e2000 rw-p 00000000 00:00 0 [heap] > b5700000-b5708000 rw-p 00000000 00:00 0 > b5708000-b5800000 ---p 00000000 00:00 0 > b588a000-b5d00000 rw-p 00000000 00:00 0 > b5d00000-b5d01000 ---p 00000000 00:00 0 > b5d01000-b6501000 rw-p 00000000 00:00 0 > b6501000-b6502000 ---p 00000000 00:00 0 > b6502000-b6d02000 rw-p 00000000 00:00 0 > b6d02000-b6d03000 ---p 00000000 00:00 0 > b6d03000-b7503000 rw-p 00000000 00:00 0 > b7503000-b7504000 ---p 00000000 00:00 0 > b7504000-b7fed000 rw-p 00000000 00:00 0 > b7fef000-b7ff0000 rw-p 00000000 00:00 0 > b7ff0000-b7ff1000 ---p 00000000 00:00 0 > b7ff1000-b7ff5000 rw-p 00000000 00:00 0 > b7ff5000-b7ff6000 ---p 00000000 00:00 0 > b7ff6000-b8000000 rw-p 00000000 00:00 0 > bffdf000-c0000000 rw-p 00000000 00:00 0 [stack] > > Program received signal SIGABRT, Aborted. > [Switching to Thread 0xb6d01b70 (LWP 3195)] > 0x0012e416 in ?? () > (gdb) where > #0 0x0012e416 in ?? () > #1 0x001e3941 in raise () from /lib/libc.so.6 > #2 0x001e6e42 in abort () from /lib/libc.so.6 > #3 0x0021b305 in ?? () from /lib/libc.so.6 > #4 0x00225501 in ?? () from /lib/libc.so.6 > #5 0x00226d70 in ?? () from /lib/libc.so.6 > #6 0x00229e5d in free () from /lib/libc.so.6 > #7 0x00493343 in fann_destroy (ann=0x0) at fann.c:786 > #8 0x004857cd in destroy_fann_pointer (env=0xb6d01144, resource=0x82849c0) > at c_src/fann_nif.c:130 > #9 0x0814397a in nif_resource_dtor (bin=0x82849a8) at beam/erl_nif.c:1176 > #10 0x080c6946 in erts_bin_free (offheap=0xb5bc35f8) at > beam/erl_binary.h:294 > #11 erts_cleanup_offheap (offheap=0xb5bc35f8) at beam/erl_message.c:174 > #12 0x080d8a8c in delete_process (p=0xb5bc34f4, pix_lock= out>) > at beam/erl_process.c:8328 > #13 continue_exit_process (p=0xb5bc34f4, pix_lock=) > at beam/erl_process.c:9370 > #14 0x0814d90e in process_main () at beam/beam_emu.c:2976 > #15 0x080d165e in sched_thread_func (vesdp=0xb7dff000) > at beam/erl_process.c:3635 > #16 0x081b1485 in thr_wrapper (vtwd=0xbfffee10) at pthread/ethread.c:106 > #17 0x0019bcc9 in start_thread () from /lib/libpthread.so.0 > #18 0x0028969e in clone () from /lib/libc.so.6 > (gdb) > > second error: > Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] > [kernel-poll:false] > > Eshell V5.8.1 (abort with ^G) > 1> l(fannerl). > {module,fannerl} > 2> c("test/fannerl_tests"). > {ok,fannerl_tests} > 3> fannerl > fannerl fannerl_tests > 3> fannerl_tests:create_test(). > test/fannerl_tests.erl:8: exiting create_test > ok > 4> fannerl_tests:get_mse_test(). > Destroy the fann pointer > test/fannerl_tests.erl:13: exiting get_mse_test > ok > 5> fannerl_tests:activation_function_test(). > Destroy the fann pointer > test/fannerl_tests.erl:19: before last get > test/fannerl_tests.erl:21: after last get > testing to see if we get here > Destroy the fann pointer > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0xb6d01b70 (LWP 3214)] > 0x002268ac in ?? () from /lib/libc.so.6 > (gdb) where > #0 0x002268ac in ?? () from /lib/libc.so.6 > #1 0x00229e5d in free () from /lib/libc.so.6 > #2 0x00326549 in fann_destroy (ann=0xa3c4517b) at fann.c:811 > #3 0x003187cd in destroy_fann_pointer (env=0xb6d010c4, resource=0x82833f8) > at c_src/fann_nif.c:130 > #4 0x0814397a in nif_resource_dtor (bin=0x82833e0) at beam/erl_nif.c:1176 > #5 0x080c6946 in erts_bin_free (offheap=0xb7eccba0) at > beam/erl_binary.h:294 > #6 erts_cleanup_offheap (offheap=0xb7eccba0) at beam/erl_message.c:174 > #7 0x080d8a8c in delete_process (p=0xb7ecca9c, pix_lock= out>) > at beam/erl_process.c:8328 > #8 continue_exit_process (p=0xb7ecca9c, pix_lock=) > at beam/erl_process.c:9370 > #9 0x0814c14e in terminate_proc (c_p=0xb7ecca9c, pc=, > reg=, bf=0x80af500) at beam/beam_emu.c:5363 > #10 handle_error (c_p=0xb7ecca9c, pc=, > reg=, bf=0x80af500) at beam/beam_emu.c:5238 > #11 0x081539d5 in process_main () at beam/beam_emu.c:2260 > #12 0x080d165e in sched_thread_func (vesdp=0xb7dff000) > at beam/erl_process.c:3635 > #13 0x081b1485 in thr_wrapper (vtwd=0xbfffee90) at pthread/ethread.c:106 > #14 0x0019bcc9 in start_thread () from /lib/libpthread.so.0 > #15 0x0028969e in clone () from /lib/libc.so.6 > > /Erik Axling > > From koops.j@REDACTED Tue Feb 8 12:07:23 2011 From: koops.j@REDACTED (Jeroen Koops) Date: Tue, 8 Feb 2011 12:07:23 +0100 Subject: [erlang-questions] [off-topic] MWC GSMA 2011 In-Reply-To: <4D504AD3.2010008@adinet.com.uy> References: <4D504AD3.2010008@adinet.com.uy> Message-ID: The company I work for, Textendo (http://www.textendo.com/), uses Erlang extensively, and will have a presence at MWC. If you want to discuss innovative text messaging products, or hear how well Erlang works for us, then drop me an e-mail, and I'll put you in touch. Regards, Jeroen Koops On Mon, Feb 7, 2011 at 8:41 PM, Bernardo Alvez wrote: > Hi All, > > Are there any companies using Erlang attending MWC Barcelona 2011? > > Regards, > Bernardo > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From alain.odea@REDACTED Tue Feb 8 13:18:00 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Tue, 8 Feb 2011 08:48:00 -0330 Subject: [erlang-questions] Opportunity for working with Erlang and Lua in Edinburgh, Scotland In-Reply-To: References: Message-ID: <8C9CC6F7-F911-4820-89B4-6482F59ADAFF@gmail.com> On 2011-02-08, at 6:12, Robert Raschke wrote: > Hi, > > if it is no longer OK to post job ads to the list, let me know and I'll > desist in future. > > The company I work for (Axios Systems) is looking for someone to join our > Integrations Team (http://www.axiossystems.com/en/company/jobdetail.php/206 > ). > > We write software that connects our main product to all kinds of third-party > systems. For this we use an Erlang part for comms and orchestration and Lua > for scripting business logic. It is a job that requires direct end-customer > communication (and sometimes travel) for requirements, testing, > installation, and handover. > > If you want to have more info, feel free to get in touch with me directly, > either on this address or my work one (robert.raschke@REDACTED). > > Thanks, > Robby Hi Robert: It may be worth posting this job to http://totally-erlang.com/register as well. It is a free service, but please donate if you find good candidates through it :) Cheers, Alain From baryluk@REDACTED Tue Feb 8 17:56:10 2011 From: baryluk@REDACTED (Witold Baryluk) Date: Tue, 8 Feb 2011 17:56:10 +0100 Subject: [erlang-questions] DNS is slow when run from many processes In-Reply-To: References: Message-ID: <20110208165610.GD5777@smp.if.uj.edu.pl> On 02-08 01:50, ori brost wrote: > I've written a small program to demonstrate this: > > ---BEGIN PROGRAM--- > -module(ghbm_bug_test). > -compile(export_all). > > run(PortCount,Url) -> > run(2000,PortCount,Url). > > run(Port,PortCount,Url) -> > case Port < PortCount of > true -> > spawn(fun() -> loop(Port,Url) end), > run(Port + 1,PortCount,Url); > false -> > ok > end. > > loop(Port,Url) -> > case Port of > 2003 -> io:format("Doing connect\n"); > _ -> ok > end, > Sock = gen_tcp:connect(Url, 8080, [], 5000), > case Port of > 2003 -> io:format("Did connect\n"); > _ -> ok > end, > case (catch gen_tcp:close(Sock)) of _ -> ok end, > loop(Port,Url). > ---END PROGRAM--- > > > This program takes a lot of time to connect when connecting via > run(50000,"127.0.0.1") and yet connects quickly when I do run(50000, > {127,0,0,1}). I am running it with a max number of processes of > 1000000 (set via +P). > > After checking the states of processes I see that many of them spend > time in: > 3> i(0,200,0). > [{current_function,{inet_gethost_native,getit,2}}, > > > What are my possibilities for better DNS? I now that I can use erlang > dns instead of native dns, this solves the problem for 127.0.0.1, but > when I try a real address (i.e. run(50000,"some.server.of.mine.com")) > connections are very slow with both native and erlang DNS. > > Any advice on a solution? I will give some tips. Please folow them in order, as most simple tricks are given here first and should solve most of the problems. make sure '127.0.0.1 localhost' entry is in /etc/hosts, and /etc/nsswitch.conf isn't to complicated for 'hosts:' entry, preferably something like 'hosts: dns' should be all there). make also sure there is no "domain" or "search" or "options" in /etc/resolv.conf, (eventually "option inet6" can be accepted) and /etc/gai.conf is unchanged (in case you are using ipv6). If needed use NSCD (or better unscd, which have improved cache and multithreading and fault-tolerance), in front of libc. Tune its cache size if needed. (libc by itself do not cache answers, but 127.0.0.1 in /etc/hosts (which is cache AFAIK until next change), should be fast). If you are going to perform lots of dns lookups of remote/Internet hosts, I suggest you should install DNS rescursive server with cache, (there are lots of good servers) on local machine (or at least on local network). And set 'nameserver 127.0.0.1' or 'nameserver ::1' in /etc/resolv.conf If this will still be too slow, you can skip libc/nscd layer, by makeing Erlang connect directly to nameserver (preferably on same machine), and not use native resolver. See "ERTS User's Guide, Inet configuration". http://www.erlang.org/doc/apps/erts/inet_cfg.html Current configuration can be retrived by inet:get_rc(). You will basically need to put, for example: clear_hosts. clear_ns. clear_search. {resolv_conf, "/dev/null"}. % or any empty file, or just separate resolv.conf for erlang {hosts_file, "/dev/null"}. % or any empty file, or some file with few needed entries {nameserver, {127,0,0,1}}. %% or {nameserver, {127,0,0,1}, 53}., you can add more nameserver for fault-tolerance {lookup, [dns]}. {cache_size, 10000}. into some file (./somewhere/erl_inetrc), and perform, something like export ERL_INETRC=./somehwere/erl_inetrc before starting erlang virtual machine. (actuall even just {lookup, [file]}. should sufice, as erlang will autodetect nameserver from /etc/resolv.conf and entries from /etc/hosts, and even monitor this files for changes in run-time). Erlang dns resolver performs caching, it helps a lot, but have small cache by default. You can increase its size by adding {cache_size, 10000}. to erl_inetrc. (AFAIK if erlang uses native glibc resolver, it do not uses own cache, and leavs this to nscd/unscd or ther mechanisms in glibc or used servers. It is only used when erlang by itself connects to dns server, and not via glibc/nscd). Unfortunetly this cache do not cache negative answers, and probably also do not respect properly TTL values of RRs. And this is only real problem with it, as it works very nicely, and is well designed. This should help, and is very simple to configure in 15 minutes. So lets try. If this is still not sufficient, then you have bug somewhere, or you have realy strange kind of workload. Take look below, for further tips. If this will be still too slow, add Erlang application side cache to this of some kind. ETS table with some garbage collection or some kind of LRU cache, will be probably sufficient. There are many possible and/or existing solutions for caching in erlang. And lastly. If you would want even some more speed, you can run whole DNS server INSIDE erlang, (of course written in erlang), which will perform whole recursive resolving by itself from the root server down to destination and cache everything. (You can also prefill cache with known servers for all top level domains, as they changes rearly, there are only about 5000 of them, and will help a lot - you can use the same trick for locally installed DNS server/cache also). Actually it will not be a server, just a fully recursive resolver, becuase glibc and erlang aren't fully recursive. It unfortunetly will be slightly complex and I am not aware of any DNS server in erlang (beyond my own project which is in alpha stage, and not usefull for anything). I also do not think it will provide significant speedup for any practical workloads, over the already mentioned tips (caching server on local machine and dns lookup by erlang itself). It can eventually save some memory (which is good for caches), and slightly improve latency (by elimniating network traffic over loopback, and context switches). But we already talking about microseconds here, and it do not matter when actuall network traffic is in miliseconds range. (of course if you are not performing UDP send-only DDoS). It also do not helps, as erlang dns resolver already performs caching. Regards, Witek -- Witold Baryluk JID: witold.baryluk // jabster.pl From baryluk@REDACTED Tue Feb 8 18:21:05 2011 From: baryluk@REDACTED (Witold Baryluk) Date: Tue, 8 Feb 2011 18:21:05 +0100 Subject: [erlang-questions] DNS is slow when run from many processes In-Reply-To: <20110208165610.GD5777@smp.if.uj.edu.pl> References: <20110208165610.GD5777@smp.if.uj.edu.pl> Message-ID: <20110208172104.GE5777@smp.if.uj.edu.pl> On 02-08 17:56, Witold Baryluk wrote: > On 02-08 01:50, ori brost wrote: > > I've written a small program to demonstrate this: > > > > What are my possibilities for better DNS? I now that I can use erlang > > dns instead of native dns, this solves the problem for 127.0.0.1, but > > when I try a real address (i.e. run(50000,"some.server.of.mine.com")) > > connections are very slow with both native and erlang DNS. > > > > Any advice on a solution? > .... > Erlang dns resolver performs caching, it helps a lot, but have small cache by default. > You can increase its size by adding {cache_size, 10000}. to erl_inetrc. > (AFAIK if erlang uses native glibc resolver, it do not uses own cache, > and leavs this to nscd/unscd or ther mechanisms in glibc or used servers. > It is only used when erlang by itself connects to dns server, > and not via glibc/nscd). > ... .... > If this will be still too slow, add Erlang application side cache to this > of some kind. > ETS table with some garbage collection or some kind of LRU cache, > will be probably sufficient. There are many possible and/or existing > solutions for caching in erlang. Last issues to know. You can of course go into some scalabilit issues with Erlang or own cache, especially if you are performing lots of requests for the same name from lots of different processes, just like you example (question how representative it is?). I'm not sure erlang resolver detects that, and performs only one request and rest waits for it to be available in cache (as it will be). Without this second call will disocver that there is no entry in cache, and start own request to nameserver. Similary next calls. We can call it race condition. This can drastically increase network usage. For example this (with erlang dns resolver): [ spawn(fun() -> inet_res:gethostbyname("www.gazeta.pl"), ok end) || X <- lists:seq(1,100) ], ok. will lead to over 100 request to local nameserver (i have {lookup, [dns]}, and {nameserver, {127,0,0,1}}. ). (how bad it will be depends actually if nameserver is on the same machine, and if server/cache software it runs is clever enaugh to not do same mistake as erlang does). If you will perform the same code above one more time, it will generate 0 requests. (This is of course becuase responses, even from locla server, especially if local server doesn't have cached them yet, comes with considerable latency, and all inet_res:gthostbyname are executed befor any answer arives, so no erlang cachs entries are populated to stop them from execuing actuall request). This can be solved by fixing erlang, or fixing you application side cache, becuase most often such queries originate from some structure of your queries and connections. Then simply have a two levels of caches. main being a erlang dns cache, and second being a lots of small caches, each dedicated to the some group of processes which performs releated tasks. For example as they crawl same subdomain, or retrivies all content (images, scripts), from single webpage. After all processes in group are done (you hae downloaded page and all it's images and scripts), you can discard a cache (simple dict, or gb_trese will probably suffice), by terminating this cache process. This is much more scalable solution. You can also enter some dark areas like scalability of ETS, or system limits of source udp ports and sockets. This is beyond scope of this problem, and is well explained in many other places. Thats all. :) -- Witold Baryluk JID: witold.baryluk // jabster.pl -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From n.oxyde@REDACTED Tue Feb 8 18:29:10 2011 From: n.oxyde@REDACTED (nox) Date: Tue, 8 Feb 2011 18:29:10 +0100 Subject: [erlang-questions] gen_fsm:handle_state/2 In-Reply-To: References: Message-ID: Le 3 f?vr. 2011 ? 05:33, Yurii Rashkovskii a ?crit : > I am also interested to learn whether OTP would be interested to > consider it or further modification of it as a patch for proper > gen_fsm; if so, I will prepare a proper pull request on GitHub with > tests and updated documentation. I'd be nice to have it merged to > prevent gen_server2 situation. > > Questions, concerns, suggestions? > > Cheers, > Yurii. Hi Yurii, just so that you know, OTP do not use pull requests, mail a git fetch command to erlang-patches. From zvi.avraham@REDACTED Tue Feb 8 19:55:28 2011 From: zvi.avraham@REDACTED (Zvi) Date: Tue, 8 Feb 2011 10:55:28 -0800 (PST) Subject: DNS is slow when run from many processes In-Reply-To: References: Message-ID: <2466b5b5-442c-406e-832d-e50744127003@k16g2000vbq.googlegroups.com> On Feb 8, 10:51?am, Ivan Ostres wrote: > You are doing DNS lookup for every port connect even though destination > host does not change for whole PortCount. Did you consider doing DNS > lookup before entering loop and cache the result? This would get you > O(1) time for hostname lookup instead of O(n). The destination host is load balancer. I hope this make things more clear. > BR, > Ivan Ostres > From axling@REDACTED Tue Feb 8 22:06:33 2011 From: axling@REDACTED (Erik Axling) Date: Tue, 8 Feb 2011 22:06:33 +0100 Subject: [erlang-questions] Destructor called twice on NIF resource In-Reply-To: <4D512179.70000@erix.ericsson.se> References: <4D512179.70000@erix.ericsson.se> Message-ID: And I thought I understood the resources by now :-). But now I'm not sure what to do at all. The resources that I allocate using enif_alloc_resource/2 shouldn't refer to each other and if they do then it's a mistake on my part(or it could be that I have missed something fundamental about resources?). The create_standard_nif function allocates a resource and creates a handle with enif_make_resource. Then I call release and hope that the GC will take care of the rest. The user should be able create as many nets as possible so the create_standard_nif should create resources independent of each other which I thought it did today. Therefore I find it hard to understand where I should use enif_keep_resource and where I should call release? /Erik Axling On Tue, Feb 8, 2011 at 11:56 AM, Sverker Eriksson wrote: > Hi Erik > > You have NIF resources that refer to each other. I think you need to use > enif_keep_resource() to get that to play nice with the GC. > > If resource A have a pointer to resource B, then you need to do > enif_keep_resource(B) to let the GC know that you have a reference of your > own to B. If you don't, then the GC may destruct B while A is still alive > with an invalid pointer to a dead B. > > If A stops referring to B (if A is destructed for example), then you need > to call enif_release_resource(B) to let the GC know that you no longer have > a reference of your own to B. > > A resource can only be destructed when the initial call to alloc_resource > plus all calls to keep_resource have been balanced by an equal number of > calls to release_resource. > > > Tip of the day: > Run debug emulator during development to (maybe) get better fault > identification when you do something wrong in your NIFs. > Read "How to Build a Debug Enabled Erlang RunTime System" in INSTALL.md. > > Calling enif_release_resource one time to many will for example be detected > by the debug emulator. > > /Sverker, Erlang/OTP > > > Erik Axling wrote: > >> Hello! >> >> I've been creating NIFs for a C-library and when I'm starting to run EUnit >> tests on them it fails with a seg fault. It seems that the destructor for >> a >> resource is called twice which leads to the error as the pointer has >> already >> been freed. I do not unserdtand why this happens though >> >> If I run the code myself in a shell it doesn't fail but then I am not >> executing it inside of functions which, I guess, gets another behaviour >> for >> the GC. But if I compile the eunit test suite file and run the individual >> functions myself it also fails but with a different seg fault this time: >> >> fann_nif.c: https://gist.github.com/815506 >> fannerl_test.erl: https://gist.github.com/815543 >> >> First fault: >> $ ./rebar eunit >> GNU gdb (GDB) 7.2-ubuntu >> Copyright (C) 2010 Free Software Foundation, Inc. >> License GPLv3+: GNU GPL version 3 or later < >> http://gnu.org/licenses/gpl.html >> This is free software: you are free to change and redistribute it. >> There is NO WARRANTY, to the extent permitted by law. Type "show copying" >> and "show warranty" for details. >> This GDB was configured as "i686-linux-gnu". >> For bug reporting instructions, please see: >> ... >> Reading symbols from /usr/local/lib/erlang/erts-5.8.1/bin/erlexec...done. >> (gdb) r >> Starting program: /usr/local/lib/erlang/erts-5.8.1/bin/erlexec +B -boot >> start_clean -noshell -noshell -noinput -run escript start -extra ./rebar >> eunit >> process 3188 is executing new program: >> /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp >> [Thread debugging using libthread_db enabled] >> [New Thread 0xb7d03b70 (LWP 3191)] >> [New Thread 0xb7ffdb70 (LWP 3192)] >> [New Thread 0xb7502b70 (LWP 3193)] >> [New Thread 0xb7ff4b70 (LWP 3194)] >> [New Thread 0xb6d01b70 (LWP 3195)] >> [New Thread 0xb6500b70 (LWP 3196)] >> ==> fannerl (eunit) >> test/fannerl_tests.erl:8: exiting create_test >> Destroy the fann pointer >> test/fannerl_tests.erl:13: exiting get_mse_test >> test/fannerl_tests.erl:19: before last get >> test/fannerl_tests.erl:21: after last get >> testing to see if we get here >> Destroy the fann pointer >> Destroy the fann pointer >> *** glibc detected *** /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp: >> free(): invalid pointer: 0x082da1d0 *** >> ======= Backtrace: ========= >> /lib/libc.so.6(+0x6c501)[0x225501] >> /lib/libc.so.6(+0x6dd70)[0x226d70] >> /lib/libc.so.6(cfree+0x6d)[0x229e5d] >> /usr/local/lib/libfann.so.2(fann_destroy+0x43)[0x493343] >> >> /home/dude/projects/fannerl/.eunit/../priv/fannerl_nif.so(+0x27cd)[0x4857cd] >> /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp[0x814397a] >> ======= Memory map: ======== >> 00110000-0012c000 r-xp 00000000 08:01 8920592 /lib/ld-2.12.1.so >> 0012c000-0012d000 r--p 0001b000 08:01 8920592 /lib/ld-2.12.1.so >> 0012d000-0012e000 rw-p 0001c000 08:01 8920592 /lib/ld-2.12.1.so >> 0012e000-0012f000 r-xp 00000000 00:00 0 [vdso] >> 0012f000-00131000 r-xp 00000000 08:01 8920620 /lib/libutil-2.12.1.so >> 00131000-00132000 r--p 00001000 08:01 8920620 /lib/libutil-2.12.1.so >> 00132000-00133000 rw-p 00002000 08:01 8920620 /lib/libutil-2.12.1.so >> 00133000-00135000 r-xp 00000000 08:01 8920604 /lib/libdl-2.12.1.so >> 00135000-00136000 r--p 00001000 08:01 8920604 /lib/libdl-2.12.1.so >> 00136000-00137000 rw-p 00002000 08:01 8920604 /lib/libdl-2.12.1.so >> 00137000-0015b000 r-xp 00000000 08:01 8920605 /lib/libm-2.12.1.so >> 0015b000-0015c000 r--p 00023000 08:01 8920605 /lib/libm-2.12.1.so >> 0015c000-0015d000 rw-p 00024000 08:01 8920605 /lib/libm-2.12.1.so >> 0015d000-00193000 r-xp 00000000 08:01 8912994 /lib/libncurses.so.5.7 >> 00193000-00195000 r--p 00035000 08:01 8912994 /lib/libncurses.so.5.7 >> 00195000-00196000 rw-p 00037000 08:01 8912994 /lib/libncurses.so.5.7 >> 00196000-001ab000 r-xp 00000000 08:01 8920615 /lib/ >> libpthread-2.12.1.so >> 001ab000-001ac000 ---p 00015000 08:01 8920615 /lib/ >> libpthread-2.12.1.so >> 001ac000-001ad000 r--p 00015000 08:01 8920615 /lib/ >> libpthread-2.12.1.so >> 001ad000-001ae000 rw-p 00016000 08:01 8920615 /lib/ >> libpthread-2.12.1.so >> 001ae000-001b0000 rw-p 00000000 00:00 0 >> 001b0000-001b7000 r-xp 00000000 08:01 8920617 /lib/librt-2.12.1.so >> 001b7000-001b8000 r--p 00006000 08:01 8920617 /lib/librt-2.12.1.so >> 001b8000-001b9000 rw-p 00007000 08:01 8920617 /lib/librt-2.12.1.so >> 001b9000-00310000 r-xp 00000000 08:01 8920601 /lib/libc-2.12.1.so >> 00310000-00312000 r--p 00157000 08:01 8920601 /lib/libc-2.12.1.so >> 00312000-00313000 rw-p 00159000 08:01 8920601 /lib/libc-2.12.1.so >> 00313000-00316000 rw-p 00000000 00:00 0 >> 00316000-0031d000 r-xp 00000000 08:01 4987538 >> /usr/local/lib/erlang/lib/crypto-2.0.1/priv/lib/crypto.so >> 0031d000-0031e000 r--p 00006000 08:01 4987538 >> /usr/local/lib/erlang/lib/crypto-2.0.1/priv/lib/crypto.so >> 0031e000-0031f000 rw-p 00007000 08:01 4987538 >> /usr/local/lib/erlang/lib/crypto-2.0.1/priv/lib/crypto.so >> 0031f000-00454000 r-xp 00000000 08:01 8913384 /lib/libcrypto.so.0.9.8 >> 00454000-0045c000 r--p 00134000 08:01 8913384 /lib/libcrypto.so.0.9.8 >> 0045c000-0046b000 rw-p 0013c000 08:01 8913384 /lib/libcrypto.so.0.9.8 >> 0046b000-0046e000 rw-p 00000000 00:00 0 >> 0046e000-00481000 r-xp 00000000 08:01 8913090 /lib/libz.so.1.2.3.4 >> 00481000-00482000 r--p 00012000 08:01 8913090 /lib/libz.so.1.2.3.4 >> 00482000-00483000 rw-p 00013000 08:01 8913090 /lib/libz.so.1.2.3.4 >> 00483000-0048b000 r-xp 00000000 08:01 6034321 >> /home/dude/projects/fannerl/priv/fannerl_nif.so >> 0048b000-0048c000 r--p 00007000 08:01 6034321 >> /home/dude/projects/fannerl/priv/fannerl_nif.so >> 0048c000-0048d000 rw-p 00008000 08:01 6034321 >> /home/dude/projects/fannerl/priv/fannerl_nif.so >> 0048d000-0049d000 r-xp 00000000 08:01 3814722 >> /usr/local/lib/libfann.so.2.0.1 >> 0049d000-0049e000 r--p 0000f000 08:01 3814722 >> /usr/local/lib/libfann.so.2.0.1 >> 0049e000-0049f000 rw-p 00010000 08:01 3814722 >> /usr/local/lib/libfann.so.2.0.1 >> 0049f000-004b9000 r-xp 00000000 08:01 8912975 /lib/libgcc_s.so.1 >> 004b9000-004ba000 r--p 00019000 08:01 8912975 /lib/libgcc_s.so.1 >> 004ba000-004bb000 rw-p 0001a000 08:01 8912975 /lib/libgcc_s.so.1 >> 08048000-081d3000 r-xp 00000000 08:01 3809922 >> /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp >> 081d3000-081d4000 r--p 0018a000 08:01 3809922 >> /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp >> 081d4000-08201000 rw-p 0018b000 08:01 3809922 >> /usr/local/lib/erlang/erts-5.8.1/bin/beam.smp >> 08201000-082e2000 rw-p 00000000 00:00 0 [heap] >> b5700000-b5708000 rw-p 00000000 00:00 0 >> b5708000-b5800000 ---p 00000000 00:00 0 >> b588a000-b5d00000 rw-p 00000000 00:00 0 >> b5d00000-b5d01000 ---p 00000000 00:00 0 >> b5d01000-b6501000 rw-p 00000000 00:00 0 >> b6501000-b6502000 ---p 00000000 00:00 0 >> b6502000-b6d02000 rw-p 00000000 00:00 0 >> b6d02000-b6d03000 ---p 00000000 00:00 0 >> b6d03000-b7503000 rw-p 00000000 00:00 0 >> b7503000-b7504000 ---p 00000000 00:00 0 >> b7504000-b7fed000 rw-p 00000000 00:00 0 >> b7fef000-b7ff0000 rw-p 00000000 00:00 0 >> b7ff0000-b7ff1000 ---p 00000000 00:00 0 >> b7ff1000-b7ff5000 rw-p 00000000 00:00 0 >> b7ff5000-b7ff6000 ---p 00000000 00:00 0 >> b7ff6000-b8000000 rw-p 00000000 00:00 0 >> bffdf000-c0000000 rw-p 00000000 00:00 0 [stack] >> >> Program received signal SIGABRT, Aborted. >> [Switching to Thread 0xb6d01b70 (LWP 3195)] >> 0x0012e416 in ?? () >> (gdb) where >> #0 0x0012e416 in ?? () >> #1 0x001e3941 in raise () from /lib/libc.so.6 >> #2 0x001e6e42 in abort () from /lib/libc.so.6 >> #3 0x0021b305 in ?? () from /lib/libc.so.6 >> #4 0x00225501 in ?? () from /lib/libc.so.6 >> #5 0x00226d70 in ?? () from /lib/libc.so.6 >> #6 0x00229e5d in free () from /lib/libc.so.6 >> #7 0x00493343 in fann_destroy (ann=0x0) at fann.c:786 >> #8 0x004857cd in destroy_fann_pointer (env=0xb6d01144, >> resource=0x82849c0) >> at c_src/fann_nif.c:130 >> #9 0x0814397a in nif_resource_dtor (bin=0x82849a8) at beam/erl_nif.c:1176 >> #10 0x080c6946 in erts_bin_free (offheap=0xb5bc35f8) at >> beam/erl_binary.h:294 >> #11 erts_cleanup_offheap (offheap=0xb5bc35f8) at beam/erl_message.c:174 >> #12 0x080d8a8c in delete_process (p=0xb5bc34f4, pix_lock=> out>) >> at beam/erl_process.c:8328 >> #13 continue_exit_process (p=0xb5bc34f4, pix_lock=) >> at beam/erl_process.c:9370 >> #14 0x0814d90e in process_main () at beam/beam_emu.c:2976 >> #15 0x080d165e in sched_thread_func (vesdp=0xb7dff000) >> at beam/erl_process.c:3635 >> #16 0x081b1485 in thr_wrapper (vtwd=0xbfffee10) at pthread/ethread.c:106 >> #17 0x0019bcc9 in start_thread () from /lib/libpthread.so.0 >> #18 0x0028969e in clone () from /lib/libc.so.6 >> (gdb) >> >> second error: >> Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] >> [kernel-poll:false] >> >> Eshell V5.8.1 (abort with ^G) >> 1> l(fannerl). >> {module,fannerl} >> 2> c("test/fannerl_tests"). >> {ok,fannerl_tests} >> 3> fannerl >> fannerl fannerl_tests >> 3> fannerl_tests:create_test(). >> test/fannerl_tests.erl:8: exiting create_test >> ok >> 4> fannerl_tests:get_mse_test(). >> Destroy the fann pointer >> test/fannerl_tests.erl:13: exiting get_mse_test >> ok >> 5> fannerl_tests:activation_function_test(). >> Destroy the fann pointer >> test/fannerl_tests.erl:19: before last get >> test/fannerl_tests.erl:21: after last get >> testing to see if we get here >> Destroy the fann pointer >> >> Program received signal SIGSEGV, Segmentation fault. >> [Switching to Thread 0xb6d01b70 (LWP 3214)] >> 0x002268ac in ?? () from /lib/libc.so.6 >> (gdb) where >> #0 0x002268ac in ?? () from /lib/libc.so.6 >> #1 0x00229e5d in free () from /lib/libc.so.6 >> #2 0x00326549 in fann_destroy (ann=0xa3c4517b) at fann.c:811 >> #3 0x003187cd in destroy_fann_pointer (env=0xb6d010c4, >> resource=0x82833f8) >> at c_src/fann_nif.c:130 >> #4 0x0814397a in nif_resource_dtor (bin=0x82833e0) at beam/erl_nif.c:1176 >> #5 0x080c6946 in erts_bin_free (offheap=0xb7eccba0) at >> beam/erl_binary.h:294 >> #6 erts_cleanup_offheap (offheap=0xb7eccba0) at beam/erl_message.c:174 >> #7 0x080d8a8c in delete_process (p=0xb7ecca9c, pix_lock=> out>) >> at beam/erl_process.c:8328 >> #8 continue_exit_process (p=0xb7ecca9c, pix_lock=) >> at beam/erl_process.c:9370 >> #9 0x0814c14e in terminate_proc (c_p=0xb7ecca9c, pc=> out>, >> reg=, bf=0x80af500) at beam/beam_emu.c:5363 >> #10 handle_error (c_p=0xb7ecca9c, pc=, >> reg=, bf=0x80af500) at beam/beam_emu.c:5238 >> #11 0x081539d5 in process_main () at beam/beam_emu.c:2260 >> #12 0x080d165e in sched_thread_func (vesdp=0xb7dff000) >> at beam/erl_process.c:3635 >> #13 0x081b1485 in thr_wrapper (vtwd=0xbfffee90) at pthread/ethread.c:106 >> #14 0x0019bcc9 in start_thread () from /lib/libpthread.so.0 >> #15 0x0028969e in clone () from /lib/libc.so.6 >> >> /Erik Axling >> >> >> > > From max.lapshin@REDACTED Wed Feb 9 00:06:56 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 9 Feb 2011 02:06:56 +0300 Subject: Daemonize escript Message-ID: I want to try to package erlyvideo in a rebar-like bundle when all beam files are in one executable. It seems to be very convenient to switch behavior: with -d key erlyvideo is funning foreground with shell available, without it it launches as daemon. Is it possible to do it with escript or I still need several different files? From max.lapshin@REDACTED Wed Feb 9 00:19:08 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 9 Feb 2011 02:19:08 +0300 Subject: [erlang-questions] Daemonize escript In-Reply-To: References: Message-ID: >> > > Have you tried `rebar escriptize` to do this? > I need a bit more complicated script, because I have to collect files from other packages. However, problem still remains: when I launch erlyvideo:start(), it launches supervisor and returns. After this escript stops and everything is turned off. From robert.virding@REDACTED Wed Feb 9 00:24:04 2011 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 8 Feb 2011 23:24:04 +0000 (GMT) Subject: [erlang-questions] Re: Erlang: Who uses it for games? In-Reply-To: Message-ID: <25361468.416191297207444693.JavaMail.root@zimbra> There is another company who Erlang in games and that is Guild Software, http://www.guildsoftware.com/, who have an MMO game called Vendetta Online, http://www.vendetta-online.com/. They use Erlang in the servers. As far as I know they have been using Erlang for at least 3-4 years. Robert ----- "Michael Turner" wrote: > Well, *that* definitely needed to be added to the Wikipedia article > about > Erlang. Done. Now all we need is a parse transform to make a new > language > called Cylon: Erlang with FORTH syntax. Just in case you ever > wondered why > you could never understood the Cylons. > > -michael turner > > On Tue, Feb 8, 2011 at 12:27 AM, Joe Armstrong > wrote: > > > On Thu, Jul 8, 2010 at 9:54 AM, ingo.schramm > > >wrote: > > > > > Great! So many Erlang gamers out there! > > > > > > Somebody told me, a Battlestar Galactica browser game is backed up > by > > > an Erlang engine. > > > > > > > > There's a article about this here: > > > > http://www.theregister.co.uk/2011/02/05/battlestar_galactica_mmp/ > > > > > > /Joe From max.lapshin@REDACTED Wed Feb 9 00:23:30 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 9 Feb 2011 02:23:30 +0300 Subject: [erlang-questions] Daemonize escript In-Reply-To: References: Message-ID: Looking at sources, it seems impossible without escript rewriting. It runs halt right after Module:main(), so erlyvideo should not return from main untill it needs to be alive. On Wed, Feb 9, 2011 at 2:19 AM, Max Lapshin wrote: >>> >> >> Have you tried `rebar escriptize` to do this? >> > > I need a bit more complicated script, because I have to collect files > from other packages. > However, problem still remains: > when I launch erlyvideo:start(), it launches supervisor and returns. > After this escript stops and everything is turned off. > From mjtruog@REDACTED Wed Feb 9 00:34:39 2011 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 08 Feb 2011 15:34:39 -0800 Subject: [erlang-questions] Re: Erlang: Who uses it for games? In-Reply-To: <25361468.416191297207444693.JavaMail.root@zimbra> References: <25361468.416191297207444693.JavaMail.root@zimbra> Message-ID: <4D51D30F.4010108@gmail.com> They also sell the MMO server/client here http://www.guildsoftware.com/naos/inbrief.html The Kourier system uses Erlang to model NPCs (Non-Person Characters) as actors (processes), so there might be AI work in there too. On 02/08/2011 03:24 PM, Robert Virding wrote: > There is another company who Erlang in games and that is Guild Software, http://www.guildsoftware.com/, who have an MMO game called Vendetta Online, http://www.vendetta-online.com/. They use Erlang in the servers. As far as I know they have been using Erlang for at least 3-4 years. > > Robert > > ----- "Michael Turner" wrote: > >> Well, *that* definitely needed to be added to the Wikipedia article >> about >> Erlang. Done. Now all we need is a parse transform to make a new >> language >> called Cylon: Erlang with FORTH syntax. Just in case you ever >> wondered why >> you could never understood the Cylons. >> >> -michael turner >> >> On Tue, Feb 8, 2011 at 12:27 AM, Joe Armstrong >> wrote: >> >>> On Thu, Jul 8, 2010 at 9:54 AM, ingo.schramm >> >>> wrote: >>>> Great! So many Erlang gamers out there! >>>> >>>> Somebody told me, a Battlestar Galactica browser game is backed up >> by >>>> an Erlang engine. >>>> >>>> >>> There's a article about this here: >>> >>> http://www.theregister.co.uk/2011/02/05/battlestar_galactica_mmp/ >>> >>> >>> /Joe > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From michael.eugene.turner@REDACTED Wed Feb 9 07:30:54 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 9 Feb 2011 15:30:54 +0900 Subject: [erlang-questions] Re: Erlang: Who uses it for games? In-Reply-To: <4D51D30F.4010108@gmail.com> References: <25361468.416191297207444693.JavaMail.root@zimbra> <4D51D30F.4010108@gmail.com> Message-ID: Unfortunately, press releases don't qualify as reliable sources (RS) under Wikipedia's guidelines. Guild Software's NAOS engine seems to be known only through press releases, at this point. If we could get somebody from Guild to give a talk about NAOS (say, at the upcoming ACM SIGPLAN Erlang Workshop in Tokyo later this year), resulting in some paper that gets at least a page in a conference proceedings, it's Wikipedia RS, since the writing will have gone through something like peer review. -michael turner On Wed, Feb 9, 2011 at 8:34 AM, Michael Truog wrote: > They also sell the MMO server/client here > http://www.guildsoftware.com/naos/inbrief.html > The Kourier system uses Erlang to model NPCs (Non-Person Characters) as > actors (processes), so there might be AI work in there too. > > On 02/08/2011 03:24 PM, Robert Virding wrote: > > There is another company who Erlang in games and that is Guild Software, > http://www.guildsoftware.com/, who have an MMO game called Vendetta > Online, http://www.vendetta-online.com/. They use Erlang in the servers. > As far as I know they have been using Erlang for at least 3-4 years. > > > > Robert > > > > ----- "Michael Turner" wrote: > > > >> Well, *that* definitely needed to be added to the Wikipedia article > >> about > >> Erlang. Done. Now all we need is a parse transform to make a new > >> language > >> called Cylon: Erlang with FORTH syntax. Just in case you ever > >> wondered why > >> you could never understood the Cylons. > >> > >> -michael turner > >> > >> On Tue, Feb 8, 2011 at 12:27 AM, Joe Armstrong > >> wrote: > >> > >>> On Thu, Jul 8, 2010 at 9:54 AM, ingo.schramm > >> >>>> wrote: > >>>> Great! So many Erlang gamers out there! > >>>> > >>>> Somebody told me, a Battlestar Galactica browser game is backed up > >> by > >>>> an Erlang engine. > >>>> > >>>> > >>> There's a article about this here: > >>> > >>> http://www.theregister.co.uk/2011/02/05/battlestar_galactica_mmp/ > >>> > >>> > >>> /Joe > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From michael.eugene.turner@REDACTED Wed Feb 9 07:41:35 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 9 Feb 2011 15:41:35 +0900 Subject: [erlang-questions] Re: Erlang: Who uses it for games? In-Reply-To: <4D51D30F.4010108@gmail.com> References: <25361468.416191297207444693.JavaMail.root@zimbra> <4D51D30F.4010108@gmail.com> Message-ID: Unfortunately, press releases don't qualify as reliable source (RS) under Wikipedia's guidelines. Guild Software's NAOS engine seems to be known only through press releases, at this point. If we could get somebody from Guild to give a talk about NAOS (say, at the upcoming Erlang activity in the functional programming conference in Tokyo later this year), resulting in some paper that gets a page or so in a conference proceedings, it's Wikipedia RS, since it will have gone through something like peer review. -michael turner On Wed, Feb 9, 2011 at 8:34 AM, Michael Truog wrote: > They also sell the MMO server/client here > http://www.guildsoftware.com/naos/inbrief.html > The Kourier system uses Erlang to model NPCs (Non-Person Characters) as > actors (processes), so there might be AI work in there too. > > On 02/08/2011 03:24 PM, Robert Virding wrote: > > There is another company who Erlang in games and that is Guild Software, > http://www.guildsoftware.com/, who have an MMO game called Vendetta > Online, http://www.vendetta-online.com/. They use Erlang in the servers. > As far as I know they have been using Erlang for at least 3-4 years. > > > > Robert > > > > ----- "Michael Turner" wrote: > > > >> Well, *that* definitely needed to be added to the Wikipedia article > >> about > >> Erlang. Done. Now all we need is a parse transform to make a new > >> language > >> called Cylon: Erlang with FORTH syntax. Just in case you ever > >> wondered why > >> you could never understood the Cylons. > >> > >> -michael turner > >> > >> On Tue, Feb 8, 2011 at 12:27 AM, Joe Armstrong > >> wrote: > >> > >>> On Thu, Jul 8, 2010 at 9:54 AM, ingo.schramm > >> >>>> wrote: > >>>> Great! So many Erlang gamers out there! > >>>> > >>>> Somebody told me, a Battlestar Galactica browser game is backed up > >> by > >>>> an Erlang engine. > >>>> > >>>> > >>> There's a article about this here: > >>> > >>> http://www.theregister.co.uk/2011/02/05/battlestar_galactica_mmp/ > >>> > >>> > >>> /Joe > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From max.lapshin@REDACTED Wed Feb 9 08:53:19 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 9 Feb 2011 10:53:19 +0300 Subject: Optimization help required: find 0,0,1 code in binary Message-ID: Hi! I'd wish to remove NIF code from erlyvideo once and thus I want to try fix some bottlenecks. First is NAL start code lookup. H.264 video frames are packaged in byte stream where frames are divided either by 0,0,0,1 either by 0,0,1 It is not very clear when should be used 4-bytes code or when is used 3-bytes, so everybody is looking for 3-bytes code and then checks if it is 4-byte code. Here is my erlang implementation: It starts to look for a first start code from required Offset, than it looks for second border and extracts binary between them. https://github.com/erlyvideo/erlyvideo/blob/master/deps/mpegts/src/mpegts_reader.erl#L554 extract_nal_erl(Data) -> extract_nal_erl(Data, 0). extract_nal_erl(Data, Offset) -> case Data of <<_:Offset/binary>> -> undefined; <<_:Offset/binary, 1:32, _Rest/binary>> -> extract_nal_erl(Data, Offset + 1, 0); <<_:Offset/binary, 1:24, _Rest/binary>> -> extract_nal_erl(Data, Offset, 0); _ -> extract_nal_erl(Data, Offset+1) end. extract_nal_erl(Data, Offset, Length) -> case Data of <<_:Offset/binary, 1:24, NAL:Length/binary, 1:32, _/binary>> -> <<_:Offset/binary, 1:24, NAL:Length/binary, Rest/binary>> = Data, {ok, NAL, Rest}; <<_:Offset/binary, 1:24, NAL:Length/binary, 1:24, _/binary>> -> <<_:Offset/binary, 1:24, NAL:Length/binary, Rest/binary>> = Data, {ok, NAL, Rest}; <<_:Offset/binary, 1:24, NAL:Length/binary, 0>> -> {ok, NAL, <<>>}; <<_:Offset/binary, 1:24, NAL:Length/binary>> -> {ok, NAL, <<>>}; <<_:Offset/binary, 1:24, _/binary>> -> extract_nal_erl(Data, Offset, Length+1) end. Here is my NIF implementation: https://github.com/erlyvideo/erlyvideo/blob/master/deps/mpegts/src/mpegts_reader.c#L55 static int find_nal(ErlNifBinary data, int i, int border) { if (i + 3 >= data.size) { return -1; } for(; i + 2 < data.size; i++) { if (i + 3 < data.size && data.data[i] == 0 && data.data[i+1] == 0 && data.data[i+2] == 0 && data.data[i+3] == 1) { return border == START ? i + 4 : i; } if (data.data[i] == 0 && data.data[i+1] == 0 && data.data[i+2] == 1) { return border == START ? i + 3 : i; } } return -1; } It is extremely faster: (ems@REDACTED)7> mpegts_reader:benchmark(). debug 2011-02-09 10:36:32 [mpegts] mpegts_reader:655 {"Timer erl",100000,2456324} debug 2011-02-09 10:36:32 [mpegts] mpegts_reader:664 {"Timer native",100000,65452} ok C implementation is 40 times faster than erlang one. It looks for me a bit strange because this is exactly the same difference between these two implementations: https://github.com/erlyvideo/erlyvideo/blob/master/deps/mpegts/src/mpeg2_crc32.c https://github.com/erlyvideo/erlyvideo/blob/master/deps/mpegts/src/mpeg2_crc32.erl I have also tested hipe (with commented out on_load function): (ems@REDACTED)10> c("deps/mpegts/src/mpegts_reader.erl", [{ebin, "deps/mpegts/ebin"},native]). deps/mpegts/src/mpegts_reader.erl:90: Warning: function load_nif/0 is unused deps/mpegts/src/mpegts_reader.erl:93: Warning: function load_nif/1 is unused {ok,mpegts_reader} (ems@REDACTED)11> mpegts_reader:benchmark(). debug 2011-02-09 10:50:40 [mpegts] mpegts_reader:655 {"Timer erl",100000,882046} debug 2011-02-09 10:50:41 [mpegts] mpegts_reader:664 {"Timer native",100000,878927} ok So we get 3 times speed up from erlang bytecode but still it is 10 times slower than C So my questions are: 1) where should I look and how should I profile erlang code to speed up my erlang implementation of byte pattern lookup ? 2) Are there any tricks like put on all possible guards and help hipe to compile more effective code From jesper.pettersson@REDACTED Wed Feb 9 09:18:18 2011 From: jesper.pettersson@REDACTED (Jesper Pettersson) Date: Wed, 9 Feb 2011 09:18:18 +0100 Subject: Crypto-questions Message-ID: I have some questions regarding the crypto- and ssl-modules to see what the future plan is and if anyone in the community has written any functionality that may be on the way in. 1) Support for the SHA-2 hash suite (SHA-256, SHA-384 and SHA-512) In the documentation and running crypto:info/1 I can only find references to MD4, MD5 and SHA-1. rsa_digest_type() can only be md5 and sha1 according to doumentation. 2) Support for RSASSA-PSS padding with RSA Signatures. According to the documentation currently no_padding and RSAPSS-PKCS1 padding are supported. This padding technique adds message blinding, making sure that the same message will never get the same signature even when signed with the same key. Further reading: http://www.rsa.com/rsalabs/node.asp?id=2005 3) Follow-up on 1 & 2, Support for sha256WithRSAEncryption, oid 1.2.840.113549.1.1.11 4) Is there an explicit upper limit on the RSA key modulus lengths in the Erlang implementation? 5) Are there any plans to include ECC (Elliptic Curve Cryptography) as a public key algorithm in Erlang? Jesper Pettersson Klarna AB From banibrata.dutta@REDACTED Wed Feb 9 09:49:45 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Wed, 9 Feb 2011 14:19:45 +0530 Subject: Any work happing on ErlyBird, the Erlang plugin for Netbeans ? Message-ID: Does anyone have any update on ErlyBird and if there is any hope for Erlang plugins for latest version of Netbeans and functionality updates ? regards, B From bgustavsson@REDACTED Wed Feb 9 11:33:30 2011 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 9 Feb 2011 11:33:30 +0100 Subject: [erlang-questions] Optimization help required: find 0,0,1 code in binary In-Reply-To: References: Message-ID: On Wed, Feb 9, 2011 at 8:53 AM, Max Lapshin wrote: > Hi! I'd wish to remove NIF code from erlyvideo once and thus I want to > try fix some bottlenecks. > > First is NAL start code lookup. H.264 video frames are packaged in > byte stream where frames are divided either by 0,0,0,1 either by 0,0,1 > It is not very clear when should be used 4-bytes code or when is used > 3-bytes, so everybody is looking for 3-bytes code and then checks if > it is 4-byte code. Have you read the section about matching binaries in the Efficiency Guide? http://www.erlang.org/doc/efficiency_guide/binaryhandling.html#id64336 The way you have structured your binary matching used to be best way to do it before R12B (and is similar to the "DO NOT" example in that section), but now it is the slowest way. Look at the "DO" example and try to structure your matching in the same way. You can use the bin_opt_info compiler option to print information about binary optimizations being applied (or not applied). -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From dsc@REDACTED Wed Feb 9 12:22:11 2011 From: dsc@REDACTED (Dave Challis) Date: Wed, 09 Feb 2011 11:22:11 +0000 Subject: try .. of .. catch question References: <4D5278E3.5000707@ecs.soton.ac.uk> Message-ID: Hi, I'm just learning about exceptions in Erlang, and just curious as to the use of try..of..catch - is it just a shortcut to combining a case with a try/catch? For example, are the two blocks of code below identical? Or is there some subtle difference I've missed? try case foo() of true -> a; false -> b end catch Error:Reason -> {Error, Reason} end try foo() of true -> a; false -> b catch Error:Reason -> {Error, Reason} end Thanks, -- Dave Challis dsc@REDACTED From koops.j@REDACTED Wed Feb 9 12:31:25 2011 From: koops.j@REDACTED (Jeroen Koops) Date: Wed, 9 Feb 2011 12:31:25 +0100 Subject: [erlang-questions] try .. of .. catch question In-Reply-To: References: <4D5278E3.5000707@ecs.soton.ac.uk> Message-ID: Hi Dave, The two blocks are different. In the first block, if an exception occurs anywhere between the 'case' and the 'end', it will be caught by the catch-clause. But in the second block, only exceptions occurring when evaluating foo() will be caught; exceptions occuring while evaluating 'a' or 'b' won't. Regards, Jeroen On Wed, Feb 9, 2011 at 12:22 PM, Dave Challis wrote: > Hi, > I'm just learning about exceptions in Erlang, and just curious as to the > use of try..of..catch - is it just a shortcut to combining a case with a > try/catch? > > For example, are the two blocks of code below identical? Or is there some > subtle difference I've missed? > > try > case foo() of > true -> a; > false -> b > end > catch > Error:Reason -> {Error, Reason} > end > > > try > foo() > of > true -> a; > false -> b > catch > Error:Reason -> {Error, Reason} > end > > Thanks, > -- > Dave Challis > dsc@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From max.lapshin@REDACTED Wed Feb 9 12:35:38 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 9 Feb 2011 14:35:38 +0300 Subject: [erlang-questions] Optimization help required: find 0,0,1 code in binary In-Reply-To: References: Message-ID: I've refactored code and got such numbers: 9 microseconds for beam code 3 microseconds for HiPE 0,7 microseconds for C code Not bad! From jesper.louis.andersen@REDACTED Wed Feb 9 12:43:05 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 9 Feb 2011 12:43:05 +0100 Subject: [erlang-questions] Optimization help required: find 0,0,1 code in binary In-Reply-To: References: Message-ID: 2011/2/9 Max Lapshin : > I've refactored code and got such numbers: > > 9 microseconds for beam code > 3 microseconds for HiPE > 0,7 microseconds for C code I wonder if you can cheat and fast-path the common variant where the offset is 0 by handling it outside the functions with the 'Offset' parameter. The 'binary' modules (binary:split/2,3) also comes to mind as something to try. -- J. From raimo+erlang-questions@REDACTED Wed Feb 9 13:15:55 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 9 Feb 2011 13:15:55 +0100 Subject: [erlang-questions] try .. of .. catch question In-Reply-To: References: <4D5278E3.5000707@ecs.soton.ac.uk> Message-ID: <20110209121555.GA13750@erix.ericsson.se> One more subtle difference: in the first block if foo() returns something other than true or false, the case clause generated will be caught by the catch clause, but in the second block a try clause will be generated and not caught by the catch clause since it out of scope of the try..catch. On Wed, Feb 09, 2011 at 12:31:25PM +0100, Jeroen Koops wrote: > Hi Dave, > > The two blocks are different. In the first block, if an exception occurs > anywhere between the 'case' and the 'end', it will be caught by the > catch-clause. But in the second block, only exceptions occurring when > evaluating foo() will be caught; exceptions occuring while evaluating 'a' or > 'b' won't. > > Regards, > > Jeroen > > > > On Wed, Feb 9, 2011 at 12:22 PM, Dave Challis wrote: > > > Hi, > > I'm just learning about exceptions in Erlang, and just curious as to the > > use of try..of..catch - is it just a shortcut to combining a case with a > > try/catch? > > > > For example, are the two blocks of code below identical? Or is there some > > subtle difference I've missed? > > > > try > > case foo() of > > true -> a; > > false -> b > > end > > catch > > Error:Reason -> {Error, Reason} > > end > > > > > > try > > foo() > > of > > true -> a; > > false -> b > > catch > > Error:Reason -> {Error, Reason} > > end > > > > Thanks, > > -- > > Dave Challis > > dsc@REDACTED > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From dsc@REDACTED Wed Feb 9 13:19:43 2011 From: dsc@REDACTED (Dave Challis) Date: Wed, 09 Feb 2011 12:19:43 +0000 Subject: [erlang-questions] try .. of .. catch question In-Reply-To: <20110209121555.GA13750@erix.ericsson.se> References: <4D5278E3.5000707@ecs.soton.ac.uk> <20110209121555.GA13750@erix.ericsson.se> <4D52865F.6030400@ecs.soton.ac.uk> Message-ID: Ah, thanks for the answers, than clears up a lot of things! On 09/02/11 12:15, Raimo Niskanen wrote: > One more subtle difference: in the first block if foo() returns > something other than true or false, the case clause generated > will be caught by the catch clause, but in the second block > a try clause will be generated and not caught by the catch > clause since it out of scope of the try..catch. > > On Wed, Feb 09, 2011 at 12:31:25PM +0100, Jeroen Koops wrote: >> Hi Dave, >> >> The two blocks are different. In the first block, if an exception occurs >> anywhere between the 'case' and the 'end', it will be caught by the >> catch-clause. But in the second block, only exceptions occurring when >> evaluating foo() will be caught; exceptions occuring while evaluating 'a' or >> 'b' won't. >> >> Regards, >> >> Jeroen >> >> >> >> On Wed, Feb 9, 2011 at 12:22 PM, Dave Challis wrote: >> >>> Hi, >>> I'm just learning about exceptions in Erlang, and just curious as to the >>> use of try..of..catch - is it just a shortcut to combining a case with a >>> try/catch? >>> >>> For example, are the two blocks of code below identical? Or is there some >>> subtle difference I've missed? >>> >>> try >>> case foo() of >>> true -> a; >>> false -> b >>> end >>> catch >>> Error:Reason -> {Error, Reason} >>> end >>> >>> >>> try >>> foo() >>> of >>> true -> a; >>> false -> b >>> catch >>> Error:Reason -> {Error, Reason} >>> end >>> >>> Thanks, >>> -- >>> Dave Challis >>> dsc@REDACTED >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> > -- Dave Challis dsc@REDACTED From max.lapshin@REDACTED Wed Feb 9 13:45:42 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 9 Feb 2011 15:45:42 +0300 Subject: [erlang-questions] Optimization help required: find 0,0,1 code in binary In-Reply-To: References: Message-ID: 2011/2/9 Jesper Louis Andersen : > > I wonder if you can cheat and fast-path the common variant where the > offset is 0 by handling it outside the functions with the 'Offset' > parameter. The 'binary' modules (binary:split/2,3) also comes to mind > as something to try. But I have to find second offset. This is the problem. From ingela@REDACTED Wed Feb 9 14:34:51 2011 From: ingela@REDACTED (Ingela Andin) Date: Wed, 9 Feb 2011 14:34:51 +0100 Subject: [erlang-questions] Crypto-questions In-Reply-To: References: Message-ID: Hi! 2011/2/9 Jesper Pettersson : > I have some questions regarding the crypto- and ssl-modules to see what the > future plan is and if anyone in the community has written any functionality > that may be on the way in. > > 1) Support for the SHA-2 hash suite (SHA-256, SHA-384 and SHA-512) > In the documentation and running crypto:info/1 I can only find references to > MD4, MD5 and SHA-1. > rsa_digest_type() can only be md5 and sha1 according to doumentation. > > 2) Support for RSASSA-PSS padding with RSA Signatures. > According to the documentation currently no_padding and RSAPSS-PKCS1 padding > are supported. > This padding technique adds message blinding, making sure that the same > message will never get the same signature even when signed with the same > key. Further reading: http://www.rsa.com/rsalabs/node.asp?id=2005 > > 3) Follow-up on 1 & 2, Support for sha256WithRSAEncryption, oid > 1.2.840.113549.1.1.11 In the long term we plan to support TLS 1.2 and then support for this I think will follow. But as for right now openssl does not yet support TLS 1.2 and theire support for TLS 1.1 is still experimental, hence I think this is somewhere in a more distant future. > 4) Is there an explicit upper limit on the RSA key modulus lengths in the > Erlang implementation? Erlang (crypto) does not enforce any limitations that is not already present in openssl. Crypto uses openssls crypto library. But apart from cryptographic operations new ssl does not use openssl and all message and certificate handling is implemented in Erlang. > 5) Are there any plans to include ECC (Elliptic Curve Cryptography) as a > public key algorithm in Erlang? No such plans at the moment. Regards Ingela Erlang/OTP team - Ericsson AB From mevans@REDACTED Wed Feb 9 20:20:50 2011 From: mevans@REDACTED (Evans, Matthew) Date: Wed, 9 Feb 2011 14:20:50 -0500 Subject: Internet Content Adaptation Protocol (ICAP) Message-ID: Hi, I've been recently reading about ICAP - see: http://tools.ietf.org/html/rfc3507 Seems like an ideal candidate for an Erlang (e.g. YAWS, mochiweb etc). Anyone know of any implementations? Regards Matt From vances@REDACTED Wed Feb 9 21:51:30 2011 From: vances@REDACTED (Vance Shipley) Date: Wed, 9 Feb 2011 15:51:30 -0500 Subject: [erlang-questions] [off-topic] MWC GSMA 2011 In-Reply-To: <4D504AD3.2010008@adinet.com.uy> References: <4D504AD3.2010008@adinet.com.uy> Message-ID: <20110209205130.GD5674@h216-235-12-171.host.egate.net> Bernardo, I'll be there Monday and Tuesday. -- -Vance On Mon, Feb 07, 2011 at 05:41:07PM -0200, Bernardo Alvez wrote: } Are there any companies using Erlang attending MWC Barcelona 2011? From chandrashekhar.mullaparthi@REDACTED Wed Feb 9 21:58:11 2011 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Wed, 9 Feb 2011 20:58:11 +0000 Subject: [erlang-questions] Internet Content Adaptation Protocol (ICAP) In-Reply-To: References: Message-ID: On 9 February 2011 19:20, Evans, Matthew wrote: > Hi, > > I've been recently reading about ICAP - see: > http://tools.ietf.org/html/rfc3507 > > Seems like an ideal candidate for an Erlang (e.g. YAWS, mochiweb etc). > > Anyone know of any implementations? > > It is very HTTP-ish. We're working on an ICAP client implementation on top of ibrowse. If you are interested, I'll make the changes open source once we are done. cheers Chandru From mevans@REDACTED Wed Feb 9 22:01:15 2011 From: mevans@REDACTED (Evans, Matthew) Date: Wed, 9 Feb 2011 16:01:15 -0500 Subject: [erlang-questions] Internet Content Adaptation Protocol (ICAP) In-Reply-To: References: Message-ID: Hi Chandru, Yes, I'd like to see what you have; although I'm more interested in the server-side implementation. Thanks Matt ________________________________ From: Chandru [mailto:chandrashekhar.mullaparthi@REDACTED] Sent: Wednesday, February 09, 2011 3:58 PM To: Evans, Matthew Cc: Erlang Subject: Re: [erlang-questions] Internet Content Adaptation Protocol (ICAP) On 9 February 2011 19:20, Evans, Matthew > wrote: Hi, I've been recently reading about ICAP - see: http://tools.ietf.org/html/rfc3507 Seems like an ideal candidate for an Erlang (e.g. YAWS, mochiweb etc). Anyone know of any implementations? It is very HTTP-ish. We're working on an ICAP client implementation on top of ibrowse. If you are interested, I'll make the changes open source once we are done. cheers Chandru From kostis@REDACTED Thu Feb 10 02:21:11 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 10 Feb 2011 03:21:11 +0200 Subject: HiPE backend for PPC64 Message-ID: <4D533D87.6000803@cs.ntua.gr> There is a new back end of the HiPE native code compiler for PPC64 (PowerPC 64). The relevant fetch command is: git fetch git://github.com/kostis/otp.git hipe-ppc64 Please include into 'pu'/'dev'. Also, I would appreciate some feedback from interested users who want to test this early on whether this is auto-enabled in PPC64 architectures and native compiled code works OK for things they have tried -- I have only very limited access to such a machine. Kostis From kaiduanx@REDACTED Thu Feb 10 03:46:22 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 9 Feb 2011 21:46:22 -0500 Subject: [erlang-questions] ssl_connection process hibernation option In-Reply-To: References: Message-ID: Ingela, I also found ssl_connection_sup consumes a lot of memory. For the system with 20K SSL connections, ssl_connection_sup eats 65,638,608 bytes (65M), so for each SSL connection, ssl_connection_sup itself takes about 3K bytes. Do you have any idea to reduce this? This result is taken from R14A on x86/SMP. Thanks for help, /Kaiduan (erlsip_proxy@REDACTED)6> process_info(pid(0,66,0)). [{registered_name,ssl_connection_sup}, {current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}, {status,waiting}, {message_queue_len,0}, {messages,[]}, {links,[<0.16515.0>,<0.133.1>,<0.4229.1>,<0.6277.1>, <0.7301.1>,<0.7813.1>,<0.8069.1>,<0.8197.1>,<0.8261.1>, <0.8293.1>,<0.8309.1>,<0.8317.1>,<0.8321.1>,<0.8325.1>, <0.8327.1>,<0.8323.1>,<0.8319.1>,<0.8313.1>,<0.8315.1>, <0.8311.1>|...]}, {dictionary,[{'$ancestors',[ssl_sup,<0.62.0>]}, {'$initial_call',{supervisor,ssl_connection_sup,1}}]}, {trap_exit,true}, {error_handler,error_handler}, {priority,normal}, {group_leader,<0.61.0>}, {total_heap_size,8102320}, {heap_size,1682835}, {stack_size,9}, {reductions,2046919}, {garbage_collection,[{min_bin_vheap_size,46368}, {min_heap_size,233}, {fullsweep_after,65535}, {minor_gcs,6}]}, {suspending,[]}] (erlsip_proxy@REDACTED)7> process_info(pid(0,66,0),memory). {memory,65638608} On Tue, Feb 8, 2011 at 2:19 AM, Ingela Andin wrote: > Hi! > > I think hibernation should be optional and if you choose to use it, it > could have configurtion parameters such as the > one you suggest. > > Regards Ingela Erlang/OTP team - Ericsson AB > > 2011/2/7 Kaiduan Xie : >> Ingela, >> >> May I have a suggestion to have the option to set the idle time for >> SSL connection so that after a mount of time of idle, the SSL >> connection will be put into hibernation? >> >> Thanks, >> >> /Kaiduan >> >> On Mon, Feb 7, 2011 at 3:32 AM, Ingela Andin wrote: >>> Hi! >>> >>> We (the OTP team) would consider it. >>> >>> Regards Ingela Erlang/OTP team - Ericsson AB >>> >>> >>> 2011/2/4 Jeroen Koops : >>>> Hi Ingela, >>>> Thanks for your response. Suppose I come up with a patch for this, do you >>>> think the OTP team would consider it for inclusion in an upcoming release? >>>> >>>> On Fri, Feb 4, 2011 at 4:51 PM, Ingela Andin wrote: >>>>> >>>>> Hi! >>>>> >>>>> We already put some effort into cleaning the state of the gen_fsm >>>>> processes holding the connection, >>>>> but some form of hibernation could be interesting too we will take >>>>> this into consideration. >>>>> >>>>> Regards Ingela Erlang/OTP team - Ericsson AB >>>>> >>>>> 2011/2/4 Jeroen Koops : >>>>> > I would very much appreciate this too. >>>>> > >>>>> > As a test, I inserted a proxy-module between the gen_fsm and >>>>> > ssl_connection >>>>> > callback, that changes all responses from the ssl_connection callback to >>>>> > include the 'hibernate' option (it's not a hack -- it's AOP!) and the >>>>> > savings are dramatic: before the change an SSL connection used around >>>>> > 350 >>>>> > kB, after the change only around 600 bytes. If a system is maintaining >>>>> > thousands of SSL connections, this really adds up. >>>>> > >>>>> > If we could somehow get hibernation built in to the ssl application, >>>>> > that >>>>> > would be great. >>>>> > >>>>> > On Fri, Feb 4, 2011 at 9:57 AM, Ferenc Holzhauser < >>>>> > ferenc.holzhauser@REDACTED> wrote: >>>>> > >>>>> >> Hi, >>>>> >> >>>>> >> I'm using R14B01 new SSL to set up a lot of (not very active) client >>>>> >> connections. >>>>> >> >>>>> >> I have noticed that ssl_connection processes use a lot of memory >>>>> >> between >>>>> >> GC's. >>>>> >> A forced GC makes them small again until the next piece of received >>>>> >> data. >>>>> >> >>>>> >> This adds up to a significant memory usage (around 350-400 kB/session >>>>> >> on a >>>>> >> x64 system in my case) limiting the number of possible sessions. >>>>> >> >>>>> >> An option to hibernate this gen_fsm would decrease the average memory >>>>> >> usage >>>>> >> in this scenario. >>>>> >> >>>>> >> Would it be possible to introduce this option? >>>>> >> >>>>> >> Thanks in advance, >>>>> >> Ferenc >>>>> >> >>>>> > >>>> >>>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From barcojie@REDACTED Thu Feb 10 05:53:09 2011 From: barcojie@REDACTED (Barco You) Date: Thu, 10 Feb 2011 12:53:09 +0800 Subject: [Erlang] Any suggestion or Reference for Embedded Erlang programming on a mobile-phone like chip Message-ID: Dear Members, Could you please give a reference about embedded Erlang development on a mobile chip, such as iPhone, or multimedia-processing specific chip, like MP3. Thanks! Best regards, Barco From paramr@REDACTED Thu Feb 10 06:06:38 2011 From: paramr@REDACTED (Param Jyothi Reddy) Date: Wed, 9 Feb 2011 21:06:38 -0800 Subject: upgrading erlang servers Message-ID: Hi guys, I am new to erlang and was reading "Programming Erlang" by Joe Armstrong and one remark in the book surprised me. Pg 173: "And same version of erlang [on all nodes]. If you dont do this, you'll get serious and mysterious errors.". Not sure how true this is currently, however if it is, how do you upgrade cluster of nodes running X version of erlang to Y without disrupting service? I would have assumed that one can upgrade subset at a time which would also provide a vehicle to test version Y on small subset before committing the full cluster. But if this comment is true then one would have to upgrade cluster of nodes in an all or nothing fashion. There are other ways to upgrade by having two clusters and switching load between them using some protocol. Is that how it is meant to be used? Am i missing something obvious? Thanks, Param From matthias@REDACTED Thu Feb 10 12:00:48 2011 From: matthias@REDACTED (Matthias Lang) Date: Thu, 10 Feb 2011 12:00:48 +0100 Subject: why can't dialyzer see that my spec is a lie? Message-ID: <20110210110048.GA5027@corelatus.se> Hi, Running the example below through dialyzer R14B01: 143> dialyzer:run([{from, src_code}, {files, ["minimal.erl"]}]). [] I was hoping Dialyzer would complain that my spec (line 18) is a lie. Cut down, but compilable, code below. Changing "almost anything" results in a good warning, e.g. if I comment out the recursive step (line 13): 145> dialyzer:run([{from, src_code}, {files, ["minimal.erl"]}]). [{warn_contract_types,{"/tmp/minimal.erl",17}, {extra_range,[minimal,parse_and_execute,1,"'quit'", "{_,_}"]}}] What's going on? I'm stumped. Matt ---------------------------------------------------------------------- -module(minimal). -export([start/0]). -record(state, {commands}). start() -> Fun = fun() -> {quit, "string"} end, server_loop(#state{commands=Fun}). server_loop(State) -> try {New_state, _} = parse_and_execute(State), server_loop(New_state) catch _:_ -> ok end. -spec parse_and_execute(any()) -> {#state{}, string()} | 'quit'. parse_and_execute(State) -> Fun = State#state.commands, Fun(). From hynek@REDACTED Thu Feb 10 13:07:56 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Thu, 10 Feb 2011 13:07:56 +0100 Subject: [erlang-questions] Is this code tail recursive ? In-Reply-To: <552417436.406791297092078554.JavaMail.root@zimbra> References: <552417436.406791297092078554.JavaMail.root@zimbra> Message-ID: On Mon, Feb 7, 2011 at 4:21 PM, Robert Virding wrote: > Yes, it is tail-recursive in gather_files/1/2. Though it will probably not return what you want as the first clause of gather_files/2: > > gather_files([], [Acc]) -> Acc; > > will only succeed if there is ONE element (file) in the accumulator in which case it will return the file, otherwise it will crash. Did you intend > > gather_files([], Acc) -> Acc; ? > > Robert > No, it works well if it is called gether_files([{path,Dir}]) because in this case it will have only one element in Acc due ? ? directory -> ? ? ? ? gather_files(Files, [gather_directory_files(F) | Acc ]); I would rewrite it as .... gather_files([],Acc) -> Acc; ... ? ? directory -> ? ? ? ? gather_files(Files, gather_directory_files(F, Acc)); ... gather_directory_files(D, Tail) -> filelib:fold_files( D, ".*", true, fun(X,Acc) -> [X|Acc] end, Tail). > ----- "Mihai Balea" wrote: > >> On Feb 6, 2011, at 6:08 PM, Radu Brumariu wrote: >> >> > gather_files([]) -> []; >> > gather_files(F) -> >> > ? ?gather_files(F,[]). >> > gather_files([],[Acc]) -> Acc; >> > gather_files([{path,F}|Files],Acc) -> >> > ? ?io:format("Processing path ~p~n",[F]), >> > ? ?{ok, T} = file:read_file_info(F), >> > ? ?case T#file_info.type >> > ? ?of >> > ? ? regular -> >> > ? ? ? ? gather_files(Files, [F | Acc]); >> > ? ? directory -> >> > ? ? ? ? gather_files(Files, [gather_directory_files(F) | Acc ]); >> > ? ? _ -> >> > ? ? ? ? gather_files(Files, Acc) >> > ? ?end. >> > >> > gather_directory_files(D) -> >> > ? ?filelib:fold_files( D, ".*", true, fun(X,Acc) -> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [X|Acc] >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ?end, >> > ? ? ? ? ? ? ? ? ? ?[]). >> > >> >> Looks tail recursive to me, although looking at the way you build up >> your result, it might not be what you expect :) >> Not sure what you expect though, so I cannot comment on that. >> >> Hope this helps, >> Mihai >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- > Robert Virding, Erlang Solutions Ltd. > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From greim@REDACTED Thu Feb 10 15:06:36 2011 From: greim@REDACTED (greim) Date: Thu, 10 Feb 2011 15:06:36 +0100 Subject: [erlang-questions] [Erlang] Any suggestion or Reference for Embedded Erlang programming on a mobile-phone like chip In-Reply-To: References: Message-ID: <4D53F0EC.6060905@schleibinger.com> Hi Barco, http://embedded-erlang.org here you can download a complete bootable image for the beagle board or gumstix overo. http://beagleboard.org/ http://www.gumstix.org/ I have tested it on gumstix, it works, but is was not able to get the network interface running up to now. So any help is welcome here. Somebody made a port to Android http://www.burbas.se/artiklar/erlang-for-the-android-plattform/ Also this http://www.sics.se/~joe/sae.html sounds nice, but not tried yet. Regards Markus Greim > > Could you please give a reference about embedded Erlang development on a > mobile chip, such as iPhone, or multimedia-processing specific chip, like > MP3. > > Thanks! > > Best regards, > Barco > -- From dsc@REDACTED Thu Feb 10 15:34:45 2011 From: dsc@REDACTED (Dave Challis) Date: Thu, 10 Feb 2011 14:34:45 +0000 Subject: Generating readable unique IDs for use outside erlang References: <4D53F785.9030300@ecs.soton.ac.uk> Message-ID: Is there a good (and fast) way of generating a unique id for use outside Erlang? The best method I can find for generating a unique id is to use: {node(), now()} Which generates something like: {foo@REDACTED,{1297,347375,276224}} This works great, and is what I use within erlang. However, it looks a bit meaningless is likely to confuse users of the non-erlang application that uses it - is there a nice way to turn that into an integer, or series of integers instead? E.g. turning the above into: 3473290872_1297_347375_276224 (or ideally a single integer) would be fine, but I'm not sure how to get an integer from a term. Chances are I'll be generating hundreds of thousands to several million of these, so I'm also a bit concerned about efficiency - turning the {node(), now()} tuple into a string would no doubt take up more memory (and some processing time). Any suggestions on this? -- Dave Challis dsc@REDACTED From ingela@REDACTED Thu Feb 10 15:34:34 2011 From: ingela@REDACTED (Ingela Anderton Andin) Date: Thu, 10 Feb 2011 15:34:34 +0100 Subject: [erlang-questions] ssl_connection process hibernation option In-Reply-To: References: Message-ID: <4D53F77A.7010603@erix.ericsson.se> Hi! Well this seems to be a supervisor-problem rather than a ssl-problem. Supervisors saves init-parameters to be able to restart processes, which is natural, but it seems to do this even for temporary processes which is unnecessary and could be quite expensive. We will look in to this. Regards Ingela Erlang/OTP - Ericsson AB Kaiduan Xie wrote: > Ingela, > > I also found ssl_connection_sup consumes a lot of memory. For the > system with 20K SSL connections, ssl_connection_sup eats 65,638,608 > bytes (65M), so for each SSL connection, ssl_connection_sup itself > takes about 3K bytes. Do you have any idea to reduce this? This result > is taken from R14A on x86/SMP. > > Thanks for help, > > /Kaiduan > > (erlsip_proxy@REDACTED)6> process_info(pid(0,66,0)). > [{registered_name,ssl_connection_sup}, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}, > {status,waiting}, > {message_queue_len,0}, > {messages,[]}, > {links,[<0.16515.0>,<0.133.1>,<0.4229.1>,<0.6277.1>, > <0.7301.1>,<0.7813.1>,<0.8069.1>,<0.8197.1>,<0.8261.1>, > <0.8293.1>,<0.8309.1>,<0.8317.1>,<0.8321.1>,<0.8325.1>, > <0.8327.1>,<0.8323.1>,<0.8319.1>,<0.8313.1>,<0.8315.1>, > <0.8311.1>|...]}, > {dictionary,[{'$ancestors',[ssl_sup,<0.62.0>]}, > {'$initial_call',{supervisor,ssl_connection_sup,1}}]}, > {trap_exit,true}, > {error_handler,error_handler}, > {priority,normal}, > {group_leader,<0.61.0>}, > {total_heap_size,8102320}, > {heap_size,1682835}, > {stack_size,9}, > {reductions,2046919}, > {garbage_collection,[{min_bin_vheap_size,46368}, > {min_heap_size,233}, > {fullsweep_after,65535}, > {minor_gcs,6}]}, > {suspending,[]}] > (erlsip_proxy@REDACTED)7> process_info(pid(0,66,0),memory). > {memory,65638608} > > > On Tue, Feb 8, 2011 at 2:19 AM, Ingela Andin wrote: > >> Hi! >> >> I think hibernation should be optional and if you choose to use it, it >> could have configurtion parameters such as the >> one you suggest. >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> 2011/2/7 Kaiduan Xie : >> >>> Ingela, >>> >>> May I have a suggestion to have the option to set the idle time for >>> SSL connection so that after a mount of time of idle, the SSL >>> connection will be put into hibernation? >>> >>> Thanks, >>> >>> /Kaiduan >>> >>> On Mon, Feb 7, 2011 at 3:32 AM, Ingela Andin wrote: >>> >>>> Hi! >>>> >>>> We (the OTP team) would consider it. >>>> >>>> Regards Ingela Erlang/OTP team - Ericsson AB >>>> >>>> >>>> 2011/2/4 Jeroen Koops : >>>> >>>>> Hi Ingela, >>>>> Thanks for your response. Suppose I come up with a patch for this, do you >>>>> think the OTP team would consider it for inclusion in an upcoming release? >>>>> >>>>> On Fri, Feb 4, 2011 at 4:51 PM, Ingela Andin wrote: >>>>> >>>>>> Hi! >>>>>> >>>>>> We already put some effort into cleaning the state of the gen_fsm >>>>>> processes holding the connection, >>>>>> but some form of hibernation could be interesting too we will take >>>>>> this into consideration. >>>>>> >>>>>> Regards Ingela Erlang/OTP team - Ericsson AB >>>>>> >>>>>> 2011/2/4 Jeroen Koops : >>>>>> >>>>>>> I would very much appreciate this too. >>>>>>> >>>>>>> As a test, I inserted a proxy-module between the gen_fsm and >>>>>>> ssl_connection >>>>>>> callback, that changes all responses from the ssl_connection callback to >>>>>>> include the 'hibernate' option (it's not a hack -- it's AOP!) and the >>>>>>> savings are dramatic: before the change an SSL connection used around >>>>>>> 350 >>>>>>> kB, after the change only around 600 bytes. If a system is maintaining >>>>>>> thousands of SSL connections, this really adds up. >>>>>>> >>>>>>> If we could somehow get hibernation built in to the ssl application, >>>>>>> that >>>>>>> would be great. >>>>>>> >>>>>>> On Fri, Feb 4, 2011 at 9:57 AM, Ferenc Holzhauser < >>>>>>> ferenc.holzhauser@REDACTED> wrote: >>>>>>> >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I'm using R14B01 new SSL to set up a lot of (not very active) client >>>>>>>> connections. >>>>>>>> >>>>>>>> I have noticed that ssl_connection processes use a lot of memory >>>>>>>> between >>>>>>>> GC's. >>>>>>>> A forced GC makes them small again until the next piece of received >>>>>>>> data. >>>>>>>> >>>>>>>> This adds up to a significant memory usage (around 350-400 kB/session >>>>>>>> on a >>>>>>>> x64 system in my case) limiting the number of possible sessions. >>>>>>>> >>>>>>>> An option to hibernate this gen_fsm would decrease the average memory >>>>>>>> usage >>>>>>>> in this scenario. >>>>>>>> >>>>>>>> Would it be possible to introduce this option? >>>>>>>> >>>>>>>> Thanks in advance, >>>>>>>> Ferenc >>>>>>>> >>>>>>>> >>>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> >>>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >>> > > From mek@REDACTED Thu Feb 10 15:40:51 2011 From: mek@REDACTED (Max E. Kuznecov) Date: Thu, 10 Feb 2011 16:40:51 +0200 Subject: [erlang-questions] Generating readable unique IDs for use outside erlang In-Reply-To: References: <4D53F785.9030300@ecs.soton.ac.uk> Message-ID: Looks like erlang:phash2/1 may help: > erlang:phash2({node(), now()}). 59735927 2011/2/10 Dave Challis : > Is there a good (and fast) way of generating a unique id for use outside > Erlang? > > The best method I can find for generating a unique id is to use: > {node(), now()} > > Which generates something like: > {foo@REDACTED,{1297,347375,276224}} > > This works great, and is what I use within erlang. > > However, it looks a bit meaningless is likely to confuse users of the > non-erlang application that uses it - is there a nice way to turn that into > an integer, or series of integers instead? > > E.g. turning the above into: > 3473290872_1297_347375_276224 > (or ideally a single integer) > > would be fine, but I'm not sure how to get an integer from a term. > > Chances are I'll be generating hundreds of thousands to several million of > these, so I'm also a bit concerned about efficiency - turning the {node(), > now()} tuple into a string would no doubt take up more memory (and some > processing time). > > Any suggestions on this? > > -- > Dave Challis > dsc@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- ~syhpoon From dsc@REDACTED Thu Feb 10 15:48:53 2011 From: dsc@REDACTED (Dave Challis) Date: Thu, 10 Feb 2011 14:48:53 +0000 Subject: [erlang-questions] Generating readable unique IDs for use outside erlang In-Reply-To: References: <4D53F785.9030300@ecs.soton.ac.uk> <4D53FAD5.4020500@ecs.soton.ac.uk> Message-ID: Perfect, that looks like it does exactly as I need, thanks! On 10/02/11 14:40, Max E. Kuznecov wrote: > Looks like erlang:phash2/1 may help: > >> erlang:phash2({node(), now()}). > 59735927 > > 2011/2/10 Dave Challis: >> Is there a good (and fast) way of generating a unique id for use outside >> Erlang? >> >> The best method I can find for generating a unique id is to use: >> {node(), now()} >> >> Which generates something like: >> {foo@REDACTED,{1297,347375,276224}} >> >> This works great, and is what I use within erlang. >> >> However, it looks a bit meaningless is likely to confuse users of the >> non-erlang application that uses it - is there a nice way to turn that into >> an integer, or series of integers instead? >> >> E.g. turning the above into: >> 3473290872_1297_347375_276224 >> (or ideally a single integer) >> >> would be fine, but I'm not sure how to get an integer from a term. >> >> Chances are I'll be generating hundreds of thousands to several million of >> these, so I'm also a bit concerned about efficiency - turning the {node(), >> now()} tuple into a string would no doubt take up more memory (and some >> processing time). >> >> Any suggestions on this? >> >> -- >> Dave Challis >> dsc@REDACTED >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > > -- Dave Challis dsc@REDACTED From mevans@REDACTED Thu Feb 10 15:47:15 2011 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 10 Feb 2011 09:47:15 -0500 Subject: [erlang-questions] Generating readable unique IDs for use outside erlang In-Reply-To: References: <4D53F785.9030300@ecs.soton.ac.uk> Message-ID: A few ideas come to mind: 1) erlang:term_to_binary/1? (taking your tuple as input) 2) erlang:md5/1 (taking the result of term_to_binary/1 as an input, returning a binary as the result). 3) erlang:phash/2 (with some crazy large number to try to guarantee uniqueness). 4) erlang:crc32/1 (taking the result of term_to_binary/2 as input, this returns an int as the result). Matt -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Dave Challis Sent: Thursday, February 10, 2011 9:35 AM To: erlang-questions@REDACTED Subject: [erlang-questions] Generating readable unique IDs for use outside erlang Is there a good (and fast) way of generating a unique id for use outside Erlang? The best method I can find for generating a unique id is to use: {node(), now()} Which generates something like: {foo@REDACTED,{1297,347375,276224}} This works great, and is what I use within erlang. However, it looks a bit meaningless is likely to confuse users of the non-erlang application that uses it - is there a nice way to turn that into an integer, or series of integers instead? E.g. turning the above into: 3473290872_1297_347375_276224 (or ideally a single integer) would be fine, but I'm not sure how to get an integer from a term. Chances are I'll be generating hundreds of thousands to several million of these, so I'm also a bit concerned about efficiency - turning the {node(), now()} tuple into a string would no doubt take up more memory (and some processing time). Any suggestions on this? -- Dave Challis dsc@REDACTED ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From peralta.alejandro@REDACTED Thu Feb 10 15:47:26 2011 From: peralta.alejandro@REDACTED (Ale) Date: Thu, 10 Feb 2011 11:47:26 -0300 Subject: [erlang-questions] Generating readable unique IDs for use outside erlang In-Reply-To: References: <4D53F785.9030300@ecs.soton.ac.uk> Message-ID: > > Any suggestions on this? > Why don't you generate UUID? http://en.wikipedia.org/wiki/Universally_unique_identifier I found this googling: https://github.com/travis/erlang-uuid/blob/master/uuid.erl cheers, -- Ale. From olivier.girondel@REDACTED Thu Feb 10 16:02:34 2011 From: olivier.girondel@REDACTED (Olivier Girondel) Date: Thu, 10 Feb 2011 16:02:34 +0100 Subject: [erlang-questions] Generating readable unique IDs for use outside erlang In-Reply-To: References: <4D53F785.9030300@ecs.soton.ac.uk> Message-ID: Quite old code, could be rewritten as NIF now: http://erlang.biniou.org/erluuid On Thursday, February 10, 2011, Dave Challis wrote: > Is there a good (and fast) way of generating a unique id for use outside Erlang? > > The best method I can find for generating a unique id is to use: > {node(), now()} > > Which generates something like: > {foo@REDACTED,{1297,347375,276224}} > > This works great, and is what I use within erlang. > > However, it looks a bit meaningless is likely to confuse users of the non-erlang application that uses it - is there a nice way to turn that into an integer, or series of integers instead? > > E.g. turning the above into: > 3473290872_1297_347375_276224 > (or ideally a single integer) > > would be fine, but I'm not sure how to get an integer from a term. > > Chances are I'll be generating hundreds of thousands to several million of these, so I'm also a bit concerned about efficiency - turning the {node(), now()} tuple into a string would no doubt take up more memory (and some processing time). > > Any suggestions on this? > > -- > Dave Challis > dsc@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Olivier From atilla.erdodi@REDACTED Thu Feb 10 16:45:15 2011 From: atilla.erdodi@REDACTED (=?ISO-8859-2?Q?Atilla_Erd=F5di?=) Date: Thu, 10 Feb 2011 16:45:15 +0100 Subject: [erlang-questions] web framework, db, architecture query, with requirements In-Reply-To: References: Message-ID: 2011/2/7 Icarus Alive : > Thanks for the pointer to Zotonic. The feature set is quite > impressive, however my understanding is that CMS is good where content > needs to be seen by lots and lots of people, and you have lots of > content to organize, make it searchable, classifiable etc. Just to > clarify my requirement a bit further, this is for a rather niche > service, where small groups of people have access to a fairly limited > set of content, and that content changes quite frequently, i.e. > content older than few days gets purged. And there are possibly > hundreds of thousands of such small groups. Would you say a CMS is > good for such application ? The scenario you are describing here is similar what we're using Zotonic for at Maxclass (http://www.maxclass.com), and we're developing it towards that direction. Maybe it's not appropriate to call Zotonic a CMS anymore, however it does include one. Regards, Atilla From ulf.wiger@REDACTED Thu Feb 10 17:01:23 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 10 Feb 2011 17:01:23 +0100 Subject: [erlang-questions] Generating readable unique IDs for use outside erlang In-Reply-To: References: <4D53F785.9030300@ecs.soton.ac.uk> Message-ID: <5AA018C0-A500-4764-8092-54B5866F68FC@erlang-solutions.com> On 10 Feb 2011, at 15:34, Dave Challis wrote: > Is there a good (and fast) way of generating a unique id for use outside Erlang? Depending on your needs, it could be useful to provide a mapping between an internal id and an external, reasonably memorable, one. This code is from a blog example in ErlHive (http://erlhive.svn.sourceforge.net/viewvc/erlhive/trunk/lib/erlhive/examples/blog/src/flexiblog.erl?revision=69&view=markup) The trick for generating a unique Id in this case was to rely on the Erlhive 'stream' data type: new_blog(Properties) -> Caller = erlhive.user:auth(), B_props = blog_props(Properties), N = erlhive.user:append([blogs, Caller], B_props), Key = [blogs, Caller, N], then providing conversion functions to/from external presentation: %%% @spec (item()) -> string() %%% @doc Converts an internal blog, article or comment identifier %%% to an external representation. %%% id_to_string([blogs,U,I]) when is_binary(U), is_integer(I) -> "blogs/" ++ .lists:concat([b2l(U), "/", i2l(I)]); id_to_string([ba,U,I,J]) when is_binary(U), is_integer(I), is_integer(J) -> "ba/" ++ blog_item_serial(U,I,J); id_to_string([bc,U,I,J]) when is_binary(U), is_integer(I), is_integer(J) -> "bc/" ++ blog_item_serial(U,I,J). blog_item_serial(U, I, J) -> .lists:concat([b2l(U), "/", i2l(I), "/", i2l(J)]). b2l(B) -> binary_to_list(B). i2l(I) -> integer_to_list(I). %%% @spec (string()) -> item() %%% @doc Converts an external string representation to an internal %%% blog, article or comment identifier. %%% string_to_id("blogs/" ++ BStr) -> {UStr, I} = one_int(BStr), [blogs, list_to_binary(UStr), I]; string_to_id("ba/" ++ BIStr) -> {UStr, I, J} = two_ints(BIStr), [ba, list_to_binary(UStr), I, J]; string_to_id("bc/" ++ BIStr) -> {UStr, I, J} = two_ints(BIStr), [bc, list_to_binary(UStr), I, J]. one_int(Str) -> {RevS, I} = pick(.lists:reverse(Str), []), {.lists:reverse(RevS), I}. two_ints(Str) -> {Rest1, J} = pick(.lists:reverse(Str), []), {Rest2, I} = pick(Rest1, []), {.lists:reverse(Rest2), I, J}. pick("/" ++ Rest, Acc) -> {Rest, list_to_integer(Acc)}; pick([H|T], Acc) when H >= $0, H =< $9 -> pick(T, [H|Acc]). Otherwise, I too like the {node(), now()} version. BR, Ulf > > The best method I can find for generating a unique id is to use: > {node(), now()} > > Which generates something like: > {foo@REDACTED,{1297,347375,276224}} > > This works great, and is what I use within erlang. > > However, it looks a bit meaningless is likely to confuse users of the non-erlang application that uses it - is there a nice way to turn that into an integer, or series of integers instead? > > E.g. turning the above into: > 3473290872_1297_347375_276224 > (or ideally a single integer) > > would be fine, but I'm not sure how to get an integer from a term. > > Chances are I'll be generating hundreds of thousands to several million of these, so I'm also a bit concerned about efficiency - turning the {node(), now()} tuple into a string would no doubt take up more memory (and some processing time). > > Any suggestions on this? > > -- > Dave Challis > dsc@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From alessandro.sivieri@REDACTED Thu Feb 10 17:05:52 2011 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Thu, 10 Feb 2011 17:05:52 +0100 Subject: Dict size in guards Message-ID: Hi all, is there a good reason for not allowing dict:size() in guards? I mean, it looks exactly like length() for lists, in theory (of course, their implementations are different). -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From dsc@REDACTED Thu Feb 10 17:45:12 2011 From: dsc@REDACTED (Dave Challis) Date: Thu, 10 Feb 2011 16:45:12 +0000 Subject: [erlang-questions] Generating readable unique IDs for use outside erlang In-Reply-To: References: <4D53F785.9030300@ecs.soton.ac.uk> <4D541618.5030906@ecs.soton.ac.uk> Message-ID: On 10/02/11 14:47, Ale wrote: >> >> Any suggestions on this? >> > > Why don't you generate UUID? > http://en.wikipedia.org/wiki/Universally_unique_identifier I mostly wanted to stick to the standard library for speed, I assumed that calling the BIFs now() and node() would be much more efficient than running code to generate a uuid. Memory-wise, a tuple containing {node(), now()} should use up much less space than a 128 bit uuid. But good to know the code is out there, will come in handy someday I'm sure! -- Dave Challis dsc@REDACTED From dan.heidebrecht@REDACTED Thu Feb 10 17:48:08 2011 From: dan.heidebrecht@REDACTED (Daniel Heidebrecht) Date: Thu, 10 Feb 2011 09:48:08 -0700 Subject: [erlang-questions] Generating readable unique IDs for use outside erlang In-Reply-To: References: <4D53F785.9030300@ecs.soton.ac.uk> <4D541618.5030906@ecs.soton.ac.uk> Message-ID: hi, i'm not sure about the performance of now(). from the rabbitmq rabbit_guid.erl inline comments: %% We don't use erlang:now() here because a) it may return %% duplicates when the system clock has been rewound prior to a %% restart, or ids were generated at a high rate (which causes %% now() to move ahead of the system time), and b) it is really %% slow since it takes a global lock and makes a system call. dan. On Thursday, February 10, 2011 at 9:45 AM, Dave Challis wrote: On 10/02/11 14:47, Ale wrote: > > > > > > Any suggestions on this? > > > > Why don't you generate UUID? > > http://en.wikipedia.org/wiki/Universally_unique_identifier > > I mostly wanted to stick to the standard library for speed, I assumed > that calling the BIFs now() and node() would be much more efficient than > running code to generate a uuid. > > Memory-wise, a tuple containing {node(), now()} should use up much less > space than a 128 bit uuid. > > But good to know the code is out there, will come in handy someday I'm sure! > > -- > Dave Challis > dsc@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From jared.nance@REDACTED Thu Feb 10 17:28:30 2011 From: jared.nance@REDACTED (Jared Kofron) Date: Thu, 10 Feb 2011 08:28:30 -0800 Subject: [erlang-questions] Generating readable unique IDs for use outside erlang In-Reply-To: References: <4D53F785.9030300@ecs.soton.ac.uk> Message-ID: <1D911796-2B94-4185-AF8A-527464F126F7@gmail.com> I believe couch uses crypto's rand_bytes to accomplish this... something like to_hex(crypto:rand_bytes(16)) generates a UUID for them. On Feb 10, 2011, at 6:47 AM, Ale wrote: >> >> Any suggestions on this? >> > > Why don't you generate UUID? > http://en.wikipedia.org/wiki/Universally_unique_identifier > > I found this googling: > https://github.com/travis/erlang-uuid/blob/master/uuid.erl > > cheers, > -- > Ale. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From orionqwest@REDACTED Thu Feb 10 18:17:26 2011 From: orionqwest@REDACTED (Nav) Date: Thu, 10 Feb 2011 19:17:26 +0200 Subject: [erlang-questions] TCP flow control problem In-Reply-To: References: <1296460155.5188.3.camel@seasc1137> Message-ID: Hi Bengt and Alain, Many thanks for pointing out my logical error. As I am now rewriting my whole network layer, what would be the best recommended design. My requirement is that the Network Layer should just read data, and pass it on to other OTP component using handle_cast. Network Layer Logic (gen_fsm or gen_server) psuedo code ------------------------------------------------------------------------------------- - Socket in {active, once} mode {to keep flow under control with remote server} - Read available data using handle_info() - Parse message-data using marker, and store rest of message-data (=DataRest) inside State variable of OTP - Set socket to {active, once} mode - Go back to start, read data and add after DataRest, and parse message, and repeat above steps. Since I am aiming to handle very high amount of traffic, would this design suffice? For e.g., - Could handle_cast queue run out of memory, if network layer is pumping messages at faster rate? Any better design to make server faster and be able to handle high capacity? Thanks again, Nav On Mon, Jan 31, 2011 at 7:27 PM, Alain O'Dea wrote: > Bengt is right. ?I suggest using a gen_server to collect the data and look for message boundaries. ?As it collects data it would prepend it to an iodata() in its own State structure. ?When it encounters a message boundary it would cast the data in its State plus the data up to and including the message boundary onto a delegate gen_server for processing. ?The data in the packet after the message boundary would then become the new partial message data in it's State. > > Below is pseudo-code for this: > > -record(state, {partial_message :: iodata(), delegate :: pid()}). > > handle_info({tcp, {data, Data}}, _From, #state{partial_message = Partial} = State) -> > ? ?NextPartial = case boundary_split(Data) of > ? ? ? ?{MsgEnd, NextMsg} -> > ? ? ? ? ? ?tcp_message_handler:handle_message(State#state.delegate, [MsgEnd | Partial]), > ? ? ? ? ? ?NextMsg; > ? ? ? ?Data -> > ? ? ? ? ? ?[Data | Partial] > ? ?end, > ? ?{noreply, State#state{partial_message = NextPartial}. > > > On 2011-01-31, at 4:19, Bengt Kleberg wrote: > >> Greetings, >> >> TCP is a stream protocol. It does not preserve message boundaries. You >> have to handle that yourself. Sleeping one second is one way. When you >> do not sleep, how do you separate the messages? >> >> >> bengt >> >> On Sun, 2011-01-30 at 23:38 +0100, Nav wrote: >>> I have implemented two TCP servers using gen_fsm, and gen_server, respectively. >>> I am using {active, once} option, after getting each message. >>> >>> When I try to send data from one side to another in a loop, the data >>> gets corrupted (first 1 or 2 message arrive ok, but following gets >>> corrupted) when it arrives at other side. >>> >>> However, if I put a timer:sleep(1000) before sending each message, >>> then everything seems to go fast. >>> >>> What seems to go wrong by sending data too fast? Do I need to set some >>> more socket options? >>> >>> Thanks, Nav >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From robert.virding@REDACTED Thu Feb 10 18:27:14 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 10 Feb 2011 17:27:14 +0000 (GMT) Subject: [erlang-questions] Dict size in guards In-Reply-To: Message-ID: <1515729009.429611297358834296.JavaMail.root@zimbra> There is a general restriction on guard tests that no calls to user defined functions are allowed. This irrespective of how simple these functions may be, as there is no way of guaranteeing this at run-time. Robert ----- "Alessandro Sivieri" wrote: > Hi all, > > is there a good reason for not allowing dict:size() in guards? I mean, > it > looks exactly like length() for lists, in theory (of course, their > implementations are different). > > -- > Sivieri Alessandro > alessandro.sivieri@REDACTED > http://www.chimera-bellerofonte.eu/ > http://www.poul.org/ -- Robert Virding, Erlang Solutions Ltd. From alessandro.sivieri@REDACTED Thu Feb 10 18:28:59 2011 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Thu, 10 Feb 2011 18:28:59 +0100 Subject: [erlang-questions] Dict size in guards In-Reply-To: <1515729009.429611297358834296.JavaMail.root@zimbra> References: <1515729009.429611297358834296.JavaMail.root@zimbra> Message-ID: On Thu, Feb 10, 2011 at 6:27 PM, Robert Virding < robert.virding@REDACTED> wrote: > There is a general restriction on guard tests that no calls to user defined > functions are allowed. This irrespective of how simple these functions may > be, as there is no way of guaranteeing this at run-time. > > I know, but this does not change my question, at least about that specific function (which is not user defined, is in stdlib...). -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From robert.virding@REDACTED Thu Feb 10 18:33:32 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 10 Feb 2011 17:33:32 +0000 (GMT) Subject: [erlang-questions] Dict size in guards In-Reply-To: <830245676.429751297359207197.JavaMail.root@zimbra> Message-ID: <913126443.429771297359212064.JavaMail.root@zimbra> ----- "Alessandro Sivieri" wrote: > > On Thu, Feb 10, 2011 at 6:27 PM, Robert Virding < robert.virding@REDACTED > wrote: > There is a general restriction on guard tests that no calls to user defined functions are allowed. This irrespective of how simple these functions may be, as there is no way of guaranteeing this at run-time. > > > I know, but this does not change my question, at least about that specific function (which is not user defined, is in stdlib...). In this case "user-defined" means written in Erlang. And irrespective of who wrote the module and whether it is in the release there are still no guarantees that at run-time this is the code which is run. Robert -- Robert Virding, Erlang Solutions Ltd. From kaiduanx@REDACTED Thu Feb 10 19:43:22 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Thu, 10 Feb 2011 13:43:22 -0500 Subject: [erlang-questions] TCP flow control problem In-Reply-To: References: <1296460155.5188.3.camel@seasc1137> Message-ID: Nav, I have been working on a server with TCP/TLS/UDP support for a while, the following is my experience, 1. For TCP/TLS, spawns one gen_fsm for a connection, and sets the socket as {active, once} first, and then handles incoming packet in handle_info(), parses the packet in the gen_fsm (this is the place where gen_fsm comes into play, state machine is very handy to parse message because TCP/TLS is stream based), after reading a packet from network, sets the socket as {active, once} again. 2. You can accept TCP/TLS connections from multiple processes, this will improve your server's connection acceptance speed. 3. If your TCP/TLS connection is long-live, and idle most of the time, hibernates the gen_fsm to save memory. Cheers, /Kaiduan On Thu, Feb 10, 2011 at 12:17 PM, Nav wrote: > Hi Bengt and Alain, > > Many thanks for pointing out my logical error. > > As I am now rewriting my whole network layer, what would be the best > recommended design. > > My requirement is that the Network Layer should just read data, and > pass it on to other OTP component using handle_cast. > > Network Layer Logic (gen_fsm or gen_server) psuedo code > ------------------------------------------------------------------------------------- > > - Socket in {active, once} mode ?{to keep flow under control with remote server} > - Read available data using handle_info() > - Parse message-data using marker, and store rest of message-data > (=DataRest) inside State variable of OTP > - Set socket to {active, once} mode > - Go back to start, read data and add after DataRest, and parse > message, and repeat above steps. > > > Since I am aiming to handle very high amount of traffic, would this > design suffice? > > For e.g., > - Could handle_cast queue run out of memory, if network layer is > pumping messages at faster rate? > > Any better design to make server faster and be able to handle high capacity? > > > Thanks again, > > Nav > > > On Mon, Jan 31, 2011 at 7:27 PM, Alain O'Dea wrote: >> Bengt is right. ?I suggest using a gen_server to collect the data and look for message boundaries. ?As it collects data it would prepend it to an iodata() in its own State structure. ?When it encounters a message boundary it would cast the data in its State plus the data up to and including the message boundary onto a delegate gen_server for processing. ?The data in the packet after the message boundary would then become the new partial message data in it's State. >> >> Below is pseudo-code for this: >> >> -record(state, {partial_message :: iodata(), delegate :: pid()}). >> >> handle_info({tcp, {data, Data}}, _From, #state{partial_message = Partial} = State) -> >> ? ?NextPartial = case boundary_split(Data) of >> ? ? ? ?{MsgEnd, NextMsg} -> >> ? ? ? ? ? ?tcp_message_handler:handle_message(State#state.delegate, [MsgEnd | Partial]), >> ? ? ? ? ? ?NextMsg; >> ? ? ? ?Data -> >> ? ? ? ? ? ?[Data | Partial] >> ? ?end, >> ? ?{noreply, State#state{partial_message = NextPartial}. >> >> >> On 2011-01-31, at 4:19, Bengt Kleberg wrote: >> >>> Greetings, >>> >>> TCP is a stream protocol. It does not preserve message boundaries. You >>> have to handle that yourself. Sleeping one second is one way. When you >>> do not sleep, how do you separate the messages? >>> >>> >>> bengt >>> >>> On Sun, 2011-01-30 at 23:38 +0100, Nav wrote: >>>> I have implemented two TCP servers using gen_fsm, and gen_server, respectively. >>>> I am using {active, once} option, after getting each message. >>>> >>>> When I try to send data from one side to another in a loop, the data >>>> gets corrupted (first 1 or 2 message arrive ok, but following gets >>>> corrupted) when it arrives at other side. >>>> >>>> However, if I put a timer:sleep(1000) before sending each message, >>>> then everything seems to go fast. >>>> >>>> What seems to go wrong by sending data too fast? Do I need to set some >>>> more socket options? >>>> >>>> Thanks, Nav >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From overminddl1@REDACTED Thu Feb 10 22:28:50 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Thu, 10 Feb 2011 14:28:50 -0700 Subject: [erlang-questions] Dict size in guards In-Reply-To: <913126443.429771297359212064.JavaMail.root@zimbra> References: <830245676.429751297359207197.JavaMail.root@zimbra> <913126443.429771297359212064.JavaMail.root@zimbra> Message-ID: On Thu, Feb 10, 2011 at 10:33 AM, Robert Virding wrote: > ----- "Alessandro Sivieri" wrote: >> >> On Thu, Feb 10, 2011 at 6:27 PM, Robert Virding < robert.virding@REDACTED > wrote: >> > > There is a general restriction on guard tests that no calls to user defined functions are allowed. This irrespective of how simple these functions may be, as there is no way of guaranteeing this at run-time. >> >> >> I know, but this does not change my question, at least about that specific function (which is not user defined, is in stdlib...). > > In this case "user-defined" means written in Erlang. And irrespective of who wrote the module and whether it is in the release there are still no guarantees that at run-time this is the code which is run. Would not the Abstract Patterns and other enhancements put through be Richard A. O'Keefe help this issue a great deal? From jameschurchman@REDACTED Fri Feb 11 02:15:04 2011 From: jameschurchman@REDACTED (James Churchman) Date: Fri, 11 Feb 2011 01:15:04 +0000 Subject: [erlang-questions] global module limits In-Reply-To: References: Message-ID: <09B385E0-1C87-401F-9C14-93D28A8B9117@gmail.com> Download gproc, find some suitable hardware for you (does ec2 fit? ) benchmark it on 1 2 4 & 8 connected nodes.. and write a blog post to share all the finding :-) On 7 Feb 2011, at 17:07, Paolo Negri wrote: > Dear list > > I've been reading various opinions about the global module and I'd > like some light to be shed on its limits and their underlying reasons. > Particularly I'd like to understand what are its limit relative to > register a big number of processes. > > How do the following factors affect the global module performance > > - number of registered processes at a given time, is this a problem if > the registered processes are in the order of 1K 10K or 100K > - registration rate, what if I register 10 100 or 1000 processes per seconds > - lookup rate, this is how often the global name registry is queried > - de-registration rate, symmetrical to the question above, 10 100 or > 1000 registered processes terminate per second > - overall number of registered processes over time: assuming that in a > given second processes are registered at the same rate at which old > registered processes terminate, the number of currently registered > processes is constant (say 10K) but overtime I might have registered > 1000K different process names, does the "register history" get cleaned > or it'll stick around in the global environment? > - cluster replication overhead, how does the number of nodes in a > system affect the performance of the global registration facility? > > On a side note I would be very interested in people willing to share > experiences in using in production the gproc[1] project using it's > global facility to handle in the region of 100K to 1000K processes. > > [1] https://github.com/esl/gproc > > Thanks in advance for your answers. > > Paolo > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ok@REDACTED Fri Feb 11 02:54:33 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 11 Feb 2011 14:54:33 +1300 Subject: [erlang-questions] upgrading erlang servers In-Reply-To: References: Message-ID: On 10/02/2011, at 6:06 PM, Param Jyothi Reddy wrote: > Hi guys, > I am new to erlang and was reading "Programming Erlang" by Joe > Armstrong and one remark in the book surprised me. > > Pg 173: "And same version of erlang [on all nodes]. If you dont do > this, you'll get serious and mysterious errors.". There's one possible reason for this. When you send terms from processes on one node to processes on another, they are sent in a binary format. This format has changed from time to time, and could certainly be improved further. See the "minor_version" option in the documentation for erlang:term_to_binary/2. Since mnesia/ets data also use this format, it's not something anyone wants to change _often_. From ok@REDACTED Fri Feb 11 03:02:57 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 11 Feb 2011 15:02:57 +1300 Subject: [erlang-questions] Dict size in guards In-Reply-To: References: Message-ID: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> On 11/02/2011, at 5:05 AM, Alessandro Sivieri wrote: > Hi all, > > is there a good reason for not allowing dict:size() in guards? Yes. Who knows what dict:size() actually does? Guards cannot and should not call functions. > I mean, it > looks exactly like length() for lists, in theory (of course, their > implementations are different). The inclusion of length/1 in guard tests has always been more than a bit dodgy. It's better taste to avoid it. Not least because if you do something like f(L) when length(L) > 10 -> long_case(L); when length(L) > 5 -> medium_case(L); when true -> short_case(L). doesn't do what people often think. I've several times been caught by that. f(L) -> N = length(L), if N > 10 -> long_case(L) ; N > 5 -> medium_case(L) ; true -> short_case(L) end. is safer and only evaluates the length once. A style warning "length/1 in guard" would be nice to have. From ok@REDACTED Fri Feb 11 03:04:25 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 11 Feb 2011 15:04:25 +1300 Subject: [erlang-questions] Dict size in guards In-Reply-To: References: <1515729009.429611297358834296.JavaMail.root@zimbra> Message-ID: <46200F92-0502-4FF6-A79D-F4EC154DE290@cs.otago.ac.nz> On 11/02/2011, at 6:28 AM, Alessandro Sivieri wrote: > On Thu, Feb 10, 2011 at 6:27 PM, Robert Virding < > robert.virding@REDACTED> wrote: > >> There is a general restriction on guard tests that no calls to user defined >> functions are allowed. This irrespective of how simple these functions may >> be, as there is no way of guaranteeing this at run-time. >> >> > I know, but this does not change my question, at least about that specific > function (which is not user defined, is in stdlib...). I have been known to patch the occasional stdlib function... From mapandfold@REDACTED Fri Feb 11 04:30:58 2011 From: mapandfold@REDACTED (MapAndFold) Date: Fri, 11 Feb 2011 11:30:58 +0800 Subject: how to check if VM is in detached mode Message-ID: Hi all, Is it possible to check if VM is in detached mode at run-time? My use case: When wx window is closed: If in detached mode, then erlang:halt(). -- From james@REDACTED Fri Feb 11 04:37:15 2011 From: james@REDACTED (James Aimonetti) Date: Thu, 10 Feb 2011 19:37:15 -0800 Subject: gen_server call when return value isn't wanted Message-ID: <4D54AEEB.3040900@2600hz.com> I have a gen_server function, that as part of the body does a DB save, and returns the saved version of the data. save_data(Data) -> gen_server:call(?SERVER, {save_data, Data}). Occasionally I don't care about the result of the save and was wondering about couple of ideas. Is there a difference to the compiler in doing ... my_server:save_data(Stuff), ... versus ... _ = my_server:save_data(Stuff), ... or should I just create a version of save_data/1 that uses cast instead? Perhaps spawn the save_data/1 call? -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 From bengt.kleberg@REDACTED Fri Feb 11 07:40:35 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 11 Feb 2011 07:40:35 +0100 Subject: [erlang-questions] gen_server call when return value isn't wanted In-Reply-To: <4D54AEEB.3040900@2600hz.com> References: <4D54AEEB.3040900@2600hz.com> Message-ID: <1297406435.5101.10.camel@seasc1137> Greetings, There is no difference in the time it takes to complete the function call if you have no assignment or if you assign to '_'. If it is not the time you can save, by not waiting for the gen_fsm:call/2 to return that interest you, I suggest using no assignment. If you want to continue asynchronous in the (now) calling process you will need a new function using cast, or as you suggest, spawning a new process before calling the old function. I would spawn if this only happens once, but have a new function if it is done more than once. bengt On Fri, 2011-02-11 at 04:37 +0100, James Aimonetti wrote: > I have a gen_server function, that as part of the body does a DB save, > and returns the saved version of the data. > > save_data(Data) -> > gen_server:call(?SERVER, {save_data, Data}). > > Occasionally I don't care about the result of the save and was wondering > about couple of ideas. > > Is there a difference to the compiler in doing > > ... > my_server:save_data(Stuff), > ... > > versus > ... > _ = my_server:save_data(Stuff), > ... > > or should I just create a version of save_data/1 that uses cast instead? > Perhaps spawn the save_data/1 call? > From ulf.wiger@REDACTED Fri Feb 11 09:03:17 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 11 Feb 2011 08:03:17 +0000 Subject: [erlang-questions] Dict size in guards In-Reply-To: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> References: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> Message-ID: <3FF17835-13E7-4DBA-A6FF-8DBBC2FC3916@erlang-solutions.com> On 11 Feb 2011, at 02:02, Richard O'Keefe wrote: > The inclusion of length/1 in guard tests has always been more than > a bit dodgy. It's better taste to avoid it. Not least because > if you do something like > > f(L) > when length(L) > 10 -> long_case(L); > when length(L) > 5 -> medium_case(L); > when true -> short_case(L). > > doesn't do what people often think. I've several times been > caught by that. Also, as pointed out by Kostis in his Tidier presentations, a fairly common use is f(L) when length(L) == 0 -> ? which of course has to traverse the whole (potentially very long) list. BR, Ulf Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From tony@REDACTED Fri Feb 11 08:35:36 2011 From: tony@REDACTED (Tony Rogvall) Date: Fri, 11 Feb 2011 08:35:36 +0100 Subject: [erlang-questions] Dict size in guards In-Reply-To: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> References: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> Message-ID: <81190123-346F-4951-83A9-A339FA64F3CC@rogvall.se> On 11 feb 2011, at 03.02, Richard O'Keefe wrote: > > On 11/02/2011, at 5:05 AM, Alessandro Sivieri wrote: > >> Hi all, >> >> is there a good reason for not allowing dict:size() in guards? > > Yes. Who knows what dict:size() actually does? > Guards cannot and should not call functions. > >> I mean, it >> looks exactly like length() for lists, in theory (of course, their >> implementations are different). > > The inclusion of length/1 in guard tests has always been more than > a bit dodgy. It's better taste to avoid it. Not least because > if you do something like > > f(L) > when length(L) > 10 -> long_case(L); > when length(L) > 5 -> medium_case(L); > when true -> short_case(L). > > doesn't do what people often think. I've several times been > caught by that. > > f(L) -> > N = length(L), > if N > 10 -> long_case(L) > ; N > 5 -> medium_case(L) > ; true -> short_case(L) > end. > > is safer and only evaluates the length once. > One way of avoiding length in this case is of course: f(L) -> case L of [_,_,_,_,_,_,_,_,_,_,_|_] -> long_case(L); [_,_,_,_,_,_|_] -> medium_case(L); _ -> short_case(L) end. May be the compiler could take care of the "simple" cases like this, and then issue a warning if it can not do the transformation ? BTW. The compiler needs a fix to handle this transform in general. It uses way to many registers. Maybe it's a problem with compiling guards? > A style warning "length/1 in guard" would be nice to have. > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > "Have run Make so many times I dunno what's installed anymore" From tony@REDACTED Fri Feb 11 09:54:37 2011 From: tony@REDACTED (Tony Rogvall) Date: Fri, 11 Feb 2011 09:54:37 +0100 Subject: [erlang-questions] Dict size in guards In-Reply-To: <3FF17835-13E7-4DBA-A6FF-8DBBC2FC3916@erlang-solutions.com> References: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> <3FF17835-13E7-4DBA-A6FF-8DBBC2FC3916@erlang-solutions.com> Message-ID: <10D1489F-CDFB-4C6E-93F6-8FB961910CF3@rogvall.se> And in general guard you can use something like this: f(L) -> if is_list(tl(tl(tl(tl(tl(tl(tl(tl(tl(tl(tl(L)))))))))))) -> long_case(L); is_list(tl(tl(tl(tl(tl(tl(L))))))) -> medium_case(L); true -> short_case(L) end. It compiles a LOT better than the corresponding pattern version. /Tony On 11 feb 2011, at 09.03, Ulf Wiger wrote: > > On 11 Feb 2011, at 02:02, Richard O'Keefe wrote: > >> The inclusion of length/1 in guard tests has always been more than >> a bit dodgy. It's better taste to avoid it. Not least because >> if you do something like >> >> f(L) >> when length(L) > 10 -> long_case(L); >> when length(L) > 5 -> medium_case(L); >> when true -> short_case(L). >> >> doesn't do what people often think. I've several times been >> caught by that. > > > Also, as pointed out by Kostis in his Tidier presentations, a fairly > common use is > > f(L) when length(L) == 0 -> ? > > which of course has to traverse the whole (potentially very long) list. > > BR, > Ulf > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > "Have run Make so many times I dunno what's installed anymore" From michael.eugene.turner@REDACTED Fri Feb 11 10:40:12 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Fri, 11 Feb 2011 18:40:12 +0900 Subject: [erlang-questions] Dict size in guards In-Reply-To: <10D1489F-CDFB-4C6E-93F6-8FB961910CF3@rogvall.se> References: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> <3FF17835-13E7-4DBA-A6FF-8DBBC2FC3916@erlang-solutions.com> <10D1489F-CDFB-4C6E-93F6-8FB961910CF3@rogvall.se> Message-ID: FWIW, Doug Clark found that the average length of lists in large, mature lisp applications was about four. http://portal.acm.org/citation.cfm?id=359423.359427 Even if f() [see below] were statistically tuned, however, the basic problem remains -- you wouldn't be able to use that f() function in a guard, and you CAN use length() in a guard, despite a possible compromise of the "soft real-time" promise. How hard would it be to make length() execute in (short) constant time, e.g., with an actual length attribute? I know this implies consing up a length for each list returned by any evaluation, so (except for length()-heavy list computation) it would tend to hurt performance. But if the compiler has good live-variable analysis, it might be possible to avoid computing list lengths (and storing them) in the overwhelming majority of cases. Some such data flow analysis might also (with a little work) do double-duty by making more opportunistic garbage collection of temporary lists possible. For applications where space is more important than time, the extra space required for storing list lengths could then be *much* more than made up for, with cdr-coding and other time-honored tricks for reducing list bulk. -michael turner On Fri, Feb 11, 2011 at 5:54 PM, Tony Rogvall wrote: > And in general guard you can use something like this: > > f(L) -> > if is_list(tl(tl(tl(tl(tl(tl(tl(tl(tl(tl(tl(L)))))))))))) -> > long_case(L); > is_list(tl(tl(tl(tl(tl(tl(L))))))) -> medium_case(L); > true -> short_case(L) > end. > > It compiles a LOT better than the corresponding pattern version. > > /Tony > > > On 11 feb 2011, at 09.03, Ulf Wiger wrote: > > > > > On 11 Feb 2011, at 02:02, Richard O'Keefe wrote: > > > >> The inclusion of length/1 in guard tests has always been more than > >> a bit dodgy. It's better taste to avoid it. Not least because > >> if you do something like > >> > >> f(L) > >> when length(L) > 10 -> long_case(L); > >> when length(L) > 5 -> medium_case(L); > >> when true -> short_case(L). > >> > >> doesn't do what people often think. I've several times been > >> caught by that. > > > > > > Also, as pointed out by Kostis in his Tidier presentations, a fairly > > common use is > > > > f(L) when length(L) == 0 -> ? > > > > which of course has to traverse the whole (potentially very long) list. > > > > BR, > > Ulf > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > > http://erlang-solutions.com > > > > > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > "Have run Make so many times I dunno what's installed anymore" > > From roberto@REDACTED Fri Feb 11 12:11:36 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 11 Feb 2011 13:11:36 +0200 Subject: supervise a python process Message-ID: dear list, i need to integrate existing python code with a SOA erlang layer, which provides some distribution facilities. i'm thinking to use BERT to enable communication between the erlang layer and python. the main point to not use ports is that, afaik, that if a python port crashes the whole erlang VM goes down, and the point of using erlang supervisors to provide fault-tolerancy gets lost. therefore, i'm fishing for suggestions on how to enable erlang to somehow provide 'supervisor' functionalities for a python process. the first thing that came to my mind is opening a TCP connection for the BERT communication between an erlang and a python module, so that in case the connection gets lost, the python module is instructed to exit while the erlang process restarts the python process (probably with a os:cmd/1 command or similar). are there already attempts at this i might not be aware of, or any additional ideas you might recommend/consider? any input welcome. thank you, r. From dima@REDACTED Fri Feb 11 13:17:01 2011 From: dima@REDACTED (Dmitry Vasiliev) Date: Fri, 11 Feb 2011 15:17:01 +0300 Subject: [erlang-questions] supervise a python process In-Reply-To: References: Message-ID: <4D5528BD.1080705@hlabs.org> 11.02.2011 14:11, Roberto Ostinelli ?????: > i need to integrate existing python code with a SOA erlang layer, which > provides some distribution facilities. > > i'm thinking to use BERT to enable communication between the erlang layer > and python. the main point to not use ports is that, afaik, that if a python > port crashes the whole erlang VM goes down, and the point of using erlang > supervisors to provide fault-tolerancy gets lost. It's just not true. Only the process which created the port (the connected process of the port which is linked to the port) is terminates when port is closed. So you can easily use supervisors with ports. You can try ErlPort (http://erlport.org/) for ports based integration of Erlang and Python. -- Dmitry Vasiliev http://hlabs.org http://twitter.com/hdima From karlsson.rm@REDACTED Fri Feb 11 13:20:40 2011 From: karlsson.rm@REDACTED (Mikael Karlsson) Date: Fri, 11 Feb 2011 04:20:40 -0800 (PST) Subject: Immediately reflect sudden changes in system time Message-ID: Hi, according to the documentation erlang:now/0 will not immediately reflect sudden changes in the system time, but slowly adjust towards the new system time. (one percent adjustments), unless the Erlang VM is started with the +c option. Is there a way to to make an immediate change of erlang:now/0 without restarting the VM even if it has been started without the +c flag? Some undocumented erlang function call or sending SIGHUP (kill -1) to the beam process...? Mikael From igarai@REDACTED Fri Feb 11 13:30:15 2011 From: igarai@REDACTED (=?UTF-8?B?ScOxYWtpIEdhcmF5?=) Date: Fri, 11 Feb 2011 09:30:15 -0300 Subject: [erlang-questions] supervise a python process In-Reply-To: <4D5528BD.1080705@hlabs.org> References: <4D5528BD.1080705@hlabs.org> Message-ID: On Fri, Feb 11, 2011 at 09:17, Dmitry Vasiliev wrote: > 11.02.2011 14:11, Roberto Ostinelli ?????: >> >> i need to integrate existing python code with a SOA erlang layer, which >> provides some distribution facilities. >> >> i'm thinking to use BERT to enable communication between the erlang layer >> and python. the main point to not use ports is that, afaik, that if a >> python >> port crashes the whole erlang VM goes down, and the point of using erlang >> supervisors to provide fault-tolerancy gets lost. > > It's just not true. Only the process which created the port (the connected > process of the port which is linked to the port) is terminates when port is > closed. So you can easily use supervisors with ports. Perhaps there is some confusion regarding ports and linked-in drivers? cf. Ports vs. Port Drivers: http://www.erlang.org/doc/tutorial/c_port.html http://www.erlang.org/doc/tutorial/c_portdriver.html -- I?aki Garay. From alain.odea@REDACTED Fri Feb 11 13:35:29 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Fri, 11 Feb 2011 09:05:29 -0330 Subject: [erlang-questions] supervise a python process In-Reply-To: References: Message-ID: <9FFE3EE9-0630-465D-B1EC-3DEF2759824F@gmail.com> On 2011-02-11, at 7:41, Roberto Ostinelli wrote: > dear list, > > i need to integrate existing python code with a SOA erlang layer, which > provides some distribution facilities. > > i'm thinking to use BERT to enable communication between the erlang layer > and python. the main point to not use ports is that, afaik, that if a python > port crashes the whole erlang VM goes down, and the point of using erlang > supervisors to provide fault-tolerancy gets lost. > > therefore, i'm fishing for suggestions on how to enable erlang to somehow > provide 'supervisor' functionalities for a python process. the first thing > that came to my mind is opening a TCP connection for the BERT communication > between an erlang and a python module, so that in case the connection gets > lost, the python module is instructed to exit while the erlang process > restarts the python process (probably with a os:cmd/1 command or similar). > > are there already attempts at this i might not be aware of, or any > additional ideas you might recommend/consider? > > any input welcome. > > thank you, > > r. Hi Roberto: It seems like Nokia's http://discoproject.org/ might be useful. It lets Erlang schedule and manage Python jobs in a cluster. Cheers, Alain From ebegumisa@REDACTED Fri Feb 11 14:07:08 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sat, 12 Feb 2011 00:07:08 +1100 Subject: [erlang-questions] supervise a python process In-Reply-To: References: Message-ID: On Fri, 11 Feb 2011 22:11:36 +1100, Roberto Ostinelli wrote: > dear list, > > i need to integrate existing python code with a SOA erlang layer, which > provides some distribution facilities. > > i'm thinking to use BERT to enable communication between the erlang layer > and python. the main point to not use ports is that, afaik, that if a > python > port crashes the whole erlang VM goes down, and the point of using erlang > supervisors to provide fault-tolerancy gets lost. > Not exactly. There are two "types" of ports: a plain port and a port driver (AKA linked-in driver). A *port* runs in it's own OS process - it will not crash the VM if it crashes itself. A *port driver* is linked-in to the VM - if it crashes it will take the VM down with it. You could take a second look at using a port for your purposes. > therefore, i'm fishing for suggestions on how to enable erlang to somehow > provide 'supervisor' functionalities for a python process. From Erlang, ports look [kinda] like linked processes. If a port crashes, you receive an {'EXIT',Port,Why} message and you can restart it. Likewise if the Erlang process that owns the port dies on the Erlang side, the python OS process will be terminated*. These fit nicely into enabling the "supervisor functionalities" you desire. *If the port hangs however, that's a different story. > the first thing > that came to my mind is opening a TCP connection for the BERT > communication > between an erlang and a python module, so that in case the connection > gets > lost, the python module is instructed to exit while the erlang process > restarts the python process (probably with a os:cmd/1 command or > similar). > There are two questions you should treat separately here... 1. How to best to do the inter-OS-process IPC 2. How best to encode and decode messages between OS processes once you've chosen the IPC technique. For question 1) You can use a port (don't immediately dismiss this), or you could indeed use TCP sockets. Either technique can be successfully worked into your supervision tree. Each technique has advantages over the other. Also, from my experience, both techniques may require you to implement some sort of heartbeat for long-running external code. The ultimate answer will be a mixture of application specificity and taste. For question 2) You could use the External Term Format, you could indeed use BERT, you could roll-your-own format. Personally, I prefer using UBF because it's compact, it comes with a contract checker, and most IPC I do is FSM-like. But if you've decided you like BERT, heck, use BERT. > are there already attempts at this i might not be aware of, or any > additional ideas you might recommend/consider? Have a look at UBF. Norton's implementation is pretty flexible (it can even use JSON for message encoding)... http://norton.github.com/ubf/ubf-user-guide.en.html - Edmond - > > any input welcome. > > thank you, > > r. -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From gleber.p@REDACTED Fri Feb 11 14:39:05 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 11 Feb 2011 14:39:05 +0100 Subject: [erlang-questions] supervise a python process In-Reply-To: <4D5528BD.1080705@hlabs.org> References: <4D5528BD.1080705@hlabs.org> Message-ID: If you don't need some blazing performance and need a quick solution, then ErlPort is a great choice. So +1 On 11 Feb 2011 13:20, "Dmitry Vasiliev" wrote: > 11.02.2011 14:11, Roberto Ostinelli ?????: >> i need to integrate existing python code with a SOA erlang layer, which >> provides some distribution facilities. >> >> i'm thinking to use BERT to enable communication between the erlang layer >> and python. the main point to not use ports is that, afaik, that if a python >> port crashes the whole erlang VM goes down, and the point of using erlang >> supervisors to provide fault-tolerancy gets lost. > > It's just not true. Only the process which created the port (the > connected process of the port which is linked to the port) is terminates > when port is closed. So you can easily use supervisors with ports. > > You can try ErlPort (http://erlport.org/) for ports based integration of > Erlang and Python. > > -- > Dmitry Vasiliev > http://hlabs.org > http://twitter.com/hdima > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From norton@REDACTED Fri Feb 11 15:19:44 2011 From: norton@REDACTED (Joseph Wayne Norton) Date: Fri, 11 Feb 2011 23:19:44 +0900 Subject: [erlang-questions] supervise a python process In-Reply-To: References: <4D5528BD.1080705@hlabs.org> Message-ID: <678594F9-4B0B-4960-ACFC-BEFD3D3CC257@lovely.email.ne.jp> For Python/Erlang integration, I have had success with the following approaches: - py_interface http://www.lysator.liu.se/~tab/erlang/py_interface/ - Python EBF client https://github.com/norton/ubf/blob/master/priv/python/pyebf.py - Python JSON RPC client https://github.com/norton/ubf/blob/master/priv/python/jsonrpc.py The latter two methods haven't been documented yet. Though, there is some general UBF documentation in place (http://norton.github.com/ubf/ubf-user-guide.en.html). On 2011/02/11, at 22:39, Gleb Peregud wrote: > If you don't need some blazing performance and need a quick solution, then > ErlPort is a great choice. So +1 > On 11 Feb 2011 13:20, "Dmitry Vasiliev" wrote: >> 11.02.2011 14:11, Roberto Ostinelli ?????: >>> i need to integrate existing python code with a SOA erlang layer, which >>> provides some distribution facilities. >>> >>> i'm thinking to use BERT to enable communication between the erlang layer >>> and python. the main point to not use ports is that, afaik, that if a > python >>> port crashes the whole erlang VM goes down, and the point of using erlang >>> supervisors to provide fault-tolerancy gets lost. >> >> It's just not true. Only the process which created the port (the >> connected process of the port which is linked to the port) is terminates >> when port is closed. So you can easily use supervisors with ports. >> >> You can try ErlPort (http://erlport.org/) for ports based integration of >> Erlang and Python. >> >> -- >> Dmitry Vasiliev >> http://hlabs.org >> http://twitter.com/hdima >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> Joseph Wayne Norton norton@REDACTED From dmercer@REDACTED Fri Feb 11 15:35:01 2011 From: dmercer@REDACTED (David Mercer) Date: Fri, 11 Feb 2011 08:35:01 -0600 Subject: [erlang-questions] upgrading erlang servers In-Reply-To: References: Message-ID: <000e01cbc9f8$e04e2080$a0ea6180$@com> On Wednesday, February 09, 2011, Param Jyothi Reddy wrote: > I am new to erlang and was reading "Programming Erlang" by Joe > Armstrong and one remark in the book surprised me. > > Pg 173: "And same version of erlang [on all nodes]. If you dont do > this, you'll get serious and mysterious errors.". Not sure how true > this is currently, however if it is, how do you upgrade cluster of > nodes running X version of erlang to Y without disrupting service? I > would have assumed that one can upgrade subset at a time which would > also provide a vehicle to test version Y on small subset before > committing the full cluster. But if this comment is true then one > would have to upgrade cluster of nodes in an all or nothing fashion. > > There are other ways to upgrade by having two clusters and switching > load between them using some protocol. Is that how it is meant to be > used? Am i missing something obvious? This is a good question, which relates not only to upgrades of the Erlang runtime system, but also of any code that changes its data structures: How do you upgrade a distributed system when the interface changes? I asked a similar question in October of '07 (http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:30311:200710:fanenpeabclnldcl aikd), but your question is even more intractable than mine, since yours cannot be solved by code tricks. If you need simultaneous upgrade of all 3,000 nodes, especially when confounded by some of the geopolitical considerations described in , then I'm not sure there is a practical solution, which is why ten years from now "Pan-Global Worldwide Consolidated" will still be running Erlang R3A1. Cheers, DBM From dmercer@REDACTED Fri Feb 11 15:35:01 2011 From: dmercer@REDACTED (David Mercer) Date: Fri, 11 Feb 2011 08:35:01 -0600 Subject: [erlang-questions] Dict size in guards In-Reply-To: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> References: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> Message-ID: <000d01cbc9f8$df325040$9d96f0c0$@com> On Thursday, February 10, 2011, Richard O'Keefe wrote: > Guards cannot and should not call functions. ^^^^^^^^^^^^^^ Is the "should not" because of side-effects in Erlang functions? If Erlang functions were pure, would you remove the "and should not" from your statement? I realize there may be issues of performance if they are allowed, but that would be up to the programmer, I suppose. Does Haskell permit user-defined (non-monadic) functions in guards? Just asking from a language theory point of view, since I know you're an expert in this area. Thanks, ROK. Cheers, DBM From raphael.korsoski@REDACTED Fri Feb 11 16:29:09 2011 From: raphael.korsoski@REDACTED (Raphael Korsoski) Date: Fri, 11 Feb 2011 16:29:09 +0100 Subject: [erlang-questions] Dict size in guards In-Reply-To: <000d01cbc9f8$df325040$9d96f0c0$@com> References: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> <000d01cbc9f8$df325040$9d96f0c0$@com> Message-ID: Hi, To my understanding, Haskell guards are very different from Erlang guards. Haskell compiles guards to case statements (afaik), while Erlang evaluates them for selection at runtime. In Haskell we can write: f x | my_length(x) > 3 = "longer than 3" | otherwise = "shorter than or equal to 3" where my_length = ?a ? which does contain a user-defined non-monadic funtion. In GHC it's also possible to do binding within guards, like so: g x | s ? my_length(x) , s > 3 = "longer than 3" | otherwise = "shorter than or equal to 3" where my_length = ?a ? Hope that answers something of your question. BR, Raphael K -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of David Mercer Sent: den 11 februari 2011 15:35 To: 'Richard O'Keefe' Cc: 'Erlang Questions' Subject: RE: [erlang-questions] Dict size in guards On Thursday, February 10, 2011, Richard O'Keefe wrote: > Guards cannot and should not call functions. ^^^^^^^^^^^^^^ Is the "should not" because of side-effects in Erlang functions? If Erlang functions were pure, would you remove the "and should not" from your statement? I realize there may be issues of performance if they are allowed, but that would be up to the programmer, I suppose. Does Haskell permit user-defined (non-monadic) functions in guards? Just asking from a language theory point of view, since I know you're an expert in this area. Thanks, ROK. Cheers, DBM ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From alessandro.sivieri@REDACTED Fri Feb 11 16:47:19 2011 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Fri, 11 Feb 2011 16:47:19 +0100 Subject: [erlang-questions] Dict size in guards In-Reply-To: References: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> <000d01cbc9f8$df325040$9d96f0c0$@com> Message-ID: Well, I think guards are more elegant than case/if statements; but the Erlang limitation forces you to use the latter, that's all. I mean, it is not a problem to get the dictionary length as the first operation of a function and create an if statement, but... -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From kuleshovmail@REDACTED Fri Feb 11 17:50:48 2011 From: kuleshovmail@REDACTED (shk) Date: Fri, 11 Feb 2011 08:50:48 -0800 (PST) Subject: Split string in nested list Message-ID: <1297443048857-3301595.post@n4.nabble.com> Hello, I have list of next view: [["admin@REDACTED"],[localhost@REDACTED],[user@REDACTED]] How can i split by '@" this list? I need following: [["admin", "localhost"], ["localhost", "localhost"], [user, localhost]] Thank you. -- View this message in context: http://erlang.2086793.n4.nabble.com/Split-string-in-nested-list-tp3301595p3301595.html Sent from the Erlang Questions mailing list archive at Nabble.com. From jesper.louis.andersen@REDACTED Fri Feb 11 18:02:40 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 11 Feb 2011 18:02:40 +0100 Subject: [erlang-questions] Split string in nested list In-Reply-To: <1297443048857-3301595.post@n4.nabble.com> References: <1297443048857-3301595.post@n4.nabble.com> Message-ID: On Fri, Feb 11, 2011 at 17:50, shk wrote: > I have list of next view: > > [["admin@REDACTED"],[localhost@REDACTED],[user@REDACTED]] I assume you mean that localhost@REDACTED is a string and not an atom in the following: 5> [string:tokens(S, "@") || [S] <- [["admin@REDACTED"],["localhost@REDACTED"],["user@REDACTED"]]]. [["admin","localhost"], ["localhost","localhost"], ["user","localhost"]] 6> But you may want to pull out the underlying strings out of the lists first depending on what you are trying to do. In the above we use a list comprehension under the *assumption* that its elements are of the form [S] for some S which is a string. It will fail silently if they are not and skip the element. Also string:tokens/2 is under the assumption there are no double occurrence of an @ character in the string. -- J. From kuleshovmail@REDACTED Fri Feb 11 18:07:51 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Fri, 11 Feb 2011 17:07:51 +0000 Subject: [erlang-questions] Split string in nested list In-Reply-To: References: <1297443048857-3301595.post@n4.nabble.com> Message-ID: Hello Jesper Louis Andersen. Thank you very much for your answer, you very helped me. From singpor@REDACTED Fri Feb 11 18:08:45 2011 From: singpor@REDACTED (Weerasak Chongnguluam) Date: Sat, 12 Feb 2011 00:08:45 +0700 Subject: [erlang-questions] Split string in nested list In-Reply-To: <1297443048857-3301595.post@n4.nabble.com> References: <1297443048857-3301595.post@n4.nabble.com> Message-ID: <1297444125.2991.5.camel@iporsut> Hi, You can use lists:map with string:tokens, like this. lists:map(fun(URL) -> if is_atom(URL) -> string:tokens(atom_to_list(URL),"@"); true -> string:tokens(URL,"@") end end,[["admin@REDACTED"],[localhost@REDACTED],[user@REDACTED]] ). On Fri, 2011-02-11 at 08:50 -0800, shk wrote: > Hello, > > I have list of next view: > > [["admin@REDACTED"],[localhost@REDACTED],[user@REDACTED]] > > How can i split by '@" this list? > > I need following: > > [["admin", "localhost"], ["localhost", "localhost"], [user, localhost]] > > Thank you. From jameschurchman@REDACTED Fri Feb 11 18:10:02 2011 From: jameschurchman@REDACTED (James Churchman) Date: Fri, 11 Feb 2011 17:10:02 +0000 Subject: [erlang-questions] Dict size in guards In-Reply-To: References: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> <000d01cbc9f8$df325040$9d96f0c0$@com> Message-ID: <46B9BBC8-4AFB-42B8-9751-F46DCBEA034F@gmail.com> f(L) when length(L) > 10 -> long_case(L); when length(L) > 5 -> medium_case(L); when true -> short_case(L). (which is subtly wrong, as its misses the two following f(L)'s :-) ) can also be written like : f(L)->f(L,length(L)). f(L,Len) when Len > 10 -> long_case(L); f(L,Len) when Len > 5 -> medium_case(L); f(L,_) -> short_case(L). which i think i personally prefer.... and if the list L is only needed for length it gets even shorter (tho f/2 needs renaming if becoming f/1 ) also you could consider using an orddict, the api is identical to the dict, but it consists of a normal list so both of the above code works with out modification. There will obviously just be a lookup time difference for large lists (eg ones that are more than "4" :-) ) http://www.erlang.org/doc/man/orddict.html and if using normal dict, just replace length with dict:size() in the 2nd code Also why does the compiler do a length check for every time you use length in a guard ?? I thought it did not do repetitive pattern matches that are "the same", so surly repetitive guard would also be similarly optimised? also "hacks" like [_,_,_,_,_,_,_,_,_,_,_|_] look very clever, and i guess for very long lists have a speed benefit, but again could the compiler not realise that when you put "when Len > 10" it only needs to check that the list is longer than 10? not get the genuine numeric length and then numeric compare to 10. That seems a basic optimisation that could be made also, why does is_list(tl(tl(tl(tl(tl(tl(L))))))) compile better?? it this true?? it looks really nasty! On 11 Feb 2011, at 15:47, Alessandro Sivieri wrote: > Well, I think guards are more elegant than case/if statements; but the > Erlang limitation forces you to use the latter, that's all. > I mean, it is not a problem to get the dictionary length as the first > operation of a function and create an if statement, but... > > -- > Sivieri Alessandro > alessandro.sivieri@REDACTED > http://www.chimera-bellerofonte.eu/ > http://www.poul.org/ From mevans@REDACTED Fri Feb 11 18:14:59 2011 From: mevans@REDACTED (Evans, Matthew) Date: Fri, 11 Feb 2011 12:14:59 -0500 Subject: Mnesia Fragmentation - Adding a new fragment Message-ID: Hi, When I add a new node with a new fragment to an existing fragmented table the documentation says: {add_frag, NodesOrDist} Adds one new fragment to a fragmented table. All records in one of the old fragments will be rehashed and about half of them will be moved to the new (last) fragment. All other fragmented tables, which refers to this table in their foreign key, will automatically get a new fragment, and their records will also be dynamically rehashed in the same manner as for the main table. The question is that what happens to db queries during this period? If they use the new hash on the key to find the fragment then during the rehashing period lookups will fail (likewise if they use the old hash). The other question relates to backups of fragments (the n_disc_copies option). Can we guarantee that this backup fragment will be on a different node (if many nodes are available) to its mate? Thanks Matt From banibrata.dutta@REDACTED Fri Feb 11 18:27:42 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Fri, 11 Feb 2011 22:57:42 +0530 Subject: Any work happing on ErlyBird, the Erlang plugin for Netbeans ? In-Reply-To: References: Message-ID: Strangly, I can see a response to this query on the trapexit.org sidebar, but never got anything in my mailbox! even checked spam folder and deleted mails. Anyhow, in the response was asked if I faced issue with netbeans & erlybird. Well the answer is -- no, not really, am yet to try. Asked the question because Erlybird looked quite promising, and the last known Netbeans support has been out for quite a while with several updates post it. Thus checking. On Wed, Feb 9, 2011 at 2:19 PM, Banibrata Dutta wrote: > Does anyone have any update on ErlyBird and if there is any hope for Erlang > plugins for latest version of Netbeans and functionality updates ? > > regards, > B > From james@REDACTED Fri Feb 11 18:05:28 2011 From: james@REDACTED (James Aimonetti) Date: Fri, 11 Feb 2011 09:05:28 -0800 Subject: [erlang-questions] Split string in nested list In-Reply-To: <1297443048857-3301595.post@n4.nabble.com> References: <1297443048857-3301595.post@n4.nabble.com> Message-ID: <4D556C58.2080703@2600hz.com> On 02/11/2011 08:50 AM, shk wrote: > Hello, > > I have list of next view: > > [["admin@REDACTED"],[localhost@REDACTED],[user@REDACTED]] > > How can i split by '@" this list? > > I need following: > > [["admin", "localhost"], ["localhost", "localhost"], [user, localhost]] > > Thank you. Is the list ["admin@REDACTED", "localhost@REDACTED", "user@REDACTED"] or the one you wrote? They are different; yours is a list of lists, each of which is a list with one element (the string), while mine is a list of strings. Either way, lists:map/2 should suffice to help convert the list; you will need to unpack your inner lists within the map fun to get your actual string. -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 From singpor@REDACTED Fri Feb 11 18:33:29 2011 From: singpor@REDACTED (Weerasak Chongnguluam) Date: Sat, 12 Feb 2011 00:33:29 +0700 Subject: [erlang-questions] Split string in nested list In-Reply-To: <1297444125.2991.5.camel@iporsut> References: <1297443048857-3301595.post@n4.nabble.com> <1297444125.2991.5.camel@iporsut> Message-ID: Sorry I'm incorrect argument pattern. lists:map(fun([URL]) -> if is_atom(URL) -> string:tokens(atom_to_list( > > URL),"@"); > true -> > string:tokens(URL,"@") > end > end,[["admin@REDACTED"],[localhost@REDACTED],[user@REDACTED]] > ). 2011/2/12 Weerasak Chongnguluam > Hi, > > You can use lists:map with string:tokens, like this. > lists:map(fun(URL) -> > if is_atom(URL) -> > string:tokens(atom_to_list(URL),"@"); > true -> > string:tokens(URL,"@") > end > end,[["admin@REDACTED"],[localhost@REDACTED],[user@REDACTED]] > ). > > On Fri, 2011-02-11 at 08:50 -0800, shk wrote: > > Hello, > > > > I have list of next view: > > > > [["admin@REDACTED"],[localhost@REDACTED],[user@REDACTED]] > > > > How can i split by '@" this list? > > > > I need following: > > > > [["admin", "localhost"], ["localhost", "localhost"], [user, localhost]] > > > > Thank you. > > > From mevans@REDACTED Fri Feb 11 18:58:22 2011 From: mevans@REDACTED (Evans, Matthew) Date: Fri, 11 Feb 2011 12:58:22 -0500 Subject: Mnesia Fragmentation - Adding a new fragment In-Reply-To: References: Message-ID: I'll reformat the question a bit better :) ----Original Message----- Hi, When I add a new node with a new fragment to an existing fragmented table the documentation says: ============================ {add_frag, NodesOrDist} Adds one new fragment to a fragmented table. All records in one of the old fragments will be rehashed and about half of them will be moved to the new (last) fragment. All other fragmented tables, which refers to this table in their foreign key, will automatically get a new fragment, and their records will also be dynamically rehashed in the same manner as for the main table. ============================ The question is that what happens to db queries during this period? If they use the new hash on the key to find the fragment then during the rehashing period lookups will fail (likewise if they use the old hash). The other question relates to backups of fragments (the n_disc_copies option). Can we guarantee that this backup fragment will be on a different node (if many nodes are available) to its mate? Thanks Matt From erlang@REDACTED Fri Feb 11 19:08:35 2011 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 11 Feb 2011 19:08:35 +0100 Subject: announce SEGB-0.1 Message-ID: Announcing SEBG-0.1 SEBG = Simple Erlang Browser Graphics This is a system to push graphics to a browser. Using this you can make interactive graphics in a websockets enabled browser in a few lines of code. It consists of a simple web server. The browser opens a web socket and the web server spawns a process to handle the socket. I set up a middle man to handle the socket. Thereafter the handler can push asynchronous messages to the browser. These message contain Javascript expressions. All the code in the browser does is wait for a message, eval it and wait for the next message. Then I make sure jquery and Raphael are loaded, after which I send Javascript messages to the browser contain Javascript expressions that create SVG objects by calling library functions in Raphael The code for all of this is surprisingly simple, and it is very easy to push asynchronous message to the browser. I have also included a number of suggests for projects based on this. This list often gets request for suitable problems to solve. Building on this simple code it should be possible to implement simple multi-user games and so on. Being able to push asynchronous messages to the browser makes like very easy. Have a look. The code is at https://github.com/joearms/SEBG Cheers /Joe From zabrane3@REDACTED Fri Feb 11 19:37:04 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 11 Feb 2011 19:37:04 +0100 Subject: [erlang-questions] announce SEGB-0.1 In-Reply-To: References: Message-ID: Hi Joe, > These message contain Javascript expressions. All the code in the > browser does is wait for a message, eval it and wait for the next message. You forgot to mention the SEGB needs "elib1" (elib1_misc.erl). > Then I make sure jquery and Raphael are loaded, after which > I send Javascript messages to the browser contain Javascript expressions > that create SVG objects by calling library functions in Raphael > > The code for all of this is surprisingly simple, and it is very easy to push > asynchronous message to the browser. Very impressive ... > I have also included a number of suggests for projects based on this. > This list often gets request for suitable problems to solve. Building on > this simple code it should be possible to implement simple multi-user > games and so on. Being able to push asynchronous messages to the browser > makes like very easy. > > Have a look. The code is at > > https://github.com/joearms/SEBG I'll do. > /Joe Regards, Zabrane From jameschurchman@REDACTED Fri Feb 11 20:43:20 2011 From: jameschurchman@REDACTED (James Churchman) Date: Fri, 11 Feb 2011 19:43:20 +0000 Subject: [erlang-questions] announce SEGB-0.1 In-Reply-To: References: Message-ID: <69442BC6-5BA3-430A-B798-6B59AAC084DD@gmail.com> looks really cool! two things tho, 1) i cant get it to work, i get when visiting http://localhost:1234/generic.html Uncaught ReferenceError: SVG is not defined in chromium and generic.html:20ReferenceError: Can't find variable: SVG in safari :-) maybe its because i dot have genuine chrome on my mac, but the function SVG() does not exist and 2) im fairly sure that this in the docs is not quite correct var glob = new Array(); To persist a variable outside the scope of eval we can evaluate: eval('var glob[Name] = ...'); sinceglob is defined outside of the scope of eval its value will be persisted outside the scope of eval. I think that eval always runs at the scope that its at, but putting in "var" restricts the scope to either the eval or two the scope that its in (eg the function). I think its the latter. Simply leaving out the var statement (as it does in the code :-) ) makes it global (eg attach to the window object) james On 11 Feb 2011, at 18:08, Joe Armstrong wrote: > Announcing SEBG-0.1 > > SEBG = Simple Erlang Browser Graphics > > This is a system to push graphics to a browser. > > Using this you can make interactive graphics in a websockets enabled browser > in a few lines of code. > > It consists of a simple web server. The browser > opens a web socket and the web server spawns a process to > handle the socket. I set up a middle man to handle the socket. > Thereafter the handler can push asynchronous messages to the browser. > > These message contain Javascript expressions. All the code in the > browser does is wait for a message, eval it and wait for the next message. > > Then I make sure jquery and Raphael are loaded, after which > I send Javascript messages to the browser contain Javascript expressions > that create SVG objects by calling library functions in Raphael > > The code for all of this is surprisingly simple, and it is very easy to push > asynchronous message to the browser. > > I have also included a number of suggests for projects based on this. > This list often gets request for suitable problems to solve. Building on > this simple code it should be possible to implement simple multi-user > games and so on. Being able to push asynchronous messages to the browser > makes like very easy. > > Have a look. The code is at > > https://github.com/joearms/SEBG > > Cheers > > /Joe From dmitriy.kargapolov@REDACTED Fri Feb 11 20:46:25 2011 From: dmitriy.kargapolov@REDACTED (Dmitriy Kargapolov) Date: Fri, 11 Feb 2011 14:46:25 -0500 Subject: process memory and performance Message-ID: Hi all, I have some process (gen_sever) performing multiple mnesia queries, so complex lookup takes about 840 mks (pretty old h/w, non-smp, R11B-5). When internal in-memory storage added to this process, so its memory increased up to ~ 60MB, same lookup slow down to 2200 mks. Note that in this test lookup did not use big memory. This big memory organized as complex recursive data structure. Why memory allocated by process indirectly affects its performance? Is it related to how GC works? Thanks. From erlang@REDACTED Fri Feb 11 21:20:40 2011 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 11 Feb 2011 21:20:40 +0100 Subject: [erlang-questions] announce SEGB-0.1 In-Reply-To: <69442BC6-5BA3-430A-B798-6B59AAC084DD@gmail.com> References: <69442BC6-5BA3-430A-B798-6B59AAC084DD@gmail.com> Message-ID: On Fri, Feb 11, 2011 at 8:43 PM, James Churchman wrote: > looks really cool! > > two things tho, > > 1) i cant get it to work, i get when visiting > http://localhost:1234/generic.html > Uncaught ReferenceError: SVG is not defined > in chromium > and > generic.html:20 ReferenceError: Can't > find variable: SVG > in safari :-) > > maybe its because i dot have genuine chrome on my mac, but the function > SVG() does not exist > Oops - I've fixed this - thanks. SVG was a private library that I was playing with > > and 2) > > im fairly sure that this in the docs is not quite correct > > *var glob = new Array();* > > To persist a variable outside the scope of eval we can evaluate: *eval('var > glob[Name] = ...');* since*glob* is defined outside of the scope of eval > its value will be persisted outside the scope of eval. > > I think that eval always runs at the scope that its at, but putting in > "var" restricts the scope to either the eval or two the scope that its in > (eg the function). I think its the latter. Simply leaving out the var > statement (as it does in the code :-) ) makes it global (eg attach to the > window object) > I didn't know that. I tried eval("var foo=.....") follow by an access to var via eval("...x(foo) ..") and concluded that foo was not known to the garbage collector and removed. I didn't try the eval("foo = ..") variant. I actually have no idea how long data persists in javascript, especially code. I assume data structures are garbed away if they are not reachable form the DOM or the top level of all javscript that has been loaded, but how to remove a js script is a mystery ... /Joe > james > > On 11 Feb 2011, at 18:08, Joe Armstrong wrote: > > Announcing SEBG-0.1 > > SEBG = Simple Erlang Browser Graphics > > This is a system to push graphics to a browser. > > Using this you can make interactive graphics in a websockets enabled > browser > in a few lines of code. > > It consists of a simple web server. The browser > opens a web socket and the web server spawns a process to > handle the socket. I set up a middle man to handle the socket. > Thereafter the handler can push asynchronous messages to the browser. > > These message contain Javascript expressions. All the code in the > browser does is wait for a message, eval it and wait for the next message. > > Then I make sure jquery and Raphael are loaded, after which > I send Javascript messages to the browser contain Javascript expressions > that create SVG objects by calling library functions in Raphael > > The code for all of this is surprisingly simple, and it is very easy to > push > asynchronous message to the browser. > > I have also included a number of suggests for projects based on this. > This list often gets request for suitable problems to solve. Building on > this simple code it should be possible to implement simple multi-user > games and so on. Being able to push asynchronous messages to the browser > makes like very easy. > > Have a look. The code is at > > https://github.com/joearms/SEBG > > Cheers > > /Joe > > > From jameschurchman@REDACTED Fri Feb 11 22:23:56 2011 From: jameschurchman@REDACTED (James Churchman) Date: Fri, 11 Feb 2011 21:23:56 +0000 Subject: [erlang-questions] announce SEGB-0.1 In-Reply-To: References: <69442BC6-5BA3-430A-B798-6B59AAC084DD@gmail.com> Message-ID: <372925AA-912A-4514-9F68-0F358A488F80@gmail.com> js is so full of quirks... ( and therefore using it as a server side backend is a perfect idea :-) ) not using "var" makes it look up the scope hierarchy to see if it can find the same name variable and if not then use window.the_var_name but using var has other effects (for example it changes the meaning of code written above it the variable name matches in the same scope.. which can mean adding new code that accidentally reuses a variable name in code above can have very odd effects, usually making it have the value undefined) > I actually have no idea how long data persists in javascript, especially code. i am not even sure that people implementing ie 6 knew.... as far as i know some styles of code will never have their variables garbage collected even tho its impossible to access them! in that case it works on the principle that you will reload / quit the browser before the machine runs out of ram :-) On 11 Feb 2011, at 20:20, Joe Armstrong wrote: > > > On Fri, Feb 11, 2011 at 8:43 PM, James Churchman wrote: > looks really cool! > > two things tho, > > 1) i cant get it to work, i get when visiting http://localhost:1234/generic.html > Uncaught ReferenceError: SVG is not defined > in chromium > and > generic.html:20ReferenceError: Can't find variable: SVG > in safari :-) > > maybe its because i dot have genuine chrome on my mac, but the function SVG() does not exist > > Oops - I've fixed this - thanks. SVG was a private library that I was playing with > > > and 2) > > im fairly sure that this in the docs is not quite correct > var glob = new Array(); > > To persist a variable outside the scope of eval we can evaluate: eval('var glob[Name] = ...'); sinceglob is defined outside of the scope of eval its value will be persisted outside the scope of eval. > > > I think that eval always runs at the scope that its at, but putting in "var" restricts the scope to either the eval or two the scope that its in (eg the function). I think its the latter. Simply leaving out the var statement (as it does in the code :-) ) makes it global (eg attach to the window object) > > I didn't know that. I tried eval("var foo=.....") follow by an > access to var via eval("...x(foo) ..") and concluded that foo > was not known to the garbage collector and removed. I didn't try > the eval("foo = ..") variant. > > I actually have no idea how long data persists in javascript, especially code. I assume data structures are garbed away > if they are not reachable form the DOM or the top level of > all javscript that has been loaded, but how to remove a js script > is a mystery ... > > /Joe > > > > > > james > > On 11 Feb 2011, at 18:08, Joe Armstrong wrote: > >> Announcing SEBG-0.1 >> >> SEBG = Simple Erlang Browser Graphics >> >> This is a system to push graphics to a browser. >> >> Using this you can make interactive graphics in a websockets enabled browser >> in a few lines of code. >> >> It consists of a simple web server. The browser >> opens a web socket and the web server spawns a process to >> handle the socket. I set up a middle man to handle the socket. >> Thereafter the handler can push asynchronous messages to the browser. >> >> These message contain Javascript expressions. All the code in the >> browser does is wait for a message, eval it and wait for the next message. >> >> Then I make sure jquery and Raphael are loaded, after which >> I send Javascript messages to the browser contain Javascript expressions >> that create SVG objects by calling library functions in Raphael >> >> The code for all of this is surprisingly simple, and it is very easy to push >> asynchronous message to the browser. >> >> I have also included a number of suggests for projects based on this. >> This list often gets request for suitable problems to solve. Building on >> this simple code it should be possible to implement simple multi-user >> games and so on. Being able to push asynchronous messages to the browser >> makes like very easy. >> >> Have a look. The code is at >> >> https://github.com/joearms/SEBG >> >> Cheers >> >> /Joe > > From kenneth.lundin@REDACTED Fri Feb 11 22:47:44 2011 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Fri, 11 Feb 2011 22:47:44 +0100 Subject: [erlang-questions] upgrading erlang servers In-Reply-To: <000e01cbc9f8$e04e2080$a0ea6180$@com> References: <000e01cbc9f8$e04e2080$a0ea6180$@com> Message-ID: In practice it works to do an upgrade node by node in a cluster of Erlang nodes. We always keep the runtime system including the distribution protocol compatible 2 major releases back. Of course you can create troubles by changing some protocol of your own that you use between nodes. You have to handle both old and new behaviour in that case. Other problems are incompatible changes of funs from another node that are stored. Mnesia database schemas/format can also be a problem, but there is support for converting tables. All in all it is in practice possible to do a rolling upgrade of the Erlang modules and/or the Erlang VM if you plan the changes beforehand. /Kenneth , Erlang/OTP Ericsson On Fri, Feb 11, 2011 at 3:35 PM, David Mercer wrote: > On Wednesday, February 09, 2011, Param Jyothi Reddy wrote: > >> I am new to erlang and was reading "Programming Erlang" by Joe >> Armstrong and one remark in the book surprised me. >> >> Pg 173: "And same version of erlang [on all nodes]. If you dont do >> this, you'll get serious and mysterious errors.". Not sure how true >> this is currently, however if it is, how do you upgrade cluster of >> nodes running X version of erlang to Y without disrupting service? I >> would have assumed that one can upgrade subset at a time which would >> also provide a vehicle to test version Y on small subset before >> committing the full cluster. But if this comment is true then one >> would have to upgrade cluster of nodes in an all or nothing fashion. >> >> There are other ways to upgrade by having two clusters and switching >> load between them using some protocol. Is that how it is meant to be >> used? Am i missing something obvious? > > This is a good question, which relates not only to upgrades of the Erlang > runtime system, but also of any code that changes its data structures: How > do you upgrade a distributed system when the interface changes? ?I asked a > similar question in October of '07 > (http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:30311:200710:fanenpeabclnldcl > aikd), but your question is even more intractable than mine, since yours > cannot be solved by code tricks. ?If you need simultaneous upgrade of all > 3,000 nodes, especially when confounded by some of the geopolitical > considerations described in > , > then I'm not sure there is a practical solution, which is why ten years from > now "Pan-Global Worldwide Consolidated" will still be running Erlang R3A1. > > Cheers, > > DBM > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From paramr@REDACTED Fri Feb 11 23:20:02 2011 From: paramr@REDACTED (Param Jyothi Reddy) Date: Fri, 11 Feb 2011 14:20:02 -0800 Subject: [erlang-questions] upgrading erlang servers In-Reply-To: References: <000e01cbc9f8$e04e2080$a0ea6180$@com> Message-ID: Thanks for the info on compatibility. "Other problems are incompatible changes of funs from another node that are stored. Mnesia database schemas/format can also be a problem, but there is support for converting tables." Would it be fair to say funs and DB schema/format will also be compatible for 2 versions? Are they auto converted on version updgrade or does one usually run a batch process to upgrade the db? Also curious if these data are versioned i.e. there is version number associated with the blob which tells if it makes sense to read it with current readers or not? Are conversions supported for 2 versions or arbitrary versions? Pointers to docs will help me not ask too many questions on the list :) Param On Fri, Feb 11, 2011 at 1:47 PM, Kenneth Lundin wrote: > In practice it works to do an upgrade node by node in a cluster of Erlang nodes. > We always keep the runtime system including the distribution protocol compatible > 2 major releases back. > Of course you can create troubles by changing some protocol of your own that you > use between nodes. > You have to handle both old and new behaviour in that case. > Other problems are incompatible changes of funs from another node that > are stored. > Mnesia database schemas/format can also be a problem, but there is > support for converting tables. > > All in all it is in practice possible to do a rolling upgrade of the > Erlang modules > and/or ?the Erlang VM if you plan the changes beforehand. > > /Kenneth , Erlang/OTP Ericsson > > On Fri, Feb 11, 2011 at 3:35 PM, David Mercer wrote: >> On Wednesday, February 09, 2011, Param Jyothi Reddy wrote: >> >>> I am new to erlang and was reading "Programming Erlang" by Joe >>> Armstrong and one remark in the book surprised me. >>> >>> Pg 173: "And same version of erlang [on all nodes]. If you dont do >>> this, you'll get serious and mysterious errors.". Not sure how true >>> this is currently, however if it is, how do you upgrade cluster of >>> nodes running X version of erlang to Y without disrupting service? I >>> would have assumed that one can upgrade subset at a time which would >>> also provide a vehicle to test version Y on small subset before >>> committing the full cluster. But if this comment is true then one >>> would have to upgrade cluster of nodes in an all or nothing fashion. >>> >>> There are other ways to upgrade by having two clusters and switching >>> load between them using some protocol. Is that how it is meant to be >>> used? Am i missing something obvious? >> >> This is a good question, which relates not only to upgrades of the Erlang >> runtime system, but also of any code that changes its data structures: How >> do you upgrade a distributed system when the interface changes? ?I asked a >> similar question in October of '07 >> (http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:30311:200710:fanenpeabclnldcl >> aikd), but your question is even more intractable than mine, since yours >> cannot be solved by code tricks. ?If you need simultaneous upgrade of all >> 3,000 nodes, especially when confounded by some of the geopolitical >> considerations described in >> , >> then I'm not sure there is a practical solution, which is why ten years from >> now "Pan-Global Worldwide Consolidated" will still be running Erlang R3A1. >> >> Cheers, >> >> DBM >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From ulf.wiger@REDACTED Sat Feb 12 03:18:23 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 12 Feb 2011 03:18:23 +0100 Subject: [erlang-questions] RE: Mnesia Fragmentation - Adding a new fragment In-Reply-To: References: Message-ID: <45209195-FFFB-4C1A-B788-15CFA40156B4@erlang-solutions.com> Hi Matthew, Mnesia actually does all the rehashing within the schema transaction that adds the fragment, so the table will be locked. Queries within transactions will run either before or after the rehash. Dirty queries may return unexpected results. And, yes, mnesia will put the disc_copies on different nodes. It is not in fact possible to have two replicas of a table on the same node. Mnesia will abort if there are not enough nodes in the node pool to create the requested number of replicas. BR, Ulf W On 11 Feb 2011, at 18:58, Evans, Matthew wrote: > I'll reformat the question a bit better :) > > > ----Original Message----- > Hi, > > When I add a new node with a new fragment to an existing fragmented table the documentation says: > > ============================ > {add_frag, NodesOrDist} > > Adds one new fragment to a fragmented table. All records in one of the old fragments will be rehashed and about half of them will be moved to the new (last) fragment. All other fragmented tables, which refers to this table in their foreign key, will automatically get a new fragment, and their records will also be dynamically rehashed in the same manner as for the main table. > ============================ > > The question is that what happens to db queries during this period? If they use the new hash on the key to find the fragment then during the rehashing period lookups will fail (likewise if they use the old hash). > > The other question relates to backups of fragments (the n_disc_copies option). Can we guarantee that this backup fragment will be on a different node (if many nodes are available) to its mate? > > Thanks > > Matt > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From steven.charles.davis@REDACTED Sat Feb 12 03:19:22 2011 From: steven.charles.davis@REDACTED (Steve Davis) Date: Fri, 11 Feb 2011 18:19:22 -0800 (PST) Subject: how to check if VM is in detached mode In-Reply-To: References: Message-ID: <1297477162061-3302316.post@n4.nabble.com> Hi, Are you looking for init:get_arguments() maybe? /s -- View this message in context: http://erlang.2086793.n4.nabble.com/how-to-check-if-VM-is-in-detached-mode-tp3300714p3302316.html Sent from the Erlang Questions mailing list archive at Nabble.com. From ulf.wiger@REDACTED Sat Feb 12 03:44:40 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 12 Feb 2011 03:44:40 +0100 Subject: [erlang-questions] upgrading erlang servers In-Reply-To: References: <000e01cbc9f8$e04e2080$a0ea6180$@com> Message-ID: <5C472E53-16D4-4F01-A1FC-36405C332A40@erlang-solutions.com> The mnesia schema problem is related to the table definitions, not mnesia's internal representation. Mnesia doesn't support different concurrent versions of the same table, so if you would upgrade one node, and in the process modify a table definition, this change would immediately become visible on the other - not yet upgraded - nodes, potentially with unwanted consequences. BR, Ulf W On 11 Feb 2011, at 23:20, Param Jyothi Reddy wrote: > Thanks for the info on compatibility. > > "Other problems are incompatible changes of funs from another node that > are stored. > Mnesia database schemas/format can also be a problem, but there is > support for converting tables." > > Would it be fair to say funs and DB schema/format will also be > compatible for 2 versions? Are they auto converted on version updgrade > or does one usually run a batch process to upgrade the db? > > Also curious if these data are versioned i.e. there is version number > associated with the blob which tells if it makes sense to read it with > current readers or not? Are conversions supported for 2 versions or > arbitrary versions? > > Pointers to docs will help me not ask too many questions on the list :) > > Param > > On Fri, Feb 11, 2011 at 1:47 PM, Kenneth Lundin > wrote: >> In practice it works to do an upgrade node by node in a cluster of Erlang nodes. >> We always keep the runtime system including the distribution protocol compatible >> 2 major releases back. >> Of course you can create troubles by changing some protocol of your own that you >> use between nodes. >> You have to handle both old and new behaviour in that case. >> Other problems are incompatible changes of funs from another node that >> are stored. >> Mnesia database schemas/format can also be a problem, but there is >> support for converting tables. >> >> All in all it is in practice possible to do a rolling upgrade of the >> Erlang modules >> and/or the Erlang VM if you plan the changes beforehand. >> >> /Kenneth , Erlang/OTP Ericsson >> >> On Fri, Feb 11, 2011 at 3:35 PM, David Mercer wrote: >>> On Wednesday, February 09, 2011, Param Jyothi Reddy wrote: >>> >>>> I am new to erlang and was reading "Programming Erlang" by Joe >>>> Armstrong and one remark in the book surprised me. >>>> >>>> Pg 173: "And same version of erlang [on all nodes]. If you dont do >>>> this, you'll get serious and mysterious errors.". Not sure how true >>>> this is currently, however if it is, how do you upgrade cluster of >>>> nodes running X version of erlang to Y without disrupting service? I >>>> would have assumed that one can upgrade subset at a time which would >>>> also provide a vehicle to test version Y on small subset before >>>> committing the full cluster. But if this comment is true then one >>>> would have to upgrade cluster of nodes in an all or nothing fashion. >>>> >>>> There are other ways to upgrade by having two clusters and switching >>>> load between them using some protocol. Is that how it is meant to be >>>> used? Am i missing something obvious? >>> >>> This is a good question, which relates not only to upgrades of the Erlang >>> runtime system, but also of any code that changes its data structures: How >>> do you upgrade a distributed system when the interface changes? I asked a >>> similar question in October of '07 >>> (http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:30311:200710:fanenpeabclnldcl >>> aikd), but your question is even more intractable than mine, since yours >>> cannot be solved by code tricks. If you need simultaneous upgrade of all >>> 3,000 nodes, especially when confounded by some of the geopolitical >>> considerations described in >>> , >>> then I'm not sure there is a practical solution, which is why ten years from >>> now "Pan-Global Worldwide Consolidated" will still be running Erlang R3A1. >>> >>> Cheers, >>> >>> DBM >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From andrew@REDACTED Sat Feb 12 04:21:35 2011 From: andrew@REDACTED (Andrew Thompson) Date: Fri, 11 Feb 2011 22:21:35 -0500 Subject: Some notes on making a random erlang program (egitd) faster Message-ID: <20110212032135.GF17893@hijacked.us> Just for something to do this week, I decided to pick a random erlang project that was a failure due to its speed and see if I could make it fast enough to compete. I picked egitd (the erlang git daemon that github wrote) because it was a fairly public failure. It took me ~4 hours from first grabbing the code to get its performance very close to git-daemon. I also removed ~1500 LOC along the way. Here's the in-depth notes on what I did, as a series of blog articles: http://andrew.hijacked.us/by_keyword/328/egit Hopefully someone will find this useful and/or won't blame the failure of egitd on erlang. Andrew From kenji.rikitake@REDACTED Sat Feb 12 08:22:20 2011 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Sat, 12 Feb 2011 16:22:20 +0900 Subject: [erlang-questions] SSL number of connections In-Reply-To: References: Message-ID: <20110212072220.GA40526@k2r.org> On SSL performance: The OTP crypto module has been NIF-nized since R14B, and the openssl functions are called directly from the module. Kenji Rikitake From bgustavsson@REDACTED Sat Feb 12 08:28:12 2011 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sat, 12 Feb 2011 08:28:12 +0100 Subject: [erlang-questions] Dict size in guards In-Reply-To: <46B9BBC8-4AFB-42B8-9751-F46DCBEA034F@gmail.com> References: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> <000d01cbc9f8$df325040$9d96f0c0$@com> <46B9BBC8-4AFB-42B8-9751-F46DCBEA034F@gmail.com> Message-ID: On Fri, Feb 11, 2011 at 6:10 PM, James Churchman wrote: > Also why does the compiler do a length check for every time you use length in a guard ?? I thought it did not do repetitive pattern matches that are "the same", so surly repetitive guard would also be similarly optimised? That would be possible, but it is not done yet. The compiler currently does not do much optimization of guards and it does not do common sub-expression elimination. > > also "hacks" like [_,_,_,_,_,_,_,_,_,_,_|_] look very clever, and i guess for very long lists have a speed benefit, but again could the compiler not realise that when you put "when Len > 10" it only needs to check that the list is longer than 10? not get the genuine numeric length and then numeric compare to 10. That seems a basic optimisation that could be made That basic optimization is not done because the length(L) will cause an exception if the L is improper, but only checking 10 elements will not cause an exception. The compiler does not do optimizations that would silence potential exceptions (silencing exceptions could hide bugs in the program and make debugging more difficult). Here is an example of an improper list that will cause a badarg exception in length/1: lists:seq(1, 20) ++ [a|b] -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From kenji.rikitake@REDACTED Sat Feb 12 08:58:16 2011 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Sat, 12 Feb 2011 16:58:16 +0900 Subject: [erlang-questions] DNS is slow when run from many processes In-Reply-To: <20110208165610.GD5777@smp.if.uj.edu.pl> References: <20110208165610.GD5777@smp.if.uj.edu.pl> Message-ID: <20110212075816.GB40526@k2r.org> Witold posted an excellent summary. Just one more tip: Having a DNS server faster than 10000 queries/second is not easy. (See the benchmarks of BIND, nsd/unbound, djbdns, and others for the details.) Kenji Rikitake From kenji.rikitake@REDACTED Sat Feb 12 09:25:14 2011 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Sat, 12 Feb 2011 17:25:14 +0900 Subject: [erlang-questions] Generating readable unique IDs for use outside erlang In-Reply-To: <1D911796-2B94-4185-AF8A-527464F126F7@gmail.com> References: <4D53F785.9030300@ecs.soton.ac.uk> <1D911796-2B94-4185-AF8A-527464F126F7@gmail.com> Message-ID: <20110212082514.GA42218@k2r.org> Solely depending on randomness does not necessarily guarantee the uniqueness of the output; two calls from crypto:rand_bytes/1 may result in the same value. UUID has 128-bit value space, however, so the probability of accidental duplication of two UUIDs may be small enough. See RFC4122 for UUID URN Namespace. The Section 4.1.4 requires a timestamp included in the UUID Version 1. Kenji Rikitake ++> Jared Kofron [2011-02-10 08:28:06 -0800]: > I believe couch uses crypto's rand_bytes to accomplish this... something like > > to_hex(crypto:rand_bytes(16)) > > generates a UUID for them. From jws@REDACTED Sat Feb 12 09:29:51 2011 From: jws@REDACTED (Jeff Schultz) Date: Sat, 12 Feb 2011 19:29:51 +1100 Subject: [erlang-questions] Dict size in guards In-Reply-To: References: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> <3FF17835-13E7-4DBA-A6FF-8DBBC2FC3916@erlang-solutions.com> <10D1489F-CDFB-4C6E-93F6-8FB961910CF3@rogvall.se> Message-ID: <20110212082951.GA11966@mulga.csse.unimelb.edu.au> On Fri, Feb 11, 2011 at 06:40:12PM +0900, Michael Turner wrote: > FWIW, Doug Clark found that the average length of lists in large, mature > lisp applications was about four. FWIW, 1. As far as I remember, LISP programs don't typically manage strings of characters as lists of small integers. 2. LISP doesn't have tuples, so things that in Erlang are usually tuples are represented in LISP as small lists. The one makes average Erlang lists longer, and the other makes average LISP lists smaller. Jeff Schultz From tony@REDACTED Sat Feb 12 10:29:27 2011 From: tony@REDACTED (Tony Rogvall) Date: Sat, 12 Feb 2011 10:29:27 +0100 Subject: [erlang-questions] Some notes on making a random erlang program (egitd) faster In-Reply-To: <20110212032135.GF17893@hijacked.us> References: <20110212032135.GF17893@hijacked.us> Message-ID: Cool. Nice reading. If you need money, may be you should consider sell your services. with a site name like: save-crashed-projects.com :-) /Tony On 12 feb 2011, at 04.21, Andrew Thompson wrote: > Just for something to do this week, I decided to pick a random erlang > project that was a failure due to its speed and see if I could make it > fast enough to compete. I picked egitd (the erlang git daemon that > github wrote) because it was a fairly public failure. It took me ~4 > hours from first grabbing the code to get its performance very close to > git-daemon. I also removed ~1500 LOC along the way. > > Here's the in-depth notes on what I did, as a series of blog articles: > > http://andrew.hijacked.us/by_keyword/328/egit > > Hopefully someone will find this useful and/or won't blame the failure > of egitd on erlang. > > Andrew > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > "Have run Make so many times I dunno what's installed anymore" From erlang@REDACTED Sat Feb 12 12:33:36 2011 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 12 Feb 2011 12:33:36 +0100 Subject: Two beautiful programs - or web programming made easy Message-ID: Two beautiful programs For a long time one of my favorite Erlang program was this: loop() -> receive F -> F(), loop() end. It's nice because it does very little, but what is does is universal. It enables mobile code. Well now I can do this in Javascript. The Javascript equivalent is: function onMessage(evt) { eval(evt.data); } Where the data comes from a websocket. Websockets are controlled with a simple API: websocket = new WebSocket(wsUri); websocket.onopen = function(evt) { onOpen(evt) }; websocket.onclose = function(evt) { onClose(evt) }; websocket.onmessage = function(evt) { onMessage(evt) }; Linking a websocket and Erlang is pretty easy. So now I can write code like this in erlang -module(demo1). -export([start/1]). start(Pid) -> Pid ! {eval, "document.body.innerHTML='';"}, Pid ! {eval, "document.body.style.backgroundColor='#eeffaa';"}, Pid ! {eval, "document.body.innerHTML+='

Hello World

'"}, event_loop(Pid). event_loop(Pid) -> receive Any -> io:format("??event loop:~p~n",[Any]), event_loop(Pid) end. This code pushes asynchronous messages containing javascript to a generic web page, and the web page evals the result. This technique is amazingly powerful. So now I only need one generic web page. Think of that. Only one page is needed - forever. All the generic pages does is: loop: wait for a message containing Javascript eval the message Beautiful This is the easiest way to program a GUI I can conceive of. You can download and run the examples from: https://github.com/joearms/SEBG You'll need something like Google chrome to run them. Have fun /Joe From masklinn@REDACTED Sat Feb 12 12:46:39 2011 From: masklinn@REDACTED (Masklinn) Date: Sat, 12 Feb 2011 12:46:39 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: Message-ID: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> On 2011-02-12, at 12:33 , Joe Armstrong wrote: > > You'll need something like Google chrome to run them. Warning: the status of the (currently broken [0]) WebSockets is unsure: Firefox and Opera already dropped it, it is unknown whether Chrome and Safari will keep it enabled by default. Chrome-wise, they've kept it for now but they plan on disabling it as soon as they hear of an exploit in the wild[1] Also, it's going to be a complete and utter pain to debug: even the more modern JS debuggers of Firefox (via Firebug) and Webkit have insane amounts of trouble finding and debugging dynamically evaluated code. [0] http://www.ietf.org/mail-archive/web/hybi/current/msg04744.html [1] http://codereview.chromium.org/5643005/ From erlang@REDACTED Sat Feb 12 12:51:16 2011 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 12 Feb 2011 12:51:16 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> Message-ID: Which is why you should write correct code that does not need debugging. The use of higher order functions that generate the code will make this a lot easier. The next stop is to generate the JS with HOFs and make sure the generators are correct. You don't want to write JS by hand - that's work - let a program do it for you :-) /Joe On Sat, Feb 12, 2011 at 12:46 PM, Masklinn wrote: > On 2011-02-12, at 12:33 , Joe Armstrong wrote: > > > > You'll need something like Google chrome to run them. > Warning: the status of the (currently broken [0]) WebSockets is unsure: > Firefox and Opera already dropped it, it is unknown whether Chrome and > Safari will keep it enabled by default. Chrome-wise, they've kept it for now > but they plan on disabling it as soon as they hear of an exploit in the > wild[1] > > Also, it's going to be a complete and utter pain to debug: even the more > modern JS debuggers of Firefox (via Firebug) and Webkit have insane amounts > of trouble finding and debugging dynamically evaluated code. > > [0] http://www.ietf.org/mail-archive/web/hybi/current/msg04744.html > [1] http://codereview.chromium.org/5643005/ From ulf.wiger@REDACTED Sat Feb 12 13:15:12 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 12 Feb 2011 13:15:12 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> Message-ID: <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> Well, I read those links not as an indictment of websockets per se, but of the UPGRADE-based handshake. I think Joe is definitely on to something here, and this sort of setup fits Erlang beautifully. Essentially, Joe may have found a way to dust off his ex11 idea[1], in a context where it is much more likely to succeed. ;-) (But please, Joe, leave out the !! this time - one revolution at a time.) BR, Ulf W [1] http://ftp.sunet.se/pub/lang/erlang/workshop/2004/ex11.pdf On 12 Feb 2011, at 12:46, Masklinn wrote: > On 2011-02-12, at 12:33 , Joe Armstrong wrote: >> >> You'll need something like Google chrome to run them. > Warning: the status of the (currently broken [0]) WebSockets is unsure: Firefox and Opera already dropped it, it is unknown whether Chrome and Safari will keep it enabled by default. Chrome-wise, they've kept it for now but they plan on disabling it as soon as they hear of an exploit in the wild[1] > > Also, it's going to be a complete and utter pain to debug: even the more modern JS debuggers of Firefox (via Firebug) and Webkit have insane amounts of trouble finding and debugging dynamically evaluated code. > > [0] http://www.ietf.org/mail-archive/web/hybi/current/msg04744.html > [1] http://codereview.chromium.org/5643005/ > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From erlang@REDACTED Sat Feb 12 13:23:58 2011 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 12 Feb 2011 13:23:58 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> Message-ID: On Sat, Feb 12, 2011 at 1:15 PM, Ulf Wiger wrote: > > Well, I read those links not as an indictment of websockets per se, but of > the UPGRADE-based handshake. I think Joe is definitely on to something here, > and this sort of setup fits Erlang beautifully. > > Essentially, Joe may have found a way to dust off his ex11 idea[1], in a > context where it is much more likely to succeed. ;-) > > (But please, Joe, leave out the !! this time - one revolution at a time.) > !! would be evil in this context. As soon as you get outside erlang !! is bad. ! is good because it creates no dependencies, the message either gets there or it does not. Code that can deal with this uncertainty will be good, other code is not. It's actually pretty awe-inspiring to say Pid ! button("click me") and see a button just appear in the browser Great fun. The great thing is that we can now abstract the interface button("text") means return a thing that when evaluated makes a button this code return Javascript, but it could equally well return xul, or html, or llvm assembler it doesn't matter - as long as the eval end knows how to eval it. So the top level code can be written entirely without knowledge of what's going on under the covers /Joe /Joe > > BR, > Ulf W > > [1] http://ftp.sunet.se/pub/lang/erlang/workshop/2004/ex11.pdf > > On 12 Feb 2011, at 12:46, Masklinn wrote: > > > On 2011-02-12, at 12:33 , Joe Armstrong wrote: > >> > >> You'll need something like Google chrome to run them. > > Warning: the status of the (currently broken [0]) WebSockets is unsure: > Firefox and Opera already dropped it, it is unknown whether Chrome and > Safari will keep it enabled by default. Chrome-wise, they've kept it for now > but they plan on disabling it as soon as they hear of an exploit in the > wild[1] > > > > Also, it's going to be a complete and utter pain to debug: even the more > modern JS debuggers of Firefox (via Firebug) and Webkit have insane amounts > of trouble finding and debugging dynamically evaluated code. > > > > [0] http://www.ietf.org/mail-archive/web/hybi/current/msg04744.html > > [1] http://codereview.chromium.org/5643005/ > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > From tramjoe.merin@REDACTED Sat Feb 12 13:28:31 2011 From: tramjoe.merin@REDACTED (Jerome Martin) Date: Sat, 12 Feb 2011 13:28:31 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> Message-ID: If only the browser could adopt for good a CONNECT-based handshake and enable it by default, the world would be a better place to deploy web UIs :-) Because that remains the biggest issue for now, until I see this happening, all that beauty might be lost :-( On Sat, Feb 12, 2011 at 1:23 PM, Joe Armstrong wrote: > On Sat, Feb 12, 2011 at 1:15 PM, Ulf Wiger > wrote: > > > > > Well, I read those links not as an indictment of websockets per se, but > of > > the UPGRADE-based handshake. I think Joe is definitely on to something > here, > > and this sort of setup fits Erlang beautifully. > > > > Essentially, Joe may have found a way to dust off his ex11 idea[1], in a > > context where it is much more likely to succeed. ;-) > > > > (But please, Joe, leave out the !! this time - one revolution at a time.) > > > > !! would be evil in this context. As soon as you get outside erlang !! is > bad. > ! is good because it creates no dependencies, the message either gets there > or > it does not. Code that can deal with this uncertainty will be good, other > code is > not. > > It's actually pretty awe-inspiring to say > > Pid ! button("click me") and see a button just appear in the browser > > Great fun. > > The great thing is that we can now abstract the interface > > button("text") means return a thing that when evaluated makes a button > this code return Javascript, but it could equally well return xul, or html, > or llvm assembler > it doesn't matter - as long as the eval end knows how to eval it. So the > top > level > code can be written entirely without knowledge of what's going on under the > covers > > /Joe > > > /Joe > > > > > > > BR, > > Ulf W > > > > [1] http://ftp.sunet.se/pub/lang/erlang/workshop/2004/ex11.pdf > > > > On 12 Feb 2011, at 12:46, Masklinn wrote: > > > > > On 2011-02-12, at 12:33 , Joe Armstrong wrote: > > >> > > >> You'll need something like Google chrome to run them. > > > Warning: the status of the (currently broken [0]) WebSockets is unsure: > > Firefox and Opera already dropped it, it is unknown whether Chrome and > > Safari will keep it enabled by default. Chrome-wise, they've kept it for > now > > but they plan on disabling it as soon as they hear of an exploit in the > > wild[1] > > > > > > Also, it's going to be a complete and utter pain to debug: even the > more > > modern JS debuggers of Firefox (via Firebug) and Webkit have insane > amounts > > of trouble finding and debugging dynamically evaluated code. > > > > > > [0] http://www.ietf.org/mail-archive/web/hybi/current/msg04744.html > > > [1] http://codereview.chromium.org/5643005/ > > > ________________________________________________________________ > > > erlang-questions (at) erlang.org mailing list. > > > See http://www.erlang.org/faq.html > > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > > http://erlang-solutions.com > > > > > > > > > -- J?r?me Martin From masklinn@REDACTED Sat Feb 12 13:49:34 2011 From: masklinn@REDACTED (Masklinn) Date: Sat, 12 Feb 2011 13:49:34 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> Message-ID: On 2011-02-12, at 13:15 , Ulf Wiger wrote: > Well, I read those links not as an indictment of websockets per se Why would you? They're not, they're just warnings that websockets are getting disabled in browsers due to being currently broken, and that this kind of approach is therefore nice as a research toy but can not be considered for serious work at this point. I don't believe I wrote anything against Joe's idea, I just forwarded short and mid-term issues with actually applying them outside of well-controlled demonstration environments. From erlang@REDACTED Sat Feb 12 14:03:00 2011 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 12 Feb 2011 14:03:00 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> Message-ID: As I see it the web is predominantly built with one concurrency pattern. A client (web browser) can do an RPC to a server We can't (easily) do an RPC from the server to the client send an asynchronous message from server to client send an asynchronous message from the client to the server Of these four concurrency patterns three are crippled. Instant messageing, for example,is trivial if you can push messages to a client. Turning web sockets off for security reasons is stupid. The solution is to sandbox the thing that receives the messages or if unsandboxed strongly authenticate the messages. /Joe and On Sat, Feb 12, 2011 at 1:49 PM, Masklinn wrote: > On 2011-02-12, at 13:15 , Ulf Wiger wrote: > > Well, I read those links not as an indictment of websockets per se > Why would you? They're not, they're just warnings that websockets are > getting disabled in browsers due to being currently broken, and that this > kind of approach is therefore nice as a research toy but can not be > considered for serious work at this point. > > I don't believe I wrote anything against Joe's idea, I just forwarded short > and mid-term issues with actually applying them outside of well-controlled > demonstration environments. From jameschurchman@REDACTED Sat Feb 12 14:12:33 2011 From: jameschurchman@REDACTED (James Churchman) Date: Sat, 12 Feb 2011 13:12:33 +0000 Subject: [erlang-questions] Dict size in guards In-Reply-To: References: <2D256E72-6485-4514-8640-AC8652945AD4@cs.otago.ac.nz> <000d01cbc9f8$df325040$9d96f0c0$@com> <46B9BBC8-4AFB-42B8-9751-F46DCBEA034F@gmail.com> Message-ID: <95DA08F6-EF06-4E08-A738-58AB392152D4@gmail.com> i seee, so due to improper lists they have different behaviour Trying : Len = length([1 , 2 | ok ]). Gives : exception error: bad argument in function length/1 called as length([1,2|ok]) which is guess is ok because the list is improper and length is therefore undefined and with a guard : LengthFun = fun (List) when length(List) > 1 -> big; (_) -> small end. LengthFun( [1 , 2 | ok ] ) Gives small as the guard silently fails.. I guess the optimisation could still be made internally in beam somehow but its definitely more complex that it seems at first. On 12 Feb 2011, at 07:28, Bj?rn Gustavsson wrote: > That basic optimization is not done because the > length(L) will cause an exception if the L is improper, > but only checking 10 elements will not cause an exception. > The compiler does not do optimizations that would > silence potential exceptions (silencing exceptions could > hide bugs in the program and make debugging more > difficult). > > Here is an example of an improper list that will > cause a badarg exception in length/1: > > lists:seq(1, 20) ++ [a|b] From alain.odea@REDACTED Sat Feb 12 14:45:42 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sat, 12 Feb 2011 10:15:42 -0330 Subject: [erlang-questions] Some notes on making a random erlang program (egitd) faster In-Reply-To: <20110212032135.GF17893@hijacked.us> References: <20110212032135.GF17893@hijacked.us> Message-ID: <9D8B0EE3-05DC-4670-8FDC-5EBE194975E5@gmail.com> On 2011-02-11, at 23:51, Andrew Thompson wrote: > Just for something to do this week, I decided to pick a random erlang > project that was a failure due to its speed and see if I could make it > fast enough to compete. I picked egitd (the erlang git daemon that > github wrote) because it was a fairly public failure. It took me ~4 > hours from first grabbing the code to get its performance very close to > git-daemon. I also removed ~1500 LOC along the way. > > Here's the in-depth notes on what I did, as a series of blog articles: > > http://andrew.hijacked.us/by_keyword/328/egit > > Hopefully someone will find this useful and/or won't blame the failure > of egitd on erlang. > > Andrew > That was awesome Andrew! The fixes you found and the rate at which yoy verified them is prodigal. I highly recommend that everyone on this list read Andrew's Optimizing egitd blog post series starting with http://andrew.hijacked.us/article/324/optimizing-egitd-introduction Optimizing egitd is a clearly written dissection of a performance problem and a beautifully documented path to its resolution. Congratulations Andrew and thank you for sharing your experience :) From masklinn@REDACTED Sat Feb 12 15:29:09 2011 From: masklinn@REDACTED (Masklinn) Date: Sat, 12 Feb 2011 15:29:09 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> Message-ID: <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> On 2011-02-12, at 14:03 , Joe Armstrong wrote: > As I see it the web is predominantly built with one concurrency pattern. > > A client (web browser) can do an RPC to a server > > We can't (easily) > > do an RPC from the server to the client > send an asynchronous message from server to client > send an asynchronous message from the client to the server This is incorrect. HTTP does not support "push" messaging (from a server to a client), but the main (and default) communication mechanism in Javascript is asynchronous (and event-driven). Unless you have a very different definition of "asynchronous message" than the one I'm used to. In fact, synchronous RPC-type calls are definitely frowned upon as they create terrible use experience (due to in-browser javascript being a purely single-threaded event loop). > Turning web sockets off for security reasons is stupid. > > The solution is to sandbox the thing that receives the messages > or if unsandboxed strongly authenticate the messages. I don't follow you here. You're saying it's OK to release broken standards (even though they're fixable) and asserting that every implementor should go through the process of creating his own broken sandbox which will fail to be correctly secured? From marc@REDACTED Sat Feb 12 15:34:40 2011 From: marc@REDACTED (Marc Worrell) Date: Sat, 12 Feb 2011 15:34:40 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> Message-ID: <09486744-8EA4-455C-BF03-1109F27F9850@worrell.nl> The funny thing is that zotonic and nitrogen already use this way of pushing dynamically generated JavaScript to the user agent. It works great. Zotonic switches between a long polling comet connection and websockets depending on the capabilities of the browser. By the way, websockets is not broken. Some proxies are broken. - Marc On 12 feb 2011, at 14:03, Joe Armstrong wrote: > As I see it the web is predominantly built with one concurrency pattern. > > A client (web browser) can do an RPC to a server > > We can't (easily) > > do an RPC from the server to the client > send an asynchronous message from server to client > send an asynchronous message from the client to the server > > Of these four concurrency patterns three are crippled. > > Instant messageing, for example,is trivial if you can push messages to a > client. > > Turning web sockets off for security reasons is stupid. > > The solution is to sandbox the thing that receives the messages > or if unsandboxed strongly authenticate the messages. > > /Joe > From ebegumisa@REDACTED Sat Feb 12 15:36:06 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sun, 13 Feb 2011 01:36:06 +1100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> Message-ID: On Sun, 13 Feb 2011 00:03:00 +1100, Joe Armstrong wrote: > As I see it the web is predominantly built with one concurrency pattern. > > A client (web browser) can do an RPC to a server > > We can't (easily) > > do an RPC from the server to the client > send an asynchronous message from server to client These first 2 difficulties have been the biggest reason why I avoided writing web-apps that run in the browser for so long (I still avoid it). Just give me a socket and I'm happy. When you're used to socket programming, where after a connection is made the line between who is the server and who is the client becomes blurry, it gets really hard to adapt to the idea that only the connectER can make requests to the connectEE (pardon my word invention.) Designs that you took for granted become next to impossible to implement. > send an asynchronous message from the client to the server > > Of these four concurrency patterns three are crippled. > > Instant messageing, for example,is trivial if you can push messages to a > client. > > Turning web sockets off for security reasons is stupid. > I agree 110% With modern web-app development targeting browsers, a large amount of time is being spent finding work-arounds to enable programmers to do things that have been done in other environments. From bi-directional requests (websockets/comet/long-poll) to rich-UI (menus, drag and drop, etc.) I think web-development is out-growing the technologies it's based on -- web developers obviously want more. Attempts to stifle ideas like websockets that are trying to respond to the demand won't succeed in the long run. In the meantime, runtime environments like Mozilla XULRunner and Adobe AIR that give developers a wider range of patterns to choose from (while giving them more responsibility for things like security) will gain more traction while browsers debate on whether to lift their constraints. Me thinks. - Edmond - > The solution is to sandbox the thing that receives the messages > or if unsandboxed strongly authenticate the messages. > > /Joe > > > > and > > On Sat, Feb 12, 2011 at 1:49 PM, Masklinn wrote: > >> On 2011-02-12, at 13:15 , Ulf Wiger wrote: >> > Well, I read those links not as an indictment of websockets per se >> Why would you? They're not, they're just warnings that websockets are >> getting disabled in browsers due to being currently broken, and that >> this >> kind of approach is therefore nice as a research toy but can not be >> considered for serious work at this point. >> >> I don't believe I wrote anything against Joe's idea, I just forwarded >> short >> and mid-term issues with actually applying them outside of >> well-controlled >> demonstration environments. -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From erlang@REDACTED Sat Feb 12 15:52:05 2011 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 12 Feb 2011 15:52:05 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> Message-ID: On Sat, Feb 12, 2011 at 3:29 PM, Masklinn wrote: > On 2011-02-12, at 14:03 , Joe Armstrong wrote: > > As I see it the web is predominantly built with one concurrency pattern. > > > > A client (web browser) can do an RPC to a server > > > > We can't (easily) > > > > do an RPC from the server to the client > > send an asynchronous message from server to client > > send an asynchronous message from the client to the server > This is incorrect. HTTP does not support "push" messaging (from a server to > a client), but the main (and default) communication mechanism in Javascript > is asynchronous (and event-driven). Unless you have a very different > definition of "asynchronous message" than the one I'm used to. > To what does the word "this" apply? "this" (ie the first word of your comment) could refer to any combination of four things - I said we can't easily ... send an asynchronous message from the server to the client - I am fully aware the HTTP does not support this that why I said "easily" it can be done with horrible work arounds like comet etc. > > In fact, synchronous RPC-type calls are definitely frowned upon as they > create terrible use experience (due to in-browser javascript being a purely > single-threaded event loop). > > > > Turning web sockets off for security reasons is stupid. > > > > The solution is to sandbox the thing that receives the messages > > or if unsandboxed strongly authenticate the messages. > I don't follow you here. You're saying it's OK to release broken standards > (even though they're fixable) and asserting that every implementor should go > through the process of creating his own broken sandbox which will fail to be > correctly secured? I didn't say it was ok to release broken standards. I didn't say "broken sandbox" I said "sandbox" - If you want to be secure you should put every application in a trusted sandbox. That way you can play with web sockets even if they are broken. You need a security policy for your sandbox that disallows access to local storage, opening new socket etc. /Joe From masklinn@REDACTED Sat Feb 12 16:34:46 2011 From: masklinn@REDACTED (Masklinn) Date: Sat, 12 Feb 2011 16:34:46 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: <09486744-8EA4-455C-BF03-1109F27F9850@worrell.nl> References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <09486744-8EA4-455C-BF03-1109F27F9850@worrell.nl> Message-ID: On 2011-02-12, at 15:34 , Marc Worrell wrote: > > By the way, websockets is not broken. Some proxies are broken. If these proxies being broken create a *security issue* with using websockets, then websockets themselves are broken. From masklinn@REDACTED Sat Feb 12 16:40:09 2011 From: masklinn@REDACTED (Masklinn) Date: Sat, 12 Feb 2011 16:40:09 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> Message-ID: On 2011-02-12, at 15:52 , Joe Armstrong wrote: > On Sat, Feb 12, 2011 at 3:29 PM, Masklinn wrote: > >> On 2011-02-12, at 14:03 , Joe Armstrong wrote: >>> As I see it the web is predominantly built with one concurrency pattern. >>> >>> A client (web browser) can do an RPC to a server >>> >>> We can't (easily) >>> >>> do an RPC from the server to the client >>> send an asynchronous message from server to client >>> send an asynchronous message from the client to the server >> This is incorrect. HTTP does not support "push" messaging (from a server to >> a client), but the main (and default) communication mechanism in Javascript >> is asynchronous (and event-driven). Unless you have a very different >> definition of "asynchronous message" than the one I'm used to. >> > > To what does the word "this" apply? > > "this" (ie the first word of your comment) could refer to > any combination of four things This referes to what i quoted. The sum is incorrect since some of its parts (namely your assertion that asynchronous client->server communications are "not easy") are incorrect. Which is what I explain in the next three lines. > - I said we can't easily ... send an > asynchronous message > from the server to the client - I am fully aware the HTTP does not support > this that > why I said "easily" it can be done with horrible work arounds like comet > etc. You also said, in the same bundle, that "we can't easily send an asynchronous message from the client to the server". Which is completely incorrect. >>> Turning web sockets off for security reasons is stupid. >>> >>> The solution is to sandbox the thing that receives the messages >>> or if unsandboxed strongly authenticate the messages. >> I don't follow you here. You're saying it's OK to release broken standards >> (even though they're fixable) and asserting that every implementor should go >> through the process of creating his own broken sandbox which will fail to be >> correctly secured? > I didn't say it was ok to release broken standards. You said turning off web sockets due to them being broken was stupid. It follows that you advocate releasing and enabling websockets in a broken state. From arjan@REDACTED Sat Feb 12 17:24:57 2011 From: arjan@REDACTED (Arjan Scherpenisse) Date: Sat, 12 Feb 2011 17:24:57 +0100 Subject: Zotonic 0.6.0 released Message-ID: <4D56B459.8060900@scherpenisse.net> Dear Zotonic users and Erlang enthusiasts, I am pleased to announce the release of Zotonic 0.6.0, the Erlang CMS and web framework. It can be downloaded from the usual place on google code: http://code.google.com/p/zotonic/downloads/list The most prominent changes in this release are multi-language translateable content and SSL support. The changelog containing all major changes and fixes is attached below. Many thanks to all the people who have contributed to this release by sending patches or filing issues on the bug tracker. kind regards, Arjan Scherpenisse Marc Worrell Release 0.6.0, released on 2011-02-12 ------------------------------------- * New features: ** SSL support Zotonic has gotten support for serving web pages over secure HTTPS connections. When configured, it listens by default on port 8443. See http://zotonic.com/https-support for details. ** z_logger A new subsystem for the low-level logging and tracing of requests. This module should be used to log lower level events during development time. Higher-level log messages (e.g. events by Zotonic modules) are still handled by 'mod_logging'. ** multilingual content Every resource can have be translated in as many languages as you like. The admin has gotten an interface to provide the editing of the multiple language versions. Available languages are fully dynamically configurable. ** z_depcache Partial rewrite of depcache system, is now faster and using more the process dictionary of the calling process to cache often used values. * New and changed modules: ** mod_signal New module providing a handy signal and slot mechanism for use in templates. ** mod_tkvstore New module providing a simple typed key/value store for modules and Erlang code. ** mod_translation Check if the user has a preferred language (in the user's persistent data). If not then check the accept-language header (if any) against the available languages. ** mod_mailinglist Tweaks in the templates, updated dutch translations; do not send mail when deleting recipient from admin; Added 'recipient_id' to some e-mails so that the e-mails are sent in the correct language. ** mod_authentication Fix user name display in password reminder e-mail. ** mod_emailer Fix for e-mail override, escape the '@' in the original e-mail address. Added flushing of poll messages ** mod_seo Added option to set a no-index for a complete site. New Google Analytics tracker code. With thanks to Richard Fergie. ** mod_contact Configurable from address for contact email ** mod_admin_identity Fix for finding users, select only identity records with type 'username_pw' ** mod_calendar Better handling for undefined date_end values. ** mod_search Improper months ordering in archive_year_month query. (#134) ** mod_menu Possibility to create an arbitrary number of different menu's. Also a new filter (menu_trail) which gets the menu trail for the main menu. * Changes to template filters and tags ** 'first' filter added optional length parameter ** min/max and minmax 3 new filters were added to clamp a value in an (integer) range. ** filesizeformat New filter, similar to the Django filesizeformat filter. ** lib tag Extended the lib tag with a 'use_absolute_url' option. ** confirm/alert actions These actions were changed and now use HTML dialogs instead of javascript popups. ** reversed New filter to reverse a list ** menu tag Added 'menu_id' parameter to specify which menu to render ** date_diff New filter to calculate the difference between two dates ** tinymce_add, tinymce_remove New actions to dynamically initialize of de-initialize rich textareas ** trigger_event New action to trigger a named wire. ** wire Added a new 'visible' wire type, which triggers when the wired element comes into view (by scrolling or using 'show'). ** lazy New scomp which shows a 'loader' image and performs onetime actions when loader comes into view. * General bug fixes - Fix for 'double-dot' in e-mails when using postfix. Also encode the $. characters using quoted-printable. - Fix for format_price filter. Show thousands when no cents. - Make video embed code editable. - Merged various webmachine fixes, updating it to 1.7.3: - support {stream, TotalSize, StreamFun} body result for range-capable streams - Add infinity timeout to gen_server calls - Allow multiple IP/port bindings - split chunk header on semicolon just in case a client is using chunk extensions - properly extract peername from all rfc1918 addrs - change H7 to match on any if-match, not just * - webmachine: WM-1.7.3(compat) ignores client's Content-Type on HTTP PUT requests (#130) - webmachine: prevent using chunked transfer encoding with HTTP/1.0. - increase the startup timeouts for the gen_servers to prevent startup race condition - Update mochiweb to latest version from mochi/mochiweb github repository (1.5.0) - Pulled latest epgsql driver to support Postgres notifications. - Added additional mime types (Office 2007, .rar) - z_session: Only mark the persistent store as dirty when a persistent value changes. - pgsql: Fix for a problem where a postgres connection was not returned to the pool in case of a sql error. - z_media_preview: some files without a preview where not showing an icon. - fixed an DoS vulnerability in Mochiweb/SSL - Added flushing for most periodic internal messages (e.g. tick, poll) - windows: fix build.cmd; remove some unix-specificness from imagemagick shell commands - mochiweb: Cookie expire date format string now follows rfc2109 - ACL checks on static file serving - Comet: support for cross-domain comet connections From ebegumisa@REDACTED Sat Feb 12 18:02:23 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sun, 13 Feb 2011 04:02:23 +1100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> Message-ID: On Sun, 13 Feb 2011 02:40:09 +1100, Masklinn wrote: > > On 2011-02-12, at 15:52 , Joe Armstrong wrote: > >> On Sat, Feb 12, 2011 at 3:29 PM, Masklinn wrote: >> >>> On 2011-02-12, at 14:03 , Joe Armstrong wrote: >>>> As I see it the web is predominantly built with one concurrency >>>> pattern. >>>> >>>> A client (web browser) can do an RPC to a server >>>> >>>> We can't (easily) >>>> >>>> do an RPC from the server to the client >>>> send an asynchronous message from server to client >>>> send an asynchronous message from the client to the server >>> This is incorrect. HTTP does not support "push" messaging (from a >>> server to >>> a client), but the main (and default) communication mechanism in >>> Javascript >>> is asynchronous (and event-driven). Unless you have a very different >>> definition of "asynchronous message" than the one I'm used to. >>> Possibly you do have different definitions of "asynchronous message" :) I don't think client-side JavaScript actually push asynchronous messages over the wire, it just looks like this from the web-programmer's perspective. See below... >> >> To what does the word "this" apply? >> >> "this" (ie the first word of your comment) could refer to >> any combination of four things > This referes to what i quoted. The sum is incorrect since some of its > parts (namely your assertion that asynchronous client->server > communications are "not easy") are incorrect. Which is what I explain in > the next three lines. > >> - I said we can't easily ... send an >> asynchronous message >> from the server to the client - I am fully aware the HTTP does not >> support >> this that >> why I said "easily" it can be done with horrible work arounds like comet >> etc. > You also said, in the same bundle, that "we can't easily send an > asynchronous message from the client to the server". Which is completely > incorrect. > I have to disagree, I think the statement is perfectly correct. I believe what Joe meant here is: * From the client, HTTP requests are not asynchronous because HTTP requests (GET/PUT/POST) by definition await a response from the server (200 OK, 404, etc.) This is not send and forget. It's send and wait. It's synchronous communication built on asynchronous communication. * The browser hides the above fact from you (the web-developer) by enabling the single-threaded JavaScript environment to do other work in-between the request and the response even executing other requests (aka AJAX.) So your JavaScript is async but the actual messages (HTTP requests) are not. * One workaround to send true async messages from the client to the server (which *I* know of and have used) is to multipart POST with a very large content-length header value and keep adding to the body, say lines of JSON, and allow the server to "eat" them as they come while not concluding the request for as long as possible. A kind long-poll in reverse. This is a pain for both client code and server code which is why I think Joe is correct when he says it's not "easy". - Edmond - >>>> Turning web sockets off for security reasons is stupid. >>>> >>>> The solution is to sandbox the thing that receives the messages >>>> or if unsandboxed strongly authenticate the messages. >>> I don't follow you here. You're saying it's OK to release broken >>> standards >>> (even though they're fixable) and asserting that every implementor >>> should go >>> through the process of creating his own broken sandbox which will fail >>> to be >>> correctly secured? >> I didn't say it was ok to release broken standards. > You said turning off web sockets due to them being broken was stupid. It > follows that you advocate releasing and enabling websockets in a broken > state. > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From erlang@REDACTED Sat Feb 12 18:50:33 2011 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 12 Feb 2011 18:50:33 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> Message-ID: On Sat, Feb 12, 2011 at 4:40 PM, Masklinn wrote: > > On 2011-02-12, at 15:52 , Joe Armstrong wrote: > > > On Sat, Feb 12, 2011 at 3:29 PM, Masklinn wrote: > > > >> On 2011-02-12, at 14:03 , Joe Armstrong wrote: > >>> As I see it the web is predominantly built with one concurrency > pattern. > >>> > >>> A client (web browser) can do an RPC to a server > >>> > >>> We can't (easily) > >>> > >>> do an RPC from the server to the client > >>> send an asynchronous message from server to client > >>> send an asynchronous message from the client to the server > >> This is incorrect. HTTP does not support "push" messaging (from a server > to > >> a client), but the main (and default) communication mechanism in > Javascript > >> is asynchronous (and event-driven). Unless you have a very different > >> definition of "asynchronous message" than the one I'm used to. > >> > > > > To what does the word "this" apply? > > > > "this" (ie the first word of your comment) could refer to > > any combination of four things > This referes to what i quoted. The sum is incorrect since some of its parts > (namely your assertion that asynchronous client->server communications are > "not easy") are incorrect. Which is what I explain in the next three lines. > > > - I said we can't easily ... send an > > asynchronous message > > from the server to the client - I am fully aware the HTTP does not > support > > this that > > why I said "easily" it can be done with horrible work arounds like comet > > etc. > You also said, in the same bundle, that "we can't easily send an > asynchronous message from the client to the server". Which is completely > incorrect. > How the heck can you easily send an an asynchronous message from a client to a server? HTTP get and post wait for a reply and thus are synchronous. By asynchronous I mean "fire and forget" like UDP. If it easy easy them I'd love to know how .. > > >>> Turning web sockets off for security reasons is stupid. > >>> > >>> The solution is to sandbox the thing that receives the messages > >>> or if unsandboxed strongly authenticate the messages. > >> I don't follow you here. You're saying it's OK to release broken > standards > >> (even though they're fixable) and asserting that every implementor > should go > >> through the process of creating his own broken sandbox which will fail > to be > >> correctly secured? > > I didn't say it was ok to release broken standards. > You said turning off web sockets due to them being broken was stupid. It > follows that you advocate releasing and enabling websockets in a broken > state. I never mentioned "releasing" or "enabling" in my post so you can't say I advocated either of these. I said turning them off was stupid - nothing else. Actually i wasn't bothered about whether they were broken or not. I was advocating using them because they are useful. And turning off something that is useful is stupid. Windows, for example, allows viruses (by stupid design) but it is still useful. Virtually all software I've ever seen is broken in one way or another. It can still be useful even if it is broken. It's not a question of absolutes here. Even if they are broken they are still useful and i can easily put a layer of strong crypto on top if I were worried - which I'm not. /Joe From kvs@REDACTED Sat Feb 12 19:06:15 2011 From: kvs@REDACTED (Konstantin Sorokin) Date: Sat, 12 Feb 2011 21:06:15 +0300 Subject: rebar and subversion Message-ID: Hello! I just noticed that that 'rebar clean' command removes ebin/.svn/tmp directory so that subversion feels very unhappy about it kvs@REDACTED myapp:> ls ebin/.svn/ entries text-base tmp kvs@REDACTED myapp:> ./rebar clean ==> lhttpc (clean) ==> mochiweb (clean) ==> rel (clean) ==> myapp (clean) kvs@REDACTED myapp:> ls ebin/.svn/ entries text-base kvs@REDACTED myapp:> -- Konstantin Sorokin From jxm@REDACTED Sat Feb 12 20:10:55 2011 From: jxm@REDACTED (Jerome Martin) Date: Sat, 12 Feb 2011 20:10:55 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <09486744-8EA4-455C-BF03-1109F27F9850@worrell.nl> Message-ID: On Sat, Feb 12, 2011 at 4:34 PM, Masklinn wrote: > On 2011-02-12, at 15:34 , Marc Worrell wrote: > > > > By the way, websockets is not broken. Some proxies are broken. > If these proxies being broken create a *security issue* with using > websockets, then websockets themselves are broken. > This must be one of the most nonsensical sentence I have read since a long time from someone supposedly mastering basic logic. Nice one. -- J?r?me Martin From jameschurchman@REDACTED Sat Feb 12 20:35:21 2011 From: jameschurchman@REDACTED (James Churchman) Date: Sat, 12 Feb 2011 19:35:21 +0000 Subject: gproc Message-ID: <1FFD5974-48DC-43F6-85FB-F9EDB4BAF42A@gmail.com> Firstly, gprocs great :-) should really come as part of the erlang distro! but i have two questions : "First" : ------- first/1 first(Type::type()) -> key() | '$end_of_table' Behaves as ets:first(Tab) for a given type of registration object. See http://www.erlang.org/doc/man/ets.html#first-1. The registry behaves as an ordered_set table. ------- Why does it behave as an ordered set? does this have any advantages ? i see with select/123 you can pattern match but even in this case does it help much ? and does begin ordered strongly effect performance, especially as i am only ever using it as a (single) process registry? can you pick an unordered set if you don't need it? "Second" : I want to do a fairly common pattern of registering a process and using it, unless its already registered where i use the registered one so for a local registration : gproc:reg({n, l,"ID"}). however on a second (and therefor "failing" ) registration it gives exception error: bad argument: [{gproc,chk_reply,{reg,{n,l,ok2},undefined}}, {erl_eval,do_apply,5}, {shell,exprs,7}, {shell,eval_exprs,7}, {shell,eval_loop,3}] i can obviously catch this, but it would be nicer if it did the genserver style of returning {error , already_exists , PID / Value_Stored_At_That_Key} the alternate is to do a lookup first, but as ~98 % of the time in my case it wont be registered this seems a bit counter intuitive also in either case, there is a small race condition in that : 1) i look up the process, its not there, but before i register it, someone else did (so it throws... ) 2) register the process, it exists, look it up, and in that time the process has ended its usefulness and has shutdown, so when i message it, its gone in both cases i can add extra code to register a 2nd time etc... but it seems a tad excessive are there better ways of doing things that im missing, or is this simply the way things are in a distributed multithreaded environment :-) From ebegumisa@REDACTED Sat Feb 12 21:03:03 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sun, 13 Feb 2011 07:03:03 +1100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <09486744-8EA4-455C-BF03-1109F27F9850@worrell.nl> Message-ID: On Sun, 13 Feb 2011 06:10:55 +1100, Jerome Martin wrote: > On Sat, Feb 12, 2011 at 4:34 PM, Masklinn wrote: > >> On 2011-02-12, at 15:34 , Marc Worrell wrote: >> > >> > By the way, websockets is not broken. Some proxies are broken. >> If these proxies being broken create a *security issue* with using >> websockets, then websockets themselves are broken. >> > > This must be one of the most nonsensical sentence I have read since a > long > time from someone supposedly mastering basic logic. Nice one. > > Sounded like circular logic, but I _kind of_ see what he was trying to say ;) Browsers, much like OSs, are in the unfortunate position of having to be careful what they introduce coz of the impact on other-people's faulty code. Too careful in the websockets case. Web developers have been suffering from the uni-directional issue for too long. It's not a minor limitation, it's major. Broken proxies just need to be fixed and not pass the blame. I've waited in vain for the Mozilla Framework (and hence Firefox) to support websockets. The debate reminds me of Linus rather rudely complaining about the undue status given to fixing "security" bugs over "normal" bugs... http://thread.gmane.org/gmane.linux.kernel/701694/focus=706950 The solution might be for web-developers to move away from browsers and handle their own security. - Edmond - -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From overminddl1@REDACTED Sat Feb 12 21:27:17 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Sat, 12 Feb 2011 13:27:17 -0700 Subject: [erlang-questions] gproc In-Reply-To: <1FFD5974-48DC-43F6-85FB-F9EDB4BAF42A@gmail.com> References: <1FFD5974-48DC-43F6-85FB-F9EDB4BAF42A@gmail.com> Message-ID: On Sat, Feb 12, 2011 at 12:35 PM, James Churchman wrote: > Firstly, gprocs great :-) should really come as part of the erlang distro! > > but i have two questions : > > "First" : > > ------- > first/1 > > first(Type::type()) -> key() | '$end_of_table' > > > Behaves as ets:first(Tab) for a given type of registration object. > > See http://www.erlang.org/doc/man/ets.html#first-1. The registry behaves as an ordered_set table. > > ------- > > Why does it behave as an ordered set? does this have any advantages ? i see with select/123 you can pattern match but even in this case does it help much ? and does begin ordered strongly effect performance, especially as i am only ever using it as a (single) process registry? can you pick an unordered set if you don't need it? > > > "Second" : > > I want to do a fairly common pattern of registering a process and using it, unless its already registered where i use the registered one > > so for a local registration : > > gproc:reg({n, l,"ID"}). > > however on a second (and therefor "failing" ) registration it gives > > exception error: bad argument: [{gproc,chk_reply,{reg,{n,l,ok2},undefined}}, ? ? ? ? {erl_eval,do_apply,5}, ? ? {shell,exprs,7}, ? ? {shell,eval_exprs,7}, ? ? ?{shell,eval_loop,3}] > > i can obviously catch this, but it would be nicer if it did the genserver style of returning {error , already_exists , PID / Value_Stored_At_That_Key} > > the alternate is to do a lookup first, but as ~98 % of the time in my case it wont be registered this seems a bit counter intuitive > > also in either case, there is a small race condition in that : > > 1) i look up the process, its not there, but before i register it, someone else did (so it throws... ) > 2) register the process, it exists, look it up, and in that time the process has ended its usefulness and has shutdown, so when i message it, its gone > > in both cases i can add extra code to register a 2nd time etc... but it seems a tad excessive > > are there better ways of doing things that im missing, or is this simply the way things are in a distributed multithreaded environment :-) nprocreg (made by but is a standalone dependency of Nitrogen) does what you want (and a bit more): https://github.com/nitrogen/nprocreg Just link it in, start up its sup (or application), and use it like this: nprocreg:get_pid(pid_name). % This will return an existing proc with pid_name name as Pid or will return undefined nprocreg:get_pid(pid_name, Fun). % This will return an existing proc with pid_name name as Pid, or will create a new process with the name pid_name if the Fun in a fun, or will return undefined is Fun is undefined. The extra that it does that you may or may not want is that it distributed processes across all connected nodes that are running nprocreg, which may or may not be what you want. Do not quote me on this, but I do not think it handles nodes being dynamically added/removed. From alain.odea@REDACTED Sat Feb 12 22:53:30 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sat, 12 Feb 2011 18:23:30 -0330 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <09486744-8EA4-455C-BF03-1109F27F9850@worrell.nl> Message-ID: On 2011-02-12, at 12:04, Masklinn wrote: > On 2011-02-12, at 15:34 , Marc Worrell wrote: >> >> By the way, websockets is not broken. Some proxies are broken. > If these proxies being broken create a *security issue* with using websockets, then websockets themselves are broken. To clarify for the unwary: No exploit lab or otherwise exists that functions in the context of the whole websockets stack from Chrome or Firefox. The vulnerability is identified on one component in isolation where it never would be in any real system. From ulf.wiger@REDACTED Sun Feb 13 08:54:11 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 13 Feb 2011 08:54:11 +0100 Subject: [erlang-questions] gproc In-Reply-To: <1FFD5974-48DC-43F6-85FB-F9EDB4BAF42A@gmail.com> References: <1FFD5974-48DC-43F6-85FB-F9EDB4BAF42A@gmail.com> Message-ID: On 12 Feb 2011, at 20:35, James Churchman wrote: > haves as ets:first(Tab) for a given type of registration object. > > See http://www.erlang.org/doc/man/ets.html#first-1. The registry behaves as an ordered_set table. > > ------- > > Why does it behave as an ordered set? does this have any advantages ? i see with select/123 you can pattern match but even in this case does it help much ? and does begin ordered strongly effect performance, especially as i am only ever using it as a (single) process registry? can you pick an unordered set if you don't need it? Actually, gproc relies on the ordered_set semantics in several places, not least when cleaning up after process death. It is possible that one could slightly improve performance using bag semantics, but then, having ordered_set semantics allows users to register complex properties, and do efficient queries on them (by binding the first part of the key). Performance of ordered_set is quite good; I did use initial tests with 100,000 registered processes, and was quite pleased with how it scaled. > "Second" : > > I want to do a fairly common pattern of registering a process and using it, unless its already registered where i use the registered one > > so for a local registration : > > gproc:reg({n, l,"ID"}). > > however on a second (and therefor "failing" ) registration it gives > > exception error: bad argument: [{gproc,chk_reply,{reg,{n,l,ok2},undefined}}, {erl_eval,do_apply,5}, {shell,exprs,7}, {shell,eval_exprs,7}, {shell,eval_loop,3}] > > i can obviously catch this, but it would be nicer if it did the genserver style of returning {error , already_exists , PID / Value_Stored_At_That_Key} Hmm, one way you could do this would be: get_pid(Name, F) -> spawn(fun() -> gproc:add_local_name(Name), F() end), gproc:await({n,l,Name}). ?or did you want a function similar to wait, where the calling process registers itself if there isn't already a process registered? BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From fw@REDACTED Sun Feb 13 11:18:53 2011 From: fw@REDACTED (Florian Weimer) Date: Sun, 13 Feb 2011 11:18:53 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: (Joe Armstrong's message of "Sat, 12 Feb 2011 18:50:33 +0100") References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> Message-ID: <87pqqwme42.fsf@mid.deneb.enyo.de> * Joe Armstrong: > How the heck can you easily send an an asynchronous message from a client to > a server? > > HTTP get and post wait for a reply and thus are synchronous. > > By asynchronous I mean "fire and forget" like UDP. > > If it easy easy them I'd love to know how .. XMLHttpRequest has an asynchronous option. And you can get emulate a single full-duplex connection using two half-duplex ones, so Websockets is just an optimization, and likely not even a very much needed one. The barrier right now seems to be the BSD sockets API and kernel state management, and you still have that with Websockets. From kostis@REDACTED Sun Feb 13 13:22:24 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sun, 13 Feb 2011 14:22:24 +0200 Subject: [erlang-questions] why can't dialyzer see that my spec is a lie? In-Reply-To: <20110210110048.GA5027@corelatus.se> References: <20110210110048.GA5027@corelatus.se> Message-ID: <4D57CD00.5070203@cs.ntua.gr> Matthias Lang wrote: > Hi, > > Running the example below through dialyzer R14B01: > > 143> dialyzer:run([{from, src_code}, {files, ["minimal.erl"]}]). > [] > > I was hoping Dialyzer would complain that my spec (line 18) is a lie. > > Cut down, but compilable, code below. Changing "almost anything" results > in a good warning, e.g. if I comment out the recursive step (line 13): > > 145> dialyzer:run([{from, src_code}, {files, ["minimal.erl"]}]). > [{warn_contract_types,{"/tmp/minimal.erl",17}, > {extra_range,[minimal,parse_and_execute,1,"'quit'", > "{_,_}"]}}] > > What's going on? I'm stumped. In program analysis tools one needs to choose between soundness for correctness (i.e., find all errors and then some) and soundness for defect detection (i.e., find only true discrepancies but raise no false alarms). Dialyzer has opted for the latter, which means that it can occasionally fail to report some discrepancies between the code and its intended uses (as documented for example in programmer-supplied specs). That's what's happening here. In your program, dialyzer infers a return type of any() for your parse_and_execute/1 function and cannot see that it is invalid (actually, even the warning you get when you do the change you suggest in your mail is not the "right" one). I think this happens because it does not propagate information about closures across functions, esp. when an infinite loop is involved as in your program. We will look at this issue (thanks for reporting it) and try to fix it in a future dialyzer version. Kostis > ---------------------------------------------------------------------- > -module(minimal). > -export([start/0]). > > -record(state, {commands}). > > start() -> > Fun = fun() -> {quit, "string"} end, > server_loop(#state{commands=Fun}). > > server_loop(State) -> > try > {New_state, _} = parse_and_execute(State), > server_loop(New_state) > catch > _:_ -> ok > end. > > -spec parse_and_execute(any()) -> {#state{}, string()} | 'quit'. > > parse_and_execute(State) -> > Fun = State#state.commands, > Fun(). > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From dizzyd@REDACTED Sun Feb 13 14:30:45 2011 From: dizzyd@REDACTED (Dave Smith) Date: Sun, 13 Feb 2011 06:30:45 -0700 Subject: [erlang-questions] rebar and subversion In-Reply-To: References: Message-ID: Hi Konstantin, On Sat, Feb 12, 2011 at 11:06 AM, Konstantin Sorokin wrote: > Hello! > > I just noticed that that 'rebar clean' command removes ebin/.svn/tmp > directory so that subversion feels very unhappy about it Yes, rebar tries to clean out the ebin/ directory (and nested subdirs) rather thoroughly. The Erlang compilation can generate nested directories (if you're using packages), so rebar tries to accomodate that. D. From alain.odea@REDACTED Sun Feb 13 14:42:30 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sun, 13 Feb 2011 10:12:30 -0330 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: <87pqqwme42.fsf@mid.deneb.enyo.de> References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> <87pqqwme42.fsf@mid.deneb.enyo.de> Message-ID: <88F9C3CA-C2DB-44AA-8092-52D78DA0D7BF@gmail.com> On 2011-02-13, at 6:48, Florian Weimer wrote: > * Joe Armstrong: > >> How the heck can you easily send an an asynchronous message from a client to >> a server? >> >> HTTP get and post wait for a reply and thus are synchronous. >> >> By asynchronous I mean "fire and forget" like UDP. >> >> If it easy easy them I'd love to know how .. > > XMLHttpRequest has an asynchronous option. > > And you can get emulate a single full-duplex connection using two > half-duplex ones, so Websockets is just an optimization, and likely > not even a very much needed one. The barrier right now seems to be > the BSD sockets API and kernel state management, and you still have > that with Websockets. HTTP is a synchronous protocol. The async option is XmlHttpRequest is only to background the synchronous request processing so that the browser remains responsive. Websockets is not merely an optimization. It removes the packaging around conversational events in a web application. This has far-reaching scalability impact since it reduces the cost per client substantially in a chatty application. For an application using SEBG tge reduced cost per request will reduce the delay between a server event and its realization on the client. Again, XmlHttpRequest's async option does not make the HTTP request itself asynchronous (it wouldn't be HTTP). It merely runs the synchronous requedt in the background on the client. From bob@REDACTED Sun Feb 13 14:41:02 2011 From: bob@REDACTED (Bob Ippolito) Date: Sun, 13 Feb 2011 21:41:02 +0800 Subject: [erlang-questions] rebar and subversion In-Reply-To: References: Message-ID: On Sun, Feb 13, 2011 at 9:30 PM, Dave Smith wrote: > Hi Konstantin, > > On Sat, Feb 12, 2011 at 11:06 AM, Konstantin Sorokin wrote: >> Hello! >> >> I just noticed that that 'rebar clean' command removes ebin/.svn/tmp >> directory so that subversion feels very unhappy about it > > Yes, rebar tries to clean out the ebin/ directory (and nested subdirs) > rather thoroughly. The Erlang compilation can generate nested > directories (if you're using packages), so rebar tries to accomodate > that. Another option is to simply not have an ebin dir, and have the .app file get generated by rebar with an .app.src file in your src directory. -bob From kaellis@REDACTED Sun Feb 13 14:45:11 2011 From: kaellis@REDACTED (Ken Ellis) Date: Sun, 13 Feb 2011 08:45:11 -0500 Subject: [erlang-questions] rebar and subversion In-Reply-To: References: Message-ID: Konstantin, Note that it will recursively delete all *empty* directories under ebin, after deleting all .beam and .[xy]rl files. So will only remove .svn/tmp if its empty. A little heavy-handed. But you might be able to put a dummy file in there if its a problem, or patch (rebar_erlc_compiler:clean) for your needs. Ken On Sun, Feb 13, 2011 at 8:30 AM, Dave Smith wrote: > Hi Konstantin, > > On Sat, Feb 12, 2011 at 11:06 AM, Konstantin Sorokin wrote: >> Hello! >> >> I just noticed that that 'rebar clean' command removes ebin/.svn/tmp >> directory so that subversion feels very unhappy about it > > Yes, rebar tries to clean out the ebin/ directory (and nested subdirs) > rather thoroughly. The Erlang compilation can generate nested > directories (if you're using packages), so rebar tries to accomodate > that. > > D. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jwilberding@REDACTED Sun Feb 13 14:54:26 2011 From: jwilberding@REDACTED (Jordan Wilberding) Date: Sun, 13 Feb 2011 08:54:26 -0500 Subject: Deprecating projects Message-ID: I was trying to find a nice way to deprecate a project, but it appears the way Erlang does it is by adding the deprecations into stdlib/src/otp_internal.erl. Is there no other way to make the compiler warn about using a deprecated module/function? Thanks! Jordan Wilberding From andrey.pampukha@REDACTED Sun Feb 13 15:25:56 2011 From: andrey.pampukha@REDACTED (Andrey Pampukha) Date: Sun, 13 Feb 2011 17:25:56 +0300 Subject: Java-style "packaged" module names in bif.tab Message-ID: Hi all, Just curious about some BIFs named in the following notation found in erts/emulator/beam/bif.tab: bif 'erl.lang.float':from_string/1 bif 'erl.lang.integer':from_string/1 bif 'erl.lang.proc':string_to_pid/1 bif 'erl.lang.tuple':from_list/1 bif 'erl.system.code':load/2 bif 'erl.system.code':loaded/0 bif 'erl.util.date':local/0 For how long and why they are there? I've read manual page for 'packages' ( http://erlang.org/doc/man/packages.html) and I guess all those 'erl'.* modules were introduced at the same moment as this unsupported feature. Am I right? Is there any plans to occasionally start supporting this? I know it's not very important "feature", but it could be interesting... Thanx! Andrey. From ebegumisa@REDACTED Sun Feb 13 15:33:36 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Mon, 14 Feb 2011 01:33:36 +1100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: <87pqqwme42.fsf@mid.deneb.enyo.de> References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> <87pqqwme42.fsf@mid.deneb.enyo.de> Message-ID: On Sun, 13 Feb 2011 21:18:53 +1100, Florian Weimer wrote: > * Joe Armstrong: > >> How the heck can you easily send an an asynchronous message from a >> client to >> a server? >> >> HTTP get and post wait for a reply and thus are synchronous. >> >> By asynchronous I mean "fire and forget" like UDP. >> >> If it easy easy them I'd love to know how .. > > XMLHttpRequest has an asynchronous option. > There seems to be some confusion. Forgive me, but aren't asynchronous HTTP requests very different from asynchronous messages? Isn't the "asynchronous" in XMLHttpRequest talking about the scripting environment in relation to thread-blocking and not the messages themselves? Asynchronous XMLHttpRequest just means that you can do something else while you wait for the reply, but a failed reply is still a fault so the message depends on the reply (or the reply can be viewed as part of the message). XMLHttpRequest is just a workaround to allow single-threaded scripting environments not to block so they can handle multiple requests without introducing multi-threading, but you *always* have to deal with the reply. My understanding is XMLHttpRequest in whatever mode does not magically turn HTTP requests into asynchronous *messages*. What is an "asynchronous message" anyway? AFAIK, it means send and forget. For Erlang's case this is... Recipient ! Message From here on, you are free to go about your business and not have to deal with any response. The recipient need not send you back any acknowledgement to make the message "final". The message is thus asynchronous because you send and forget. Compare this with HTTP... PUT /foo?message=Message HTTP/1.1 From here, the message is not complete until the server responds... HTTP/1.1 201 Created [or 504 or whatever] For XMLHttpRequest in synchronous mode, the scripting environment's thread blocks until the message is concluded. For XMLHttpRequest in asynchronous mode, the scripting environment's thread churns on but the message is still not concluded until the reply arrives. The message is thus synchronous either way because you don't send and forget, you have to write code to deal with the reply. Correct me if I'm wrong, but if you send and have to deal with a response from the recipient, regardless of whether you can do something else as you await the response, the message itself is not asynchronous. An HTTP message consists of 1) send and 2) get reply, so is synchronous by definition. XMLHttpRequest can't erase this, but it can allow you to workaround it. The workaround is an ugly hack that involves packing many "sub" messages into the request and waiting forever for a reply while taking advantage of non-thread-blocking in the scripting environment (i.e. comet/long-poll)... So, to send truly asynchronous *messages* from server to client, the client would... GET /foo/comet_interface HTTP/1.1 Server responds, streaming back a reply while not concluding the reply for as long as it can hold the connection open. It first sends headers, then keeps appending the asynchronous messages... HTTP/1.1 200 OK ..Headers.. InnerMessage1 InnerMessage2 .. InnerMessageN Client then uses it's non-thread-blocking XMLHttpRequest to read the inner messages as they arrive and tries not to close the connection. The client has to know how to chop them up and the server might need to keep sending a regular special heartbeat sequence when idle to keep the connection open. So the "outer" HTTP GET message itself is synchronous (send and wait for reply), but the "inner" messages packed inside are asynchronous (send and forget). This is ugly, but web-developers have made it work out of desperation. To send truly asynchronous messages from client to server is even tricker and uglier. It might looks something like... POST /foo/multipart_interface HTTP/1.1 Content-type: multipart/form-data, boundary=AaB03x --AaB03x content-disposition: form-data; name="message1" InnerMessage1 --AaB03x content-disposition: form-data; name="message2" InnerMessage2 --AaB03x ... content-disposition: form-data; name="messageN" InnerMessageN --AaB03x-- Again, the "outer" HTTP POST message is synchronous (send and wait for reply), but the "inner" messages are asynchronous (send and forget). This hackery is harder than it needs to be. > And you can get emulate a single full-duplex connection using two > half-duplex ones, Synchronous full-duplex messages are still synchronous messages. Both sides still expect replies from the other, they can just do so at the same time. What you can do, as illustrated above, is simulate asynchronous messages with synchronous HTTP. And it's complete pain in the arse. > so Websockets is just an optimization, I disagree. WebSockets allow you to send asynchronous full-duplex messages without the hackery of comet/long-polling. Just look at how clean Joe's code is. A websocket-less version would be possible, but uglier. > and likely not even a very much needed one. IMO, the very fact that web-developers have frequently used various workarounds proves that it's needed. There is a disconnect between what web developers want and what the infrastructure currently provides. Websockets tries to address some of that. Personally, I think it's *very* much needed because it opens up the possibilities of what you can do with the web-browser as a client. The web browser starts to look more like a ordinary TCP client. You can do more of the things that are easy with plain sockets which have always frustrated me with browsers. Just look at Joe's idea. Those sorts of ideas don't immediately spring to mind when you're stuck in the mindset of uni-directional synchronous messages. - Edmond - > The barrier right now seems to be > the BSD sockets API and kernel state management, and you still have > that with Websockets. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From tramjoe.merin@REDACTED Sun Feb 13 18:08:05 2011 From: tramjoe.merin@REDACTED (Jerome Martin) Date: Sun, 13 Feb 2011 18:08:05 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> <87pqqwme42.fsf@mid.deneb.enyo.de> Message-ID: Just my 2cents.... (warning, if you are prompt to be upset by dramatically different opinion than your, please do not read the following) On a totally (not so ?) subjective mode: - (my) Taste tells me Joe's code is beautiful. - If such beautiful code is conceptually possible, it seems the right thing to do (remember, we are in subjective mode) - Preventing that beauty from happening (not supporting features enabling it), or claiming it is not necessary (presenting ugly code that does not feel right and claiming it is the same) seems like the worst thing to do, even if alternatives can be motivated (security reasons, etc.). Now, on a purely pragmatic mode: - My experience (dealing with computer code since 20 years) tells me that my taste buds are not that subjective, and are more than often right: usually, I find (or someone else does) later a technical demonstration that intuition and taste got it right. - It is rare enough to trigger taste to a level that we can call beautiful, so definitely Joe must be on to something there :) To everyone with rotten taste buds (serious mode but I won't argue or defend my views): - No, taste is not that subjective. I do think some people have bad taste and some other good taste, in an absolute sense. Matters of taste are a tale imagined not to upset anyone. - Can't you see that there is something beyond technical arguments thee that yells "this is the right way to do it" ? I mean, really ? On Sun, Feb 13, 2011 at 3:33 PM, Edmond Begumisa < ebegumisa@REDACTED> wrote: > On Sun, 13 Feb 2011 21:18:53 +1100, Florian Weimer > wrote: > > * Joe Armstrong: >> >> How the heck can you easily send an an asynchronous message from a client >>> to >>> a server? >>> >>> HTTP get and post wait for a reply and thus are synchronous. >>> >>> By asynchronous I mean "fire and forget" like UDP. >>> >>> If it easy easy them I'd love to know how .. >>> >> >> XMLHttpRequest has an asynchronous option. >> >> > There seems to be some confusion. Forgive me, but aren't asynchronous HTTP > requests very different from asynchronous messages? Isn't the "asynchronous" > in XMLHttpRequest talking about the scripting environment in relation to > thread-blocking and not the messages themselves? > > Asynchronous XMLHttpRequest just means that you can do something else while > you wait for the reply, but a failed reply is still a fault so the message > depends on the reply (or the reply can be viewed as part of the message). > XMLHttpRequest is just a workaround to allow single-threaded scripting > environments not to block so they can handle multiple requests without > introducing multi-threading, but you *always* have to deal with the reply. > My understanding is XMLHttpRequest in whatever mode does not magically turn > HTTP requests into asynchronous *messages*. > > What is an "asynchronous message" anyway? > > AFAIK, it means send and forget. For Erlang's case this is... > > Recipient ! Message > > From here on, you are free to go about your business and not have to deal > with any response. The recipient need not send you back any acknowledgement > to make the message "final". The message is thus asynchronous because you > send and forget. > > Compare this with HTTP... > > PUT /foo?message=Message HTTP/1.1 > > From here, the message is not complete until the server responds... > > HTTP/1.1 201 Created [or 504 or whatever] > > For XMLHttpRequest in synchronous mode, the scripting environment's thread > blocks until the message is concluded. For XMLHttpRequest in asynchronous > mode, the scripting environment's thread churns on but the message is still > not concluded until the reply arrives. The message is thus synchronous > either way because you don't send and forget, you have to write code to deal > with the reply. > > Correct me if I'm wrong, but if you send and have to deal with a response > from the recipient, regardless of whether you can do something else as you > await the response, the message itself is not asynchronous. An HTTP message > consists of 1) send and 2) get reply, so is synchronous by definition. > XMLHttpRequest can't erase this, but it can allow you to workaround it. > > The workaround is an ugly hack that involves packing many "sub" messages > into the request and waiting forever for a reply while taking advantage of > non-thread-blocking in the scripting environment (i.e. comet/long-poll)... > > So, to send truly asynchronous *messages* from server to client, the client > would... > > GET /foo/comet_interface HTTP/1.1 > > Server responds, streaming back a reply while not concluding the reply for > as long as it can hold the connection open. It first sends headers, then > keeps appending the asynchronous messages... > > HTTP/1.1 200 OK > ..Headers.. > InnerMessage1 > InnerMessage2 > .. > InnerMessageN > > Client then uses it's non-thread-blocking XMLHttpRequest to read the inner > messages as they arrive and tries not to close the connection. The client > has to know how to chop them up and the server might need to keep sending a > regular special heartbeat sequence when idle to keep the connection open. So > the "outer" HTTP GET message itself is synchronous (send and wait for > reply), but the "inner" messages packed inside are asynchronous (send and > forget). This is ugly, but web-developers have made it work out of > desperation. > > To send truly asynchronous messages from client to server is even tricker > and uglier. It might looks something like... > > POST /foo/multipart_interface HTTP/1.1 > > Content-type: multipart/form-data, boundary=AaB03x > > --AaB03x > content-disposition: form-data; name="message1" > > InnerMessage1 > --AaB03x > content-disposition: form-data; name="message2" > > InnerMessage2 > --AaB03x > > ... > > content-disposition: form-data; name="messageN" > > InnerMessageN > --AaB03x-- > > Again, the "outer" HTTP POST message is synchronous (send and wait for > reply), but the "inner" messages are asynchronous (send and forget). This > hackery is harder than it needs to be. > > > And you can get emulate a single full-duplex connection using two >> half-duplex ones, >> > > Synchronous full-duplex messages are still synchronous messages. Both sides > still expect replies from the other, they can just do so at the same time. > What you can do, as illustrated above, is simulate asynchronous messages > with synchronous HTTP. And it's complete pain in the arse. > > > so Websockets is just an optimization, >> > > I disagree. WebSockets allow you to send asynchronous full-duplex messages > without the hackery of comet/long-polling. Just look at how clean Joe's code > is. A websocket-less version would be possible, but uglier. > > > and likely not even a very much needed one. >> > > IMO, the very fact that web-developers have frequently used various > workarounds proves that it's needed. There is a disconnect between what web > developers want and what the infrastructure currently provides. Websockets > tries to address some of that. > > Personally, I think it's *very* much needed because it opens up the > possibilities of what you can do with the web-browser as a client. The web > browser starts to look more like a ordinary TCP client. You can do more of > the things that are easy with plain sockets which have always frustrated me > with browsers. Just look at Joe's idea. Those sorts of ideas don't > immediately spring to mind when you're stuck in the mindset of > uni-directional synchronous messages. > > - Edmond - > > > The barrier right now seems to be >> the BSD sockets API and kernel state management, and you still have >> that with Websockets. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- J?r?me Martin From ebegumisa@REDACTED Sun Feb 13 18:25:58 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Mon, 14 Feb 2011 04:25:58 +1100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> <87pqqwme42.fsf@mid.deneb.enyo.de> Message-ID: +1 The thread went off topic. Joe e-mailed me with an idea like this many months ago off-list. I liked it then and I like it now. I even borrowed little bits of it in my current XULRunner projects (XUL has some limitations preventing the exact model). In fact, I've been thinking of some ways adding some templating to his code. So you can send "groups of widgets" down to the browser to display. It also got me thinking about some sort of flow language that can be used for people that don't know Erlang so they can easily string screens and widgets together in a flow chart manner. - Edmond - On Mon, 14 Feb 2011 04:08:05 +1100, Jerome Martin wrote: > Just my 2cents.... > (warning, if you are prompt to be upset by dramatically different opinion > than your, please do not read the following) > > On a totally (not so ?) subjective mode: > - (my) Taste tells me Joe's code is beautiful. > - If such beautiful code is conceptually possible, it seems the right > thing to do (remember, we are in subjective mode) > - Preventing that beauty from happening (not supporting features > enabling > it), or claiming it is not necessary (presenting ugly code that does not > feel right and claiming it is the same) seems like the worst thing to do, > even if alternatives can be motivated (security reasons, etc.). > > Now, on a purely pragmatic mode: > - My experience (dealing with computer code since 20 years) tells me > that > my taste buds are not that subjective, and are more than often right: > usually, I find (or someone else does) later a technical demonstration > that > intuition and taste got it right. > - It is rare enough to trigger taste to a level that we can call > beautiful, > so definitely Joe must be on to something there :) > > To everyone with rotten taste buds (serious mode but I won't argue or > defend > my views): > - No, taste is not that subjective. I do think some people have bad > taste > and some other good taste, in an absolute sense. Matters of taste are a > tale > imagined not to upset anyone. > - Can't you see that there is something beyond technical arguments thee > that yells "this is the right way to do it" ? I mean, really ? > > On Sun, Feb 13, 2011 at 3:33 PM, Edmond Begumisa < > ebegumisa@REDACTED> wrote: > >> On Sun, 13 Feb 2011 21:18:53 +1100, Florian Weimer >> wrote: >> >> * Joe Armstrong: >>> >>> How the heck can you easily send an an asynchronous message from a >>> client >>>> to >>>> a server? >>>> >>>> HTTP get and post wait for a reply and thus are synchronous. >>>> >>>> By asynchronous I mean "fire and forget" like UDP. >>>> >>>> If it easy easy them I'd love to know how .. >>>> >>> >>> XMLHttpRequest has an asynchronous option. >>> >>> >> There seems to be some confusion. Forgive me, but aren't asynchronous >> HTTP >> requests very different from asynchronous messages? Isn't the >> "asynchronous" >> in XMLHttpRequest talking about the scripting environment in relation to >> thread-blocking and not the messages themselves? >> >> Asynchronous XMLHttpRequest just means that you can do something else >> while >> you wait for the reply, but a failed reply is still a fault so the >> message >> depends on the reply (or the reply can be viewed as part of the >> message). >> XMLHttpRequest is just a workaround to allow single-threaded scripting >> environments not to block so they can handle multiple requests without >> introducing multi-threading, but you *always* have to deal with the >> reply. >> My understanding is XMLHttpRequest in whatever mode does not magically >> turn >> HTTP requests into asynchronous *messages*. >> >> What is an "asynchronous message" anyway? >> >> AFAIK, it means send and forget. For Erlang's case this is... >> >> Recipient ! Message >> >> From here on, you are free to go about your business and not have to >> deal >> with any response. The recipient need not send you back any >> acknowledgement >> to make the message "final". The message is thus asynchronous because >> you >> send and forget. >> >> Compare this with HTTP... >> >> PUT /foo?message=Message HTTP/1.1 >> >> From here, the message is not complete until the server responds... >> >> HTTP/1.1 201 Created [or 504 or whatever] >> >> For XMLHttpRequest in synchronous mode, the scripting environment's >> thread >> blocks until the message is concluded. For XMLHttpRequest in >> asynchronous >> mode, the scripting environment's thread churns on but the message is >> still >> not concluded until the reply arrives. The message is thus synchronous >> either way because you don't send and forget, you have to write code to >> deal >> with the reply. >> >> Correct me if I'm wrong, but if you send and have to deal with a >> response >> from the recipient, regardless of whether you can do something else as >> you >> await the response, the message itself is not asynchronous. An HTTP >> message >> consists of 1) send and 2) get reply, so is synchronous by definition. >> XMLHttpRequest can't erase this, but it can allow you to workaround it. >> >> The workaround is an ugly hack that involves packing many "sub" messages >> into the request and waiting forever for a reply while taking advantage >> of >> non-thread-blocking in the scripting environment (i.e. >> comet/long-poll)... >> >> So, to send truly asynchronous *messages* from server to client, the >> client >> would... >> >> GET /foo/comet_interface HTTP/1.1 >> >> Server responds, streaming back a reply while not concluding the reply >> for >> as long as it can hold the connection open. It first sends headers, then >> keeps appending the asynchronous messages... >> >> HTTP/1.1 200 OK >> ..Headers.. >> InnerMessage1 >> InnerMessage2 >> .. >> InnerMessageN >> >> Client then uses it's non-thread-blocking XMLHttpRequest to read the >> inner >> messages as they arrive and tries not to close the connection. The >> client >> has to know how to chop them up and the server might need to keep >> sending a >> regular special heartbeat sequence when idle to keep the connection >> open. So >> the "outer" HTTP GET message itself is synchronous (send and wait for >> reply), but the "inner" messages packed inside are asynchronous (send >> and >> forget). This is ugly, but web-developers have made it work out of >> desperation. >> >> To send truly asynchronous messages from client to server is even >> tricker >> and uglier. It might looks something like... >> >> POST /foo/multipart_interface HTTP/1.1 >> >> Content-type: multipart/form-data, boundary=AaB03x >> >> --AaB03x >> content-disposition: form-data; name="message1" >> >> InnerMessage1 >> --AaB03x >> content-disposition: form-data; name="message2" >> >> InnerMessage2 >> --AaB03x >> >> ... >> >> content-disposition: form-data; name="messageN" >> >> InnerMessageN >> --AaB03x-- >> >> Again, the "outer" HTTP POST message is synchronous (send and wait for >> reply), but the "inner" messages are asynchronous (send and forget). >> This >> hackery is harder than it needs to be. >> >> >> And you can get emulate a single full-duplex connection using two >>> half-duplex ones, >>> >> >> Synchronous full-duplex messages are still synchronous messages. Both >> sides >> still expect replies from the other, they can just do so at the same >> time. >> What you can do, as illustrated above, is simulate asynchronous messages >> with synchronous HTTP. And it's complete pain in the arse. >> >> >> so Websockets is just an optimization, >>> >> >> I disagree. WebSockets allow you to send asynchronous full-duplex >> messages >> without the hackery of comet/long-polling. Just look at how clean Joe's >> code >> is. A websocket-less version would be possible, but uglier. >> >> >> and likely not even a very much needed one. >>> >> >> IMO, the very fact that web-developers have frequently used various >> workarounds proves that it's needed. There is a disconnect between what >> web >> developers want and what the infrastructure currently provides. >> Websockets >> tries to address some of that. >> >> Personally, I think it's *very* much needed because it opens up the >> possibilities of what you can do with the web-browser as a client. The >> web >> browser starts to look more like a ordinary TCP client. You can do more >> of >> the things that are easy with plain sockets which have always >> frustrated me >> with browsers. Just look at Joe's idea. Those sorts of ideas don't >> immediately spring to mind when you're stuck in the mindset of >> uni-directional synchronous messages. >> >> - Edmond - >> >> >> The barrier right now seems to be >>> the BSD sockets API and kernel state management, and you still have >>> that with Websockets. >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Sun Feb 13 18:54:59 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Mon, 14 Feb 2011 04:54:59 +1100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> <87pqqwme42.fsf@mid.deneb.enyo.de> Message-ID: On Mon, 14 Feb 2011 04:08:05 +1100, Jerome Martin wrote: > - Preventing that beauty from happening (not supporting features > enabling it), or claiming it is not necessary (presenting ugly code that > does not > feel right and claiming it is the same) seems like the worst thing to do, > even if alternatives can be motivated (security reasons, etc.). And BTW: I was presenting the ugly code precisely to show that although similar, it doesn't not feel right, and hence, NOT supporting the features enabling the beautiful code is holding everyone back from coming up with great ideas like Joe's. - Edmond - -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Sun Feb 13 18:57:24 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Mon, 14 Feb 2011 04:57:24 +1100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> <87pqqwme42.fsf@mid.deneb.enyo.de> Message-ID: On Mon, 14 Feb 2011 04:54:59 +1100, Edmond Begumisa wrote: > On Mon, 14 Feb 2011 04:08:05 +1100, Jerome Martin > wrote: > >> - Preventing that beauty from happening (not supporting features >> enabling it), or claiming it is not necessary (presenting ugly code >> that does not >> feel right and claiming it is the same) seems like the worst thing to >> do, >> even if alternatives can be motivated (security reasons, etc.). > > And BTW: I was presenting the ugly code precisely to show that although > similar, it doesn't not feel right, and hence, NOT supporting the OOPS: ^^^^^^^^^^^^^^^^^^^^^^^^^ TYPO: it doesn't feel right > features enabling the beautiful code is holding everyone back from > coming up with great ideas like Joe's. > > - Edmond - > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From tramjoe.merin@REDACTED Sun Feb 13 19:30:46 2011 From: tramjoe.merin@REDACTED (Jerome Martin) Date: Sun, 13 Feb 2011 19:30:46 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> <87pqqwme42.fsf@mid.deneb.enyo.de> Message-ID: On Sun, Feb 13, 2011 at 6:54 PM, Edmond Begumisa < ebegumisa@REDACTED> wrote: > On Mon, 14 Feb 2011 04:08:05 +1100, Jerome Martin > wrote: > > - Preventing that beauty from happening (not supporting features enabling >> it), or claiming it is not necessary (presenting ugly code that does not >> feel right and claiming it is the same) seems like the worst thing to do, >> even if alternatives can be motivated (security reasons, etc.). >> > > And BTW: I was presenting the ugly code precisely to show that although > similar, it doesn't not feel right, Yup, that was clear, don't worry :-) Sorry for phrasing my point in a way that could make one think you were defending that ugly code or presenting it as a viable alternative. and hence, NOT supporting the features enabling the beautiful code is > holding everyone back from coming up with great ideas like Joe's. > -- J?r?me Martin From tramjoe.merin@REDACTED Sun Feb 13 19:36:26 2011 From: tramjoe.merin@REDACTED (Jerome Martin) Date: Sun, 13 Feb 2011 19:36:26 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> <87pqqwme42.fsf@mid.deneb.enyo.de> Message-ID: Yes, since Joe's WebSocket blog post I think many people have had that idea in the back of their minds, floating around, waiting for some signal that WebSocket was out to the general public for good and could be considered as a usable in real life. Unfortunately, as we all realize, it is not the case yet, and most frustrating, probably for the wrong reasons :-( On Sun, Feb 13, 2011 at 6:25 PM, Edmond Begumisa < ebegumisa@REDACTED> wrote: > +1 > > The thread went off topic. Joe e-mailed me with an idea like this many > months ago off-list. I liked it then and I like it now. I even borrowed > little bits of it in my current XULRunner projects (XUL has some limitations > preventing the exact model). > > In fact, I've been thinking of some ways adding some templating to his > code. So you can send "groups of widgets" down to the browser to display. It > also got me thinking about some sort of flow language that can be used for > people that don't know Erlang so they can easily string screens and widgets > together in a flow chart manner. > > - Edmond - > > > > On Mon, 14 Feb 2011 04:08:05 +1100, Jerome Martin > wrote: > > Just my 2cents.... >> (warning, if you are prompt to be upset by dramatically different opinion >> than your, please do not read the following) >> >> On a totally (not so ?) subjective mode: >> - (my) Taste tells me Joe's code is beautiful. >> - If such beautiful code is conceptually possible, it seems the right >> thing to do (remember, we are in subjective mode) >> - Preventing that beauty from happening (not supporting features enabling >> it), or claiming it is not necessary (presenting ugly code that does not >> feel right and claiming it is the same) seems like the worst thing to do, >> even if alternatives can be motivated (security reasons, etc.). >> >> Now, on a purely pragmatic mode: >> - My experience (dealing with computer code since 20 years) tells me that >> my taste buds are not that subjective, and are more than often right: >> usually, I find (or someone else does) later a technical demonstration >> that >> intuition and taste got it right. >> - It is rare enough to trigger taste to a level that we can call >> beautiful, >> so definitely Joe must be on to something there :) >> >> To everyone with rotten taste buds (serious mode but I won't argue or >> defend >> my views): >> - No, taste is not that subjective. I do think some people have bad taste >> and some other good taste, in an absolute sense. Matters of taste are a >> tale >> imagined not to upset anyone. >> - Can't you see that there is something beyond technical arguments thee >> that yells "this is the right way to do it" ? I mean, really ? >> >> On Sun, Feb 13, 2011 at 3:33 PM, Edmond Begumisa < >> ebegumisa@REDACTED> wrote: >> >> On Sun, 13 Feb 2011 21:18:53 +1100, Florian Weimer >>> wrote: >>> >>> * Joe Armstrong: >>> >>>> >>>> How the heck can you easily send an an asynchronous message from a >>>> client >>>> >>>>> to >>>>> a server? >>>>> >>>>> HTTP get and post wait for a reply and thus are synchronous. >>>>> >>>>> By asynchronous I mean "fire and forget" like UDP. >>>>> >>>>> If it easy easy them I'd love to know how .. >>>>> >>>>> >>>> XMLHttpRequest has an asynchronous option. >>>> >>>> >>>> There seems to be some confusion. Forgive me, but aren't asynchronous >>> HTTP >>> requests very different from asynchronous messages? Isn't the >>> "asynchronous" >>> in XMLHttpRequest talking about the scripting environment in relation to >>> thread-blocking and not the messages themselves? >>> >>> Asynchronous XMLHttpRequest just means that you can do something else >>> while >>> you wait for the reply, but a failed reply is still a fault so the >>> message >>> depends on the reply (or the reply can be viewed as part of the message). >>> XMLHttpRequest is just a workaround to allow single-threaded scripting >>> environments not to block so they can handle multiple requests without >>> introducing multi-threading, but you *always* have to deal with the >>> reply. >>> My understanding is XMLHttpRequest in whatever mode does not magically >>> turn >>> HTTP requests into asynchronous *messages*. >>> >>> What is an "asynchronous message" anyway? >>> >>> AFAIK, it means send and forget. For Erlang's case this is... >>> >>> Recipient ! Message >>> >>> From here on, you are free to go about your business and not have to deal >>> with any response. The recipient need not send you back any >>> acknowledgement >>> to make the message "final". The message is thus asynchronous because you >>> send and forget. >>> >>> Compare this with HTTP... >>> >>> PUT /foo?message=Message HTTP/1.1 >>> >>> From here, the message is not complete until the server responds... >>> >>> HTTP/1.1 201 Created [or 504 or whatever] >>> >>> For XMLHttpRequest in synchronous mode, the scripting environment's >>> thread >>> blocks until the message is concluded. For XMLHttpRequest in asynchronous >>> mode, the scripting environment's thread churns on but the message is >>> still >>> not concluded until the reply arrives. The message is thus synchronous >>> either way because you don't send and forget, you have to write code to >>> deal >>> with the reply. >>> >>> Correct me if I'm wrong, but if you send and have to deal with a response >>> from the recipient, regardless of whether you can do something else as >>> you >>> await the response, the message itself is not asynchronous. An HTTP >>> message >>> consists of 1) send and 2) get reply, so is synchronous by definition. >>> XMLHttpRequest can't erase this, but it can allow you to workaround it. >>> >>> The workaround is an ugly hack that involves packing many "sub" messages >>> into the request and waiting forever for a reply while taking advantage >>> of >>> non-thread-blocking in the scripting environment (i.e. >>> comet/long-poll)... >>> >>> So, to send truly asynchronous *messages* from server to client, the >>> client >>> would... >>> >>> GET /foo/comet_interface HTTP/1.1 >>> >>> Server responds, streaming back a reply while not concluding the reply >>> for >>> as long as it can hold the connection open. It first sends headers, then >>> keeps appending the asynchronous messages... >>> >>> HTTP/1.1 200 OK >>> ..Headers.. >>> InnerMessage1 >>> InnerMessage2 >>> .. >>> InnerMessageN >>> >>> Client then uses it's non-thread-blocking XMLHttpRequest to read the >>> inner >>> messages as they arrive and tries not to close the connection. The client >>> has to know how to chop them up and the server might need to keep sending >>> a >>> regular special heartbeat sequence when idle to keep the connection open. >>> So >>> the "outer" HTTP GET message itself is synchronous (send and wait for >>> reply), but the "inner" messages packed inside are asynchronous (send and >>> forget). This is ugly, but web-developers have made it work out of >>> desperation. >>> >>> To send truly asynchronous messages from client to server is even tricker >>> and uglier. It might looks something like... >>> >>> POST /foo/multipart_interface HTTP/1.1 >>> >>> Content-type: multipart/form-data, boundary=AaB03x >>> >>> --AaB03x >>> content-disposition: form-data; name="message1" >>> >>> InnerMessage1 >>> --AaB03x >>> content-disposition: form-data; name="message2" >>> >>> InnerMessage2 >>> --AaB03x >>> >>> ... >>> >>> content-disposition: form-data; name="messageN" >>> >>> InnerMessageN >>> --AaB03x-- >>> >>> Again, the "outer" HTTP POST message is synchronous (send and wait for >>> reply), but the "inner" messages are asynchronous (send and forget). This >>> hackery is harder than it needs to be. >>> >>> >>> And you can get emulate a single full-duplex connection using two >>> >>>> half-duplex ones, >>>> >>>> >>> Synchronous full-duplex messages are still synchronous messages. Both >>> sides >>> still expect replies from the other, they can just do so at the same >>> time. >>> What you can do, as illustrated above, is simulate asynchronous messages >>> with synchronous HTTP. And it's complete pain in the arse. >>> >>> >>> so Websockets is just an optimization, >>> >>>> >>>> >>> I disagree. WebSockets allow you to send asynchronous full-duplex >>> messages >>> without the hackery of comet/long-polling. Just look at how clean Joe's >>> code >>> is. A websocket-less version would be possible, but uglier. >>> >>> >>> and likely not even a very much needed one. >>> >>>> >>>> >>> IMO, the very fact that web-developers have frequently used various >>> workarounds proves that it's needed. There is a disconnect between what >>> web >>> developers want and what the infrastructure currently provides. >>> Websockets >>> tries to address some of that. >>> >>> Personally, I think it's *very* much needed because it opens up the >>> possibilities of what you can do with the web-browser as a client. The >>> web >>> browser starts to look more like a ordinary TCP client. You can do more >>> of >>> the things that are easy with plain sockets which have always frustrated >>> me >>> with browsers. Just look at Joe's idea. Those sorts of ideas don't >>> immediately spring to mind when you're stuck in the mindset of >>> uni-directional synchronous messages. >>> >>> - Edmond - >>> >>> >>> The barrier right now seems to be >>> >>>> the BSD sockets API and kernel state management, and you still have >>>> that with Websockets. >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> >>>> >>> -- >>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >>> >> >> > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > -- J?r?me Martin From erlang@REDACTED Sun Feb 13 19:44:47 2011 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 13 Feb 2011 19:44:47 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> <87pqqwme42.fsf@mid.deneb.enyo.de> Message-ID: On Sun, Feb 13, 2011 at 7:36 PM, Jerome Martin wrote: > Yes, since Joe's WebSocket blog post I > think many people have had that idea in the back of their minds, floating > around, waiting for some signal that WebSocket was out to the general public > for good and could be considered as a usable in real life. Unfortunately, as > we all realize, it is not the case yet, and most frustrating, probably for > the wrong reasons :-( > > Well it might not be universally available, but it's here and now on my machine. All the general public have to do is change browsers, and how difficult is that? It's one click away. While the giants fight for market share by making their browsers mutually incompatible I just download the one that best suits my purposes. I just want to make things as simple as possible - life is to short to play browser wars. Hopefully a kind of software Darwinism will prevail - that's my naive belief. Life is too short to fix broken software. /Joe > > On Sun, Feb 13, 2011 at 6:25 PM, Edmond Begumisa < > ebegumisa@REDACTED> wrote: > >> +1 >> >> The thread went off topic. Joe e-mailed me with an idea like this many >> months ago off-list. I liked it then and I like it now. I even borrowed >> little bits of it in my current XULRunner projects (XUL has some limitations >> preventing the exact model). >> >> In fact, I've been thinking of some ways adding some templating to his >> code. So you can send "groups of widgets" down to the browser to display. It >> also got me thinking about some sort of flow language that can be used for >> people that don't know Erlang so they can easily string screens and widgets >> together in a flow chart manner. >> >> - Edmond - >> >> >> >> On Mon, 14 Feb 2011 04:08:05 +1100, Jerome Martin < >> tramjoe.merin@REDACTED> wrote: >> >> Just my 2cents.... >>> (warning, if you are prompt to be upset by dramatically different opinion >>> than your, please do not read the following) >>> >>> On a totally (not so ?) subjective mode: >>> - (my) Taste tells me Joe's code is beautiful. >>> - If such beautiful code is conceptually possible, it seems the right >>> thing to do (remember, we are in subjective mode) >>> - Preventing that beauty from happening (not supporting features >>> enabling >>> it), or claiming it is not necessary (presenting ugly code that does not >>> feel right and claiming it is the same) seems like the worst thing to do, >>> even if alternatives can be motivated (security reasons, etc.). >>> >>> Now, on a purely pragmatic mode: >>> - My experience (dealing with computer code since 20 years) tells me >>> that >>> my taste buds are not that subjective, and are more than often right: >>> usually, I find (or someone else does) later a technical demonstration >>> that >>> intuition and taste got it right. >>> - It is rare enough to trigger taste to a level that we can call >>> beautiful, >>> so definitely Joe must be on to something there :) >>> >>> To everyone with rotten taste buds (serious mode but I won't argue or >>> defend >>> my views): >>> - No, taste is not that subjective. I do think some people have bad >>> taste >>> and some other good taste, in an absolute sense. Matters of taste are a >>> tale >>> imagined not to upset anyone. >>> - Can't you see that there is something beyond technical arguments thee >>> that yells "this is the right way to do it" ? I mean, really ? >>> >>> On Sun, Feb 13, 2011 at 3:33 PM, Edmond Begumisa < >>> ebegumisa@REDACTED> wrote: >>> >>> On Sun, 13 Feb 2011 21:18:53 +1100, Florian Weimer >>>> wrote: >>>> >>>> * Joe Armstrong: >>>> >>>>> >>>>> How the heck can you easily send an an asynchronous message from a >>>>> client >>>>> >>>>>> to >>>>>> a server? >>>>>> >>>>>> HTTP get and post wait for a reply and thus are synchronous. >>>>>> >>>>>> By asynchronous I mean "fire and forget" like UDP. >>>>>> >>>>>> If it easy easy them I'd love to know how .. >>>>>> >>>>>> >>>>> XMLHttpRequest has an asynchronous option. >>>>> >>>>> >>>>> There seems to be some confusion. Forgive me, but aren't asynchronous >>>> HTTP >>>> requests very different from asynchronous messages? Isn't the >>>> "asynchronous" >>>> in XMLHttpRequest talking about the scripting environment in relation to >>>> thread-blocking and not the messages themselves? >>>> >>>> Asynchronous XMLHttpRequest just means that you can do something else >>>> while >>>> you wait for the reply, but a failed reply is still a fault so the >>>> message >>>> depends on the reply (or the reply can be viewed as part of the >>>> message). >>>> XMLHttpRequest is just a workaround to allow single-threaded scripting >>>> environments not to block so they can handle multiple requests without >>>> introducing multi-threading, but you *always* have to deal with the >>>> reply. >>>> My understanding is XMLHttpRequest in whatever mode does not magically >>>> turn >>>> HTTP requests into asynchronous *messages*. >>>> >>>> What is an "asynchronous message" anyway? >>>> >>>> AFAIK, it means send and forget. For Erlang's case this is... >>>> >>>> Recipient ! Message >>>> >>>> From here on, you are free to go about your business and not have to >>>> deal >>>> with any response. The recipient need not send you back any >>>> acknowledgement >>>> to make the message "final". The message is thus asynchronous because >>>> you >>>> send and forget. >>>> >>>> Compare this with HTTP... >>>> >>>> PUT /foo?message=Message HTTP/1.1 >>>> >>>> From here, the message is not complete until the server responds... >>>> >>>> HTTP/1.1 201 Created [or 504 or whatever] >>>> >>>> For XMLHttpRequest in synchronous mode, the scripting environment's >>>> thread >>>> blocks until the message is concluded. For XMLHttpRequest in >>>> asynchronous >>>> mode, the scripting environment's thread churns on but the message is >>>> still >>>> not concluded until the reply arrives. The message is thus synchronous >>>> either way because you don't send and forget, you have to write code to >>>> deal >>>> with the reply. >>>> >>>> Correct me if I'm wrong, but if you send and have to deal with a >>>> response >>>> from the recipient, regardless of whether you can do something else as >>>> you >>>> await the response, the message itself is not asynchronous. An HTTP >>>> message >>>> consists of 1) send and 2) get reply, so is synchronous by definition. >>>> XMLHttpRequest can't erase this, but it can allow you to workaround it. >>>> >>>> The workaround is an ugly hack that involves packing many "sub" messages >>>> into the request and waiting forever for a reply while taking advantage >>>> of >>>> non-thread-blocking in the scripting environment (i.e. >>>> comet/long-poll)... >>>> >>>> So, to send truly asynchronous *messages* from server to client, the >>>> client >>>> would... >>>> >>>> GET /foo/comet_interface HTTP/1.1 >>>> >>>> Server responds, streaming back a reply while not concluding the reply >>>> for >>>> as long as it can hold the connection open. It first sends headers, then >>>> keeps appending the asynchronous messages... >>>> >>>> HTTP/1.1 200 OK >>>> ..Headers.. >>>> InnerMessage1 >>>> InnerMessage2 >>>> .. >>>> InnerMessageN >>>> >>>> Client then uses it's non-thread-blocking XMLHttpRequest to read the >>>> inner >>>> messages as they arrive and tries not to close the connection. The >>>> client >>>> has to know how to chop them up and the server might need to keep >>>> sending a >>>> regular special heartbeat sequence when idle to keep the connection >>>> open. So >>>> the "outer" HTTP GET message itself is synchronous (send and wait for >>>> reply), but the "inner" messages packed inside are asynchronous (send >>>> and >>>> forget). This is ugly, but web-developers have made it work out of >>>> desperation. >>>> >>>> To send truly asynchronous messages from client to server is even >>>> tricker >>>> and uglier. It might looks something like... >>>> >>>> POST /foo/multipart_interface HTTP/1.1 >>>> >>>> Content-type: multipart/form-data, boundary=AaB03x >>>> >>>> --AaB03x >>>> content-disposition: form-data; name="message1" >>>> >>>> InnerMessage1 >>>> --AaB03x >>>> content-disposition: form-data; name="message2" >>>> >>>> InnerMessage2 >>>> --AaB03x >>>> >>>> ... >>>> >>>> content-disposition: form-data; name="messageN" >>>> >>>> InnerMessageN >>>> --AaB03x-- >>>> >>>> Again, the "outer" HTTP POST message is synchronous (send and wait for >>>> reply), but the "inner" messages are asynchronous (send and forget). >>>> This >>>> hackery is harder than it needs to be. >>>> >>>> >>>> And you can get emulate a single full-duplex connection using two >>>> >>>>> half-duplex ones, >>>>> >>>>> >>>> Synchronous full-duplex messages are still synchronous messages. Both >>>> sides >>>> still expect replies from the other, they can just do so at the same >>>> time. >>>> What you can do, as illustrated above, is simulate asynchronous messages >>>> with synchronous HTTP. And it's complete pain in the arse. >>>> >>>> >>>> so Websockets is just an optimization, >>>> >>>>> >>>>> >>>> I disagree. WebSockets allow you to send asynchronous full-duplex >>>> messages >>>> without the hackery of comet/long-polling. Just look at how clean Joe's >>>> code >>>> is. A websocket-less version would be possible, but uglier. >>>> >>>> >>>> and likely not even a very much needed one. >>>> >>>>> >>>>> >>>> IMO, the very fact that web-developers have frequently used various >>>> workarounds proves that it's needed. There is a disconnect between what >>>> web >>>> developers want and what the infrastructure currently provides. >>>> Websockets >>>> tries to address some of that. >>>> >>>> Personally, I think it's *very* much needed because it opens up the >>>> possibilities of what you can do with the web-browser as a client. The >>>> web >>>> browser starts to look more like a ordinary TCP client. You can do more >>>> of >>>> the things that are easy with plain sockets which have always frustrated >>>> me >>>> with browsers. Just look at Joe's idea. Those sorts of ideas don't >>>> immediately spring to mind when you're stuck in the mindset of >>>> uni-directional synchronous messages. >>>> >>>> - Edmond - >>>> >>>> >>>> The barrier right now seems to be >>>> >>>>> the BSD sockets API and kernel state management, and you still have >>>>> that with Websockets. >>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions (at) erlang.org mailing list. >>>>> See http://www.erlang.org/faq.html >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>> >>>>> >>>>> >>>> -- >>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> >>>> >>> >>> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> > > > > -- > J?r?me Martin > From fred.hebert@REDACTED Sun Feb 13 19:59:19 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Sun, 13 Feb 2011 13:59:19 -0500 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: Message-ID: <845A27C8-47B2-4955-9F95-94D6AC90377E@erlang-solutions.com> On 2011-02-12, at 06:33 AM, Joe Armstrong wrote: > > The Javascript equivalent is: > > function onMessage(evt) { > eval(evt.data); > } > > Where the data comes from a websocket. > This is rather risky. Eval will take any code whatsoever and run it for you. If you have dynamic content, without proper escaping and being very careful, users could run arbitrary code in your page, including stuff to steal session data and send it over to either some other site, or perform actions for the user which they do not necessarily approve on (making their profile public, closing their account, worms, etc.) In fact, this is a reason why people like Douglas Crockford prefered to write JSON parsers rather than just evaluating them. It's just not safe enough. Plus you have to call the javascript parser and whatnot, which is usually rather slow. The whole idea is pretty bad on the web, where you have to assume that people will actively try to break your stuff and steal data from other users (or you). > > This technique is amazingly powerful. > > So now I only need one generic web page. Think of that. > > Only one page is needed - forever. > This is a problem when it comes to bookmarks, sharing the link with a friend, searchability, browser history, etc. The web wasn't exactly intended to be a stateful thing and you'll have to resort to hacks such as hash-bangs to get around it. I suggest reading Tim Bray's Broken Links to see why that isn't a good solution anyway. Plus I'd argue that javascript and Erlang should be kept separate and you shouldn't try to generate one with the other, but at this point, I figure it's more of a matter of who wants to give himself the trouble than anything. -- Fred H?bert http://www.erlang-solutions.com From alain.odea@REDACTED Sun Feb 13 20:30:48 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sun, 13 Feb 2011 16:00:48 -0330 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: <845A27C8-47B2-4955-9F95-94D6AC90377E@erlang-solutions.com> References: <845A27C8-47B2-4955-9F95-94D6AC90377E@erlang-solutions.com> Message-ID: <5DDAB1D4-C058-4A74-BBD9-2EF9D1904E93@gmail.com> On 2011-02-13, at 15:29, Fr?d?ric Trottier-H?bert wrote: > > On 2011-02-12, at 06:33 AM, Joe Armstrong wrote: > >> >> The Javascript equivalent is: >> >> function onMessage(evt) { >> eval(evt.data); >> } >> >> Where the data comes from a websocket. >> > This is rather risky. Eval will take any code whatsoever and run it for you. If you have dynamic content, without proper escaping and being very careful, users could run arbitrary code in your page, including stuff to steal session data and send it over to either some other site, or perform actions for the user which they do not necessarily approve on (making their profile public, closing their account, worms, etc.) > > In fact, this is a reason why people like Douglas Crockford prefered to write JSON parsers rather than just evaluating them. It's just not safe enough. > > Plus you have to call the javascript parser and whatnot, which is usually rather slow. The whole idea is pretty bad on the web, where you have to assume that people will actively try to break your stuff and steal data from other users (or you). The source of of the eval'd script is the websocket which is domain locked. The scripts can only act in the context of the page. This is much safer than an iPhone app for example which has far more liberal and troubling data access. The trouble with eval is when you are using it for scripts whose source is unverified. The security risks are not serious in this case so lobg as you take care on the server side. > >> >> This technique is amazingly powerful. >> >> So now I only need one generic web page. Think of that. >> >> Only one page is needed - forever. >> > This is a problem when it comes to bookmarks, sharing the link with a friend, searchability, browser history, etc. The web wasn't exactly intended to be a stateful thing and you'll have to resort to hacks such as hash-bangs to get around it. I suggest reading Tim Bray's Broken Links to see why that isn't a good solution anyway. Bookmarking is definitely hard to achieve in this scenario, but hashes (like you say) could be used to restore previous states. Alternatively, you could take the view that the current state of the resource is its persistent state so long as the initial content the user gets upon visiting the page is the aggregate of all previous incremental websocket changes. In general this could be achieved by appending all the JS of those together and storing it or by having some form of caching user agent that tracks the outcomes. > > Plus I'd argue that javascript and Erlang should be kept separate and you shouldn't try to generate one with the other, but at this point, I figure it's more of a matter of who wants to give himself the trouble than anything. > > > -- > Fred H?bert > http://www.erlang-solutions.com > > From roberto@REDACTED Sun Feb 13 20:50:44 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 13 Feb 2011 20:50:44 +0100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <50612B0E-938C-4B0F-827C-04E0604396B3@masklinn.net> <2B1BF1F5-8E4E-4981-AF56-5E92642EA03C@erlang-solutions.com> <213049E1-5B57-412C-BCD6-01CC6B03C789@masklinn.net> <87pqqwme42.fsf@mid.deneb.enyo.de> Message-ID: 2011/2/13 Jerome Martin > Yes, since Joe's WebSocket blog > post< > http://armstrongonsoftware.blogspot.com/2009/12/comet-is-dead-long-live-websockets.html > > > I > think many people have had that idea in the back of their minds, floating > around, waiting for some signal that WebSocket was out to the general > public > for good and could be considered as a usable in real life. Unfortunately, > as > we all realize, it is not the case yet, and most frustrating, probably for > the wrong reasons :-( > i have implemented websockets in misultin, which is built to support multiple websockets versions at the same time. i have to say that i've currently stopped developing, waiting for a finalized version to come out. tired of losing time on this, and eager to settle to something finally usable out there. r. From kvs@REDACTED Sun Feb 13 21:01:40 2011 From: kvs@REDACTED (Konstantin Sorokin) Date: Sun, 13 Feb 2011 23:01:40 +0300 Subject: [erlang-questions] rebar and subversion In-Reply-To: References: Message-ID: On Sun, Feb 13, 2011 at 4:41 PM, Bob Ippolito wrote: > On Sun, Feb 13, 2011 at 9:30 PM, Dave Smith wrote: > > Hi Konstantin, > > > > On Sat, Feb 12, 2011 at 11:06 AM, Konstantin Sorokin > wrote: > >> Hello! > >> > >> I just noticed that that 'rebar clean' command removes ebin/.svn/tmp > >> directory so that subversion feels very unhappy about it > > > > Yes, rebar tries to clean out the ebin/ directory (and nested subdirs) > > rather thoroughly. The Erlang compilation can generate nested > > directories (if you're using packages), so rebar tries to accomodate > > that. > > Another option is to simply not have an ebin dir, and have the .app > file get generated by rebar with an .app.src file in your src > directory. > > Thanks for the hint, Bob, it worked like a charm! -- Konstantin Sorokin From ebegumisa@REDACTED Sun Feb 13 21:43:09 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Mon, 14 Feb 2011 07:43:09 +1100 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: <845A27C8-47B2-4955-9F95-94D6AC90377E@erlang-solutions.com> References: <845A27C8-47B2-4955-9F95-94D6AC90377E@erlang-solutions.com> Message-ID: On Mon, 14 Feb 2011 05:59:19 +1100, Fr?d?ric Trottier-H?bert wrote: > > On 2011-02-12, at 06:33 AM, Joe Armstrong wrote: > >> >> The Javascript equivalent is: >> >> function onMessage(evt) { >> eval(evt.data); >> } >> >> Where the data comes from a websocket. >> > This is rather risky. Eval will take any code whatsoever and run it for > you. Likewise the browser will take any static js (