From info@REDACTED Wed Jul 1 01:00:39 2009 From: info@REDACTED (info) Date: Wed, 1 Jul 2009 00:00:39 +0100 Subject: problem with string:tokens References: <200906301516188997141@its3.ch>, <87r5x1h23j.fsf@dixie.cronqvi.st> Message-ID: <200907010000388002871@its3.ch> No ! Once again it's a general question !! "info" writes: > Hi All, > > I have two strings: S1="A,,C,D" and S2="A,B,,D" > > I do string:tokens(S1) : "A","C","D" length =3 > I do string:tokens(S2) : "A","B","D" length = 3 > > Imagine the string S = S1 or S2 > How can I know if B is absent or C ? > > It is a general question of course ! the tokens function cannot > separate the tokens with a composite string unfortunately (here ",,") perhaps this is what you need? lists:member($B,S1) - > false lists:member($C,S1) - > true From info@REDACTED Wed Jul 1 01:02:35 2009 From: info@REDACTED (info) Date: Wed, 1 Jul 2009 00:02:35 +0100 Subject: [erlang-questions] Re: problem with string:tokens References: <200906301516188997141@its3.ch>, <3b1dbeaa-61c9-4fdb-ae52-10fcf2af9906@k26g2000vbp.googlegroups.com> Message-ID: <200907010002351431877@its3.ch> Oh Oh very good ! I didn't think to the re module. Thank you so much > It is a general question of course ! the tokens function cannot > separate the tokens with a composite string unfortunately (here ",,") As its name suggests, it *doesn't* behave like the more common "split" function in other languages. Have you tried the re module? For example: > re:split("A,,C,D", ",", [{return,list}]). ["A",[],"C","D"] > re:split("A,B,,D", ",", [{return,list}]). ["A","B",[],"D"] ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From ok@REDACTED Wed Jul 1 03:40:04 2009 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 1 Jul 2009 13:40:04 +1200 Subject: [erlang-questions] How to do counters? In-Reply-To: References: <960867.99882.qm@web111404.mail.gq1.yahoo.com> <20090625231037.GF181@h216-235-12-174.host.egate.net> <1245996165.4770.114.camel@piko.site> <7C165A49-29FB-4A73-B5C2-581AF6420F40@cs.otago.ac.nz> Message-ID: <2AB376D4-00D9-4BD0-82E6-1AF7EA3745CF@cs.otago.ac.nz> On Jun 30, 2009, at 1:35 PM, Jarrod Roberson wrote: > On Mon, Jun 29, 2009 at 7:46 PM, Richard O'Keefe > wrote: > >> As a general rule, don't register a process unless there is >> a REALLY good reason to do so. > > > Is the reason not to register processes performance related > or is it a "best practices" thing? It's the fact that the process registry is *global*. At startup, length(registered()) is currently 16. But that's only the processes that are _initially_ registered, not all that Erlang/OTP _might_ already register for itself. There's at least another 20. So it's not just that each possible registered name counts as a global variable, it's that you don't know which of them may be needed by the current release of the system and which of them might be needed in a future release. If you have a strong reason to register a process, and there _can_ be good reasons, try some convention like counter@REDACTED (name @ your initials) or something like that; OTP doesn't seem to use atoms with @ signs as registered names. > > And if it is performance related, It isn't. > how would someone know if they have a > REALLY good reason to register the process? You have a REALLY good reason to register a process when there is something you need to do that cannot be done any other way. In particular, when process A needs to talk to process B and process A didn't create process B and nobody else has told process A which process B is (the pid hasn't been sent in a message). In this case registering B under a name that's known to both parties will do the trick. Another situation is where the thing you want to talk to is a "role" rather than an actor, as in "I want to talk to whoever is in charge of alarm reporting right now". A role might be filled by a succession of actors because the actors die, or because work is redistributed for load balancing, or for debugging reasons, ... A project might well run its own registry. That's just a process that keeps a mapping from names (or other descriptions) to pids. But the processes in that project have to find _that_ registry. Using *one* registered name in the project, for the project's own registry, makes sense. From ok@REDACTED Wed Jul 1 04:05:44 2009 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 1 Jul 2009 14:05:44 +1200 Subject: [erlang-questions] problem with string:tokens In-Reply-To: <200906301516188997141@its3.ch> References: <200906301516188997141@its3.ch> Message-ID: <81D643C0-EAFC-4D0A-8AA8-CE6EE5BBBC3B@cs.otago.ac.nz> On Jul 1, 2009, at 1:16 AM, info wrote: > Hi All, > > I have two strings: S1="A,,C,D" and S2="A,B,,D" > > I do string:tokens(S1) : "A","C","D" length =3 > I do string:tokens(S2) : "A","B","D" length = 3 > > Imagine the string S = S1 or S2 > How can I know if B is absent or C ? > > It is a general question of course ! the tokens function cannot > separate the tokens with a composite string unfortunately (here ",,") There is probably something else in the library you could use, but I rolled my own. So could you. 2> s:separate("A,,C,D", $,). ["A",[],"C","D"] 3> s:separate("A,B,,D", ","). ["A","B",[],"D"] separate(Characters, Separator) - Characters must be a flat list of character codes - Separator may be a single character $x or a non-empty flat list of character codes "x" "xyz" ... - breaks Characters into X1++Separator++X2++Separator++...++Xn and returns [X1,...,Xn]. separate(Chars, [S|Sep]) when is_list(Chars), is_integer(S), S >= 0, is_list(Sep) -> separate(Chars, S, Sep); separate(Chars, S) when is_list(Chars), is_integer(S), S >= 0 -> separate(Chars, S, []). separate(Chars, S, Sep) -> case part(Chars, S, Sep, []) of no -> [Chars] ; {Word,Tail} -> [Word|separate(Tail, S, Sep)] end. part([], _, _, _) -> no; part([C|Chars], C, Sep, Rev) -> case prefix_check(Chars, Sep) of no -> part(Chars, C, Sep, [C|Rev]) ; Tail -> {lists:reverse(Rev), Tail} end; part([X|Chars], C, Sep, Rev) -> part(Chars, C, Sep, [X|Rev]). prefix_check(Tail, []) -> Tail; prefix_check([X|Tail], [X|Sep]) -> prefix_check(Tail, Sep); prefix_check(_, _) -> no. From zerthurd@REDACTED Wed Jul 1 10:26:43 2009 From: zerthurd@REDACTED (Maxim Treskin) Date: Wed, 1 Jul 2009 15:26:43 +0700 Subject: [erlang-questions] Different arity of macroses Message-ID: Hello EPP does not handle macroses with different arity as different macroses, but sometimes it can be a good property. I.e.: -define(DBG(F, A), io:format("DBG(~w:~b): " ++ F ++ "~n", [?MODULE, ?LINE] ++ A)). -define(DBG(F), io:format("DBG(~w:~b): " ++ F ++ "~n", [?MODULE, ?LINE])). and write ?DBG("Something here") instead of ?DBG("Something here", []) May be it is appropriated to add this behaviour in EPP? Thank you. -- Maxim Treskin From masse@REDACTED Wed Jul 1 11:39:45 2009 From: masse@REDACTED (mats cronqvist) Date: Wed, 01 Jul 2009 11:39:45 +0200 Subject: What is the best software development platform to use when working with Erlang? In-Reply-To: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> (G. S.'s message of "Wed\, 17 Jun 2009 15\:34\:58 -0700") References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> Message-ID: <87r5x0wxlq.fsf@sterlett.hq.kred> "G.S." writes: > Hello everyone, > > Is Eclipse the best SDE to use? For a long time, emacs was the only editor that had any kind of official support from OTP. There is also distel, which provides a useful emacs-erlang bridge, e.g. allowing you to run the debugger inside emacs. I gather that OTP is now focussing on providing industrial strength Eclipse support, also including debugger access. For all other environments you're pretty much on your own. This is obviously not an answer to your question, but hopefully of some interest anyway. mats From masse@REDACTED Wed Jul 1 12:05:27 2009 From: masse@REDACTED (mats cronqvist) Date: Wed, 01 Jul 2009 12:05:27 +0200 Subject: Inconsistent catch behaviour In-Reply-To: <4A39FC62.7040103@cs.ntua.gr> (Kostis Sagonas's message of "Thu\, 18 Jun 2009 11\:35\:46 +0300") References: <13274494.193991245313431276.JavaMail.root@zimbra> <4A39FC62.7040103@cs.ntua.gr> Message-ID: <87my7owwew.fsf@sterlett.hq.kred> Kostis Sagonas writes: > Adam Lindberg wrote: >> Hi! >> >> How come I can write: >> >> 1> catch 1 + a. >> 2> case catch 1 + a of A -> A end. >> 3> A = (catch 1 + a). >> >> and even >> >> 4> catch 1 + a. >> 5> A = v(-1). >> >> but not >> >> 4> A = catch 1 + a. > > Ulf Wiger has already answered this question, but I want to ask a > different one. Why should one bother these days with catch (even > mention it on this mailing list) when try-catch is available? If you don't care about the result. E.g. catch ets:delete(foo), to assert that the table foo no longer exists. mats From dmitriid@REDACTED Wed Jul 1 12:56:08 2009 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Wed, 1 Jul 2009 13:56:08 +0300 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> Message-ID: <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> NetBeans 6.7 RC3 with Erlang Plugin. See http://blogtrader.net/dcaoyuan/entry/erlang_plugin_version_1_for On Jun 18, 2009, at 1:34 AM, G.S. wrote: > Hello everyone, > > Is Eclipse the best SDE to use? I usually don't bother using > anything other > than a simple text editor with a highlighter since I can keep my > code in my > head, and even when debugging I can just work my way through the > logic to > find the errors I'm seeing even with 10k+ lines of code systems. But > I'll > have to begin working with someone else's code in a while, and it > will most > likely be over 25k lines of code, so I'll need something more > robust. Anyone > suggest anything other than Eclipse with the Erlang plug-in? > > Regards, > -Gene From rapsey@REDACTED Wed Jul 1 14:03:14 2009 From: rapsey@REDACTED (Rapsey) Date: Wed, 1 Jul 2009 14:03:14 +0200 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> Message-ID: <97619b170907010503y29d8918an7c5ec117f0a55e4@mail.gmail.com> If you are on a mac TextMate is pretty good. That's what I use at least. Sergej On Thu, Jun 18, 2009 at 12:34 AM, G.S. wrote: > Hello everyone, > > Is Eclipse the best SDE to use? I usually don't bother using anything other > than a simple text editor with a highlighter since I can keep my code in my > head, and even when debugging I can just work my way through the logic to > find the errors I'm seeing even with 10k+ lines of code systems. But I'll > have to begin working with someone else's code in a while, and it will most > likely be over 25k lines of code, so I'll need something more robust. > Anyone > suggest anything other than Eclipse with the Erlang plug-in? > > Regards, > -Gene > From amit.murthy@REDACTED Wed Jul 1 14:40:18 2009 From: amit.murthy@REDACTED (Amit Murthy) Date: Wed, 1 Jul 2009 18:10:18 +0530 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> Message-ID: <4f5fdb630907010540i2f8dcd9dv4e6d0682833ae24@mail.gmail.com> SCiTE in erlang mode works awesomely for me. On Thu, Jun 18, 2009 at 4:04 AM, G.S. wrote: > Hello everyone, > > Is Eclipse the best SDE to use? I usually don't bother using anything other > than a simple text editor with a highlighter since I can keep my code in my > head, and even when debugging I can just work my way through the logic to > find the errors I'm seeing even with 10k+ lines of code systems. But I'll > have to begin working with someone else's code in a while, and it will most > likely be over 25k lines of code, so I'll need something more robust. > Anyone > suggest anything other than Eclipse with the Erlang plug-in? > > Regards, > -Gene > From illo@REDACTED Wed Jul 1 15:36:46 2009 From: illo@REDACTED (Illo de Illis) Date: Wed, 1 Jul 2009 15:36:46 +0200 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: <97619b170907010503y29d8918an7c5ec117f0a55e4@mail.gmail.com> References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <97619b170907010503y29d8918an7c5ec117f0a55e4@mail.gmail.com> Message-ID: <541FCFF1-818B-4029-BCD7-C12075C51639@totalwire.it> On Jul 1, 2009, at 2:03 PM, Rapsey wrote: > If you are on a mac TextMate is pretty good. That's what I use at > least. Yep, me too. I had to write a bunch of templates and commands though, since none of them is included in the Erlang bundle. They're a little rough on the edges but they work. Ciao, Illo. From daveb@REDACTED Wed Jul 1 15:45:26 2009 From: daveb@REDACTED (Dave Bryson) Date: Wed, 1 Jul 2009 08:45:26 -0500 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> Message-ID: Emacs and Make. Two tools that are always there for you. I got away from Emacs for few years. Thanks to Erlang for bringing me back to my roots... Dave On Jul 1, 2009, at 5:56 AM, Dmitrii Dimandt wrote: > > NetBeans 6.7 RC3 with Erlang Plugin. See http://blogtrader.net/dcaoyuan/entry/erlang_plugin_version_1_for > > > On Jun 18, 2009, at 1:34 AM, G.S. wrote: > >> Hello everyone, >> >> Is Eclipse the best SDE to use? I usually don't bother using >> anything other >> than a simple text editor with a highlighter since I can keep my >> code in my >> head, and even when debugging I can just work my way through the >> logic to >> find the errors I'm seeing even with 10k+ lines of code systems. >> But I'll >> have to begin working with someone else's code in a while, and it >> will most >> likely be over 25k lines of code, so I'll need something more >> robust. Anyone >> suggest anything other than Eclipse with the Erlang plug-in? >> >> Regards, >> -Gene > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From sverker@REDACTED Wed Jul 1 16:02:39 2009 From: sverker@REDACTED (Sverker Eriksson) Date: Wed, 01 Jul 2009 16:02:39 +0200 Subject: [erlang-questions] Is ets:insert/2 (with multiple objects) isolated with respect to concurrent readers? In-Reply-To: <4A4A7A64.7050509@erlang-consulting.com> References: <781dd98c0906301123h4235262fq671f520c3060c029@mail.gmail.com> <4A4A7A64.7050509@erlang-consulting.com> Message-ID: <4A4B6C7F.70805@erix.ericsson.se> Ulf Wiger wrote: > Chris Newcombe wrote: >> Please could the isolation behavior be officially documented for >> ets:insert, ets:insert_new, and ets:delete_all_objects? > > Granted, you're asking for official documentation, and I > agree that this is important. > As Ulf mentioned; insert, insert_new and delete_all_objects are all both atomic and isolated. I can't see that we ever would want (and dare) to break that. I will make that guarantee more explicit in the documentation for next release. /Sverker, Erlang/OTP From chris.newcombe@REDACTED Wed Jul 1 16:12:36 2009 From: chris.newcombe@REDACTED (Chris Newcombe) Date: Wed, 1 Jul 2009 07:12:36 -0700 Subject: [erlang-questions] Is ets:insert/2 (with multiple objects) isolated with respect to concurrent readers? In-Reply-To: <4A4B6C7F.70805@erix.ericsson.se> References: <781dd98c0906301123h4235262fq671f520c3060c029@mail.gmail.com> <4A4A7A64.7050509@erlang-consulting.com> <4A4B6C7F.70805@erix.ericsson.se> Message-ID: <781dd98c0907010712s38e1323fmacb3b869da69627d@mail.gmail.com> > I will make that guarantee more explicit in the documentation for next > release. Excellent -- thankyou very much! Chris On Wed, Jul 1, 2009 at 7:02 AM, Sverker Eriksson wrote: > Ulf Wiger wrote: >> >> Chris Newcombe wrote: >>> >>> Please could the isolation behavior be officially documented for >>> ets:insert, ets:insert_new, and ets:delete_all_objects? >> >> Granted, you're asking for official documentation, and I >> agree that this is important. >> > As Ulf mentioned; insert, insert_new and delete_all_objects are all both > atomic and isolated. > > I can't see that we ever would want (and dare) to break that. > I will make that guarantee more explicit in the documentation for next > release. > > /Sverker, Erlang/OTP > > > > From tsuraan@REDACTED Wed Jul 1 17:57:53 2009 From: tsuraan@REDACTED (tsuraan) Date: Wed, 1 Jul 2009 10:57:53 -0500 Subject: [erlang-questions] erlang ffi this summer? In-Reply-To: References: Message-ID: <84fb38e30907010857t7404d20fid460df314cabb77e@mail.gmail.com> > I apologize if this has been visited recently but what's the status on > the Erlang FFI EEP? Last thing I heard it was going to be out this > spring. Any word on this? FFI seems to be a great idea that's even been implemented as a set of independent patches, but there isn't much discussion on it. Is FFI in the pipeline? From joelr1@REDACTED Wed Jul 1 18:12:43 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 1 Jul 2009 17:12:43 +0100 Subject: pushing messages to large number of subscribers with minimal latency Message-ID: <7CE08E26-359B-45FE-9A4A-3F5482BF985F@gmail.com> I'm trying to build a solution where I can push a message to hundreds of thousands of topic subscribers with minimal latency. Topic subscribers may come and go at a rapid clip but only point of the solution is to push messages out to subscribers as quickly as possible. I'm writing in Erlang and wondering if RabbitMQ is really buying me much given that it relies so much on Erlang. Pushing messages to subscribers also seems to be a basic telco problem, surely solved a few times over. I'm thinking of a design where each node in the cluster keeps track of subscribers connected to that node (over TCP) and messages are pushed to each node which then proceeds to push to its share of subscribers. I could then keep track of topics on each node by wrapping a dict or ets table in a process and keying on the pid for ease of removal. There are no locking or replication needs if topic tracking processes are running on each node and the number of topics in the system would only be limited by memory if a dict is used instead of ets. Any comments on my design? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From roberto@REDACTED Wed Jul 1 18:17:08 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 1 Jul 2009 18:17:08 +0200 Subject: [erlang-questions] distributed system: same app, different params In-Reply-To: <9a09ca9a0906221604o56fd4c80sfe55e1db8320863@mail.gmail.com> References: <4A39E84B.1010408@erlang-consulting.com> <087E0400-7545-49FC-8D0A-CD888DF15796@widetag.com> <4A3FA6F9.1060306@erlang-consulting.com> <9a09ca9a0906221604o56fd4c80sfe55e1db8320863@mail.gmail.com> Message-ID: ulf and kunthar, i'm currently using scripts via python, but i wished there was a more natural way for erlang to keep track of running procs. i thought otp would take care of this. thank you :) r. 2009/6/23 Kunthar > I don't know if those fit to your needs but here are two options; > > Check pssh, fancy pink background :) > http://www.theether.org/pssh/ > > Check capistrano, Ruby guys holiday inn. > http://www.capify.org/ > > > Peace > \|/ Kunth > > > On Mon, Jun 22, 2009 at 6:44 PM, Ulf > Wiger wrote: > > Roberto Ostinelli wrote: > >> > >> hi, can i ask for a little clarification more: what would you use to > >> deploy this script on many different machines? mechanisms such as bash > >> script managers over the network, application_controller, ...? > > > > I guess this is a bit application-specific, but one may e.g. distribute > > a global config document, and let each machine find its own particulars > > by matching on the host name. It can then complement the global config > > with data found through local discovery. > > > > A bash or python script might be the best tool for this kind of > > thing, but escript could of course also work. > > > > Another fairly nice approach, if you're running within an Erlang > > environment already, is file:script(File). It's like > > file:consult/1, but returns the value of the last expression. > > This function actually came from the 'builder' project, where > > I generated app files, config, boot and script files through > > a combination of preset values and local discovery. > > > > BR, > > Ulf W > > -- > > Ulf Wiger > > CTO, Erlang Training & Consulting Ltd > > http://www.erlang-consulting.com > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- ---------------------------------------------------------- Roberto Ostinelli CTO, WideTag Inc. - Towards an Internet Of Things widetag.com openspime.com skype rostinelli mobile +39 335 6 100 22 6 From joe@REDACTED Wed Jul 1 19:04:32 2009 From: joe@REDACTED (Joe Williams) Date: Wed, 1 Jul 2009 10:04:32 -0700 Subject: [erlang-questions] erlang ffi this summer? In-Reply-To: <84fb38e30907010857t7404d20fid460df314cabb77e@mail.gmail.com> References: <84fb38e30907010857t7404d20fid460df314cabb77e@mail.gmail.com> Message-ID: <20090701100432.784a6866@der-dieb> Agreed FFI would be a great addition, any info on it's status would be appreciated. -Joe On Wed, 1 Jul 2009 10:57:53 -0500 tsuraan wrote: > > I apologize if this has been visited recently but what's the status > > on the Erlang FFI EEP? Last thing I heard it was going to be out > > this spring. > > Any word on this? FFI seems to be a great idea that's even been > implemented as a set of independent patches, but there isn't much > discussion on it. Is FFI in the pipeline? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Name: Joseph A. Williams Email: joe@REDACTED Blog: http://www.joeandmotorboat.com/ From kaiduanx@REDACTED Wed Jul 1 20:28:10 2009 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 1 Jul 2009 14:28:10 -0400 Subject: dbg:p(all, [c]) hangs the shell Message-ID: Hi, all, I would like to trace all calls in all processes, however the erlang shell hangs after dbg:p(all, [c]). It even does not accept Ctrl-G. kaiduanx@REDACTED:~$ erl Erlang (BEAM) emulator version 5.6.5 [source] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.6.5 (abort with ^G) 1> dbg:tracer(). {ok,<0.33.0>} 2> dbg:tpl('_', '_', '_', dbg:fun2ms(fun(_) -> return_trace() end)). {ok,[{matched,nonode@REDACTED,5905},{saved,1}]} 3> dbg:get_tracer(). {ok,<0.34.0>} 4> process_info(list_to_pid("<0.34.0>")). [{current_function,{dbg,tracer_loop,2}}, {initial_call,{erlang,apply,2}}, {status,waiting}, {message_queue_len,0}, {messages,[]}, {links,[<0.33.0>]}, {dictionary,[]}, {trap_exit,true}, {error_handler,error_handler}, {priority,max}, {group_leader,<0.24.0>}, {total_heap_size,233}, {heap_size,233}, {stack_size,3}, {reductions,5}, {garbage_collection,[{fullsweep_after,65535},{minor_gcs,0}]}, {suspending,[]}] 5> dbg:p(list_to_pid("<0.24.0>"), clear). {ok,[{matched,nonode@REDACTED,1}]} 6> dbg:p(all, [c]). (<0.33.0>) call lists:map(#Fun,[]) (<0.33.0>) returned from lists:map/2 -> [] (<0.33.0>) call dbg:reply(<0.31.0>,{ok,[{matched,nonode@REDACTED,25}]}) (<0.33.0>) returned from dbg:reply/2 -> {dbg, {ok,[{matched,nonode@REDACTED,25}]}} {ok,[{matched,nonode@REDACTED,25}]} Any idea on what was wrong? Thanks, kaiduan From chris.newcombe@REDACTED Wed Jul 1 21:27:09 2009 From: chris.newcombe@REDACTED (Chris Newcombe) Date: Wed, 1 Jul 2009 12:27:09 -0700 Subject: [erlang-questions] Is ets:insert/2 (with multiple objects) isolated with respect to concurrent readers? In-Reply-To: <781dd98c0907010712s38e1323fmacb3b869da69627d@mail.gmail.com> References: <781dd98c0906301123h4235262fq671f520c3060c029@mail.gmail.com> <4A4A7A64.7050509@erlang-consulting.com> <4A4B6C7F.70805@erix.ericsson.se> <781dd98c0907010712s38e1323fmacb3b869da69627d@mail.gmail.com> Message-ID: <781dd98c0907011227q5ae023aftcce56160a1ffe914@mail.gmail.com> >> As Ulf mentioned; insert, insert_new and delete_all_objects are all both >> atomic and isolated. >> >> I can't see that we ever would want (and dare) to break that. I just noticed that the new {write_concurrency, true} option says that write-locks might no-longer be taken at the table level. "Different parts of the same table can be mutated (and read) by concurrent processes." (full text below) It does not say which write/read APIs are allowed to be concurrent. So there's the usual natural tension between clean API semantics for compound write operations, and increased concurrency. e.g. Some applications might want atomicity, but might care more about increased concurrency than full isolation. Other applications (like my current one) might really need strong isolation. But I guess that backwards-compatibility reasons will dominate your decision (quite understandably). Given the historic implicit behavior (strong isolation) for delete_all_objects, insert, and insert_new, it would be dangerous to change them now. Also, strong isolation follows the principle of least surprise. It would be great if the updated documentation for the APIs specifically described the isolation semantics when {write_concurrency,true} is used. And vice-versa too; e.g. it would good if the documentation for write_concurrency mentioned that compound-write operations will either acquire multiple fine-grain write-locks (i.e. acquire all necessary locks before modifying anything), or may choose to acquire a table-level lock, to ensure (in either case) that their historic isolation behavior is preserved. Therefore applications that make heavy use of compound-write operations might see less benefit from {write_concurrency, true}. thanks, Chris "{write_concurrency,bool()} Performance tuning. Default is false, which means that the table is optimized towards concurrent read access. An operation that mutates (writes to) the table will obtain exclusive access, blocking any concurrent access of the same table until finished. If set to true, the table is optimized towards concurrent write access. Different parts of the same table can be mutated (and read) by concurrent processes. This is achieve to some degree at the expense of single access and concurrent reader performance. Table typ ordered_set is not affected by this option in current implementation." On Wed, Jul 1, 2009 at 7:12 AM, Chris Newcombe wrote: >> I will make that guarantee more explicit in the documentation for next >> release. > > Excellent -- thankyou very much! > > Chris > > > On Wed, Jul 1, 2009 at 7:02 AM, Sverker > Eriksson wrote: >> Ulf Wiger wrote: >>> >>> Chris Newcombe wrote: >>>> >>>> Please could the isolation behavior be officially documented for >>>> ets:insert, ets:insert_new, and ets:delete_all_objects? >>> >>> Granted, you're asking for official documentation, and I >>> agree that this is important. >>> >> As Ulf mentioned; insert, insert_new and delete_all_objects are all both >> atomic and isolated. >> >> I can't see that we ever would want (and dare) to break that. >> I will make that guarantee more explicit in the documentation for next >> release. >> >> /Sverker, Erlang/OTP >> >> >> >> > From chandrashekhar.mullaparthi@REDACTED Thu Jul 2 01:06:07 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Thu, 2 Jul 2009 00:06:07 +0100 Subject: [ANN] ibrowse-1.5.0 Message-ID: Hi, In response to comments I received from Oscar Hellstr?m, Adam Kocoloski and Chris Newcombe, I've updated ibrowse to use binaries internally. A few other changes came up while these were being done. A list of changes is included below (extract from README). From now on, github will be the primary repository for ibrowse. I'll still push changes to jungerl in Sourceforge. Latest version : git://github.com/cmullaparthi/ibrowse.git API : http://wiki.github.com/cmullaparthi/ibrowse/ibrowse-api CONTRIBUTIONS & CHANGE HISTORY ============================== 29-06-2009 - * Fixed following issues reported by Oscar Hellstr?m ?????????????? - Use {active, once} instead of {active, true} ?????????????? - Fix 'dodgy' timeout handling ?????????????? - Use binaries internally instead of lists to reduce memory ???????????????? consumption on 64 bit platforms. The default response format ???????????????? is still 'list' to maintain backwards compatibility. Use the ???????????????? option {response_format, binary} to get responses as binaries. ???????????? * Fixed chunking bug (reported by Adam Kocoloski) ???????????? * Added new option {inactivity_timeout, Milliseconds} to timeout ?????????????? requests if no data is received on the link for the specified ?????????????? interval. Useful when responses are large and links are flaky. * Added option {'connect_timeout', Milliseconds} ???????????? * Added ibrowse:all_trace_off/0 to turn off all tracing ???????????? * Change to the way responses to asynchronous requests are ?????????????? returned. The following messages have been removed. ??????????????? * {ibrowse_async_response, Req_id, {chunk_start, Chunk_size}} ???????????????? * {ibrowse_async_response, Req_id, chunk_end} ???????????? * Fixed Makefiles as part of Debian packaging ?????????????? (thanks to Thomas Lindgren) ???????????? * Moved repository from Sourceforge to Github Thanks to Adam Kocoloski and Chris Newcombe for help with testing. cheers Chandru From dizzyd@REDACTED Thu Jul 2 02:34:31 2009 From: dizzyd@REDACTED (Dave Smith) Date: Wed, 1 Jul 2009 18:34:31 -0600 Subject: [erlang-questions] [ANN] ibrowse-1.5.0 In-Reply-To: References: Message-ID: On Wed, Jul 1, 2009 at 5:06 PM, Chandru wrote: > github will be the primary repository for ibrowse. I'll still push > changes to jungerl in Sourceforge. > > Latest version : git://github.com/cmullaparthi/ibrowse.git That's great news -- I'm deleting my old repo of ibrowse on github in favor this. Thanks Chandru! :) D. From jarrod@REDACTED Thu Jul 2 03:29:22 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Wed, 1 Jul 2009 21:29:22 -0400 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> Message-ID: On Wed, Jul 1, 2009 at 6:56 AM, Dmitrii Dimandt wrote: > > NetBeans 6.7 RC3 with Erlang Plugin. See > http://blogtrader.net/dcaoyuan/entry/erlang_plugin_version_1_for > I have to say, this is the one I am playing with right now. It really helps out with when to use .,; or nothing at the end of a line. From caox@REDACTED Thu Jul 2 03:32:24 2009 From: caox@REDACTED (caox) Date: Thu, 02 Jul 2009 09:32:24 +0800 Subject: why can not call erl_init() in drivers Message-ID: <4A4C0E28.5050408@lightpole.net> Hi I found the erl_interface lib can not be used when I was working on some c-drivers for erlang, since there is no response when calling erl_init(). Instead, we can only use the ei lib. So, is there any articles or references to introduce the reason? From bryantln@REDACTED Thu Jul 2 04:09:39 2009 From: bryantln@REDACTED (Lincoln Bryant) Date: Wed, 1 Jul 2009 22:09:39 -0400 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> Message-ID: Indeed. The Erlang mode for Emacs is excellent. To be honest, I'm surprised how many Erlang users are using Emacs. Most of the folks I know are huge Vi fans! On Wed, Jul 1, 2009 at 9:45 AM, Dave Bryson wrote: > Emacs and Make. Two tools that are always there for you. > > I got away from Emacs for few years. Thanks to Erlang for bringing me back > to my roots... > > Dave > > On Jul 1, 2009, at 5:56 AM, Dmitrii Dimandt wrote: > > >> NetBeans 6.7 RC3 with Erlang Plugin. See >> http://blogtrader.net/dcaoyuan/entry/erlang_plugin_version_1_for >> >> >> On Jun 18, 2009, at 1:34 AM, G.S. wrote: >> >> Hello everyone, >>> >>> Is Eclipse the best SDE to use? I usually don't bother using anything >>> other >>> than a simple text editor with a highlighter since I can keep my code in >>> my >>> head, and even when debugging I can just work my way through the logic to >>> find the errors I'm seeing even with 10k+ lines of code systems. But I'll >>> have to begin working with someone else's code in a while, and it will >>> most >>> likely be over 25k lines of code, so I'll need something more robust. >>> Anyone >>> suggest anything other than Eclipse with the Erlang plug-in? >>> >>> Regards, >>> -Gene >>> >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- Lincoln Bryant From wenewboy@REDACTED Thu Jul 2 05:51:57 2009 From: wenewboy@REDACTED (wenew zhang) Date: Thu, 2 Jul 2009 11:51:57 +0800 Subject: erlang-mysql doesn't release the connection when worker crash and restart! Message-ID: <4eaa09eb0907012051i3204676m943e06be5af5ab99@mail.gmail.com> use erlang-mysql in my project, it?s a supervisor->work structure and start mysql:start_link() in the work, when the work crashed, the mysql-connections to still keep alive, i add process_flag(trap_exit, true) in mysql:init and mysql_conn:init but it doesn?t working! what shall i do? wenew zhang From jebenitez@REDACTED Thu Jul 2 05:55:17 2009 From: jebenitez@REDACTED (Jose Enrique Benitez Jimenez) Date: Wed, 1 Jul 2009 23:55:17 -0400 (CDT) Subject: Web services in erlang Message-ID: <2099288794.3851246506917180.JavaMail.root@ucimail3.uci.cu> Hello comunity Where I can download erlyweb-0.7.1.tar.gz from anywhere but erlyweb.googlecode.com (I am from Cuba so I dont have access to this URL) and some documentation about how make web services in erlang. Thanks Enrique. From wenewboy@REDACTED Thu Jul 2 06:01:47 2009 From: wenewboy@REDACTED (wenew zhang) Date: Thu, 2 Jul 2009 12:01:47 +0800 Subject: how to pass a error_loger to erlang-mysql? Message-ID: <4eaa09eb0907012101k25a4806cy772dd6b428c1908@mail.gmail.com> erlang-mysql driver have a LogFun Arg,i want pass error_loger:info_msg to it, i try: mysql:start_link(...,error_loger:info_msg), mysql:start_link(...,info_msg/2), mysql:start_link(...,info_msg), i can't work out! some code below,i'm confused with log/4 and the macros defined start_link(PoolId, Host, Port, User, Password, Database, LogFun) -> init([PoolId, Host, Port, User, Password, Database, LogFun, Encoding]) -> LogFun1 = if LogFun == undefined -> fun log/4; true -> LogFun end, -- -define(Log(LogFun,Level,Msg), LogFun(?MODULE, ?LINE,Level,fun()-> {Msg,[]} end)). -define(Log2(LogFun,Level,Msg,Params), LogFun(?MODULE, ?LINE,Level,fun()-> {Msg,Params} end)). -define(L(Msg), io:format("~p:~b ~p ~n", [?MODULE, ?LINE, Msg])). From attila.r.nohl@REDACTED Thu Jul 2 09:04:34 2009 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 2 Jul 2009 09:04:34 +0200 Subject: [erlang-questions] dbg:p(all, [c]) hangs the shell In-Reply-To: References: Message-ID: <401d3ba30907020004h1abc0003tdde7f02b823bad30@mail.gmail.com> 2009/7/1, Kaiduan Xie : > Hi, all, > > I would like to trace all calls in all processes, [...] > Any idea on what was wrong? I'd guess that at the tracing framework is (at least partially) written in erlang, so you're effectively tracing the tracing framework, which leads to deadlock. From kamiseq@REDACTED Thu Jul 2 09:40:56 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Thu, 2 Jul 2009 09:40:56 +0200 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> Message-ID: maybe it is just a matter of configuration of my emacs but I miss so simple features like -word highlighting if you selected the word in editor -auto complition of existing function's or var/atoms's names - now I just copy paste -the overall user experience is dramatic, sometimes is hard to select and then copy the same chunk of code many times in the row. -what the hell is make :) I want my editor to build my project and copy all to dist or other boring things and still I have no time to configure all this distels things :) anyway it works and I am coding in it for last 4 months :) I ve just installed earlybird plugin and what i miss is short list of templates (all gen_* skeletons) pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From gleber.p@REDACTED Thu Jul 2 10:49:29 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 2 Jul 2009 10:49:29 +0200 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> Message-ID: <14f0e3620907020149p3e216a92h9e6a6ce2020a0ad5@mail.gmail.com> Hello 2009/7/2 pawe? kami?ski : > maybe it is just a matter of configuration of my emacs but I miss so simple > features like > -word highlighting if you selected the word in editor Just add (transient-mark-mode t) to your ~/.emacs file. Adding (global-hl-line-mode t) (set-face-background 'hl-line "#444") will enable current line highlighting (very handy. You need a terminal which supports at least 256 colors so it looks good) > -auto complition of existing function's or var/atoms's names - now I just > copy paste I use M-/ to complete the words (it ignores any semantics - it just completes the words based on all open buffers). It is quite handy but not perfect. Distel supports proper erlang module and function name completion, but it is hard to configure it properly > -the overall user experience is dramatic, Emacs is not easy to learn, but when you have enough experience you will think that any other editor is handicapped and not pleasant to use. > sometimes is hard to select and > then copy the same chunk of code many times in the row. C-SPC, arrows, M-w - to copy text C-y to paste and afterward use M-y to cycle through the yank (copy/cut) buffer. It is really handy. > -what the hell is make :) I want my editor to build my project and copy all > to dist or other boring things > and still I have no time to configure all this distels things :) With Distel C-c C-k compiles a module in the current buffer. I've added support for including hrl's from deps/*/include and deps/*/src in my ~/.emacs - I can share it if you want me to. Best regards, Gleb Peregud P.S. IDO mode is really handy too: (require 'ido) (ido-mode t) It adds a better completition for file-open (and similar) mini-buffer -- Gleb Peregud http://gleber.pl/ Every minute is to be grasped. Time waits for nobody. -- Inscription on a Zen Gong From kamiseq@REDACTED Thu Jul 2 11:47:02 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Thu, 2 Jul 2009 11:47:02 +0200 Subject: document oriented mapReduce databases - schema and queries Message-ID: hi, all recently I ve started to look at couchDB and what I see sounds great but I have few general questions 1)when it is better to use MR db and not RDMS (I have structurised documents split into tables) 2)why schema free is good, how then ensure that what somebody puts to db is what it should be especially when JSON can handle any data type and it is easy to make a mistake 3)as a old SQL developer Im force now to think how to implement all map and reduce function to make them efficient. with select statement I just write what I want and leave all to the system. 4)how to design my db, curently my db is about 500 master to items tables, so I understand how to aggragate with MR but how to join and how to make it work fast and use as small disk as possible and that data are not redundant. 5)If my RDBS is running on 2core (ore sometimes one core) machine, and that machine is alone server storing data from local clients is there any gain of using systems such as couchDB or others (ok replication and other features are still nice) 6)this is somehow related to point 5) what scale down really means (to work on mobiles, browsers etc)? If I have one core machine then will the system be fast enough than RDMS (sqlite) thanks for helping me in that pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From sverker@REDACTED Thu Jul 2 12:21:29 2009 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 02 Jul 2009 12:21:29 +0200 Subject: [erlang-questions] Is ets:insert/2 (with multiple objects) isolated with respect to concurrent readers? In-Reply-To: <781dd98c0907011227q5ae023aftcce56160a1ffe914@mail.gmail.com> References: <781dd98c0906301123h4235262fq671f520c3060c029@mail.gmail.com> <4A4A7A64.7050509@erlang-consulting.com> <4A4B6C7F.70805@erix.ericsson.se> <781dd98c0907010712s38e1323fmacb3b869da69627d@mail.gmail.com> <781dd98c0907011227q5ae023aftcce56160a1ffe914@mail.gmail.com> Message-ID: <4A4C8A29.80103@erix.ericsson.se> Chris Newcombe wrote: > I just noticed that the new {write_concurrency, true} option says that > write-locks might no-longer be taken at the table level. > > "Different parts of the same table can be mutated (and read) by > concurrent processes." > > (full text below) > > It does not say which write/read APIs are allowed to be concurrent. > > The idea with write_concurrency was that it should be pure performance tuning and not change any guarentees about API semantics. > So there's the usual natural tension between clean API semantics for > compound write operations, and increased concurrency. e.g. Some > applications might want atomicity, but might care more about increased > concurrency than full isolation. Other applications (like my current > one) might really need strong isolation. > > But I guess that backwards-compatibility reasons will dominate your > decision (quite understandably). Given the historic implicit behavior > (strong isolation) for delete_all_objects, insert, and insert_new, it > would be dangerous to change them now. Also, strong isolation > follows the principle of least surprise. > > True, backward-compatibility was the main reason for deciding now about making the atomic and isolated semantics of insert, insert_new and delete_all_objects to be guaranteed in the docs. The introduction of write_concurrency was however about backward-compatibility with respect only to performance and not semantics. > It would be great if the updated documentation for the APIs > specifically described the isolation semantics when > {write_concurrency,true} is used. > > And vice-versa too; e.g. it would good if the documentation for > write_concurrency mentioned that compound-write operations will either > acquire multiple fine-grain write-locks (i.e. acquire all necessary > locks before modifying anything), or may choose to acquire a > table-level lock, to ensure (in either case) that their historic > isolation behavior is preserved. > Therefore applications that make heavy use of compound-write > operations might see less benefit from {write_concurrency, true}. > > Don't you think it's enough if stated about {write_concurrency,bool()} that is does not break any semantic promises about atomicy and isolation. Maybe note that operations that makes such promises will gain less (or nothing) from {write_concurrency,true}. I don't think we should need to describe the current internal locking strategy. Also, any guarantees about atomicy and isolation only applies to the data that the function is operating on. /Sverker, Erlang/OTP From olopierpa@REDACTED Thu Jul 2 13:11:59 2009 From: olopierpa@REDACTED (Pierpaolo Bernardi) Date: Thu, 2 Jul 2009 13:11:59 +0200 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> Message-ID: <7352e43a0907020411q42d30dc9g3a9dd75497f0c83@mail.gmail.com> 2009/7/2 pawe? kami?ski : > -the overall user experience is dramatic, sometimes is hard to select and > then copy the same chunk of code many times in the row. ... > anyway it works and I am coding in it for last 4 months :) You are using it for the last 4 months but you coudn't find the 15 minutes necessary to work through the tutorial that emacs itself suggests in the opening screen. You will save yourself a lot of time in exchange for these 15 minutes. Just type C-h t. Cheers P. From kamiseq@REDACTED Thu Jul 2 13:21:57 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Thu, 2 Jul 2009 13:21:57 +0200 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: <7352e43a0907020411q42d30dc9g3a9dd75497f0c83@mail.gmail.com> References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> <7352e43a0907020411q42d30dc9g3a9dd75497f0c83@mail.gmail.com> Message-ID: i did and then made a cheatlist of shorcuts (about 50) and then couldnt help using emacs like all other editors i used to ;) sorry but Im working in delphi/sql/java/flex and try to learn erlang in my free time so forgive me for focusing on better things than configuring editor pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From olopierpa@REDACTED Thu Jul 2 13:36:55 2009 From: olopierpa@REDACTED (Pierpaolo Bernardi) Date: Thu, 2 Jul 2009 13:36:55 +0200 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> <7352e43a0907020411q42d30dc9g3a9dd75497f0c83@mail.gmail.com> Message-ID: <7352e43a0907020436g6de05906j29dd790090acfbc2@mail.gmail.com> 2009/7/2 pawe? kami?ski : > i did and then made a cheatlist of shorcuts (about 50) and then couldnt help > using emacs like all other editors i used to ;) > sorry but Im working in delphi/sql/java/flex and try to learn erlang in my > free time so forgive me for focusing on better things than configuring > editor I think I know the reason you are so short of time. ;) Cheers P. From ajuttner.list@REDACTED Thu Jul 2 13:37:46 2009 From: ajuttner.list@REDACTED (Alpar Juttner) Date: Thu, 02 Jul 2009 11:37:46 +0000 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> Message-ID: <1246534666.6215.23.camel@piko.site> > -what the hell is make :) M-x compile You can issue any shell command from there (is it 'make -k' by default). Emacs executes it and the result will go to a buffer called *compilation*. If there was any error during compilation you can go to the related source code position with C-x ` (and repeat it for jumping to the next error.) Regards, Alpar From matthew@REDACTED Thu Jul 2 12:39:20 2009 From: matthew@REDACTED (Matthew Sackman) Date: Thu, 2 Jul 2009 11:39:20 +0100 Subject: Mnesia memory, size and effects of table copy types Message-ID: <20090702103919.GA1984@mrnibble.lshift.net> I'm doing some manual memory management and when memory gets tight, I'm converting some mnesia tables from disc_copies to disc_only_copies. But I have a few questions because what I'm seeing reported through table_info seems odd. ets and mnesia claim to report memory size in words. But dets reports in bytes. When a table is put in disc_only_copies, can someone confirm it's still words I'm getting back, and not bytes? Because the following looks very very fishy > mnesia:create_table(mytable, [{disc_copies, [node()]}]). {atomic,ok} > mnesia:table_info(mytable, size). 0 > mnesia:table_info(mytable, memory). 299 Ok, so presumeably, that's the number of words in RAM. Maybe. Docs don't actually say - it could very well be the sum of bytes on disk and words in ram, but let's assume it's not double counting. > mnesia:change_table_copy_type(mytable, node(), disc_only_copies). {atomic,ok} > mnesia:table_info(mytable, size). 0 > mnesia:table_info(mytable, memory). 5752 Um. Ok, so maybe that's just some RAM based overhead or something. But it's gone up quite a long way... far enough that I'm not convinced that's words and not bytes. > mnesia:change_table_copy_type(mytable, node(), ram_copies). {atomic,ok} > mnesia:table_info(mytable, size). 0 > mnesia:table_info(mytable, memory). 299 Oh well, at least we're back. Now, let's fill the table up: > [ mnesia:dirty_write(mytable, {mytable, N, <<0:8192>>}) || N <- lists:seq(1, 1000000) ]. [ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok, ok,ok,ok,ok,ok,ok,ok,ok,ok,ok|...] > mnesia:table_info(mytable, size). 1000000 > mnesia:table_info(mytable, memory). 17144943 > mnesia:change_table_copy_type(mytable, node(), disc_copies). {atomic,ok} > mnesia:table_info(mytable, size). 1000000 > mnesia:table_info(mytable, memory). 17144943 Still good... but then: > mnesia:change_table_copy_type(mytable, node(), disc_only_copies). {atomic,ok} > mnesia:table_info(mytable, size). 1000000 > mnesia:table_info(mytable, memory). 1929666056 Oh goodie. 1929666056 / 17144943 = 112.6. So going to disk has made it MUCH bigger. I am less than convinced by these numbers. > size(term_to_binary({mytable, 1000000, <<0:8192>>})) * 1000000. 1047000000 So, given that's at least in the same magnitude, I'm really suspecting the number returned by table_info when in disc_only_copies is in *bytes*, not words. > round ((size(term_to_binary({mytable, 1000000, <<0:8192>>})) * 1000000) / erlang:system_info(wordsize)). 130875000 The minimum number of words is some 7 times bigger than the amount of memory reported when using ets. However, I can well believe this assuming we don't really have 1e6 copies of that binary. So, from this, I deduce that calling table_info(Tab, memory) reports: a) if ets is being used, the number of words of RAM used by the table b) if ets isn't being used, the number of bytes of the table on disk Is this right? Cheers, Matthew From info@REDACTED Thu Jul 2 14:02:48 2009 From: info@REDACTED (info) Date: Thu, 2 Jul 2009 14:02:48 +0200 Subject: sending of messages to several childs Message-ID: <200907021402466706652@its3.ch> Hi all, Is it possible for a "father process" to send the same message to all spawned childs ? What is the syntax / or reference ? Thank's John From kamiseq@REDACTED Thu Jul 2 14:07:07 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Thu, 2 Jul 2009 14:07:07 +0200 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: <1246534666.6215.23.camel@piko.site> References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> <1246534666.6215.23.camel@piko.site> Message-ID: hehe if erlang is a way to get to coders haven then maybe emacs is it's purgatory :) pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From kamiseq@REDACTED Thu Jul 2 14:33:41 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Thu, 2 Jul 2009 14:33:41 +0200 Subject: [erlang-questions] sending of messages to several childs In-Reply-To: <200907021402466706652@its3.ch> References: <200907021402466706652@its3.ch> Message-ID: maybe you should keep pids of all your children and then iterate over the list and send same msg? pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From matthew@REDACTED Thu Jul 2 14:37:41 2009 From: matthew@REDACTED (Matthew Sackman) Date: Thu, 2 Jul 2009 13:37:41 +0100 Subject: [erlang-questions] Mnesia memory, size and effects of table copy types In-Reply-To: <20090702121536.6BD702000156@mwinf2024.orange.fr> References: <20090702121536.6BD702000156@mwinf2024.orange.fr> Message-ID: <20090702123740.GD1984@mrnibble.lshift.net> On Thu, Jul 02, 2009 at 02:15:39PM +0200, wde wrote: > by reading the mnesia source code i found this : > > table_info/2 -> > raw_table_info/2 -> > if storage type == disc_only_copies -> > dets:info/2 -> > dets_utils:position/3 -> > file:position/2 -> > > return offset counted in bytes Yeah, that's pretty much what I found from some quick grepping through. This surely is a bug then. Units should not change depending on table type. Also, the documentation should probably reenforce that this is the number of words (if we must, but why on earth was words chosen and not bytes?) used by the table content, irregardless of whether that content is on disk or not. I.e. it's quite sensible to think that when using disc_only_copies, memory should return the amount of RAM used and thus be small and reasonably constant. Now in my case, I definitely want the size of the dets table, but that's not the only interpretation, and the documentation should be clearer. Matthew From wde@REDACTED Thu Jul 2 14:15:39 2009 From: wde@REDACTED (wde) Date: Thu, 2 Jul 2009 14:15:39 +0200 Subject: [erlang-questions] Mnesia memory, size and effects of table copy types Message-ID: <20090702121536.6BD702000156@mwinf2024.orange.fr> by reading the mnesia source code i found this : table_info/2 -> raw_table_info/2 -> if storage type == disc_only_copies -> dets:info/2 -> dets_utils:position/3 -> file:position/2 -> return offset counted in bytes ======= le 02/07/2009, 12:39:20 vous ?criviez: ======= >I'm doing some manual memory management and when memory gets tight, I'm >converting some mnesia tables from disc_copies to disc_only_copies. But >I have a few questions because what I'm seeing reported through table_info >seems odd. > >ets and mnesia claim to report memory size in words. But dets reports in >bytes. When a table is put in disc_only_copies, can someone confirm it's >still words I'm getting back, and not bytes? Because the following looks >very very fishy > >> mnesia:create_table(mytable, [{disc_copies, [node()]}]). >{atomic,ok} >> mnesia:table_info(mytable, size). >0 >> mnesia:table_info(mytable, memory). >299 > >Ok, so presumeably, that's the number of words in RAM. Maybe. Docs don't >actually say - it could very well be the sum of bytes on disk and words >in ram, but let's assume it's not double counting. > >> mnesia:change_table_copy_type(mytable, node(), disc_only_copies). >{atomic,ok} >> mnesia:table_info(mytable, size). >0 >> mnesia:table_info(mytable, memory). >5752 > >Um. Ok, so maybe that's just some RAM based overhead or something. But >it's gone up quite a long way... far enough that I'm not convinced >that's words and not bytes. > >> mnesia:change_table_copy_type(mytable, node(), ram_copies). >{atomic,ok} >> mnesia:table_info(mytable, size). >0 >> mnesia:table_info(mytable, memory). >299 > >Oh well, at least we're back. Now, let's fill the table up: > >> [ mnesia:dirty_write(mytable, {mytable, N, <<0:8192>>}) || N <- lists:seq(1, 1000000) ]. >[ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok, > ok,ok,ok,ok,ok,ok,ok,ok,ok,ok|...] >> mnesia:table_info(mytable, size). >1000000 >> mnesia:table_info(mytable, memory). >17144943 > >> mnesia:change_table_copy_type(mytable, node(), disc_copies). >{atomic,ok} >> mnesia:table_info(mytable, size). >1000000 >> mnesia:table_info(mytable, memory). >17144943 > >Still good... but then: > >> mnesia:change_table_copy_type(mytable, node(), disc_only_copies). >{atomic,ok} >> mnesia:table_info(mytable, size). >1000000 >> mnesia:table_info(mytable, memory). >1929666056 > >Oh goodie. 1929666056 / 17144943 = 112.6. So going to disk has made it >MUCH bigger. I am less than convinced by these numbers. > >> size(term_to_binary({mytable, 1000000, <<0:8192>>})) * 1000000. >1047000000 > >So, given that's at least in the same magnitude, I'm really suspecting >the number returned by table_info when in disc_only_copies is in >*bytes*, not words. > >> round ((size(term_to_binary({mytable, 1000000, <<0:8192>>})) * 1000000) / erlang:system_info(wordsize)). >130875000 > >The minimum number of words is some 7 times bigger than the amount of >memory reported when using ets. However, I can well believe this >assuming we don't really have 1e6 copies of that binary. > >So, from this, I deduce that calling table_info(Tab, memory) reports: >a) if ets is being used, the number of words of RAM used by the table >b) if ets isn't being used, the number of bytes of the table on disk > >Is this right? > >Cheers, > >Matthew > >________________________________________________________________ >erlang-questions mailing list. See http://www.erlang.org/faq.html >erlang-questions (at) erlang.org > > = = = = = = = = = ========= = = = = = = = = = = wde wde@REDACTED 02/07/2009 From torben.lehoff@REDACTED Thu Jul 2 15:09:52 2009 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Thu, 2 Jul 2009 15:09:52 +0200 Subject: [erlang-questions] sending of messages to several childs In-Reply-To: <200907021402466706652@its3.ch> References: <200907021402466706652@its3.ch> Message-ID: Hi John. There is no built in operator for that, bou could look at: global_group pg2 pg Either way you have to do some book keeping or leg work to make it happen. Cheers, Torben On Thu, Jul 2, 2009 at 14:02, info wrote: > Hi all, > > Is it possible for a "father process" to send the same message to all > spawned childs ? > What is the syntax / or reference ? > Thank's > > John > -- http://www.linkedin.com/in/torbenhoffmann From psa@REDACTED Thu Jul 2 15:31:09 2009 From: psa@REDACTED (=?ISO-8859-1?Q?Paulo_S=E9rgio_Almeida?=) Date: Thu, 02 Jul 2009 14:31:09 +0100 Subject: [erlang-questions] Mnesia memory, size and effects of table copy types In-Reply-To: <20090702103919.GA1984@mrnibble.lshift.net> References: <20090702103919.GA1984@mrnibble.lshift.net> Message-ID: <4A4CB69D.303@di.uminho.pt> Hi, Matthew Sackman wrote: >> [ mnesia:dirty_write(mytable, {mytable, N, <<0:8192>>}) || N <- lists:seq(1, 1000000) ]. > [ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok, > ok,ok,ok,ok,ok,ok,ok,ok,ok,ok|...] >> mnesia:table_info(mytable, size). > 1000000 >> mnesia:table_info(mytable, memory). > 17144943 You are inserting large binaries which are not accounted for here. Only the references to them. > The minimum number of words is some 7 times bigger than the amount of > memory reported when using ets. However, I can well believe this > assuming we don't really have 1e6 copies of that binary. I tested and we have really 1e6 copies of that binary. i.e. doing [ets:insert(aaa, {N, <<0:8192>>}) || N <- lists:seq(1,1000000)] consumes much more memory than doing: B = <<0:8192>>, [ets:insert(aaa, {N, B}) || N <- lists:seq(1,1000000)]. This means that binary constants have a different behaviour than other constants. (Now there is a global pool for constants to avoid recreating them e.g. once each function invocation.) Is this accidental or some design decision, considering that the above is not common and that one wants to frequently append things to binaries? Regards, Paulo From info@REDACTED Thu Jul 2 16:02:18 2009 From: info@REDACTED (info) Date: Thu, 2 Jul 2009 16:02:18 +0200 Subject: [erlang-questions] sending of messages to several childs References: <20090702123624.472962000060@mwinf2028.orange.fr> Message-ID: <200907021602178941882@its3.ch> This solution seems elegant. I will test it. Thank you John you can use spawn_link/2 to spawn your children and use erlang:process_info(Father,links) to get all linked process if the father process is a supervisor you can use supervisor:which_children/1 you can use pg or pg2 module to add all chidren to the same group ======= le 02/07/2009, 14:02:48 vous ?criviez: ======= >Hi all, > >Is it possible for a "father process" to send the same message to all spawned childs ? >What is the syntax / or reference ? >Thank's > >John > = = = = = = = = = ========= = = = = = = = = = = wde wde@REDACTED 02/07/2009 From per@REDACTED Thu Jul 2 16:29:53 2009 From: per@REDACTED (Per Hedeland) Date: Thu, 2 Jul 2009 16:29:53 +0200 (CEST) Subject: [erlang-questions] why can not call erl_init() in drivers In-Reply-To: <4A4C0E28.5050408@lightpole.net> Message-ID: <200907021429.n62ETrG1064278@pluto.hedeland.org> caox wrote: > > I found the erl_interface lib can not be used when I was working on >some c-drivers for erlang, since there is no response when calling >erl_init(). This is probably because there is an erl_init() function in the emulator itself (in erts/emulator/beam/erl_init.c) that does "something completely different", and that's the one you end up calling. I happened to notice that as I wanted to use erl_interface in a driver a while back, never actually tried calling erl_init() though.:-) In my case it never was a problem, because in the driver I actually dynamically loaded a library (i.e. via dlopen()) that was built from erl_interface and some other stuff, and so while the driver does call the erl_init() in erl_interface, it does it via the pointer returned by dlsym(), not a direct call - hence no collision possible (for erl_init() or any other function that happens to have the same name as one in the emulator). There are probably other ways to avoid the issue, e.g. you could have the 3 lines of code that erl_init() consist of in your driver instead - or maybe it would work if you link your driver statically with erl_interface. --Per From chris.newcombe@REDACTED Thu Jul 2 17:02:57 2009 From: chris.newcombe@REDACTED (Chris Newcombe) Date: Thu, 2 Jul 2009 08:02:57 -0700 Subject: [erlang-questions] Is ets:insert/2 (with multiple objects) isolated with respect to concurrent readers? In-Reply-To: <4A4C8A29.80103@erix.ericsson.se> References: <781dd98c0906301123h4235262fq671f520c3060c029@mail.gmail.com> <4A4A7A64.7050509@erlang-consulting.com> <4A4B6C7F.70805@erix.ericsson.se> <781dd98c0907010712s38e1323fmacb3b869da69627d@mail.gmail.com> <781dd98c0907011227q5ae023aftcce56160a1ffe914@mail.gmail.com> <4A4C8A29.80103@erix.ericsson.se> Message-ID: <781dd98c0907020802s76fdec28m8f0d30bd2fafd1aa@mail.gmail.com> >>Don't you think it's enough if stated about {write_concurrency,bool()} that is does not break any semantic promises about atomicy and isolation. Maybe note that operations that makes such promises will gain less (or nothing) from {write_concurrency,true}. Yes, that would be fine IMO. > I don't think we should need to describe the current internal locking > strategy. Sorry, I wasn't more clear. I wasn't advocating exposing/documenting implementation details -- I realize that you need as much implementation freedom as possible. I was trying to show that some examples of potential implementation strategies might help users understand what is not guaranteed. But it was a bad suggestion -- it would be better to simply refer indirectly to the API semantics documented elsewhere, as you say. > Also, any guarantees about atomicy and isolation only applies to the data > that the function is operating on. Yes. IMO the API semantics should describe this. However, I think you need more detail. e.g. Atomic+isolated operations on multiple objects are really emulating simple transactions. What kind of isolation guarantee is offered? - 'serializable' isolation would On Thu, Jul 2, 2009 at 3:21 AM, Sverker Eriksson wrote: > Chris Newcombe wrote: >> >> I just noticed that the new {write_concurrency, true} option says that >> write-locks might no-longer be taken at the table level. >> >> ? ?"Different parts of the same table can be mutated (and read) by >> concurrent processes." >> >> (full text below) >> >> It does not say which write/read APIs are allowed to be concurrent. >> >> > > The idea with write_concurrency was that it should be pure performance > tuning > and not change any guarentees about API semantics. > >> So there's the usual natural tension between clean API semantics for >> compound write operations, and increased concurrency. ?e.g. Some >> applications might want atomicity, but might care more about increased >> concurrency than full isolation. ?Other applications (like my current >> one) might really need strong isolation. >> >> But I guess that backwards-compatibility reasons will dominate your >> decision (quite understandably). ?Given the historic implicit behavior >> (strong isolation) for delete_all_objects, insert, and insert_new, it >> would be dangerous to change them now. ? Also, strong isolation >> follows the principle of least surprise. >> >> > > True, backward-compatibility was the main reason for deciding now about > making > the atomic and isolated semantics of insert, insert_new and > delete_all_objects > to be guaranteed in the docs. > The introduction of write_concurrency was however about > backward-compatibility > with respect only to performance and not semantics. > >> It would be great if the updated documentation for the APIs >> specifically described the isolation semantics when >> {write_concurrency,true} is used. >> ?And vice-versa too; e.g. it would good if the documentation for >> write_concurrency mentioned that compound-write operations will either >> acquire multiple fine-grain write-locks (i.e. acquire all necessary >> locks before modifying anything), or may choose to acquire a >> table-level lock, to ensure (in either case) that their historic >> isolation behavior is preserved. >> Therefore applications that make heavy use of compound-write >> operations might see less benefit from {write_concurrency, true}. >> >> > > Don't you think it's enough if stated about {write_concurrency,bool()} that > is does > not break any semantic promises about atomicy and isolation. Maybe note that > operations > that makes such promises will gain less (or nothing) from > {write_concurrency,true}. > I don't think we should need to describe the current internal locking > strategy. > > Also, any guarantees about atomicy and isolation only applies to the data > that the function is operating on. > > /Sverker, Erlang/OTP > > > From chris.newcombe@REDACTED Thu Jul 2 17:02:57 2009 From: chris.newcombe@REDACTED (Chris Newcombe) Date: Thu, 2 Jul 2009 08:02:57 -0700 Subject: [erlang-questions] Is ets:insert/2 (with multiple objects) isolated with respect to concurrent readers? In-Reply-To: <4A4C8A29.80103@erix.ericsson.se> References: <781dd98c0906301123h4235262fq671f520c3060c029@mail.gmail.com> <4A4A7A64.7050509@erlang-consulting.com> <4A4B6C7F.70805@erix.ericsson.se> <781dd98c0907010712s38e1323fmacb3b869da69627d@mail.gmail.com> <781dd98c0907011227q5ae023aftcce56160a1ffe914@mail.gmail.com> <4A4C8A29.80103@erix.ericsson.se> Message-ID: <781dd98c0907020802s76fdec28m8f0d30bd2fafd1aa@mail.gmail.com> >>Don't you think it's enough if stated about {write_concurrency,bool()} that is does not break any semantic promises about atomicy and isolation. Maybe note that operations that makes such promises will gain less (or nothing) from {write_concurrency,true}. Yes, that would be fine IMO. > I don't think we should need to describe the current internal locking > strategy. Sorry, I wasn't more clear. I wasn't advocating exposing/documenting implementation details -- I realize that you need as much implementation freedom as possible. I was trying to show that some examples of potential implementation strategies might help users understand what is not guaranteed. But it was a bad suggestion -- it would be better to simply refer indirectly to the API semantics documented elsewhere, as you say. > Also, any guarantees about atomicy and isolation only applies to the data > that the function is operating on. Yes. IMO the API semantics should describe this. However, I think you need more detail. e.g. Atomic+isolated operations on multiple objects are really emulating simple transactions. What kind of isolation guarantee is offered? - 'serializable' isolation would On Thu, Jul 2, 2009 at 3:21 AM, Sverker Eriksson wrote: > Chris Newcombe wrote: >> >> I just noticed that the new {write_concurrency, true} option says that >> write-locks might no-longer be taken at the table level. >> >> ? ?"Different parts of the same table can be mutated (and read) by >> concurrent processes." >> >> (full text below) >> >> It does not say which write/read APIs are allowed to be concurrent. >> >> > > The idea with write_concurrency was that it should be pure performance > tuning > and not change any guarentees about API semantics. > >> So there's the usual natural tension between clean API semantics for >> compound write operations, and increased concurrency. ?e.g. Some >> applications might want atomicity, but might care more about increased >> concurrency than full isolation. ?Other applications (like my current >> one) might really need strong isolation. >> >> But I guess that backwards-compatibility reasons will dominate your >> decision (quite understandably). ?Given the historic implicit behavior >> (strong isolation) for delete_all_objects, insert, and insert_new, it >> would be dangerous to change them now. ? Also, strong isolation >> follows the principle of least surprise. >> >> > > True, backward-compatibility was the main reason for deciding now about > making > the atomic and isolated semantics of insert, insert_new and > delete_all_objects > to be guaranteed in the docs. > The introduction of write_concurrency was however about > backward-compatibility > with respect only to performance and not semantics. > >> It would be great if the updated documentation for the APIs >> specifically described the isolation semantics when >> {write_concurrency,true} is used. >> ?And vice-versa too; e.g. it would good if the documentation for >> write_concurrency mentioned that compound-write operations will either >> acquire multiple fine-grain write-locks (i.e. acquire all necessary >> locks before modifying anything), or may choose to acquire a >> table-level lock, to ensure (in either case) that their historic >> isolation behavior is preserved. >> Therefore applications that make heavy use of compound-write >> operations might see less benefit from {write_concurrency, true}. >> >> > > Don't you think it's enough if stated about {write_concurrency,bool()} that > is does > not break any semantic promises about atomicy and isolation. Maybe note that > operations > that makes such promises will gain less (or nothing) from > {write_concurrency,true}. > I don't think we should need to describe the current internal locking > strategy. > > Also, any guarantees about atomicy and isolation only applies to the data > that the function is operating on. > > /Sverker, Erlang/OTP > > > From andrea.dipersio@REDACTED Thu Jul 2 17:56:55 2009 From: andrea.dipersio@REDACTED (Andrea Di Persio) Date: Thu, 2 Jul 2009 08:56:55 -0700 (PDT) Subject: Simple OTP Application error Message-ID: <92cca880-d3f3-45b0-b5ad-a842458cacb8@g19g2000yql.googlegroups.com> Hi! I'm trying to build a simple otp application to dive deeper in erlang. I'm following the example in Joe Armstrong 'Programmin Erlang' and I also tried with some example on the web but I can't make it work correctly. The application load correctly but when I run application:start (appname) I get this error: application: hda exited: {bad_return, {{hda_app,start,[normal,[]]}, {'EXIT', {undef, [{hda_app,start,[normal,[]]}, {application_master,start_it_old,4}]}}}} type: temporary {error,{bad_return,{{hda_app,start,[normal,[]]}, {'EXIT',{undef,[{hda_app,start,[normal,[]]}, {application_master,start_it_old, 4}]}}}}} If I run the supervisor alone, it work correctly. This is my _app file: -module(hda_app). -behaviour(application). -export([start/2, stop/1]). start(_Type, _Args) -> hda_supervisor:start_link(). stop(_State) -> ok. This is my .app file: {application, hda, [{description, "Help Desk Assistant"}, {vsn, "1.0"}, {modules, [hda_app, hda_supervisor, customer_server, hda_alarm_handler]}, {registered, [customer_server]}, {applications, [kernel, stdlib, sasl]}, {mod, {hda_app, []}} ]}. This is my supervisor: -module(sellaprime_supervisor). -behaviour(supervisor). % see erl -man supervisor -export([start/0, start_in_shell_for_testing/0, start_link/1, init/ 1]). start() -> spawn(fun() -> supervisor:start_link({local,?MODULE}, ?MODULE, _Arg = []) end). start_in_shell_for_testing() -> {ok, Pid} = supervisor:start_link({local,?MODULE}, ?MODULE, _Arg = []), unlink(Pid). start_link(Args) -> supervisor:start_link({local,?MODULE}, ?MODULE, Args). init([]) -> %% Install my personal error handler gen_event:swap_handler(alarm_handler, {alarm_handler, swap}, {my_alarm_handler, xyz}), {ok, {{one_for_one, 3, 10}, {ok, {{one_for_one, 3, 10}, [{tag1, {customer_server, start_link, []}, permanent, 10000, worker, [customer_server]} ]}}. Best regards, Andrea. From thomasl_erlang@REDACTED Thu Jul 2 17:11:54 2009 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 2 Jul 2009 08:11:54 -0700 (PDT) Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> <1246534666.6215.23.camel@piko.site> Message-ID: <991018.15211.qm@web111414.mail.gq1.yahoo.com> ----- Original Message ---- > From: pawe? kami?ski > > hehe if erlang is a way to get to coders haven then maybe emacs is it's > purgatory :) Emacs is the gnostic secret of programming ... Best, Thomas PS. Devoted emacs user since 1984 (pre-gnu emacs! :-) From chris.newcombe@REDACTED Thu Jul 2 18:40:58 2009 From: chris.newcombe@REDACTED (Chris Newcombe) Date: Thu, 2 Jul 2009 09:40:58 -0700 Subject: [erlang-questions] Is ets:insert/2 (with multiple objects) isolated with respect to concurrent readers? In-Reply-To: <781dd98c0907020802s76fdec28m8f0d30bd2fafd1aa@mail.gmail.com> References: <781dd98c0906301123h4235262fq671f520c3060c029@mail.gmail.com> <4A4A7A64.7050509@erlang-consulting.com> <4A4B6C7F.70805@erix.ericsson.se> <781dd98c0907010712s38e1323fmacb3b869da69627d@mail.gmail.com> <781dd98c0907011227q5ae023aftcce56160a1ffe914@mail.gmail.com> <4A4C8A29.80103@erix.ericsson.se> <781dd98c0907020802s76fdec28m8f0d30bd2fafd1aa@mail.gmail.com> Message-ID: <781dd98c0907020940r4a0f27dfof5f06d752ac689fb@mail.gmail.com> Very sorry -- I somehow managed to get gmail to send an incomplete message unintentionally The unfinished part was:, > e.g. Atomic+isolated operations on multiple objects are really > emulating simple transactions. What kind of isolation guarantee is > offered? > - 'serializable' isolation would I was about to say that to achieve full 'serializable' isolation may require locking keys/objects that are currently absent (but which will be inserted by the current insert operation). (This is analogous to the 'phantom rows' problem in RDBMs). i.e. It requires two-pass locking; accumulate all necessary locks, including on the logical slots for any keys that are to be created. If you don't do this then concurrent multi-object writes may appear to be interleaved (not serializable) after the fact. (Of course, acquiring a table-level lock achieves this but is undesirable in other ways.) There are of course other standard serialization levels to choose from, e.g. 'repeatable-read'. So whichever type of isolation you choose to support for ETS, it may help users if the documentation describes it in standard vocabulary (assuming one of the standard levels matches the semantics of ETS.) Apologies again for the earlier incomplete message. regards, Chris On Thu, Jul 2, 2009 at 8:02 AM, Chris Newcombe wrote: >>>Don't you think it's enough if stated about {write_concurrency,bool()} that is does > not break any semantic promises about atomicy and isolation. Maybe > note that operations > that makes such promises will gain less (or nothing) from > {write_concurrency,true}. > > Yes, that would be fine IMO. > > >> I don't think we should need to describe the current internal locking >> strategy. > > Sorry, I wasn't more clear. ? I wasn't advocating exposing/documenting > implementation details -- I realize that you need as much > implementation freedom as possible. > I was trying to show that some examples of potential implementation > strategies might help users understand what is not guaranteed. > But it was a bad suggestion -- it would be better to simply refer > indirectly to the API semantics documented elsewhere, as you say. > > >> Also, any guarantees about atomicy and isolation only applies to the data >> that the function is operating on. > > Yes. ?IMO the API semantics should describe this. ?However, I think > you need more detail. > e.g. Atomic+isolated operations on multiple objects are really > emulating simple transactions. ?What kind of isolation guarantee is > offered? > ? - 'serializable' isolation would > > > > > On Thu, Jul 2, 2009 at 3:21 AM, Sverker > Eriksson wrote: >> Chris Newcombe wrote: >>> >>> I just noticed that the new {write_concurrency, true} option says that >>> write-locks might no-longer be taken at the table level. >>> >>> ? ?"Different parts of the same table can be mutated (and read) by >>> concurrent processes." >>> >>> (full text below) >>> >>> It does not say which write/read APIs are allowed to be concurrent. >>> >>> >> >> The idea with write_concurrency was that it should be pure performance >> tuning >> and not change any guarentees about API semantics. >> >>> So there's the usual natural tension between clean API semantics for >>> compound write operations, and increased concurrency. ?e.g. Some >>> applications might want atomicity, but might care more about increased >>> concurrency than full isolation. ?Other applications (like my current >>> one) might really need strong isolation. >>> >>> But I guess that backwards-compatibility reasons will dominate your >>> decision (quite understandably). ?Given the historic implicit behavior >>> (strong isolation) for delete_all_objects, insert, and insert_new, it >>> would be dangerous to change them now. ? Also, strong isolation >>> follows the principle of least surprise. >>> >>> >> >> True, backward-compatibility was the main reason for deciding now about >> making >> the atomic and isolated semantics of insert, insert_new and >> delete_all_objects >> to be guaranteed in the docs. >> The introduction of write_concurrency was however about >> backward-compatibility >> with respect only to performance and not semantics. >> >>> It would be great if the updated documentation for the APIs >>> specifically described the isolation semantics when >>> {write_concurrency,true} is used. >>> ?And vice-versa too; e.g. it would good if the documentation for >>> write_concurrency mentioned that compound-write operations will either >>> acquire multiple fine-grain write-locks (i.e. acquire all necessary >>> locks before modifying anything), or may choose to acquire a >>> table-level lock, to ensure (in either case) that their historic >>> isolation behavior is preserved. >>> Therefore applications that make heavy use of compound-write >>> operations might see less benefit from {write_concurrency, true}. >>> >>> >> >> Don't you think it's enough if stated about {write_concurrency,bool()} that >> is does >> not break any semantic promises about atomicy and isolation. Maybe note that >> operations >> that makes such promises will gain less (or nothing) from >> {write_concurrency,true}. >> I don't think we should need to describe the current internal locking >> strategy. >> >> Also, any guarantees about atomicy and isolation only applies to the data >> that the function is operating on. >> >> /Sverker, Erlang/OTP >> >> >> > From chris.newcombe@REDACTED Thu Jul 2 18:45:08 2009 From: chris.newcombe@REDACTED (Chris Newcombe) Date: Thu, 2 Jul 2009 09:45:08 -0700 Subject: [erlang-questions] dbg:p(all, [c]) hangs the shell In-Reply-To: References: Message-ID: <781dd98c0907020945n4c12fdacyaf6305c137117085@mail.gmail.com> > I would like to trace all calls in all processes, however the erlang > shell hangs after dbg:p(all, [c]). It even does not accept Ctrl-G. The documentation describes one issue that can cause deadlock, and suggests some work-arounds. Chris >From the end of http://erlang.org/doc/man/dbg.html "Note of caution When tracing function calls on a group leader process (an IO process), there is risk of causing a deadlock. This will happen if a group leader process generates a trace message and the tracer process, by calling the trace handler function, sends an IO request to the same group leader. The problem can only occur if the trace handler prints to tty using an io function such as format/2. Note that when dbg:p(all,call) is called, IO processes are also traced. Here's an example: %% Using a default line editing shell 1> dbg:tracer(process, {fun(Msg,_) -> io:format("~p~n", [Msg]), 0 end, 0}). {ok,<0.37.0>} 2> dbg:p(all, [call]). {ok,[{matched,nonode@REDACTED,25}]} 3> dbg:tp(mymod,[{'_',[],[]}]). {ok,[{matched,nonode@REDACTED,0},{saved,1}]} 4> mymod: % TAB pressed here %% -- Deadlock -- Here's another example: %% Using a shell without line editing (oldshell) 1> dbg:tracer(process). {ok,<0.31.0>} 2> dbg:p(all, [call]). {ok,[{matched,nonode@REDACTED,25}]} 3> dbg:tp(lists,[{'_',[],[]}]). {ok,[{matched,nonode@REDACTED,0},{saved,1}]} % -- Deadlock -- The reason we get a deadlock in the first example is because when TAB is pressed to expand the function name, the group leader (which handles character input) calls mymod:module_info(). This generates a trace message which, in turn, causes the tracer process to send an IO request to the group leader (by calling io:format/2). We end up in a deadlock. In the second example we use the default trace handler function. This handler prints to tty by sending IO requests to the user process. When Erlang is started in oldshell mode, the shell process will have user as its group leader and so will the tracer process in this example. Since user calls functions in lists we end up in a deadlock as soon as the first IO request is sent. Here are a few suggestions for how to avoid deadlock: * Don't trace the group leader of the tracer process. If tracing has been switched on for all processes, call dbg:p(TracerGLPid,clear) to stop tracing the group leader (TracerGLPid). process_info(TracerPid,group_leader) tells you which process this is (TracerPid is returned from dbg:get_tracer/0). * Don't trace the user process if using the default trace handler function. * In your own trace handler function, call erlang:display/1 instead of an io function or, if user is not used as group leader, print to user instead of the default group leader. Example: io:format(user,Str,Args). On Wed, Jul 1, 2009 at 11:28 AM, Kaiduan Xie wrote: > Hi, all, > > I would like to trace all calls in all processes, however the erlang > shell hangs after dbg:p(all, [c]). It even does not accept Ctrl-G. > > kaiduanx@REDACTED:~$ erl > Erlang (BEAM) emulator version 5.6.5 [source] [async-threads:0] [hipe] > [kernel-poll:false] > > Eshell V5.6.5 ?(abort with ^G) > 1> dbg:tracer(). > {ok,<0.33.0>} > 2> dbg:tpl('_', '_', '_', dbg:fun2ms(fun(_) -> return_trace() end)). > {ok,[{matched,nonode@REDACTED,5905},{saved,1}]} > 3> dbg:get_tracer(). > {ok,<0.34.0>} > 4> process_info(list_to_pid("<0.34.0>")). > [{current_function,{dbg,tracer_loop,2}}, > ?{initial_call,{erlang,apply,2}}, > ?{status,waiting}, > ?{message_queue_len,0}, > ?{messages,[]}, > ?{links,[<0.33.0>]}, > ?{dictionary,[]}, > ?{trap_exit,true}, > ?{error_handler,error_handler}, > ?{priority,max}, > ?{group_leader,<0.24.0>}, > ?{total_heap_size,233}, > ?{heap_size,233}, > ?{stack_size,3}, > ?{reductions,5}, > ?{garbage_collection,[{fullsweep_after,65535},{minor_gcs,0}]}, > ?{suspending,[]}] > 5> dbg:p(list_to_pid("<0.24.0>"), clear). > {ok,[{matched,nonode@REDACTED,1}]} > 6> dbg:p(all, [c]). > (<0.33.0>) call lists:map(#Fun,[]) > (<0.33.0>) returned from lists:map/2 -> [] > (<0.33.0>) call dbg:reply(<0.31.0>,{ok,[{matched,nonode@REDACTED,25}]}) > (<0.33.0>) returned from dbg:reply/2 -> {dbg, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {ok,[{matched,nonode@REDACTED,25}]}} > {ok,[{matched,nonode@REDACTED,25}]} > > Any idea on what was wrong? > > Thanks, > > kaiduan > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From wde@REDACTED Thu Jul 2 14:36:27 2009 From: wde@REDACTED (wde) Date: Thu, 2 Jul 2009 14:36:27 +0200 Subject: [erlang-questions] sending of messages to several childs Message-ID: <20090702123624.472962000060@mwinf2028.orange.fr> you can use spawn_link/2 to spawn your children and use erlang:process_info(Father,links) to get all linked process if the father process is a supervisor you can use supervisor:which_children/1 you can use pg or pg2 module to add all chidren to the same group ======= le 02/07/2009, 14:02:48 vous ?criviez: ======= >Hi all, > >Is it possible for a "father process" to send the same message to all spawned childs ? >What is the syntax / or reference ? >Thank's > >John > = = = = = = = = = ========= = = = = = = = = = = wde wde@REDACTED 02/07/2009 From jacek99@REDACTED Thu Jul 2 19:09:31 2009 From: jacek99@REDACTED (Jacek Furmankiewicz) Date: Thu, 2 Jul 2009 13:09:31 -0400 Subject: Unable to start YAWS Message-ID: <269a5c30907021009x7b8db75dgcbf3473249b77bc4@mail.gmail.com> We've been developing an OTP app on Ubuntu with embedded YAWS and all is good. We tried moving it to our production OS (CentOS 4.7) where I've manually compiled R12B-5. Unfortunately on this OS we get a startup error like this: application: das exited: {bad_return, {{das_app,start,[normal,[]]}, {'EXIT', {noproc, {gen_server,call, [yaws_server,check_certs,infinity]}}}}} Is there something missing in my setup? Does anyone know what is the cause of the check_certs error? I was guessing it may be SSL-related, so I downloaded openssl and openssl-devel on CentOS and recompiled Erlang from scratch with them..but same error persists. Any suggestions would be welcome... Cheers, Jacek P.S. basic sample of our YAWS code (nothing fancy, as you can see): GL = [{enable_soap,true}, {tmpdir,"res"},{logdir,"res"}, {flags,[{auth_log,false},{trace,true},{tty_trace,true},{copy_errlog,true}]}], SL = [{port,?YAWS_PORT},{servername,"localhost"},{dir_listings,true}, {appmods,[ {"das",das_http}, ]}, {listen,{0,0,0,0}},{flags,[{access_log,false}]}], yaws:start_embedded(".", SL, GL), yaws_soap_srv:setup({das_http,handler}, ?WSDL). From behdad.forghani@REDACTED Thu Jul 2 20:01:03 2009 From: behdad.forghani@REDACTED (Behdad Forghani) Date: Thu, 02 Jul 2009 14:01:03 -0400 Subject: Asn.1 Question Message-ID: <4A4CF5DF.1000205@samsarasol.com> Hi, I compiled LTE ASN.1 S1-AP specification with asn1ct. When I try to decode a message, I get the error: exception error: no match of right hand side value. The line that produces the error is: {Tmpterm1, Bytes3} = ?RT_PER:decode_open_type(Bytes2, []) It seems to me the decoder is calling decode_open_type and passing an empty list as the second parameter. Tmpterm1, Bytes3 and Bytes2 are unbound. Is my analysis correct? Sorry, I am very new to Erlang. Is this a bug in Asn.1 compiler? Is this the correct forum to ask this question? I really appreciate if any help you can give me. I am just getting my feet wet on Erlang. I am using OTP R13B01. The detailed error is: ** exception error: no match of right hand side value <<0,0,3,0,8,0,3,0,0,0,0,67,0,6,5,37,245,64,0,1,0,26,0,12, 11>> in function asn1rt_per_bin:getoctets_as_bin/2 in call from 'S1AP-PDU-Descriptions':dec_InitiatingMessage/2 called as 'S1AP-PDU-Descriptions':dec_InitiatingMessage({3, <<0,12,64,36, 0,0,3,0,8, 0,3,0,0,0, 0,67,0,6,5, 37,245,64, 0,1,0,26, ...>>}, telltype) in call from 'S1AP-PDU-Descriptions':'dec_S1AP-PDU'/2 in call from 'S1AP-PDU-Descriptions':decode_disp/2 The function is: dec_InitiatingMessage'(Bytes,_) -> %% attribute number 1 with type procedureCode {Term1,Bytes1} = ?RT_PER:decode_integer(Bytes,[{'ValueRange',{0,255}}]), %% attribute number 2 with type criticality {Term2,Bytes2} = ?RT_PER:decode_enumerated(Bytes1,[{'ValueRange',{0,2}}],{reject,ignore,notify}), %% attribute number 3 with type InitiatingMessage {Tmpterm1, Bytes3} = ?RT_PER:decode_open_type(Bytes2, []), Regards, Behdad From ulf.wiger@REDACTED Thu Jul 2 20:08:59 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 02 Jul 2009 20:08:59 +0200 Subject: [erlang-questions] Simple OTP Application error In-Reply-To: <92cca880-d3f3-45b0-b5ad-a842458cacb8@g19g2000yql.googlegroups.com> References: <92cca880-d3f3-45b0-b5ad-a842458cacb8@g19g2000yql.googlegroups.com> Message-ID: <4A4CF7BB.7080300@erlang-consulting.com> The 'undef' part of the error means that erlang couldn't find the function hda_app:start/2. Check that your module has been compiled and is accessible via the code path. BR, Ulf W Andrea Di Persio wrote: > Hi! > > I'm trying to build a simple otp application to dive deeper in erlang. > > I'm following the example in Joe Armstrong 'Programmin Erlang' and I > also tried with some example on the web but I can't make it work > correctly. > > The application load correctly but when I run application:start > (appname) I get this error: > > application: hda > exited: {bad_return, > {{hda_app,start,[normal,[]]}, > {'EXIT', > {undef, > [{hda_app,start,[normal,[]]}, > {application_master,start_it_old,4}]}}}} > type: temporary > {error,{bad_return,{{hda_app,start,[normal,[]]}, > {'EXIT',{undef,[{hda_app,start,[normal,[]]}, > {application_master,start_it_old, > 4}]}}}}} > > > > If I run the supervisor alone, it work correctly. > > > > This is my _app file: > > -module(hda_app). > -behaviour(application). > -export([start/2, stop/1]). > > > > start(_Type, _Args) -> > hda_supervisor:start_link(). > > > stop(_State) -> > ok. > > > > This is my .app file: > > {application, hda, > [{description, "Help Desk Assistant"}, > {vsn, "1.0"}, > {modules, [hda_app, hda_supervisor, customer_server, > hda_alarm_handler]}, > {registered, [customer_server]}, > {applications, [kernel, stdlib, sasl]}, > {mod, {hda_app, []}} > ]}. > > > This is my supervisor: > > > -module(sellaprime_supervisor). > -behaviour(supervisor). % see erl -man supervisor > > -export([start/0, start_in_shell_for_testing/0, start_link/1, init/ > 1]). > > > > start() -> > spawn(fun() -> > supervisor:start_link({local,?MODULE}, ?MODULE, _Arg = > []) > end). > > > start_in_shell_for_testing() -> > {ok, Pid} = supervisor:start_link({local,?MODULE}, ?MODULE, _Arg = > []), > unlink(Pid). > > > start_link(Args) -> > supervisor:start_link({local,?MODULE}, ?MODULE, Args). > > > init([]) -> > %% Install my personal error handler > gen_event:swap_handler(alarm_handler, > {alarm_handler, swap}, > {my_alarm_handler, xyz}), > {ok, {{one_for_one, 3, 10}, > > {ok, {{one_for_one, 3, 10}, > [{tag1, > {customer_server, start_link, []}, > permanent, > 10000, > worker, > [customer_server]} > ]}}. > > > > Best regards, > Andrea. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From mevans@REDACTED Thu Jul 2 21:12:03 2009 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 2 Jul 2009 15:12:03 -0400 Subject: Erlang receive / after performance strangeness Message-ID: Hi, This is interesting, not sure if it is expected or not. Two functions below. One WITH the timer on the receive loop: get_asset_data(ProcessId, Offset, Misc) -> ProcessId ! {get_asset_data, self(), Offset, Misc}, receive Data -> Data after 2000 -> {error, []} end. One WITHOUT the timer on the receive loop: get_asset_data(ProcessId, Offset, Misc) -> ProcessId ! {get_asset_data, self(), Offset, Misc}, receive Data -> Data end. When running on a VM the one WITH the timer gets about 9500 messages per second. The one WITHOUT gets 15000 messages per second. That's quite a performance drop (this is on a VM, running version 12B). Is that expected? Thanks Matt From yoursurrogategod@REDACTED Thu Jul 2 21:37:11 2009 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Thu, 2 Jul 2009 12:37:11 -0700 (PDT) Subject: What is the best software development platform to use when working with Erlang? In-Reply-To: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> Message-ID: <74d78504-2e64-4969-b7cc-ee4727169da3@o6g2000yqj.googlegroups.com> Usually I use gEdit. It's simple to use and prolific. I like the tabs at the top (yes, I know that emacs has 'buffers', but I'm stuck on gEdit :) ). If you're going to start modifying someone else's code, I'd suggest Eclipse. It seems like a nice way of standardizing your development environment. Users can potentially change things behind the scenes which could affect the manner in which code is run. Outside your development machine. I hope that you're using CVS/SVN/ git :) . The last thing that you want is to do is to start doing diffs manually ;) . On Jun 17, 6:34?pm, "G.S." wrote: > Hello everyone, > > Is Eclipse the best SDE to use? I usually don't bother using anything other > than a simple text editor with a highlighter since I can keep my code in my > head, and even when debugging I can just work my way through the logic to > find the errors I'm seeing even with 10k+ lines of code systems. But I'll > have to begin working with someone else's code in a while, and it will most > likely be over 25k lines of code, so I'll need something more robust. Anyone > suggest anything other than Eclipse with the Erlang plug-in? > > Regards, > -Gene From pfisher@REDACTED Thu Jul 2 21:40:51 2009 From: pfisher@REDACTED (Paul Fisher) Date: Thu, 02 Jul 2009 14:40:51 -0500 Subject: salesforce integration? Message-ID: <4A4D0D43.7080908@alertlogic.net> Anyone have a basic saleforce.com integration written in erlang available? -- paul From illo@REDACTED Thu Jul 2 22:13:33 2009 From: illo@REDACTED (Illo de' Illis) Date: Thu, 2 Jul 2009 22:13:33 +0200 Subject: Erlang/OTP on Vista 64bit Message-ID: <4297632C-E3D9-4823-A9B0-A61469E24C67@totalwire.it> G'dday folks. Is there anyone who's aware of glitches or bugs of Erlang/OTP on a Vista/64bit platform? I'm particulary interested in RabbitMQ, but I'd like to know whether having a stable Erlang/OTP generic application on Vista/64bit is dream or reality. Thank you very much. Ciao, Illo. From praveen.ray@REDACTED Thu Jul 2 22:22:41 2009 From: praveen.ray@REDACTED (Praveen Ray) Date: Thu, 2 Jul 2009 16:22:41 -0400 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: <991018.15211.qm@web111414.mail.gq1.yahoo.com> References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> <1246534666.6215.23.camel@piko.site> <991018.15211.qm@web111414.mail.gq1.yahoo.com> Message-ID: Maybe some Emacs gurus can help me out with my annoying Emacs string issue. I use stock erlang-mode on Ubuntu. Whenever I type a string with double slashes, emacs goes crazy and refuses to recognize the ending double quote as the end of the string and all following lines are now colored as if I'm still inside that string! Tab completions, indentations don't work and it looks ugly! re:run(MyString, "(\\d+)"), All lines here and after are thought to be inside that regex(by emacs)! I'm sure others have solved this problem! Thanks -Praveen On Thu, Jul 2, 2009 at 11:11 AM, Thomas Lindgren wrote: > > > > > > ----- Original Message ---- > > From: pawe? kami?ski > > > > hehe if erlang is a way to get to coders haven then maybe emacs is it's > > purgatory :) > > > Emacs is the gnostic secret of programming ... > > Best, > Thomas > > PS. Devoted emacs user since 1984 (pre-gnu emacs! :-) > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- Yellowfish Technologies Inc http://www.yellowfish.biz praveen.ray@REDACTED (888) 817 2969 x 233 gtalk/skype: praveenray From harveyd@REDACTED Thu Jul 2 22:54:00 2009 From: harveyd@REDACTED (Dale Harvey) Date: Thu, 2 Jul 2009 21:54:00 +0100 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> <1246534666.6215.23.camel@piko.site> <991018.15211.qm@web111414.mail.gq1.yahoo.com> Message-ID: I believe thats been fixed in the erlware-mode which is erlang mode but often a bit more up to date http://code.google.com/p/erlware-mode/ the other way is to put a comment after the line that terminates the escape, its usually for when people use $' though, I cant see how yours goes funny, but some combination of " and \ after the comment will probably fix it re:run(MyString, "(\\d+)"), % " 2009/7/2 Praveen Ray > Maybe some Emacs gurus can help me out with my annoying Emacs string issue. > I use stock erlang-mode on Ubuntu. Whenever I type a string with double > slashes, emacs goes crazy and refuses to recognize the ending double quote > as the end of the string and all following lines are now colored as if I'm > still inside that string! Tab completions, indentations don't work and it > looks ugly! > > re:run(MyString, "(\\d+)"), > All lines here and after are thought to be inside that regex(by emacs)! > > I'm sure others have solved this problem! > > Thanks > -Praveen > > On Thu, Jul 2, 2009 at 11:11 AM, Thomas Lindgren > wrote: > > > > > > > > > > > > > ----- Original Message ---- > > > From: pawe? kami?ski > > > > > > hehe if erlang is a way to get to coders haven then maybe emacs is it's > > > purgatory :) > > > > > > Emacs is the gnostic secret of programming ... > > > > Best, > > Thomas > > > > PS. Devoted emacs user since 1984 (pre-gnu emacs! :-) > > > > > > > > > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > > > -- > Yellowfish Technologies Inc > http://www.yellowfish.biz > praveen.ray@REDACTED > (888) 817 2969 x 233 > gtalk/skype: praveenray > From mikpe@REDACTED Fri Jul 3 00:09:57 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Fri, 3 Jul 2009 00:09:57 +0200 Subject: [erlang-questions] Erlang receive / after performance strangeness In-Reply-To: References: Message-ID: <19021.12341.311696.163046@pilspetsen.it.uu.se> Evans, Matthew writes: > Hi, > > This is interesting, not sure if it is expected or not. > > Two functions below. > > One WITH the timer on the receive loop: > > get_asset_data(ProcessId, Offset, Misc) -> > ProcessId ! {get_asset_data, self(), Offset, Misc}, > receive > Data -> Data > after 2000 -> > {error, []} > end. > > > One WITHOUT the timer on the receive loop: > > get_asset_data(ProcessId, Offset, Misc) -> > ProcessId ! {get_asset_data, self(), Offset, Misc}, > receive > Data -> Data > end. > > When running on a VM the one WITH the timer gets about 9500 messages per second. The one WITHOUT gets 15000 messages per second. > > That's quite a performance drop (this is on a VM, running version 12B). > > Is that expected? Some performance drop can be expected from using 'after' since whenever the receive blocks, it has to perform additional work to set up a timer, and timers aren't cheap. And assuming a reply is received in time, the timer has to be cancelled, and that's also an overhead. And your !/receive combo looks like it will always have to block waiting for ProcessId to do its thing, so you'll always incur the timer overhead. What you're doing here is a blocking RPC. A more Erlangy way of doing things is to have multiple pending asynchronous requests, and to have some loop that both generates new requests and receives replies to pending requests. That way delays in replies don't block other work. From mikpe@REDACTED Fri Jul 3 00:23:20 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Fri, 3 Jul 2009 00:23:20 +0200 Subject: [erlang-questions] Erlang/OTP on Vista 64bit In-Reply-To: <4297632C-E3D9-4823-A9B0-A61469E24C67@totalwire.it> References: <4297632C-E3D9-4823-A9B0-A61469E24C67@totalwire.it> Message-ID: <19021.13144.760602.324537@pilspetsen.it.uu.se> Illo de' Illis writes: > G'dday folks. > > Is there anyone who's aware of glitches or bugs of Erlang/OTP on a > Vista/64bit platform? I'm particulary interested in RabbitMQ, but I'd > like to know whether having a stable Erlang/OTP generic application on > Vista/64bit is dream or reality. A 32-bit Windows version should work fine running in Wow64 mode. A native 64-bit Windows version of Erlang/OTP was not possible last time I checked (late last year). The problem is Microsoft's decision to make the C 'long' type 32 bits although pointers are 64 bits. Fixing this is easy in principle (use uintptr_t), but requires a lot of tedious work and rigorous testing, so I suspect it will take a while before that happens. Perhaps run a 64-bit Linux/Unix version in a VMM on Vista/x64 instead? From corticalcomputer@REDACTED Fri Jul 3 02:01:32 2009 From: corticalcomputer@REDACTED (G.S.) Date: Thu, 2 Jul 2009 17:01:32 -0700 Subject: Strange CPU usage... Message-ID: <2a67d3ff0907021701i366f0915pa949c0b36f424df3@mail.gmail.com> Hello everyone, Has anyone else had a problem where after running their Erlang node for some distributed (multi process) program, their CPU (whether single, or dual, or quad core), went into 100% use, even after you killed the node completely? I've tried everything, but after a month of testing different stuff, the only consistent part in this 100% cpu usage is that it occurs every once in a while after I run my Erlang program, the thing that is strange is that even after I kill the node, the cpu is still bogged down. The System Monitor (Top), doesn't even show a process that uses 100% of the cpu, yet at the same time shows the CPU as being in 100% use. I can understand if my program goes into some infinite loop, but it persists after the node is killed. Any suggestions? -Gene p.s. I'm on linux, Debian. From jeffm@REDACTED Fri Jul 3 02:14:43 2009 From: jeffm@REDACTED (jm) Date: Fri, 03 Jul 2009 10:14:43 +1000 Subject: [erlang-questions] Strange CPU usage... In-Reply-To: <2a67d3ff0907021701i366f0915pa949c0b36f424df3@mail.gmail.com> References: <2a67d3ff0907021701i366f0915pa949c0b36f424df3@mail.gmail.com> Message-ID: <4A4D4D73.10307@ghostgun.com> G.S. wrote: > I've tried everything, but after a month of testing different stuff, the > only consistent part in this 100% cpu usage is that it occurs every once in > a while after I run my Erlang program, the thing that is strange is that > even after I kill the node, the cpu is still bogged down. The System Monitor > (Top), doesn't even show a process that uses 100% of the cpu, yet at the > same time shows the CPU as being in 100% use. > > Strange. What's the breakdown of the CPU usage? is it user, wait, or something else? in top hit the "1" key this should break the usage down by CPU you should see something similar to top - 10:13:01 up 288 days, 12:32, 1 user, load average: 0.00, 0.02, 0.06 Tasks: 88 total, 1 running, 87 sleeping, 0 stopped, 0 zombie Cpu0 : 0.7%us, 0.0%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st <--*** Cpu1 : 0.7%us, 0.3%sy, 0.0%ni, 98.7%id, 0.0%wa, 0.0%hi, 0.3%si, 0.0%st <--*** Mem: 3367504k total, 3224704k used, 142800k free, 149220k buffers Swap: 2031608k total, 36k used, 2031572k free, 2744644k cached Try a different kernel version. It could be a kernel or driver bug. Jeff. From kaiduanx@REDACTED Fri Jul 3 05:33:32 2009 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Thu, 2 Jul 2009 23:33:32 -0400 Subject: dbg:trace does not work on remote shell Message-ID: This is another question on dbg, it looks weird. If I start an OTP application in the shell, the trace works well. However, if an remote shell is attached to the running OTP application using -remsh (Remote shell is running on the same machine), trace does not work. There is no output of the trace on remote shell. Any idea? Thanks, kaiduan From ngocdaothanh@REDACTED Fri Jul 3 06:14:42 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 3 Jul 2009 13:14:42 +0900 Subject: [erlang-questions] Web services in erlang In-Reply-To: <2099288794.3851246506917180.JavaMail.root@ucimail3.uci.cu> References: <2099288794.3851246506917180.JavaMail.root@ucimail3.uci.cu> Message-ID: <5c493e530907022114u6e0d5f10xc02d3909c9443cdb@mail.gmail.com> Try this: http://github.com/yariv/erlyweb Ngoc. On Thu, Jul 2, 2009 at 12:55 PM, Jose Enrique Benitez Jimenez < jebenitez@REDACTED> wrote: > Hello comunity > > Where I can download erlyweb-0.7.1.tar.gz from anywhere but > erlyweb.googlecode.com (I am from Cuba so I dont have access to this URL) > and some documentation about how make web services in erlang. > > Thanks > > Enrique. From zerthurd@REDACTED Fri Jul 3 06:36:27 2009 From: zerthurd@REDACTED (Maxim Treskin) Date: Fri, 3 Jul 2009 11:36:27 +0700 Subject: [erlang-questions] Asn.1 Question In-Reply-To: <4A4CF5DF.1000205@samsarasol.com> References: <4A4CF5DF.1000205@samsarasol.com> Message-ID: It possible a bug in code generation. Can you provide this ASN.1 file? -- Maxim Treskin From adam.kocoloski@REDACTED Fri Jul 3 07:15:49 2009 From: adam.kocoloski@REDACTED (Adam Kocoloski) Date: Fri, 3 Jul 2009 01:15:49 -0400 Subject: is http:request's {self,once} option broken? Message-ID: Hi, I'd really like to use this option, but as far as I can tell it does not work in R13B. I'd be very happy if someone could demonstrate that the HTTP client actually does wait for a call to stream_next before sending subsequent chunks when this option is specified. When I try I get all the chunks in my mailbox automatically. Best, Adam From bflatmaj7th@REDACTED Fri Jul 3 07:34:30 2009 From: bflatmaj7th@REDACTED (Richard Andrews) Date: Fri, 3 Jul 2009 15:34:30 +1000 Subject: [erlang-questions] Strange CPU usage... In-Reply-To: <2a67d3ff0907021701i366f0915pa949c0b36f424df3@mail.gmail.com> References: <2a67d3ff0907021701i366f0915pa949c0b36f424df3@mail.gmail.com> Message-ID: <7702c0610907022234r1d601867ic38446ea17a3225c@mail.gmail.com> On Fri, Jul 3, 2009 at 10:01 AM, G.S. wrote: > Hello everyone, > > Has anyone else had a problem where after running their Erlang node for some > distributed (multi process) program, their CPU (whether single, or dual, or > quad core), went into 100% use, even after you killed the node completely? > > I've tried everything, but after a month of testing different stuff, the > only consistent part in this 100% cpu usage is that it occurs every once in > a while after I run my Erlang program, the thing that is strange is that > even after I kill the node, the cpu is still bogged down. The System Monitor > (Top), doesn't even show a process that uses 100% of the cpu, yet at the > same time shows the CPU as being in 100% use. > > I can understand if my program goes into some infinite loop, but it persists > after the node is killed. > > Any suggestions? > -Gene > p.s. I'm on linux, Debian. > If it is not attributed to a process in top then it will probably be iowait on a disk. Are you running a database or other software that might do large writes to disk from time to time? From nc@REDACTED Fri Jul 3 08:51:45 2009 From: nc@REDACTED (Nicolas Charpentier) Date: Fri, 03 Jul 2009 08:51:45 +0200 Subject: [erlang-questions] dbg:trace does not work on remote shell In-Reply-To: References: Message-ID: <4A4DAA81.6080809@charpi.net> Kaiduan Xie wrote: > This is another question on dbg, it looks weird. > > If I start an OTP application in the shell, the trace works well. > > However, if an remote shell is attached to the running OTP application > using -remsh (Remote shell is running on the same machine), trace does > not work. There is no output of the trace on remote shell. > > Any idea? > Hi, The output of dbg is done on the standard output of the first node. Dbg tracer that you have started use the group leader of your orininal node. If you need to have the output on the second node you can use dbg:n/1 to add all the node where trace pattern have to be activated. Start the first node. $ erl -sname first $ erl -sname debug debug@REDACTED> dbg:tracer(). {ok,} debug@REDACTED> dbg:n(first@REDACTED). {ok,first@REDACTED} debug@REDACTED> dbg:p(all,[c]). {ok,[{matched,first@REDACTED,33},{matched,debug@REDACTED,34}]} debug@REDACTED> dbg:tpl(....). All the output will be done on the node debug@REDACTED Regards, --- Nicolas Charpentier http://charpi.net From sameer.p.pradhan@REDACTED Fri Jul 3 09:38:53 2009 From: sameer.p.pradhan@REDACTED (sameer pradhan) Date: Fri, 3 Jul 2009 13:08:53 +0530 Subject: occi error due to erl_init(). Message-ID: <9ef06be80907030038sad93872l7cdf5284c87e6769@mail.gmail.com> hi all I'm using erlora-read-only package for occi and i got an error in cpp program during GDB as---- (gdb) 67 erl_init(NULL, 0); (gdb) erl_init (hp=0x0, heap_size=0) at legacy/erl_eterm.c:60 60 legacy/erl_eterm.c: No such file or directory. in legacy/erl_eterm.c Current language: auto; currently c (gdb) 61 in legacy/erl_eterm.c (gdb) erl_init_malloc (hp=0x0, heap_size=0) at legacy/erl_malloc.c:34 34 legacy/erl_malloc.c: No such file or directory. in legacy/erl_malloc.c (gdb) erl_init_eterm_alloc () at legacy/erl_fix_alloc.c:60 60 legacy/erl_fix_alloc.c: No such file or directory. in legacy/erl_fix_alloc.c (gdb) 65 in legacy/erl_fix_alloc.c (gdb) BUt in legacy folder that erl_eterm.c , erl_malloc.c and erl_fix_alloc.c all are present in usr/lib64/erlang/lib/erl_interface-3.6.1/src/legacy . can any body help me out... -- Thanks & Regards Sameer Prakash Pradhan From torben.lehoff@REDACTED Fri Jul 3 10:23:19 2009 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 3 Jul 2009 10:23:19 +0200 Subject: Insight on OTP vs plain Erlang Message-ID: Hi, Quick insight: use OTP instead of plain Erlang whenever possible. Elaboration: I had a task before me where I had to experiment with a design of a communication layer. Prototyping is easy with Erlang and since I was looking for a quick way of getting some feedback, so I thought that using plain Erlang without OTP might be the fastest way to get the task done. Boy, was I wrong... 1. You have to write a lot of boilerplate code that the OTP behaviours just does for you for free. 2. Navigating your code is more difficult since everything is hidden inside a big loop function. This is an issue with IDEs that gives you an outline of the program - just painful. 3. Going to production quality, i.e., real OTP components is a tedious task... basically a waste of time. The exercise in using plain Erlang is good for tuning ones skills in doing small sniplets of code that are too laborious to put into the OTP framework, but apart from that I would not reccomend it. OTP is flexible enough to be used for prototyping and it lends itself extremely well to an iterative development process. So the bottom line is that OTP has a learning curve, but once you are past that you are cruising at a speed no one else can achieve. I have just finished a function point analysis of a project done with OTP at work and it showed a 7x improvement over doing the same project using C++ (comparing to C: 11x) so I have learned my lesson now! Cheers, Torben -- http://www.linkedin.com/in/torbenhoffmann From kenneth.lundin@REDACTED Fri Jul 3 11:40:05 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Fri, 3 Jul 2009 11:40:05 +0200 Subject: [erlang-questions] Asn.1 Question In-Reply-To: <4A4CF5DF.1000205@samsarasol.com> References: <4A4CF5DF.1000205@samsarasol.com> Message-ID: Hi, Please provide the ASN.1 spec(s) , info about how you compiled them and the input bytes you are trying to decode and I will take a look. If you have more information about what the input bytes ought to be decoded to please provide that too. Regards Kenneth, Erlang/OTP, Ericsson On Thu, Jul 2, 2009 at 8:01 PM, Behdad Forghani wrote: > Hi, > > I compiled LTE ASN.1 S1-AP specification with asn1ct. When I try to decode a > message, I get the error: > exception error: no match of right hand side value. > > The line that produces the error is: > {Tmpterm1, Bytes3} = ?RT_PER:decode_open_type(Bytes2, []) > > It seems to me the decoder is calling decode_open_type and passing an empty > list as the second parameter. > Tmpterm1, Bytes3 and Bytes2 are unbound. Is my analysis correct? Sorry, I am > very new to Erlang. > Is this a bug in Asn.1 compiler? > Is this the correct forum to ask this question? > I really appreciate if any help you can give me. I am just getting my feet > wet on Erlang. > > I am using OTP R13B01. > The detailed error is: > > ** exception error: no match of right hand side value > ? ? ? ? ? ? ? ? ? <<0,0,3,0,8,0,3,0,0,0,0,67,0,6,5,37,245,64,0,1,0,26,0,12, > ? ? ? ? ? ? ? ? ? ? 11>> > ? ?in function ?asn1rt_per_bin:getoctets_as_bin/2 > ? ?in call from 'S1AP-PDU-Descriptions':dec_InitiatingMessage/2 > ? ? ? called as 'S1AP-PDU-Descriptions':dec_InitiatingMessage({3, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?<<0,12,64,36, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0,0,3,0,8, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0,3,0,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0,67,0,6,5, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?37,245,64, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0,1,0,26, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?...>>}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? telltype) > ? ?in call from 'S1AP-PDU-Descriptions':'dec_S1AP-PDU'/2 > ? ?in call from 'S1AP-PDU-Descriptions':decode_disp/2 > > ? The function is: > dec_InitiatingMessage'(Bytes,_) -> > > %% ?attribute number 1 with type procedureCode > {Term1,Bytes1} = ?RT_PER:decode_integer(Bytes,[{'ValueRange',{0,255}}]), > > %% ?attribute number 2 with type criticality > {Term2,Bytes2} = > ?RT_PER:decode_enumerated(Bytes1,[{'ValueRange',{0,2}}],{reject,ignore,notify}), > > %% ?attribute number 3 with type InitiatingMessage > {Tmpterm1, Bytes3} = ?RT_PER:decode_open_type(Bytes2, []), > > > Regards, > Behdad > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From jerome@REDACTED Fri Jul 3 13:13:22 2009 From: jerome@REDACTED (=?ISO-8859-1?Q?J=E9r=F4me_Desquilbet?=) Date: Fri, 03 Jul 2009 13:13:22 +0200 Subject: Erlang, OpenBSD and multicore Message-ID: <4A4DE7D2.70605@desquilbet.org> I am using OpenBSD as an OS on my server, and Erlang as well. I am considering now to upgrade to an Intel dualcore platform, taking advantage of OpenBSD support for SMP (see http://openbsd.org/faq/faq8.html#SMP ). I understand that Erlang won't automatically detect the dualcore with OpenBSD (as it would with some Linuxes). But will everything be OK if I "force" the CPU topology with erlang:system_flag()? Or is there a known supportability issue with OpenBSD? Thanks, J?r?me. From andrea.dipersio@REDACTED Fri Jul 3 16:07:10 2009 From: andrea.dipersio@REDACTED (Andrea Di Persio) Date: Fri, 3 Jul 2009 07:07:10 -0700 (PDT) Subject: Simple OTP Application error In-Reply-To: <4A4CF7BB.7080300@erlang-consulting.com> References: <92cca880-d3f3-45b0-b5ad-a842458cacb8@g19g2000yql.googlegroups.com> <4A4CF7BB.7080300@erlang-consulting.com> Message-ID: <06cd3ce1-ed49-4e83-92b6-ce07b3934222@a7g2000yqk.googlegroups.com> Bingo! I didn't compiled all the application file. Now it work like a charm! Thank you very much! On Jul 2, 8:08?pm, Ulf Wiger wrote: > The 'undef' part of the error means that erlang > couldn't find the function hda_app:start/2. > > Check that your module has been compiled and is > accessible via the code path. > > BR, > Ulf W > > > > > > Andrea Di Persio wrote: > > Hi! > > > I'm trying to build a simple otp application to dive deeper in erlang. > > > I'm following the example in Joe Armstrong 'Programmin Erlang' ?and I > > also tried with some example on the web but I can't make it work > > correctly. > > > The application load correctly but when I run application:start > > (appname) I get this error: > > > application: hda > > ? ? exited: {bad_return, > > ? ? ? ? ? ? ? ? {{hda_app,start,[normal,[]]}, > > ? ? ? ? ? ? ? ? ?{'EXIT', > > ? ? ? ? ? ? ? ? ? ? ?{undef, > > ? ? ? ? ? ? ? ? ? ? ? ? ?[{hda_app,start,[normal,[]]}, > > ? ? ? ? ? ? ? ? ? ? ? ? ? {application_master,start_it_old,4}]}}}} > > ? ? type: temporary > > {error,{bad_return,{{hda_app,start,[normal,[]]}, > > ? ? ? ? ? ? ? ? ? ? {'EXIT',{undef,[{hda_app,start,[normal,[]]}, > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {application_master,start_it_old, > > 4}]}}}}} > > > If I run the supervisor alone, it work correctly. > > > This is my ? _app file: > > > -module(hda_app). > > -behaviour(application). > > -export([start/2, stop/1]). > > > start(_Type, _Args) -> > > ? ? hda_supervisor:start_link(). > > > stop(_State) -> > > ? ? ok. > > > This is my .app file: > > > {application, hda, > > ?[{description, "Help Desk Assistant"}, > > ? {vsn, "1.0"}, > > ? {modules, [hda_app, hda_supervisor, customer_server, > > hda_alarm_handler]}, > > ? {registered, [customer_server]}, > > ? {applications, [kernel, stdlib, sasl]}, > > ? {mod, {hda_app, []}} > > ?]}. > > > This is my supervisor: > > > -module(sellaprime_supervisor). > > -behaviour(supervisor). % see erl -man supervisor > > > -export([start/0, start_in_shell_for_testing/0, start_link/1, init/ > > 1]). > > > start() -> > > ? ? spawn(fun() -> > > ? ? ? ? ? ? supervisor:start_link({local,?MODULE}, ?MODULE, _Arg = > > []) > > ? ? end). > > > start_in_shell_for_testing() -> > > ? ? {ok, Pid} = supervisor:start_link({local,?MODULE}, ?MODULE, _Arg = > > []), > > ? ? unlink(Pid). > > > start_link(Args) -> > > ? ? supervisor:start_link({local,?MODULE}, ?MODULE, Args). > > > init([]) -> > > ? ? %% Install my personal error handler > > ? ? gen_event:swap_handler(alarm_handler, > > ? ? ? ? ? ? ? ? ? ? ? ? ? ?{alarm_handler, swap}, > > ? ? ? ? ? ? ? ? ? ? ? ? ? ?{my_alarm_handler, xyz}), > > ? ? ? ? ? ? ? ? ? ? ? ? ? ?{ok, {{one_for_one, 3, 10}, > > > ? ? {ok, {{one_for_one, 3, 10}, > > ? ? ? ? ? [{tag1, > > ? ? ? ? ? ? {customer_server, start_link, []}, > > ? ? ? ? ? ? ?permanent, > > ? ? ? ? ? ? ?10000, > > ? ? ? ? ? ? ?worker, > > ? ? ? ? ? ? ?[customer_server]} > > ? ? ? ? ? ]}}. > > > Best regards, > > Andrea. > > > ________________________________________________________________ > > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltdhttp://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From prikrutil@REDACTED Fri Jul 3 20:43:05 2009 From: prikrutil@REDACTED (Sergey Samokhin) Date: Fri, 3 Jul 2009 11:43:05 -0700 Subject: Four ways to start mnesia in OTP environment. Which one should one use? Message-ID: Hello. I've found four ways to start mnesia in OTP environment. Which one should I use? I was a bit puzzled to decide which of them is better. 1) Boot scripts It seems to be the default approach when it comes to making sure that all the dependencies of your app will be started. But there are some disadvantages which make it quite difficult to use boot scripts as some kind of "dependencies controller": [-] Runtime doesn't give you a standard way to prepare the system before a dependency has been started. E.g. it isn't possible for me to ensure that a directory pointed by "-mnesia dir" is really exist before mnesia has been started. To make sure that the appropriate directory exists I have to move the corresponding code to my escript-based CLI. I don't think a developer will be happy to know that such a part of code as checks is no longer part of an Erlang Application. This code can't be executed if the application is started manually under the fresh erl-shell. [-] If you use a boot script you have to generate a new one for every OTP release your application is supposed to work with. [-] Mnesia isn't automatically stopped after you application has been terminated. I'm not sure if it's a problem, but if one is going to start another application which is supposed to use mnesia in the same VM, it's a good idea to restart mnesia with a fresh schema generated. 2) erl -run mnesia To start mnesia one can also decide to use either "-run" or "-s" parameters to erl. Disadvantages of this approach are the same as of the previous one except that in this case you don't have to regenerate boot scripts every time you upgrade your erlang package (because there is no any boot scripts =). 3) From init/1 function of a gen_server using mnesia. You just write something like this to control when mnesia is started: init(_) -> mnesia:start(), %... [-] It seems a good solution, but you can't stop mnesia in terminate/1 function (mnesia:stop() call hangs). It seems a bit inconsistent to leave mnesia running after your application has been stopped. 4) Start mnesia as included application Although how to work with Included Applications is described in OTP Design Principles I don't see this way being used widely. Maybe because of the following questions: [-] Is it safe to start mnesia as included application by starting mnesia_sup:start/0 from standard OTP supervsor? [-] Is it safe to set mnesia options (like dir) using set_env before your_sup:init() has been returned? Although I've noted only disadvantages (it's more difficult for me to estimate advantages of those ways right now) I try to use them all. Which one do you prefer and why? Thanks. -- Sergey Samokhin From kunthar@REDACTED Sat Jul 4 00:59:57 2009 From: kunthar@REDACTED (Kunthar) Date: Sat, 4 Jul 2009 01:59:57 +0300 Subject: Mumps M/DB and M/DBX Message-ID: <9a09ca9a0907031559h19c0b4b7n2664d895ebb8db70@mail.gmail.com> I'd really appreciated to get pros and cons, if anyone used before those DB types. What you think about GT.M and extended clone of M/DB? http://www.mgateway.com/ http://groups.google.co.uk/group/mdb-community-forum http://www.fidelityinfoservices.com/FNFIS/Markets/NonfinancialIndustries/Healthcare/gtm/ Peace \|/ Kunth From kaiduanx@REDACTED Sat Jul 4 04:40:19 2009 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Fri, 3 Jul 2009 22:40:19 -0400 Subject: [erlang-questions] dbg:trace does not work on remote shell In-Reply-To: <4A4DAA81.6080809@charpi.net> References: <4A4DAA81.6080809@charpi.net> Message-ID: Nicolas, This does not work out. When I type the dbg:p(list_to_pid("<0.X.0>"), [c, m]), 4> dbg:p(list_to_pid("<0.264.0>"), [c, m]). {ok,[{matched,'debug1@REDACTED',0,{no_proc,<0.264.0>}}]} Process <0.264.0> is on running OTP application. Also I am attaching an remote shell to the running OTP application with -remsh, this remote shell is a local shell on the running OTP application. Thanks, kaiduan On Fri, Jul 3, 2009 at 2:51 AM, Nicolas Charpentier wrote: > Kaiduan Xie wrote: >> >> This is another question on dbg, it looks weird. >> >> If I start an OTP application in the shell, the trace works well. >> >> However, if an remote shell is attached to the running OTP application >> using -remsh (Remote shell is running on the same machine), trace does >> not work. There is no output of the trace on remote shell. >> >> Any idea? >> > > Hi, > The output of dbg is done on the standard output of the first node. Dbg > tracer that you have started use the group leader of your orininal node. > If you need to have the output on the second node you can use dbg:n/1 to > ?add all the node where trace pattern have to be activated. > > ?Start the first node. > > $ erl -sname first > > > > $ erl -sname debug > debug@REDACTED> dbg:tracer(). > {ok,} > debug@REDACTED> dbg:n(first@REDACTED). > {ok,first@REDACTED} > debug@REDACTED> dbg:p(all,[c]). > {ok,[{matched,first@REDACTED,33},{matched,debug@REDACTED,34}]} > debug@REDACTED> dbg:tpl(....). > > > All the output will be done on the node debug@REDACTED > > > > Regards, > > --- > Nicolas Charpentier > http://charpi.net > From victor.sovetov@REDACTED Sat Jul 4 17:34:44 2009 From: victor.sovetov@REDACTED (Viktor Sovietov) Date: Sat, 4 Jul 2009 08:34:44 -0700 (PDT) Subject: Mumps M/DB and M/DBX In-Reply-To: <9a09ca9a0907031559h19c0b4b7n2664d895ebb8db70@mail.gmail.com> References: <9a09ca9a0907031559h19c0b4b7n2664d895ebb8db70@mail.gmail.com> Message-ID: Hi Knuth MUMPS concept is really powerful, but the good part of this power exists because MUMPS interpreter is the part of DB engine and MUMPS code is able to operate with native datastructures without any transformations (like marshalling/unmarshalling). MUMPS databases are fast and very scalable, but IMHO it would be difficult to avoid dramatic drop of performance on bridge between Erlang and MUMPS. Unfortunately. Some time ago I tried to create Erlang/OTP-based replacement for MUMPS with using tuples and lists to represent hierarchical index, but it seemed that no one was interested to use such database. Sincerely, --Victor On Jul 4, 1:59?am, Kunthar wrote: > I'd really appreciated to get pros and cons, if anyone used before > those DB types. > What you think about GT.M and extended clone of M/DB? > > http://www.mgateway.com/http://groups.google.co.uk/group/mdb-community-forumhttp://www.fidelityinfoservices.com/FNFIS/Markets/NonfinancialIndustr... > > Peace > \|/ Kunth > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From buricchio@REDACTED Sun Jul 5 00:04:06 2009 From: buricchio@REDACTED (Andrew) Date: Sun, 05 Jul 2009 01:04:06 +0300 Subject: [erlang-questions] Mumps M/DB and M/DBX In-Reply-To: <9a09ca9a0907031559h19c0b4b7n2664d895ebb8db70@mail.gmail.com> References: <9a09ca9a0907031559h19c0b4b7n2664d895ebb8db70@mail.gmail.com> Message-ID: <4A4FD1D6.6010308@gmail.com> Hi, MUMPS is very powerful. We use it in conjunction with Erlang. Architecture is {Web application -> Erlang middleware -> MUMPS}. In fact, we use MUMPS from Intersystems Cache DB, but it is quite the same as GT.M. I can't remember any cons, it really save much time for us. Both MUMPS (especially Cache) and Erlang complement each other quite well. Perhaps, you have more concrete questions? Andrew Kunthar wrote: > I'd really appreciated to get pros and cons, if anyone used before > those DB types. > What you think about GT.M and extended clone of M/DB? > > > http://www.mgateway.com/ > http://groups.google.co.uk/group/mdb-community-forum > http://www.fidelityinfoservices.com/FNFIS/Markets/NonfinancialIndustries/Healthcare/gtm/ > > Peace > \|/ Kunth > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From kunthar@REDACTED Sun Jul 5 01:18:38 2009 From: kunthar@REDACTED (Kunthar) Date: Sun, 5 Jul 2009 02:18:38 +0300 Subject: [erlang-questions] Mumps M/DB and M/DBX In-Reply-To: <4A4FD1D6.6010308@gmail.com> References: <9a09ca9a0907031559h19c0b4b7n2664d895ebb8db70@mail.gmail.com> <4A4FD1D6.6010308@gmail.com> Message-ID: <9a09ca9a0907041618p1c670d85p15e56a509b7b5096@mail.gmail.com> Let me be more specific then :) 1. Seems it supports ACID, is this clear? 2. What about transaction speed? Any benchmarks would be cool. 3. Is Erlang Middleware is also shareable like new M/DB or should i need to dig in myself? 4. How scalable is? 5. Is this support like multidimensional array data types? 6. How painful to/from convert data types of Erlang? Peace \|/ Kunthar On Sun, Jul 5, 2009 at 1:04 AM, Andrew wrote: > Hi, > MUMPS is very powerful. We use it in conjunction with Erlang. Architecture > is {Web application -> Erlang middleware -> MUMPS}. In fact, we use MUMPS > from Intersystems Cache DB, but it is quite the same as GT.M. I can't > remember any cons, it really save much time for us. Both MUMPS (especially > Cache) and Erlang complement each other quite well. Perhaps, you have more > concrete questions? > > Andrew > > > Kunthar wrote: >> >> I'd really appreciated to get pros and cons, if anyone used before >> those DB types. >> What you think about GT.M and extended clone of M/DB? >> >> >> http://www.mgateway.com/ >> http://groups.google.co.uk/group/mdb-community-forum >> >> http://www.fidelityinfoservices.com/FNFIS/Markets/NonfinancialIndustries/Healthcare/gtm/ >> >> Peace >> \|/ Kunth >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> > > From steven.charles.davis@REDACTED Sun Jul 5 01:44:45 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sat, 4 Jul 2009 16:44:45 -0700 (PDT) Subject: Four ways to start mnesia in OTP environment. Which one should one use? In-Reply-To: References: Message-ID: Since nobody else has taken this up this holiday weekend, here's my take: 1) When you intend to stop mnesia -- how are you sure that it's not being used by another application? 2) Why would you stop your application (apart for development purposes or decommissioning)? 3) Since Mnesia is an entire subsystem you could either put it in your application/applications requirement or ensure it's started in the application/mod supervisor startup call. That's my take - I'm sure others have more/better suggestions. BR, /sd From ulf.wiger@REDACTED Sun Jul 5 10:15:53 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 05 Jul 2009 10:15:53 +0200 Subject: [erlang-questions] Four ways to start mnesia in OTP environment. Which one should one use? In-Reply-To: References: Message-ID: <4A506139.5030609@erlang-consulting.com> Sergey Samokhin wrote: > Hello. > > I've found four ways to start mnesia in OTP environment. Which one > should I use? I was a bit puzzled to decide which of them is better. > > 1) Boot scripts > [...] > > [-] Runtime doesn't give you a standard way to prepare the system > before a dependency has been started. E.g. it isn't possible for me to > ensure that a directory pointed by "-mnesia dir" is really exist > before mnesia has been started. The conventional way to deal with this is to require an explicit install phase. One way to accomplish this is to create an application that starts before mnesia. The way to ensure that it runs before mnesia when using a boot script is to list the application before mnesia in the .rel file. The script builder may adjust the order of applications based on dependencies, but it will not reorder applications un- necessarily. Thus, if you put an application before mnesia, and it doesn't depend on mnesia, it will start before it. This app can check whether the system has been installed, or bootstrap the installation otherwise. It could also check e.g. whether the node has restarted due to partitioned network, and try to figure out whether to set master nodes in mnesia, or perhaps whether it should simply wait for some condition to be fulfilled before letting mnesia synch with the others. > [-] If you use a boot script you have to generate a new one for every > OTP release your application is supposed to work with. Well, yes, but erlc will do that for you. The pain is that you have to generate a new .erl file, as the application versions are listed in it. The 'builder' contrib that I wrote years ago addressed that, and I believe that Sinan/Faxien does that too. http://www.erlware.org > [-] Mnesia isn't automatically stopped after you application has been > terminated. I'm not sure if it's a problem, but if one is going to > start another application which is supposed to use mnesia in the same > VM, it's a good idea to restart mnesia with a fresh schema generated. This sounds like you really have two independent applications that do /not/ need to run in the same VM. Why do you want to reuse the VM instance? I'd shut down the erlang node in an orderly fashion, and start a new one, using a different boot script and a different mnesia dir. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From prikrutil@REDACTED Sun Jul 5 20:34:15 2009 From: prikrutil@REDACTED (Sergey Samokhin) Date: Sun, 5 Jul 2009 11:34:15 -0700 Subject: [erlang-questions] Re: Four ways to start mnesia in OTP environment. Which one should one use? In-Reply-To: References: Message-ID: Hello, Steve. Thanks for your answer. > 1) When you intend to stop mnesia -- how are you sure that it's not > being used by another application? It's often possible, though it may require some additional steps to take. There is at least one way to ensure that all of your applications using mnesia terminated before mnesia has been stopped: every application you start has to write #app{app = AppName, node = node()} record during the initialization. Later you will be able to check if some of the allowed applications are still running by looking at #app{} records present in mnesia. It only works if applications you're trying to "ping" that way are written in a uniform manner, i.e. register themself through mnesia:write(AppRecord). There must be also a corresponding function unregistering dead apps in a safe/distributed manner. I don't think it will be possible if one has decided to start mnesia as Included Applcation. > 2) Why would you stop your application (apart for development purposes > or decommissioning)? To dynamically reconfigure a cluster: add and remove some applications on the fly. But I don't know what I should answer to "Why do you want to stop mnesia?" =) Maybe because of the thought that all dependencies one has started in init/1 has to be stopped in terminate/1. > 3) Since Mnesia is an entire subsystem you could either put it in your > application/applications requirement or ensure it's started in the > application/mod supervisor startup call. Does this mean that Included application is a wrong way? -- Sergey Samokhin From prikrutil@REDACTED Sun Jul 5 20:34:27 2009 From: prikrutil@REDACTED (Sergey Samokhin) Date: Sun, 5 Jul 2009 11:34:27 -0700 Subject: [erlang-questions] Four ways to start mnesia in OTP environment. Which one should one use? In-Reply-To: <4A506139.5030609@erlang-consulting.com> References: <4A506139.5030609@erlang-consulting.com> Message-ID: Hello, Ulf. > The conventional way to deal with this is to require > an explicit install phase. I'll try it, thanks! > The 'builder' contrib that I wrote years ago addressed that, and I > believe that Sinan/Faxien does that too. http://www.erlware.org Thanks again! :] > This sounds like you really have two independent applications > that do /not/ need to run in the same VM. Yeah, it is (details are below). It looks like I don't have to stop mnesia when an app is terminating. > Why do you want to reuse the VM instance? > I'd shut down the erlang node in an orderly fashion, and > start a new one, using a different boot script and > a different mnesia dir. Let me explain what I was thinking about when writing the first letter: Let's suppose you need to write a few applications which are supposed to work in a cluster and be added/removed regardless of which applications are already running: 'FrontEnd', 'BackEnd', 'WebInterface', 'Monitoring'. The obvious solution to make a cluster is by using mnesia as a kind of foundation. {first_machine, ['FrontEnd', 'WebInterface']} {second_machine, ['BackEnd', 'Monitoring']} Next question is "How should I split applications across VMs?" There are two answers: *one VM - one application* That is what I'm using now. I.e. all applications are started in their own VMs vs *One VM - many applications* I've never tried it, but maybe it's a more convenient way? And here it would be cool to stop all the dependencies of an application (but not mnesia, because it's used to connect nodes together). -- Sergey Samokhin From rvirding@REDACTED Mon Jul 6 01:11:16 2009 From: rvirding@REDACTED (Robert Virding) Date: Mon, 6 Jul 2009 01:11:16 +0200 Subject: [erlang-questions] Why doesn't erl_eval compile code? In-Reply-To: References: Message-ID: <3dbc6d1c0907051611s2a7b1aa5m1a391e0bae613c8f@mail.gmail.com> Seeing no one has replied I will give some comments: - What problem are you trying to solve? I admit that it would be cool to do it, but why? - While the fun code is not very elegant :-), it does work as it should. What does not work as it should is interpreting receive. - If you remove the module when you are done, which you must explicitly do, then there could be some problem with funs if you have passed them on and they still are in use as they will not work after their module has been removed. - The interpreter was never intended to run larger programs so in one respect it wold be a waste of time. - If you implemented this then PDQ you would get requests to be able to define functions and modules in the shell. :-) But it would be cool code, Robert 2009/6/24 Tony Arcieri > In its present implementation erl_eval accepts an Erlang parse tree and it > walks the nodes of the parse tree, evaluating them node by node. This is > somewhat inelegant in many cases, such as when you run into constructs like > funs: > > http://gist.github.com/79022 > > Why not transform the parse tree into something that provides erl_eval's > return value (i.e. real return value + final bindings), compile that to a > temporary module, load that into the code server, and execute it, removing > the module when it's done. Wouldn't that solution provide more consistent > semantics while also being faster at things like tail calls and other > computationally complex problems? > > I'll go ahead and concede there's a great deal of complexity to both > solutions, and the compiled solution presents some concurrency problems, as > I understand. Please expound on that issue either way. > > As a bit of background, I'm working on moving the eval strategy of my > language Reia (which runs on the Erlang VM) off of erl_eval completely, > having it always compile to BEAM/HiPE bytecode, to make the semantics more > consistent and also improve performance. I'm curious why Erlang doesn't > take this approach. > > -- > Tony Arcieri > From kaiduanx@REDACTED Mon Jul 6 03:09:41 2009 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Sun, 5 Jul 2009 21:09:41 -0400 Subject: Repost/dbg:trace does not work on remote shell Message-ID: Hi, all, Repost on weird problem on dbg. 1. If I start ejabberd (version 2.0.1) with the following options (I modify ejabberdctl) start() { erl \ -name $NODE@$HOST \ -pa $EJABBERD_EBIN \ -mnesia dir "\"$EJABBERD_DB\"" \ -s ejabberd \ -ejabberd config \"$EJABBERD_CONFIG_PATH\" \ log_path \"$EJABBERD_LOG_PATH\" \ -sasl sasl_error_logger \{file,\"$SASL_LOG_PATH\"\} } dbg:trace can work on LIVE shell. 2. However, if I start ejabberd with a different option (using original ejabberdctl start) start() { erl \ -noinput -detached \ -name $NODE@$HOST \ -pa $EJABBERD_EBIN \ -mnesia dir "\"$EJABBERD_DB\"" \ -s ejabberd \ -ejabberd config \"$EJABBERD_CONFIG_PATH\" \ log_path \"$EJABBERD_LOG_PATH\" \ -sasl sasl_error_logger \{file,\"$SASL_LOG_PATH\"\} } and attach a remote shell to the running ejabberd server using ejabberdctl debug debug() { erl \ -name debug$NODE@$HOST \ -pa $EJABBERD_EBIN \ -mnesia dir "\"$EJABBERD_DB\"" \ -remsh $NODE@$HOST } dbg:trace() does not work on the remote shell. kaiduanx@REDACTED:~/xmpp/ejabberd-2.0.1/tools$ ./ejabberdctl start kaiduanx@REDACTED:~/xmpp/ejabberd-2.0.1/tools$ ./ejabberdctl debug Erlang (BEAM) emulator version 5.6.5 [source] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.6.5 (abort with ^G) (ejabberd@REDACTED)1> appmon:start(). {ok,<0.343.0>} (ejabberd@REDACTED)2> dbg:tracer(). {ok,<0.353.0>} (ejabberd@REDACTED)3> dbg:tpl('_', '_', '_', dbf:fun2ms(fun(_) -> return_trace() end)). ** exception error: undefined function dbf:fun2ms/1 (ejabberd@REDACTED)4> dbg:tpl('_', '_', '_', dbg:fun2ms(fun(_) -> return_trace() end)). {ok,[{matched,'ejabberd@REDACTED',14795},{saved,1}]} (ejabberd@REDACTED)5> dbg:p(list_to_pid("<0.258.0>"), [c, m]). {ok,[{matched,'ejabberd@REDACTED',1}]} (ejabberd@REDACTED)6> Any idea why dbg does not work on remote shell? What is the practice to trace a running OTP application , for example, a server running on customer site remotely? Thanks, kaiduan From paul-trapexit@REDACTED Mon Jul 6 03:20:08 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Sun, 5 Jul 2009 18:20:08 -0700 (PDT) Subject: [erlang-questions] Four ways to start mnesia in OTP environment. Which one should one use? In-Reply-To: References: Message-ID: I ended up going with #4, included application, on a large project and was satisfied. The mnesia bits were open sourced, you can check out http://code.google.com/p/schemafinder/ for details. The main reason for going with included application was to be able to ensure that nodes were joining the distributed schema in a reasonable state. We also started our nodes with a shell script that checked all sorts of preconditions before invoking erl. That seems a fact of life to me under all scenarios (e.g., where are you putting standard out and standard error from the emulator?). For a simpler hobby project I'm working on starting mnesia as an app has been working fine, mnesia is really good about letting you do all sorts of manipulations while it is running so letting it start on it's own is not that limiting. -- p On Fri, 3 Jul 2009, Sergey Samokhin wrote: > Hello. > > I've found four ways to start mnesia in OTP environment. Which one > should I use? I was a bit puzzled to decide which of them is better. > > 1) Boot scripts > > It seems to be the default approach when it comes to making sure that > all the dependencies of your app will be started. > > But there are some disadvantages which make it quite difficult to use > boot scripts as some kind of "dependencies controller": > > [-] Runtime doesn't give you a standard way to prepare the system > before a dependency has been started. E.g. it isn't possible for me to > ensure that a directory pointed by "-mnesia dir" is really exist > before mnesia has been started. To make sure that the appropriate > directory exists I have to move the corresponding code to my > escript-based CLI. I don't think a developer will be happy to know > that such a part of code as checks is no longer part of an Erlang > Application. This code can't be executed if the application is started > manually under the fresh erl-shell. > [-] If you use a boot script you have to generate a new one for every > OTP release your application is supposed to work with. > [-] Mnesia isn't automatically stopped after you application has been > terminated. I'm not sure if it's a problem, but if one is going to > start another application which is supposed to use mnesia in the same > VM, it's a good idea to restart mnesia with a fresh schema generated. > > 2) erl -run mnesia > > To start mnesia one can also decide to use either "-run" or "-s" > parameters to erl. Disadvantages of this approach are the same as of > the previous one except that in this case you don't have to regenerate > boot scripts every time you upgrade your erlang package (because there > is no any boot scripts =). > > 3) From init/1 function of a gen_server using mnesia. > > You just write something like this to control when mnesia is started: > > init(_) -> > mnesia:start(), > %... > > [-] It seems a good solution, but you can't stop mnesia in terminate/1 > function (mnesia:stop() call hangs). It seems a bit inconsistent to > leave mnesia running after your application has been stopped. > > 4) Start mnesia as included application > > Although how to work with Included Applications is described in OTP > Design Principles I don't see this way being used widely. Maybe > because of the following questions: > > [-] Is it safe to start mnesia as included application by starting > mnesia_sup:start/0 from standard OTP supervsor? > [-] Is it safe to set mnesia options (like dir) using set_env before > your_sup:init() has been returned? > > Although I've noted only disadvantages (it's more difficult for me to > estimate advantages of those ways right now) I try to use them all. > Which one do you prefer and why? > > Thanks. > > -- > Sergey Samokhin > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From vik@REDACTED Mon Jul 6 03:27:04 2009 From: vik@REDACTED (Vik Olliver) Date: Mon, 06 Jul 2009 13:27:04 +1200 Subject: Debugging with toolbar: stopping the spawn Message-ID: <4A5152E8.6060904@catalyst.net.nz> I'm debugging some code - or trying to - with the toolbar debugger. The problem is that I only want to break on the first process to hit a function. Otherwise 3 billion other processes hit the break and flood my screen with debug windows. So, how do I break on the first function break, and only on the first function break? Vik :v) From tony@REDACTED Mon Jul 6 04:03:11 2009 From: tony@REDACTED (Tony Arcieri) Date: Sun, 5 Jul 2009 20:03:11 -0600 Subject: [erlang-questions] Why doesn't erl_eval compile code? In-Reply-To: <3dbc6d1c0907051611s2a7b1aa5m1a391e0bae613c8f@mail.gmail.com> References: <3dbc6d1c0907051611s2a7b1aa5m1a391e0bae613c8f@mail.gmail.com> Message-ID: On Sun, Jul 5, 2009 at 5:11 PM, Robert Virding wrote: > Seeing no one has replied I will give some comments: > > - What problem are you trying to solve? I admit that it would be cool to do > it, but why? Well, specifically I'm curious about this approach for Reia. Right now I've had difficulty keeping the semantics consistent between the compiled and evaluated versions (and this is using erl_eval to handle the evaluation). I figure by using a compiled solution for eval there's a single path all code goes through to get executed. > - While the fun code is not very elegant :-), it does work as it should. Unless you're crazy enough to pass more than 20 arguments :) > - If you remove the module when you are done, which you must explicitly do, > then there could be some problem with funs if you have passed them on and > they still are in use as they will not work after their module has been > removed. Oh right, crap... that's a tricky problem. I can't think of a good solution offhand either. -- Tony Arcieri medioh.com From ruslan@REDACTED Mon Jul 6 09:01:49 2009 From: ruslan@REDACTED (Ruslan Babayev) Date: Mon, 6 Jul 2009 00:01:49 -0700 Subject: erlang-amf Message-ID: <6211282E-2D62-4A6C-9758-E2A13D8D1D9D@babayev.com> I have finally published the Erlang AMF library I have been working on to github. It has full support for Adobe AMF0 and AMF3 specs. Enjoy responsibly. From ruslan@REDACTED Mon Jul 6 09:03:51 2009 From: ruslan@REDACTED (Ruslan) Date: Mon, 6 Jul 2009 00:03:51 -0700 (PDT) Subject: erlang-amf In-Reply-To: <6211282E-2D62-4A6C-9758-E2A13D8D1D9D@babayev.com> References: <6211282E-2D62-4A6C-9758-E2A13D8D1D9D@babayev.com> Message-ID: <3792a5cf-8649-489b-9d78-36800d12e102@g7g2000prg.googlegroups.com> Forgot the actual link http://github.com/mujaheed/erlang-amf/tree/master On Jul 6, 12:01?am, Ruslan Babayev wrote: > I have finally published the Erlang AMF library I have been working on ? > to github. It has full support for Adobe AMF0 and AMF3 specs. > > Enjoy responsibly. From xuxiang1@REDACTED Mon Jul 6 11:13:32 2009 From: xuxiang1@REDACTED (George Xu) Date: Mon, 6 Jul 2009 17:13:32 +0800 Subject: Cannot find the function httpd_util:header as described by documents Message-ID: The erlang documents describes a function named header in the module of http_util (http://erlang.org/doc/man/httpd_util.html) header(StatusCode,PersistentConn) header(StatusCode,Date) header(StatusCode,MimeType,Date) header(StatusCode,MimeType,PersistentConn,Date) -> HTTPHeader when I want to use them to fill my http response header, error accoured: 7> httpd_util:header(403, true). ** exception error: undefined function httpd_util:header/2 I looked the R13B01, R13A and R12B-5's source code and it really do not there!!! Which version of inets has this function? George Fly From raimo+erlang-questions@REDACTED Mon Jul 6 11:48:01 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 6 Jul 2009 11:48:01 +0200 Subject: [erlang-questions] Erlang, OpenBSD and multicore In-Reply-To: <4A4DE7D2.70605@desquilbet.org> References: <4A4DE7D2.70605@desquilbet.org> Message-ID: <20090706094801.GA4401@erix.ericsson.se> On Fri, Jul 03, 2009 at 01:13:22PM +0200, J?r?me Desquilbet wrote: > I am using OpenBSD as an OS on my server, and Erlang as well. I am > considering now to upgrade to an Intel dualcore platform, taking > advantage of OpenBSD support for SMP (see > http://openbsd.org/faq/faq8.html#SMP ). I understand that Erlang won't > automatically detect the dualcore with OpenBSD (as it would with some > Linuxes). But will everything be OK if I "force" the CPU topology with > erlang:system_flag()? Or is there a known supportability issue with OpenBSD? > Thanks, > J?r?me. Well, OpenBSD's SMP support is as far as I know limited to scheduling different processes on different CPU core. A process that uses multiple threads (as Erlang's SMP emulator) gets user threads within one OS process hence one Erlang emulator can only utilize one CPU core. For better SMP on OpenBSD you have to build an RTHREADS-enabled kernel and librthread.so, and link (dynamically) the Erlang emulator against it. This is highly unsuported, not even alpha status, for testing only, and RTHREADS only parallellize disk IO (at least not network IO). I ment to try this but has not got around to it yet. A dualcore platform can still be interesting since on any normal system there is other things for the other CPU core to do, or you can run two Erlang emulators. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From joelr1@REDACTED Mon Jul 6 13:13:04 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 6 Jul 2009 12:13:04 +0100 Subject: comma vs andalso Message-ID: I noticed that inets code uses foo(...) when Condition1 andalso Condition2 -> ... Is the comma deprecated? For example, why not foo(...) when Condition1, Condition2 -> ... A related pet peeve of mine is that the following does not properly indent in the latest Emacs mode handle_info({tcp, Socket, Bin}, State) when State#state.transport /= undefined, State#state.socket == Socket -> ... I have to manually insert spaces to align State#state... above and using andalso above does not help either. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From magnus@REDACTED Mon Jul 6 13:58:57 2009 From: magnus@REDACTED (Magnus Henoch) Date: Mon, 06 Jul 2009 12:58:57 +0100 Subject: comma vs andalso In-Reply-To: (Joel Reymont's message of "Mon, 6 Jul 2009 12:13:04 +0100") References: Message-ID: <841voudnum.fsf@linux-b2a3.site> Joel Reymont writes: > A related pet peeve of mine is that the following does not properly > indent in the latest Emacs mode > > handle_info({tcp, Socket, Bin}, State) > when State#state.transport /= undefined, > State#state.socket == Socket -> > ... > > I have to manually insert spaces to align State#state... above and > using andalso above does not help either. I found that this happens because there is a "." in the guard. This change to erlang-mode (actually erlware-mode, but the code should be similar) seems to fix it: diff --git a/erlang.el b/erlang.el index 6ad03ea..916037e 100644 --- a/erlang.el +++ b/erlang.el @@ -2097,7 +2097,7 @@ Value is list (stack token-start token-type in-what)." ((looking-at "when[^->\.]*case[^->\.]*->")) ((looking-at "when[^->\.]*receive[^->\.]*->")) ;; Normal when case - ((looking-at "when [^->\.]*->") + ((looking-at "when [^->]*->") (erlang-push (list 'when token (current-column)) stack)) ((looking-at "after[.]+->") (erlang-push (list 'icr token (current-column)) stack)) Not sure what other cases that would mess up, though. The comment above mentions one possible gotcha: ;; In test suites you may want to do something like ;; ?match(Mem when integer(Mem), mnesia:table_info(Tab, ;; memory)), and then the following if/case/receive ;; statement will mess up the indentation by fooling the ;; erlang mode to think the 'when' in the argument is a ;; "real" when. The following three clauses will avoid ;; this problem. -- Magnus Henoch, magnus@REDACTED Erlang Training and Consulting http://www.erlang-consulting.com/ From roberto@REDACTED Mon Jul 6 14:19:59 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 6 Jul 2009 14:19:59 +0200 Subject: [erlang-questions] comma vs andalso In-Reply-To: References: Message-ID: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> On 06/lug/09, at 13:13, Joel Reymont wrote: > I noticed that inets code uses > > foo(...) when Condition1 andalso Condition2 -> ... > > Is the comma deprecated? For example, why not > > foo(...) when Condition1, Condition2 -> ... the difference is very simple. comma indicates that both guard operators need to be evaluated, while on the contrary 'andalso' means that the second condition is evaluated ONLY if condition1 is true. same goes for 'orselse'. this is particularly relevant when condition2 would result for instance in an evaluation error if condition1 is not met, as in: foo(..) when (islist(X) == true ansalso (lists:nth(1,X) == 'test') -> .. hope this clears. r. From roberto@REDACTED Mon Jul 6 14:21:46 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 6 Jul 2009 14:21:46 +0200 Subject: [erlang-questions] comma vs andalso In-Reply-To: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> Message-ID: <88CA9E66-E138-416C-926C-D2B9AD0C2AA0@widetag.com> On 06/lug/09, at 14:19, Roberto Ostinelli wrote: > foo(..) when (islist(X) == true ansalso (lists:nth(1,X) == 'test') - > > .. that would be foo(..) when (islist(X) =:= true) ansalso (lists:nth(1,X) =:= 'test') - > .. pardon me i was in python mode. r. From whongo@REDACTED Mon Jul 6 15:07:56 2009 From: whongo@REDACTED (=?ISO-8859-1?Q?Martin_Engstr=F6m?=) Date: Mon, 6 Jul 2009 15:07:56 +0200 Subject: [erlang-questions] comma vs andalso In-Reply-To: <88CA9E66-E138-416C-926C-D2B9AD0C2AA0@widetag.com> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <88CA9E66-E138-416C-926C-D2B9AD0C2AA0@widetag.com> Message-ID: <28423d490907060607x1f3a12d9mc06025b1f44ea6a9@mail.gmail.com> On Mon, Jul 6, 2009 at 2:21 PM, Roberto Ostinelli wrote: > > On 06/lug/09, at 14:19, Roberto Ostinelli wrote: > > foo(..) when (islist(X) == true ansalso (lists:nth(1,X) == 'test') -> .. >> > > that would be > > foo(..) when (islist(X) =:= true) ansalso (lists:nth(1,X) =:= 'test') -> .. > '==' and '=:=' are equivalent except when applied to numbers, are they not? (1 == 1.0) is true, while (1 =:= 1.0) is false. When applied to atoms, as in your example, there is no difference. And the difference between comma and 'andalso' doesn't actually matter much for the purpose of avoiding evaluation errors in guard context, since such an error in a guard just means that the guard fails. For instance, try this: 1> length(a). ** exception error: bad argument in function length/1 called as length(a) 2> Foo = fun(X) when length(X) > 3 -> "Long list"; (X) -> "Short or no list" end. 3> Foo(a). "Short or no list" Also, lists:nth(X) is an illegal guard expression. ;) From roberto@REDACTED Mon Jul 6 15:09:52 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 6 Jul 2009 15:09:52 +0200 Subject: [erlang-questions] comma vs andalso In-Reply-To: <4d08db370907060600t134957d2ld5179b8416a7cb00@mail.gmail.com> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <88CA9E66-E138-416C-926C-D2B9AD0C2AA0@widetag.com> <4d08db370907060600t134957d2ld5179b8416a7cb00@mail.gmail.com> Message-ID: On 06/lug/09, at 15:00, Hynek Vychodil wrote: > > It seems you are still in python mode. lists:nth/2 is not allowed in > guard expression. you are definitely right :) let me point another (and correct) example then: foo(..) when X =/= 0 andalso 1/X > 0.1 -> .. the following expression would fail instead: foo(..) when X =/= 0, 1/X > 0.1 -> .. r. From richardc@REDACTED Mon Jul 6 15:10:16 2009 From: richardc@REDACTED (Richard Carlsson) Date: Mon, 06 Jul 2009 16:10:16 +0300 Subject: [erlang-questions] comma vs andalso In-Reply-To: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> Message-ID: <4A51F7B8.2020004@it.uu.se> Roberto Ostinelli wrote: > the difference is very simple. comma indicates that both guard operators > need to be evaluated, while on the contrary 'andalso' means that the > second condition is evaluated ONLY if condition1 is true. same goes for > 'orselse'. No, the comma separator in guards also uses short-cut evaluation and will not evaluate the subsequent tests if one test fails. If there are alternatives expressed with semicolon, the next such alternative will be tried instead; otherwise, the whole guard fails. What you describe is true for the 'and' operator compared to 'andalso'. /Richard From roberto@REDACTED Mon Jul 6 15:25:50 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 6 Jul 2009 15:25:50 +0200 Subject: [erlang-questions] comma vs andalso In-Reply-To: <4d08db370907060621k63e24967wbb7cff51ce15770d@mail.gmail.com> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <88CA9E66-E138-416C-926C-D2B9AD0C2AA0@widetag.com> <4d08db370907060600t134957d2ld5179b8416a7cb00@mail.gmail.com> <4d08db370907060621k63e24967wbb7cff51ce15770d@mail.gmail.com> Message-ID: <3AC9B53E-294B-4FB5-8A8C-4F46B652C2B9@widetag.com> On 06/lug/09, at 15:21, Hynek Vychodil wrote: > It seems no difference for me: > > 1> F = fun(X) when X=/=0, 1/X > 0.1 -> true; (_) -> false end. > #Fun > 2> F2 = fun(X) when X=/=0 andalso 1/X > 0.1 -> true; (_) -> false end. > #Fun > 3> [ F(X) || X <- [0, 0.0, 0.1, 10, 9]]. > [false,false,true,false,true] > 4> [ F2(X) || X <- [0, 0.0, 0.1, 10, 9]]. > [false,false,true,false,true] > > -- > --Hynek (Pichi) Vychodil yes, at it has been pointed out by richard already: On 06/lug/09, at 15:10, Richard Carlsson wrote: > and will not evaluate the subsequent tests if one test fails. If > there are alternatives expressed with semicolon, the next such > alternative will be tried instead; otherwise, the whole guard fails. > > What you describe is true for the 'and' operator compared to > 'andalso'. my fault. as you can see: 1> F3 = fun(X) when X=/=0 and (1/X > 0.1) -> true; (_) -> false end. #Fun 2> [ F3(X) || X <- [0, 0.0, 0.1, 10, 9]]. [false,false,false,false,false] r. From richardc@REDACTED Mon Jul 6 15:41:15 2009 From: richardc@REDACTED (Richard Carlsson) Date: Mon, 06 Jul 2009 16:41:15 +0300 Subject: [erlang-questions] comma vs andalso In-Reply-To: References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <88CA9E66-E138-416C-926C-D2B9AD0C2AA0@widetag.com> <4d08db370907060600t134957d2ld5179b8416a7cb00@mail.gmail.com> Message-ID: <4A51FEFB.2090904@it.uu.se> Roberto Ostinelli wrote: > let me point another (and correct) example then: > > foo(..) when X =/= 0 andalso 1/X > 0.1 -> .. > > the following expression would fail instead: > > foo(..) when X =/= 0, 1/X > 0.1 -> .. Aye, but an exception in a guard test is always caught and treated as if the test evaluated to "false". In this example, there is no observable difference between the two. /Richard From whongo@REDACTED Mon Jul 6 15:52:14 2009 From: whongo@REDACTED (whongo@REDACTED) Date: Mon, 6 Jul 2009 06:52:14 -0700 (PDT) Subject: comma vs andalso In-Reply-To: <3AC9B53E-294B-4FB5-8A8C-4F46B652C2B9@widetag.com> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <88CA9E66-E138-416C-926C-D2B9AD0C2AA0@widetag.com> <4d08db370907060600t134957d2ld5179b8416a7cb00@mail.gmail.com> <4d08db370907060621k63e24967wbb7cff51ce15770d@mail.gmail.com> <3AC9B53E-294B-4FB5-8A8C-4F46B652C2B9@widetag.com> Message-ID: <62c65ab8-d0a6-4ed6-9462-3ec16c0a80dd@18g2000yqa.googlegroups.com> On Jul 6, 3:25?pm, Roberto Ostinelli wrote: > > 1> F3 = fun(X) when X=/=0 and (1/X > 0.1) -> true; (_) -> false end. > #Fun > 2> [ F3(X) || X <- [0, 0.0, 0.1, 10, 9]]. > [false,false,false,false,false] Ah, but that's because the expression (0 and (1/X > 0.1)) is bad. (I've learned to be very afraid of how strongly 'and' binds. ;)) By contrast, 25> F4 = fun(X) when (X =/= 0) and (1/X > 0.1) -> true; (_) -> false end. #Fun 26> [ F4(X) || X <- [0, 0.0, 0.1, 10, 9]]. [false,false,true,false,true] And to iterate my previous point, 28> F5 = fun(X) when 1/X > 0.1 -> true; (_) -> false end. #Fun 29> [F5(X) || X <- [0, 0.0, 0.1, 10, 9]]. [false,false,true,false,true] From roberto@REDACTED Mon Jul 6 15:58:13 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 6 Jul 2009 15:58:13 +0200 Subject: [erlang-questions] Re: comma vs andalso In-Reply-To: <62c65ab8-d0a6-4ed6-9462-3ec16c0a80dd@18g2000yqa.googlegroups.com> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <88CA9E66-E138-416C-926C-D2B9AD0C2AA0@widetag.com> <4d08db370907060600t134957d2ld5179b8416a7cb00@mail.gmail.com> <4d08db370907060621k63e24967wbb7cff51ce15770d@mail.gmail.com> <3AC9B53E-294B-4FB5-8A8C-4F46B652C2B9@widetag.com> <62c65ab8-d0a6-4ed6-9462-3ec16c0a80dd@18g2000yqa.googlegroups.com> Message-ID: <66C6FFC8-761E-4330-8F1E-FADDB811BFC2@widetag.com> On 06/lug/09, at 15:52, whongo@REDACTED wrote: > > Ah, but that's because the expression (0 and (1/X > 0.1)) is bad. > (I've learned to be very afraid of how strongly 'and' binds. ;)) definitely interesting. i guess you've learned it the hard way :) r. From devdoer2@REDACTED Mon Jul 6 17:35:18 2009 From: devdoer2@REDACTED (devdoer bird) Date: Mon, 6 Jul 2009 23:35:18 +0800 Subject: Will read/write failed while I do fragment in mnesia? Message-ID: Will read / write failed while I do fragment in mnesia? Eg. when a record is moved to a new fragmet ,the early read/write operaton will failed to find the record From joelr1@REDACTED Mon Jul 6 18:20:30 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 6 Jul 2009 17:20:30 +0100 Subject: tcp_close during async accept Message-ID: I'm playing with Serge's "Non-blocking TCP server using OTP principles" [1]. When spawning a large number of connections almost at once, I'm seeing tcp_close messages on the client end, even before seeing inet_async on the client end. I may spawn 700 client connections and only half may go through, even though my backlog option is 1024, e.g. Opts = [binary, {packet, 0}, {reuseaddr, true}, {keepalive, true}, {backlog, 1024}, {active, false}], case gen_tcp:listen(Port, Opts) of {ok, Socket} -> %% Create first accepting process {ok, Ref} = prim_inet:async_accept(Socket, -1), ... Any suggestions on what's going on? Is it kosher to use prim_inet:async_accept/2? Am I using backlog correctly? Thanks, Joel [1] http://www.trapexit.org/Building_a_Non-blocking_TCP_server_using_OTP_principles --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From cbenac@REDACTED Mon Jul 6 21:30:20 2009 From: cbenac@REDACTED (Clara Benac Earle) Date: Mon, 06 Jul 2009 21:30:20 +0200 Subject: Erlang workshop in Edinburgh September 5th Message-ID: <4A5250CC.7010903@fi.upm.es> Please check the Erlang workshop program at http://erlang.org/workshop/2009/index.html The deadline for the early registration is the 30th of July http://www.regmaster.com/conf/icfp2009.html As usual we will meet after the workshop for some food and beer. We are looking forward to seeing you at Edinburgh!! Clara From joelr1@REDACTED Mon Jul 6 22:30:59 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 6 Jul 2009 21:30:59 +0100 Subject: kpoll still using select Message-ID: Any suggestions? Thanks in advance, Joel --- Mac OSX 10.5.7 ulimit -n 10240 Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:true] =ERROR REPORT==== 6-Jul-2009::20:51:54 === driver_select(0x00000041, 1024, ERL_DRV_WRITE ERL_DRV_USE, 1) by tcp_inet driver #Port<0.3137> failed: fd=1024 is larger than the largest allowed fd=1023 =ERROR REPORT==== 6-Jul-2009::20:51:55 === File operation error: system_limit. Target: s3erl/ebin/random.beam. Function: get_file. Process: code_server. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From tony@REDACTED Tue Jul 7 01:53:56 2009 From: tony@REDACTED (Tony Arcieri) Date: Mon, 6 Jul 2009 17:53:56 -0600 Subject: [erlang-questions] Why doesn't erl_eval compile code? In-Reply-To: <3dbc6d1c0907051611s2a7b1aa5m1a391e0bae613c8f@mail.gmail.com> References: <3dbc6d1c0907051611s2a7b1aa5m1a391e0bae613c8f@mail.gmail.com> Message-ID: On Sun, Jul 5, 2009 at 5:11 PM, Robert Virding wrote: > - If you remove the module when you are done, which you must explicitly do, > then there could be some problem with funs if you have passed them on and > they still are in use as they will not work after their module has been > removed. So here's a question... if I code:soft_purge a module which was used to create funs which are still in scope in some process somewhere, will that call to soft_purge fail with false? If so, as a hack I could have a "garbage collector" process which tries to soft purge the temporary modules periodically until it succeeds. This is a far uglier solution that I thought it would be when I started out, though :( Would it work though? -- Tony Arcieri medioh.com From eliliang@REDACTED Tue Jul 7 00:55:06 2009 From: eliliang@REDACTED (Eli Liang) Date: Mon, 6 Jul 2009 15:55:06 -0700 (PDT) Subject: Erlang made the news (though not in a nice way) In-Reply-To: References: Message-ID: <207314.96967.qm@web30406.mail.mud.yahoo.com> http://seekingalpha.com/article/147104-is-a-case-of-quant-trading-sabotage-about-to-destroy-goldman-sachs From erlangy@REDACTED Tue Jul 7 02:38:36 2009 From: erlangy@REDACTED (Michael McDaniel) Date: Mon, 6 Jul 2009 17:38:36 -0700 Subject: [question] timed_supervisor, extended timer module Message-ID: <20090707003836.GB7036@delora.autosys.us> Two questions for Serge Aleynikov , 1) does the extended timer module work you did supersede any updates to timed_supervisor.erl ? 2) where can I download latest of whichever is most updated ? thanks, and please advise, ~Michael From ok@REDACTED Tue Jul 7 02:47:23 2009 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 7 Jul 2009 12:47:23 +1200 Subject: [erlang-questions] comma vs andalso In-Reply-To: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> Message-ID: <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> On Jul 7, 2009, at 12:19 AM, Roberto Ostinelli wrote: > the difference is very simple. comma indicates that both guard > operators need to be evaluated, while on the contrary 'andalso' > means that the second condition is evaluated ONLY if condition1 is > true. same goes for 'orselse'. He's wrong, but he's not _crazy_. Section 6.24 of the reference manual _ought_ to explain this clearly, but doesn't. -------------------------- 6.24 Guard Sequences A guard sequence is a sequence of guards, separated by semicolon (;). The guard sequence is true if at least one of the guards is true. (The remaining guards, if any, will not be evaluated.) Guard1;...;GuardK A guard is a sequence of guard expressions, separated by comma (,). The guard is true if all guard expressions evaluate to true. GuardExpr1,...,GuardExprN -------------------------- I propose rewriting these paragraphs something like this: A guard sequence is a sequence (G1;...;Gk) of guards separated by semicolons. Guards are tested from left to right. If some guard succeeds, the guard sequence as a whole succeeds, and the remaining guards will not be checked. If all guards fail, the guard sequence as a whole fails. A guard is a sequence (GE1,...,GEn) of guard expressions separated by commas. Guard expressions are tested from left to right. If some guard expression fails, the guard as a whole fails, and the remaining guard expressions will not be checked. If all guard expressions succeed, the guard as a whole suceeds. A guard expression is a very special restricted expression. It is said to succeed it it evaluates to true. It is said to fail if it evaluates to anything else or raises an exceptions. Exceptions never propagate out of guard expressions, they are just taken as lack of success. (then continue with the existing third paragraph) From ryan.pratt@REDACTED Tue Jul 7 05:20:32 2009 From: ryan.pratt@REDACTED (Ryan Pratt) Date: Mon, 6 Jul 2009 22:20:32 -0500 Subject: webappmon Message-ID: I am noticing what appears to be a bug (?) in webappmon and I am hoping someone on the list has some insight. When you run it against an app that has many linked processes (like 100000+) it seems to lock up access to remote nodes, and sometimes completely hangs the VM. The first symptom of the problem that I noticed is that one of my processes that is monitoring a remote node (actually a JInterface node) got a nodedown message even though the remote node was not down. This got me looking into how monitor_node works but I decided to give up and ask when I got into the BIF code in dist.c So, besides the webappmon issue, does anyone know how monitor_node works for remote nodes? Does it go through rex or some other common process? Webappmon appears to use rpc so if monitor_node was also doing so then I guess that would explain why I would get the nodedown message. Any ideas? Thanks Ryan From andrew@REDACTED Tue Jul 7 06:16:23 2009 From: andrew@REDACTED (Andrew Thompson) Date: Tue, 7 Jul 2009 00:16:23 -0400 Subject: [erlang-questions] comma vs andalso In-Reply-To: <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> Message-ID: <20090707041622.GB28401@hijacked.us> On Tue, Jul 07, 2009 at 12:47:23PM +1200, Richard O'Keefe wrote: > > I propose rewriting these paragraphs something like this: I found your explanation a *lot* clearer and even learned something reading it, so I'd definitely agree that it's an improvement. I'm still unclear as to where 'andalso' differs from a comma though, how is true, false. any different from true andalso false. Andrew From matt@REDACTED Tue Jul 7 07:31:13 2009 From: matt@REDACTED (Matt Reynolds) Date: Mon, 6 Jul 2009 22:31:13 -0700 Subject: Why doesn't ssh module start crypto module? Message-ID: Am I misunderstanding the use of the "applications" atom in the Applications configuration (http://erlang.org/doc/design_principles/applications.html)? Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) 1> application:start(ssh). {error,{not_started,crypto}} Does the "applications" configuration simply specify what must be started first, without starting those modules? So the Application specifying dependencies must start it itself? If so, why doesn't SSH start crypto? From roberto@REDACTED Tue Jul 7 08:09:17 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 7 Jul 2009 08:09:17 +0200 Subject: [erlang-questions] comma vs andalso In-Reply-To: <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> Message-ID: On 07/lug/2009, at 02.47, "Richard O'Keefe" wrote: > > On Jul 7, 2009, at 12:19 AM, Roberto Ostinelli wrote: >> the difference is very simple. comma indicates that both guard >> operators need to be evaluated, while on the contrary 'andalso' >> means that the second condition is evaluated ONLY if condition1 is >> true. same goes for 'orselse'. > > He's wrong, but he's not _crazy_. > Section 6.24 of the reference manual _ought_ to explain this > clearly, but doesn't. that is the paragraph i'd been referring to and apparently i had been interpreting it wrongly. i thought this had already been cleared yesterday and yes, an editing as proposed would be a good thing, i might not be the only one doing so. r. From joelr1@REDACTED Tue Jul 7 10:52:59 2009 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 7 Jul 2009 09:52:59 +0100 Subject: [erlang-questions] Erlang made the news (though not in a nice way) In-Reply-To: <207314.96967.qm@web30406.mail.mud.yahoo.com> References: <207314.96967.qm@web30406.mail.mud.yahoo.com> Message-ID: <5B3D3E85-4072-43E6-BB87-CF11E6010935@gmail.com> On Jul 6, 2009, at 11:55 PM, Eli Liang wrote: > http://seekingalpha.com/article/147104-is-a-case-of-quant-trading-sabotage-about-to-destroy-goldman-sachs I interviewed with Serge after he left Goldman, just a short while ago. Decided to stay put in Tenerife rather than work for the trading startup in Chicago. The official complaint is here: http://www.scribd.com/doc/17118166/Complaint-Aleynikov Also, Bloomberg has a much better story: http://www.bloomberg.com/apps/news?pid=20601103&sid=axYw_ykTBokE --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From btolputt@REDACTED Tue Jul 7 06:58:48 2009 From: btolputt@REDACTED (Benjamin Tolputt) Date: Tue, 07 Jul 2009 14:58:48 +1000 Subject: [erlang-questions] Erlang made the news (though not in a nice way) In-Reply-To: <207314.96967.qm@web30406.mail.mud.yahoo.com> References: <207314.96967.qm@web30406.mail.mud.yahoo.com> Message-ID: <4A52D608.2060208@bigpond.net.au> Eli Liang wrote: > http://seekingalpha.com/article/147104-is-a-case-of-quant-trading-sabotage-about-to-destroy-goldman-sachs > I wouldn't say that the news is all bad. As far as Erlang/OTP is concerned, it is part of a vital distributed, high-speed, high-reliability, high-income producing system put together for one of the world's leading financial firms. The fact it is buried in a story regarding regarding corporate sabotage/theft shouldn't take away from the best part of the story for Erlang. That is, a system put together using Erlang/OTP (amongst other technologies) is so critical to the "low latency trading" operations of Goldman Sachs that when it was shut down (for legal, not technical reasons) - it removed the top trading financial form from first place in the NYSE reports to non-existent! What better reference in the financial industry could you ask for? -- Regards, Benjamin Tolputt Analyst Programmer From Woolla_T@REDACTED Tue Jul 7 13:10:37 2009 From: Woolla_T@REDACTED (Trevor Woollacott [ MTN - Innovation Centre ]) Date: Tue, 7 Jul 2009 13:10:37 +0200 Subject: [erlang-questions] tcp_close during async accept In-Reply-To: References: Message-ID: What OS are you using? Maybe the maximum backlog on your platform is less than 1024? Regards, Trevor > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] > On Behalf Of Joel Reymont > Sent: Monday, 06 July 2009 06:21 PM > To: Erlang Users' List > Subject: [erlang-questions] tcp_close during async accept > > I'm playing with Serge's "Non-blocking TCP server using OTP > principles" [1]. When spawning a large number of connections almost at > once, I'm seeing tcp_close messages on the client end, even before > seeing inet_async on the client end. > > I may spawn 700 client connections and only half may go through, even > though my backlog option is 1024, e.g. > > Opts = [binary, > {packet, 0}, > {reuseaddr, true}, > {keepalive, true}, > {backlog, 1024}, > {active, false}], > case gen_tcp:listen(Port, Opts) of > {ok, Socket} -> > %% Create first accepting process > {ok, Ref} = prim_inet:async_accept(Socket, -1), > ... > > Any suggestions on what's going on? > > Is it kosher to use prim_inet:async_accept/2? > > Am I using backlog correctly? > > Thanks, Joel > > [1] http://www.trapexit.org/Building_a_Non- > blocking_TCP_server_using_OTP_principles > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org NOTE: This e-mail message is subject to the MTN Group disclaimer see http://www.mtn.co.za/SUPPORT/LEGAL/Pages/EmailDisclaimer.aspx From jacek99@REDACTED Tue Jul 7 14:48:47 2009 From: jacek99@REDACTED (Jacek Furmankiewicz) Date: Tue, 7 Jul 2009 08:48:47 -0400 Subject: Replicated Mnesia table on dynamic nodes - how? Message-ID: <269a5c30907070548k7dd9722fu3edf86f4f699968d@mail.gmail.com> When you create a Mnesia table, you are supposed to pass it a list of nodes that it should be replicated on. Since this a creation-time element, how do you handle nodes dynamically being added later on? Is there a way to specify a Mnesia table with some sort of cookie parameter, so all nodes with the same cookie will automatically replicate the disk table? Thanks, Jacek From nick@REDACTED Tue Jul 7 15:05:35 2009 From: nick@REDACTED (Niclas Eklund) Date: Tue, 7 Jul 2009 15:05:35 +0200 (CEST) Subject: [erlang-questions] Why doesn't ssh module start crypto module? In-Reply-To: References: Message-ID: Hello! The 'applications' value shall list other applications that the application depends on. If the required applicatios hans't been started an error message is returned. Why doesn't SSH start crypto? For example might other applications also depend on crypto and the required type (permanent | transient | temporary) may differ. Another reason is that many applications export specific start functions (Mnesia, SSL, Orber etc) which in some cases require configuration parameters. Typically these functions do more stuff than just invoking application:start/1/2. Hence, you should use ssh:start/1/2 instead. Also consider that if application A depends on that B is started. If B is updated and require that application C i started prior to B. In that case A must be patched so that it starts not only B but also C. I hope you see my point why SSH shall not start Crypto ;-) The short version, you must start the other applications in your code. Best Regards, Niclas @ Erlang/OTP On Mon, 6 Jul 2009, Matt Reynolds wrote: > Am I misunderstanding the use of the "applications" atom in the > Applications configuration > (http://erlang.org/doc/design_principles/applications.html)? > > Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] > [kernel-poll:false] > > Eshell V5.7.2 (abort with ^G) > 1> application:start(ssh). > {error,{not_started,crypto}} > > Does the "applications" configuration simply specify what must be > started first, without starting those modules? So the Application > specifying dependencies must start it itself? If so, why doesn't SSH > start crypto? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From rtrlists@REDACTED Tue Jul 7 15:39:52 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 7 Jul 2009 14:39:52 +0100 Subject: [erlang-questions] Why doesn't ssh module start crypto module? In-Reply-To: References: Message-ID: <6a3ae47e0907070639i2566450fnb3c7597fa8094306@mail.gmail.com> Also note that if you build an OTP release, then the boot script usually takes care of starting all your applications in the right order. If you're not using releases, then you need to do that yourself. Robby On Tue, Jul 7, 2009 at 2:05 PM, Niclas Eklund wrote: > > Hello! > > The 'applications' value shall list other applications that the application > depends on. If the required applicatios hans't been started an error message > is returned. > > Why doesn't SSH start crypto? For example might other applications also > depend on crypto and the required type (permanent | transient | temporary) > may differ. Another reason is that many applications export specific start > functions (Mnesia, SSL, Orber etc) which in some cases require configuration > parameters. Typically these functions do more stuff than just invoking > application:start/1/2. Hence, you should use ssh:start/1/2 instead. > > Also consider that if application A depends on that B is started. If B is > updated and require that application C i started prior to B. In that case A > must be patched so that it starts not only B but also C. I hope you see my > point why SSH shall not start Crypto ;-) > > The short version, you must start the other applications in your code. > > Best Regards, > > Niclas @ Erlang/OTP > > > > On Mon, 6 Jul 2009, Matt Reynolds wrote: > > Am I misunderstanding the use of the "applications" atom in the >> Applications configuration >> (http://erlang.org/doc/design_principles/applications.html)? >> >> Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] >> [kernel-poll:false] >> >> Eshell V5.7.2 (abort with ^G) >> 1> application:start(ssh). >> {error,{not_started,crypto}} >> >> Does the "applications" configuration simply specify what must be >> started first, without starting those modules? So the Application >> specifying dependencies must start it itself? If so, why doesn't SSH >> start crypto? >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From thanos.vassilakis@REDACTED Tue Jul 7 16:04:19 2009 From: thanos.vassilakis@REDACTED (Vassilakis, Thanos) Date: Tue, 7 Jul 2009 10:04:19 -0400 Subject: [erlang-questions] distributed system: same app, different params In-Reply-To: References: <4A39E84B.1010408@erlang-consulting.com> <087E0400-7545-49FC-8D0A-CD888DF15796@widetag.com> <4A3FA6F9.1060306@erlang-consulting.com> <9a09ca9a0906221604o56fd4c80sfe55e1db8320863@mail.gmail.com> Message-ID: <05F3D38D836B6248BF7452CA10036F500412ECDF@EPRI17P32003B.csfb.cs-group.com> I use a pylons web system that runs with Fabric (http://docs.fabfile.org/0.9/) to manage our many wonderful releases ! -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Roberto Ostinelli Sent: Wednesday, July 01, 2009 12:17 PM To: Kunthar Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] distributed system: same app, different params ulf and kunthar, i'm currently using scripts via python, but i wished there was a more natural way for erlang to keep track of running procs. i thought otp would take care of this. thank you :) r. 2009/6/23 Kunthar > I don't know if those fit to your needs but here are two options; > > Check pssh, fancy pink background :) > http://www.theether.org/pssh/ > > Check capistrano, Ruby guys holiday inn. > http://www.capify.org/ > > > Peace > \|/ Kunth > > > On Mon, Jun 22, 2009 at 6:44 PM, Ulf > Wiger wrote: > > Roberto Ostinelli wrote: > >> > >> hi, can i ask for a little clarification more: what would you use > >> to deploy this script on many different machines? mechanisms such > >> as bash script managers over the network, application_controller, ...? > > > > I guess this is a bit application-specific, but one may e.g. > > distribute a global config document, and let each machine find its > > own particulars by matching on the host name. It can then complement > > the global config with data found through local discovery. > > > > A bash or python script might be the best tool for this kind of > > thing, but escript could of course also work. > > > > Another fairly nice approach, if you're running within an Erlang > > environment already, is file:script(File). It's like file:consult/1, > > but returns the value of the last expression. > > This function actually came from the 'builder' project, where I > > generated app files, config, boot and script files through a > > combination of preset values and local discovery. > > > > BR, > > Ulf W > > -- > > Ulf Wiger > > CTO, Erlang Training & Consulting Ltd > > http://www.erlang-consulting.com > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- ---------------------------------------------------------- Roberto Ostinelli CTO, WideTag Inc. - Towards an Internet Of Things widetag.com openspime.com skype rostinelli mobile +39 335 6 100 22 6 =============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html =============================================================================== From joelr1@REDACTED Tue Jul 7 16:30:55 2009 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 7 Jul 2009 15:30:55 +0100 Subject: [erlang-questions] tcp_close during async accept In-Reply-To: References: Message-ID: <4EEFAA5F-00D9-45FE-A1D0-A278A94F142D@gmail.com> On Jul 7, 2009, at 12:10 PM, Trevor Woollacott [ MTN - Innovation Centre ] wrote: > What OS are you using? Maybe the maximum backlog on your platform is > less than 1024? Mac OSX 10.5.7 --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From paul-trapexit@REDACTED Tue Jul 7 18:48:34 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Tue, 7 Jul 2009 09:48:34 -0700 (PDT) Subject: [erlang-questions] Replicated Mnesia table on dynamic nodes - how? In-Reply-To: <269a5c30907070548k7dd9722fu3edf86f4f699968d@mail.gmail.com> References: <269a5c30907070548k7dd9722fu3edf86f4f699968d@mail.gmail.com> Message-ID: to add nodes to the schema: mnesia:change_config/2 and mnesia:change_table_copy_type/3 to add a table to a node: mnesia:add_table_copy/3 -- p On Tue, 7 Jul 2009, Jacek Furmankiewicz wrote: > When you create a Mnesia table, you are supposed to pass it a list of nodes > that it should be replicated on. > Since this a creation-time element, how do you handle nodes dynamically > being added later on? > Is there a way to specify a Mnesia table with some sort of cookie parameter, > so all nodes with the same cookie will automatically replicate the disk > table? > > Thanks, > Jacek > From jacek99@REDACTED Tue Jul 7 19:46:32 2009 From: jacek99@REDACTED (Jacek Furmankiewicz) Date: Tue, 7 Jul 2009 13:46:32 -0400 Subject: [erlang-questions] Replicated Mnesia table on dynamic nodes - how? In-Reply-To: References: <269a5c30907070548k7dd9722fu3edf86f4f699968d@mail.gmail.com> Message-ID: <269a5c30907071046j6c3f5667q7df9cd16f0d76918@mail.gmail.com> OK, so how would you write common code to make sure a node creates the initial schema (if it's the first one) or adds itself to an existing schema (if another node has already created one). I had this type of logic to create a schema on startup: * case mnesia:create_schema([node()]) of {atomic,ok} -> ok; {error,{_,{already_exists,_}}} -> ok; Error -> error_logger:error_msg("Failed to create Mnesia schema: ~p.~nStopping application.~n",[Error]), init:stop(1) end,* and then tried something like this for a new node to attach itself: * case mnesia:change_config(extra_db_nodes,node()) of {ok,_Nodes} -> ok; {error, Reason} -> error_logger:error_msg("Failed to connect to Mnesia schema: ~p.~nStopping application.~n",[Reason]), init:stop(1) end,* Our app does not have a single "master" node, all nodes are equal. The system should basically auto-configure Mnesia (create schema by first node that gets created, attach itself to an existing schema for all other nodes). I'm not quite sure how to write this single code that all nodes could use to reliably fulfill this requirement... Cheers, Jacek From paul-trapexit@REDACTED Tue Jul 7 20:00:20 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Tue, 7 Jul 2009 11:00:20 -0700 (PDT) Subject: [erlang-questions] Replicated Mnesia table on dynamic nodes - how? In-Reply-To: <269a5c30907071046j6c3f5667q7df9cd16f0d76918@mail.gmail.com> References: <269a5c30907070548k7dd9722fu3edf86f4f699968d@mail.gmail.com> <269a5c30907071046j6c3f5667q7df9cd16f0d76918@mail.gmail.com> Message-ID: http://erlanganswers.com/web/mcedemo/mnesia/DistributedHelloWorld.html -- p p.z. Sounds like you want http://code.google.com/p/schemafinder/ On Tue, 7 Jul 2009, Jacek Furmankiewicz wrote: > OK, so how would you write common code to make sure a node creates the > initial schema (if it's the first one) or adds itself to an existing schema > (if another node has already created one). > > I had this type of logic to create a schema on startup: > * > case mnesia:create_schema([node()]) of > {atomic,ok} -> ok; > {error,{_,{already_exists,_}}} -> ok; > Error -> > error_logger:error_msg("Failed to create Mnesia schema: > ~p.~nStopping application.~n",[Error]), > init:stop(1) > end,* > > and then tried something like this for a new node to attach itself: > * > case mnesia:change_config(extra_db_nodes,node()) of > {ok,_Nodes} -> ok; > {error, Reason} -> > error_logger:error_msg("Failed to connect to Mnesia schema: > ~p.~nStopping application.~n",[Reason]), > init:stop(1) > end,* > > Our app does not have a single "master" node, all nodes are equal. The > system should basically auto-configure Mnesia (create schema by first node > that gets created, attach itself to an existing schema for all other nodes). > I'm not quite sure how to write this single code that all nodes could use to > reliably fulfill this requirement... > > Cheers, Jacek > From jacek99@REDACTED Tue Jul 7 20:05:42 2009 From: jacek99@REDACTED (Jacek Furmankiewicz) Date: Tue, 7 Jul 2009 14:05:42 -0400 Subject: [erlang-questions] Replicated Mnesia table on dynamic nodes - how? In-Reply-To: References: <269a5c30907070548k7dd9722fu3edf86f4f699968d@mail.gmail.com> <269a5c30907071046j6c3f5667q7df9cd16f0d76918@mail.gmail.com> Message-ID: <269a5c30907071105t625fbd04na13002cafdcc0ded@mail.gmail.com> Thank you Paul, I will investigate those further. Jacek From paul-trapexit@REDACTED Tue Jul 7 20:06:05 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Tue, 7 Jul 2009 11:06:05 -0700 (PDT) Subject: type specs and mnesia selects Message-ID: folks, i was being a good citizen and adding type specifications to my records. this caused dialyzer to complain because in my mnesia:select/2,3,4 calls I am binding a member of the record to a variable (which is an atom like '$1') and this was violating the type spec. are there any workarounds, other than augmenting or dropping the type spec? -- p From jacek99@REDACTED Tue Jul 7 21:15:05 2009 From: jacek99@REDACTED (Jacek Furmankiewicz) Date: Tue, 7 Jul 2009 15:15:05 -0400 Subject: [erlang-questions] Replicated Mnesia table on dynamic nodes - how? In-Reply-To: <269a5c30907071105t625fbd04na13002cafdcc0ded@mail.gmail.com> References: <269a5c30907070548k7dd9722fu3edf86f4f699968d@mail.gmail.com> <269a5c30907071046j6c3f5667q7df9cd16f0d76918@mail.gmail.com> <269a5c30907071105t625fbd04na13002cafdcc0ded@mail.gmail.com> Message-ID: <269a5c30907071215q2697354ep4ca20a4fafcb38b2@mail.gmail.com> OK, so I created something like this to transparently handle creating or connecting to an existing Mnesia cluster (depending if node is first one or not). Does it look OK to you or did I miss something? *start_mnesia() -> mnesia:start(), DbNodes = mnesia:system_info(running_db_nodes), error_logger:info_msg("Running Mnesia database nodes:: ~p~n",[DbNodes]), %% register this node with Mnesia, handle if already part of it case lists:any(fun(X) -> node() =:= X end, DbNodes) of true -> error_logger:info_msg("Current node already registered with Mnesia.~n"); false -> error_logger:info_msg("Current node being registered with Mnesia.~n"), mnesia:change_config (extra_db_nodes, [node()]) end, %% ensure local schema is disk-based case mnesia:change_table_copy_type(schema, node(), disc_copies) of {atomic,ok} -> error_logger:info_msg("Successfully modified Mnesia schema to be disk-based.~n"); {aborted,{already_exists,schema,_Node,,disc_copies}} -> error_logger:info_msg("Disk-based Mnesia schema found.~n"); Error -> error_logger:error_msg("Unable to change Mnesia schema to disc-based: ~p~n",[Error]), init:halt(1) end, ok.* From erlang@REDACTED Tue Jul 7 21:58:07 2009 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 7 Jul 2009 21:58:07 +0200 Subject: changing the module name attribute in a parse transform Message-ID: <9b08084c0907071258w6845f368qccdc53407c4754af@mail.gmail.com> If I change the module attribute in a parse transform I get an error. The call compile:file("foo.erl", [{parse_transform, bar}]) Generates an error if bar:parse_transform changes the module name attribute from foo to anything else Is there an easy work around to this? I want to change the module name in foo.erl to a new name (say bar1234) and generate bar1234.beam which when loaded defines the module bar1234 /Joe From joelr1@REDACTED Tue Jul 7 22:17:27 2009 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 7 Jul 2009 21:17:27 +0100 Subject: optimizing an asynchronous architecture Message-ID: Suppose I'm tasked with building a server that can broadcast a message to 20K subscribers to a topic. No, I do not want to use RabbitMQ since my implementation fits into 2-3 of pages of code. I'm using gen_server:cast all around and the trip from publishing a message to gen_tcp:send is through 5 gen_servers. #1 is the "transport process" that receives the message and forwards it to #2, a locally registered "topic manager". I need the topic manager intermediary because the names of locally registered servers must be atoms and so I cannot locally register a server for each topic. The topic manager gen_server keeps track of locally registered "subscriber servers" (#3) which keep track of subscribers for each topic. Both #2 and #3 use dicts to map topics to processes and keep track of subscriber processes respectively. #4 is a client proxy that forwards the message back to the transport (#1) which pushes it out to the client socket. How do I go about cutting the message trip time in half? I tried using fprof but the output is mostly gen_server:loop, proc_lib:sync_wait, etc. I cannot disable certain functions since fprof always calls erlang:trace(PidSpec, true, ...), that is I can't disable tracing for unwanted. What I would like, ideally, is to timestamp the message as it goes through the 1-2-3-4-1 pipeline and calculate the deltas once I'm back at #1. Any other suggestions? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From koops.j@REDACTED Tue Jul 7 22:40:44 2009 From: koops.j@REDACTED (Jeroen Koops) Date: Tue, 7 Jul 2009 22:40:44 +0200 Subject: [erlang-questions] optimizing an asynchronous architecture In-Reply-To: References: Message-ID: <331a9abb0907071340r7464e39cy2d01cc4ad8ac03ca@mail.gmail.com> Hi Joel, What I would like, ideally, is to timestamp the message as it goes through > the 1-2-3-4-1 pipeline and calculate the deltas once I'm back at #1. Any > other suggestions? > Not that I have any experience with it, but did you have a look at the sequential tracing facility (http://www.erlang.org/doc/man/seq_trace.html)? Looks like it does pretty much what you need. ---- Jeroen From dizzyd@REDACTED Tue Jul 7 22:46:51 2009 From: dizzyd@REDACTED (Dave Smith) Date: Tue, 7 Jul 2009 14:46:51 -0600 Subject: [erlang-questions] optimizing an asynchronous architecture In-Reply-To: References: Message-ID: On Tue, Jul 7, 2009 at 2:17 PM, Joel Reymont wrote: > > How do I go about cutting the message trip time in half? In half of what? D. From joelr1@REDACTED Tue Jul 7 22:50:09 2009 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 7 Jul 2009 21:50:09 +0100 Subject: [erlang-questions] optimizing an asynchronous architecture In-Reply-To: References: Message-ID: In half of its original time, of course :-). The actual value is not important since I'm looking for optimization approaches. Sequential tracing is one approach that fits the bill, perhaps. I'll try to make use of it and will report my findings. Are there other approaches? On Jul 7, 2009, at 9:46 PM, Dave Smith wrote: > On Tue, Jul 7, 2009 at 2:17 PM, Joel Reymont wrote: >> >> How do I go about cutting the message trip time in half? > > In half of what? > > D. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From cliff@REDACTED Tue Jul 7 23:00:38 2009 From: cliff@REDACTED (Cliff Moon) Date: Tue, 07 Jul 2009 14:00:38 -0700 Subject: [erlang-questions] optimizing an asynchronous architecture In-Reply-To: References: Message-ID: <4A53B776.3020001@moonpolysoft.com> One technique that I was successful in using was timing exit and entry points for the various servers I was interested in using a one off profiling server to collect the statistics. Code for the profiling server is here: http://github.com/cliffmoon/dynomite/blob/b621762c1d052f8e4e85f5f4e057ed04389014d2/elibs/dynomite_prof.erl And profiling is turned off or on via a macro during the build: http://github.com/cliffmoon/dynomite/blob/cfe6dd6768d39a268ff2d49203bb2b70e19cba16/include/profile.hrl So long as your exit and entry points have unique labels, you can get an idea of where time is going. This helped me figure out that a massive amount of time was going to run queue contention in R12B-5, for instance. Joel Reymont wrote: > Suppose I'm tasked with building a server that can broadcast a message > to 20K subscribers to a topic. No, I do not want to use RabbitMQ since > my implementation fits into 2-3 of pages of code. > > I'm using gen_server:cast all around and the trip from publishing a > message to gen_tcp:send is through 5 gen_servers. > > #1 is the "transport process" that receives the message and forwards > it to #2, a locally registered "topic manager". > > I need the topic manager intermediary because the names of locally > registered servers must be atoms and so I cannot locally register a > server for each topic. The topic manager gen_server keeps track of > locally registered "subscriber servers" (#3) which keep track of > subscribers for each topic. Both #2 and #3 use dicts to map topics to > processes and keep track of subscriber processes respectively. > > #4 is a client proxy that forwards the message back to the transport > (#1) which pushes it out to the client socket. > > How do I go about cutting the message trip time in half? > > I tried using fprof but the output is mostly gen_server:loop, > proc_lib:sync_wait, etc. I cannot disable certain functions since > fprof always calls erlang:trace(PidSpec, true, ...), that is I can't > disable tracing for unwanted. > > What I would like, ideally, is to timestamp the message as it goes > through the 1-2-3-4-1 pipeline and calculate the deltas once I'm back > at #1. Any other suggestions? > > Thanks, Joel > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From joelr1@REDACTED Tue Jul 7 23:15:29 2009 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 7 Jul 2009 22:15:29 +0100 Subject: [erlang-questions] optimizing an asynchronous architecture In-Reply-To: <4A53B731.7000209@moonpolysoft.com> References: <4A53B731.7000209@moonpolysoft.com> Message-ID: Cliff, Do you have any usage examples? I'm using a home-grown stats server myself. It keeps track of min, max, avg, etc. values for any label I choose and dumps accumulated stats every so often. Thanks, Joel On Jul 7, 2009, at 9:59 PM, Cliff Moon wrote: > One technique that I was successful in using was timing exit and > entry points for the various servers I was interested in using a one > off profiling server to collect the statistics. Code for the > profiling server is here: > > http://github.com/cliffmoon/dynomite/blob/b621762c1d052f8e4e85f5f4e057ed04389014d2/elibs/dynomite_prof.erl > > And profiling is turned off or on via a macro during the build: > > http://github.com/cliffmoon/dynomite/blob/cfe6dd6768d39a268ff2d49203bb2b70e19cba16/include/profile.hrl --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From cliff@REDACTED Tue Jul 7 23:21:16 2009 From: cliff@REDACTED (Cliff Moon) Date: Tue, 07 Jul 2009 14:21:16 -0700 Subject: [erlang-questions] optimizing an asynchronous architecture In-Reply-To: References: <4A53B731.7000209@moonpolysoft.com> Message-ID: <4A53BC4C.3000505@moonpolysoft.com> http://github.com/cliffmoon/dynomite/blob/b621762c1d052f8e4e85f5f4e057ed04389014d2/elibs/storage_server.erl That's where it gets used the most. The nice thing about this implementation is that you can nest the timings however you want and it will not get different labels confused with one another. Joel Reymont wrote: > Cliff, > > Do you have any usage examples? > > I'm using a home-grown stats server myself. It keeps track of min, > max, avg, etc. values for any label I choose and dumps accumulated > stats every so often. > > Thanks, Joel > > On Jul 7, 2009, at 9:59 PM, Cliff Moon wrote: > >> One technique that I was successful in using was timing exit and >> entry points for the various servers I was interested in using a one >> off profiling server to collect the statistics. Code for the >> profiling server is here: >> >> http://github.com/cliffmoon/dynomite/blob/b621762c1d052f8e4e85f5f4e057ed04389014d2/elibs/dynomite_prof.erl >> >> >> And profiling is turned off or on via a macro during the build: >> >> http://github.com/cliffmoon/dynomite/blob/cfe6dd6768d39a268ff2d49203bb2b70e19cba16/include/profile.hrl >> > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > From chandrashekhar.mullaparthi@REDACTED Wed Jul 8 00:29:11 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 7 Jul 2009 23:29:11 +0100 Subject: [ANN] ibrowse-1.5.1 Message-ID: Hi all, New feature added to ibrowse upon request. Extract from README below. 03-07-2009 - Added option {stream_to, {Pid, once}} which allows the caller to control when it wants to receive more data. If this option is used, the call ibrowse:stream_next(Req_id) should be used to get more data. - Patch submitted by Steve Vinoski to remove compiler warnings about the use of obsolete guards Git it from git://github.com/cmullaparthi/ibrowse.git cheers Chandru From ok@REDACTED Wed Jul 8 03:59:35 2009 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 8 Jul 2009 13:59:35 +1200 Subject: [erlang-questions] comma vs andalso In-Reply-To: <20090707041622.GB28401@hijacked.us> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> <20090707041622.GB28401@hijacked.us> Message-ID: <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> On Jul 7, 2009, at 4:16 PM, Andrew Thompson wrote: > I found your explanation a *lot* clearer and even learned something > reading it, so I'd definitely agree that it's an improvement. Purr. > > I'm still unclear as to where 'andalso' differs from a comma though, > how > is > > true, false. > > any different from > > true andalso false. First, some historical background. Erlang was influenced by Prolog via Strand-88. (By the way, does anyone have a Strand manual they don't want?) In Prolog, there is one and only one way to get and-then (,) and one and only one way to get or-else (;). To oversimplify, ::= {';' }... ::= {',' }... ::= ['\+'] ( | '(' ')' ) ::= ['(' ')'] One of the things I've left out of that is infix, prefix, and postfix operators. So something like cousin_sibling_or_self(X, Y) :- ( father(X, P) ; mother(X, P) ), ( father(Y, Q) ; mother(Y, Q) ), ( father(P, G) ; mother(P, G) ), ( father(Q, G) ; mother(Q, G) ). is possible, where "or" is nested inside "and". Let's return to Erlang. This syntax was adopted for Erlang guards, *ALMOST*. (It was also adopted for Erlang function bodies, almost. The C-like use of comma in Erlang expressions is rooted in Prolog's comma, not C's.) The *ALMOST* part is that Erlang does not allow nesting. You cannot write f(X, Y) when (X == 1 ; X == 2), (Y == 1 ; Y == 2) -> You aren't even allowed to write (GE1,GE2),GE3 but *must* write GE1,GE2,GE3. Nor may you write G1;(G2;G3). Weird. This has never made sense to me. Such combinations are perfectly meaningful, and allowing them would make the definition of some guard macros considerably easier. The obvious thing to do is to remove this arbitrary restriction. That's not the path Erlang took. (Honestly, you couldn't make this stuff up.) Instead, *new* operators 'andalso' and 'orelse' (copied from SML, which is a fine language with some fine compilers) were added. Most of the time, you can use 'andalso' and 'orelse' in a guard, where you would have used ',' and ';'. "Most of the time" means "when under the normal rules of evaluation you would certainly not have had an exception." Let's take two apparently similar functions: f(X) when is_atom(element(3, X)) ; true -> 42. g(X) when is_atom(element(3, X)) orelse true -> 42. By actual test, f(99) -> 42. By actual test, g(99) -> a function_clause exception. That is, in a guard, an expression using 'andalso' or 'orelse' is still an expression, and if an exception occurs in the left-hand operand, the whole expression is still skipped. The thing is that 'andalso' and 'orelse' can be used in any expression, not just guards. Erlang is treating them in guard expressions exactly the same way that it treats them in any other expressions. And that's _different_ from the way ',' and ';' are treated when there's an exception. I conclude that (1) you had best be EXTREMELY cautious about using 'andalso' and 'orelse' in guards; they are NOT drop-in replacements for ',' and ';', and (2) it is long past time that Erlang allowed nested use of ',' and ';' in guards. From ajuttner.list@REDACTED Wed Jul 8 08:21:41 2009 From: ajuttner.list@REDACTED (Alpar Juttner) Date: Wed, 08 Jul 2009 07:21:41 +0100 Subject: [erlang-questions] comma vs andalso In-Reply-To: <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> <20090707041622.GB28401@hijacked.us> <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> Message-ID: <1247034101.6040.5.camel@piko.site> > I conclude that > (1) you had best be EXTREMELY cautious about using 'andalso' > and 'orelse' in guards; they are NOT drop-in replacements > for ',' and ';', and > (2) it is long past time that Erlang allowed nested use of > ',' and ';' in guards. (3) this topic should be discussed in details (with examples) in the doc. Alpar From roberto@REDACTED Wed Jul 8 10:25:46 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 8 Jul 2009 10:25:46 +0200 Subject: [erlang-questions] distributed system: same app, different params In-Reply-To: <05F3D38D836B6248BF7452CA10036F500412ECDF@EPRI17P32003B.csfb.cs-group.com> References: <4A39E84B.1010408@erlang-consulting.com> <087E0400-7545-49FC-8D0A-CD888DF15796@widetag.com> <4A3FA6F9.1060306@erlang-consulting.com> <9a09ca9a0906221604o56fd4c80sfe55e1db8320863@mail.gmail.com> <05F3D38D836B6248BF7452CA10036F500412ECDF@EPRI17P32003B.csfb.cs-group.com> Message-ID: <322EB8D0-4C3C-47CE-96BE-FFD86F890EC2@widetag.com> thank you thanos, that is also interesting. cheers, r. On 07/lug/09, at 16:04, Vassilakis, Thanos wrote: > I use a pylons web system that runs with Fabric > (http://docs.fabfile.org/0.9/) to manage our many wonderful releases ! > From raimo+erlang-questions@REDACTED Wed Jul 8 10:42:37 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 8 Jul 2009 10:42:37 +0200 Subject: [erlang-questions] Why does this not work in the latest otp In-Reply-To: <20090626125811.GA28941@erix.ericsson.se> References: <0A5E0904-3D3A-47FA-958B-E16DA574A6A1@gmail.com> <20090626125811.GA28941@erix.ericsson.se> Message-ID: <20090708084237.GA1680@erix.ericsson.se> On Fri, Jun 26, 2009 at 02:58:11PM +0200, Raimo Niskanen wrote: > Congratulations, you have found a serious bug in > the new atom cache code! > > Simply when sending [] (the empty list) as the _complete_ > message to a remote process, decoding fails in R13B01. > That is serious and it is interesting to note that > sending [] as the complete message between remote > processes must be _very_ uncommon, oddly enough. > You are the only one so far that has encountered it. > > The developer that wrote the new atom cache (now on vacation) > has given me a two-line patch to regression test. First shot > on one platform works fine and I do not expect any problems. > Next week I will post the patch if the regression tests run fine. Sorry about the delay. I had almost totally unrelated problems in the regression tests. (caused by the same developer, otherwise totally unrelated). But here is the two-line patch (attached). It works fine in our regression tests (when they run). > > > > On Wed, Jun 24, 2009 at 03:17:02PM +0100, Thomas James Malone wrote: > > > > > > If I use the module I have included as an attachement and run the > > following code in R13B01, > > > > db_dis:new(database, remote@REDACTED). > > db_dis:write(tom, london, database, remote@REDACTED). > > db_dis:write(stuart, london, database, remote@REDACTED). > > db_dis:match(tom, database, remote@REDACTED). > > > > > > =ERROR REPORT==== 24-Jun-2009::15:13:11 === > > 'local@REDACTED' got a corrupted external term from 'remote@REDACTED' on > > distribution channel 5571 > > <<...,106>> > > ATOM_CACHE_REF translations: 0='local@REDACTED', 1='' > > timeout > > > > In R12B-5 if run the same code I get my expected response of [] > > > > what am I missing. > > > > thanks in advance > > > > Tom > > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB -------------- next part -------------- A non-text attachment was scrubbed... Name: external.c.diff Type: text/x-patch Size: 667 bytes Desc: not available URL: From thomasl_erlang@REDACTED Wed Jul 8 10:38:30 2009 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 8 Jul 2009 01:38:30 -0700 (PDT) Subject: [erlang-questions] comma vs andalso In-Reply-To: <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> <20090707041622.GB28401@hijacked.us> <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> Message-ID: <769861.4108.qm@web111415.mail.gq1.yahoo.com> ----- Original Message ---- > From: Richard O'Keefe > > Let's take two apparently similar functions: > > f(X) when is_atom(element(3, X)) ; true -> 42. > > g(X) when is_atom(element(3, X)) orelse true -> 42. > > By actual test, f(99) -> 42. > By actual test, g(99) -> a function_clause exception. > > That is, in a guard, an expression using 'andalso' or > 'orelse' is still an expression, and if an exception > occurs in the left-hand operand, the whole expression > is still skipped. .... > I conclude that > (1) you had best be EXTREMELY cautious about using 'andalso' > and 'orelse' in guards; they are NOT drop-in replacements > for ',' and ';', and > (2) it is long past time that Erlang allowed nested use of > ',' and ';' in guards. Guards are an awful mess in Erlang. I'd be quite surprised if the above difference in behaviour was intentional. If so, what possible purpose does it serve? At one point, you could also use "and"/"or", the boolean operators manque, in guards. (I haven't checked recently whether this is still the case.) So we then have three very similar sets of guard operators. Not to mention the twin set of type tests, introduced, as far as I know, to get rid of the lone double entendre of float/1 (as a test or conversion not the most frequent of operations). And now, for our convenience, the shorter form of these tests is being deprecated. Sigh. Best, Thomas From Woolla_T@REDACTED Wed Jul 8 12:45:23 2009 From: Woolla_T@REDACTED (Trevor Woollacott [ MTN - Innovation Centre ]) Date: Wed, 8 Jul 2009 12:45:23 +0200 Subject: [erlang-questions] tcp_close during async accept In-Reply-To: <4EEFAA5F-00D9-45FE-A1D0-A278A94F142D@gmail.com> References: <4EEFAA5F-00D9-45FE-A1D0-A278A94F142D@gmail.com> Message-ID: I'm not sure for Max OSX, but I think it is 128. Although that wouldn't explain why only half of your 700 connections are successful. Maybe there is a fudge factor involved in the calculation of the actual backlog value, or maybe the listener process isn't accepting connections quickly enough? I know for Windows Sockets 1.1 the maximum value is 5, and some versions of Windows (such as Windows 2000 Server) allow a backlog value of up to 200. > -----Original Message----- > From: Joel Reymont [mailto:joelr1@REDACTED] > Sent: Tuesday, 07 July 2009 04:31 PM > To: Trevor Woollacott [ MTN - Innovation Centre ] > Cc: Erlang Users' List > Subject: Re: [erlang-questions] tcp_close during async accept > > > On Jul 7, 2009, at 12:10 PM, Trevor Woollacott [ MTN - Innovation > Centre ] wrote: > > > What OS are you using? Maybe the maximum backlog on your platform is > > less than 1024? > > > Mac OSX 10.5.7 > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont NOTE: This e-mail message is subject to the MTN Group disclaimer see http://www.mtn.co.za/SUPPORT/LEGAL/Pages/EmailDisclaimer.aspx From oscar@REDACTED Wed Jul 8 12:45:03 2009 From: oscar@REDACTED (=?ISO-8859-1?Q?Oscar_Hellstr=F6m?=) Date: Wed, 08 Jul 2009 11:45:03 +0100 Subject: Dialyzer and erlang:error/1 Message-ID: <4A5478AF.6000807@erlang-consulting.com> Hi All, I'm trying to express that a valid list of options will return ok while any invalid option would eventually throw an exception, but Dialyzer and I don't get along. Dialyzer says: foo.erl:14: The call foo:verify(Options::any(),[any(),...]) will never return since it differs in argument position 2 from the success typing arguments: (['bar' | 'foo'],[]) This is entirely true, but it's not an error, it's what I want! I guess I'm failing to express what I want here, so could anybody help me? Best regards -- Oscar Hellstr?m, oscar@REDACTED Office: +44 20 7655 0337 Mobile: +44 798 45 44 773 Erlang Training and Consulting Ltd http://www.erlang-consulting.com/ -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: foo.erl URL: From oscar@REDACTED Wed Jul 8 12:48:41 2009 From: oscar@REDACTED (=?ISO-8859-1?Q?Oscar_Hellstr=F6m?=) Date: Wed, 08 Jul 2009 11:48:41 +0100 Subject: [erlang-questions] tcp_close during async accept In-Reply-To: References: <4EEFAA5F-00D9-45FE-A1D0-A278A94F142D@gmail.com> Message-ID: <4A547989.1060106@erlang-consulting.com> There might be syn cookies involved as well? That way you kind of believe (in the client) that the connection is successful, or at least that's how I understand it. I also don't know about OSX, but at least in Linux you can reconfigure most of this with sysctl... Trevor Woollacott [ MTN - Innovation Centre ] wrote: > I'm not sure for Max OSX, but I think it is 128. Although that wouldn't explain why only half of your 700 connections are successful. Maybe there is a fudge factor involved in the calculation of the actual backlog value, or maybe the listener process isn't accepting connections quickly enough? > I know for Windows Sockets 1.1 the maximum value is 5, and some versions of Windows (such as Windows 2000 Server) allow a backlog value of up to 200. > > >> -----Original Message----- >> From: Joel Reymont [mailto:joelr1@REDACTED] >> Sent: Tuesday, 07 July 2009 04:31 PM >> To: Trevor Woollacott [ MTN - Innovation Centre ] >> Cc: Erlang Users' List >> Subject: Re: [erlang-questions] tcp_close during async accept >> >> >> On Jul 7, 2009, at 12:10 PM, Trevor Woollacott [ MTN - Innovation >> Centre ] wrote: >> >> >>> What OS are you using? Maybe the maximum backlog on your platform is >>> less than 1024? >>> >> Mac OSX 10.5.7 >> >> --- >> Mac hacker with a performance bent >> http://www.linkedin.com/in/joelreymont >> > > > NOTE: This e-mail message is subject to the MTN Group disclaimer see http://www.mtn.co.za/SUPPORT/LEGAL/Pages/EmailDisclaimer.aspx > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- Oscar Hellstr?m, oscar@REDACTED Office: +44 20 7655 0337 Mobile: +44 798 45 44 773 Erlang Training and Consulting Ltd http://www.erlang-consulting.com/ From eletuchy@REDACTED Wed Jul 8 13:16:41 2009 From: eletuchy@REDACTED (eletuchy@REDACTED) Date: Wed, 8 Jul 2009 04:16:41 -0700 (PDT) Subject: Erlang at Facebook - hibernation and gen_server In-Reply-To: <523869a70906301041m1017055alafe279f156f1824a@mail.gmail.com> References: <523869a70906292134j5d51d679nd72f0060a0317b2f@mail.gmail.com> <1246364964.5363.5.camel@sredniczarny.smp.if.uj.edu.pl> <523869a70906301041m1017055alafe279f156f1824a@mail.gmail.com> Message-ID: <2a78ca8c-6d6f-441a-95a8-e4b717c0fe46@x5g2000prf.googlegroups.com> The incompatibility is on the client side (gen:call), not the server side (gen_server:handle_call). There's a receive() loop inside gen:call that prevents a client from hibernating between dispatching the $call message and getting back the gen_server:reply(). On Jun 30, 10:41?am, Davide Marqu?s wrote: > > This is really strange, that only gen_event supports explicit hibernate > > ie. returning {ok, Reply, State1, hibernate} from handle_call. > > > gen_server and fsm lacks support for hibernate. > > -- > > Witold Baryluk > > That's the thing... from what I can tell gen_server's support for > hibernate is present and working!http://www.erlang.org/doc/man/gen_server.html#Module:init-1 > > Using a gen_server looking like this: > init([Data]) -> > ? ? {ok, #state{data=Data}}. > > handle_call(hibernate, _From, State) -> > ? ? {reply, going_to_hibernate, State, hibernate}; > handle_call(get_state, _From, State) -> > ? ? {reply, State, State}; > handle_call(_Request, _From, State) -> > ? ? {reply, ok, State}. > > We can see a gen_server hibernating: > Eshell V5.7.2 ?(abort with ^G) > 1> hibernate:start_link(initial_state). > {ok,<0.34.0>} > 2> gen_server:call(hibernate, get_state). > {state,initial_state} > 3> erlang:process_info(whereis(hibernate), total_heap_size). > {total_heap_size,233} > 4> gen_server:call(hibernate, hibernate). > going_to_hibernate > 5> erlang:process_info(whereis(hibernate), total_heap_size). > {total_heap_size,31} > 6> gen_server:call(hibernate, get_state). > {state,initial_state} > 7> erlang:process_info(whereis(hibernate), total_heap_size). > {total_heap_size,267} > > It seems to work, so I'm left to wonder what "incompatibility" > was described in Eugene's presentation. > > Cheers, > :Davide From richardc@REDACTED Wed Jul 8 14:06:05 2009 From: richardc@REDACTED (Richard Carlsson) Date: Wed, 08 Jul 2009 14:06:05 +0200 Subject: [erlang-questions] comma vs andalso In-Reply-To: <769861.4108.qm@web111415.mail.gq1.yahoo.com> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> <20090707041622.GB28401@hijacked.us> <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> <769861.4108.qm@web111415.mail.gq1.yahoo.com> Message-ID: <4A548BAD.1030401@it.uu.se> Thomas Lindgren wrote: > ----- Original Message ---- >> From: Richard O'Keefe >> That is, in a guard, an expression using 'andalso' or 'orelse' is >> still an expression, and if an exception occurs in the left-hand >> operand, the whole expression is still skipped. > > Guards are an awful mess in Erlang. I'd be quite surprised if the > above difference in behaviour was intentional. If so, what possible > purpose does it serve? It is intentional in the respect that 'andalso'/'orelse' behave just like any other operator in a guard test. As for purpose, all Erlang's operators are allowed in guard expressions, so why make an exception? > At one point, you could also use "and"/"or", the boolean operators > manque, in guards. (I haven't checked recently whether this is still > the case.) So we then have three very similar sets of guard > operators. 'and'/'or' have always (as far back as I can remember, anyway) been allowed in guards, again, probably simply by virtue of being general operators in the language. And they don't behave like ','/';' either. So to add to Richard O'Keefe's examples: h(X) when is_atom(element(3, X)) or true -> 42. which for h(99) behaves just like the orelse version, i.e., throws a function_clause error. Quite simply, and/or/andalso/orelse behave just like +, ==, or any other operator; compare e.g.: i(X) when is_atom(element(3, X)) == true -> 42. > Not to mention the twin set of type tests, introduced, as far as I > know, to get rid of the lone double entendre of float/1 (as a test or > conversion not the most frequent of operations). That was one details, yes, but the main reason was to make it possible to refactor a piece of code without being forced to change the names of the function tests if you moved an expression from within a guard to an ordinary expression and vice versa. Recall that before the is_... versions, there was no way of saying e.g., "Bool = (is_integer(X) and is_atom(Y))" without writing a case/if or a separate predicate function. The old type tests didn't correspond to any built-in functions, so you had to treat them as "primops" inside the compiler, you couldn't refer to them and pass them around, etc. But the old type test names "atom(X)" and so forth could not simply be made to work outside guards because there would be name clashes (with the float(X) bif and with any existing code that defined a function such as list(X) or integer(X)), hence the is_... prefix for the generally usable versions that are proper built-in functions (defined in the 'erlang' module along with all the others). Now that we have the new is_-forms, the old forms are merely an obstacle to refactoring. As a simple example, this macro nowadays works in both guard expressions and in general expressions - it didn't back then: -define(is_a_foo(X), (is_integer(X) or is_binary(X))). (we can even use orelse here, and it will still work in a guard). > And now, for our convenience, the shorter form of these tests is being > deprecated. Hold your horses - nobody is deprecating the use of ',' and ';'. But as Richard O'Keefe has rightly described, they *are* different from the plain boolean operators. A guard has the following general form: ... when *, ..., * ; *, ..., * ; ... -> that is, one or more semicolon-separated alternatives, each consisting of one or more comma-separated _tests_ (marked with a * here). Each test is (conceptually) evaluated in a try/catch that simply transforms any exception into 'false'. (Subexpressions of a test are evaluated as normal, which is why 'orelse' does not continue executing if there is an exception in the left-hand expression.) This fail-to-false behaviour was in my opinion a mistake, because it occasionally hides a bug in the guard and turns what should have been an observable runtime crash into a silent "well, we take the next clause then". Some people like to use this as a trick to write more compact guards, but that makes it hard for someone reading the code to tell whether the crash-jumps-to-the-next-case is intentional or not. However, Erlang seems to be stuck with these semantics of guards. But it means that if you do things inside your guard test like element(N,T), length(Xs), or anything else that could crash if it gets bad arguments, you need to think about the effect on what clause, if any, will be selected. The thing to remember is that exceptions always stop at the top of each separate test in the guard. Complicated expressions that might throw an unwanted exception are generally better evaluated before the switch, so that bugs are not hidden. I'm not sure I have any real arguments against nesting of ','/';', but I fear the grammar could get rather messy, and that such nested guards could be quite difficult to read in practice. /Richard From richardc@REDACTED Wed Jul 8 14:30:58 2009 From: richardc@REDACTED (Richard Carlsson) Date: Wed, 08 Jul 2009 14:30:58 +0200 Subject: [erlang-questions] comma vs andalso In-Reply-To: <769861.4108.qm@web111415.mail.gq1.yahoo.com> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> <20090707041622.GB28401@hijacked.us> <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> <769861.4108.qm@web111415.mail.gq1.yahoo.com> Message-ID: <4A549182.4070908@it.uu.se> Thomas Lindgren wrote: > Not to mention the twin set of type tests, introduced, as far as I > know, to get rid of the lone double entendre of float/1 (as a test or > conversion not the most frequent of operations). And now, for our > convenience, the shorter form of these tests is being deprecated. > Sigh. Oh, and I forgot this beauty, which is a consequence of the doubleplusweird scope rules for the old type tests in guards: 1> if integer(42) -> ok; true -> no end. ok 2> if integer(42) == true -> ok; true -> no end. * 1: illegal guard expression because, you see, it is only at the absolute top of the guard test that the name 'integer(X)' can be used to refer to the type test. Shove it into one side of an '==', and it is no longer recognized. The new forms work everywhere: 3> if is_integer(42) == true -> ok; true -> no end. ok 4> if is_integer(42) == true -> ok; true -> no end. ok And for my final trick, here's the old double entendre you mentioned: 5> if float(3.14) -> ok; true -> no end. ok 6> if float(3.14) == true -> ok; true -> no end. no why, isn't it obvious? The first clause uses the type test float(X), while the second ensures that 3.14 is cast to a float and then compares the number to 'true'. Sticking to the modern is_-forms of the type tests lets you stay sane. /Richard From el03665@REDACTED Wed Jul 8 14:34:46 2009 From: el03665@REDACTED (Maria Christakis) Date: Wed, 08 Jul 2009 15:34:46 +0300 Subject: [erlang-questions] Dialyzer and erlang:error/1 In-Reply-To: <4A5478AF.6000807@erlang-consulting.com> References: <4A5478AF.6000807@erlang-consulting.com> Message-ID: <4A549266.2010704@mail.ntua.gr> Hello, I think that dialyzer assumes that no exception will ever be thrown. When you remove the specs, it comes up with: -spec verify(['bar' | 'foo']) -> 'ok'. -spec verify(['bar' | 'foo'],[]) -> 'ok'. because no exception will be thrown when your Options list is either [] or contains 'foo' and / or 'bar' elements only. Then your Errors list will always be empty. Hope this helped but a more educated opinion would be better, Maria Oscar Hellstr?m wrote: > Hi All, > > I'm trying to express that a valid list of options will return ok while > any invalid option would eventually throw an exception, but Dialyzer and > I don't get along. > > Dialyzer says: > foo.erl:14: The call foo:verify(Options::any(),[any(),...]) will never > return since it differs in argument position 2 from the success typing > arguments: (['bar' | 'foo'],[]) > > This is entirely true, but it's not an error, it's what I want! I guess > I'm failing to express what I want here, so could anybody help me? > > Best regards > > > ------------------------------------------------------------------------ > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org From v@REDACTED Wed Jul 8 14:41:04 2009 From: v@REDACTED (Valentin Micic) Date: Wed, 8 Jul 2009 14:41:04 +0200 Subject: [erlang-questions] tcp_close during async accept In-Reply-To: Message-ID: <200907081239.n68Cd6lc019001@mail.pharos-avantgard.com> In my experience, a backlog value is rarely an issue in practice (although testing may be another matter). What's more often a problem when you do high-volume testing is number of file descriptors (e.g. maybe you did change it in ERTS but not at kernel level, or vice-versa), or even value for TIME_WAIT period etc. What makes you believe that the problem is with backlog? V. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Trevor Woollacott [ MTN - Innovation Centre ] Sent: 08 July 2009 12:45 PM To: Joel Reymont Cc: erlang-questions@REDACTED Subject: RE: [erlang-questions] tcp_close during async accept I'm not sure for Max OSX, but I think it is 128. Although that wouldn't explain why only half of your 700 connections are successful. Maybe there is a fudge factor involved in the calculation of the actual backlog value, or maybe the listener process isn't accepting connections quickly enough? I know for Windows Sockets 1.1 the maximum value is 5, and some versions of Windows (such as Windows 2000 Server) allow a backlog value of up to 200. > -----Original Message----- > From: Joel Reymont [mailto:joelr1@REDACTED] > Sent: Tuesday, 07 July 2009 04:31 PM > To: Trevor Woollacott [ MTN - Innovation Centre ] > Cc: Erlang Users' List > Subject: Re: [erlang-questions] tcp_close during async accept > > > On Jul 7, 2009, at 12:10 PM, Trevor Woollacott [ MTN - Innovation > Centre ] wrote: > > > What OS are you using? Maybe the maximum backlog on your platform is > > less than 1024? > > > Mac OSX 10.5.7 > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont NOTE: This e-mail message is subject to the MTN Group disclaimer see http://www.mtn.co.za/SUPPORT/LEGAL/Pages/EmailDisclaimer.aspx ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From oscar@REDACTED Wed Jul 8 14:55:46 2009 From: oscar@REDACTED (=?ISO-8859-1?Q?Oscar_Hellstr=F6m?=) Date: Wed, 08 Jul 2009 13:55:46 +0100 Subject: [erlang-questions] Dialyzer and erlang:error/1 In-Reply-To: <4A549266.2010704@mail.ntua.gr> References: <4A5478AF.6000807@erlang-consulting.com> <4A549266.2010704@mail.ntua.gr> Message-ID: <4A549752.4@erlang-consulting.com> Hi Maria, Well, it doesn't help, since it will complain about line 14 calling verify with the second argument not being an empty list. That's actually what it's complaining about in the original version as well. And yes, that call will never return, but that's kinda what I tried to express with the return type ok | no_return() Thanks Maria Christakis wrote: > Hello, > > I think that dialyzer assumes that no exception will ever be thrown. > When you remove the specs, it comes up with: > > -spec verify(['bar' | 'foo']) -> 'ok'. > -spec verify(['bar' | 'foo'],[]) -> 'ok'. > > because no exception will be thrown when your Options list is either > [] or contains 'foo' and / or 'bar' elements only. Then your Errors > list will always be empty. > > Hope this helped but a more educated opinion would be better, > Maria > > Oscar Hellstr?m wrote: >> Hi All, >> >> I'm trying to express that a valid list of options will return ok while >> any invalid option would eventually throw an exception, but Dialyzer and >> I don't get along. >> >> Dialyzer says: >> foo.erl:14: The call foo:verify(Options::any(),[any(),...]) will never >> return since it differs in argument position 2 from the success typing >> arguments: (['bar' | 'foo'],[]) >> >> This is entirely true, but it's not an error, it's what I want! I guess >> I'm failing to express what I want here, so could anybody help me? >> >> Best regards >> >> >> ------------------------------------------------------------------------ >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org > -- Oscar Hellstr?m, oscar@REDACTED Office: +44 20 7655 0337 Mobile: +44 798 45 44 773 Erlang Training and Consulting Ltd http://www.erlang-consulting.com/ From tobias.lindahl@REDACTED Wed Jul 8 15:41:32 2009 From: tobias.lindahl@REDACTED (Tobias Lindahl) Date: Wed, 08 Jul 2009 15:41:32 +0200 Subject: [erlang-questions] Dialyzer and erlang:error/1 In-Reply-To: <4A5478AF.6000807@erlang-consulting.com> References: <4A5478AF.6000807@erlang-consulting.com> Message-ID: <4A54A20C.80607@it.uu.se> Oscar Hellstr?m wrote: > Hi All, > > I'm trying to express that a valid list of options will return ok while > any invalid option would eventually throw an exception, but Dialyzer and > I don't get along. > > Dialyzer says: > foo.erl:14: The call foo:verify(Options::any(),[any(),...]) will never > return since it differs in argument position 2 from the success typing > arguments: (['bar' | 'foo'],[]) > > This is entirely true, but it's not an error, it's what I want! I guess > I'm failing to express what I want here, so could anybody help me? > I see what you are trying to do, but unfortunately it does not work this way. The no_return() will disappear when you make the union. What you need to do, although I admit that it is a bit constructed, is to break out the clause that throws the exception to a separate function. In this way the no_return() allows you to accept the exception clause. (Code below. Should work. Haven't tried it) Best regards, Tobias -module(foo). -export([verify/1]). -spec verify([atom()]) -> ok. verify(Options) -> verify(Options, []). -spec verify([atom()], [atom()]) -> ok. verify([foo | Options], Errors) -> verify(Options, Errors); verify([bar | Options], Errors) -> verify(Options, Errors); verify([Option | Options], Errors) -> verify(Options, [Option | Errors]); verify([], []) -> ok; verify([], Errors) -> bad_options(Errors). -spec bad_options([atom()]) -> no_return(). bad_options(Errors) -> erlang:error({bad_options, Errors}). > Best regards > > > > ------------------------------------------------------------------------ > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org From oscar@REDACTED Wed Jul 8 16:21:10 2009 From: oscar@REDACTED (=?ISO-8859-1?Q?Oscar_Hellstr=F6m?=) Date: Wed, 08 Jul 2009 15:21:10 +0100 Subject: [erlang-questions] Dialyzer and erlang:error/1 In-Reply-To: <4A54A20C.80607@it.uu.se> References: <4A5478AF.6000807@erlang-consulting.com> <4A54A20C.80607@it.uu.se> Message-ID: <4A54AB56.7060306@erlang-consulting.com> Hi Tobias, Thank you very much, this works well :) Tobias Lindahl wrote: > > > Oscar Hellstr?m wrote: >> Hi All, >> >> I'm trying to express that a valid list of options will return ok while >> any invalid option would eventually throw an exception, but Dialyzer and >> I don't get along. >> >> Dialyzer says: >> foo.erl:14: The call foo:verify(Options::any(),[any(),...]) will never >> return since it differs in argument position 2 from the success typing >> arguments: (['bar' | 'foo'],[]) >> >> This is entirely true, but it's not an error, it's what I want! I guess >> I'm failing to express what I want here, so could anybody help me? >> > I see what you are trying to do, but unfortunately it does not work > this way. The no_return() will disappear when you make the union. > > What you need to do, although I admit that it is a bit constructed, is > to break out the clause that throws the exception to a separate > function. In this way the no_return() allows you to accept the > exception clause. > > (Code below. Should work. Haven't tried it) > > Best regards, > > Tobias > > > -module(foo). > -export([verify/1]). > > -spec verify([atom()]) -> ok. > > verify(Options) -> > verify(Options, []). > > -spec verify([atom()], [atom()]) -> ok. > > verify([foo | Options], Errors) -> > verify(Options, Errors); > verify([bar | Options], Errors) -> > verify(Options, Errors); > verify([Option | Options], Errors) -> > verify(Options, [Option | Errors]); > verify([], []) -> > ok; > verify([], Errors) -> > bad_options(Errors). > > -spec bad_options([atom()]) -> no_return(). > > bad_options(Errors) -> > erlang:error({bad_options, Errors}). > > > >> Best regards >> >> >> >> ------------------------------------------------------------------------ >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org -- Oscar Hellstr?m, oscar@REDACTED Office: +44 20 7655 0337 Mobile: +44 798 45 44 773 Erlang Training and Consulting Ltd http://www.erlang-consulting.com/ From wolfmanjm@REDACTED Wed Jul 8 17:19:17 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Wed, 8 Jul 2009 08:19:17 -0700 (PDT) Subject: tcp_close during async accept In-Reply-To: <200907081239.n68Cd6lc019001@mail.pharos-avantgard.com> References: <200907081239.n68Cd6lc019001@mail.pharos-avantgard.com> Message-ID: <0e972a47-e8ac-41e8-a9c9-2bc8190928d1@b25g2000prb.googlegroups.com> I am using the same async code in my server, and have tested several thousand connections with no problem. Are you testing client and server on same machine? if not are they on the same network? Is there something in between that is doing TCP SYN attack mitigation? Maybe OS/X does SYN attack protection builtin? The trouble with testing so many simultaneous connections in a short time is it does look like a SYN attack. One thing I did find is my tester client had to have a very short delay between the tcp opens, around 10ms was enough to significantly speed up the test. Hitting my server with a thousand simultaneous opens did cause the server some issues, however not the same as you report, pacing the opens slightly had a significant impact on performance. On Jul 8, 5:41?am, "Valentin Micic" wrote: > In my experience, a backlog value is rarely an issue in practice (although > testing may be another matter). What's more often a problem when you do > high-volume testing is number of file descriptors (e.g. maybe you did change > it in ERTS but not at kernel level, or vice-versa), or even value for > TIME_WAIT period etc. What makes you believe that the problem is with > backlog? > > V. > > -----Original Message----- > From: erlang-questi...@REDACTED [mailto:erlang-questi...@REDACTED] On > > Behalf Of Trevor Woollacott [ MTN - Innovation Centre ] > Sent: 08 July 2009 12:45 PM > To: Joel Reymont > Cc: erlang-questi...@REDACTED > Subject: RE: [erlang-questions] tcp_close during async accept > > I'm not sure for Max OSX, but I think it is 128. Although that wouldn't > explain why only half of your 700 connections are successful. Maybe there is > a fudge factor involved in the calculation of the actual backlog value, or > maybe the listener process isn't accepting connections quickly enough? > I know for Windows Sockets 1.1 the maximum value is 5, and some versions of > Windows (such as Windows 2000 Server) allow a backlog value of up to 200. > > > -----Original Message----- > > From: Joel Reymont [mailto:joe...@REDACTED] > > Sent: Tuesday, 07 July 2009 04:31 PM > > To: Trevor Woollacott [ MTN - Innovation Centre ] > > Cc: Erlang Users' List > > Subject: Re: [erlang-questions] tcp_close during async accept > > > On Jul 7, 2009, at 12:10 PM, Trevor Woollacott [ MTN - Innovation > > Centre ] wrote: > > > > What OS are you using? Maybe the maximum backlog on your platform is > > > less than 1024? > > > Mac OSX 10.5.7 > > > --- > > Mac hacker with a performance bent > >http://www.linkedin.com/in/joelreymont > > NOTE: This e-mail message is subject to the MTN Group disclaimer seehttp://www.mtn.co.za/SUPPORT/LEGAL/Pages/EmailDisclaimer.aspx > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From trevorw@REDACTED Wed Jul 8 17:45:58 2009 From: trevorw@REDACTED (Trevor Woollacott) Date: Wed, 8 Jul 2009 17:45:58 +0200 Subject: [erlang-questions] tcp_close during async accept References: <200907081239.n68Cd6lc019001@mail.pharos-avantgard.com> Message-ID: <003101c9ffe3$32f0a2e0$706bd90a@mtn.co.za> Thats true, running out of file descriptors is likely. If that is the case then you should be seeing EMFILE or ENFILE errors. I had assumed that the TIME_WAIT period wouldn't be a problem because you are using SO_REUSEADDR in the listener. But I suppose it could be a problem if the clients have their ports defined manually. For example, if the client on PORT A is connected to the remote server on PORT B, and then the connection closes, the client will still be in TIME_WAIT state. The same client on PORT A will not be able to reconnect to the servers remote address until the TIME_WAIT state is over, even with SO_REUSEADDR. The best thing to do is snoop the network traffic to see exactly what is going on. ----- Original Message ----- From: "Valentin Micic" To: "'Trevor Woollacott [ MTN - Innovation Centre ]'" ; "'Joel Reymont'" Cc: Sent: Wednesday, July 08, 2009 2:41 PM Subject: RE: [erlang-questions] tcp_close during async accept > In my experience, a backlog value is rarely an issue in practice (although > testing may be another matter). What's more often a problem when you do > high-volume testing is number of file descriptors (e.g. maybe you did > change > it in ERTS but not at kernel level, or vice-versa), or even value for > TIME_WAIT period etc. What makes you believe that the problem is with > backlog? > > V. > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Trevor Woollacott [ MTN - Innovation Centre ] > Sent: 08 July 2009 12:45 PM > To: Joel Reymont > Cc: erlang-questions@REDACTED > Subject: RE: [erlang-questions] tcp_close during async accept > > I'm not sure for Max OSX, but I think it is 128. Although that wouldn't > explain why only half of your 700 connections are successful. Maybe there > is > a fudge factor involved in the calculation of the actual backlog value, or > maybe the listener process isn't accepting connections quickly enough? > I know for Windows Sockets 1.1 the maximum value is 5, and some versions > of > Windows (such as Windows 2000 Server) allow a backlog value of up to 200. > >> -----Original Message----- >> From: Joel Reymont [mailto:joelr1@REDACTED] >> Sent: Tuesday, 07 July 2009 04:31 PM >> To: Trevor Woollacott [ MTN - Innovation Centre ] >> Cc: Erlang Users' List >> Subject: Re: [erlang-questions] tcp_close during async accept >> >> >> On Jul 7, 2009, at 12:10 PM, Trevor Woollacott [ MTN - Innovation >> Centre ] wrote: >> >> > What OS are you using? Maybe the maximum backlog on your platform is >> > less than 1024? >> >> >> Mac OSX 10.5.7 >> >> --- >> Mac hacker with a performance bent >> http://www.linkedin.com/in/joelreymont > > > NOTE: This e-mail message is subject to the MTN Group disclaimer see > http://www.mtn.co.za/SUPPORT/LEGAL/Pages/EmailDisclaimer.aspx > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org From marc@REDACTED Wed Jul 8 18:13:15 2009 From: marc@REDACTED (Marc Worrell) Date: Wed, 8 Jul 2009 18:13:15 +0200 Subject: Erlang programmers in NL? Message-ID: <226D6C47-FCCB-45B9-BDB9-BB5064241D7D@worrell.nl> Hello, For a customer I would like to know how many Erlang programmers are available in the Netherlands, and if there are any companies able to supply support for Erlang based systems and/or products. Any pointers greatly appreciated. Kind Regards, Marc Worrell. From joelr1@REDACTED Wed Jul 8 18:27:33 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 8 Jul 2009 17:27:33 +0100 Subject: [erlang-questions] Re: tcp_close during async accept In-Reply-To: <0e972a47-e8ac-41e8-a9c9-2bc8190928d1@b25g2000prb.googlegroups.com> References: <200907081239.n68Cd6lc019001@mail.pharos-avantgard.com> <0e972a47-e8ac-41e8-a9c9-2bc8190928d1@b25g2000prb.googlegroups.com> Message-ID: On Jul 8, 2009, at 4:19 PM, Jim Morris wrote: > Are you testing client and server on same machine? Yes. > if not are they on > the same network? Is there something in between that is doing TCP SYN > attack mitigation? I believe I saw the same effect on Amazon EC2. > One thing I did find is my tester client had to have a very short > delay between the tcp opens, around 10ms was enough to significantly > speed up the test. I'm reconnecting after a random delay when this happens. I was wondering why it's a successful connection on the client side and then a tcp_close since it's not even getting to the Erlang accept on the server side. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Wed Jul 8 18:29:38 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 8 Jul 2009 17:29:38 +0100 Subject: [erlang-questions] tcp_close during async accept In-Reply-To: <003101c9ffe3$32f0a2e0$706bd90a@mtn.co.za> References: <200907081239.n68Cd6lc019001@mail.pharos-avantgard.com> <003101c9ffe3$32f0a2e0$706bd90a@mtn.co.za> Message-ID: On Jul 8, 2009, at 4:45 PM, Trevor Woollacott wrote: > Thats true, running out of file descriptors is likely. If that is > the case then you should be seeing EMFILE or ENFILE errors. This is not the case in my test, although I did report a related issue: Mac OSX 10.5.7 ulimit -n 10240 Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:true] =ERROR REPORT==== 6-Jul-2009::20:51:54 === driver_select(0x00000041, 1024, ERL_DRV_WRITE ERL_DRV_USE, 1) by tcp_inet driver #Port<0.3137> failed: fd=1024 is larger than the largest allowed fd=1023 =ERROR REPORT==== 6-Jul-2009::20:51:55 === File operation error: system_limit. Target: s3erl/ebin/random.beam. Function: get_file. Process: code_server. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Wed Jul 8 19:30:32 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 8 Jul 2009 18:30:32 +0100 Subject: [erlang-questions] optimizing an asynchronous architecture In-Reply-To: <331a9abb0907071340r7464e39cy2d01cc4ad8ac03ca@mail.gmail.com> References: <331a9abb0907071340r7464e39cy2d01cc4ad8ac03ca@mail.gmail.com> Message-ID: On Jul 7, 2009, at 9:40 PM, Jeroen Koops wrote: > Not that I have any experience with it, but did you have a look at > the sequential tracing facility (http://www.erlang.org/doc/man/seq_trace.html)? > Looks like it does pretty much what you need. I rolled my own version of sequential tracing by wrapping a message up in another message with a timestamp at each stage. This way I can calculate how long it took the message to get from process to process, a total of 5 stages. The pipeline stops right before gen_tcp:send but it's still only 50% the time I calculate by other means, described below. I can't believe that 50% of the time is spent going through the TCP stack so perhaps my testing methodology is flawed. I need to broadcast my message once all the bots have connected to the server and subscribed to a particular topic. To make this happen, I have a "barrier" process that I start with the total number of bots I plan to run. Bots ping the barrier once they are ready which decrements the counter. Once the counter is 0 the barrier process exits. Bots monitor the barrier process to get notified of its exit and so does the broadcasting process. A single message is broadcast to all bots once the barrier opens. Bots log the time when they received the barrier exit notification as well as the time when they receive the message itself. The difference is what I compare against the "pipeline length" at the beginning of this message. It turns out that a message takes twice as long to get to a bot as the total time it takes a message to travel through all stages. Is it safe to assume that erlang:monitor is wicked fast and has no problem notifying 10-20k processes monitoring a single one? Are there better ways to implement a "barrier" in Erlang? Is there anything else, apart from gen_tcp:send, that could be accounting for that 50% difference between bot notification time and message travel time within the server? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Wed Jul 8 19:53:12 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 8 Jul 2009 18:53:12 +0100 Subject: ets vs process-based registry + local vs global dispatch Message-ID: <2448FB09-DBD0-498D-A0B1-9BFC2F6020FF@gmail.com> Are there any performance numbers on using ETS to keep track of processes vs using a process-based registry? What about local vs global gen_server casts? I'm keeping track of processes associated with session tokens. I can globally register a process with a name that includes any term. I can only use an atom when registering a process locally. I suspect that sending messages to globally registered processes ({global, {client_proxy, Token}}) is slowing me down quite a bit. I'm thinking of keeping processes in a token->pid map but I'd like to know if ETS is generally recommended over a process wrapping a dict or gb_tree map. I'm going to run tests and get some hard numbers. I'm wondering if there's an existing body of knowledge on this, though. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From vincent.dephily@REDACTED Wed Jul 8 19:55:50 2009 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Wed, 8 Jul 2009 19:55:50 +0200 Subject: http feature proposal: speifying which pid should receive async responses Message-ID: <200907081955.51089.vincent.dephily@mobile-devices.fr> Hi, currently, http:request(...{sync,false}...) returns responses to the calling process. This is not allways convenient : sometimes we want another process to receive the response. It's possible work around this using spawn/receive machinery, but that's pointlessly painfull. This patch adds a 'reply_pid' option to http:request(), cleanly solving the problem : --- src/http.erl~ 2009-06-12 12:04:35.000000000 +0200 +++ src/http.erl 2009-07-08 18:53:47.000000000 +0200 @@ -284,6 +284,7 @@ HTTPRecordOptions = http_options(HTTPOptions, #http_options{}), Sync = proplists:get_value(sync, Options, true), + ReplyPid = proplists:get_value(reply_pid, Options, self()), NewHeaders = lists:map(fun({Key, Val}) -> {http_util:to_lower(Key), Val} end, Headers), @@ -296,7 +297,7 @@ _ -> RecordHeaders = header_record(NewHeaders, #http_request_h{}, Host, Version), - Request = #request{from = self(), + Request = #request{from = ReplyPid, scheme = Scheme, address = {Host,Port}, path = Path, The documentation would need to be updated too, but I'm not sure where to patch that. -- Vincent de Phily Mobile Devices +33 (0) 666 301 306 +33 (0) 142 119 325 Warning This message (and any associated files) is intended only for the use of its intended recipient and may contain information that is confidential, subject to copyright or constitutes a trade secret. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this message, or files associated with this message, is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. Any views or opinions presented are solely those of the author vincent.dephily@REDACTED and do not necessarily represent those of the company. Although the company has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. From dizzyd@REDACTED Wed Jul 8 20:00:15 2009 From: dizzyd@REDACTED (Dave Smith) Date: Wed, 8 Jul 2009 12:00:15 -0600 Subject: [erlang-questions] ets vs process-based registry + local vs global dispatch In-Reply-To: <2448FB09-DBD0-498D-A0B1-9BFC2F6020FF@gmail.com> References: <2448FB09-DBD0-498D-A0B1-9BFC2F6020FF@gmail.com> Message-ID: You might want to look at Ulf Wiger's "proc" app -- I think it's already solved this problem (at least within the scope of a single node). My github clone of it is here: http://github.com/dizzyd/proc/tree/master Originally found in: http://jungerl.sourceforge.net/ Somewhere, I think Ulf has a SVN repo as well -- but can't remember where. D. On Wed, Jul 8, 2009 at 11:53 AM, Joel Reymont wrote: > Are there any performance numbers on using ETS to keep track of processes vs > using a process-based registry? What about local vs global gen_server casts? > > I'm keeping track of processes associated with session tokens. I can > globally register a process with a name that includes any term. I can only > use an atom when registering a process locally. > > I suspect that sending messages to globally registered processes ({global, > {client_proxy, Token}}) is slowing me down quite a bit. I'm thinking of > keeping processes in a token->pid map but I'd like to know if ETS is > generally recommended over a process wrapping a dict or gb_tree map. > > I'm going to run tests and get some hard numbers. I'm wondering if there's > an existing body of knowledge on this, though. > > ? ? ? ?Thanks, Joel > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ulf.wiger@REDACTED Wed Jul 8 20:12:53 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 08 Jul 2009 20:12:53 +0200 Subject: [erlang-questions] ets vs process-based registry + local vs global dispatch In-Reply-To: References: <2448FB09-DBD0-498D-A0B1-9BFC2F6020FF@gmail.com> Message-ID: <4A54E1A5.6070205@erlang-consulting.com> Dave Smith wrote: > You might want to look at Ulf Wiger's "proc" app -- I think it's > already solved this problem (at least within the scope of a single > node). > > My github clone of it is here: http://github.com/dizzyd/proc/tree/master > > Originally found in: http://jungerl.sourceforge.net/ Perhaps even better to look at gproc at http://svn.ulf.wiger.net/gproc/ In the paper (found in the doc/ directory) I have some benchmark figures, which among other things illustrate that global is very slow. Ets is faster than dict, gb_trees, etc. esp in this case, where the registry is kept by a central server anyway, and the objects stored are small. At least, this is the conventional wisdom. If the access patterns to the ETS table include many processes, and the system is running on multi-core, I'd say ETS performance is a moving target. The new concurrency support for ETS added in R13B01 may improve matters for an application like gproc, but I don't know how much. I chose ETS mainly because I wanted to use the select() behaviour for extracting sets of names. At Ericsson, we found it fast enough for our purposes, but for processes that register many shared attributes, it's a very good idea to register them in bulk, rather than going to the server once for every attribute. BR, Ulf W > > Somewhere, I think Ulf has a SVN repo as well -- but can't remember where. > > D. > > On Wed, Jul 8, 2009 at 11:53 AM, Joel Reymont wrote: >> Are there any performance numbers on using ETS to keep track of processes vs >> using a process-based registry? What about local vs global gen_server casts? >> >> I'm keeping track of processes associated with session tokens. I can >> globally register a process with a name that includes any term. I can only >> use an atom when registering a process locally. >> >> I suspect that sending messages to globally registered processes ({global, >> {client_proxy, Token}}) is slowing me down quite a bit. I'm thinking of >> keeping processes in a token->pid map but I'd like to know if ETS is >> generally recommended over a process wrapping a dict or gb_tree map. >> >> I'm going to run tests and get some hard numbers. I'm wondering if there's >> an existing body of knowledge on this, though. >> >> Thanks, Joel >> >> --- >> Mac hacker with a performance bent >> http://www.linkedin.com/in/joelreymont >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From joelr1@REDACTED Wed Jul 8 20:15:50 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 8 Jul 2009 19:15:50 +0100 Subject: [erlang-questions] ets vs process-based registry + local vs global dispatch In-Reply-To: <4A54E1A5.6070205@erlang-consulting.com> References: <2448FB09-DBD0-498D-A0B1-9BFC2F6020FF@gmail.com> <4A54E1A5.6070205@erlang-consulting.com> Message-ID: <02426D8D-9CFE-4543-9D6F-63E25C36AE4A@gmail.com> On Jul 8, 2009, at 7:12 PM, Ulf Wiger wrote: > Perhaps even better to look at gproc at > http://svn.ulf.wiger.net/gproc/ Darn, just as I added proc from github as a submodule :-(. Is there a substantial difference between proc and gproc if all I want to do is register processes locally using any Erlang term? I assume iteration works the same in both proc and gproc. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From formido@REDACTED Wed Jul 8 22:01:30 2009 From: formido@REDACTED (Michael Terry) Date: Wed, 8 Jul 2009 13:01:30 -0700 Subject: Can't connect to mysql Message-ID: <3e9445610907081301n6a982bb6g64cc61b53543e659@mail.gmail.com> I'm trying to connect to a mysql database like so: C = "Driver={MySQL ODBC 3.51 Driver};Server=sub.domain.com;Database=db_name;User=db_user;Password=my_password;Option=3;". odbc:connect(C, []). ...and all I get is: {error,"No SQL-driver information available. Connection to database failed."} I can connect to the db from mysql client. I've been over the server address, user, and password many, many times. I'm on CentOS and have unixODBC-devel and mysql-connector-odbc installed. I can't think what else to try to troubleshoot. What are possible causes for this? Is there a native erlang mysql adapter somewhere? I couldn't find one with examples or documentation. Michael From wolfmanjm@REDACTED Wed Jul 8 22:23:13 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Wed, 8 Jul 2009 13:23:13 -0700 (PDT) Subject: tcp_close during async accept In-Reply-To: References: <200907081239.n68Cd6lc019001@mail.pharos-avantgard.com> <0e972a47-e8ac-41e8-a9c9-2bc8190928d1@b25g2000prb.googlegroups.com> Message-ID: Oh BTW my server is running on a recent Linux Kernel... Erlang R13B (erts-5.7.1) [source] [smp:2:2] [rq:2] [async-threads:10] [hipe] [kernel-poll:true] So maybe it is an OS/X thing? On Jul 8, 9:27?am, Joel Reymont wrote: > On Jul 8, 2009, at 4:19 PM, Jim Morris wrote: > > > Are you testing client and server on same machine? > > Yes. > > > if not are they on > > the same network? Is there something in between that is doing TCP SYN > > attack mitigation? > > I believe I saw the same effect on Amazon EC2. > > > One thing I did find is my tester client had to have a very short > > delay between the tcp opens, around 10ms was enough to significantly > > speed up the test. > > I'm reconnecting after a random delay when this happens. I was ? > wondering why it's a successful connection on the client side and then ? > a tcp_close since it's not even getting to the Erlang accept on the ? > server side. > > ? ? ? ? Thanks, Joel > > --- > Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From anton.krasovsky@REDACTED Wed Jul 8 23:30:07 2009 From: anton.krasovsky@REDACTED (Anton Krasovsky) Date: Wed, 8 Jul 2009 22:30:07 +0100 Subject: [erlang-questions] Can't connect to mysql In-Reply-To: <3e9445610907081301n6a982bb6g64cc61b53543e659@mail.gmail.com> References: <3e9445610907081301n6a982bb6g64cc61b53543e659@mail.gmail.com> Message-ID: <46167e6a0907081430k33ca927xfed6ec78527c9dae@mail.gmail.com> Try that one http://code.google.com/p/erlang-mysql-driver/ Also make sure your mysqld does not have skip-networking set, as it works over TCP/IP only. anton On Wed, Jul 8, 2009 at 9:01 PM, Michael Terry wrote: > I'm trying to connect to a mysql database like so: > > C = "Driver={MySQL ODBC 3.51 > Driver};Server=sub.domain.com;Database=db_name;User=db_user;Password=my_password;Option=3;". > odbc:connect(C, []). > > ...and all I get is: > > {error,"No SQL-driver information available. Connection to database failed."} > > I can connect to the db from mysql client. I've been over the server > address, user, and password many, many times. I'm on CentOS and have > unixODBC-devel and mysql-connector-odbc installed. I can't think what > else to try to troubleshoot. > > What are possible causes for this? Is there a native erlang mysql > adapter somewhere? I couldn't find one with examples or documentation. > > Michael > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From paul-trapexit@REDACTED Wed Jul 8 23:59:24 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Wed, 8 Jul 2009 14:59:24 -0700 (PDT) Subject: type specs and mnesia selects In-Reply-To: References: Message-ID: from the silence i assume no. for the (search engine) record, i ended up augmenting the type specs with sum types which included the (first few) variable atoms. this weakens the type enforcement in other parts of the code, and i mused having two headers, one external and one for (mnesia) internal implementation, but laziness kicked in. it's not all bad ... by not allowing variable binding in the first element of various tuple keys, i can forbid full table scans using the type system. cheers, -- p p.z. dialyzer rulz! On Tue, 7 Jul 2009, Paul Mineiro wrote: > folks, > > i was being a good citizen and adding type specifications to my records. > this caused dialyzer to complain because in my mnesia:select/2,3,4 calls I > am binding a member of the record to a variable (which is an atom like > '$1') and this was violating the type spec. > > are there any workarounds, other than augmenting or dropping the type > spec? > > -- p > > From javierparis@REDACTED Wed Jul 8 23:42:55 2009 From: javierparis@REDACTED (=?ISO-8859-1?Q?Javier_Par=EDs_Fern=E1ndez?=) Date: Wed, 8 Jul 2009 23:42:55 +0200 Subject: [erlang-questions] Re: tcp_close during async accept In-Reply-To: References: <200907081239.n68Cd6lc019001@mail.pharos-avantgard.com> <0e972a47-e8ac-41e8-a9c9-2bc8190928d1@b25g2000prb.googlegroups.com> Message-ID: <4C3B81C9-2AEE-441C-923D-9204F5BC75CF@udc.es> El 08/07/2009, a las 18:27, Joel Reymont escribi?: > > I'm reconnecting after a random delay when this happens. I was > wondering why it's a successful connection on the client side and > then a tcp_close since it's not even getting to the Erlang accept on > the server side. The easiest way to debug these kind of errors is to have a look at the actual packet trace with a sniffer. Look at wether the server is sending syn+acks to all the connection attempts. You can also look for rst that may explain that "open in the client but immediately close" behaviour. Regarding what Oscar suggested about syn cookies, they are indeed a syn flood prevention technique, but they do not actually drop any connection so even if it is being used here it should not be an issue. You can guess if it is indeed used by the server if you see that some of the syn+acks replies it sends do not have any tcp option. Regards. From steven.charles.davis@REDACTED Thu Jul 9 00:17:38 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 8 Jul 2009 15:17:38 -0700 (PDT) Subject: tcp_close during async accept In-Reply-To: References: Message-ID: <6123c2f3-ab99-43e7-bf4b-8f935958357a@h18g2000yqj.googlegroups.com> Joel et al. I'm not sure if this is related but I'm seeing the same thing about 30-50$ of the time on a client socket connection when using... ok = inet:setopts(Socket, [{packet, http_bin}, {keepalive, true}]), case gen_tcp:recv(Socket, 0, Timeout) of ... ...possible bug in R13B01, or perhaps a bug in my usage of the socket? /sd On Jul 6, 11:20?am, Joel Reymont wrote: > I'm playing with Serge's "Non-blocking TCP server using OTP ? > principles" [1]. When spawning a large number of connections almost at ? > once, I'm seeing tcp_close messages on the client end, even before ? > seeing inet_async on the client end. > > I may spawn 700 client connections and only half may go through, even ? > though my backlog option is 1024, e.g. > > ? ? ?Opts = [binary, > ? ? ? ? ? ? ?{packet, 0}, > ? ? ? ? ? ? ?{reuseaddr, true}, > ? ? ? ? ? ? ?{keepalive, true}, > ? ? ? ? ? ? ?{backlog, 1024}, > ? ? ? ? ? ? ?{active, false}], > ? ? ?case gen_tcp:listen(Port, Opts) of > ? ? ? ? ?{ok, Socket} -> > ? ? ? ? ? ? ?%% Create first accepting process > ? ? ? ? ? ? ?{ok, Ref} = prim_inet:async_accept(Socket, -1), > ... > > Any suggestions on what's going on? > From bob@REDACTED Thu Jul 9 00:26:34 2009 From: bob@REDACTED (Bob Ippolito) Date: Wed, 8 Jul 2009 15:26:34 -0700 Subject: ERTS_FP_CHECK_INIT at 0x4ac84b: detected unhandled FPE at 0x2 Message-ID: <6a36e7290907081526y5b353f9ared4e9e4d64dd22ec@mail.gmail.com> We've started upgrading from R12B-3 to R13B01 on some of our nodes and we have seen this sporadically: ERTS_FP_CHECK_INIT at 0x4ac84b: detected unhandled FPE at 0x2 Has anyone seen this before? Is this a kernel bug, VM bug, CPU bug? I haven't been able to reliably reproduce it and I haven't yet seen any crashes that seem to be directly associated with it. Linux version 2.6.18-92.el5 (mockbuild@REDACTED) (gcc version 4.1.2 20071124 (Red Hat 4.1.2-42)) #1 SMP Tue Jun 10 18:51:06 EDT 2008 Erlang R13B01 (erts-5.7.2) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:4] [hipe] [kernel-poll:true] vendor_id : AuthenticAMD cpu family : 15 model : 65 model name : Dual-Core AMD Opteron(tm) Processor 2212 stepping : 3 -bob From torben.lehoff@REDACTED Thu Jul 9 00:28:34 2009 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Thu, 9 Jul 2009 00:28:34 +0200 Subject: Extracting constants from a header file Message-ID: Hi, I would like to get a hold of all the -define(CONSTANT,42). constant definitions so I can output them for use in an external tool. I have played with epp_dodger, erl_syntax_lib et al, but I have not found an easy way of doing it. Right now the following approach seems to be what I must do: 1. Get the AST using epp:dodger:parse_file/1 2. Filter all the elements from the AST where erl_syntax_lib:analyze_attribute/1 returns preprocessor 3. Take the subtrees of the each element from the step above (using erl_syntaxt_lib:subtrees/1) and then use the fact that 1. the first subtree is a list that contains {atom,_,define} and 2. the second subtree is a list consisting of {var,_,'PDU_TYPE'} and {integer,_,0} (i.e., the constant's name and type+value) Is this the one and only way of doing it? Not that I would mind terribly to code the above, but it seems like something that ought to be easier and/or have been solved by someone else before me. Cheers, Torben -- http://www.linkedin.com/in/torbenhoffmann From rvirding@REDACTED Thu Jul 9 00:46:47 2009 From: rvirding@REDACTED (Robert Virding) Date: Thu, 9 Jul 2009 00:46:47 +0200 Subject: [erlang-questions] Why doesn't erl_eval compile code? In-Reply-To: References: <3dbc6d1c0907051611s2a7b1aa5m1a391e0bae613c8f@mail.gmail.com> Message-ID: <3dbc6d1c0907081546r7c9d2fa5oc706b510329e4153@mail.gmail.com> Sorry, don't know about soft_purge. In my time we were hard. :-) The basic problem is that you are trying to use the code system in a way we never intended. To do this properly as you intend would mean a rewrite of the code system, which would probably result in changes all over the place. But it would be more beautiful. :-( Robert 2009/7/7 Tony Arcieri > On Sun, Jul 5, 2009 at 5:11 PM, Robert Virding wrote: > >> - If you remove the module when you are done, which you must explicitly >> do, then there could be some problem with funs if you have passed them on >> and they still are in use as they will not work after their module has been >> removed. > > > So here's a question... if I code:soft_purge a module which was used to > create funs which are still in scope in some process somewhere, will that > call to soft_purge fail with false? > > If so, as a hack I could have a "garbage collector" process which tries to > soft purge the temporary modules periodically until it succeeds. > > This is a far uglier solution that I thought it would be when I started > out, though :( > > Would it work though? > > -- > Tony Arcieri > medioh.com > From tony@REDACTED Thu Jul 9 00:58:39 2009 From: tony@REDACTED (Tony Arcieri) Date: Wed, 8 Jul 2009 16:58:39 -0600 Subject: [erlang-questions] Why doesn't erl_eval compile code? In-Reply-To: <3dbc6d1c0907081546r7c9d2fa5oc706b510329e4153@mail.gmail.com> References: <3dbc6d1c0907051611s2a7b1aa5m1a391e0bae613c8f@mail.gmail.com> <3dbc6d1c0907081546r7c9d2fa5oc706b510329e4153@mail.gmail.com> Message-ID: On Wed, Jul 8, 2009 at 4:46 PM, Robert Virding wrote: > The basic problem is that you are trying to use the code system in a way we > never intended. Yes, at times I feel like I'm in square peg round hole territory. Usually when I reach this point I try to take a step back and do things in an Erlangier way. I now see the reasoning behind why Erlang implements eval metacircularly. However, if there's one thing that sets scripting languages apart from other languages I believe it's extensive use of eval and runtime code generation/execution. In that regard you might call Lisp the world's first scripting language (although I believe that honor is typically attributed to awk) However, it's a lot easier to write a metacircular evaluator for a Lisp than for something like Reia... > To do this properly as you intend would mean a rewrite of the code system, > which would probably result in changes all over the place. But it would be > more beautiful. :-( > I'm okay with hacks as long as they work :) -- Tony Arcieri medioh.com From rvirding@REDACTED Thu Jul 9 01:10:59 2009 From: rvirding@REDACTED (Robert Virding) Date: Thu, 9 Jul 2009 01:10:59 +0200 Subject: [erlang-questions] comma vs andalso In-Reply-To: <4A549182.4070908@it.uu.se> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> <20090707041622.GB28401@hijacked.us> <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> <769861.4108.qm@web111415.mail.gq1.yahoo.com> <4A549182.4070908@it.uu.se> Message-ID: <3dbc6d1c0907081610u3d0154adrda6db14e648f0889@mail.gmail.com> 2009/7/8 Richard Carlsson > Thomas Lindgren wrote: > > Not to mention the twin set of type tests, introduced, as far as I > > know, to get rid of the lone double entendre of float/1 (as a test or > > conversion not the most frequent of operations). And now, for our > > convenience, the shorter form of these tests is being deprecated. > > Sigh. > > Oh, and I forgot this beauty, which is a consequence of the > doubleplusweird scope rules for the old type tests in guards: > > 1> if integer(42) -> ok; true -> no end. > ok > 2> if integer(42) == true -> ok; true -> no end. > * 1: illegal guard expression > > because, you see, it is only at the absolute top of the guard test > that the name 'integer(X)' can be used to refer to the type test. > Shove it into one side of an '==', and it is no longer recognized. It has nothing to do with scope rules really. In the original guards only simple tests were allowed so there was no problem. It only became weird when expressions were allowed. Life was simpler in the old days! :-) Robert From richardc@REDACTED Thu Jul 9 01:12:31 2009 From: richardc@REDACTED (Richard Carlsson) Date: Thu, 09 Jul 2009 01:12:31 +0200 Subject: [erlang-questions] Extracting constants from a header file In-Reply-To: References: Message-ID: <4A5527DF.40901@it.uu.se> Torben Hoffmann wrote: > Hi, > > I would like to get a hold of all the > -define(CONSTANT,42). > constant definitions so I can output them for use in an external tool. > > I have played with epp_dodger, erl_syntax_lib et al, but I have not found an > easy way of doing it. > > Right now the following approach seems to be what I must do: > > 1. Get the AST using epp:dodger:parse_file/1 > 2. Filter all the elements from the AST where > erl_syntax_lib:analyze_attribute/1 returns preprocessor > 3. Take the subtrees of the each element from the step above (using > erl_syntaxt_lib:subtrees/1) and then use the fact that > 1. the first subtree is a list that contains {atom,_,define} and > 2. the second subtree is a list consisting of {var,_,'PDU_TYPE'} and > {integer,_,0} (i.e., the constant's name and type+value) I'd call that pretty much an easy way of doing it. In step 3, you might want to use the functions in erl_syntax all the way instead of breaking the abstraction and matching directly on tuples, in particular to decompose the attribute (rather than using subtrees/1). > Is this the one and only way of doing it? You could do your own parsing of the token stream and detect forms that begin with '-','define','(', then extract the interesting tokens and output them in whatever format you want. Might or might not be easier for you. /Richard From wolfmanjm@REDACTED Thu Jul 9 01:16:15 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Wed, 8 Jul 2009 16:16:15 -0700 (PDT) Subject: tcp_close during async accept In-Reply-To: <6123c2f3-ab99-43e7-bf4b-8f935958357a@h18g2000yqj.googlegroups.com> References: <6123c2f3-ab99-43e7-bf4b-8f935958357a@h18g2000yqj.googlegroups.com> Message-ID: I doubt this is the problem, but I notice you both use tcp keep alive. In the past this option has had issues in various TCP stacks, and if my memory serves me correctly its use was frowned upon. That may be old data, but I have been writing servers for many many years and have never used that option. I prefer to have the application protocol include some type of ping that the client sends to the server every 30 seconds, this keeps any intermediate NAT/Firewall box from losing the connection, and allows the server to know when a client has vanished due to a cable unplug or router going away (ie it never sees the FIN or RST). This works well for HTTP long-poll type clients as well as proprietary clients. On Jul 8, 3:17?pm, Steve Davis wrote: > Joel et al. > > I'm not sure if this is related but I'm seeing the same thing about > 30-50$ of the time on a client socket connection when using... > > ? ok = inet:setopts(Socket, [{packet, http_bin}, {keepalive, true}]), > ? case gen_tcp:recv(Socket, 0, Timeout) of > ? ... > > ...possible bug in R13B01, or perhaps a bug in my usage of the socket? > > /sd > > On Jul 6, 11:20?am, Joel Reymont wrote: > > > > > I'm playing with Serge's "Non-blocking TCP server using OTP ? > > principles" [1]. When spawning a large number of connections almost at ? > > once, I'm seeing tcp_close messages on the client end, even before ? > > seeing inet_async on the client end. > > > I may spawn 700 client connections and only half may go through, even ? > > though my backlog option is 1024, e.g. > > > ? ? ?Opts = [binary, > > ? ? ? ? ? ? ?{packet, 0}, > > ? ? ? ? ? ? ?{reuseaddr, true}, > > ? ? ? ? ? ? ?{keepalive, true}, > > ? ? ? ? ? ? ?{backlog, 1024}, > > ? ? ? ? ? ? ?{active, false}], > > ? ? ?case gen_tcp:listen(Port, Opts) of > > ? ? ? ? ?{ok, Socket} -> > > ? ? ? ? ? ? ?%% Create first accepting process > > ? ? ? ? ? ? ?{ok, Ref} = prim_inet:async_accept(Socket, -1), > > ... > > > Any suggestions on what's going on? > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From rvirding@REDACTED Thu Jul 9 01:28:23 2009 From: rvirding@REDACTED (Robert Virding) Date: Thu, 9 Jul 2009 01:28:23 +0200 Subject: [erlang-questions] comma vs andalso In-Reply-To: <769861.4108.qm@web111415.mail.gq1.yahoo.com> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> <20090707041622.GB28401@hijacked.us> <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> <769861.4108.qm@web111415.mail.gq1.yahoo.com> Message-ID: <3dbc6d1c0907081628j587cb1dfm4f398120e822e126@mail.gmail.com> 2009/7/8 Thomas Lindgren > ----- Original Message ---- > > From: Richard O'Keefe > > > > Let's take two apparently similar functions: > > > > f(X) when is_atom(element(3, X)) ; true -> 42. > > > > g(X) when is_atom(element(3, X)) orelse true -> 42. > > > > By actual test, f(99) -> 42. > > By actual test, g(99) -> a function_clause exception. > > > > That is, in a guard, an expression using 'andalso' or > > 'orelse' is still an expression, and if an exception > > occurs in the left-hand operand, the whole expression > > is still skipped. > .... > > I conclude that > > (1) you had best be EXTREMELY cautious about using 'andalso' > > and 'orelse' in guards; they are NOT drop-in replacements > > for ',' and ';', and > > (2) it is long past time that Erlang allowed nested use of > > ',' and ';' in guards. > > Guards are an awful mess in Erlang. I'd be quite surprised if the above > difference in behaviour was intentional. If so, what possible purpose does > it serve? > The above difference is intentional! There is one fundamental and important difference between using orelse/or or ';' and that is in the case of exceptions. If an exception occurs in the case: G1 ; G2 ; G3 ; ... say in G1 then G1 will fail and G2 will be attempted, as an exception is equivalent to failure. This is logical as they are separate guards. However, if an exception occurs in: G1 orelse G2 orelse G3 ... in G1 again then the WHOLE guard expression will fail, G2 will never be tried as it is part of the same expression. This IS intentional as using ';' is exactly equivalent to separate clauses with the same RHS. So f(...) when G1 ; G2 ; G3 -> . is the same as: f(...) when G1 -> ; f(...) when G2 -> ; f(...) when G3 -> . This functionality was added for just this case. There is logic in the madness. It was simple before with only simple guard tests, it only became confusing when logical expressions were allowed in guards. They are practical but were only added later, THEY are the odd man out. This is also why originally there were no problems with type tests without is_ in guards. Guards were always meant to be part of pattern matching and no more. Robert From steven.charles.davis@REDACTED Thu Jul 9 01:35:26 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 8 Jul 2009 16:35:26 -0700 (PDT) Subject: tcp_close during async accept In-Reply-To: References: <6123c2f3-ab99-43e7-bf4b-8f935958357a@h18g2000yqj.googlegroups.com> Message-ID: Hi Jim, Indeed you are correct. I don't need it and my expectation was wrong about its underlying meaning! I was just coming back to post that the bug was in my brain/code and not the platform code. Thank you for response (sic!). Regards, Steve On Jul 8, 6:16?pm, Jim Morris wrote: > I doubt this is the problem, but I notice you both use tcp keep alive. > In the past this option has had issues in various TCP stacks, and if > my memory serves me correctly its use was frowned upon. That may be > old data, but I have been writing servers for many many years and have > never used that option. > > I prefer to have the application protocol include some type of ping > that the client sends to the server every 30 seconds, this keeps any > intermediate NAT/Firewall box from losing the connection, and allows > the server to know when a client has vanished due to a cable unplug or > router going away (ie it never sees the FIN or RST). This works well > for HTTP long-poll type clients as well as proprietary clients. > > On Jul 8, 3:17?pm, Steve Davis wrote: > > > > > > > Joel et al. > > > I'm not sure if this is related but I'm seeing the same thing about > > 30-50$ of the time on a client socket connection when using... > > > ? ok = inet:setopts(Socket, [{packet, http_bin}, {keepalive, true}]), > > ? case gen_tcp:recv(Socket, 0, Timeout) of > > ? ... > > > ...possible bug in R13B01, or perhaps a bug in my usage of the socket? > > > /sd > > > On Jul 6, 11:20?am, Joel Reymont wrote: > > > > I'm playing with Serge's "Non-blocking TCP server using OTP ? > > > principles" [1]. When spawning a large number of connections almost at ? > > > once, I'm seeing tcp_close messages on the client end, even before ? > > > seeing inet_async on the client end. > > > > I may spawn 700 client connections and only half may go through, even ? > > > though my backlog option is 1024, e.g. > > > > ? ? ?Opts = [binary, > > > ? ? ? ? ? ? ?{packet, 0}, > > > ? ? ? ? ? ? ?{reuseaddr, true}, > > > ? ? ? ? ? ? ?{keepalive, true}, > > > ? ? ? ? ? ? ?{backlog, 1024}, > > > ? ? ? ? ? ? ?{active, false}], > > > ? ? ?case gen_tcp:listen(Port, Opts) of > > > ? ? ? ? ?{ok, Socket} -> > > > ? ? ? ? ? ? ?%% Create first accepting process > > > ? ? ? ? ? ? ?{ok, Ref} = prim_inet:async_accept(Socket, -1), > > > ... > > > > Any suggestions on what's going on? > > > ________________________________________________________________ > > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From wolfmanjm@REDACTED Thu Jul 9 01:55:50 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Wed, 8 Jul 2009 16:55:50 -0700 (PDT) Subject: tcp_close during async accept In-Reply-To: References: <200907081239.n68Cd6lc019001@mail.pharos-avantgard.com> <003101c9ffe3$32f0a2e0$706bd90a@mtn.co.za> Message-ID: <2d519385-61f1-428e-9ff3-69cce625d2fe@x25g2000prf.googlegroups.com> Joel, one other thing to try is use [kernel-poll:true] by adding +K true to the erl command line, on both the client and server, see if that helps. I just double checked and I can open 1000 virtually simultaneous connections (sleep(1) between each open on client) using the code from http://www.trapexit.org/Building_a_Non-blocking_TCP_server_using_OTP_principles with no problems on my Linux box using +K true. (Haven't tried without that though). On Jul 8, 9:29?am, Joel Reymont wrote: > On Jul 8, 2009, at 4:45 PM, Trevor Woollacott wrote: > > > Thats true, running out of file descriptors is likely. If that is ? > > the case then you should be seeing EMFILE or ENFILE errors. > > This is not the case in my test, although I did report a related issue: > > Mac OSX 10.5.7 > > ulimit -n > 10240 > > Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] ? > [kernel-poll:true] > > =ERROR REPORT==== 6-Jul-2009::20:51:54 === > driver_select(0x00000041, 1024, ERL_DRV_WRITE ERL_DRV_USE, 1) by ? > tcp_inet driver #Port<0.3137> failed: fd=1024 is larger than the ? > largest allowed fd=1023 > > =ERROR REPORT==== 6-Jul-2009::20:51:55 === > File operation error: system_limit. Target: s3erl/ebin/random.beam. ? > Function: get_file. Process: code_server. > > --- > Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From kevin@REDACTED Thu Jul 9 01:51:34 2009 From: kevin@REDACTED (Kevin A. Smith) Date: Wed, 8 Jul 2009 19:51:34 -0400 Subject: 64-bit Linked-In Drivers Message-ID: <40B65E54-A984-4896-8E90-D4639FC7600C@hypotheticalabs.com> I'm looking for some pointers on compiling linked-in drivers on OS X with 64 bit Erlang. On true 64-bit Macs calls to erl_ddll:load/2 fail. Unfortunately I'm not able to reproduce these failures locally so I don't have more details about the actual error. I've attached build output for both 32 and 64 bit builds. I haven't been able to reproduce the problem on Linux which I'm more familiar with. I'd appreciate any pointer or suggestions from those more savvy than I. --Kevin -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: build64.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: build32.txt URL: From joelr1@REDACTED Thu Jul 9 02:18:21 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 9 Jul 2009 01:18:21 +0100 Subject: [erlang-questions] Re: tcp_close during async accept In-Reply-To: <2d519385-61f1-428e-9ff3-69cce625d2fe@x25g2000prf.googlegroups.com> References: <200907081239.n68Cd6lc019001@mail.pharos-avantgard.com> <003101c9ffe3$32f0a2e0$706bd90a@mtn.co.za> <2d519385-61f1-428e-9ff3-69cce625d2fe@x25g2000prf.googlegroups.com> Message-ID: Jim, Take a look at the Erlang prompt right below where ulimit -n gives 10240, i.e. >> Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads: >> 0] >> [kernel-poll:true] I thought this means I do have kernel poll enabled. I should since I pass +K. The part that I'm surprised about is that select is still used. On Jul 9, 2009, at 12:55 AM, Jim Morris wrote: > Joel, one other thing to try is use [kernel-poll:true] by adding +K > true to the erl command line, on both the client and server, see if > that helps. > > I just double checked and I can open 1000 virtually simultaneous > connections (sleep(1) between each open on client) using the code from > http://www.trapexit.org/Building_a_Non-blocking_TCP_server_using_OTP_principles > with no problems on my Linux box using +K true. (Haven't tried without > that though). > > > > On Jul 8, 9:29 am, Joel Reymont wrote: >> On Jul 8, 2009, at 4:45 PM, Trevor Woollacott wrote: >> >>> Thats true, running out of file descriptors is likely. If that is >>> the case then you should be seeing EMFILE or ENFILE errors. >> >> This is not the case in my test, although I did report a related >> issue: >> >> Mac OSX 10.5.7 >> >> ulimit -n >> 10240 >> >> Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads: >> 0] >> [kernel-poll:true] >> >> =ERROR REPORT==== 6-Jul-2009::20:51:54 === >> driver_select(0x00000041, 1024, ERL_DRV_WRITE ERL_DRV_USE, 1) by >> tcp_inet driver #Port<0.3137> failed: fd=1024 is larger than the >> largest allowed fd=1023 >> >> =ERROR REPORT==== 6-Jul-2009::20:51:55 === >> File operation error: system_limit. Target: s3erl/ebin/random.beam. >> Function: get_file. Process: code_server. >> >> --- >> Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont >> >> ________________________________________________________________ >> erlang-questions mailing list. Seehttp://www.erlang.org/faq.html >> erlang-questions (at) erlang.org > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From matt@REDACTED Thu Jul 9 04:59:40 2009 From: matt@REDACTED (Matt Reynolds) Date: Wed, 8 Jul 2009 19:59:40 -0700 Subject: [erlang-questions] Why doesn't ssh module start crypto module? In-Reply-To: <6a3ae47e0907070639i2566450fnb3c7597fa8094306@mail.gmail.com> References: <6a3ae47e0907070639i2566450fnb3c7597fa8094306@mail.gmail.com> Message-ID: Thanks all for the answers! On Tue, Jul 7, 2009 at 6:39 AM, Robert Raschke wrote: > Also note that if you build an OTP release, then the boot script usually > takes care of starting all your applications in the right order. If you're > not using releases, then you need to do that yourself. > > Robby > > On Tue, Jul 7, 2009 at 2:05 PM, Niclas Eklund wrote: > >> >> Hello! >> >> The 'applications' value shall list other applications that the application >> depends on. If the required applicatios hans't been started an error message >> is returned. >> >> Why doesn't SSH start crypto? For example might other applications also >> depend on crypto and the required type (permanent | transient | temporary) >> may differ. Another reason is that many applications export specific start >> functions (Mnesia, SSL, Orber etc) which in some cases require configuration >> parameters. Typically these functions do more stuff than just invoking >> application:start/1/2. Hence, you should use ssh:start/1/2 instead. >> >> Also consider that if application A depends on that B is started. If B is >> updated and require that application C i started prior to B. In that case A >> must be patched so that it starts not only B but also C. I hope you see my >> point why SSH shall not start Crypto ;-) >> >> The short version, you must start the other applications in your code. >> >> Best Regards, >> >> Niclas @ Erlang/OTP >> >> >> >> On Mon, 6 Jul 2009, Matt Reynolds wrote: >> >> ?Am I misunderstanding the use of the "applications" atom in the >>> Applications configuration >>> (http://erlang.org/doc/design_principles/applications.html)? >>> >>> Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] >>> [kernel-poll:false] >>> >>> Eshell V5.7.2 ?(abort with ^G) >>> 1> application:start(ssh). >>> {error,{not_started,crypto}} >>> >>> Does the "applications" configuration simply specify what must be >>> started first, without starting those modules? ?So the Application >>> specifying dependencies must start it itself? ?If so, why doesn't SSH >>> start crypto? >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > From kaiduanx@REDACTED Thu Jul 9 06:25:06 2009 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Thu, 9 Jul 2009 00:25:06 -0400 Subject: Trace a particular record with dbg:fun2ms Message-ID: Hi, Suppose I have a function which takes a record Person as parameter, how to write the fun2ms() to trace the call for a particular value? -record(person, {firstname, lastname}). test(Person) -> ... For example, I would like to trace the call where the firstname is erlanger, how to write fun2ms? I tried as below, 6> dbg:fun2ms(fun(#person{firstname = F, lastname = L}) when F == "erlanger" -> true end). Error: dbg:fun2ms requires fun with single variable or list parameter {error,transform_error} 7> Thanks, kaiduan From ok@REDACTED Thu Jul 9 07:32:27 2009 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 9 Jul 2009 17:32:27 +1200 Subject: [erlang-questions] comma vs andalso In-Reply-To: <4A548BAD.1030401@it.uu.se> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> <20090707041622.GB28401@hijacked.us> <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> <769861.4108.qm@web111415.mail.gq1.yahoo.com> <4A548BAD.1030401@it.uu.se> Message-ID: <461A85DE-BF04-44FC-B0CF-FFE63979FAB2@cs.otago.ac.nz> Let's see if we can make this clear: `and` `or` `andalso` and `orelse` are allowed ANYWHERE in a guard expression are are treated IDENTICALLY to the way they are treated elsewhere. Of those things that are accepted at all in guards, ONLY ',' and ';' are treated differently. X , Y acts like true=X, Y X ; Y acts a bit like case (catch X) of true -> true ; _ -> Y end. > That was one details, yes, but the main reason was to make it possible > to refactor a piece of code without being forced to change the names > of the function tests if you moved an expression from within a guard > to an ordinary expression and vice versa. It has never been clear to me why this was considered important. It's not something I often do in Haskell or Clean, where guards _are_ perfectly ordinary expressions, yet I refactor Haskell code all the time. > Recall that before the is_... > versions, there was no way of saying e.g., "Bool = (is_integer(X) and > is_atom(Y))" without writing a case/if or a separate predicate > function. It's _still_ the case that almost no non-trivial expressions can be moved into a guard. > and with any existing > code that defined a function such as list(X) or integer(X)), And the new names created clashes with any previously existing code that defined a function such as is_list(X). In fact, given the obvious rule if a module defines or explicitly imports a function F/N that matches the name of a built-in function, emit a warning, and for that module pretend that the built-in function does not exist no code would have been affected at all, other than to get new *warning* messages. A language may need to add new built-ins at any time. Erlang has often done so. Such a rule is valuable. So the argument for the "is_" prefix is a non-sequitur: it was neither necessary nor sufficient to avoid code breakage. > As a simple example, this macro nowadays works in both > guard expressions and in general expressions - it didn't back then: > > -define(is_a_foo(X), (is_integer(X) or is_binary(X))). > > (we can even use orelse here, and it will still work in a guard). But we _can't_ write -define(is_a_bar(X), (Y = X*X, (Y < -2 or Y > 2)). and expect _that_ to work in both guards and expressions. [Abstract patterns would solve this problem another way.] > > >> And now, for our convenience, the shorter form of these tests is >> being >> deprecated. > > Hold your horses - nobody is deprecating the use of ',' and ';'. He didn't say that. He meant that the convenient short tests like atom(X) are being deprecated in favour of the long names. > This fail-to-false > behaviour was in my opinion a mistake, because it occasionally hides > a bug in the guard and turns what should have been an observable > runtime > crash into a silent "well, we take the next clause then". There are arguments on both sides here. > > I'm not sure I have any real arguments against nesting of ','/';', > but I fear the grammar could get rather messy, and that such nested > guards could be quite difficult to read in practice. I've been asked to help with a C compiler for a transport- triggered architecture, so my micro-Erlang compiler is on indefinite hold. But I did actually tackle this problem in the grammar. The grammar rules for guards need to be separate from the grammar rules for expressions, but neither gets particularly "messy". If I recall correctly, it took me about half an hour to make the changes. Such guards _could_ be hard to read in practice, but there's no reason they _have_ to be. It is certain that they would make _some_ guards clearer. Consider this example: -define(is_vowel(X), ( X == $a ; X == $e ; X == $i ; X == $o ; X == $u )). count_adjacent_vowels(L) -> count_adjacent_vowels(L, 0). count_adjacent_vowels([V|S=[W|_]], N) when ?is_vowel(V), ?is_vowel(W) -> count_adjacent_vowels(S, N+1); count_adjacent_vowels([_|S], N) -> count_adjacent_vowels(S, N); count_adjacent_vowels([], N) -> N. This could also be done using `orelse`, of course. But what if you want to define guard tests that *can't* be (ab)used as expressions? Right now, we can't even do -define(is_probability(X), ( is_float(X), X >= 0.0, 1.0 >= X )). f(X, Y) when ?is_probability(X), ?is_probability(Y) -> X*Y. \ From wolfmanjm@REDACTED Thu Jul 9 08:32:54 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Wed, 8 Jul 2009 23:32:54 -0700 (PDT) Subject: tcp_close during async accept In-Reply-To: References: <200907081239.n68Cd6lc019001@mail.pharos-avantgard.com> <003101c9ffe3$32f0a2e0$706bd90a@mtn.co.za> <2d519385-61f1-428e-9ff3-69cce625d2fe@x25g2000prf.googlegroups.com> Message-ID: <7af82ef5-1a5d-4aaf-963d-3f495fdb996c@k13g2000prh.googlegroups.com> Sorry I missed that. So the error does make it look like select is being used, did you build the erlang from source or get a package? I suspect that kernel poll is not actually being set because it is not compiled in, and erl is reporting it being turned on when in fact it is turned off. ./configure has some settings which should force kernel poll to be compiled, presuming that the OS/X kernel supports it. > ./configure --enable-kernel-poll check the output to make sure it can find everything it needs to enable that option. I did find that I had to set a few things to get certain features to be compiled in even though they are meant to be default. On Jul 8, 5:18?pm, Joel Reymont wrote: > Jim, > > Take a look at the Erlang prompt right below where ulimit -n gives ? > 10240, i.e. > > >> Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads: > >> 0] > >> [kernel-poll:true] > > I thought this means I do have kernel poll enabled. I should since I ? > pass +K. > > The part that I'm surprised about is that select is still used. > > On Jul 9, 2009, at 12:55 AM, Jim Morris wrote: > > > > > Joel, one other thing to try is use [kernel-poll:true] by adding +K > > true to the erl command line, on both the client and server, see if > > that helps. > > > I just double checked and I can open 1000 virtually simultaneous > > connections (sleep(1) between each open on client) using the code from > >http://www.trapexit.org/Building_a_Non-blocking_TCP_server_using_OTP_... > > with no problems on my Linux box using +K true. (Haven't tried without > > that though). > > > On Jul 8, 9:29 am, Joel Reymont wrote: > >> On Jul 8, 2009, at 4:45 PM, Trevor Woollacott wrote: > > >>> Thats true, running out of file descriptors is likely. If that is > >>> the case then you should be seeing EMFILE or ENFILE errors. > > >> This is not the case in my test, although I did report a related ? > >> issue: > > >> Mac OSX 10.5.7 > > >> ulimit -n > >> 10240 > > >> Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads: > >> 0] > >> [kernel-poll:true] > > >> =ERROR REPORT==== 6-Jul-2009::20:51:54 === > >> driver_select(0x00000041, 1024, ERL_DRV_WRITE ERL_DRV_USE, 1) by > >> tcp_inet driver #Port<0.3137> failed: fd=1024 is larger than the > >> largest allowed fd=1023 > > >> =ERROR REPORT==== 6-Jul-2009::20:51:55 === > >> File operation error: system_limit. Target: s3erl/ebin/random.beam. > >> Function: get_file. Process: code_server. > > >> --- > >> Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont > > >> ________________________________________________________________ > >> erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > >> erlang-questions (at) erlang.org > > > ________________________________________________________________ > > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > --- > Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From raimo+erlang-questions@REDACTED Thu Jul 9 08:38:50 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 9 Jul 2009 08:38:50 +0200 Subject: [erlang-questions] ERTS_FP_CHECK_INIT at 0x4ac84b: detected unhandled FPE at 0x2 In-Reply-To: <6a36e7290907081526y5b353f9ared4e9e4d64dd22ec@mail.gmail.com> References: <6a36e7290907081526y5b353f9ared4e9e4d64dd22ec@mail.gmail.com> Message-ID: <20090709063850.GA31288@erix.ericsson.se> Sounds like this one: http://www.erlang.org/cgi-bin/ezmlm-cgi/3/430 On Wed, Jul 08, 2009 at 03:26:34PM -0700, Bob Ippolito wrote: > We've started upgrading from R12B-3 to R13B01 on some of our nodes and > we have seen this sporadically: > ERTS_FP_CHECK_INIT at 0x4ac84b: detected unhandled FPE at 0x2 > > Has anyone seen this before? Is this a kernel bug, VM bug, CPU bug? I > haven't been able to reliably reproduce it and I haven't yet seen any > crashes that seem to be directly associated with it. > > Linux version 2.6.18-92.el5 (mockbuild@REDACTED) (gcc > version 4.1.2 20071124 (Red Hat 4.1.2-42)) #1 SMP Tue Jun 10 18:51:06 > EDT 2008 > > Erlang R13B01 (erts-5.7.2) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:4] [hipe] [kernel-poll:true] > > vendor_id : AuthenticAMD > cpu family : 15 > model : 65 > model name : Dual-Core AMD Opteron(tm) Processor 2212 > stepping : 3 > > -bob > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From richardc@REDACTED Thu Jul 9 10:02:04 2009 From: richardc@REDACTED (Richard Carlsson) Date: Thu, 09 Jul 2009 10:02:04 +0200 Subject: [erlang-questions] comma vs andalso In-Reply-To: <461A85DE-BF04-44FC-B0CF-FFE63979FAB2@cs.otago.ac.nz> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> <20090707041622.GB28401@hijacked.us> <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> <769861.4108.qm@web111415.mail.gq1.yahoo.com> <4A548BAD.1030401@it.uu.se> <461A85DE-BF04-44FC-B0CF-FFE63979FAB2@cs.otago.ac.nz> Message-ID: <4A55A3FC.8070109@it.uu.se> Richard O'Keefe wrote: > Let's see if we can make this clear: > `and` `or` `andalso` and `orelse` are allowed ANYWHERE > in a guard expression are are treated IDENTICALLY to > the way they are treated elsewhere. Quite. > Of those things that are accepted at all in guards, > ONLY ',' and ';' are treated differently. > > X , Y acts like true=X, Y > X ; Y acts a bit like case (catch X) of true -> true > ; _ -> Y end. A bit, yes. Just recall that even in the ',' case, no exception will ever escape the guard as a whole (not sure if this was implied above). A more exact equivalence could be written: -define(wrap(A), try (A)=:=true catch _:_ -> false end). -define(comma(A,B), (wrap(A) andalso wrap(B))). -define(semi(A,B), (wrap(A) orelse wrap(B))). >> That was one details, yes, but the main reason was to make it possible >> to refactor a piece of code without being forced to change the names >> of the function tests if you moved an expression from within a guard >> to an ordinary expression and vice versa. > > It has never been clear to me why this was considered > important. It's not something I often do in Haskell or > Clean, where guards _are_ perfectly ordinary expressions, > yet I refactor Haskell code all the time. Important... maybe not, but definitely nicer when you do want to do it. It makes it easier to teach the language (even though those who grew up with Erlang may find the changes uncomfortable). And there were as I said also considerations such as why you could talk about erlang:'+'/2 but not erlang:'list(X)'. In all, it was worth doing. >> Recall that before the is_... >> versions, there was no way of saying e.g., "Bool = (is_integer(X) and >> is_atom(Y))" without writing a case/if or a separate predicate function. > > It's _still_ the case that almost no non-trivial expressions > can be moved into a guard. True. But every bit helps. Ive never found "this does not solve all problems" to be an argument for not making a partial improvement (as long as it does not create an obstacle for future development). > And the new names created clashes with any previously existing > code that defined a function such as is_list(X). In fact, > given the obvious rule > > if a module defines or explicitly imports a function > F/N that matches the name of a built-in function, > emit a warning, and for that module pretend that the > built-in function does not exist > > no code would have been affected at all, other than to get > new *warning* messages. A language may need to add new > built-ins at any time. Erlang has often done so. Such a > rule is valuable. Yes, however, there was already a similar rule in place since ancient times, and it stated that in the case you describe, the built-in function takes precedence. Bummer. (We are now slowly trying to phase out this old rule, though, taking baby steps.) > So the argument for the "is_" prefix is a non-sequitur: it > was neither necessary nor sufficient to avoid code breakage. Neither necessary nor sufficient, but likely. It's a game of probabilities. I _had_ seen several existing modules that used the name list(Xs), and float(X) was already in use as a BIF for casting. In comparison, the is_ convention was much less likely to cause clashes (indeed, I recall no reports of any such when we introduced the new names). And the convention has kept working for those type tests that were added later, e.g., is_boolean(X), is_bitstring(X). >>> And now, for our convenience, the shorter form of these tests is being >>> deprecated. >> >> Hold your horses - nobody is deprecating the use of ',' and ';'. > > He didn't say that. He meant that the convenient short tests > like atom(X) are being deprecated in favour of the long names. I'm sorry, I misread. But the "long" names are only 3 more characters, and at least in my opinion, they improve readability. >> This fail-to-false >> behaviour was in my opinion a mistake, because it occasionally hides >> a bug in the guard and turns what should have been an observable runtime >> crash into a silent "well, we take the next clause then". > > There are arguments on both sides here. Absolutely. But I can tell you that it's one of the least fun kind of facepalm-inducing bugs that can happen. I've seen programs that have been running for years and years that now and then took the wrong case but nobody detected it. Or worse, always took the wrong case. > I've been asked to help with a C compiler for a transport- > triggered architecture, so my micro-Erlang compiler is on > indefinite hold. But I did actually tackle this problem > in the grammar. The grammar rules for guards need to be > separate from the grammar rules for expressions, but neither > gets particularly "messy". If I recall correctly, it took > me about half an hour to make the changes. I'll take your word for that. Good to know. > Such guards _could_ be hard to read in practice, but there's > no reason they _have_ to be. It is certain that they would > make _some_ guards clearer. Consider this example: > > -define(is_vowel(X), > ( X == $a ; X == $e ; X == $i ; X == $o ; X == $u )). Could be nice, but I'm not so much worried about what a distinguished computer scientist will do with it, as what will happen in a large body of code written by well-meaning but average programmers. I'm open to persuasion, though. > But what if you want to define guard tests that *can't* > be (ab)used as expressions? I don't quite see the point in that, though. Do you feel a similar urge when programming Haskell, that you'd like to be able to write things that have a meaning in guards only? /Richard From rtrlists@REDACTED Thu Jul 9 10:42:55 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 9 Jul 2009 09:42:55 +0100 Subject: [erlang-questions] Can't connect to mysql In-Reply-To: <3e9445610907081301n6a982bb6g64cc61b53543e659@mail.gmail.com> References: <3e9445610907081301n6a982bb6g64cc61b53543e659@mail.gmail.com> Message-ID: <6a3ae47e0907090142k290d9877r6d0d0db8c254f548@mail.gmail.com> On Wed, Jul 8, 2009 at 9:01 PM, Michael Terry wrote: > I'm trying to connect to a mysql database like so: > > C = "Driver={MySQL ODBC 3.51 > Driver};Server=sub.domain.com > ;Database=db_name;User=db_user;Password=my_password;Option=3;". > odbc:connect(C, []). > > ...and all I get is: > > {error,"No SQL-driver information available. Connection to database > failed."} > Try adding the option {scrollable_cursors, off} . I've had to add this in most of my connections. Not sure this'll "fix" it but worth a try. Robby From sverker@REDACTED Thu Jul 9 11:28:44 2009 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 09 Jul 2009 11:28:44 +0200 Subject: [erlang-questions] kpoll still using select In-Reply-To: References: Message-ID: <4A55B84C.7040303@erix.ericsson.se> Joel Reymont wrote: > Mac OSX 10.5.7 > ulimit -n > 10240 > Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] > [kernel-poll:true] > > =ERROR REPORT==== 6-Jul-2009::20:51:54 === > driver_select(0x00000041, 1024, ERL_DRV_WRITE ERL_DRV_USE, 1) by > tcp_inet driver #Port<0.3137> failed: fd=1024 is larger than the > largest allowed fd=1023 > > =ERROR REPORT==== 6-Jul-2009::20:51:55 === > File operation error: system_limit. Target: s3erl/ebin/random.beam. > Function: get_file. Process: code_server. A late answer: What makes you think select is still used? driver_select will use select, poll or any kernel poll variant depending on configuration and if kernel poll is turn on or not. In case of kernel poll, driver_select may in some cases fall back to use select or poll for individual file descriptors if the OS does not support kernel poll for that device. /Sverker, Erlang/OTP, Ericsson From joelr1@REDACTED Thu Jul 9 11:52:39 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 9 Jul 2009 10:52:39 +0100 Subject: [erlang-questions] kpoll still using select In-Reply-To: <4A55B84C.7040303@erix.ericsson.se> References: <4A55B84C.7040303@erix.ericsson.se> Message-ID: <7A6A571E-9FF2-4736-BBFA-7B09F80A14B1@gmail.com> On Jul 9, 2009, at 10:28 AM, Sverker Eriksson wrote: > In case of kernel poll, driver_select may in some > cases fall back to use select or poll for individual file > descriptors if the > OS does not support kernel poll for that device. ./configure --enable-threads --enable-smp-support --enable-kernel-poll --enable-hipe ... checking for working poll()... broken or based on select() checking whether kqueue() fd can be select()ed on... yes checking whether kernel poll support should be enabled... yes; kqueue ... It should be using kqueue and the Erlang prompt reports that it's using kqueue. In fact, I distinctly remember being able to go to at least 20k connections (descriptors) when banging on OpenPoker last year. It puzzles me greatly that I cannot do better than 1024 this time around. Any suggestions on how to troubleshoot? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From sverker@REDACTED Thu Jul 9 12:08:17 2009 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 09 Jul 2009 12:08:17 +0200 Subject: [erlang-questions] kpoll still using select In-Reply-To: <7A6A571E-9FF2-4736-BBFA-7B09F80A14B1@gmail.com> References: <4A55B84C.7040303@erix.ericsson.se> <7A6A571E-9FF2-4736-BBFA-7B09F80A14B1@gmail.com> Message-ID: <4A55C191.9080302@erix.ericsson.se> Joel Reymont wrote: > > On Jul 9, 2009, at 10:28 AM, Sverker Eriksson wrote: > >> In case of kernel poll, driver_select may in some >> cases fall back to use select or poll for individual file descriptors >> if the >> OS does not support kernel poll for that device. > > ./configure --enable-threads --enable-smp-support --enable-kernel-poll > --enable-hipe > ... > checking for working poll()... broken or based on select() > checking whether kqueue() fd can be select()ed on... yes > checking whether kernel poll support should be enabled... yes; kqueue > ... > > It should be using kqueue and the Erlang prompt reports that it's > using kqueue. In fact, I distinctly remember being able to go to at > least 20k connections (descriptors) when banging on OpenPoker last year. > > It puzzles me greatly that I cannot do better than 1024 this time around. > > Any suggestions on how to troubleshoot? > Put some printf's in erts/emulator/sys/common/erl_poll.c. This is where the file descriptor limit is set at start: void ERTS_POLL_EXPORT(erts_poll_init)(void) { erts_smp_spinlock_init(&pollsets_lock, "pollsets_lock"); pollsets = NULL; errno = 0; #if defined(VXWORKS) max_fds = erts_vxworks_max_files; #elif !defined(NO_SYSCONF) max_fds = sysconf(_SC_OPEN_MAX); #elif ERTS_POLL_USE_SELECT max_fds = NOFILE; #else max_fds = OPEN_MAX; #endif #if ERTS_POLL_USE_SELECT && defined(FD_SETSIZE) if (max_fds > FD_SETSIZE) max_fds = FD_SETSIZE; #endif if (max_fds < 0) fatal_error("erts_poll_init(): Failed to get max number of files: %s\n", erl_errno_id(errno)); #ifdef ERTS_POLL_DEBUG_PRINT print_misc_debug_info(); #endif } /Sverker, Erlang/OTP From joelr1@REDACTED Thu Jul 9 12:12:25 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 9 Jul 2009 11:12:25 +0100 Subject: [erlang-questions] kpoll still using select In-Reply-To: <4A55C191.9080302@erix.ericsson.se> References: <4A55B84C.7040303@erix.ericsson.se> <7A6A571E-9FF2-4736-BBFA-7B09F80A14B1@gmail.com> <4A55C191.9080302@erix.ericsson.se> Message-ID: On Jul 9, 2009, at 11:08 AM, Sverker Eriksson wrote: > Put some printf's in erts/emulator/sys/common/erl_poll.c. This is > where the file descriptor limit is set at start: Would this work instead? Breakpoint 2, driver_select (port=86, event=86, mode=86, on=1) at sys/ unix/sys.c:275 275 } (gdb) p max_fds $3 = -1 Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Thu Jul 9 12:25:46 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 9 Jul 2009 11:25:46 +0100 Subject: [erlang-questions] kpoll still using select In-Reply-To: <4A55C191.9080302@erix.ericsson.se> References: <4A55B84C.7040303@erix.ericsson.se> <7A6A571E-9FF2-4736-BBFA-7B09F80A14B1@gmail.com> <4A55C191.9080302@erix.ericsson.se> Message-ID: <16EFDFD5-BB3E-4C8D-844E-B1819E83D733@gmail.com> On Jul 9, 2009, at 11:08 AM, Sverker Eriksson wrote: > Put some printf's in erts/emulator/sys/common/erl_poll.c. This is > where the file descriptor limit is set at start: Another data point: (gdb) p (int)erts_poll_max_fds_kp() $2 = 1024 Why? --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Thu Jul 9 12:46:26 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 9 Jul 2009 11:46:26 +0100 Subject: [erlang-questions] kpoll still using select In-Reply-To: <4A55C191.9080302@erix.ericsson.se> References: <4A55B84C.7040303@erix.ericsson.se> <7A6A571E-9FF2-4736-BBFA-7B09F80A14B1@gmail.com> <4A55C191.9080302@erix.ericsson.se> Message-ID: <39D826D0-ACA7-431D-A30D-97BD929C7D53@gmail.com> It goes like this... Breakpoint 2, erts_poll_init_kp () at sys/common/erl_poll.c:2174 2174 max_fds = sysconf(_SC_OPEN_MAX); reakpoint 3, erts_poll_init_kp () at sys/common/erl_poll.c:2183 2183 max_fds = FD_SETSIZE; (gdb) p max_fds $1 = -1 kernel poll is enabled with kqueue (as per configure), thus the _kp suffix. ERTS_POLL_USE_SELECT is still defined, though, thus the clipping. I have a strange feeling of deja vu, having had looked into this last year. I remember that ERTS_POLL_USE_SELECT must be defined because kernel poll may fall back to select under certain exceptional circumstances. I don't understand why I'm forced to 1024 fds under normal kernel poll conditions, though. Any explanation or workarounds? Thanks, Joel --- 2171 #if defined(VXWORKS) 2172 max_fds = erts_vxworks_max_files; 2173 #elif !defined(NO_SYSCONF) 2174 max_fds = sysconf(_SC_OPEN_MAX); 2175 #elif ERTS_POLL_USE_SELECT 2176 max_fds = NOFILE; 2177 #else 2178 max_fds = OPEN_MAX; 2179 #endif 2180 2181 #if ERTS_POLL_USE_SELECT && defined(FD_SETSIZE) 2182 if (max_fds > FD_SETSIZE) 2183 max_fds = FD_SETSIZE; 2184 #endif 2185 2186 if (max_fds < 0) 2187 fatal_error("erts_poll_init(): Failed to get max number of files: %s\n", 2188 erl_errno_id(errno)); --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Thu Jul 9 13:09:06 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 9 Jul 2009 12:09:06 +0100 Subject: [erlang-questions] kpoll still using select In-Reply-To: <4A55C191.9080302@erix.ericsson.se> References: <4A55B84C.7040303@erix.ericsson.se> <7A6A571E-9FF2-4736-BBFA-7B09F80A14B1@gmail.com> <4A55C191.9080302@erix.ericsson.se> Message-ID: <86D82E8D-3E80-42A8-8515-05D486378DDA@gmail.com> It's all coming back to me... http://erlang.org/pipermail/erlang-questions/2008-August/037574.html http://erlang.org/pipermail/erlang-questions/2008-August/037595.html I think it's wrong for the OTP team to hide behind broken poll on Mac OSX to clip fds to 1024 when kernel poll is used. The whole point of using kernel poll is to effectively manage more than 1024 descriptors. I also think it's wrong to force users to edit system header files to bump the available number of file descriptors. I bumped FD_SETSIZE to 30720 in /usr/include/sys/select.h (/usr/include/sys/_structs.h technically) so select may choke if it falls back. I'm fine with select choking on a larger fd if it falls back. I don't want any select fallbacks, otherwise I would not be using kernel poll. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From peter@REDACTED Thu Jul 9 14:45:19 2009 From: peter@REDACTED (Peter Sabaini) Date: Thu, 09 Jul 2009 14:45:19 +0200 Subject: Max. number of distributed nodes? Message-ID: <1247143519.20218.6.camel@ockham.local> Hi, I'm currently experimenting with a distributed Erlang setup and I'm trying to get ~120 nodes spread (unevenly) over 5 machines to talk to each other. Unfortunately, I often get errors of the kind: =ERROR REPORT==== 9-Jul-2009::13:56:07 === The global_name_server locker process received an unexpected message: {{#Ref<0.0.0.1957>,'xy@REDACTED'},false} Or =ERROR REPORT==== 9-Jul-2009::14:03:33 === global: 'foo@REDACTED' failed to connect to 'qux@REDACTED' I tried: * setting ERL_MAX_PORTS to 100 000 * using kernel-poll * setting net_setuptime to 60s but the errors persist. Is there anything else I could try? Thanks, peter. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From torben.lehoff@REDACTED Thu Jul 9 15:18:15 2009 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Thu, 9 Jul 2009 15:18:15 +0200 Subject: [erlang-questions] Extracting constants from a header file In-Reply-To: <4A55D8AA.6020709@kreditor.se> References: <4A55D8AA.6020709@kreditor.se> Message-ID: On Thu, Jul 9, 2009 at 13:46, Fabian Alenius wrote: > Torben Hoffmann wrote: > >> Hi, >> >> I would like to get a hold of all the >> -define(CONSTANT,42). >> constant definitions so I can output them for use in an external tool. >> >> I have played with epp_dodger, erl_syntax_lib et al, but I have not found >> an >> easy way of doing it. >> >> Right now the following approach seems to be what I must do: >> >> 1. Get the AST using epp:dodger:parse_file/1 >> 2. Filter all the elements from the AST where >> erl_syntax_lib:analyze_attribute/1 returns preprocessor >> 3. Take the subtrees of the each element from the step above (using >> erl_syntaxt_lib:subtrees/1) and then use the fact that >> 1. the first subtree is a list that contains {atom,_,define} and >> 2. the second subtree is a list consisting of {var,_,'PDU_TYPE'} and >> {integer,_,0} (i.e., the constant's name and type+value) >> >> Is this the one and only way of doing it? >> Not that I would mind terribly to code the above, but it seems like >> something that ought to be easier and/or have been solved by someone else >> before me. >> >> Cheers, >> Torben >> >> > Is there any reason why you can't just use a simple reg-exp? > > E.g. > > {ok, Bin} = file:read_file("erl_bits.hrl"), > re:run(Bin, ".*-define[(](?.*),[ ]*(?.*)[)]",[global, > {capture, ['NAME','VALUE'], list}]). > {match,[["SYS_ENDIAN","big"], > ["SIZEOF_CHAR","1"], > ["SIZEOF_DOUBLE","8"], > ["SIZEOF_FLOAT","4"], > ["SIZEOF_INT","4"], > ["SIZEOF_LONG","4"], > ["SIZEOF_LONG_LONG","8"], > ["SIZEOF_SHORT","2"]]} > > / Fabian Alenius > > Not really - just didn't think of it... which is why I ran the question by the mailing list! I will experiment with re and erl_syntaxt as Richard suggested and report back on what is the easiest to work with. Thanks to both of you for your hints. Cheers, Torben -- http://www.linkedin.com/in/torbenhoffmann From peter@REDACTED Thu Jul 9 16:11:56 2009 From: peter@REDACTED (Peter Sabaini) Date: Thu, 09 Jul 2009 16:11:56 +0200 Subject: [erlang-questions] Max. number of distributed nodes? In-Reply-To: <1247143519.20218.6.camel@ockham.local> References: <1247143519.20218.6.camel@ockham.local> Message-ID: <1247148716.16255.5.camel@gram.local> I should maybe add that this is on a local network (gigabit) and the network seems reasonably healthy: # ping -f -s1024 mass PING mass.local (192.168.1.199) 1024(1052) bytes of data. .^C --- mass.local ping statistics --- 116392 packets transmitted, 116391 received, 0% packet loss, time 24487ms rtt min/avg/max/mdev = 0.118/0.154/0.330/0.017 ms, ipg/ewma 0.210/0.153 ms I notice some network traffic when the nodes start up (spikes up to ~5Mbit/s) but far from being saturated; also there is some load (loadfactor ~1-3) on the machines during startup but nothing extraordinary. On Thu, 2009-07-09 at 14:45 +0200, Peter Sabaini wrote: > Hi, > > I'm currently experimenting with a distributed Erlang setup and I'm > trying to get ~120 nodes spread (unevenly) over 5 machines to talk to > each other. > > Unfortunately, I often get errors of the kind: > > =ERROR REPORT==== 9-Jul-2009::13:56:07 === > The global_name_server locker process received an unexpected message: > {{#Ref<0.0.0.1957>,'xy@REDACTED'},false} > > Or > > =ERROR REPORT==== 9-Jul-2009::14:03:33 === > global: 'foo@REDACTED' failed to connect to 'qux@REDACTED' > > > I tried: > > * setting ERL_MAX_PORTS to 100 000 > > * using kernel-poll > > * setting net_setuptime to 60s > > but the errors persist. > > Is there anything else I could try? > > Thanks, > peter. > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From magnus@REDACTED Thu Jul 9 16:20:44 2009 From: magnus@REDACTED (Magnus Henoch) Date: Thu, 09 Jul 2009 15:20:44 +0100 Subject: Trace a particular record with dbg:fun2ms In-Reply-To: (Kaiduan Xie's message of "Thu, 9 Jul 2009 00:25:06 -0400") References: Message-ID: <84bpnu0wg3.fsf@linux-b2a3.site> Kaiduan Xie writes: > 6> dbg:fun2ms(fun(#person{firstname = F, lastname = L}) when F == > "erlanger" -> true end). > Error: dbg:fun2ms requires fun with single variable or list parameter > {error,transform_error} The function passed to dbg:fun2ms takes a single argument, the _list_ of the arguments to the traced function - in this case, a list with a single element. Like this: 5> dbg:fun2ms(fun([#person{firstname = F, lastname = L}]) when F == "erlanger" -> true end). [{[#person{firstname = '$1',lastname = '$2'}], [{'==','$1',"erlanger"}], [true]}] -- Magnus Henoch, magnus@REDACTED Erlang Training and Consulting http://www.erlang-consulting.com/ From sverker@REDACTED Thu Jul 9 16:20:21 2009 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 09 Jul 2009 16:20:21 +0200 Subject: [erlang-questions] kpoll still using select In-Reply-To: <86D82E8D-3E80-42A8-8515-05D486378DDA@gmail.com> References: <4A55B84C.7040303@erix.ericsson.se> <7A6A571E-9FF2-4736-BBFA-7B09F80A14B1@gmail.com> <4A55C191.9080302@erix.ericsson.se> <86D82E8D-3E80-42A8-8515-05D486378DDA@gmail.com> Message-ID: <4A55FCA5.2000902@erix.ericsson.se> Joel Reymont wrote: > It's all coming back to me... > > http://erlang.org/pipermail/erlang-questions/2008-August/037574.html > http://erlang.org/pipermail/erlang-questions/2008-August/037595.html > > I think it's wrong for the OTP team to hide behind broken poll on Mac > OSX to clip fds to 1024 when kernel poll is used. The whole point of > using kernel poll is to effectively manage more than 1024 descriptors. > > I also think it's wrong to force users to edit system header files to > bump the available number of file descriptors. I bumped FD_SETSIZE to > 30720 in /usr/include/sys/select.h (/usr/include/sys/_structs.h > technically) so select may choke if it falls back. > > I'm fine with select choking on a larger fd if it falls back. I don't > want any select fallbacks, otherwise I would not be using kernel poll. > > Disabling fallback on select is not an option. That will fail to poll the tty device and make the emulator unable to read from STDIN. To make select fail for large fd's would probably work better. But you will have an emulator with unstable semantics. driver_select may fail arbitrary depending on the value the fd happended to get. If you want this, you can patch erl_poll.c as suggested in the earlier thread. You don't have to edit any system headers. Let's hope for a healthier poll() in the next release of OSX. /Sverker, Erlang/OTP From sverker@REDACTED Thu Jul 9 16:54:30 2009 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 09 Jul 2009 16:54:30 +0200 Subject: [erlang-questions] kpoll still using select In-Reply-To: <4A55FCA5.2000902@erix.ericsson.se> References: <4A55B84C.7040303@erix.ericsson.se> <7A6A571E-9FF2-4736-BBFA-7B09F80A14B1@gmail.com> <4A55C191.9080302@erix.ericsson.se> <86D82E8D-3E80-42A8-8515-05D486378DDA@gmail.com> <4A55FCA5.2000902@erix.ericsson.se> Message-ID: <4A5604A6.6010909@erix.ericsson.se> I wrote: > You don't have to edit any system headers. > Incorrect. I see your point. You patch FD_SETSIZE to make the bit-mask in fd_set larger and avoid memory overwrites by FD_SET and FD_CLR. But that will give you even more dangerous semantics. A call to driver_select with a large fd that kernel poll does not support will be silently accepted but you will never get any events from it. What would be needed is a patch to erl_poll.c that checks fd against FD_SETSIZE before calling FD_SET or FD_CLR. /Sverker, Erlang/OTP From erlangy@REDACTED Thu Jul 9 17:05:25 2009 From: erlangy@REDACTED (Michael McDaniel) Date: Thu, 9 Jul 2009 08:05:25 -0700 Subject: [erlang-questions] comma vs andalso In-Reply-To: <3dbc6d1c0907081628j587cb1dfm4f398120e822e126@mail.gmail.com> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> <20090707041622.GB28401@hijacked.us> <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> <769861.4108.qm@web111415.mail.gq1.yahoo.com> <3dbc6d1c0907081628j587cb1dfm4f398120e822e126@mail.gmail.com> Message-ID: <20090709150525.GB7036@delora.autosys.us> On Thu, Jul 09, 2009 at 01:28:23AM +0200, Robert Virding wrote: > 2009/7/8 Thomas Lindgren > > > ----- Original Message ---- > > > From: Richard O'Keefe > > > > > > Let's take two apparently similar functions: > > > > > > f(X) when is_atom(element(3, X)) ; true -> 42. > > > > > > g(X) when is_atom(element(3, X)) orelse true -> 42. > > > > > > By actual test, f(99) -> 42. > > > By actual test, g(99) -> a function_clause exception. > > > > > > That is, in a guard, an expression using 'andalso' or > > > 'orelse' is still an expression, and if an exception > > > occurs in the left-hand operand, the whole expression > > > is still skipped. > > .... > > > I conclude that > > > (1) you had best be EXTREMELY cautious about using 'andalso' > > > and 'orelse' in guards; they are NOT drop-in replacements > > > for ',' and ';', and > > > (2) it is long past time that Erlang allowed nested use of > > > ',' and ';' in guards. > > > > Guards are an awful mess in Erlang. I'd be quite surprised if the above > > difference in behaviour was intentional. If so, what possible purpose does > > it serve? > > > > The above difference is intentional! There is one fundamental and important > difference between using orelse/or or ';' and that is in the case of > exceptions. If an exception occurs in the case: > > G1 ; G2 ; G3 ; ... > > say in G1 then G1 will fail and G2 will be attempted, as an exception is > equivalent to failure. This is logical as they are separate guards. However, > if an exception occurs in: > > G1 orelse G2 orelse G3 ... > > in G1 again then the WHOLE guard expression will fail, G2 will never be > tried as it is part of the same expression. This IS intentional as using ';' > is exactly equivalent to separate clauses with the same RHS. So > > f(...) when G1 ; G2 ; G3 -> . > > is the same as: > > f(...) when G1 -> ; > f(...) when G2 -> ; > f(...) when G3 -> . > > This functionality was added for just this case. There is logic in the > madness. It was simple before with only simple guard tests, it only became > confusing when logical expressions were allowed in guards. They are > practical but were only added later, THEY are the odd man out. This is also > why originally there were no problems with type tests without is_ in guards. > > Guards were always meant to be part of pattern matching and no more. > > Robert ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Thank you for this history and explanation, Robert. It makes it much easier for me to remember and understand how the ';' works. (and, by extension, the other guard separators) ~Michael From kaiduanx@REDACTED Thu Jul 9 19:24:39 2009 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Thu, 9 Jul 2009 13:24:39 -0400 Subject: Trace a particular record with dbg:fun2ms In-Reply-To: <84bpnu0wg3.fsf@linux-b2a3.site> References: <84bpnu0wg3.fsf@linux-b2a3.site> Message-ID: Thanks a lot, Magnus. I misunderstood that a record is a single variable too. kaiduan On Thu, Jul 9, 2009 at 10:20 AM, Magnus Henoch wrote: > Kaiduan Xie writes: > >> 6> dbg:fun2ms(fun(#person{firstname = F, lastname = L}) when F == >> "erlanger" -> true end). >> Error: dbg:fun2ms requires fun with single variable or list parameter >> {error,transform_error} > > The function passed to dbg:fun2ms takes a single argument, the _list_ of > the arguments to the traced function - in this case, a list with a > single element. ?Like this: > > 5> dbg:fun2ms(fun([#person{firstname = F, lastname = L}]) when F == > "erlanger" -> true end). > [{[#person{firstname = '$1',lastname = '$2'}], > ?[{'==','$1',"erlanger"}], > ?[true]}] > > -- > Magnus Henoch, magnus@REDACTED > Erlang Training and Consulting > http://www.erlang-consulting.com/ > From mikpe@REDACTED Fri Jul 10 01:44:59 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Fri, 10 Jul 2009 01:44:59 +0200 Subject: [erlang-bugs] Possible bug in HiPE pattern matching In-Reply-To: <877FEB4B-A358-4C41-B831-21B61D5CE101@jonmeredith.com> References: <877FEB4B-A358-4C41-B831-21B61D5CE101@jonmeredith.com> Message-ID: <19030.33019.386108.973565@pilspetsen.it.uu.se> Jon Meredith writes: > Hi all, > > I think I've discovered an issue with pattern matching under HiPE > going back on at least 5.6.3 under Debian/lenny and 5.7.2 under OS X > > Here is a module that demonstrates the issue > > -module(bug). > -export([f/1,g/1]). > f(0) -> zero; > f(-10) -> one; > f(-20) -> two; > f(-30) -> three; > f(-40) -> four; > f(-50) -> five; > f(-60) -> six; > f(-70) -> seven; > f(-80) -> eight; %% Uncomment this line to make it work with HiPE > f(X) -> {error, X}. > > g(X) -> > case X of > 0 -> zero; > -10 -> one; > -20 -> two; > -30 -> three; > -40 -> four; > -50 -> five; > -60 -> six; > -70 -> seven; > -80 -> eight; %% Uncomment this line to make it work with HiPE > X -> {error, X} > end. > > Running normally without HiPE it does what I expect > > jmeredith@REDACTED:~$ /usr/bin/erlc bug.erl > jmeredith@REDACTED:~$ /usr/bin/erl > Erlang (BEAM) emulator version 5.6.3 [source] [64-bit] [smp:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.6.3 (abort with ^G) > 1> bug:f(0). > zero > 2> bug:g(0). > zero > > If I compile with +native, the pattern doesn't match the first case as > it should > > jmeredith@REDACTED:~$ /usr/bin/erlc +native bug.erl > jmeredith@REDACTED:~$ /usr/bin/erl > Erlang (BEAM) emulator version 5.6.3 [source] [64-bit] [smp:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.6.3 (abort with ^G) > 1> bug:f(0). > {error,0} > 2> bug:g(0). > {error,0} Confirmed on i686, x86-64, and powerpc, so it looks like a middle-end (Icode or RTL) rather than a backend bug. Someone, likely me or Kostis, will take a look at it. > Is there > something I don't understand about HiPE? Bugs happen, that's all. Thanks for reporting this problem. /Mikael From cyberlync@REDACTED Fri Jul 10 03:43:46 2009 From: cyberlync@REDACTED (Eric Merritt) Date: Thu, 9 Jul 2009 20:43:46 -0500 Subject: Changes to group leader with introduction of unicode features Message-ID: Guys, I am trying to debug a problem in my code. I have a group leader that 'captures' io from a couple of Erlang libraries that expect to communicate directly with the user. It seems that the introduction of unicode changed the implementation of group leaders in some obvious and non-obvious ways. The obvious ways where pretty strait forward to cover. However, I am having a bit of trouble with the non-obvious problems. I have a gen server that acts as a group leader. It receives IO messages in its handle_info the exiting code looks as follows. Somehow I am garbling the io message. I am consistently getting the following error. The no_translation message seems to be coming from deep in the io layer. I suspect that I am missing something obvious but I could use a pointer to the right direction. doc:io generated event:error in doclet 'edoc_doclet': {'EXIT',{no_translation,[{io,put_chars,[<0.724.0>,unicode,[[60,33,68,79,67,84,89|...],[45,47,47,87,51,67|...],[34],[],[62,10],[[[...]|...],[...]]]]},{edoc_lib,write_file,4},{edoc_doclet,gen,6},{edoc_lib,run_plugin,5},{dbg_debugged,reply,1},{dbg_debugged,handle_command,2},{dbg_debugged,msg_loop,3},{eta_task_runner,apply_task,3}]}}. . handle_info({io_request, From, ReplyAs, {put_chars, io_lib, Func, [Format, Args]}}, State) when is_list(Format) -> Msg = (catch io_lib:Func(Format,Args)), handle_msg(ReplyAs, Msg, From, Func, State), {noreply, State}; handle_info({io_request, From, ReplyAs, {put_chars, unicode, io_lib, Func, [Format, Args]}}, State) when is_list(Format) -> Msg = (catch io_lib:Func(Format,Args)), handle_msg(ReplyAs, Msg, From, Func, State), {noreply, State}; handle_info({io_request, From, ReplyAs, {put_chars, io_lib, Func, [Format, Args]}}, State) when is_atom(Format) -> Msg = (catch io_lib:Func(Format,Args)), handle_msg(ReplyAs, Msg, From, Func, State), {noreply, State}; handle_info({io_request, From, ReplyAs, {put_chars, unicode, io_lib, Func, [Format, Args]}}, State) when is_atom(Format) -> Msg = (catch io_lib:Func(Format,Args)), handle_msg(ReplyAs, Msg, From, Func, State), {noreply, State}; handle_info({io_request, From, ReplyAs, {put_chars, Bytes}}, State) -> handle_msg(ReplyAs, Bytes, From, put_chars, State), {noreply, State}; handle_info({io_request, From, ReplyAs, {put_chars,unicode,Bytes}}, State) -> handle_msg(ReplyAs, Bytes, From, put_chars, State), {noreply, State}; handle_info(IoReq, State) when element(1, IoReq) == io_request -> %% something else, just pass it on group_leader() ! IoReq, {noreply, State}; handle_info(_, State) -> {noreply, State}. handle_msg(ReplyAs, Msg, From, Func, #state{build_id=BuildId, task=Task, type=Type}) -> case Msg of {'EXIT',_} -> From ! {io_reply,ReplyAs,{error,Func}}; _ -> From ! {io_reply,ReplyAs,ok} end, eta_event:task_event(BuildId, Task, Type, Msg). From shelton.ms@REDACTED Fri Jul 10 04:53:04 2009 From: shelton.ms@REDACTED (Shelton Tang) Date: Fri, 10 Jul 2009 10:53:04 +0800 Subject: any limitation on outbound tcp connections on Windows Message-ID: <27ccae8e0907091953m3548e557w49db4d2cc9663726@mail.gmail.com> Hi All, I use gen_tcp:connect to make outbound TCP connection. The argument passed to the call is [list, {packet, 0},{reuseaddr, true},{active, false}] (I also tried binary or passive mode but no luck). However, my test shows the maximum concurrent connections are <= 10 on my xp and 5 - 8 on a X64 win2k8 machine. All other connections got timeout error. So I captured a network traffic and didn't find any frame for the timeout connections. Then I try to connect to the local host with 127.0.0.1. On my WinXP box, I can create about 32 connections and other got econnrefused error (the server is an erlang socket server). I know there is a 10 SYN limitation on winxp tcpip.sys. However, the confused is why i hit the problem on a powerful (16G memory, 4 processors with dual cores) x64 machine? Possible reasons: 1. is there any limitation in erlang? (becoz i can make > 10 connections to 127.0.0.1, most like it is not an erlang issue). 2. is there any limitation on tcpip.sys on win2k8? 3. is it possible there is a restraction on my firewall? (but if so, i should see any network frames but actually i didn't find it). any ideas? Regards, Shelton From shelton.ms@REDACTED Fri Jul 10 08:37:07 2009 From: shelton.ms@REDACTED (Shelton Tang) Date: Fri, 10 Jul 2009 14:37:07 +0800 Subject: any limitation on outbound tcp connections on Windows In-Reply-To: <27ccae8e0907091953m3548e557w49db4d2cc9663726@mail.gmail.com> References: <27ccae8e0907091953m3548e557w49db4d2cc9663726@mail.gmail.com> Message-ID: <27ccae8e0907092337h5837177xdf7d61dac2445f5c@mail.gmail.com> I wrote a test program with C# and didn't see such problem. is this a limitation in erlang@REDACTED? On Fri, Jul 10, 2009 at 10:53 AM, Shelton Tang wrote: > Hi All, > > I use gen_tcp:connect to make outbound TCP connection. ?The argument > passed to the call is ?[list, {packet, 0},{reuseaddr, true},{active, > false}] (I also tried binary or passive mode but no luck). ?However, > my test shows the maximum concurrent connections are <= 10 on my xp > and 5 - 8 on a X64 win2k8 machine. All other connections got timeout > error. So I captured a network traffic and didn't find any frame for > the timeout connections. Then I try to connect to the local host with > 127.0.0.1. On my WinXP box, I can create about 32 connections and > other got econnrefused error (the server is an erlang socket server). > > I know there is a 10 SYN limitation on winxp tcpip.sys. ?However, the > confused is why i hit the problem on a powerful (16G memory, 4 > processors with dual cores) x64 machine? Possible reasons: > 1. is there any limitation in erlang? ?(becoz i can make > 10 > connections to 127.0.0.1, most like it is not an erlang issue). > 2. is there any limitation on tcpip.sys on win2k8? > 3. is it possible there is a restraction on my firewall? (but if so, i > should see any network frames but actually i didn't find it). > > any ideas? > > Regards, > Shelton > From erlang@REDACTED Fri Jul 10 08:55:00 2009 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 10 Jul 2009 08:55:00 +0200 Subject: compile transforms which change the module name Message-ID: <9b08084c0907092355t2f649bf6tfea654f971ad991e@mail.gmail.com> If I change the module name in a compile transform I get an error. If there a work-around for this? I really want to change the module name so that when I compile foo.erl it produces a new name xyz.beam containing code for the module xyz. /Joe From peppe@REDACTED Fri Jul 10 11:02:53 2009 From: peppe@REDACTED (Peter Andersson) Date: Fri, 10 Jul 2009 11:02:53 +0200 Subject: [erlang-questions] compile transforms which change the module name In-Reply-To: <9b08084c0907092355t2f649bf6tfea654f971ad991e@mail.gmail.com> References: <9b08084c0907092355t2f649bf6tfea654f971ad991e@mail.gmail.com> Message-ID: <4A5703BD.6060705@erix.ericsson.se> The 'no_error_module_mismatch' option should do the trick: -module(foo). -compile({parse_transform,rename}). ... 1> compile:file(foo, [no_error_module_mismatch]). {ok,xyz} /Peppe Joe Armstrong wrote: > If I change the module name in a compile transform I get an error. > > If there a work-around for this? > > I really want to change the module name so that when I compile foo.erl it > produces a new name xyz.beam containing code for the module xyz. > > /Joe > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From rschonberger@REDACTED Fri Jul 10 11:34:06 2009 From: rschonberger@REDACTED (Robert Schonberger) Date: Fri, 10 Jul 2009 19:34:06 +1000 Subject: [erlang-questions] SSL RNG seeding In-Reply-To: References: Message-ID: did you look at crypto:rand_bytes/1 ? 2009/6/1 Greg Perry > Ugh. From the ssl module documentation: > > ---------- > > seed(Data) -> ok | {error, Reason} > > Types: > > Data = iolist() | binary() > > Seeds the ssl random generator. > > It is strongly advised to seed the random generator after the ssl > application has been started, and before any connections are > established. Although the port program interfacing to the OpenSSL > libraries does a "random" seeding of its own in order to make everything > work properly, that seeding is by no means random for the world since it > has a constant value which is known to everyone reading the source code > of the seeding. > > A notable return value is {error, edata}} indicating that Data was not a > binary nor an iolist. > > ---------- > > So SSL uses a hardcoded seed value within the source code, is this > standard across all of the Erland cryto libraries? > > Regards > > Greg > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Robert Schonberger rschonberger@REDACTED From zerthurd@REDACTED Fri Jul 10 12:08:45 2009 From: zerthurd@REDACTED (Maxim Treskin) Date: Fri, 10 Jul 2009 17:08:45 +0700 Subject: [erlang-questions] Different arity of macroses In-Reply-To: References: Message-ID: So, no way? -- Maxim Treskin From ulf.wiger@REDACTED Fri Jul 10 12:00:54 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 10 Jul 2009 12:00:54 +0200 Subject: [erlang-questions] compile transforms which change the module name In-Reply-To: <4A5703BD.6060705@erix.ericsson.se> References: <9b08084c0907092355t2f649bf6tfea654f971ad991e@mail.gmail.com> <4A5703BD.6060705@erix.ericsson.se> Message-ID: <4A571156.3060208@erlang-consulting.com> Peter Andersson wrote: > The 'no_error_module_mismatch' option should do the trick: > > -module(foo). > -compile({parse_transform,rename}). > ... > > 1> compile:file(foo, [no_error_module_mismatch]). > {ok,xyz} ...except it still produces a file called foo.beam, so c(foo, [no_error_module_mismatch]) still won't work. It seems as if a wrapper is needed: compile(F, Opts) -> case compile:file(F, [{parse_transform,rename},binary, no_error_module_mismatch|Opts]) of {ok, Mod, Bin} -> Dir = filename:dirname(F), OutF = filename:join(Dir, atom_to_list(Mod) ++ ".beam"), ok = file:write_file(OutF, Bin), {ok, Mod}; Error -> Error end. (Above code not compiled - let alone tested.) BR, Ulf W > /Peppe > > > Joe Armstrong wrote: >> If I change the module name in a compile transform I get an error. >> >> If there a work-around for this? >> >> I really want to change the module name so that when I compile foo.erl it >> produces a new name xyz.beam containing code for the module xyz. >> >> /Joe >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From erlang@REDACTED Fri Jul 10 12:36:02 2009 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 10 Jul 2009 12:36:02 +0200 Subject: [erlang-questions] compile transforms which change the module name In-Reply-To: <4A571156.3060208@erlang-consulting.com> References: <9b08084c0907092355t2f649bf6tfea654f971ad991e@mail.gmail.com> <4A5703BD.6060705@erix.ericsson.se> <4A571156.3060208@erlang-consulting.com> Message-ID: <9b08084c0907100336j3b222881y30e16afbab9542c1@mail.gmail.com> Works a charm - thanks I am making a very nice transform. Hint: take all files in an application call these {x1.erl, x2.erl ...} 1) Make a name map map() -> [{"x1", m27313afsadfddsf, "joe", 213517657153675} ... This means that the md5 checksum of x1.erl is m273 .... and 2135... is the MD5 sum signed with joe's public key 2) transform all code in x1.erl x2.erl replacing the module names x1, x2 ... etc. with the MD5 sums so x1:abc(...) becomes m273...:abc( ...) 3) replace apply(M,F,A) with newapply(map(),M, F, A) result - a) no name space collisions b) exact version control c) safety (if you can't guess the MD5 sum :-) It's kind of like adding a big "where" clause round a module -module(foo). a() -> lists:reverse(...) WHERE foo = m1283abcd ... lists = m34aefg ... ie all modules names are converted to the MD5 sums of the original sources. I think this might solve a lot of problems with safety, versioning, testing, namespaces etc. (safe erlang can be made by replacing calls to dangerous mods like file etc. with safe_file ... etc.) /Joe On Fri, Jul 10, 2009 at 12:00 PM, Ulf Wiger wrote: > Peter Andersson wrote: >> >> The ?'no_error_module_mismatch' option should do the trick: >> >> -module(foo). >> -compile({parse_transform,rename}). >> ... >> >> 1> compile:file(foo, [no_error_module_mismatch]). >> {ok,xyz} > > ...except it still produces a file called foo.beam, > so c(foo, [no_error_module_mismatch]) still won't work. > > It seems as if a wrapper is needed: > > compile(F, Opts) -> > ?case compile:file(F, [{parse_transform,rename},binary, > ? ? ? ? ? ? ? ? ? ? ? ?no_error_module_mismatch|Opts]) of > ? ?{ok, Mod, Bin} -> > ? ? ?Dir = filename:dirname(F), > ? ? ?OutF = filename:join(Dir, atom_to_list(Mod) ++ ".beam"), > ? ? ?ok = file:write_file(OutF, Bin), > ? ? ?{ok, Mod}; > ? ?Error -> > ? ? ?Error > ?end. > > (Above code not compiled - let alone tested.) > > BR, > Ulf W > > >> ?/Peppe >> >> >> Joe Armstrong wrote: >>> >>> If I change the module name in a compile transform I get an error. >>> >>> If there a work-around for this? >>> >>> I really want to change the module name so that when I compile foo.erl it >>> produces a new name xyz.beam containing code for the module xyz. >>> >>> /Joe >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > > > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > From vladdu55@REDACTED Fri Jul 10 12:52:45 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 10 Jul 2009 12:52:45 +0200 Subject: [erlang-questions] compile transforms which change the module name In-Reply-To: <9b08084c0907100336j3b222881y30e16afbab9542c1@mail.gmail.com> References: <9b08084c0907092355t2f649bf6tfea654f971ad991e@mail.gmail.com> <4A5703BD.6060705@erix.ericsson.se> <4A571156.3060208@erlang-consulting.com> <9b08084c0907100336j3b222881y30e16afbab9542c1@mail.gmail.com> Message-ID: <95be1d3b0907100352s7cb2ecd8ka2b9c7e6793f32ab@mail.gmail.com> On Fri, Jul 10, 2009 at 12:36, Joe Armstrong wrote: > I am making a very nice transform. > Hint: take all files in an application call these {x1.erl, x2.erl ...} > 1) Make a name map > 2) transform all code in x1.erl x2.erl replacing the module names x1, > x2 ... etc. > ? ?with the MD5 sums so x1:abc(...) becomes m273...:abc( ...) > 3) replace apply(M,F,A) with newapply(map(),M, F, A) Hi Joe, Looks nice, but I have a few followup questions: - where will the map() function reside? - what about M:F() calls and M:F/A references sent as data? - updating a comment will invalidate the map... - it would be nice to have tools to do the reverse transformation for logs and error messages; also, to be able to find the right version of a module in ClearCase or SVN given the name and a MD5 hash. best regards, Vlad From rvirding@REDACTED Fri Jul 10 14:30:51 2009 From: rvirding@REDACTED (Robert Virding) Date: Fri, 10 Jul 2009 14:30:51 +0200 Subject: [erlang-questions] Different arity of macroses In-Reply-To: References: Message-ID: <3dbc6d1c0907100530o6be06ff1i792b4e44fe211f54@mail.gmail.com> No, this feature was never added. Originally I was just mimicking the C preprocessor (yes I know it was a bad choice, but I didn't like macros) so it didn't allow macros with multiple number of arguments, and then there was never any real need for it so I never got around to it. Just give them different names DBG1, DBG2, etc. Not the erlang way I know and it would be be nicer if it worked the same way as for functions. Maybe some time in the future. Robert 2009/7/10 Maxim Treskin > So, no way? > > -- > Maxim Treskin > From rvirding@REDACTED Fri Jul 10 14:42:17 2009 From: rvirding@REDACTED (Robert Virding) Date: Fri, 10 Jul 2009 14:42:17 +0200 Subject: [erlang-questions] Why doesn't erl_eval compile code? In-Reply-To: References: <3dbc6d1c0907051611s2a7b1aa5m1a391e0bae613c8f@mail.gmail.com> <3dbc6d1c0907081546r7c9d2fa5oc706b510329e4153@mail.gmail.com> Message-ID: <3dbc6d1c0907100542v1fdd5794qaf2ab055d9e74180@mail.gmail.com> 2009/7/9 Tony Arcieri > On Wed, Jul 8, 2009 at 4:46 PM, Robert Virding wrote: > >> The basic problem is that you are trying to use the code system in a way >> we never intended. > > > Yes, at times I feel like I'm in square peg round hole territory. Usually > when I reach this point I try to take a step back and do things in an > Erlangier way. I now see the reasoning behind why Erlang implements eval > metacircularly. However, if there's one thing that sets scripting languages > apart from other languages I believe it's extensive use of eval and runtime > code generation/execution. In that regard you might call Lisp the world's > first scripting language (although I believe that honor is typically > attributed to awk) > I think that many scripting languages probably do interpret the code internally and don't compile it, though I haven't studied them so I am not certain. What makes lisp (and prolog for that matter) so useful as a scripting language is that most implementations allow you to freely intermix compiled and interpreted functions. Which Erlang does not, and that when it manages code it manages whole modules not functions, and that modules are compiled. I ran into exactly the same problem with LFE, plus that lisp also has macros as separate entities. However, it's a lot easier to write a metacircular evaluator for a Lisp than > for something like Reia... > > >> To do this properly as you intend would mean a rewrite of the code system, >> which would probably result in changes all over the place. But it would be >> more beautiful. :-( >> > > I'm okay with hacks as long as they work :) In this case using soft_purge and having a process which occasionally tries to remove your temporary modules would be the hack. Looking at the documentation for soft_purge seems to imply that it is fun safe, but try it. Robert From chad@REDACTED Fri Jul 10 15:20:27 2009 From: chad@REDACTED (Chad DePue) Date: Fri, 10 Jul 2009 10:20:27 -0300 Subject: [erlang-questions] compile transforms which change the module name In-Reply-To: <9b08084c0907100336j3b222881y30e16afbab9542c1@mail.gmail.com> References: <9b08084c0907092355t2f649bf6tfea654f971ad991e@mail.gmail.com> <4A5703BD.6060705@erix.ericsson.se> <4A571156.3060208@erlang-consulting.com> <9b08084c0907100336j3b222881y30e16afbab9542c1@mail.gmail.com> Message-ID: this is cool. is MD5 good enough or would it be better to use SHA1? On Jul 10, 2009, at 7:36 AM, Joe Armstrong wrote: > Works a charm - thanks > > I am making a very nice transform. > > Hint: take all files in an application call these {x1.erl, x2.erl ...} > > 1) Make a name map > > map() -> [{"x1", m27313afsadfddsf, "joe", 213517657153675} > ... > > This means that the md5 checksum of x1.erl is m273 .... and 2135... is > the MD5 sum signed > with joe's public key > > 2) transform all code in x1.erl x2.erl replacing the module names x1, > x2 ... etc. > with the MD5 sums so x1:abc(...) becomes m273...:abc( ...) > > 3) replace apply(M,F,A) with newapply(map(),M, F, A) > > result - a) no name space collisions > b) exact version control > c) safety (if you can't guess the MD5 sum :-) > > It's kind of like adding a big "where" clause round a module > > -module(foo). > > a() -> lists:reverse(...) > > WHERE > foo = m1283abcd ... > lists = m34aefg ... > > ie all modules names are converted to the MD5 sums of the original > sources. > > I think this might solve a lot of problems with safety, versioning, > testing, namespaces etc. > > (safe erlang can be made by replacing calls to dangerous mods like > file etc. > with safe_file ... etc.) > > /Joe > > > > On Fri, Jul 10, 2009 at 12:00 PM, Ulf > Wiger wrote: >> Peter Andersson wrote: >>> >>> The 'no_error_module_mismatch' option should do the trick: >>> >>> -module(foo). >>> -compile({parse_transform,rename}). >>> ... >>> >>> 1> compile:file(foo, [no_error_module_mismatch]). >>> {ok,xyz} >> >> ...except it still produces a file called foo.beam, >> so c(foo, [no_error_module_mismatch]) still won't work. >> >> It seems as if a wrapper is needed: >> >> compile(F, Opts) -> >> case compile:file(F, [{parse_transform,rename},binary, >> no_error_module_mismatch|Opts]) of >> {ok, Mod, Bin} -> >> Dir = filename:dirname(F), >> OutF = filename:join(Dir, atom_to_list(Mod) ++ ".beam"), >> ok = file:write_file(OutF, Bin), >> {ok, Mod}; >> Error -> >> Error >> end. >> >> (Above code not compiled - let alone tested.) >> >> BR, >> Ulf W >> >> >>> /Peppe >>> >>> >>> Joe Armstrong wrote: >>>> >>>> If I change the module name in a compile transform I get an error. >>>> >>>> If there a work-around for this? >>>> >>>> I really want to change the module name so that when I compile >>>> foo.erl it >>>> produces a new name xyz.beam containing code for the module xyz. >>>> >>>> /Joe >>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>>> >>>> >>> >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >> >> >> -- >> Ulf Wiger >> CTO, Erlang Training & Consulting Ltd >> http://www.erlang-consulting.com >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From joelr1@REDACTED Fri Jul 10 15:42:04 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 10 Jul 2009 14:42:04 +0100 Subject: pg2 on ec2 Message-ID: I'm using R13B01 and making my gen_server join the pg2 group like this: init([Trace]) -> process_flag(trap_exit, true), pg2:create(?MODULE), ok = pg2:join(?MODULE, self()), {ok, new(Trace)}. I start two nodes and launch my gen_server on node #2. I can see the process in the group with pg2:get_members/1. Problem is, initialization on node #1 succeeds both on my Mac and on Amazon EC2 (Ubuntu 9.04) but I don't see the local node's process on EC2. It's as if pg2 was ignoring my attempts to join the group on the other node while dutifully reporting that everything is fine. Furthermore, if I convert the pid returned in {ok, Pid} from list to pid and manually make it join the group then everything is fine on EC2. Any suggestions? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From erlang@REDACTED Fri Jul 10 15:50:19 2009 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 10 Jul 2009 15:50:19 +0200 Subject: [erlang-questions] compile transforms which change the module name In-Reply-To: <95be1d3b0907100352s7cb2ecd8ka2b9c7e6793f32ab@mail.gmail.com> References: <9b08084c0907092355t2f649bf6tfea654f971ad991e@mail.gmail.com> <4A5703BD.6060705@erix.ericsson.se> <4A571156.3060208@erlang-consulting.com> <9b08084c0907100336j3b222881y30e16afbab9542c1@mail.gmail.com> <95be1d3b0907100352s7cb2ecd8ka2b9c7e6793f32ab@mail.gmail.com> Message-ID: <9b08084c0907100650s22940ed9ob767b5a7a838ff9@mail.gmail.com> On Fri, Jul 10, 2009 at 12:52 PM, Vlad Dumitrescu wrote: > On Fri, Jul 10, 2009 at 12:36, Joe Armstrong wrote: >> I am making a very nice transform. >> Hint: take all files in an application call these {x1.erl, x2.erl ...} >> 1) Make a name map >> 2) transform all code in x1.erl x2.erl replacing the module names x1, >> x2 ... etc. >> ? ?with the MD5 sums so x1:abc(...) becomes m273...:abc( ...) >> 3) replace apply(M,F,A) with newapply(map(),M, F, A) > > Hi Joe, > > Looks nice, but I have a few followup questions: > > - where will the map() function reside? Somewhere :-) (in the group leader) > - what about M:F() calls and M:F/A references sent as data? Will break existing code - we need a new bif Mod1 = theModName(Mod) which converts a module name like "lists" into the correct version "424acfe..." > - updating a comment will invalidate the map... Yes - I'm not sure about this. I could make a check sum from epp:parse_file then remove the line numbers. If an average erlang module is 20KBytes then could fit 50,000 modules in 1GB > - it would be nice to have tools to do the reverse transformation for > logs and error messages; also, to be able to find the right version of > a module in ClearCase or SVN given the name and a MD5 hash. No need to the transformed module can have an added exported function: my_origonal_name() -> xxxx. /Joe > > best regards, > Vlad > From joelr1@REDACTED Fri Jul 10 15:52:08 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 10 Jul 2009 14:52:08 +0100 Subject: pg2 on ec2 In-Reply-To: References: Message-ID: <1920C083-5DF3-460E-88FF-870A82F5DF1D@gmail.com> Please disregard! Foot in mouth! On Jul 10, 2009, at 2:42 PM, Joel Reymont wrote: > I'm using R13B01 and making my gen_server join the pg2 group like > this: > > init([Trace]) -> > process_flag(trap_exit, true), > pg2:create(?MODULE), > ok = pg2:join(?MODULE, self()), > {ok, new(Trace)}. > > I start two nodes and launch my gen_server on node #2. I can see the > process in the group with pg2:get_members/1. Problem is, > initialization on node #1 succeeds both on my Mac and on Amazon EC2 > (Ubuntu 9.04) but I don't see the local node's process on EC2. > > It's as if pg2 was ignoring my attempts to join the group on the > other node while dutifully reporting that everything is fine. > > Furthermore, if I convert the pid returned in {ok, Pid} from list to > pid and manually make it join the group then everything is fine on > EC2. > > Any suggestions? > > Thanks, Joel > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From matthew@REDACTED Fri Jul 10 15:54:45 2009 From: matthew@REDACTED (Matthew Sackman) Date: Fri, 10 Jul 2009 14:54:45 +0100 Subject: receive _ -> ok end is exponential in queue length?!! Message-ID: <20090710135445.GB4171@wellquite.org> [begin [ self() ! a || _ <- lists:seq(1,N) ], S = now(), [ receive _ -> ok end || _ <- lists:seq(1,N) ], io:format("~w~n", [timer:now_diff(now(), S) / N]) end || N <- [1,10,100,1000,10000]]. 13.0 3.0 11.91 221.703 3421.773 I'm willing to believe I'm being staggaringly dumb here, but why on earth isn't this constant? We're just removing from the head of the message queue. The above is direct in the erlang shell. If I put it into a module, add 1e5 and 1e6 then I get: 1> test:test(). 1.0 0.1 0.06 0.114 0.1554 0.36398 0.358997 Which looks rather more sensible. So why is it so much slower doing it direct in the shell? It's not just a case of it looking like compiler optimisations (which you'd expect to produce a linear factor speedup), it's more that it looks like the shell is doing the wrong algorithm (like removing from the end of the queue instead of the head?!). Matthew From ulf.wiger@REDACTED Fri Jul 10 16:39:09 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 10 Jul 2009 16:39:09 +0200 Subject: [erlang-questions] receive _ -> ok end is exponential in queue length?!! In-Reply-To: <20090710135445.GB4171@wellquite.org> References: <20090710135445.GB4171@wellquite.org> Message-ID: <4A57528D.7040802@erlang-consulting.com> Matthew Sackman wrote: > [begin [ self() ! a || _ <- lists:seq(1,N) ], > S = now(), > [ receive _ -> ok end || _ <- lists:seq(1,N) ], > io:format("~w~n", [timer:now_diff(now(), S) / N]) > end || N <- [1,10,100,1000,10000]]. > 13.0 > 3.0 > 11.91 > 221.703 > 3421.773 > > I'm willing to believe I'm being staggaringly dumb here, but why on > earth isn't this constant? We're just removing from the head of the > message queue. The above is direct in the erlang shell. The module erl_eval.erl has some interesting message queue gymnastics: receive_clauses(Cs, Bs, Lf, Ef, Ms, RBs) -> receive Val -> case match_clause(Cs, [Val], Bs, Lf, Ef) of {B, Bs1} -> merge_queue(Ms), exprs(B, Bs1, Lf, Ef, RBs); nomatch -> receive_clauses(Cs, Bs, Lf, Ef, [Val|Ms], RBs) end end. merge_queue(Ms) -> send_all(recv_all(Ms), self()). recv_all(Xs) -> receive X -> recv_all([X|Xs]) after 0 -> reverse(Xs) end. send_all([X|Xs], Self) -> Self ! X, send_all(Xs, Self); send_all([], _) -> true. The case where the first message matches would be the easy one, if it weren't for the fact that one also needs to handle the case where the first message /doesn't/ match. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From richardc@REDACTED Fri Jul 10 16:41:29 2009 From: richardc@REDACTED (Richard Carlsson) Date: Fri, 10 Jul 2009 16:41:29 +0200 Subject: [erlang-questions] receive _ -> ok end is exponential in queue length?!! In-Reply-To: <20090710135445.GB4171@wellquite.org> References: <20090710135445.GB4171@wellquite.org> Message-ID: <4A575319.2060702@it.uu.se> Matthew Sackman wrote: > [begin [ self() ! a || _ <- lists:seq(1,N) ], > S = now(), > [ receive _ -> ok end || _ <- lists:seq(1,N) ], > io:format("~w~n", [timer:now_diff(now(), S) / N]) > end || N <- [1,10,100,1000,10000]]. > 13.0 > 3.0 > 11.91 > 221.703 > 3421.773 > > I'm willing to believe I'm being staggaringly dumb here, but why on > earth isn't this constant? We're just removing from the head of the > message queue. The above is direct in the erlang shell. It seems that since the interpreter (erl_eval) must simulate the pattern matching over the mailbox, it must extract messages unconditionally from the head of the mailbox, one at a time, try to match them, and put them aside for later if they don't match. When a matching message is found, or a timeout occurs, it must restore the mailbox as it is supposed to look, by putting the extracted messages back _before_ any messages still left in the real queue. The only way it can do that is by extracting all of the remaining messages as well, and then send the whole lot back to itself (hoping that no further messages arrive while it is doing this). This tends to work in a shell process with little traffic and a generally small mailbox, but obviously doesn't scale. The quick fix would be to change the following in erl_eval: merge_queue(Ms) -> send_all(recv_all(Ms), self()). to merge_queue([]) -> true; merge_queue(Ms) -> ... which at least avoids unnecessary work when no messages have been extracted which need to be put back. /Richard From info@REDACTED Fri Jul 10 16:49:09 2009 From: info@REDACTED (info) Date: Fri, 10 Jul 2009 16:49:09 +0200 Subject: build a time&date string Message-ID: <200907101649089620930@its3.ch> Hi all, Given a date and a time according the following formats: Date = DDMMYY Time = HHMMSS Is it possible to build a composite string only by using the formats ? My problem is in the splitting ... Composite = YYYY-MM-DD HH:MM:SS John From vladdu55@REDACTED Fri Jul 10 16:50:40 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 10 Jul 2009 16:50:40 +0200 Subject: [erlang-questions] compile transforms which change the module name In-Reply-To: <9b08084c0907100650s22940ed9ob767b5a7a838ff9@mail.gmail.com> References: <9b08084c0907092355t2f649bf6tfea654f971ad991e@mail.gmail.com> <4A5703BD.6060705@erix.ericsson.se> <4A571156.3060208@erlang-consulting.com> <9b08084c0907100336j3b222881y30e16afbab9542c1@mail.gmail.com> <95be1d3b0907100352s7cb2ecd8ka2b9c7e6793f32ab@mail.gmail.com> <9b08084c0907100650s22940ed9ob767b5a7a838ff9@mail.gmail.com> Message-ID: <95be1d3b0907100750r7a498ej789f585e94f6877@mail.gmail.com> On Fri, Jul 10, 2009 at 15:50, Joe Armstrong wrote: >> - it would be nice to have tools to do the reverse transformation for >> logs and error messages; also, to be able to find the right version of >> a module in ClearCase or SVN given the name and a MD5 hash. > > No need to the transformed module can have an added exported function: > ? ?my_origonal_name() -> > ? ? ? ? xxxx. That will handle the name, but the version number in the version control system might be important too. Possibly another generated function my_original_version_number() could handle that, but it's not easy to cover all VCSs... Then again, the map() data might need to be saved somewhere too (and version controlled). There seem to be many small things that have to be put in place. If there weren't had been, the whole issue would have been trivial :-) regards, Vlad From matthew@REDACTED Fri Jul 10 16:57:59 2009 From: matthew@REDACTED (Matthew Sackman) Date: Fri, 10 Jul 2009 15:57:59 +0100 Subject: [erlang-questions] receive _ -> ok end is exponential in queue length?!! In-Reply-To: <4A575319.2060702@it.uu.se> References: <20090710135445.GB4171@wellquite.org> <4A575319.2060702@it.uu.se> Message-ID: <20090710145759.GD4171@wellquite.org> On Fri, Jul 10, 2009 at 04:41:29PM +0200, Richard Carlsson wrote: > It seems that since the interpreter (erl_eval) must simulate the > pattern matching over the mailbox, it must extract messages > unconditionally from the head of the mailbox, one at a time, > try to match them, and put them aside for later if they don't > match. Understood. Thank you and Ulf Wiger for the explanations. Interesting how simulating Erlang in Erlang is so hard ;) I'm a little surprised more helpful VM primitives haven't been built so that this step in the shell evaluator can't utilise exactly the same functionality as post compilation - i.e. removing the need to implement the evaluation of receive through calling receive. It does somewhat limit the scalability of the shell. Matthew From Greg.Perry@REDACTED Fri Jul 10 17:17:39 2009 From: Greg.Perry@REDACTED (Greg Perry) Date: Fri, 10 Jul 2009 08:17:39 -0700 Subject: Question on building a variable with multiple math operations Message-ID: I have a unique challenge that I can't seem to find a right way to do based on the immutability of Erlang. I have built a unique factoring application with Erlang, which first analyzes any given N and depending on the properties of that subprime number a list of coefficients are assigned to a list (the list contains tuples actually). http://blog.liveammo.com/2009/06/factoring-fun/ For example: $ erl Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) 1> c("reductionsieve.erl"). {ok,reductionsieve} 2> c("lin.erl"). {ok,lin} 3> N1 = reductionsieve:make_prime(5) * reductionsieve:make_prime(5). Generating a 5 digit prime ...... Generating a 5 digit prime .................. 823221601 4> reductionsieve:analyze(N1). [{1,7},{2,8},{4,4},{5,5}] 5> N2 = reductionsieve:make_prime(5) * reductionsieve:make_prime(5). Generating a 5 digit prime . Generating a 5 digit prime .... 3193425841 6> reductionsieve:analyze(N2). [{1,4},{2,2},{5,8},{7,7}] 7> N3 = reductionsieve:make_prime(5) * reductionsieve:make_prime(5). Generating a 5 digit prime .... Generating a 5 digit prime . 5575777817 8> reductionsieve:analyze(N3). [{1,5},{2,7},{4,8}] The list filled with tuples that returns is variable in length, depending on the properties of any given N. I would like to be able to build a variable that takes each tuple from a list and multiplies them together, then adds them to the other products. For example, if N yields properties of [{1,5},{2,7},{4,8}], I need to be able to build a variable which is: (1 * 5) + (2 * 7) + (4 * 8); some N will return a list as small as three tuples, other N will return a list that may have as many as nine tuples that need to be multiplied in that fashion. I have been experimenting with lists:fold1 but it seems that is expecting a simple list without the use of tuples. Any ideas on this one? Thanks in advance From illo@REDACTED Fri Jul 10 17:29:18 2009 From: illo@REDACTED (Illo de' Illis) Date: Fri, 10 Jul 2009 17:29:18 +0200 Subject: [erlang-questions] Question on building a variable with multiple math operations In-Reply-To: References: Message-ID: <3B6A2669-4AFA-4B4B-BE82-39321BCFCD98@totalwire.it> On Jul 10, 2009, at 5:17 PM, Greg Perry wrote: > [...] The list filled with tuples that returns is variable in > length, depending on the properties of any given N. I would like to > be able to build a variable that takes each tuple from a list and > multiplies them together, then adds them to the other products. For > example, if N yields properties of [{1,5},{2,7},{4,8}], I need to be > able to build a variable which is: (1 * 5) + (2 * 7) + (4 * 8); some > N will return a list as small as three tuples, other N will return a > list that may have as many as nine tuples that need to be multiplied > in that fashion. > > I have been experimenting with lists:fold1 but it seems that is > expecting a simple list without the use of tuples. > > Any ideas on this one? Isn't it as simple as doing: lists:foldl(fun erlang:'+'/2, 0, [X*Y || {X,Y} <- N1]). ? Ciao, Illo. From erlang@REDACTED Fri Jul 10 17:30:14 2009 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 10 Jul 2009 17:30:14 +0200 Subject: [erlang-questions] compile transforms which change the module name In-Reply-To: <95be1d3b0907100750r7a498ej789f585e94f6877@mail.gmail.com> References: <9b08084c0907092355t2f649bf6tfea654f971ad991e@mail.gmail.com> <4A5703BD.6060705@erix.ericsson.se> <4A571156.3060208@erlang-consulting.com> <9b08084c0907100336j3b222881y30e16afbab9542c1@mail.gmail.com> <95be1d3b0907100352s7cb2ecd8ka2b9c7e6793f32ab@mail.gmail.com> <9b08084c0907100650s22940ed9ob767b5a7a838ff9@mail.gmail.com> <95be1d3b0907100750r7a498ej789f585e94f6877@mail.gmail.com> Message-ID: <9b08084c0907100830t2b73180dq2886aeadac5c87b5@mail.gmail.com> On Fri, Jul 10, 2009 at 4:50 PM, Vlad Dumitrescu wrote: > On Fri, Jul 10, 2009 at 15:50, Joe Armstrong wrote: >>> - it would be nice to have tools to do the reverse transformation for >>> logs and error messages; also, to be able to find the right version of >>> a module in ClearCase or SVN given the name and a MD5 hash. >> >> No need to the transformed module can have an added exported function: >> ? ?my_origonal_name() -> >> ? ? ? ? xxxx. > > That will handle the name, but the version number in the version > control system might be important too. Possibly another generated > function my_original_version_number() could handle that, but it's not > easy to cover all VCSs... > > Then again, the map() data might need to be saved somewhere too (and > version controlled). > > There seem to be many small things that have to be put in place. If > there weren't had been, the whole issue would have been trivial :-) > How about like GIT? every program has a parent (in the sense that you often just edit a file and it progresses through several versions) - so you just record the MD5 of the parent(s) /Joe > regards, > Vlad > From sverker@REDACTED Fri Jul 10 17:32:00 2009 From: sverker@REDACTED (Sverker Eriksson) Date: Fri, 10 Jul 2009 17:32:00 +0200 Subject: [erlang-questions] Question on building a variable with multiple math operations In-Reply-To: References: Message-ID: <4A575EF0.8000405@erix.ericsson.se> 1> lists:foldl(fun({A,B}, Acc) -> Acc + A*B end, 0, [{1,5},{2,7},{4,8}]). 51 /Sverker Greg Perry wrote: > I have a unique challenge that I can't seem to find a right way to do based on the immutability of Erlang. > > I have built a unique factoring application with Erlang, which first analyzes any given N and depending on the properties of that subprime number a list of coefficients are assigned to a list (the list contains tuples actually). > > http://blog.liveammo.com/2009/06/factoring-fun/ > > For example: > > $ erl > Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.7.2 (abort with ^G) > 1> c("reductionsieve.erl"). > {ok,reductionsieve} > 2> c("lin.erl"). > {ok,lin} > 3> N1 = reductionsieve:make_prime(5) * reductionsieve:make_prime(5). > Generating a 5 digit prime ...... > Generating a 5 digit prime .................. > 823221601 > 4> reductionsieve:analyze(N1). > [{1,7},{2,8},{4,4},{5,5}] > 5> N2 = reductionsieve:make_prime(5) * reductionsieve:make_prime(5). > Generating a 5 digit prime . > Generating a 5 digit prime .... > 3193425841 > 6> reductionsieve:analyze(N2). > [{1,4},{2,2},{5,8},{7,7}] > 7> N3 = reductionsieve:make_prime(5) * reductionsieve:make_prime(5). > Generating a 5 digit prime .... > Generating a 5 digit prime . > 5575777817 > 8> reductionsieve:analyze(N3). > [{1,5},{2,7},{4,8}] > > The list filled with tuples that returns is variable in length, depending on the properties of any given N. I would like to be able to build a variable that takes each tuple from a list and multiplies them together, then adds them to the other products. For example, if N yields properties of [{1,5},{2,7},{4,8}], I need to be able to build a variable which is: (1 * 5) + (2 * 7) + (4 * 8); some N will return a list as small as three tuples, other N will return a list that may have as many as nine tuples that need to be multiplied in that fashion. > > I have been experimenting with lists:fold1 but it seems that is expecting a simple list without the use of tuples. > > Any ideas on this one? > > Thanks in advance > > > > From illo@REDACTED Fri Jul 10 17:34:56 2009 From: illo@REDACTED (Illo de' Illis) Date: Fri, 10 Jul 2009 17:34:56 +0200 Subject: [erlang-questions] Question on building a variable with multiple math operations In-Reply-To: <3B6A2669-4AFA-4B4B-BE82-39321BCFCD98@totalwire.it> References: <3B6A2669-4AFA-4B4B-BE82-39321BCFCD98@totalwire.it> Message-ID: On Jul 10, 2009, at 5:29 PM, Illo de' Illis wrote: > On Jul 10, 2009, at 5:17 PM, Greg Perry wrote: > >> [...] >> I have been experimenting with lists:fold1 but it seems that is >> expecting a simple list without the use of tuples. >> >> Any ideas on this one? > > Isn't it as simple as doing: > > lists:foldl(fun erlang:'+'/2, 0, [X*Y || {X,Y} <- N1]). Or lists:foldl(fun ({X,Y},A) -> (X*Y)+A end, 0, N1). Ciao, Illo. From vladdu55@REDACTED Fri Jul 10 17:36:34 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 10 Jul 2009 17:36:34 +0200 Subject: [erlang-questions] compile transforms which change the module name In-Reply-To: <9b08084c0907100830t2b73180dq2886aeadac5c87b5@mail.gmail.com> References: <9b08084c0907092355t2f649bf6tfea654f971ad991e@mail.gmail.com> <4A5703BD.6060705@erix.ericsson.se> <4A571156.3060208@erlang-consulting.com> <9b08084c0907100336j3b222881y30e16afbab9542c1@mail.gmail.com> <95be1d3b0907100352s7cb2ecd8ka2b9c7e6793f32ab@mail.gmail.com> <9b08084c0907100650s22940ed9ob767b5a7a838ff9@mail.gmail.com> <95be1d3b0907100750r7a498ej789f585e94f6877@mail.gmail.com> <9b08084c0907100830t2b73180dq2886aeadac5c87b5@mail.gmail.com> Message-ID: <95be1d3b0907100836s53c0c480wce94b3b2618be7c3@mail.gmail.com> On Fri, Jul 10, 2009 at 17:30, Joe Armstrong wrote: > On Fri, Jul 10, 2009 at 4:50 PM, Vlad Dumitrescu wrote: >> On Fri, Jul 10, 2009 at 15:50, Joe Armstrong wrote: >>>> - it would be nice to have tools to do the reverse transformation for >>>> logs and error messages; also, to be able to find the right version of >>>> a module in ClearCase or SVN given the name and a MD5 hash. > > How about like GIT? every program has a parent (in the sense that you often > just edit a file and it progresses through several versions) - so you > just record the > MD5 of the parent(s) Of course, that is a solution, but to another problem. If I have my code already in a VCS, I still need to be able to match a certain hash to a certain version of the file. I assume here that the setting isn't the "normal" one where only one version is deployed and the version number is in a file somewhere, but you can have multiple versions running simultaneously in the system. regards, Vlad From peter@REDACTED Fri Jul 10 18:06:18 2009 From: peter@REDACTED (Peter Sabaini) Date: Fri, 10 Jul 2009 18:06:18 +0200 Subject: [erlang-questions] Max. number of distributed nodes? In-Reply-To: <1247148716.16255.5.camel@gram.local> References: <1247143519.20218.6.camel@ockham.local> <1247148716.16255.5.camel@gram.local> Message-ID: <1247241978.3900.6.camel@gram.local> Hm, I seem to keep talking to myself... Please let me rephrase: I want to get a feeling for to how many nodes an Erlang network can grow. The "Efficiency Guide" (http://erlang.org/doc/efficiency_guide/advanced.html#7.2 ) lists some system limits, but my tests suggest that the limits are much lower. Is that the case? Do people have networks with more than some tens of nodes in operation? Thank you, peter. On Thu, 2009-07-09 at 16:11 +0200, Peter Sabaini wrote: > I should maybe add that this is on a local network (gigabit) and the > network seems reasonably healthy: > > # ping -f -s1024 mass > PING mass.local (192.168.1.199) 1024(1052) bytes of data. > .^C > --- mass.local ping statistics --- > 116392 packets transmitted, 116391 received, 0% packet loss, time > 24487ms > rtt min/avg/max/mdev = 0.118/0.154/0.330/0.017 ms, ipg/ewma 0.210/0.153 > ms > > > I notice some network traffic when the nodes start up (spikes up to > ~5Mbit/s) but far from being saturated; also there is some load > (loadfactor ~1-3) on the machines during startup but nothing > extraordinary. > > On Thu, 2009-07-09 at 14:45 +0200, Peter Sabaini wrote: > > Hi, > > > > I'm currently experimenting with a distributed Erlang setup and I'm > > trying to get ~120 nodes spread (unevenly) over 5 machines to talk to > > each other. > > > > Unfortunately, I often get errors of the kind: > > > > =ERROR REPORT==== 9-Jul-2009::13:56:07 === > > The global_name_server locker process received an unexpected message: > > {{#Ref<0.0.0.1957>,'xy@REDACTED'},false} > > > > Or > > > > =ERROR REPORT==== 9-Jul-2009::14:03:33 === > > global: 'foo@REDACTED' failed to connect to 'qux@REDACTED' > > > > > > I tried: > > > > * setting ERL_MAX_PORTS to 100 000 > > > > * using kernel-poll > > > > * setting net_setuptime to 60s > > > > but the errors persist. > > > > Is there anything else I could try? > > > > Thanks, > > peter. > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From info@REDACTED Fri Jul 10 18:31:57 2009 From: info@REDACTED (info) Date: Fri, 10 Jul 2009 18:31:57 +0200 Subject: build a time&date string References: <200907101649089620930@its3.ch>, <3537d8a5-639d-40c0-9274-e36c1bc402f9@18g2000yqa.googlegroups.com> Message-ID: <200907101831572682521@its3.ch> Hi Jacob, Yes for the splitting: {ok, [Year, Mon, Day], []} = io_lib:fread("~2d~2d~2d", Date), {ok, [Hour, Min, Sec], []} = io_lib:fread("~2d~2d~2d", Time), and ... Args = [Year, Mon, Day, Hour, Min, Sec], S = io_lib:format("~B-~2.10.0B-~2.10.0BT~2.10.0B:~2.10.0B:~2.10.0BZ", Args), lists:flatten(S)I must find now how to add 2000 to Year.Thank you so much,John Hi John, I've got some for ISO datetime string conversion at http://bitbucket.org/japerk/elib/src/tip/src/datetime.erl Does that help? Jacob On Jul 10, 7:49 am, "info" wrote: > Hi all, > > Given a date and a time according the following formats: > Date = DDMMYY > Time = HHMMSS > > Is it possible to build a composite string only by using the formats ? > My problem is in the splitting ... > > Composite = YYYY-MM-DD HH:MM:SS > > John From Greg.Perry@REDACTED Fri Jul 10 20:13:27 2009 From: Greg.Perry@REDACTED (Greg Perry) Date: Fri, 10 Jul 2009 11:13:27 -0700 Subject: Problems with math:sqrt module Message-ID: I found what appears to be a problem with the multi precision math capabilities of Erlang, math:sqrt works fine for fairly large numbers but errors with a 2048 bit semiprime number, for example: 11> math:sqrt(3583144093694177840015703951812811819685389811654227013562705143239732783869692090471778208318589531694265620448681377942529266640295861138161515558643627087037345328746390159109263363877363094390). 5.985936930585034e97 ...works fine, but a 2048 bit number does not: 12> math:sqrt(358314409369417784001570395181281181968538981165422701356270514323973278386969209047177820831858953169426562044868137794252926664029586113816151555864362708703734532874639015910926336387736309439085113066643724657508768560218902742830031805141689201216753646475132835879627107400965023578409921487110512182961741061327409228793561655530765043642780534852247342070264867175687046721404689620610752507). ** exception error: bad argument in an arithmetic expression in function math:sqrt/1 called as math:sqrt(358314409369417784001570395181281181968538981165422701356270514323973278386969209047177820831858953169426562044868137794252926664029586113816151555864362708703734532874639015910926336387736309439085113066643724657508768560218902742830031805141689201216753646475132835879627107400965023578409921487110512182961741061327409228793561655530765043642780534852247342070264867175687046721404689620610752507) 13> any ideas? From joelr1@REDACTED Fri Jul 10 20:24:07 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 10 Jul 2009 19:24:07 +0100 Subject: 20k messages in 4s but want to go faster! Message-ID: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> I can't seem to send 20k messages in much less than 4 seconds and would appreciate any suggestions on how to improve my broadcasting speed. The messages are binary, 69 bytes, including the trailing 0 (Flash). There's no size prefix. Flow control is {active, once}, e.g. Opts = [binary, {packet, 0}, {reuseaddr, true}, {backlog, 1024}, {active, false}], The publishing routing looks like this, where Send is a closure with gen_tcp:send/2 and a captured socket. F = fun(_, {_, Send}, _) -> Send(Msg) end, F1 = fun() -> dict:fold(F, ignore, State#state.subscribers) end, spawn_link(F1), I'm running this on small EC2 instances and the actual fold takes < 200ms. I'm hesitant to attribute the difference to the TCP stack. I also tried this variant but only shaved 100-200ms from the 4s. F = fun(_, {_, Send}, _) -> spawn(fun() -> Send(Msg) end) end, dict:fold(F, ignore, State#state.subscribers), I tried distributing my test harness over 4 EC2 instances but this saved me a second or so, for the 4 second total I'm trying to beat now. What else can I do? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From matthew@REDACTED Fri Jul 10 20:29:35 2009 From: matthew@REDACTED (Matthew Sackman) Date: Fri, 10 Jul 2009 19:29:35 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> Message-ID: <20090710182935.GE4171@wellquite.org> On Fri, Jul 10, 2009 at 07:24:07PM +0100, Joel Reymont wrote: > I can't seem to send 20k messages in much less than 4 seconds and > would appreciate any suggestions on how to improve my broadcasting > speed. Have you tried turning of nagle? We've found it's almost always a bad idea to leave it on. Add {nodelay, true} to your options. Matthew From joelr1@REDACTED Fri Jul 10 20:36:45 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 10 Jul 2009 19:36:45 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <20090710182935.GE4171@wellquite.org> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> Message-ID: <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> On Jul 10, 2009, at 7:29 PM, Matthew Sackman wrote: > Have you tried turning of nagle? We've found it's almost always a bad > idea to leave it on. Add {nodelay, true} to your options. {nodelay, true} gives me 3.8-3.9s per 20k messages. I'm shooting to try to get it under 1s if possible. Is it possible? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From matthew@REDACTED Fri Jul 10 20:38:22 2009 From: matthew@REDACTED (Matthew Sackman) Date: Fri, 10 Jul 2009 19:38:22 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> Message-ID: <20090710183821.GF4171@wellquite.org> On Fri, Jul 10, 2009 at 07:36:45PM +0100, Joel Reymont wrote: > On Jul 10, 2009, at 7:29 PM, Matthew Sackman wrote: > >Have you tried turning of nagle? We've found it's almost always a bad > >idea to leave it on. Add {nodelay, true} to your options. > > {nodelay, true} gives me 3.8-3.9s per 20k messages. Try adding {sndbuf,16384} (or experiment with different sizes there). Matthew From joelr1@REDACTED Fri Jul 10 20:48:55 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 10 Jul 2009 19:48:55 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <20090710183821.GF4171@wellquite.org> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> Message-ID: <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> On Jul 10, 2009, at 7:38 PM, Matthew Sackman wrote: > Try adding {sndbuf,16384} (or experiment with different sizes there). Doesn't make much of a difference when added on the server side. My messages are just 69 bytes and each goes to a different socket. How does a large send buffer help in this case? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Fri Jul 10 20:51:32 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 10 Jul 2009 19:51:32 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <20090710183821.GF4171@wellquite.org> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> Message-ID: <29C72C5C-1E7E-4CFD-96EE-90E0C03F2A1A@gmail.com> On Jul 10, 2009, at 7:38 PM, Matthew Sackman wrote: > Try adding {sndbuf,16384} (or experiment with different sizes there). Also, I forgot to add that a single 69 byte message goes out to each of the 20k sockets. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From matthew@REDACTED Fri Jul 10 20:51:44 2009 From: matthew@REDACTED (Matthew Sackman) Date: Fri, 10 Jul 2009 19:51:44 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> Message-ID: <20090710185144.GG4171@wellquite.org> On Fri, Jul 10, 2009 at 07:48:55PM +0100, Joel Reymont wrote: > On Jul 10, 2009, at 7:38 PM, Matthew Sackman wrote: > My messages are just 69 bytes and each goes to a different socket. Ahh, sorry, hadn't realised you have a different socket for each message. Not sure if it can be done. Certainly RabbitMQ can receive, route and deliver 25k msgs/sec which are all > 69 bytes, but that's with one socket in and one socket out. I don't have any further ideas for you, sorry. Matthew From joelr1@REDACTED Fri Jul 10 21:09:28 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 10 Jul 2009 20:09:28 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <20090710185144.GG4171@wellquite.org> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> Message-ID: Any suggestions on how to measure delivery time from server to client? My "bot manager" runs on one node and monitors the 20k bots running on others. It starts the timer once it knows that all recepients are ready and waits to receive 20k 'DOWN' messages to report total broadcast time. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From andrewmmc@REDACTED Fri Jul 10 21:33:53 2009 From: andrewmmc@REDACTED (andrew mmc) Date: Fri, 10 Jul 2009 21:33:53 +0200 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> Message-ID: Hello, Just an idea, but with the example you tried before, where you spawn individual processes to send the message, the bottleneck is that the spawning process is serial. What about trying a version where you have a function that spawns two processes, one to send each half of the list. This should start off all the processes in one go once the list has been split up to lists of length 1 - which should be fast. I don't know how fully this will fill the bandwidth of your processors, but at least starting the processes will be concurrent. Andrew On Fri, Jul 10, 2009 at 9:09 PM, Joel Reymont wrote: > Any suggestions on how to measure delivery time from server to client? > > My "bot manager" runs on one node and monitors the 20k bots running on > others. It starts the timer once it knows that all recepients are ready and > waits to receive 20k 'DOWN' messages to report total broadcast time. > > Thanks, Joel > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From joelr1@REDACTED Fri Jul 10 21:36:07 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 10 Jul 2009 20:36:07 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> Message-ID: On Jul 10, 2009, at 8:33 PM, andrew mmc wrote: > This should start off all the processes in one go once the list has > been split up > to lists of length 1 - which should be fast. I apologize but I don't quite see it. I have a dict, I can convert it to a list but how do you propose to split the list, simply in half? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From andrewmmc@REDACTED Fri Jul 10 21:40:07 2009 From: andrewmmc@REDACTED (andrew mmc) Date: Fri, 10 Jul 2009 21:40:07 +0200 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> Message-ID: Yes, just in half each time. If the input list is of length 1, send the message, otherwise spawn two processes for each half of the list. On Fri, Jul 10, 2009 at 9:36 PM, Joel Reymont wrote: > > On Jul 10, 2009, at 8:33 PM, andrew mmc wrote: > > This should start off all the processes in one go once the list has been >> split up >> to lists of length 1 - which should be fast. >> > > I apologize but I don't quite see it. I have a dict, I can convert it to a > list but how do you propose to split the list, simply in half? > > > Thanks, Joel > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > From matthew@REDACTED Fri Jul 10 21:45:05 2009 From: matthew@REDACTED (Matthew Sackman) Date: Fri, 10 Jul 2009 20:45:05 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> Message-ID: <20090710194505.GH4171@wellquite.org> On Fri, Jul 10, 2009 at 09:40:07PM +0200, andrew mmc wrote: > Yes, just in half each time. If the input list is of length 1, send the > message, otherwise spawn two processes for each half of the list. Is spawning a process truly something that can happen in parallel? Are there really no VM-wide locks or data structures that have to happen at this point? Also, I have to say, I wouldn't use a dict. Immutable data structures in Erlang are _slow_ when compared to, eg, ets. OTOH, if you've timed the fold and found it's only taking 0.2 secs, then that's clearly not the problem here. Matthew From carnildo@REDACTED Fri Jul 10 21:49:16 2009 From: carnildo@REDACTED (Mark Wagner) Date: Fri, 10 Jul 2009 12:49:16 -0700 Subject: [erlang-questions] Problems with math:sqrt module In-Reply-To: References: Message-ID: <31073ef90907101249w5c56133at6be61ad9402364ff@mail.gmail.com> On Fri, Jul 10, 2009 at 11:13, Greg Perry wrote: > I found what appears to be a problem with the multi precision math capabilities of Erlang, math:sqrt works fine for fairly large numbers but errors with a 2048 bit semiprime number, for example: > > 11> math:sqrt(3583144093694177840015703951812811819685389811654227013562705143239732783869692090471778208318589531694265620448681377942529266640295861138161515558643627087037345328746390159109263363877363094390). ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?5.985936930585034e97 > > ...works fine, but a 2048 bit number does not: > > 12> math:sqrt(358314409369417784001570395181281181968538981165422701356270514323973278386969209047177820831858953169426562044868137794252926664029586113816151555864362708703734532874639015910926336387736309439085113066643724657508768560218902742830031805141689201216753646475132835879627107400965023578409921487110512182961741061327409228793561655530765043642780534852247342070264867175687046721404689620610752507). > ** exception error: bad argument in an arithmetic expression > ? ? in function ?math:sqrt/1 > ? ? ? ?called as math:sqrt(358314409369417784001570395181281181968538981165422701356270514323973278386969209047177820831858953169426562044868137794252926664029586113816151555864362708703734532874639015910926336387736309439085113066643724657508768560218902742830031805141689201216753646475132835879627107400965023578409921487110512182961741061327409228793561655530765043642780534852247342070264867175687046721404689620610752507) > 13> > > any ideas? > As I recall, the math: functions are wrappers to the underlying C library. If you're working on a typical IEEE 754 floating-point system and you've got a number larger than 10^308, you get overflow errors. From joelr1@REDACTED Fri Jul 10 21:55:05 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 10 Jul 2009 20:55:05 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <20090710194505.GH4171@wellquite.org> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> <20090710194505.GH4171@wellquite.org> Message-ID: <31B48CF7-1069-4699-ABCC-0FC83415C27B@gmail.com> On Jul 10, 2009, at 8:45 PM, Matthew Sackman wrote: > Is spawning a process truly something that can happen in parallel? Are > there really no VM-wide locks or data structures that have to happen > at > this point? The EC2 instances I'm using are single-core too. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From wolfmanjm@REDACTED Fri Jul 10 22:07:03 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Fri, 10 Jul 2009 13:07:03 -0700 (PDT) Subject: 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> Message-ID: I have a very similar system, but I "broadcast" messages sent by one client to all the other clients in the same group, there can be several thousand clients in a group. I time the latency from one client to all the other clients by putting a timestamp (using now()) in the message, then all the receiving clients calculate the delta between that timestamp and when they receive the message, this works because all the clients are running in the same node. This only times the latency between a client sending a message and a client getting the message, I am wondering if you are not including in your timing the process death and processing of the DOWN message as well, which could be a lot longer than sending/receiving the message. I suspect you are only interested in the latency between the bot sending the message and the 20k clients getting that message, not process exit time and associated garbage cleanup etc. In your case if the bots can communicate with the bot manager and all are running on the same node, maybe you could send the timestamp of when each bot gets the message to the bot manager, and the timestamp when the message was sent, and the bot manager would then collate all the timestamps once all are received and calculate average latency and min/max latency, this wouod time actual message latency and not associated setup/teardown of processes. On Jul 10, 12:09?pm, Joel Reymont wrote: > Any suggestions on how to measure delivery time from server to client? > > My "bot manager" runs on one node and monitors the 20k bots running on ? > others. It starts the timer once ?it knows that all recepients are ? > ready and waits to receive 20k 'DOWN' messages to report total ? > broadcast time. > > ? ? ? ? Thanks, Joel > > --- > Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From joelr1@REDACTED Fri Jul 10 22:17:12 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 10 Jul 2009 21:17:12 +0100 Subject: [erlang-questions] Re: 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> Message-ID: On Jul 10, 2009, at 9:07 PM, Jim Morris wrote: > I am wondering if you are not including in > your timing the process death and processing of the DOWN message as > well, which could be a lot longer than sending/receiving the message. I _am_ including all that in the timing, which probably skews my stats. That overhead would not be present when sending to a browser-based client. > I suspect you are only interested in the latency between the bot > sending the message and the 20k clients getting that message, not > process exit time and associated garbage cleanup etc. Absolutely right. > In your case if the bots can communicate with the bot manager and all > are running on the same node, Bots are running on different servers to spread the load. > maybe you could send the timestamp of > when each bot gets the message to the bot manager, and the timestamp > when the message was sent, I like the idea and I thought about it. I'm not sure if the EC2 instances are time-synced, though. There's the same monitoring and DOWN message overhead involved in firing the timer in the bots since I need to broadcast when all bots are ready and that's when bots tart their timers. A process exits when 'bumped' by the required number of bots. Both the broadcasting bot and the client bots listen for DOWN from this single process to publish and start timing. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From wolfmanjm@REDACTED Fri Jul 10 22:59:13 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Fri, 10 Jul 2009 13:59:13 -0700 (PDT) Subject: 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> Message-ID: <630dd16f-cd77-460b-b74a-e5dc2d32ef32@b15g2000yqd.googlegroups.com> For testing purposes if you could run the clients in the same node and the bot manager in a different node, then the clients would have the timestamp synced, and it would give you an idea of the latency. (Thats what I do, and I presume distributing the clients would be faster, at least I get a worst case scenario). What I do is I have all the clients send a message to the test manager (equivalent to your bot manager) when they are ready, then the test manager has one client send a message to the group, as each client gets the message it stores the ave/min/max latency, the tester manager waits a certain time (to allow all clients to get the message, I actually send multiple messages to get a good average) then sends a done message to each client. When the client gets the done message it sends the latency stats to the test manager, once the test manager has all the latencies it prints out the averages and min/max. This works pretty well and times the message latency through the system, and nothing else not even the communication time with the test manager. As I time multiple messages it also smoothes out any dynamic module load times that may occur hen getting the first message etc. What I am looking for is an accurate latency time, once the system is stable, not the startup time etc. I don't seem to be able to run 20,000 clients right now, probably memory limitation or something (it just hangs with no error dump, i need to debug that!) However I get a message latency of around 100ms worst case for 4,000 clients all running on the same node, and even the same machine as the tester. The average latency is much lower around 60ms. So I would expect to be able to get well under 1 second latency for 20,000 clients especially if they are distributed. One caveat though is my messages are UDP not TCP (its a voice server), but I can send them TCP, I'll give it a try and see if there is much difference. On Jul 10, 1:17?pm, Joel Reymont wrote: > On Jul 10, 2009, at 9:07 PM, Jim Morris wrote: > > > I am wondering if you are not including in > > your timing the process death and processing of the DOWN message as > > well, which could be a lot longer than sending/receiving the message. > > I _am_ including all that in the timing, which probably skews my stats. > That overhead would not be present when sending to a browser-based ? > client. > > > I suspect you are only interested in the latency between the bot > > sending the message and the 20k clients getting that message, not > > process exit time and associated garbage cleanup etc. > > Absolutely right. > > > In your case if the bots can communicate with the bot manager and all > > are running on the same node, > > Bots are running on different servers to spread the load. > > > maybe you could send the timestamp of > > when each bot gets the message to the bot manager, and the timestamp > > when the message was sent, > > I like the idea and I thought about it. I'm not sure if the EC2 ? > instances are time-synced, though. > > There's the same monitoring and DOWN message overhead involved in ? > firing the timer in the bots since I need to broadcast when all bots ? > are ready and that's when bots tart their timers. > > A process exits when 'bumped' by the required number of bots. Both the ? > broadcasting bot and the ?client bots listen for DOWN from this single ? > process to publish and start timing. > > ? ? ? ? Thanks, Joel > > --- > Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From mikpe@REDACTED Fri Jul 10 23:02:48 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Fri, 10 Jul 2009 23:02:48 +0200 Subject: [erlang-bugs] Possible bug in HiPE pattern matching In-Reply-To: <877FEB4B-A358-4C41-B831-21B61D5CE101@jonmeredith.com> References: <877FEB4B-A358-4C41-B831-21B61D5CE101@jonmeredith.com> Message-ID: <19031.44152.794781.272170@pilspetsen.it.uu.se> Jon Meredith writes: > Hi all, > > I think I've discovered an issue with pattern matching under HiPE > going back on at least 5.6.3 under Debian/lenny and 5.7.2 under OS X > > Here is a module that demonstrates the issue > > -module(bug). > -export([f/1,g/1]). > f(0) -> zero; > f(-10) -> one; > f(-20) -> two; > f(-30) -> three; > f(-40) -> four; > f(-50) -> five; > f(-60) -> six; > f(-70) -> seven; > f(-80) -> eight; %% Uncomment this line to make it work with HiPE > f(X) -> {error, X}. > > g(X) -> > case X of > 0 -> zero; > -10 -> one; > -20 -> two; > -30 -> three; > -40 -> four; > -50 -> five; > -60 -> six; > -70 -> seven; > -80 -> eight; %% Uncomment this line to make it work with HiPE > X -> {error, X} > end. > > Running normally without HiPE it does what I expect > > jmeredith@REDACTED:~$ /usr/bin/erlc bug.erl > jmeredith@REDACTED:~$ /usr/bin/erl > Erlang (BEAM) emulator version 5.6.3 [source] [64-bit] [smp:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.6.3 (abort with ^G) > 1> bug:f(0). > zero > 2> bug:g(0). > zero > > If I compile with +native, the pattern doesn't match the first case as > it should > > jmeredith@REDACTED:~$ /usr/bin/erlc +native bug.erl > jmeredith@REDACTED:~$ /usr/bin/erl > Erlang (BEAM) emulator version 5.6.3 [source] [64-bit] [smp:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.6.3 (abort with ^G) > 1> bug:f(0). > {error,0} > 2> bug:g(0). > {error,0} This turned out to be a long-standing bug that only triggers for largish but sparse cases on negative small integers, where the generated search code did not account for key table transformations done at load-time (to facilitate binary searches on atoms). The fix is included below. Thanks for reporting the problem and for providing such a good (small) test case. /Mikael --- otp-0710/lib/hipe/rtl/hipe_rtl_mk_switch.erl.~1~ 2009-03-18 21:56:34.000000000 +0100 +++ otp-0710/lib/hipe/rtl/hipe_rtl_mk_switch.erl 2009-07-10 13:49:11.000000000 +0200 @@ -880,7 +880,7 @@ tab(KeyList, LabelList, KeyReg, TablePnt [ hipe_rtl:mk_move(IndexReg,hipe_rtl:mk_imm(0)), hipe_rtl:mk_load(Temp,TablePntrReg,hipe_rtl:mk_imm(Init)), - hipe_rtl:mk_branch(Temp, ge, KeyReg, + hipe_rtl:mk_branch(Temp, geu, KeyReg, hipe_rtl:label_name(Lab2), hipe_rtl:label_name(Lab1), 0.5), Lab1, @@ -911,7 +911,7 @@ step(I,TablePntrReg,IndexReg,KeyReg) -> Lab2 = hipe_rtl:mk_new_label(), [hipe_rtl:mk_alu(TempIndex, IndexReg, add, hipe_rtl:mk_imm(I)), hipe_rtl:mk_load(Temp,TablePntrReg,TempIndex), - hipe_rtl:mk_branch(Temp, gt, KeyReg, + hipe_rtl:mk_branch(Temp, gtu, KeyReg, hipe_rtl:label_name(Lab2), hipe_rtl:label_name(Lab1) , 0.5), Lab1] ++ From wolfmanjm@REDACTED Fri Jul 10 23:44:38 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Fri, 10 Jul 2009 14:44:38 -0700 (PDT) Subject: 20k messages in 4s but want to go faster! In-Reply-To: <630dd16f-cd77-460b-b74a-e5dc2d32ef32@b15g2000yqd.googlegroups.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> <630dd16f-cd77-460b-b74a-e5dc2d32ef32@b15g2000yqd.googlegroups.com> Message-ID: The TCP test case is a little slower, max 216ms latency, Ave 80ms latency for 4,000 clients > I can send them TCP, I'll give it a try and see if there is much > difference. > From joelr1@REDACTED Sat Jul 11 00:36:21 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 10 Jul 2009 23:36:21 +0100 Subject: [erlang-questions] Re: 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> <630dd16f-cd77-460b-b74a-e5dc2d32ef32@b15g2000yqd.googlegroups.com> Message-ID: Is that on the same node? On Jul 10, 2009, at 10:44 PM, Jim Morris wrote: > The TCP test case is a little slower, max 216ms latency, Ave 80ms > latency for 4,000 clients > >> I can send them TCP, I'll give it a try and see if there is much >> difference. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From wolfmanjm@REDACTED Sat Jul 11 00:45:30 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Fri, 10 Jul 2009 15:45:30 -0700 (PDT) Subject: 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> <630dd16f-cd77-460b-b74a-e5dc2d32ef32@b15g2000yqd.googlegroups.com> Message-ID: All the clients are on the same node, and the test manager is on the same machine but a different Node. The test manager can be on any machine as it doesn't look at the timestamps, the clients calculate the deltas, the manager just prints out the results. On Jul 10, 3:36?pm, Joel Reymont wrote: > Is that on the same node? > > On Jul 10, 2009, at 10:44 PM, Jim Morris wrote: > > > The TCP test case is a little slower, max 216ms latency, Ave 80ms > > latency for 4,000 clients > > >> I can send them TCP, I'll give it a try and see if there is much > >> difference. > > --- > Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From joelr1@REDACTED Sat Jul 11 00:55:44 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 10 Jul 2009 23:55:44 +0100 Subject: [erlang-questions] Re: 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> <630dd16f-cd77-460b-b74a-e5dc2d32ef32@b15g2000yqd.googlegroups.com> Message-ID: <3493B3B9-76BA-4009-8F29-F8D49CC4AA91@gmail.com> On Jul 10, 2009, at 11:45 PM, Jim Morris wrote: > All the clients are on the same node, and the test manager is on the > same machine but a different Node. The test manager can be on any > machine as it doesn't look at the timestamps, the clients calculate > the deltas, the manager just prints out the results. My issue is that I need to calculate the "broadcast" latency from server to the client which means that the timestamp has to be set on the server. I took a look at the time difference between erl on two Amazon EC2 instances and there doesn't seem to be much of a difference, e.g. timer:now_diff({1247,266330,594942}, {1247,266331,294060}). -699118 I had to switch windows and type enter so that accounts for some of the difference. 70ms should not make a difference against the 4s that I'm trying to cut down. I'll try to stamp messages on the server and see what I get on the client tomorrow. I already have a stats server to calculate min, max, avg, etc. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From tony@REDACTED Sat Jul 11 00:56:43 2009 From: tony@REDACTED (Tony Arcieri) Date: Fri, 10 Jul 2009 16:56:43 -0600 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> Message-ID: On Fri, Jul 10, 2009 at 12:24 PM, Joel Reymont wrote: > I also tried this variant but only shaved 100-200ms from the 4s. > > F = fun(_, {_, Send}, _) -> spawn(fun() -> Send(Msg) end) end, > dict:fold(F, ignore, State#state.subscribers), > > I tried distributing my test harness over 4 EC2 instances but this saved me > a second or so, for the 4 second total I'm trying to beat now. > > What else can I do? > Have you tried persisting the processes? -- Tony Arcieri medioh.com From bqt@REDACTED Sat Jul 11 01:24:46 2009 From: bqt@REDACTED (Johnny Billquist) Date: Sat, 11 Jul 2009 01:24:46 +0200 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <20090710182935.GE4171@wellquite.org> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> Message-ID: <4A57CDBE.2030202@softjar.se> Matthew Sackman wrote: > On Fri, Jul 10, 2009 at 07:24:07PM +0100, Joel Reymont wrote: >> I can't seem to send 20k messages in much less than 4 seconds and >> would appreciate any suggestions on how to improve my broadcasting >> speed. > > Have you tried turning of nagle? We've found it's almost always a bad > idea to leave it on. Add {nodelay, true} to your options. Nagle is actually a good thing, which improves performance, not decrease it. There is one specific situation when you don't want nagle, and that is if you send very little data, and what it to be sent immediately. In that specific case, Nagle is an obstruction and should be shut off. So, unless we talk 20k messages to different ports, and all of them being small, turning off Nagle will only actually decrease performance, not increase it. (The reason for the gain in performance is that Nagle will actually make you send tcp packets with as much data as MSS specifies, instead of sending loads of smaller packets which will cause more overhead, since each packet also implies both a tcp and an ip header, which isn't payload.) Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: bqt@REDACTED || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol From bqt@REDACTED Sat Jul 11 01:31:38 2009 From: bqt@REDACTED (Johnny Billquist) Date: Sat, 11 Jul 2009 01:31:38 +0200 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <4A57CDBE.2030202@softjar.se> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <4A57CDBE.2030202@softjar.se> Message-ID: <4A57CF5A.8010402@softjar.se> Johnny Billquist wrote: > Matthew Sackman wrote: >> On Fri, Jul 10, 2009 at 07:24:07PM +0100, Joel Reymont wrote: >>> I can't seem to send 20k messages in much less than 4 seconds and >>> would appreciate any suggestions on how to improve my broadcasting >>> speed. >> >> Have you tried turning of nagle? We've found it's almost always a bad >> idea to leave it on. Add {nodelay, true} to your options. > > Nagle is actually a good thing, which improves performance, not decrease > it. > > There is one specific situation when you don't want nagle, and that is > if you send very little data, and what it to be sent immediately. In > that specific case, Nagle is an obstruction and should be shut off. > > So, unless we talk 20k messages to different ports, and all of them > being small, turning off Nagle will only actually decrease performance, > not increase it. > (The reason for the gain in performance is that Nagle will actually make > you send tcp packets with as much data as MSS specifies, instead of > sending loads of smaller packets which will cause more overhead, since > each packet also implies both a tcp and an ip header, which isn't payload.) DOH! And blind as a bat... Broadcast? If so, then we're not talking tcp here, and there isn't even any Nagle around... So you can forget all about this thread. I would assume you are using UDP. Since you seem to be sending each message to a different socket, I'd have to ask what you really are trying to do? :-) Maybe you are approaching the whole thing the wrong way? Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: bqt@REDACTED || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol From matthew@REDACTED Sat Jul 11 01:33:57 2009 From: matthew@REDACTED (Matthew Sackman) Date: Sat, 11 Jul 2009 00:33:57 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <4A57CDBE.2030202@softjar.se> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <4A57CDBE.2030202@softjar.se> Message-ID: <20090710233357.GA10626@wellquite.org> On Sat, Jul 11, 2009 at 01:24:46AM +0200, Johnny Billquist wrote: > Nagle is actually a good thing, which improves performance, not decrease it. > > There is one specific situation when you don't want nagle, and that > is if you send very little data, and what it to be sent immediately. > In that specific case, Nagle is an obstruction and should be shut > off. Yup. The immediate part shouldn't be underestimated. The improvement to latency can be significant. If you're doing a lot of client-server communication, then disabling Nagle can improve throughput substantially as the latency comes down thus permitting higher round trips per second. Matthew From wolfmanjm@REDACTED Sat Jul 11 02:00:09 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Fri, 10 Jul 2009 17:00:09 -0700 (PDT) Subject: 20k messages in 4s but want to go faster! In-Reply-To: <4A57CF5A.8010402@softjar.se> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <4A57CDBE.2030202@softjar.se> <4A57CF5A.8010402@softjar.se> Message-ID: On Jul 10, 4:31?pm, Johnny Billquist wrote: > Johnny Billquist wrote: > DOH! And blind as a bat... Broadcast? If so, then we're not talking tcp > here, and there isn't even any Nagle around... I think Joel means Unicast rather than Broadcast, ie sending a TCP packet to 20,000 tcp connected clients. From harveyd@REDACTED Sat Jul 11 02:04:55 2009 From: harveyd@REDACTED (Dale Harvey) Date: Sat, 11 Jul 2009 01:04:55 +0100 Subject: [erlang-questions] Documentation build process: creating DocBook XML? In-Reply-To: <47B5A2A1.8040107@cronqvi.st> References: <8EDCB178-4069-4F2B-A5BF-385C1D5D0565@bway.net> <47B14F39.9080509@gmail.com> <18357.34938.691663.177763@antilipe.corelatus.se> <47B5A2A1.8040107@cronqvi.st> Message-ID: Is there any updates on when the build sources for the documentation will be released? 2008/2/15 Mats Cronqvist > Matthias Lang wrote: > > I have a shell script which does this: > > > > ~ >eman io fwr > > showing > file:///usr/local/src/otp_doc_R11B-4/lib/stdlib-1.14.4/doc/html/io.html#fwrite/1 > > > > (it pops up the page in the currently running browser). As far as I > > know, I'm the only user. The web version doesn't get much traffic > > either. > i used a (modified) version of it. worked fine, but i liked > gotapi.com/erlang better. alas, since R12 that's > stopped working properly... > > mats > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://www.erlang.org/mailman/listinfo/erlang-questions > From mjtruog@REDACTED Sat Jul 11 02:14:10 2009 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 10 Jul 2009 17:14:10 -0700 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <20090710233357.GA10626@wellquite.org> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <4A57CDBE.2030202@softjar.se> <20090710233357.GA10626@wellquite.org> Message-ID: <4A57D952.6000801@gmail.com> Matthew Sackman wrote: > On Sat, Jul 11, 2009 at 01:24:46AM +0200, Johnny Billquist wrote: > >> Nagle is actually a good thing, which improves performance, not decrease it. >> >> There is one specific situation when you don't want nagle, and that >> is if you send very little data, and what it to be sent immediately. >> In that specific case, Nagle is an obstruction and should be shut >> off. >> > > Yup. The immediate part shouldn't be underestimated. The improvement to > latency can be significant. If you're doing a lot of client-server > communication, then disabling Nagle can improve throughput substantially > as the latency comes down thus permitting higher round trips per second. > > Matthew > As clarification, disabling Nagle is best for low latency networks and improves throughput. High latency networks can benefit from the algorithm, so it is generally used unless it is an internal cluster of computers. See http://en.wikipedia.org/wiki/Nagle's_algorithm to see why real-time systems avoid Nagle. From bqt@REDACTED Sat Jul 11 02:25:13 2009 From: bqt@REDACTED (Johnny Billquist) Date: Sat, 11 Jul 2009 02:25:13 +0200 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <4A57D952.6000801@gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <4A57CDBE.2030202@softjar.se> <20090710233357.GA10626@wellquite.org> <4A57D952.6000801@gmail.com> Message-ID: <4A57DBE9.3030007@softjar.se> Michael Truog wrote: > Matthew Sackman wrote: >> On Sat, Jul 11, 2009 at 01:24:46AM +0200, Johnny Billquist wrote: >> >>> Nagle is actually a good thing, which improves performance, not decrease it. >>> >>> There is one specific situation when you don't want nagle, and that >>> is if you send very little data, and what it to be sent immediately. >>> In that specific case, Nagle is an obstruction and should be shut >>> off. >>> >> Yup. The immediate part shouldn't be underestimated. The improvement to >> latency can be significant. If you're doing a lot of client-server >> communication, then disabling Nagle can improve throughput substantially >> as the latency comes down thus permitting higher round trips per second. >> >> Matthew >> > As clarification, disabling Nagle is best for low latency networks and > improves throughput. High latency networks can benefit from the > algorithm, so it is generally used unless it is an internal cluster of > computers. > > See http://en.wikipedia.org/wiki/Nagle's_algorithm > to see why real-time systems avoid Nagle. Hmm, I can't see how Nagle would help if you have a high latency network. Don't make no sense at all. That's not what Nagle is for. That's what you have sliding window protocols for. And large packets, large windows and large buffers. But any high volume connection will benifit from Nagle. Try turning off Nagle and do a file transfer, and you could become very disappointed with the performance. What Nagle really aims at, is when you have an application send lots of small pieces of data. Without Nagle, each little bit will be send it it's own TCP segment. With Nagle, all those small sends will be bundled together in a larger TCP segment. Now, Nagle don't neccesarily delay data, it only delays data if you have a segment smaller than MSS, otherwise it will be send right away anyway, since there is no point in delaying that packet to try and aggregate data. And file transfers are usually not sending data in chunks adapted to the MSS, but instead just sends whatever chunk size is convenient for the tcp implementation. Nagle will make sure that the link is used as efficiently as possible. As I said from the start, the only time you should disable Nagle is if you are sending *little* data, and want low latency. If you have high volumes of data, Nagle will help you, by either not interfering at all (if your applicion is sending chunks larger that the MSS), or aggregating your data into as large IP packets as possible (thus reducing the protocol overhead, and thereby improving throughput). Real time systems should probably avoid Nagle, because they should be very concerned with latency. More so than throughput. :-) Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: bqt@REDACTED || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol From mjtruog@REDACTED Sat Jul 11 02:37:37 2009 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 10 Jul 2009 17:37:37 -0700 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <4A57DBE9.3030007@softjar.se> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <4A57CDBE.2030202@softjar.se> <20090710233357.GA10626@wellquite.org> <4A57D952.6000801@gmail.com> <4A57DBE9.3030007@softjar.se> Message-ID: <4A57DED1.80005@gmail.com> Johnny Billquist wrote: > Michael Truog wrote: >> Matthew Sackman wrote: >>> On Sat, Jul 11, 2009 at 01:24:46AM +0200, Johnny Billquist wrote: >>> >>>> Nagle is actually a good thing, which improves performance, not >>>> decrease it. >>>> >>>> There is one specific situation when you don't want nagle, and that >>>> is if you send very little data, and what it to be sent immediately. >>>> In that specific case, Nagle is an obstruction and should be shut >>>> off. >>>> >>> Yup. The immediate part shouldn't be underestimated. The improvement to >>> latency can be significant. If you're doing a lot of client-server >>> communication, then disabling Nagle can improve throughput >>> substantially >>> as the latency comes down thus permitting higher round trips per >>> second. >>> >>> Matthew >>> >> As clarification, disabling Nagle is best for low latency networks and >> improves throughput. High latency networks can benefit from the >> algorithm, so it is generally used unless it is an internal cluster of >> computers. >> >> See http://en.wikipedia.org/wiki/Nagle's_algorithm >> to see why real-time systems avoid Nagle. > > Hmm, I can't see how Nagle would help if you have a high latency > network. Don't make no sense at all. That's not what Nagle is for. > That's what you have sliding window protocols for. And large packets, > large windows and large buffers. > > But any high volume connection will benifit from Nagle. Try turning > off Nagle and do a file transfer, and you could become very > disappointed with the performance. > > What Nagle really aims at, is when you have an application send lots > of small pieces of data. Without Nagle, each little bit will be send > it it's own TCP segment. With Nagle, all those small sends will be > bundled together in a larger TCP segment. Now, Nagle don't neccesarily > delay data, it only delays data if you have a segment smaller than > MSS, otherwise it will be send right away anyway, since there is no > point in delaying that packet to try and aggregate data. > > And file transfers are usually not sending data in chunks adapted to > the MSS, but instead just sends whatever chunk size is convenient for > the tcp implementation. Nagle will make sure that the link is used as > efficiently as possible. > > As I said from the start, the only time you should disable Nagle is if > you are sending *little* data, and want low latency. If you have high > volumes of data, Nagle will help you, by either not interfering at all > (if your applicion is sending chunks larger that the MSS), or > aggregating your data into as large IP packets as possible (thus > reducing the protocol overhead, and thereby improving throughput). > > Real time systems should probably avoid Nagle, because they should be > very concerned with latency. More so than throughput. :-) > > Johnny > Sorry for making this thread longer than it needs to be. For an understanding of why Nagle's algorithm is used, there is some information here: http://searchnetworking.techtarget.com/sDefinition/0,,sid7_gci754347,00.html "... this process (called /nagling/) increases the efficiency of a network application system by decreasing the number of packet s that must be sent." So decreasing number of packets helps with high latency networks. It also makes sense that most real-time systems don't care much about high throughput, but are instead more focused on small, low latency messages. However, I can imagine there are real-time systems where throughput is a concern (in addition to latency). From shelton.ms@REDACTED Sat Jul 11 10:58:53 2009 From: shelton.ms@REDACTED (Shelton Tang) Date: Sat, 11 Jul 2009 16:58:53 +0800 Subject: any limitation on outbound tcp connections on Windows In-Reply-To: <27ccae8e0907092337h5837177xdf7d61dac2445f5c@mail.gmail.com> References: <27ccae8e0907091953m3548e557w49db4d2cc9663726@mail.gmail.com> <27ccae8e0907092337h5837177xdf7d61dac2445f5c@mail.gmail.com> Message-ID: <27ccae8e0907110158hb53acbame7985885e586dd8f@mail.gmail.com> i find it's caused by the native dns resolution. thanks. On Fri, Jul 10, 2009 at 2:37 PM, Shelton Tang wrote: > I wrote a test program with C# and didn't see such problem. is this a > limitation in erlang@REDACTED? > > > On Fri, Jul 10, 2009 at 10:53 AM, Shelton Tang wrote: >> Hi All, >> >> I use gen_tcp:connect to make outbound TCP connection. ?The argument >> passed to the call is ?[list, {packet, 0},{reuseaddr, true},{active, >> false}] (I also tried binary or passive mode but no luck). ?However, >> my test shows the maximum concurrent connections are <= 10 on my xp >> and 5 - 8 on a X64 win2k8 machine. All other connections got timeout >> error. So I captured a network traffic and didn't find any frame for >> the timeout connections. Then I try to connect to the local host with >> 127.0.0.1. On my WinXP box, I can create about 32 connections and >> other got econnrefused error (the server is an erlang socket server). >> >> I know there is a 10 SYN limitation on winxp tcpip.sys. ?However, the >> confused is why i hit the problem on a powerful (16G memory, 4 >> processors with dual cores) x64 machine? Possible reasons: >> 1. is there any limitation in erlang? ?(becoz i can make > 10 >> connections to 127.0.0.1, most like it is not an erlang issue). >> 2. is there any limitation on tcpip.sys on win2k8? >> 3. is it possible there is a restraction on my firewall? (but if so, i >> should see any network frames but actually i didn't find it). >> >> any ideas? >> >> Regards, >> Shelton >> > From v@REDACTED Sat Jul 11 11:14:14 2009 From: v@REDACTED (Valentin Micic) Date: Sat, 11 Jul 2009 11:14:14 +0200 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <20090710185144.GG4171@wellquite.org> Message-ID: <200907110912.n6B9CNlc001893@mail.pharos-avantgard.com> Maybe it's a time for Joel to consider switching to UDP. V. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Matthew Sackman Sent: 10 July 2009 08:52 PM To: Joel Reymont Cc: Erlang Users' List Subject: Re: [erlang-questions] 20k messages in 4s but want to go faster! On Fri, Jul 10, 2009 at 07:48:55PM +0100, Joel Reymont wrote: > On Jul 10, 2009, at 7:38 PM, Matthew Sackman wrote: > My messages are just 69 bytes and each goes to a different socket. Ahh, sorry, hadn't realised you have a different socket for each message. Not sure if it can be done. Certainly RabbitMQ can receive, route and deliver 25k msgs/sec which are all > 69 bytes, but that's with one socket in and one socket out. I don't have any further ideas for you, sorry. Matthew ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From joelr1@REDACTED Sat Jul 11 11:33:43 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 11 Jul 2009 10:33:43 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <200907110912.n6B9CNlc001893@mail.pharos-avantgard.com> References: <200907110912.n6B9CNlc001893@mail.pharos-avantgard.com> Message-ID: <45505489-6BED-49B9-BA17-F85D1A6F9BE0@gmail.com> On Jul 11, 2009, at 10:14 AM, Valentin Micic wrote: > Maybe it's a time for Joel to consider switching to UDP. I can't, I'm dealing with browser-based clients in production. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Sat Jul 11 11:34:47 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 11 Jul 2009 10:34:47 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> Message-ID: On Jul 10, 2009, at 11:56 PM, Tony Arcieri wrote: > Have you tried persisting the processes? What do you mean? --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From bqt@REDACTED Sat Jul 11 13:02:31 2009 From: bqt@REDACTED (Johnny Billquist) Date: Sat, 11 Jul 2009 13:02:31 +0200 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <200907110912.n6B9CNlc001893@mail.pharos-avantgard.com> References: <200907110912.n6B9CNlc001893@mail.pharos-avantgard.com> Message-ID: <4A587147.3090206@softjar.se> From a performance point of view, yes. Definitely. UDP will give so much more throughput. However, it then also becomes a question of what he thinks of lost packets? Johnny Valentin Micic wrote: > Maybe it's a time for Joel to consider switching to UDP. > > V. > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Matthew Sackman > Sent: 10 July 2009 08:52 PM > To: Joel Reymont > Cc: Erlang Users' List > Subject: Re: [erlang-questions] 20k messages in 4s but want to go faster! > > On Fri, Jul 10, 2009 at 07:48:55PM +0100, Joel Reymont wrote: >> On Jul 10, 2009, at 7:38 PM, Matthew Sackman wrote: >> My messages are just 69 bytes and each goes to a different socket. > > Ahh, sorry, hadn't realised you have a different socket for each > message. Not sure if it can be done. Certainly RabbitMQ can receive, > route and deliver 25k msgs/sec which are all > 69 bytes, but that's with > one socket in and one socket out. I don't have any further ideas for > you, sorry. > > Matthew > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: bqt@REDACTED || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol From ulf.wiger@REDACTED Sat Jul 11 17:33:24 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 11 Jul 2009 17:33:24 +0200 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <31B48CF7-1069-4699-ABCC-0FC83415C27B@gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> <20090710194505.GH4171@wellquite.org> <31B48CF7-1069-4699-ABCC-0FC83415C27B@gmail.com> Message-ID: <4A58B0C4.9090705@erlang-consulting.com> Are your performance figures taken from the EC2 instances? I don't have that much experience with running erlang on virtual processors, but what I've seen so far leads me to believe that IO in particular can be a problem. BR, Ulf W Joel Reymont wrote: > > On Jul 10, 2009, at 8:45 PM, Matthew Sackman wrote: > >> Is spawning a process truly something that can happen in parallel? Are >> there really no VM-wide locks or data structures that have to happen at >> this point? > > > The EC2 instances I'm using are single-core too. > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From joelr1@REDACTED Sat Jul 11 17:35:30 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 11 Jul 2009 16:35:30 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <4A58B0C4.9090705@erlang-consulting.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> <20090710194505.GH4171@wellquite.org> <31B48CF7-1069-4699-ABCC-0FC83415C27B@gmail.com> <4A58B0C4.9090705@erlang-consulting.com> Message-ID: On Jul 11, 2009, at 4:33 PM, Ulf Wiger wrote: > I don't have that much experience with running erlang > on virtual processors, but what I've seen so far leads > me to believe that IO in particular can be a problem. Yes, these are EC2 figures. Can you elaborate on how IO is a problem on virtual processors? Would this apply to Solaris containers as well, e.g. Joyent accelerators? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From ulf.wiger@REDACTED Sat Jul 11 17:41:35 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 11 Jul 2009 17:41:35 +0200 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> <20090710194505.GH4171@wellquite.org> <31B48CF7-1069-4699-ABCC-0FC83415C27B@gmail.com> <4A58B0C4.9090705@erlang-consulting.com> Message-ID: <4A58B2AF.1090207@erlang-consulting.com> Joel Reymont wrote: > > On Jul 11, 2009, at 4:33 PM, Ulf Wiger wrote: > >> I don't have that much experience with running erlang >> on virtual processors, but what I've seen so far leads >> me to believe that IO in particular can be a problem. > > > Yes, these are EC2 figures. Can you elaborate on how > IO is a problem on virtual processors? Not much - limited (experience) was the operative word. Apart from comments here and there, we did run some SMTP benchmarks on virtual instances, and they really sucked compared to running them on dedicated machines. We got similar results when reducing the tests to raw socket benchmarks. Our conclusion in that particular project was that virtual instances were not an option for the kind of performance we had in mind. (Deliberately fuzzy on the details here. I encourage you to run your benchmarks on some reference hardware.) I believe that you can pay Amazon extra for better bandwidth. Not sure if this would have any effect on your particular throughput numbers, though. BR, Ulf W > > Would this apply to Solaris containers as well, > e.g. Joyent accelerators? > > Thanks, Joel > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From joelr1@REDACTED Sat Jul 11 17:51:16 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 11 Jul 2009 16:51:16 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> Message-ID: On Jul 10, 2009, at 8:40 PM, andrew mmc wrote: > Yes, just in half each time. If the input list is of length 1, send > the message, otherwise spawn two processes for each half of the list. I tried this by keeping two separate ets tables and alternatively inserting into them. I spawned two processes to broadcast, each iterating over their respective ets table. I did not see a performance improvement, maybe this has to do with running on a 1-core virtual CPU. I will try to send timestamps around next on the assumption that my timing calculations are flawed. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From kenneth.lundin@REDACTED Sat Jul 11 23:21:18 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Sat, 11 Jul 2009 23:21:18 +0200 Subject: [erlang-questions] Documentation build process: creating DocBook XML? In-Reply-To: References: <8EDCB178-4069-4F2B-A5BF-385C1D5D0565@bway.net> <47B14F39.9080509@gmail.com> <18357.34938.691663.177763@antilipe.corelatus.se> <47B5A2A1.8040107@cronqvi.st> Message-ID: Most of the doc sources are already there. What's missing is working make files to build the doc and the top level documentation. But for all applications there is a /doc/src directory containing the xml files and other files which is the doc source for that application. The plan is to improve this gradually i.e. with working make files for open source users and also the tools for building to html and pdf. Currently we are using the OTP application docbuilder for generating the html , man and pdf formats but we have far gone plans to change both the tool chain and the dtd's for the sources. When it comes to tools we are leaning strongly towards xsltproc for the translation to (x)html and xsltproc + Apache FOP for the translation to PDF. The DTD's we are using are described in the docbuilder docs. We have plans to reduce the number of DTD's as well. Using XMLSchema instead. /Kenneth Erlang/OTP Ericsson On Sat, Jul 11, 2009 at 2:04 AM, Dale Harvey wrote: > Is there any updates on when the build sources for the documentation > will be released? > > 2008/2/15 Mats Cronqvist > >> Matthias Lang wrote: >> > I have a shell script which does this: >> > >> > ? ~ >eman io fwr >> > ? showing >> file:///usr/local/src/otp_doc_R11B-4/lib/stdlib-1.14.4/doc/html/io.html#fwrite/1 >> > >> > (it pops up the page in the currently running browser). As far as I >> > know, I'm the only user. The web version doesn't get much traffic >> > either. >> ? i used a (modified) version of it. worked fine, but i liked >> gotapi.com/erlang better. alas, since R12 that's >> stopped working properly... >> >> ?mats >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://www.erlang.org/mailman/listinfo/erlang-questions >> > From tuncer.ayaz@REDACTED Sun Jul 12 01:29:57 2009 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 12 Jul 2009 01:29:57 +0200 Subject: [erlang-questions] Documentation build process: creating DocBook XML? In-Reply-To: References: <8EDCB178-4069-4F2B-A5BF-385C1D5D0565@bway.net> <47B14F39.9080509@gmail.com> <18357.34938.691663.177763@antilipe.corelatus.se> <47B5A2A1.8040107@cronqvi.st> Message-ID: <4ac8254d0907111629y44959f6eqf88b792054a41b16@mail.gmail.com> On Sat, Jul 11, 2009 at 11:21 PM, Kenneth Lundin wrote: > Most of the doc sources are already there. What's missing is > working make files to build the doc and the top level documentation. > But for all applications there is a /doc/src directory containing the > xml files and other files which is the doc source for that application. > > The plan is to improve this gradually i.e. with working make files for > open source users and also the tools for building to html and pdf. > > Currently we are using the OTP application docbuilder for generating > the html , man and pdf formats but we have far gone plans to change > both the tool chain and the dtd's for the sources. > When it comes to tools we are leaning strongly towards xsltproc for > the translation to (x)html and xsltproc + Apache FOP for the > translation to PDF. What's missing in docbuilder? The current output we get in HTML and man format are perfect for me (personally). I guess it's not about style or format and more about missing functionality. > The DTD's we are using are described in the docbuilder docs. > We have plans to reduce the number of DTD's as well. Using XMLSchema instead. > > /Kenneth Erlang/OTP Ericsson > > On Sat, Jul 11, 2009 at 2:04 AM, Dale Harvey wrote: >> Is there any updates on when the build sources for the documentation >> will be released? >> >> 2008/2/15 Mats Cronqvist >> >>> Matthias Lang wrote: >>> > I have a shell script which does this: >>> > >>> > ? ~ >eman io fwr >>> > ? showing >>> file:///usr/local/src/otp_doc_R11B-4/lib/stdlib-1.14.4/doc/html/io.html#fwrite/1 >>> > >>> > (it pops up the page in the currently running browser). As far as I >>> > know, I'm the only user. The web version doesn't get much traffic >>> > either. >>> ? i used a (modified) version of it. worked fine, but i liked >>> gotapi.com/erlang better. alas, since R12 that's >>> stopped working properly... >>> >>> ?mats From kenji.rikitake@REDACTED Sun Jul 12 04:56:46 2009 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Sun, 12 Jul 2009 11:56:46 +0900 Subject: Erlang R13B01 ssh-1.1.3 cipher key matching bug and documentation errors Message-ID: <20090712025646.GA44385@k2r.org> Here's a list of bugs/documentation errors of ssh-1.1.3 for R13B01 which I experienced yesterday. * [bug] ssh:shell/3 and ssh:connect/3 do not crash immediately even if they fail to negotiate the cipher to use, and hang forever How to reproduce: set NOT to accept 3des-cbc as a cipher on the server (in OpenSSH, set Ciphers directive at sshd_config, *excluding* 3des-cbc) Possible reason: failure of finding a matching cipher does not throw an exception immediately (I haven't tested yet). FYI: on Portable OpenSSH 5.1 for FreeBSD slogin client, it will turn out to be something like the following: -- quote -- debug1: match: OpenSSH_5.1p1 FreeBSD-20080901 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_5.1p1 FreeBSD-20080901 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received no matching cipher found: client 3des-cbc server aes128-ctr,blowfish-cbc,aes128-cbc,aes192-cbc,aes256-cbc -- unquote -- * [documentation error] ssh manual should include that the current ssh module only supports the following crypto parameters of SSH Version 2 protocol: (my opinion follows later in this message) cipher: 3des-cbc only MACs: hmac-sha1 only * [documentation error] ssh manual should include that only an *unencrypted* private key is supported for ssh_rsa public key authentication. The manual should also note that private keys for public key authentication used for interactive logins are mostly encrypted so cannot be used for the time being. * [documentation error] ssh:connect/1 and ssh:connect/2 no longer exist, but still documented. Description for those old functions should be eliminated, and requirement to use ssh:connect/3 instead should be described. * [my opinion] I personally think only supporting 3des-cbc is *archaic* and insufficient; implementing at least stronger ciphers such as aes128-cbc and aes256-cbc, or even blowfish-cbc, should be considered ASAP, regarding the strength of the ciphers. Regards, Kenji Rikitake From yoursurrogategod@REDACTED Sun Jul 12 05:14:03 2009 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Sat, 11 Jul 2009 20:14:03 -0700 (PDT) Subject: Attempting to create a plethora of processes Message-ID: <3a2bd8b8-fd6f-4fcd-a241-4cb98e815abb@i4g2000prm.googlegroups.com> Hi all, I'm just trying to get a better unerstanding of IPC in Erlang and I stumbled on this little problem. The problem arises when I get to the spawn method. My present understanding is that the spawn method will create a different process with the passed in method and then immediately return. I used the debugger in order to attempt to get to the bottom of this and found out that when the program gets to the simulate method (pardon the odd name, I borrowed the code from a different file that I was working on) it hangs. Understandably. It's waiting for a message (stop in this case) in order to proceed further. That's quite alright in my view, since I'm just testing something out and I could care less if this fails or not. However, what throws me for a loop is the fact that the process that called the spawn method seems to be unresponsive. This was not my intention and don't understand why this would happen. What have I misunderstood? spawn creates a new process, so in theory, it should just exit out of that method and proceed with the parent... Any help is appreciated. This is my source code: -module(superProcess). -export([start/1,stop/0]). -compile(export_all). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%% % CONTROLLING METHODS %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%% start(Population) -> run(Population). % the method that the user can call in order send the message to the main % process that it needs to stop running the application. stop() -> simAnnApp ! stop. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%% % WORKER METHODS %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%% % begin running the application. run(Population) -> registerApp(), ListOfProcs = spawnIndividuals(Population), loop(ListOfProcs). % register the overall application, this way we can send messages to the % general app. registerApp() -> register(simAnnApp, self()). % this is an interface method in order to simplify how sub-processes are % spawned. spawnIndividuals(Population) -> spawnIndividuals(Population, []). % go through a loop and create a separate process for each member of the % population. spawnIndividuals(0, ListOfProcs) -> ListOfProcs; spawnIndividuals(Population, ListOfProcs) -> Pid = spawn(simulate()), NewList = [Pid | ListOfProcs], spawnIndividuals(Population - 1, NewList). loop(ListOfProcs) -> receive stop -> killAll(ListOfProcs), exit(normal) after 0 -> true end. % will kill all sub-processes by sending it the die message. killAll([]) -> io:format("All sub-processes killed. Exiting application.~n~n"); killAll([Head | Tail]) -> die ! Head, killAll(Tail). simulate() -> receive die -> exit(normal) %after 0 -> % simulate() end. From dj3vande@REDACTED Sun Jul 12 06:40:43 2009 From: dj3vande@REDACTED (Dave Vandervies) Date: Sun, 12 Jul 2009 00:40:43 -0400 Subject: [erlang-questions] Attempting to create a plethora of processes In-Reply-To: <3a2bd8b8-fd6f-4fcd-a241-4cb98e815abb@i4g2000prm.googlegroups.com> References: <3a2bd8b8-fd6f-4fcd-a241-4cb98e815abb@i4g2000prm.googlegroups.com> Message-ID: <20090712044043.GA28588@goofy.davenet> Somebody claiming to be Yves S. Garret wrote: > Hi all, > > I'm just trying to get a better unerstanding of IPC in Erlang and I > stumbled on this little problem. The problem arises when I get to the > spawn method. My present understanding is that the spawn method will > create a different process with the passed in method and then > immediately return. [...] > However, > what throws me for a loop is the fact that the process that called the > spawn method seems to be unresponsive. This was not my intention and > don't understand why this would happen. What have I misunderstood? > spawn creates a new process, so in theory, it should just exit out of > that method and proceed with the parent... The problem isn't with spawn itself, it's with the argument you're giving it. You have: > Pid = spawn(simulate()), which calls simulate, and then calls spawn with whatever simulate returns (except you're not getting that far, since simulate blocks in a receive.) Try saying: Pid = spawn(fun simulate/0), instead; this will give spawn the function simulate/0 as an argument, and spawn should kick off a new process and return as you expect. dave -- Dave Vandervies dj3vande@REDACTED Plan your future! Make God laugh! From steven.charles.davis@REDACTED Sun Jul 12 06:46:04 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sat, 11 Jul 2009 21:46:04 -0700 (PDT) Subject: Any HTTP 1.0 clients out there? Message-ID: <990c88ad-c0a5-44ff-a87e-3b5c1c17ce77@s6g2000vbp.googlegroups.com> This is not explicitly erlang related - but the telecoms expertise of many on the list is, I think, likely to yield a reliable answer. Are there any *commercially relevant* HTTP/1.0 clients left out there? I've not seen any discussions on the web about 1.0 since the mid-2000s. That being the case -- is it reasonable to say that HTTP/ 1.0 is not just outdated but effectively dead? TIA, Steve From fw@REDACTED Sun Jul 12 11:28:35 2009 From: fw@REDACTED (Florian Weimer) Date: Sun, 12 Jul 2009 11:28:35 +0200 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: (Joel Reymont's message of "Sat, 11 Jul 2009 16:35:30 +0100") References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> <20090710194505.GH4171@wellquite.org> <31B48CF7-1069-4699-ABCC-0FC83415C27B@gmail.com> <4A58B0C4.9090705@erlang-consulting.com> Message-ID: <87ws6ei724.fsf@mid.deneb.enyo.de> * Joel Reymont: > Yes, these are EC2 figures. Can you elaborate on how > IO is a problem on virtual processors? You're sending 20K messages, which requires 40K interrupts and hypervisor/kernel transitions when naively implemented (one for the message, one for the ACK from the client), and 20K userland/kernel transitions. This is not necessarily an area where virtualization solutions shine. You need to ask the EC2 folks if this is a problem for them (interrupt mitigation can increase throughput, so the 40K number is not hard bound), and what can be done about it. Perhaps they offer some sort of TCP termination service which takes care of these details. From carlmcdade@REDACTED Sun Jul 12 13:52:28 2009 From: carlmcdade@REDACTED (Carl McDade) Date: Sun, 12 Jul 2009 13:52:28 +0200 Subject: Which ODBC and WX is needed? Message-ID: Hi, When compiling erlang on Mint Linux there is a check for Java , ODBC and WX. I have installed Java. But I can't find the right the ODBC and WX libraries. Which ones are needed? packages to install: sudo apt-get install build-essential libncurses5-dev m4 sudo apt-get install openssl libssl-dev sudo apt-get install libsctp-dev freeglut3-dev libwxgtk2.8-dev g++ Download and extract Erlang R13B: wget http://www.erlang.org/download/otp_src_R13B.tar.gz tar -xzvf otp_src_R13B.tar.gz cd otp_src_R13B Then configure, compile and install. ./configure \ --enable-threads --enable-smp-support --enable-kernel-poll \ --enable-hipe --enable-sctp --prefix=/opt/erlang make sudo make install for file in erl erlc epmd run_erl to_erl dialyzer typer escript; do sudo ln -s /opt/erlang/lib/erlang/bin/$file /usr/bin/$file; done -- Carl McDade Web developer www.hiveminds.co.uk ________________________ From bflatmaj7th@REDACTED Sun Jul 12 14:04:17 2009 From: bflatmaj7th@REDACTED (Richard Andrews) Date: Sun, 12 Jul 2009 22:04:17 +1000 Subject: [erlang-questions] Which ODBC and WX is needed? In-Reply-To: References: Message-ID: <7702c0610907120504u659850daw1b7c7dd59e82adee@mail.gmail.com> On Sun, Jul 12, 2009 at 9:52 PM, Carl McDade wrote: > Hi, > > When compiling erlang on Mint Linux there is a check for Java , ODBC > and WX. I have installed Java. ?But I can't find the right the ODBC > and WX libraries. Which ones are needed? > > > packages to install: > > sudo apt-get install build-essential libncurses5-dev m4 > sudo apt-get install openssl libssl-dev > sudo apt-get install libsctp-dev freeglut3-dev libwxgtk2.8-dev g++ As far as ODBC is concerned I think you're going to want the unixodbc-dev package (required unixodbc) and an ODBC adapter for your chosen DB. Don't know about wx. From 0x6e6562@REDACTED Sun Jul 12 14:12:09 2009 From: 0x6e6562@REDACTED (Ben Hood) Date: Sun, 12 Jul 2009 13:12:09 +0100 Subject: Loading code from an archive Message-ID: <269388e30907120512y38566659o1d155c1d15b69274@mail.gmail.com> Hi, I'm using the archive code loading mechanism introduced in R12B5 and I'm getting some strange error messages - despite the fact that the code seems to have loaded correctly. I've created an archive called mod_http.ez with the following structure: Listing archive: deps/mod_http.ez Date Time Attr Size Compressed Name ------------------- ----- ------------ ------------ ------------------------ 2009-07-12 13:56:28 D.... 0 0 mod_http 2009-07-12 13:56:28 D.... 0 0 mod_http/ebin 2009-07-12 13:56:28 ..... 284 143 mod_http/ebin/mod_http.app 2009-07-12 13:56:28 ..... 1396 1111 mod_http/ebin/mod_http.beam 2009-07-12 13:56:28 ..... 1148 862 mod_http/ebin/mod_http_app.beam 2009-07-12 13:56:28 ..... 3292 2636 mod_http/ebin/mod_http_deps.beam 2009-07-12 13:56:28 ..... 3000 2413 mod_http/ebin/mod_http_sup.beam 2009-07-12 13:56:28 ..... 2096 1749 mod_http/ebin/mod_http_web.beam ------------------- ----- ------------ ------------ ------------------------ 11216 8914 6 files, 2 folders When I load this into the interpreter using the ERL_LIBS variable, everything seems to work, apart from a spurious error message in the shell: ** Bad path can't read plugins/mod_http.ez/mod_http/ebin ** Found 0 name clashes in code paths Is this something I need to be concerned about? This behavior appears to be consistent in R12B5 and R13B01. Also, ran into a gotcha with this, which might be relevant: When I created the zip files, I used the following arguments: zip -r mod_http.ez mod_http/ebin/* which has the effect of omitting the top level directory in the zip file. This contradicts the the specification describe in the man page for the code module and hence fails to load. Cheers, Ben From 0x6e6562@REDACTED Sun Jul 12 14:59:41 2009 From: 0x6e6562@REDACTED (Ben Hood) Date: Sun, 12 Jul 2009 13:59:41 +0100 Subject: [erlang-questions] Loading code from an archive In-Reply-To: <269388e30907120512y38566659o1d155c1d15b69274@mail.gmail.com> References: <269388e30907120512y38566659o1d155c1d15b69274@mail.gmail.com> Message-ID: <269388e30907120559u1c72c8ealcbe882acbe4732d2@mail.gmail.com> Hi, I'm using the archive code loading mechanism introduced in R12B5 and I'm getting some strange error messages - despite the fact that the code seems to have loaded correctly. I've created an archive called mod_http.ez with the following structure: Listing archive: deps/mod_http.ez Date Time Attr Size Compressed Name ------------------- ----- ------------ ------------ ------------------------ 2009-07-12 13:56:28 D.... 0 0 mod_http 2009-07-12 13:56:28 D.... 0 0 mod_http/ebin 2009-07-12 13:56:28 ..... 284 143 mod_http/ebin/mod_http.app 2009-07-12 13:56:28 ..... 1396 1111 mod_http/ebin/mod_http.beam 2009-07-12 13:56:28 ..... 1148 862 mod_http/ebin/mod_http_app.beam 2009-07-12 13:56:28 ..... 3292 2636 mod_http/ebin/mod_http_deps.beam 2009-07-12 13:56:28 ..... 3000 2413 mod_http/ebin/mod_http_sup.beam 2009-07-12 13:56:28 ..... 2096 1749 mod_http/ebin/mod_http_web.beam ------------------- ----- ------------ ------------ ------------------------ 11216 8914 6 files, 2 folders When I load this into the interpreter using the ERL_LIBS variable, everything seems to work, apart from a spurious error message in the shell: ** Bad path can't read plugins/mod_http.ez/mod_http/ebin ** Found 0 name clashes in code paths Is this something I need to be concerned about? This behavior appears to be consistent in R12B5 and R13B01. Also, ran into a gotcha with this, which might be relevant: When I created the zip files, I used the following arguments: zip -r mod_http.ez mod_http/ebin/* which has the effect of omitting the top level directory in the zip file. This contradicts the the specification describe in the man page for the code module and hence fails to load. Cheers, Ben From harveyd@REDACTED Sun Jul 12 16:33:28 2009 From: harveyd@REDACTED (Dale Harvey) Date: Sun, 12 Jul 2009 15:33:28 +0100 Subject: Problem with distel debugging / breakpoint Message-ID: having avoided proper debugging with io statements for to long, giving the distel debugger a shot. following the guide at http://bc.tech.coop/blog/070528.html which is realy good, however hitting a wall. I C-c C-d i to set intepreted, works fine, I press C-x SPC to mark a breakpoint, fine, I run a function that hits the break point, it stops, I get am edb window, however when I press RET on the process, I get the message No buffer for pid [TYPE erl-pid distel_768@REDACTED 8 0 0] The function never returns and I cant seem to select a process to debug, made screenshots to hopefully make it clear. http://arandomurl.com/stuff/after.png set a breakpoint, its been hit, window pops up, press return, then http://arandomurl.com/stuff/before.png Entirely possible I am doing something silly like pressing the wrong key but debugging it isnt easy. Cheers Dale From rapsey@REDACTED Sun Jul 12 20:38:48 2009 From: rapsey@REDACTED (Rapsey) Date: Sun, 12 Jul 2009 20:38:48 +0200 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> Message-ID: <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> Well I have a similar problem with my streaming server. I suspect the main issue with using gen_tcp is that every gen_tcp:send call will involve a memory allocation and memcpy. My server needs to be able to fill up at least a gigabyte connection and this involves a very large number of gen_tcp:send calls. The only way I could achieve that number is by writing my own driver for data output. This means bypassing gen_tcp completely and writing the socket handling stuff by hand. Basically whenever I need to send the data, I loop through the array of sockets (and other information) and send from one single buffer (the sockets are non-blocking). It works great and is fast as hell. I suspect it could fill up at least 2-4gigabits. Sergej On Fri, Jul 10, 2009 at 8:24 PM, Joel Reymont wrote: > I can't seem to send 20k messages in much less than 4 seconds and would > appreciate any suggestions on how to improve my broadcasting speed. > > The messages are binary, 69 bytes, including the trailing 0 (Flash). > There's no size prefix. Flow control is {active, once}, e.g. > > Opts = [binary, > {packet, 0}, > {reuseaddr, true}, > {backlog, 1024}, > {active, false}], > > The publishing routing looks like this, where Send is a closure with > gen_tcp:send/2 and a captured socket. > > F = fun(_, {_, Send}, _) -> Send(Msg) end, > F1 = fun() -> dict:fold(F, ignore, State#state.subscribers) end, > spawn_link(F1), > > I'm running this on small EC2 instances and the actual fold takes < 200ms. > I'm hesitant to attribute the difference to the TCP stack. > > I also tried this variant but only shaved 100-200ms from the 4s. > > F = fun(_, {_, Send}, _) -> spawn(fun() -> Send(Msg) end) end, > dict:fold(F, ignore, State#state.subscribers), > > I tried distributing my test harness over 4 EC2 instances but this saved me > a second or so, for the 4 second total I'm trying to beat now. > > What else can I do? > > Thanks, Joel > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From joelr1@REDACTED Sun Jul 12 21:49:27 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sun, 12 Jul 2009 20:49:27 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> Message-ID: <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> On Jul 12, 2009, at 7:38 PM, Rapsey wrote: > Well I have a similar problem with my streaming server. I suspect > the main > issue with using gen_tcp is that every gen_tcp:send call will > involve a > memory allocation and memcpy. Why do you think so? I thought binaries over 64 bytes are not copied. > My server needs to be able to fill up at least a gigabyte connection > and > this involves a very large number of gen_tcp:send calls. The only > way I > could achieve that number is by writing my own driver for data output. Did you write your own driver already? What kind of throughput were you getting with your old Erlang code? > This means bypassing gen_tcp completely and writing the socket > handling > stuff by hand. Basically whenever I need to send the data, I loop > through > the array of sockets (and other information) and send from one > single buffer > (the sockets are non-blocking). I'll take a closer look at the TCP driver but other Erlang internals I looked at use iovecs and scatter-gather IO (writev, etc.). --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From wolfmanjm@REDACTED Mon Jul 13 00:25:31 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Sun, 12 Jul 2009 15:25:31 -0700 (PDT) Subject: 20k messages in 4s but want to go faster! In-Reply-To: <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> Message-ID: I'd be careful about going on a wild goose chase (writing drivers etc). You need to get solid data about how fast you are actually sending the messages right now. My gut feeling is you are measuring too much and not just the message transmission. ie out of the 4 seconds I'll bet that 80% of that is the time taken to tear down the processes and send the DOWN message. and the rest is the actual transmission latency (which is what you are really interested in). I reran my TCP tests, all running on the same dual processor Intel/ Linux machine, with the tester running in one VM and the server running on another VM. I can broadcast (actually technically unicast) to 8,000 clients with max latency 277ms, and an average of 192ms. This scales almost linearly (I tested 4,000 as well which was about half the time) so by extrapolation 20,000 messages should take about 700ms worst case and average under 500ms. If the clients were distributed and the server distributed I bet it could be faster. I am measuring the pure latency of the messages and no other overheads. Before writing all sorts of optimizations you really need to be confident of your test metrics. I'd test on a local machine first, get a reading, then try on your EC2 instances and see if there are any significant differences. Just my 2cents worth ;) On Jul 12, 12:49?pm, Joel Reymont wrote: > On Jul 12, 2009, at 7:38 PM, Rapsey wrote: > > > Well I have a similar problem with my streaming server. I suspect ? > > the main > > issue with using gen_tcp is that every gen_tcp:send call will ? > > involve a > > memory allocation and memcpy. > > Why do you think so? I thought binaries over 64 bytes are not copied. > > > My server needs to be able to fill up at least a gigabyte connection ? > > and > > this involves a very large number of gen_tcp:send calls. The only ? > > way I > > could achieve that number is by writing my own driver for data output. > > Did you write your own driver already? > > What kind of throughput were you getting with your old Erlang code? > > > This means bypassing gen_tcp completely and writing the socket ? > > handling > > stuff by hand. Basically whenever I need to send the data, I loop ? > > through > > the array of sockets (and other information) and send from one ? > > single buffer > > (the sockets are non-blocking). > > I'll take a closer look at the TCP driver but other Erlang internals I ? > looked at use iovecs and scatter-gather IO (writev, etc.). > > --- > Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From joelr1@REDACTED Mon Jul 13 00:35:57 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sun, 12 Jul 2009 23:35:57 +0100 Subject: [erlang-questions] Re: 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> Message-ID: <76E8BEDE-93F4-4F6C-89C6-586611EAE5D5@gmail.com> On Jul 12, 2009, at 11:25 PM, Jim Morris wrote: > I can broadcast (actually technically unicast) > to 8,000 clients with max latency 277ms, and an average of 192ms. I can only do 1.7s for 10k clients on my 2.93Ghz Core2Duo MacBook Pro with 4Gb memory and SSD. I can't do 20k, I think I need to implement a reconnect with progressive delay and perhaps tune kernel settings. I will redo my measurements tomorrow and see what changes. I'll definitely try to move away from erlang:monitor and 'DOWN' messages. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From tony@REDACTED Mon Jul 13 01:37:49 2009 From: tony@REDACTED (Tony Arcieri) Date: Sun, 12 Jul 2009 17:37:49 -0600 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> Message-ID: On Sat, Jul 11, 2009 at 3:34 AM, Joel Reymont wrote: > > On Jul 10, 2009, at 11:56 PM, Tony Arcieri wrote: > > Have you tried persisting the processes? >> > > > What do you mean? If this sending of 20k messages is a repeat affair, what if you had a process per connection you could broadcast messages to that remained alive for the duration of their TCP connections? There would be overhead for process creation/teardown, but it would not factor into the actual message transmission if the processes were persistent. Just a guess... as Jim Morris said a better profile of where your program is spending its time would be helpful in diagnosing this problem. -- Tony Arcieri medioh.com From icfp.publicity@REDACTED Mon Jul 13 04:45:51 2009 From: icfp.publicity@REDACTED (Matthew Fluet (ICFP Publicity Chair)) Date: Sun, 12 Jul 2009 21:45:51 -0500 Subject: ICFP09 Final Call for Participation Message-ID: <53ff55480907121945y7a8a6cc1vb9b85e081e224874@mail.gmail.com> ===================================================================== Final Call for Participation The 14th ACM SIGPLAN International Conference on Functional Programming (ICFP 2009) http://www.cs.nott.ac.uk/~gmh/icfp09.html Edinburgh, Scotland, 31 August - 2 September 2009 ===================================================================== ***** Accommodation Deadline: July 20, 2009 ***** Due to the overlap with Edinburgh Festival, accommodations may be difficult to secure after the deadline. Reserve now! http://www.haskell.org/haskellwiki/ICFP_2009_Local_Arrangements ***** Early Registration Deadline: July 30, 2009 ***** http://www.regmaster.com/conf/icfp2009.html ICFP 2009 provides a forum for researchers and developers to hear about the latest work on the design, implementations, principles, and uses of functional programming. The conference covers the entire spectrum of work, from practice to theory, including its peripheries. Preliminary program: * Abstracts: + http://web.cecs.pdx.edu/~apt/icfp09_accepted_papers/accepted.html * Schedule: + http://web.cecs.pdx.edu/~apt/icfp09_preliminary_program.pdf * Invited speakers: + Guy Steele -- Organizing Functional Code for Parallel Execution: or, foldl and foldr Considered Slightly Harmful + Benjamin Pierce -- Lambda, the Ultimate TA: Using a Proof Assistant to Teach Programming Language Foundations + Dan Piponi -- Commutative Monads, Diagrams and Knots Schedule including related workshops: * Aug 30: ACM SIGPLAN Workshop on ML * Aug 30: ACM SIGPLAN Workshop on Generic Programming * Aug 31-Sep 2: ICFP09 * Sep 3: ACM SIGPLAN Haskell Symposium * Sep 3: ACM SIGPLAN Developer Tracks on Functional Programming * Sep 4: Commercial Users of Functional Programming * Sep 4: ACM SIGPLAN Workshop on Mechanizing Metatheory * Sep 4: ACM SIGPLAN Workshop on Approaches and Applications of Inductive Programming * Sep 5: ACM SIGPLAN Erlang Workshop * Sep 5: ACM SIGPLAN Developer Tracks on Functional Programming * Sep 5: ACM SIGPLAN Haskell Implementors Workshop Conference organizers: * General Chair: Graham Hutton (University of Nottingham) * Program Chair: Andrew Tolmach (Portland State University) * Local Arrangements Chairs: Philip Wadler (University of Edinburgh), Kevin Hammond (University of St Andrews), and Gregory Michaelson (Heriot-Watt University) * Workshop Co-Chairs: Christopher Stone (Harvey Mudd College), and Michael Sperber (DeinProgramm) * Programming Contest Chair: Andrew Gill (University of Kansas) * Publicity Chair: Matthew Fluet (Toyota Technological Institute at Chicago) From nesrait@REDACTED Mon Jul 13 06:24:10 2009 From: nesrait@REDACTED (=?ISO-8859-1?Q?Davide_Marqu=EAs?=) Date: Mon, 13 Jul 2009 05:24:10 +0100 Subject: [erlang-questions] Re: Erlang at Facebook - hibernation and gen_server In-Reply-To: <2a78ca8c-6d6f-441a-95a8-e4b717c0fe46@x5g2000prf.googlegroups.com> References: <523869a70906292134j5d51d679nd72f0060a0317b2f@mail.gmail.com> <1246364964.5363.5.camel@sredniczarny.smp.if.uj.edu.pl> <523869a70906301041m1017055alafe279f156f1824a@mail.gmail.com> <2a78ca8c-6d6f-441a-95a8-e4b717c0fe46@x5g2000prf.googlegroups.com> Message-ID: <523869a70907122124y7eecc49cq89cb51467b7c6915@mail.gmail.com> On Wed, Jul 8, 2009 at 12:16 PM, eletuchy@REDACTED wrote: > The incompatibility is on the client side (gen:call), not the server > side (gen_server:handle_call). There's a receive() loop inside > gen:call that prevents a client from hibernating between dispatching > the $call message and getting back the gen_server:reply(). > Thanks for the feedback! Got it! :) It seems it's just a matter of (famous last words): - using erlang:send_after/3 to get the process to wake up from hibernation after the given Timeout; - hibernating the client process (with proc_lib:hibernate/3) while waiting for the reply - turn the control (after waking up) to a function that's ready to handle the reply and the erlang:send_after/3's timeout we had set up before hibernating. Trying this out now... results later on. :) Cheers, :Davide From wolfmanjm@REDACTED Mon Jul 13 07:57:57 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Sun, 12 Jul 2009 22:57:57 -0700 (PDT) Subject: 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> Message-ID: <4cf022bc-a248-44eb-aa70-d006c2b66a8b@g1g2000pra.googlegroups.com> Actually Tony has a good point, I presumed Joel was doing it that way. In my server when I unicast what I actually do is send an erlang message to each process, and that of course sends the TCP message to its client. So when I have 8,000 clients connected I have 8,000 processes, and each one sends a TCP message to its client. Not sure if having a list of 8,000 TCP sockets and iterating over them sending to each socket from a single process would be any slower, I never tried that approach. It seems more obvious to me to have a process per connection, and that process is responsible for any communication with its client, that way everything is serialized to a specific client, and the process keeps its clients state. So Joel do you have one process per client or iterate over a list of sockets? On Jul 12, 4:37?pm, Tony Arcieri wrote: > On Sat, Jul 11, 2009 at 3:34 AM, Joel Reymont wrote: > > > On Jul 10, 2009, at 11:56 PM, Tony Arcieri wrote: > > > ?Have you tried persisting the processes? > > > What do you mean? > > If this sending of 20k messages is a repeat affair, what if you had a > process per connection you could broadcast messages to that remained alive > for the duration of their TCP connections? > > There would be overhead for process creation/teardown, but it would not > factor into the actual message transmission if the processes were > persistent. > > Just a guess... as Jim Morris said a better profile of where your program is > spending its time would be helpful in diagnosing this problem. > > -- > Tony Arcieri > medioh.com From wde@REDACTED Mon Jul 13 08:17:28 2009 From: wde@REDACTED (wde) Date: Mon, 13 Jul 2009 08:17:28 +0200 Subject: [erlang-questions] Mnesia and file system snapshot Message-ID: <20090713061733.D11891C000A5@mwinf2722.orange.fr> Hello, I would like to "backup" the mnesia directory by using a file system snapshot (like zfs snapshot for example). Before to take the snapshot I must be sure that mnesia files are in consistent state. Does the mnesia checkpoint maintain files (LOG, DAT,DCD,DCL ) in a consistent state (doc refers to tables state but not files) ? Otherwise I can always snaphot a backup file . Any feedback mnesia+file system snapshot would be appreciated. Thank you, wde From rapsey@REDACTED Mon Jul 13 08:22:59 2009 From: rapsey@REDACTED (Rapsey) Date: Mon, 13 Jul 2009 08:22:59 +0200 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> Message-ID: <97619b170907122322l6d7220abv29852ef0d24568fb@mail.gmail.com> Looking at the inet_drv.c, it should not be copying memory. I was using send_timeout though, perhaps the timer for every socket was causing it. Since it's a streaming server, I was a bit nervous about using send_timeout infinity, there is a ton of data constantly moving through the server. My CPU usage was very linear with gen_tcp, I never got to try sending out data at 1gbit, but it looked like it would max out the quadcore xeon it was running on at that speed. Using my driver I hardly notice any CPU difference between 10mbit or 200mbit throughput. But you definitely should be using process per socket model. Sergej On Sun, Jul 12, 2009 at 9:49 PM, Joel Reymont wrote: > > On Jul 12, 2009, at 7:38 PM, Rapsey wrote: > > Well I have a similar problem with my streaming server. I suspect the main >> issue with using gen_tcp is that every gen_tcp:send call will involve a >> memory allocation and memcpy. >> > > Why do you think so? I thought binaries over 64 bytes are not copied. > > My server needs to be able to fill up at least a gigabyte connection and >> this involves a very large number of gen_tcp:send calls. The only way I >> could achieve that number is by writing my own driver for data output. >> > > Did you write your own driver already? > > What kind of throughput were you getting with your old Erlang code? > > This means bypassing gen_tcp completely and writing the socket handling >> stuff by hand. Basically whenever I need to send the data, I loop through >> the array of sockets (and other information) and send from one single >> buffer >> (the sockets are non-blocking). >> > > > I'll take a closer look at the TCP driver but other Erlang internals I > looked at use iovecs and scatter-gather IO (writev, etc.). > > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > From joelr1@REDACTED Mon Jul 13 09:29:42 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 13 Jul 2009 08:29:42 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> Message-ID: <4E8D2393-FC23-4060-A007-B664D4DF26C2@gmail.com> On Jul 13, 2009, at 12:37 AM, Tony Arcieri wrote: > If this sending of 20k messages is a repeat affair, what if you had a > process per connection you could broadcast messages to that remained > alive > for the duration of their TCP connections? Under what circumstances is taking a trip through 1-2 extra gen_servers is faster than pushing data directly to the socket? I do have a transport (protocol handler) process and I do have a session process that persists even while the socket is gone. I used to send to the session which would forward to the transport and I still do this for HTTP since the socket does not persist. I decided to optimize away the extra trip through session and transport for persistent Flash connections, though, by capturing gen_tcp:send with the socket in a closure and using that to push to the socket directly. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Mon Jul 13 09:34:15 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 13 Jul 2009 08:34:15 +0100 Subject: [erlang-questions] Re: 20k messages in 4s but want to go faster! In-Reply-To: <4cf022bc-a248-44eb-aa70-d006c2b66a8b@g1g2000pra.googlegroups.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <4cf022bc-a248-44eb-aa70-d006c2b66a8b@g1g2000pra.googlegroups.com> Message-ID: <920BA033-D3D2-48D7-A320-C99DC2EE3CBC@gmail.com> On Jul 13, 2009, at 6:57 AM, Jim Morris wrote: > Actually Tony has a good point, I presumed Joel was doing it that way. And I was. > Not sure if having a list of 8,000 TCP sockets and iterating over them > sending to each socket from a single process would be any slower, I > never tried that approach. It must positively be slower. I once measured the overhead of a message send to be 10x over a function call, maybe not precisely that but still _many_ times slower. A data point for all you out there trying to implement objects as processes. > So Joel do you have one process per client or iterate over a list of > sockets? gen_server is even slower than a regular message send so I can't imagine a circumstance where taking a trip through a socket-fronting process can be faster than sending to the socket directly. On to fix my measurements! --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Mon Jul 13 09:35:21 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 13 Jul 2009 08:35:21 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <97619b170907122322l6d7220abv29852ef0d24568fb@mail.gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> <97619b170907122322l6d7220abv29852ef0d24568fb@mail.gmail.com> Message-ID: On Jul 13, 2009, at 7:22 AM, Rapsey wrote: > But you definitely should be using process per socket model. Why? --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From christian.czura@REDACTED Mon Jul 13 09:36:22 2009 From: christian.czura@REDACTED (Christian Czura) Date: Mon, 13 Jul 2009 09:36:22 +0200 Subject: bzip2 port anyone? Message-ID: <4A5AE3F6.8070201@gmail.com> Hi list, I've been looking for a module/port to decompress bzip2-compressed data. My search has been unsuccessful so far. As bzip2 compression/decompression isn't THAT uncommon, I figure it's probable that either somebody has already written such module or I've missed the obvious and there is already something in the stdlib (yes, I've looked). Thanks, Chris From wolfmanjm@REDACTED Mon Jul 13 09:58:15 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Mon, 13 Jul 2009 00:58:15 -0700 (PDT) Subject: 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> <97619b170907122322l6d7220abv29852ef0d24568fb@mail.gmail.com> Message-ID: <74ddf71d-3e08-46d1-8ea5-b88bf21f62ef@u16g2000pru.googlegroups.com> I think it is actually faster. My server uses a gen_fsm for each socket/client connection. When I get a message that needs broadcasting/ unicasting to all the other clients, I get a list of all the PIDS of the gen_fsm's, and send an erlang message to each of the pids, when the 8,000 or so gen_fsm processes get that message they do a tcp send to their client. That is getting the fastest throughput for me. Also if you do a tcp send from different processes to the same socket you will get contention, and no guaranteed order, just an FYI. So in my system whenever one client needs to send a TCP message to another client, it is always done through the clients FSM process. The whole reason for me using Erlang was that I could do one process per socket, as processes were so cheap. On Jul 13, 12:35?am, Joel Reymont wrote: > On Jul 13, 2009, at 7:22 AM, Rapsey wrote: > > > But you definitely should be using process per socket model. > > Why? > > --- > Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From rapsey@REDACTED Mon Jul 13 10:03:05 2009 From: rapsey@REDACTED (Rapsey) Date: Mon, 13 Jul 2009 10:03:05 +0200 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> <97619b170907122322l6d7220abv29852ef0d24568fb@mail.gmail.com> Message-ID: <97619b170907130103k55a22d94ia7865a080d09c763@mail.gmail.com> I don't think the erlang:port_command (which is what gen_tcp:send basically is) overhead is that small. Doing it from a single process might be the bottleneck. Sergej On Mon, Jul 13, 2009 at 9:35 AM, Joel Reymont wrote: > > On Jul 13, 2009, at 7:22 AM, Rapsey wrote: > > But you definitely should be using process per socket model. >> > > Why? > > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > From masse@REDACTED Mon Jul 13 10:05:56 2009 From: masse@REDACTED (mats cronqvist) Date: Mon, 13 Jul 2009 10:05:56 +0200 Subject: Problem with distel debugging / breakpoint In-Reply-To: (Dale Harvey's message of "Sun\, 12 Jul 2009 15\:33\:28 +0100") References: Message-ID: <87zlb9j9cr.fsf@sterlett.hq.kred> Dale Harvey writes: > having avoided proper debugging with io statements for to long, > giving the distel debugger a shot. > > following the guide at http://bc.tech.coop/blog/070528.html which > is realy good, however hitting a wall. > > I C-c C-d i to set intepreted, works fine, I press C-x SPC to mark a > breakpoint, fine, I run a function that hits the break point, it stops, > I get am edb window, however when I press RET on the process, I > get the message No buffer for pid [TYPE erl-pid distel_768@REDACTED 8 0 0] are you using flymake? I've had problems using edb and flymake together... From joelr1@REDACTED Mon Jul 13 10:59:18 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 13 Jul 2009 09:59:18 +0100 Subject: [erlang-questions] Re: 20k messages in 4s but want to go faster! In-Reply-To: <74ddf71d-3e08-46d1-8ea5-b88bf21f62ef@u16g2000pru.googlegroups.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> <97619b170907122322l6d7220abv29852ef0d24568fb@mail.gmail.com> <74ddf71d-3e08-46d1-8ea5-b88bf21f62ef@u16g2000pru.googlegroups.com> Message-ID: <6B4835F7-0801-4BE5-8827-6644A0210B89@gmail.com> On Jul 13, 2009, at 8:58 AM, Jim Morris wrote: > I think it is actually faster. Have you measured it? > My server uses a gen_fsm for each > socket/client connection. When I get a message that needs > broadcasting/ > unicasting to all the other clients, I get a list of all the PIDS of > the gen_fsm's, and send an erlang message to each of the pids, That's what I (used to) do as well. > when the 8,000 or so gen_fsm processes get that message they do a > tcp send > to their client. That is getting the fastest throughput for me. How do you know it's getting the fastest throughput for you? I won't believe doing an extra round trip through a gen_* server is faster than pushing a static chunk of binary data to a list of sockets. Please prove me wrong! > Also if you do a tcp send from different processes to the same socket > you will get contention, and no guaranteed order, just an FYI. No such use case. > The whole reason for me using Erlang was that I could do one process > per socket, as processes were so cheap. I agree with you on processes being cheap and I use processes liberally. I'm trying to optimize a specific use case, though, which is pushing a chunk of binary data to 20k clients. Also, I spent a LOT of time optimizing OpenPoker last fall. I used to have an architecture where everything was a process. I had a gen_server for the common game functionality, a gen_fsm for game logic, gen_servers for handling pots, seats, etc. Drastically cutting down on the number of processes significantly improved throughput. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Mon Jul 13 11:01:41 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 13 Jul 2009 10:01:41 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <97619b170907130103k55a22d94ia7865a080d09c763@mail.gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> <97619b170907122322l6d7220abv29852ef0d24568fb@mail.gmail.com> <97619b170907130103k55a22d94ia7865a080d09c763@mail.gmail.com> Message-ID: <21490E7B-2D50-4238-ADBA-54AA1B7E3E33@gmail.com> On Jul 13, 2009, at 9:03 AM, Rapsey wrote: > Doing it from a single process might be the bottleneck. How do you tell? My biggest beef with Erlang at the moment is this total lack of transparency. I was able to run fprof on my code could could not make sense of the output since it was almost totally OTP proc_lib, etc. code. You have the option of putting in your own process tracing specifications there's no option to exclude processes in fprof, although this is possible with basic Erlang tracing. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From oscar@REDACTED Mon Jul 13 11:14:55 2009 From: oscar@REDACTED (=?ISO-8859-1?Q?Oscar_Hellstr=F6m?=) Date: Mon, 13 Jul 2009 10:14:55 +0100 Subject: [erlang-questions] Any HTTP 1.0 clients out there? In-Reply-To: <990c88ad-c0a5-44ff-a87e-3b5c1c17ce77@s6g2000vbp.googlegroups.com> References: <990c88ad-c0a5-44ff-a87e-3b5c1c17ce77@s6g2000vbp.googlegroups.com> Message-ID: <4A5AFB0F.1020806@erlang-consulting.com> Hi Steve, May I just ask why you're asking? Are you interested in using a HTTP/1.0 client, or are you asking if you should put any effort in to making a server 1.0 compliant? My only experience of HTTP/1.0 was in a commercial project not too long a ago (~2 years). The protocol an external service used was XML over HTTP, and their servers would only speak HTTP/1.0. This mainly affected connection keep-alive, since we didn't want to do any chunked upload or such. I don't know if this is still the case though. Steve Davis wrote: > This is not explicitly erlang related - but the telecoms expertise of > many on the list is, I think, likely to yield a reliable answer. > > Are there any *commercially relevant* HTTP/1.0 clients left out there? > I've not seen any discussions on the web about 1.0 since the > mid-2000s. That being the case -- is it reasonable to say that HTTP/ > 1.0 is not just outdated but effectively dead? > > TIA, > > Steve > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > Best regards -- Oscar Hellstr?m, oscar@REDACTED Office: +44 20 7655 0337 Mobile: +44 798 45 44 773 Erlang Training and Consulting Ltd http://www.erlang-consulting.com/ From steven.charles.davis@REDACTED Mon Jul 13 13:09:25 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 13 Jul 2009 04:09:25 -0700 (PDT) Subject: Any HTTP 1.0 clients out there? In-Reply-To: <4A5AFB0F.1020806@erlang-consulting.com> References: <990c88ad-c0a5-44ff-a87e-3b5c1c17ce77@s6g2000vbp.googlegroups.com> <4A5AFB0F.1020806@erlang-consulting.com> Message-ID: <923814a6-9110-44ec-ac23-feaf33635e02@o13g2000vbl.googlegroups.com> Hi Oscar, My interest is in whether it is at all relevant to support HTTP/1.0 on a server any more - currently I've stripped out that code and just return status 505 (HTTP version not supported). Regards, Steve From oscar@REDACTED Mon Jul 13 14:12:27 2009 From: oscar@REDACTED (=?ISO-8859-1?Q?Oscar_Hellstr=F6m?=) Date: Mon, 13 Jul 2009 13:12:27 +0100 Subject: [erlang-questions] Re: Any HTTP 1.0 clients out there? In-Reply-To: <923814a6-9110-44ec-ac23-feaf33635e02@o13g2000vbl.googlegroups.com> References: <990c88ad-c0a5-44ff-a87e-3b5c1c17ce77@s6g2000vbp.googlegroups.com> <4A5AFB0F.1020806@erlang-consulting.com> <923814a6-9110-44ec-ac23-feaf33635e02@o13g2000vbl.googlegroups.com> Message-ID: <4A5B24AB.7040508@erlang-consulting.com> Hi Steve, I guess that depends on what kind of clients you will have. If you have a controlled set of clients it would be very easy to make sure that there are no HTTP/1.0 clients, but in a scenario where clients is the general public I think you might still get some HTTP/1.0 clients. For instance, lynx on my machine (,2.8.6rel.5 libwww-FM/2.14...) is claiming to be an HTTP/1.0 client while links (ELinks/0.11.1-1.2etch...) on another machine of mine is using HTTP/1.1. I do use lynx and links every now and then, but I don't know if I would be a relevant user of your application. Steve Davis wrote: > Hi Oscar, > > My interest is in whether it is at all relevant to support HTTP/1.0 on > a server any more - currently I've stripped out that code and just > return status 505 (HTTP version not supported). > > Regards, > Steve > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > Best regards -- Oscar Hellstr?m, oscar@REDACTED Office: +44 20 7655 0337 Mobile: +44 798 45 44 773 Erlang Training and Consulting Ltd http://www.erlang-consulting.com/ From harveyd@REDACTED Mon Jul 13 15:15:58 2009 From: harveyd@REDACTED (Dale Harvey) Date: Mon, 13 Jul 2009 14:15:58 +0100 Subject: Problem with distel debugging / breakpoint In-Reply-To: <87zlb9j9cr.fsf@sterlett.hq.kred> References: <87zlb9j9cr.fsf@sterlett.hq.kred> Message-ID: 2009/7/13 mats cronqvist > Dale Harvey writes: > > > having avoided proper debugging with io statements for to long, > > giving the distel debugger a shot. > > > > following the guide at http://bc.tech.coop/blog/070528.html which > > is realy good, however hitting a wall. > > > > I C-c C-d i to set intepreted, works fine, I press C-x SPC to mark a > > breakpoint, fine, I run a function that hits the break point, it stops, > > I get am edb window, however when I press RET on the process, I > > get the message No buffer for pid [TYPE erl-pid distel_768@REDACTED 8 0 0] > > are you using flymake? I've had problems using edb and flymake > together... > I am, and turning flymake off fixed the problem, thanks, Ill take a shot at actually debugging the flymake vs edb now. Cheers Dale From masse@REDACTED Mon Jul 13 15:32:14 2009 From: masse@REDACTED (mats cronqvist) Date: Mon, 13 Jul 2009 15:32:14 +0200 Subject: Problem with distel debugging / breakpoint In-Reply-To: (Dale Harvey's message of "Mon\, 13 Jul 2009 14\:15\:58 +0100") References: <87zlb9j9cr.fsf@sterlett.hq.kred> Message-ID: <87vdlwk8td.fsf@sterlett.hq.kred> Dale Harvey writes: > 2009/7/13 mats cronqvist > >> Dale Harvey writes: >> >> > having avoided proper debugging with io statements for to long, >> > giving the distel debugger a shot. >> > >> > following the guide at http://bc.tech.coop/blog/070528.html which >> > is realy good, however hitting a wall. >> > >> > I C-c C-d i to set intepreted, works fine, I press C-x SPC to mark a >> > breakpoint, fine, I run a function that hits the break point, it stops, >> > I get am edb window, however when I press RET on the process, I >> > get the message No buffer for pid [TYPE erl-pid distel_768@REDACTED 8 0 0] >> >> are you using flymake? I've had problems using edb and flymake >> together... >> > > I am, and turning flymake off fixed the problem, thanks, Ill take a shot at > actually debugging the flymake vs edb now. oops. perhaps I should've mentioned that I've fixed it... you might want to take a look at http://github.com/massemanet/erlware-mode/tree/master From als@REDACTED Mon Jul 13 15:53:39 2009 From: als@REDACTED (Anthony Shipman) Date: Mon, 13 Jul 2009 23:53:39 +1000 Subject: [erlang-questions] Re: Any HTTP 1.0 clients out there? In-Reply-To: <4A5B24AB.7040508@erlang-consulting.com> References: <990c88ad-c0a5-44ff-a87e-3b5c1c17ce77@s6g2000vbp.googlegroups.com> <923814a6-9110-44ec-ac23-feaf33635e02@o13g2000vbl.googlegroups.com> <4A5B24AB.7040508@erlang-consulting.com> Message-ID: <200907132353.39595.als@iinet.net.au> On Mon, 13 Jul 2009 10:12:27 pm Oscar Hellstr?m wrote: > Hi Steve, > > I guess that depends on what kind of clients you will have. If you have > a controlled set of clients it would be very easy to make sure that > there are no HTTP/1.0 clients, but in a scenario where clients is the > general public I think you might still get some HTTP/1.0 clients. For > instance, lynx on my machine (,2.8.6rel.5 libwww-FM/2.14...) is claiming > to be an HTTP/1.0 client while links (ELinks/0.11.1-1.2etch...) on > another machine of mine is using HTTP/1.1. I do use lynx and links every > now and then, but I don't know if I would be a relevant user of your > application. > > Steve Davis wrote: > > Hi Oscar, > > > > My interest is in whether it is at all relevant to support HTTP/1.0 on > > a server any more - currently I've stripped out that code and just > > return status 505 (HTTP version not supported). > > > > Regards, > > Steve > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > Best regards I think that wget defaults to HTTP 1.0. -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From paul-trapexit@REDACTED Mon Jul 13 18:57:13 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Mon, 13 Jul 2009 09:57:13 -0700 (PDT) Subject: [erlang-questions] Mnesia and file system snapshot In-Reply-To: <20090713061733.D11891C000A5@mwinf2722.orange.fr> References: <20090713061733.D11891C000A5@mwinf2722.orange.fr> Message-ID: Wde, I have two answers. The first is that you should consider using Mnesia's built-in backup feature, and using other methods should be considered a "bad idea". It is really quite flexible and I recommend it highly. The second answer presumes you have considered and rejected the built-in backup procedure for some very good reason. In that case, grabbing a file system snapshot and then restoring it should be similar to asking Mnesia to recover from an emulator crash. I think this should mostly work. * disc_copies tables are periodically persisted to disk, and incremental changes are written to the write ahead log, so they should restore ok. dirty operations also use the logging system so even those should work. * ram_copies are not persisted normally, but you can ask mnesia to do so on a one-time basis explicitly via mnesia:dump_tables/1. * disc_only_copies are dets based, and dets makes a pretty good effort of durably persisting changes to disk when mnesia asks the storage layer to do so. You will have to suffer a dets table repair on startup after restoring your backup which can be time consuming. If you happen to use tcerl, you will run into trouble (http://code.google.com/p/tcerl/wiki/Durability) Cheers, -- p On Mon, 13 Jul 2009, wde wrote: > Hello, > > I would like to "backup" the mnesia directory by using a file system snapshot (like zfs snapshot for example). Before to take the snapshot I must be sure that mnesia files are in consistent state. Does the mnesia checkpoint maintain files (LOG, DAT,DCD,DCL ) in a consistent state (doc refers to tables state but not files) ? > > > Otherwise I can always snaphot a backup file . > > > Any feedback mnesia+file system snapshot would be appreciated. > > > Thank you, > > > wde > > > > > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From joelr1@REDACTED Mon Jul 13 20:15:24 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 13 Jul 2009 19:15:24 +0100 Subject: [erlang-questions] Re: 20k messages in 4s but want to go faster! In-Reply-To: <74ddf71d-3e08-46d1-8ea5-b88bf21f62ef@u16g2000pru.googlegroups.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> <97619b170907122322l6d7220abv29852ef0d24568fb@mail.gmail.com> <74ddf71d-3e08-46d1-8ea5-b88bf21f62ef@u16g2000pru.googlegroups.com> Message-ID: I changed my timing approach but it didn't seem to affect results much. setup time: 4412.022ms, n: 10000, run time: 6589.50ms min: 18.13ms, avg: 2074.14ms, max: 2074.22ms setup time: 7310.474ms, n: 10000, run time: 9444.31ms min: 0.39ms, avg: 2079.99ms, max: 2080.07ms setup time: 36879.026ms, n: 10000, run time: 39055.65ms min: 0.39ms, avg: 2117.59ms, max: 2117.66ms This is a 10k bot run on my MacBook Pro, 2 nodes, unicasting to the socket via a session process and then the transport process. Sending directly to the socket is not much different. I calculate setup time from when the run has started and until all bots checked in as ready. Minimum latency is really surprising here and it's something that I'm going to focus on. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From steven.charles.davis@REDACTED Mon Jul 13 20:54:07 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 13 Jul 2009 13:54:07 -0500 Subject: [erlang-questions] Re: Any HTTP 1.0 clients out there? In-Reply-To: <4A5B24AB.7040508@erlang-consulting.com> References: <990c88ad-c0a5-44ff-a87e-3b5c1c17ce77@s6g2000vbp.googlegroups.com> <4A5AFB0F.1020806@erlang-consulting.com> <923814a6-9110-44ec-ac23-feaf33635e02@o13g2000vbl.googlegroups.com> <4A5B24AB.7040508@erlang-consulting.com> Message-ID: <4A5B82CF.3050205@gmail.com> Hi Oscar, The environment is not at all controlled. Ah yes - Lynx is an interesting use case. I understand Lynx has a following in the financial sector, and so it definitely counts as "commercially relevant". I will investigate further to see what happens with Lynx when faced with an "HTTP/1.1 only" server. It may well tip the balance. My view is that server leniency has allowed MSIE, amongst others, to get away with far too much, and caused a great deal of unnecessary and frustrating hours in development. This abuse of the spirit of RFC 2616/HTTP1.1 has, I believe, actually held up the growth of the web - quite the opposite of the intentions of Berners-Lee/Fielding back in the 90's when HTTP was devised and the concept of leniency was "good". Many applications are now connection-orientated and far richer than just apache-style "serve media files" roots. Ideally, we would even have client-server ubf-style contracts; this view means the "right thing to do" would be to "just say no" to "old" clients in a standards-compliant way, and move on. Lynx may well be the use case that stops me doing this. Regards, Steve Oscar Hellstr?m wrote: > Hi Steve, > > I guess that depends on what kind of clients you will have. If you have > a controlled set of clients it would be very easy to make sure that > there are no HTTP/1.0 clients, but in a scenario where clients is the > general public I think you might still get some HTTP/1.0 clients. For > instance, lynx on my machine (,2.8.6rel.5 libwww-FM/2.14...) is claiming > to be an HTTP/1.0 client while links (ELinks/0.11.1-1.2etch...) on > another machine of mine is using HTTP/1.1. I do use lynx and links every > now and then, but I don't know if I would be a relevant user of your > application. > > Steve Davis wrote: >> Hi Oscar, >> >> My interest is in whether it is at all relevant to support HTTP/1.0 on >> a server any more - currently I've stripped out that code and just >> return status 505 (HTTP version not supported). >> >> Regards, >> Steve >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > Best regards > From steven.charles.davis@REDACTED Mon Jul 13 21:01:00 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 13 Jul 2009 12:01:00 -0700 (PDT) Subject: Any HTTP 1.0 clients out there? In-Reply-To: <200907132353.39595.als@iinet.net.au> References: <990c88ad-c0a5-44ff-a87e-3b5c1c17ce77@s6g2000vbp.googlegroups.com> <923814a6-9110-44ec-ac23-feaf33635e02@o13g2000vbl.googlegroups.com> <4A5B24AB.7040508@erlang-consulting.com> <200907132353.39595.als@iinet.net.au> Message-ID: <57c782a2-180d-4605-9251-6d456207e313@o13g2000vbl.googlegroups.com> Hi Anthony - Another one for me to check! Thank you. Steve On Jul 13, 8:53?am, Anthony Shipman wrote: > > I think that wget defaults to HTTP 1.0. From dizzyd@REDACTED Mon Jul 13 21:12:17 2009 From: dizzyd@REDACTED (Dave Smith) Date: Mon, 13 Jul 2009 13:12:17 -0600 Subject: [erlang-questions] Re: 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> <97619b170907122322l6d7220abv29852ef0d24568fb@mail.gmail.com> <74ddf71d-3e08-46d1-8ea5-b88bf21f62ef@u16g2000pru.googlegroups.com> Message-ID: Given that your average is so close to your max, I would hazard a guess that you would benefit from analyzing the latency distribution by percentile. The statistical mean is a handy tool, but for latency problems I've found it can often be very misleading. D. On Mon, Jul 13, 2009 at 12:15 PM, Joel Reymont wrote: > I changed my timing approach but it didn't seem to affect results much. > > setup time: 4412.022ms, n: 10000, run time: 6589.50ms > min: 18.13ms, avg: 2074.14ms, max: 2074.22ms > > setup time: 7310.474ms, n: 10000, run time: 9444.31ms > min: 0.39ms, avg: 2079.99ms, max: 2080.07ms > > setup time: 36879.026ms, n: 10000, run time: 39055.65ms > min: 0.39ms, avg: 2117.59ms, max: 2117.66ms > > This is a 10k bot run on my MacBook Pro, 2 nodes, unicasting to the socket > via a session process and then the transport process. > > Sending directly to the socket is not much different. > > I calculate setup time from when the run has started and until all bots > checked in as ready. > > Minimum latency is really surprising here and it's something that I'm going > to focus on. > > ? ? ? ?Thanks, Joel > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From wolfmanjm@REDACTED Mon Jul 13 21:14:19 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Mon, 13 Jul 2009 12:14:19 -0700 (PDT) Subject: 20k messages in 4s but want to go faster! In-Reply-To: <6B4835F7-0801-4BE5-8827-6644A0210B89@gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> <97619b170907122322l6d7220abv29852ef0d24568fb@mail.gmail.com> <74ddf71d-3e08-46d1-8ea5-b88bf21f62ef@u16g2000pru.googlegroups.com> <6B4835F7-0801-4BE5-8827-6644A0210B89@gmail.com> Message-ID: <75ff80cc-6dc2-4dac-a88c-deada32de5b1@a39g2000pre.googlegroups.com> On Jul 13, 1:59?am, Joel Reymont wrote: > > I think it is actually faster. > > Have you measured it? > No as my use case wouldn't work using that technique (I need to check the state of each client before sending the tcp packet). > > when the 8,000 or so gen_fsm processes get that message they do a ? > > tcp send to their client. That is getting the fastest throughput for me. > > How do you know it's getting the fastest throughput for you? Well the fact that my server seems to be much faster than your server ;) > > I won't believe doing an extra round trip through a gen_* server > is faster than pushing a static chunk of binary data to a list of ? > sockets. Actually I go through one gen_server and one gen_fsm for each message. > > Please prove me wrong! I have a theory as to why it is so much faster, however proof will require writing a lot of extra code :( Basically the theory goes like this... Using the list of sockets, you have to wait for every gen_tcp:send to complete before sending the next one, ie every send is serialized. Using a process per socket all the gen_tcp:send() are done in parallel (well kind of depending on how many threads/processes you have) but I suspect the gen_tcp:send has a yield in it once it is passed onto the OS so they are interleaved at a minimum. Sending 20,000 (or whatever) messages to 20,000 processes is extremely fast as Erlang is optimized for that use case, so the iteration that sends a message to each process will run a lot faster than an iteration that does gen_tcp:send(). Now I use a cast when sending these messages, were you using cast or call? I effectively do this to get the packet to each process... [ P ! packet || P <- Pids ] where Pids is a list of socket processes. From joelr1@REDACTED Mon Jul 13 22:48:57 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 13 Jul 2009 21:48:57 +0100 Subject: [erlang-questions] Re: 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> Message-ID: <7B9C1166-C869-445B-8696-4A03DC1770BD@gmail.com> Jim, On Jul 12, 2009, at 11:25 PM, Jim Morris wrote: > I can broadcast (actually technically unicast) > to 8,000 clients with max latency 277ms, and an average of 192ms. Are you running with -smp disabled, because I am. How many cores are you using? How does your throughput change if you disable smp? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From wolfmanjm@REDACTED Mon Jul 13 23:07:48 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Mon, 13 Jul 2009 14:07:48 -0700 (PDT) Subject: 20k messages in 4s but want to go faster! In-Reply-To: <7B9C1166-C869-445B-8696-4A03DC1770BD@gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> <7B9C1166-C869-445B-8696-4A03DC1770BD@gmail.com> Message-ID: smp was enabled, with smp disabled it seems slightly faster but not significantly... > erl -smp disable ... Latency - max: 203ms, max ave: 180K ms I have 2 cores and run with this command line... > erl +A 10 +K true ... Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads: 10] [hipe] [kernel-poll:true] Although the number of async threads doesn't make too much difference. On Jul 13, 1:48?pm, Joel Reymont wrote: > Jim, > > On Jul 12, 2009, at 11:25 PM, Jim Morris wrote: > > > I can broadcast (actually technically unicast) > > to 8,000 clients with max latency 277ms, and an average of 192ms. > > Are you running with -smp disabled, because I am. > > How many cores are you using? > > How does your throughput change if you disable smp? > > ? ? ? ? Thanks, Joel > > --- > Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From joelr1@REDACTED Tue Jul 14 00:13:33 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 13 Jul 2009 23:13:33 +0100 Subject: [erlang-questions] Re: 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> <97619b170907122322l6d7220abv29852ef0d24568fb@mail.gmail.com> <74ddf71d-3e08-46d1-8ea5-b88bf21f62ef@u16g2000pru.googlegroups.com> <7169FD85-C708-4A42-9FFD-EE76C2FA7E9F@gmail.com> Message-ID: On Jul 13, 2009, at 9:04 PM, Dave Smith wrote: > Based solely on your numbers, I think it's clear you're dealing with a > pretty skewed latency distribution and you should find a histogram > enlightening. :) Something like this? I split maximum time into 10 buckets and counted the latencies falling into them. Thanks, Joel --- setup: 4191.96ms, n: 10000, run time: 6278.44ms min: 2.05ms, avg: 2011.78ms, max: 2011.94ms 201.1936ms | 362 402.3872ms | 374 603.5808ms | 673 804.7744ms | 802 1005.9680ms | 928 1207.1616ms | 1015 1408.3552ms | 1151 1609.5488ms | 1296 1810.7424ms | 1493 2011.9360ms | 1906 setup: 576644.57ms, n: 20000, run time: 583199.64ms min: 1.95ms, avg: 6397.46ms, max: 6397.54ms 639.7537ms | 926 1279.5074ms | 894 1919.2611ms | 1054 2559.0148ms | 1393 3198.7685ms | 1580 3838.5222ms | 1735 4478.2759ms | 2053 5118.0296ms | 2319 5757.7833ms | 3184 6397.5370ms | 4862 --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Tue Jul 14 00:55:08 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 13 Jul 2009 23:55:08 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <20090710194505.GH4171@wellquite.org> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> <20090710194505.GH4171@wellquite.org> Message-ID: <2C6E800E-1D0C-4393-8676-00F64581563C@gmail.com> On Jul 10, 2009, at 8:45 PM, Matthew Sackman wrote: > Also, I have to say, I wouldn't use a dict. Immutable data > structures in > Erlang are _slow_ when compared to, eg, ets. OTOH, if you've timed the > fold and found it's only taking 0.2 secs, then that's clearly not the > problem here. I switched to ETS for my subscriber list. Two wit: info: [{memory,14284}, {owner,<0.305.0>}, {heir,none}, {name,subs}, {size,1000}, {node,'janus@REDACTED'}, {named_table,false}, {type,set}, {keypos,1}, {protection,protected}] time: 26.076 info: [{memory,141821}, {owner,<0.305.0>}, {heir,none}, {name,subs}, {size,10000}, {node,'janus@REDACTED'}, {named_table,false}, {type,set}, {keypos,1}, {protection,protected}] time: 722.08 It's ~20ms to traverse 1000 subscribers, close to 1s for 10k and about 2s for 20k. It looks like around 30% of my 10k max latency is taken up by the traversal. My code looks like this: --- Msg1 = {message, iolist_to_binary(mochijson2:encode(JSON))}, F = fun({Pid, _}, _) -> gen_server:cast(Pid, Msg1) end, F1 = fun() -> A = now(), ets:foldr(F, ignore, State#state.subs), io:format("time: ~p~n", [timer:now_diff(now(), A) / 1000]) end, spawn_link(F1), --- I think it was faster with a dict, perhaps I should time it again. It doesn't matter to me how long it takes to modify the data structure to add a new subscriber but traversal time does matter. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From steven.charles.davis@REDACTED Tue Jul 14 00:58:14 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 13 Jul 2009 15:58:14 -0700 (PDT) Subject: Loading code from an archive In-Reply-To: <269388e30907120559u1c72c8ealcbe882acbe4732d2@mail.gmail.com> References: <269388e30907120512y38566659o1d155c1d15b69274@mail.gmail.com> <269388e30907120559u1c72c8ealcbe882acbe4732d2@mail.gmail.com> Message-ID: <22b5d339-72d8-4e01-b320-25dfd8cbca52@f33g2000vbm.googlegroups.com> No answer here, but an addendum: I love the .ez idea and what it represents for the future (SAE redux) and have uses for it already (or at least as soon as it goes production ready). Incoveniently, it seems that .ez files cannot be opened by one of the most popular and widespread zip file readers i.e. 7-Zip. I think it unlikely that the bug is in 7-Zip..? /sd From 0x6e6562@REDACTED Tue Jul 14 01:14:26 2009 From: 0x6e6562@REDACTED (Ben Hood) Date: Tue, 14 Jul 2009 00:14:26 +0100 Subject: [erlang-questions] Re: Loading code from an archive In-Reply-To: <22b5d339-72d8-4e01-b320-25dfd8cbca52@f33g2000vbm.googlegroups.com> References: <269388e30907120512y38566659o1d155c1d15b69274@mail.gmail.com> <269388e30907120559u1c72c8ealcbe882acbe4732d2@mail.gmail.com> <22b5d339-72d8-4e01-b320-25dfd8cbca52@f33g2000vbm.googlegroups.com> Message-ID: <269388e30907131614r958000fp504d43e8f686a2d1@mail.gmail.com> Steve, On Mon, Jul 13, 2009 at 11:58 PM, Steve Davis wrote: > Incoveniently, it seems that .ez files cannot be opened by one of the > most popular and widespread zip file readers i.e. 7-Zip. I think it > unlikely that the bug is in 7-Zip..? The zip listing in my original post was produced by 7-Zip, albeit with the CLI version. Example usage is: 7za l deps/mochiweb.ez 7-Zip (A) 4.57 Copyright (c) 1999-2007 Igor Pavlov 2007-12-06 p7zip Version 4.57 (locale=utf8,Utf16=on,HugeFiles=on,2 CPUs) Listing archive: deps/mochiweb.ez Date Time Attr Size Compressed Name ------------------- ----- ------------ ------------ ------------------------ 2009-07-13 21:00:52 D.... 0 0 mochiweb 2009-07-14 00:28:12 D.... 0 0 mochiweb/ebin 2009-07-14 00:28:06 D.... 0 0 mochiweb/ebin/.svn 2009-07-13 21:00:52 ..... 67 64 mochiweb/ebin/.svn/all-wcprops 2009-07-13 21:00:52 ..... 39 39 mochiweb/ebin/.svn/dir-prop-base 2009-07-13 21:00:52 ..... 240 164 mochiweb/ebin/.svn/entries 2009-07-13 21:00:52 ..... 2 2 mochiweb/ebin/.svn/format 2009-07-13 21:00:52 D.... 0 0 mochiweb/ebin/.svn/prop-base 2009-07-13 21:00:52 D.... 0 0 mochiweb/ebin/.svn/props 2009-07-13 21:00:52 D.... 0 0 mochiweb/ebin/.svn/text-base 2009-07-13 21:00:52 D.... 0 0 mochiweb/ebin/.svn/tmp 2009-07-13 21:00:52 D.... 0 0 mochiweb/ebin/.svn/tmp/prop-base 2009-07-13 21:00:52 D.... 0 0 mochiweb/ebin/.svn/tmp/props 2009-07-13 21:00:52 D.... 0 0 mochiweb/ebin/.svn/tmp/text-base 2009-07-14 00:28:08 ..... 18860 15114 mochiweb/ebin/mochifmt.beam 2009-07-14 00:28:08 ..... 1840 1442 mochiweb/ebin/mochifmt_records.beam 2009-07-14 00:28:08 ..... 1604 1168 mochiweb/ebin/mochifmt_std.beam 2009-07-14 00:28:08 ..... 3116 2379 mochiweb/ebin/mochihex.beam 2009-07-14 00:28:08 ..... 24584 18524 mochiweb/ebin/mochijson.beam 2009-07-14 00:28:08 ..... 28080 20814 mochiweb/ebin/mochijson2.beam 2009-07-14 00:28:08 ..... 9664 7594 mochiweb/ebin/mochinum.beam 2009-07-14 00:28:12 ..... 732 261 mochiweb/ebin/mochiweb.app 2009-07-14 00:28:08 ..... 4500 3480 mochiweb/ebin/mochiweb.beam 2009-07-14 00:28:10 ..... 1184 828 mochiweb/ebin/mochiweb_app.beam 2009-07-14 00:28:10 ..... 19448 9132 mochiweb/ebin/mochiweb_charref.beam 2009-07-14 00:28:10 ..... 11036 8151 mochiweb/ebin/mochiweb_cookies.beam 2009-07-14 00:28:10 ..... 1732 1352 mochiweb/ebin/mochiweb_echo.beam 2009-07-14 00:28:10 ..... 9024 6111 mochiweb/ebin/mochiweb_headers.beam 2009-07-14 00:28:10 ..... 44252 33694 mochiweb/ebin/mochiweb_html.beam 2009-07-14 00:28:10 ..... 7008 5656 mochiweb/ebin/mochiweb_http.beam 2009-07-14 00:28:10 ..... 19912 15479 mochiweb/ebin/mochiweb_multipart.beam 2009-07-14 00:28:10 ..... 27456 21755 mochiweb/ebin/mochiweb_request.beam 2009-07-14 00:28:12 ..... 2340 1843 mochiweb/ebin/mochiweb_response.beam 2009-07-14 00:28:12 ..... 4740 3923 mochiweb/ebin/mochiweb_skel.beam 2009-07-14 00:28:12 ..... 9720 7933 mochiweb/ebin/mochiweb_socket_server.beam 2009-07-14 00:28:12 ..... 1644 1267 mochiweb/ebin/mochiweb_sup.beam 2009-07-14 00:28:12 ..... 30268 18238 mochiweb/ebin/mochiweb_util.beam 2009-07-14 00:28:12 ..... 5344 4498 mochiweb/ebin/reloader.beam ------------------- ----- ------------ ------------ ------------------------ 288436 210905 28 files, 10 folders HTH, Ben From richardc@REDACTED Tue Jul 14 09:25:56 2009 From: richardc@REDACTED (Richard Carlsson) Date: Tue, 14 Jul 2009 09:25:56 +0200 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <2C6E800E-1D0C-4393-8676-00F64581563C@gmail.com> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> <20090710194505.GH4171@wellquite.org> <2C6E800E-1D0C-4393-8676-00F64581563C@gmail.com> Message-ID: <4A5C3304.1080201@it.uu.se> Joel Reymont wrote: > On Jul 10, 2009, at 8:45 PM, Matthew Sackman wrote: > >> Also, I have to say, I wouldn't use a dict. Immutable data structures in >> Erlang are _slow_ when compared to, eg, ets. It depends on the data and what you need to do with it. If each entry is rather large and you do a lot of retreival and/or insertion, you will spend a whole lot of time copying data between the process heap and the table (and cause more garbage collection as a consequence). An immutable data structure resides on the process heap all the time. Of course, if the table size gets huge enough, the log(n) factor will eventually be greater than the work spent in copying; as usual, you'll simply have to measure. That said, ETS tables have some other advantages as well, arguably the most important being that you can use fast match specifications and qlc for searching. If you go with ETS tables, you may want to think about normalizing your representation and splitting it over several tables, to keep the size of each table row as small as possible. (But if you need to read every part of an entry each time anyway, this does not help.) /Richard From steven.charles.davis@REDACTED Tue Jul 14 10:40:19 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 14 Jul 2009 03:40:19 -0500 Subject: [erlang-questions] Re: Loading code from an archive In-Reply-To: <269388e30907131614r958000fp504d43e8f686a2d1@mail.gmail.com> References: <269388e30907120512y38566659o1d155c1d15b69274@mail.gmail.com> <269388e30907120559u1c72c8ealcbe882acbe4732d2@mail.gmail.com> <22b5d339-72d8-4e01-b320-25dfd8cbca52@f33g2000vbm.googlegroups.com> <269388e30907131614r958000fp504d43e8f686a2d1@mail.gmail.com> Message-ID: <4A5C4473.4020204@gmail.com> Hi Ben - Neither CLI or GUI work for me, and here's why... Upon further investigation, I found that the bug is indeed in 7-Zip for Windows. I'd report the bug to them but they have an acid "don't mail us" in the FAQ... so I won't -- instead it's back to WinRAR for me ;) OTOH Erlang/OTP wins again :) BR, Steve Ben Hood wrote: > The zip listing in my original post was produced by 7-Zip, albeit with > the CLI version. Example usage is: > > 7za l deps/mochiweb.ez > > 7-Zip (A) 4.57 Copyright (c) 1999-2007 Igor Pavlov 2007-12-06 > p7zip Version 4.57 (locale=utf8,Utf16=on,HugeFiles=on,2 CPUs) From joelr1@REDACTED Tue Jul 14 10:49:13 2009 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 14 Jul 2009 09:49:13 +0100 Subject: [erlang-questions] 20k messages in 4s but want to go faster! In-Reply-To: <4A5C3304.1080201@it.uu.se> References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <20090710182935.GE4171@wellquite.org> <05CA4C8F-90A1-4692-865D-FF5FDC1C0CB6@gmail.com> <20090710183821.GF4171@wellquite.org> <3ED50801-26E8-48AF-8B66-C7D909B046A7@gmail.com> <20090710185144.GG4171@wellquite.org> <20090710194505.GH4171@wellquite.org> <2C6E800E-1D0C-4393-8676-00F64581563C@gmail.com> <4A5C3304.1080201@it.uu.se> Message-ID: On Jul 14, 2009, at 8:25 AM, Richard Carlsson wrote: > It depends on the data and what you need to do with it. If each entry > is rather large and you do a lot of retreival and/or insertion, you > will > spend a whole lot of time copying data between the process heap and > the > table (and cause more garbage collection as a consequence). I map strings (binaries) to pids to keep track of who services what topic. I also map pids to refs returned by erlang:monitor to keep track of topic subscribers. I monitor subscribers to automatically unsubscribe them when they go down. I'll be taking apart my server today to take out all the cruft and rebuild it as something that just pushes binaries around. This is just so that I can get some basic performance measures and, hopefully, pinpoint the bottleneck. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From chandrashekhar.mullaparthi@REDACTED Tue Jul 14 11:01:47 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 14 Jul 2009 02:01:47 -0700 (PDT) Subject: HTTP:set_option() equal function in iBrowse In-Reply-To: <90b4299d0906182347v4ca3b100o5cca67028a3026f6@mail.gmail.com> References: <90b4299d0906182347v4ca3b100o5cca67028a3026f6@mail.gmail.com> Message-ID: <60fcf412-28e9-491e-a883-404d57961adf@s6g2000vbp.googlegroups.com> Hi, Cookie management is not supported in ibrowse. I firmly believe that it is a function of the application using HTTP, not the HTTP client itself. cheers Chandru On Jun 19, 7:47?am, "MAthuvathanan Mou." wrote: > Hi all, > > If I want to use ibrowse and enable cookies like the function in http module > http:set_options([{cookies, enabled}]) > how can I do? > > Thanks > > -- > Mathuvathanan Mou. From steven.charles.davis@REDACTED Tue Jul 14 12:02:36 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 14 Jul 2009 03:02:36 -0700 (PDT) Subject: Design question -- a valid or foolhardy approach to ets update? Message-ID: <6d6c448f-37cd-4ced-a2e8-990c752b883a@l32g2000vbp.googlegroups.com> I am maintaining HTTP session data as records in an ETS table. This table is managed by a gen_server. However, each time the client makes a request the session expiry time needs to be updated, which means updating one field in the session record. In order not to bottleneck the gen_server process by having all clients contact the gen_server at every request, I'm considering a direct ETS update of the session record from the client process rather than doing a safe gen_server:call to update the record. i.e.: %% TODO: probably needs a big fat review... update_session(S = #session{key=Key}) -> case ets:lookup(?ETS, Key) of [_Existing] -> NewS = S#session{expires = unow() + S#session.ttl}, ets:insert(?ETS, NewS); [] -> {error, not_found} end. My reasoning is that since **only the client process** (meaning the spawned HTTP connection) will ever access its own record in the central ?ETS session table this approach should be safe to do... am I correct in my thinking or am I badly wrong somewhere? Is there some consideration I'm missing that could mean that this could result in lock contention or deadlock? Would I be better advised to at least surround the read/insert in a try begin/end catch? TYVMIA, regs, Steve From mihai@REDACTED Tue Jul 14 13:41:31 2009 From: mihai@REDACTED (Mihai Balea) Date: Tue, 14 Jul 2009 07:41:31 -0400 Subject: [erlang-questions] Design question -- a valid or foolhardy approach to ets update? In-Reply-To: <6d6c448f-37cd-4ced-a2e8-990c752b883a@l32g2000vbp.googlegroups.com> References: <6d6c448f-37cd-4ced-a2e8-990c752b883a@l32g2000vbp.googlegroups.com> Message-ID: <1D2EDE20-A9A9-4EA5-A9EA-F5FD6AE8588E@hates.ms> On Jul 14, 2009, at 6:02 AM, Steve Davis wrote: > I am maintaining HTTP session data as records in an ETS table. This > table is managed by a gen_server. However, each time the client makes > a request the session expiry time needs to be updated, which means > updating one field in the session record. > > In order not to bottleneck the gen_server process by having all > clients contact the gen_server at every request, I'm considering a > direct ETS update of the session record from the client process rather > than doing a safe gen_server:call to update the record. i.e.: > > ... > My reasoning is that since **only the client process** (meaning the > spawned HTTP connection) will ever access its own record in the > central ?ETS session table this approach should be safe to do... am I > correct in my thinking or am I badly wrong somewhere? Is there some > consideration I'm missing that could mean that this could result in > lock contention or deadlock? I don't know if what you're suggesting is feasible, but if it is not, have you considered using ets:update_counter/3? It is probably faster than a gen_server process owning the table. Mihai From steven.charles.davis@REDACTED Tue Jul 14 14:07:57 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 14 Jul 2009 07:07:57 -0500 Subject: [erlang-questions] Design question -- a valid or foolhardy approach to ets update? In-Reply-To: <1D2EDE20-A9A9-4EA5-A9EA-F5FD6AE8588E@hates.ms> References: <6d6c448f-37cd-4ced-a2e8-990c752b883a@l32g2000vbp.googlegroups.com> <1D2EDE20-A9A9-4EA5-A9EA-F5FD6AE8588E@hates.ms> Message-ID: <4A5C751D.70901@gmail.com> Hi Mihai - Wow. That is the perfect solution. Since update_counter ensures the update will be atomic... well, that's just exactly the right thing to do. Many thanks to you! Steve Mihai Balea wrote: > > On Jul 14, 2009, at 6:02 AM, Steve Davis wrote: > I don't know if what you're suggesting is feasible, but if it is not, > have you considered using ets:update_counter/3? > It is probably faster than a gen_server process owning the table. > > Mihai > From mihai@REDACTED Tue Jul 14 15:30:19 2009 From: mihai@REDACTED (Mihai Balea) Date: Tue, 14 Jul 2009 09:30:19 -0400 Subject: [erlang-questions] Design question -- a valid or foolhardy approach to ets update? In-Reply-To: <4A5C751D.70901@gmail.com> References: <6d6c448f-37cd-4ced-a2e8-990c752b883a@l32g2000vbp.googlegroups.com> <1D2EDE20-A9A9-4EA5-A9EA-F5FD6AE8588E@hates.ms> <4A5C751D.70901@gmail.com> Message-ID: On Jul 14, 2009, at 8:07 AM, Steve Davis wrote: > Hi Mihai - > > Wow. That is the perfect solution. > > Since update_counter ensures the update will be atomic... well, > that's just exactly the right thing to do. Just be aware that atomic updates means that updates will still be serialized - you will still run into a concurrency bottleneck since the operation will prevent any other access to the table while it is in progress. It will probably be faster than the gen_server solution, but still not a fully concurrent approach. I'm not even sure a fully concurrent solution is possible with ets tables. You will probably have to encapsulate session ids in processes to do that... which will significantly increase your memory footprint and transaction latency, but will scale better. I would suggest you try update_counter first and if it turns out it is indeed a bottleneck, then go for a fancier approach. Mihai From ulf.wiger@REDACTED Tue Jul 14 16:28:07 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 14 Jul 2009 16:28:07 +0200 Subject: [erlang-questions] Design question -- a valid or foolhardy approach to ets update? In-Reply-To: References: <6d6c448f-37cd-4ced-a2e8-990c752b883a@l32g2000vbp.googlegroups.com> <1D2EDE20-A9A9-4EA5-A9EA-F5FD6AE8588E@hates.ms> <4A5C751D.70901@gmail.com> Message-ID: <4A5C95F7.3060209@erlang-consulting.com> Mihai Balea wrote: > > Just be aware that atomic updates means that updates will still be > serialized - you will still run into a concurrency bottleneck since the > operation will prevent any other access to the table while it is in > progress. It will probably be faster than the gen_server solution, but > still not a fully concurrent approach. In OTP R13B01, the concurrency granularity of ets table updates can be tuned. From the documentation of ets:new(...): "{write_concurrency,bool()} Performance tuning. Default is false, which means that the table is optimized towards concurrent read access. An operation that mutates (writes to) the table will obtain exclusive access, blocking any concurrent access of the same table until finished. If set to true, the table is optimized towards concurrent write access. Different parts of the same table can be mutated (and read) by concurrent processes. This is achieve to some degree at the expense of single access and concurrent reader performance. Table typ ordered_set is not affected by this option in current implementation." Note also that update_counter() requires the key to already exist in the table. Try to find a good place to initially create the counter. Otherwise, a fairly common pattern is: update_counter(Key, Incr) -> try ets:update_counter(?tab, Key, {#counter.value, Incr}) catch error:_ -> init_counter(Key, Incr) end. init_counter(Key, Incr) -> true = ets:insert_new(?tab, #counter{key = Key, value = Incr}), Incr. ...but this obviously messes with atomicity. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From mihai@REDACTED Tue Jul 14 17:25:01 2009 From: mihai@REDACTED (Mihai Balea) Date: Tue, 14 Jul 2009 11:25:01 -0400 Subject: [erlang-questions] Design question -- a valid or foolhardy approach to ets update? In-Reply-To: <4A5C95F7.3060209@erlang-consulting.com> References: <6d6c448f-37cd-4ced-a2e8-990c752b883a@l32g2000vbp.googlegroups.com> <1D2EDE20-A9A9-4EA5-A9EA-F5FD6AE8588E@hates.ms> <4A5C751D.70901@gmail.com> <4A5C95F7.3060209@erlang-consulting.com> Message-ID: On Jul 14, 2009, at 10:28 AM, Ulf Wiger wrote: > > From the documentation of ets:new(...): > > "{write_concurrency,bool()} > Performance tuning. Default is false, which means that the table is > optimized towards concurrent read access. An operation that mutates > (writes to) the table will obtain exclusive access, blocking any > concurrent access of the same table until finished. If set to true, > the table is optimized towards concurrent write access. Different > parts of the same table can be mutated (and read) by concurrent > processes. This is achieve to some degree at the expense of single > access and concurrent reader performance. Table typ ordered_set is > not affected by this option in current implementation." Good stuff, I wasn't aware of that. Mihai From sverker@REDACTED Tue Jul 14 17:00:13 2009 From: sverker@REDACTED (Sverker Eriksson) Date: Tue, 14 Jul 2009 17:00:13 +0200 Subject: [erlang-questions] Design question -- a valid or foolhardy approach to ets update? In-Reply-To: <4A5C95F7.3060209@erlang-consulting.com> References: <6d6c448f-37cd-4ced-a2e8-990c752b883a@l32g2000vbp.googlegroups.com> <1D2EDE20-A9A9-4EA5-A9EA-F5FD6AE8588E@hates.ms> <4A5C751D.70901@gmail.com> <4A5C95F7.3060209@erlang-consulting.com> Message-ID: <4A5C9D7D.3020007@erix.ericsson.se> Ulf Wiger wrote: > Note also that update_counter() requires the key to already > exist in the table. Try to find a good place to initially > create the counter. Otherwise, a fairly common pattern is: > > update_counter(Key, Incr) -> > try ets:update_counter(?tab, Key, {#counter.value, Incr}) > catch > error:_ -> init_counter(Key, Incr) > end. > > init_counter(Key, Incr) -> > true = ets:insert_new(?tab, #counter{key = Key, value = Incr}), > Incr. > > ...but this obviously messes with atomicity. > I think you will be "safe" as long as you catch the unlikely race when insert_new fails: init_counter(Key, Incr) -> case ets:insert_new(?tab, #counter{key = Key, value = Incr}) of true -> Incr; false -> update_counter(Key, Incr) end. /Sverker, Erlang/OTP From steven.charles.davis@REDACTED Wed Jul 15 00:08:48 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 14 Jul 2009 17:08:48 -0500 Subject: [erlang-questions] Design question -- a valid or foolhardy approach to ets update? In-Reply-To: References: <6d6c448f-37cd-4ced-a2e8-990c752b883a@l32g2000vbp.googlegroups.com> <1D2EDE20-A9A9-4EA5-A9EA-F5FD6AE8588E@hates.ms> <4A5C751D.70901@gmail.com> <4A5C95F7.3060209@erlang-consulting.com> Message-ID: <4A5D01F0.90000@gmail.com> Hi Ulf, Interesting indeed! Time I had a really close read of the latest ets documentation. In this case, that option in ets:new is highly appropriate as the table is unusually write heavy, updates of ttl for every connection, updating session data... i.e. authenticated user and various application-level userdata related to that session, and finally clearing out stale sessions. It will be a while until I am in a position to profile/load test, but I'll be sure to report results when I have them. My thanks and best regards, Steve Mihai Balea wrote: > > On Jul 14, 2009, at 10:28 AM, Ulf Wiger wrote: >> >> From the documentation of ets:new(...): >> >> "{write_concurrency,bool()} >> Performance tuning. Default is false, which means that the table is >> optimized towards concurrent read access. An operation that mutates >> (writes to) the table will obtain exclusive access, blocking any >> concurrent access of the same table until finished. If set to true, >> the table is optimized towards concurrent write access. Different >> parts of the same table can be mutated (and read) by concurrent >> processes. This is achieve to some degree at the expense of single >> access and concurrent reader performance. Table typ ordered_set is not >> affected by this option in current implementation." > > Good stuff, I wasn't aware of that. > > Mihai From kenji.rikitake@REDACTED Wed Jul 15 03:47:54 2009 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Wed, 15 Jul 2009 10:47:54 +0900 Subject: [erlang-bugs] Erlang R13B01 ssh-1.1.3 cipher key matching bug and documentation errors In-Reply-To: <20090712025646.GA44385@k2r.org> References: <20090712025646.GA44385@k2r.org> Message-ID: <20090715014754.GA9277@k2r.org> Justin Sheehy told me on Twitter that Erlang crypto module already supports AES, and I confirmed related functions in crypto.erl, so I guess it's a matter of ssh module implementation. FYI Kenji Rikitake In the message <20090712025646.GA44385@REDACTED> dated Sun, Jul 12, 2009 at 11:56:22AM +0900, Kenji Rikitake writes: > * [my opinion] I personally think only supporting 3des-cbc is *archaic* > and insufficient; implementing at least stronger ciphers such as > aes128-cbc and aes256-cbc, or even blowfish-cbc, should be considered > ASAP, regarding the strength of the ciphers. From carlmcdade@REDACTED Wed Jul 15 10:08:08 2009 From: carlmcdade@REDACTED (Carl McDade) Date: Wed, 15 Jul 2009 10:08:08 +0200 Subject: Cannot find erl.exe on Linux Mint Message-ID: Okay, I gave it the usual two days of hunting and testing but still cannot get Linux to use or see the erl.exe. Everything compiled ok and the files are there but Linux gives this message: The program 'erl' can be found in the following packages: * erlang-base * erlang-base-hipe Try: apt-get install bash: erl: command not found minty ~ # Any help is appreciated -- Carl McDade Web developer www.hiveminds.co.uk ________________________ From richardc@REDACTED Wed Jul 15 11:09:15 2009 From: richardc@REDACTED (Richard Carlsson) Date: Wed, 15 Jul 2009 11:09:15 +0200 Subject: [erlang-questions] Cannot find erl.exe on Linux Mint In-Reply-To: References: Message-ID: <4A5D9CBB.7030400@it.uu.se> Carl McDade wrote: > Okay, I gave it the usual two days of hunting and testing but still > cannot get Linux to use or see the erl.exe. Everything compiled ok and > the files are there but Linux gives this message: > > The program 'erl' can be found in the following packages: > * erlang-base > * erlang-base-hipe > Try: apt-get install > bash: erl: command not found > minty ~ # > > Any help is appreciated So, I assume that your intention is to use a version you have compiled yourself, rather than just apt-getting the distro packages. If compilation worked, did you make install? If so, where did it put the files? Did it create a link from /usr/bin/erl or similar location (which should be in your path) to the actual executable? (You can always run make install again to check what it does.) /Richard From rtrlists@REDACTED Wed Jul 15 11:25:29 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 15 Jul 2009 10:25:29 +0100 Subject: XML-binary Optimized Packaging (XOP) ? Message-ID: <6a3ae47e0907150225q7c496ea9ra8ae02891561792c@mail.gmail.com> Hi, before I embark on cobbling something together that makes sense of XML munged into this XOP format (http://www.w3.org/TR/xop10/) I thought I'd ask if anyone out there has already done this in Erlang? Thanks for any info, Robby From steven.charles.davis@REDACTED Wed Jul 15 11:44:14 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 15 Jul 2009 02:44:14 -0700 (PDT) Subject: XOP - final proof that UBF is the correct approach In-Reply-To: <6a3ae47e0907150225q7c496ea9ra8ae02891561792c@mail.gmail.com> References: <6a3ae47e0907150225q7c496ea9ra8ae02891561792c@mail.gmail.com> Message-ID: <44c39686-6fc8-41cf-9b5f-5d9ec9ec2ca3@g31g2000yqc.googlegroups.com> Isn't this XOP stuff proof that XML is struggling towards being something (UBF) but is fundamentally a )document markup) technology that is in widespread use for entirely the wrong purposes. I'm trying to get a foothold into what kind of application would encourage developers to favor UBF over XML. Anybody given any thought to this problem that perhaps they can share? /sd From carlmcdade@REDACTED Wed Jul 15 12:01:22 2009 From: carlmcdade@REDACTED (Carl McDade) Date: Wed, 15 Jul 2009 12:01:22 +0200 Subject: [erlang-questions] Cannot find erl.exe on Linux Mint In-Reply-To: <4A5D9CBB.7030400@it.uu.se> References: <4A5D9CBB.7030400@it.uu.se> Message-ID: Yep, I wanted to compile and use R13B01 because there is no repository ( debian or otherwise ) with the latest version. Apt-get only shows R12 available. The files are in a directory under /root/otp_src_R13B01 I don't know how to create such a link. I am following a tutorial and there is no mention of such. I have tried to add to the PATH variable with PATH = $PATH:/root/otp_src_R1301 But it only returns an error minty ~ # erl The program 'erl' can be found in the following packages: * erlang-base * erlang-base-hipe Try: apt-get install bash: erl: command not found minty ~ # PATH = $PATH:/root/otp_src_R13B01 bash: PATH: command not found minty ~ # On Wed, Jul 15, 2009 at 11:09 AM, Richard Carlsson wrote: > Carl McDade wrote: >> Okay, I gave it the usual two days of hunting and testing but still >> cannot get Linux to use or see the erl.exe. Everything compiled ok and >> the files are there but Linux gives this message: >> >> The program 'erl' can be found in the following packages: >> ?* erlang-base >> ?* erlang-base-hipe >> Try: apt-get install >> bash: erl: command not found >> minty ~ # >> >> Any help is appreciated > > So, I assume that your intention is to use a version you have > compiled yourself, rather than just apt-getting the distro packages. > If compilation worked, did you make install? If so, where did it > put the files? Did it create a link from /usr/bin/erl or similar > location (which should be in your path) to the actual executable? > (You can always run make install again to check what it does.) > > ? ?/Richard > -- Carl McDade Web developer www.hiveminds.co.uk ________________________ From kostis@REDACTED Wed Jul 15 12:03:15 2009 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 15 Jul 2009 13:03:15 +0300 Subject: [erlang-questions] Cannot find erl.exe on Linux Mint In-Reply-To: References: <4A5D9CBB.7030400@it.uu.se> Message-ID: <4A5DA963.4030502@cs.ntua.gr> Carl McDade wrote: > Yep, I wanted to compile and use R13B01 because there is no repository > ( debian or otherwise ) with the latest version. Apt-get only shows > R12 available. > > The files are in a directory under /root/otp_src_R13B01 > > I don't know how to create such a link. I am following a tutorial and > there is no mention of such. > > I have tried to add to the PATH variable with PATH = $PATH:/root/otp_src_R1301 > But it only returns an error If you do not want to 'make install' as Richard suggested, you need to put /root/otp_src_R1301/bin in your path, not just /root/otp_src_R1301 Kostis From carlmcdade@REDACTED Wed Jul 15 12:19:17 2009 From: carlmcdade@REDACTED (Carl McDade) Date: Wed, 15 Jul 2009 12:19:17 +0200 Subject: [erlang-questions] Cannot find erl.exe on Linux Mint In-Reply-To: <4A5DA963.4030502@cs.ntua.gr> References: <4A5D9CBB.7030400@it.uu.se> <4A5DA963.4030502@cs.ntua.gr> Message-ID: Tried . minty ~ # $PATH = $PATH:~/root/otp_src_R13B01/bin bash: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games: No such file or directory minty ~ # PATH = $PATH:/root/otp_src_R13B01/bin Without any change. Is it just me? These commands work fine on Xandros but not on this Debian distro /Carl On Wed, Jul 15, 2009 at 12:03 PM, Kostis Sagonas wrote: > Carl McDade wrote: >> >> Yep, I wanted to compile and use R13B01 because there is no repository >> ( debian or otherwise ) with the latest version. Apt-get only shows >> R12 available. >> >> The files are in a directory under /root/otp_src_R13B01 >> >> I don't know how to create such a link. I am following a tutorial and >> there is no mention of such. >> >> I have tried to add to the PATH variable with PATH = >> $PATH:/root/otp_src_R1301 >> But it only returns an error > > If you do not want to 'make install' as Richard suggested, you need to put > /root/otp_src_R1301/bin in your path, not just /root/otp_src_R1301 > > Kostis > -- Carl McDade Web developer www.hiveminds.co.uk ________________________ From richardc@REDACTED Wed Jul 15 12:29:15 2009 From: richardc@REDACTED (Richard Carlsson) Date: Wed, 15 Jul 2009 12:29:15 +0200 Subject: [erlang-questions] Cannot find erl.exe on Linux Mint In-Reply-To: References: <4A5D9CBB.7030400@it.uu.se> Message-ID: <4A5DAF7B.8020809@it.uu.se> Carl McDade wrote: > Yep, I wanted to compile and use R13B01 because there is no repository > ( debian or otherwise ) with the latest version. Apt-get only shows > R12 available. > > The files are in a directory under /root/otp_src_R13B01 Ok, so you're doing this as root. In that case, just cd into the /root/otp_src_R13B01 directory (as you did when you compiled it to begin with) and write 'make install'. This should install the executables under the default location /usr/local/bin, and you're good to go. For more information, if you want to install under a different location than the default, etc., read the file called README. It contains a clear step-by-step instruction on how to configure, compile, and install. No web tutorials needed. > I have tried to add to the PATH variable with PATH = $PATH:/root/otp_src_R1301 > But it only returns an error Well, you left out the /bin part at the end. And while you can run the system this way, you'll get a better setup if you make a proper install. /Richard From richardc@REDACTED Wed Jul 15 12:43:01 2009 From: richardc@REDACTED (Richard Carlsson) Date: Wed, 15 Jul 2009 12:43:01 +0200 Subject: [erlang-questions] Cannot find erl.exe on Linux Mint In-Reply-To: References: <4A5D9CBB.7030400@it.uu.se> <4A5DA963.4030502@cs.ntua.gr> Message-ID: <4A5DB2B5.9040406@it.uu.se> Carl McDade wrote: > Tried . > > minty ~ # $PATH = $PATH:~/root/otp_src_R13B01/bin > bash: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games: > No such file or directory > minty ~ # PATH = $PATH:/root/otp_src_R13B01/bin > > Without any change. Is it just me? These commands work fine on Xandros > but not on this Debian distro Looks like you're unused to Unix or to shells in general. What happened was that you gave the left hand side of the assignment as $PATH, instead of just PATH, so the shell expanded the variable for you, hence the left hand side of the '=' became "/usr/... ". A second problem was that you left a space between PATH and the equals sign, so the shell did in fact not interpret this as an assignment, but as a command (whatever $PATH expanded to) whose first parameter was "=". Writing PATH=$PATH:/root/otp... should do it. /Richard From aniko@REDACTED Wed Jul 15 12:43:24 2009 From: aniko@REDACTED (Aniko Nagyne Vig) Date: Wed, 15 Jul 2009 11:43:24 +0100 (BST) Subject: Linked-in drivers' symbolic info is missing from gstack log In-Reply-To: <19089130.10511247654563653.JavaMail.root@zimbra> Message-ID: <3672064.10531247654604215.JavaMail.root@zimbra> I tried to run gstack one of our node's process, but I can't see the symbolic info for the linked-in drivers of the node. The VM and the linked in drivers were compiled on the same machine, so I suspect some compilation flag incompatibility, what blocks the symbolic infos. I could not find any compiling options what should make the symbolic debugging impossible (-mlinker opt or -mrelax). The flags I'm using for compiling the linked-in drivers: -g -Wall -fpic -DDEBUG -DTHREAD_SAFE -I/usr/local/include `erl-config --cflags erts` `erl-config --cflags erl_interface` Do you know what can cause the incompatibility? Thanks, Aniko From alexander.uvarov@REDACTED Wed Jul 15 13:11:20 2009 From: alexander.uvarov@REDACTED (Alexander Uvarov) Date: Wed, 15 Jul 2009 17:11:20 +0600 Subject: Erlang TextMate bundle Message-ID: There was a thread about IDEs where textmate was mentioned. So guys please share your bundles, links. I actually can't find anything with "erlang textmate bundle" keywords. From chad@REDACTED Wed Jul 15 13:41:09 2009 From: chad@REDACTED (Chad DePue) Date: Wed, 15 Jul 2009 08:41:09 -0300 Subject: [erlang-questions] Erlang TextMate bundle In-Reply-To: References: Message-ID: Illo was the one that mentioned writing his own but didn't post them i don't think. i'd be interested as well... On Jul 15, 2009, at 8:11 AM, Alexander Uvarov wrote: > There was a thread about IDEs where textmate was mentioned. So guys > please share your bundles, links. I actually can't find anything > with "erlang textmate bundle" keywords. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From kunthar@REDACTED Wed Jul 15 13:41:09 2009 From: kunthar@REDACTED (Kunthar) Date: Wed, 15 Jul 2009 14:41:09 +0300 Subject: [erlang-questions] Erlang TextMate bundle In-Reply-To: References: Message-ID: <9a09ca9a0907150441w6c23ace5haa3c89287eb4e095@mail.gmail.com> Aware of this? http://code.google.com/p/erlymate/ On Wed, Jul 15, 2009 at 2:11 PM, Alexander Uvarov wrote: > There was a thread about IDEs where textmate was mentioned. So guys please > share your bundles, links. I actually can't find anything with "erlang > textmate bundle" keywords. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From andrewmmc@REDACTED Wed Jul 15 13:47:52 2009 From: andrewmmc@REDACTED (andrew mmc) Date: Wed, 15 Jul 2009 13:47:52 +0200 Subject: [erlang-questions] Erlang TextMate bundle In-Reply-To: References: Message-ID: Erlang textmate bundle how to: http://oneless.blogspot.com/2007/04/howto-erlang-textmate-bundle.html On Wed, Jul 15, 2009 at 1:46 PM, andrew mmc wrote: > Erlang textmate bundle how to: > > http://oneless.blogspot.com/2007/04/howto-erlang-textmate-bundle.html > > > On Wed, Jul 15, 2009 at 1:41 PM, Chad DePue wrote: > >> Illo was the one that mentioned writing his own but didn't post them i >> don't think. i'd be interested as well... >> >> >> On Jul 15, 2009, at 8:11 AM, Alexander Uvarov wrote: >> >> There was a thread about IDEs where textmate was mentioned. So guys >>> please share your bundles, links. I actually can't find anything with >>> "erlang textmate bundle" keywords. >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > From carlmcdade@REDACTED Wed Jul 15 13:58:20 2009 From: carlmcdade@REDACTED (Carl McDade) Date: Wed, 15 Jul 2009 13:58:20 +0200 Subject: [erlang-questions] Cannot find erl.exe on Linux Mint In-Reply-To: <4A5DB2B5.9040406@it.uu.se> References: <4A5D9CBB.7030400@it.uu.se> <4A5DA963.4030502@cs.ntua.gr> <4A5DB2B5.9040406@it.uu.se> Message-ID: Thanks All, That got it going. I am also going to do as suggested an go through the readme on another machine. Thanks again, On Wed, Jul 15, 2009 at 12:43 PM, Richard Carlsson wrote: > Carl McDade wrote: >> Tried . >> >> minty ~ # $PATH = $PATH:~/root/otp_src_R13B01/bin >> bash: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games: >> No such file or directory >> minty ~ # PATH = $PATH:/root/otp_src_R13B01/bin >> >> Without any change. Is it just me? These commands work fine on Xandros >> but not on this Debian distro > > Looks like you're unused to Unix or to shells in general. What happened > was that you gave the left hand side of the assignment as $PATH, instead > of just PATH, so the shell expanded the variable for you, hence the left > hand side of the '=' became "/usr/... ". A second problem was that you > left a space between PATH and the equals sign, so the shell did in fact > not interpret this as an assignment, but as a command (whatever $PATH > expanded to) whose first parameter was "=". > > Writing PATH=$PATH:/root/otp... should do it. > > ? ?/Richard > -- Carl McDade Web developer www.hiveminds.co.uk ________________________ From joelr1@REDACTED Wed Jul 15 14:31:59 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 15 Jul 2009 13:31:59 +0100 Subject: finding processes started by a given function Message-ID: <33E8DA84-FDD9-4C55-908B-F9ADF573F473@gmail.com> I'm looking to find processes started by Mod:init/1, e.g. by gen_fsm or gen_server. Is there a more elegant way of doing this? --- p(Mod) -> p(Mod, processes(), []). p(_, [], Acc) -> Acc; p(Mod, [H|T], Acc) -> {_, L} = process_info(H, dictionary), case lists:keyfind('$initial_call', 1, L) of {'$initial_call', {Mod, init, 1}} -> p(Mod, T, [H|Acc]); _ -> p(Mod, T, Acc) end. --- Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From illo@REDACTED Wed Jul 15 14:31:53 2009 From: illo@REDACTED (Illo de Illis) Date: Wed, 15 Jul 2009 14:31:53 +0200 Subject: [erlang-questions] Erlang TextMate bundle In-Reply-To: References: Message-ID: On Jul 15, 2009, at 1:41 PM, Chad DePue wrote: > Illo was the one that mentioned writing his own but didn't post them > i don't think. i'd be interested as well... Hello there! Actually I didn't write the whole erlang bundle, I just added a bunch of useful templates and a command to find the current version for an installed OTP application (for writing release files). The bundle must be checked out in "~/Library/Application Support/TextMate/Bundles" by using subversion: $ svn co http://svn.textmate.org/trunk/Bundles/Erlang.tmbundle Ciao, Illo. From sgolovan@REDACTED Wed Jul 15 14:36:51 2009 From: sgolovan@REDACTED (Sergei Golovan) Date: Wed, 15 Jul 2009 16:36:51 +0400 Subject: Problems with threaded Erlang on Linux at UltraSparc III Message-ID: Hi! I need help in debugging Erlang on Linux (specifically Debian Linux) on UltraSparc III hardware). So far only non-threaded build works fine, though it means no SMP and no wxErlang. In threads enabled build beam binary crashes often with segmentation faults or bus errors. When I build Erlang with debug info enabled (make TYPE=debug) I get very different debug messages just before crash (or before internal compiler errors): erlc -W +debug_info +warn_obsolete_guard -I/home/sgolovan/debian/otp_src_R13B01/lib/stdlib/include -o../ebin yecc.erl Function: '-read_grammar/2-lc$^0/1-0-'/3 Assertion failed: is_header(hdr) in beam/copy.c, line 95 make[3]: *** [../ebin/yecc.beam] Aborted erlc -W +debug_info +warn_obsolete_guard -I/home/sgolovan/debian/otp_src_R13B01/lib/stdlib/include -o../ebin yecc.erl Function: file/1 size_object: bad tag for 0x0 make[3]: *** [../ebin/yecc.beam] Aborted erlc -W +debug_info +warn_obsolete_guard -I/home/sgolovan/debian/otp_src_R13B01/lib/stdlib/include -o../ebin yecc.erl Function: '-check_rules/1-fun-0-'/2 ./yecc.erl:none: internal error in v3_codegen; crash reason: {{case_clause, {'EXIT', {function_clause, [{gb_trees,lookup_1, [{'-check_rules/1-fun-0-',2},{}]}, {v3_codegen,local_func_label,2}, {v3_codegen,cg_fun,6}, {v3_codegen,function,3}, {lists,mapfoldl,3}, {lists,mapfoldl,3}, {v3_codegen,module,2}, {compile,'-select_passes/2-anonymous-2-',2}]}}}, [{compile,'-select_passes/2-anonymous-2-',2}, {compile,'-internal_comp/4-anonymous-1-',2}, {compile,fold_comp,3}, {compile,internal_comp,4}, {compile,internal,3}]} make[3]: *** [../ebin/yecc.beam] Error 1 erlc -W +debug_info +warn_obsolete_guard -I/home/sgolovan/debian/otp_src_R13B01/lib/stdlib/include -o../ebin yecc.erl make[3]: *** [../ebin/yecc.beam] Bus error erlc -W +debug_info +warn_obsolete_guard -I/home/sgolovan/debian/otp_src_R13B01/lib/stdlib/include -o../ebin yecc.erl Assertion failed: !is_CP(tmp_arg1) in beam/beam_emu.c, line 2678 make[3]: *** [../ebin/yecc.beam] Aborted erlc -W +debug_info +warn_obsolete_guard -I/home/sgolovan/debian/otp_src_R13B01/lib/stdlib/include -o../ebin yecc.erl ERROR: Fence at end of memory block (p=0x1891409952, sz=2056916) clobbered. make[3]: *** [../ebin/yecc.beam] Aborted and many others. I've collected several core dumps but can't find anything useful in them (though I don't have much experience in debugging threaded programs). The experiment was performed on clean Erlang sources without any patches. Core dumps can be found at http://sgolovan.nes.ru/erlang/ I will give you any additional info which I can get if it is needed. Cheers! -- Sergei Golovan From joelr1@REDACTED Wed Jul 15 15:01:16 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 15 Jul 2009 14:01:16 +0100 Subject: save_calls Message-ID: I have process_flag(save_calls, 64) in the init function of my gen_fsm. process_info(Pid, last_calls) returns {last_calls,false}. Is save_calls working or am I doing something wrong? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From mikpe@REDACTED Wed Jul 15 15:56:47 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Wed, 15 Jul 2009 15:56:47 +0200 Subject: [erlang-questions] Problems with threaded Erlang on Linux at UltraSparc III In-Reply-To: References: Message-ID: <19037.57375.880719.203645@pilspetsen.it.uu.se> Sergei Golovan writes: > Hi! > > I need help in debugging Erlang on Linux (specifically Debian Linux) > on UltraSparc III hardware). So far only non-threaded build works > fine, though it means no SMP Try this patch: --- otp-0710/erts/include/internal/ethread.h.~1~ 2009-02-26 11:45:54.000000000 +0100 +++ otp-0710/erts/include/internal/ethread.h 2009-07-15 15:52:01.000000000 +0200 @@ -691,16 +691,12 @@ ETHR_INLINE_FUNC_NAME_(ethr_write_lock)( # include "i386/ethread.h" # elif (defined(__powerpc__) || defined(__ppc__)) && !defined(__powerpc64__) # include "ppc32/ethread.h" -# elif defined(__sparc__) -# include "sparc32/ethread.h" # elif defined(__tile__) # include "tile/ethread.h" # endif # elif ETHR_SIZEOF_PTR == 8 # if defined(__x86_64__) # include "x86_64/ethread.h" -# elif defined(__sparc__) && defined(__arch64__) -# include "sparc64/ethread.h" # endif # endif #endif /* !defined(ETHR_DISABLE_NATIVE_IMPLS) && defined(__GNUC__) */ > and no wxErlang. I consider wxErlang's SMP requirement an inexcusable bug. From sgolovan@REDACTED Wed Jul 15 16:27:07 2009 From: sgolovan@REDACTED (Sergei Golovan) Date: Wed, 15 Jul 2009 18:27:07 +0400 Subject: [erlang-questions] Problems with threaded Erlang on Linux at UltraSparc III In-Reply-To: <19037.57375.880719.203645@pilspetsen.it.uu.se> References: <19037.57375.880719.203645@pilspetsen.it.uu.se> Message-ID: On Wed, Jul 15, 2009 at 5:56 PM, Mikael Pettersson wrote: > > Try this patch: Unfortunately, the symptoms are the same. Cheers! -- Sergei Golovan From praveen.ray@REDACTED Wed Jul 15 16:34:24 2009 From: praveen.ray@REDACTED (Praveen Ray) Date: Wed, 15 Jul 2009 10:34:24 -0400 Subject: [erlang-questions] bzip2 port anyone? In-Reply-To: <4A5AE3F6.8070201@gmail.com> References: <4A5AE3F6.8070201@gmail.com> Message-ID: For such critical 'missing' components, I just use a jinterface based Java Node. On Mon, Jul 13, 2009 at 3:36 AM, Christian Czura wrote: > Hi list, > I've been looking for a module/port to decompress bzip2-compressed data. > My search has been unsuccessful so far. > > As bzip2 compression/decompression isn't THAT uncommon, I figure it's > probable that either somebody has already written such module or I've > missed the obvious and there is already something in the stdlib (yes, > I've looked). > > Thanks, > Chris > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- Yellowfish Technologies Inc http://www.yellowfish.biz praveen.ray@REDACTED (888) 817 2969 x 233 gtalk/skype: praveenray From joelr1@REDACTED Wed Jul 15 16:53:10 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 15 Jul 2009 15:53:10 +0100 Subject: ets traversal time Message-ID: I have a table with 20k entries that I'm traversing like this: Msg1 = {message, iolist_to_binary(mochijson2:encode(JSON))}, F = fun({Pid, _}, _) -> Pid ! Msg1 end, F1 = fun() -> A = now(), ets:foldr(F, ignore, State#state.subs), io:format("time: ~p~n", [timer:now_diff(now(), A) / 1000]) end, spawn_link(F1), It takes me twice as long to traverse the table the second time around. Any suggestions as to why and how to fix this? Thanks, Joel -- Case #1 info: [{memory,283357}, {owner,<0.763.0>}, {heir,none}, {name,subs}, {size,20000}, {node,'janus@REDACTED'}, {named_table,false}, {type,set}, {keypos,1}, {protection,protected}] time: 1050.958ms Case #2 info: [{memory,283357}, {owner,<0.763.0>}, {heir,none}, {name,subs}, {size,20000}, {node,'janus@REDACTED'}, {named_table,false}, {type,set}, {keypos,1}, {protection,protected}] time: 2076.814ms --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From rtrlists@REDACTED Wed Jul 15 19:11:30 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 15 Jul 2009 18:11:30 +0100 Subject: XML-binary Optimized Packaging (XOP) ? In-Reply-To: <6a3ae47e0907150225q7c496ea9ra8ae02891561792c@mail.gmail.com> References: <6a3ae47e0907150225q7c496ea9ra8ae02891561792c@mail.gmail.com> Message-ID: <6a3ae47e0907151011l67d49109kd738d9fed92dbac3@mail.gmail.com> On Wed, Jul 15, 2009 at 10:25 AM, Robert Raschke wrote: > Hi, > > before I embark on cobbling something together that makes sense of XML > munged into this XOP format (http://www.w3.org/TR/xop10/) I thought I'd > ask if anyone out there has already done this in Erlang? > > Thanks for any info, > Robby > > Right, I made myself a hacky un-XOP-er. It's a bit rough and ready, but works for my needs (I think). I'm probably not doing the setting of the #xmlText.type quite right (ie. I don't). This works for me because I don't care what's inside the xop:Include elements. But better would probably be a peek into the content-type of the referenced MIME part and set the type to cdata if needed. Dunno. This thing depends on xmerl and the mimemail module from gen_smtp (a big thanks goes out to Andrew Thompson for writing that). Anyway, if anyone has a need for something similar, here it is. Beware the bugs. Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Wed Jul 15 19:15:02 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 15 Jul 2009 18:15:02 +0100 Subject: [erlang-questions] Re: 20k messages in 4s but want to go faster! In-Reply-To: References: <2BEF4392-2D9F-45E9-964D-EB4301CD5CCB@gmail.com> <97619b170907121138t3c8868f3j5bcc347618f2b00b@mail.gmail.com> <593DD9A7-FE52-4499-873C-664F076C6E62@gmail.com> <97619b170907122322l6d7220abv29852ef0d24568fb@mail.gmail.com> <74ddf71d-3e08-46d1-8ea5-b88bf21f62ef@u16g2000pru.googlegroups.com> <7169FD85-C708-4A42-9FFD-EE76C2FA7E9F@gmail.com> Message-ID: I won't be posting to this thread for a while as I, basically, ran out of ideas. Here are the results with a histogram, as Dave Smith suggested. It's interesting to note that my ets traversal time for 20k takes up to 2s on the Mac. I wrote about this in a separate message. Thanks, Joel --- MacBook Pro Core2Duo 2.93Ghz with SSD setup: 15040.77ms, n: 20000, run: 23101.30ms 1.5010ms | min 500.0000ms | 579 1000.0000ms | 3693 1500.0000ms | 509 2000.0000ms | 1758 2500.0000ms | 3922 3000.0000ms | 2577 3500.0000ms | 2809 4000.0000ms | 1657 4500.0000ms | 2496 4185.8580ms | max setup: 14538.21ms, n: 20000, run: 22750.28ms 2.8300ms | min 500.0000ms | 400 1000.0000ms | 1541 1500.0000ms | 7107 2000.0000ms | 5402 2500.0000ms | 4670 3000.0000ms | 880 2595.8890ms | max setup: 15929.55ms, n: 20000, run: 23789.73ms 1.7730ms | min 500.0000ms | 1330 1000.0000ms | 668 1500.0000ms | 1878 2000.0000ms | 4109 2500.0000ms | 4954 3000.0000ms | 2546 3500.0000ms | 1259 4000.0000ms | 3256 3902.5510ms | max ec2, 10k, 1 instance for bots and 1 for server setup: 15463.12ms, n: 10000, run: 19252.67ms -15.0700ms | min 500.0000ms | 1136 - 11.36% 1000.0000ms | 1934 - 19.34% 1500.0000ms | 2219 - 22.19% 2000.0000ms | 2727 - 27.27% 2500.0000ms | 1984 - 19.84% 2352.5950ms | max setup: 13463.51ms, n: 10000, run: 17207.97ms -15.7610ms | min 500.0000ms | 872 - 08.72% 1000.0000ms | 2146 - 21.46% 1500.0000ms | 2993 - 29.93% 2000.0000ms | 2656 - 26.56% 2500.0000ms | 1333 - 13.33% 2257.9970ms | max setup: 33910.54ms, n: 10000, run: 37634.46ms -16.9060ms | min 500.0000ms | 1122 - 11.22% 1000.0000ms | 2194 - 21.94% 1500.0000ms | 2570 - 25.70% 2000.0000ms | 2369 - 23.69% 2500.0000ms | 1745 - 17.45% 2335.4170ms | max ec2, 20k, 1 instance for bots and 1 for server setup: 47791.69ms, n: 20000, run: 55429.86ms -21.7180ms | min 500.0000ms | 1064 - 05.32% 1000.0000ms | 828 - 04.14% 1500.0000ms | 1596 - 07.98% 2000.0000ms | 2786 - 13.93% 2500.0000ms | 2272 - 11.36% 3000.0000ms | 2574 - 12.87% 3500.0000ms | 2717 - 13.58% 4000.0000ms | 2418 - 12.09% 4500.0000ms | 2594 - 12.97% 5000.0000ms | 1151 - 05.75% 4679.1540ms | max setup: 50666.82ms, n: 20000, run: 58361.03ms -25.4550ms | min 500.0000ms | 1330 - 06.65% 1000.0000ms | 811 - 04.06% 1500.0000ms | 792 - 03.96% 2000.0000ms | 2800 - 14.00% 2500.0000ms | 2484 - 12.42% 3000.0000ms | 2902 - 14.51% 3500.0000ms | 2526 - 12.63% 4000.0000ms | 2861 - 14.31% 4500.0000ms | 2777 - 13.88% 5000.0000ms | 717 - 03.58% 4716.3200ms | max setup: 42524.39ms, n: 20000, run: 50148.11ms 6.6660ms | min 500.0000ms | 1168 - 05.84% 1000.0000ms | 1016 - 05.08% 1500.0000ms | 1546 - 07.73% 2000.0000ms | 2442 - 12.21% 2500.0000ms | 2713 - 13.56% 3000.0000ms | 2684 - 13.42% 3500.0000ms | 2636 - 13.18% 4000.0000ms | 2363 - 11.82% 4500.0000ms | 2248 - 11.24% 5000.0000ms | 1184 - 05.92% 4680.1510ms | max --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Wed Jul 15 19:39:18 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 15 Jul 2009 18:39:18 +0100 Subject: Unicast 20k messages, $500-$1000 bounty Message-ID: Folks, I ran out of ideas on how to optimize my code but my client was kind enough to let me open source a basic version of the code under the MIT and offer a bounty for performance improvements. The rules are very simple... 1) Unicast to 20K clients in < 1s nets you a $1000 bounty. 2) Unicast to 20K clients in < 2s nets you a $500 bounty. 3) Final proof has to be from EC2, one small instance for the server and one for the bots. 4) {packet, 0} 5) TCP Lets keep this thread for technical discussion and use #erlang on irc.freenode.com to discuss. You can also email me with any questions and follow wagerlabs on Twitter. I'll be posting the reference implementation on Github tonight. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From chris.tilt@REDACTED Wed Jul 15 22:20:26 2009 From: chris.tilt@REDACTED (Chris Tilt) Date: Wed, 15 Jul 2009 16:20:26 -0400 Subject: Any interest in an RDS distribution? Message-ID: Hi. I have just written my first Erlang program, which is a port driver for Infiniband (using our com layer). It is a prototype and not written as a distribution replacement. It's just a regular port driver. But, the performance was great. An echo/ping application that uses the port was measured at 20 usec per 1K buffer (one hop - round trip was 40 usec to echo back the data packet), which includes the time in and out of the Erlang port owner's process to the echo server and ping client. I am investigating the possible use of Erlang to implement a reliable system, but we need IB speeds. So, I thought I could write a distribution layer on top of the RDS api of OFED. One of our technical founders is the original designer of RDS (Ranjit Pandit) and we have some expertise in writing IB drivers. Anyone who has tried to write directly on top of OFED knows how hard that is :-( I was going to start with the kernel/examples/uds_dist example and modify it. If anyone has already done this, then great - share if you can. If anyone has written a different distribution, using the port driver mechanism, that uses the EPDM I'd like to talk with you. Although RDS uses datagrams, it's still reliable. I think I want to use the EPDM to discover ports. BTW, the cool thing about RDS is that it uses a single connection between two nodes, no matter how many "connections" exist between those two nodes. For us, this is quite important to conserve end point buffers in clusters that have >500 nodes connected over Infiniband (the hardware runs out of resources otherwise). We would be happy to share this work back to the community (I say this before I've spoken with our lawyer :-) as open source. We'd have to research the license issues, etc, but first things first - let's get it working. Any advice is welcomed. Or just any cheers of support are welcomed too :-) Best regards, Chris Tilt From ulf.wiger@REDACTED Wed Jul 15 22:53:32 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 15 Jul 2009 22:53:32 +0200 Subject: [erlang-questions] Any interest in an RDS distribution? In-Reply-To: References: Message-ID: <4A5E41CC.50904@erlang-consulting.com> Chris Tilt wrote: > Hi. I have just written my first Erlang program, which is a port > driver for Infiniband (using our com layer). It is a prototype and > not written as a distribution replacement. It's just a regular port > driver. But, the performance was great. An echo/ping application that > uses the port was measured at 20 usec per 1K buffer (one hop - round > trip was 40 usec to echo back the data packet), which includes the > time in and out of the Erlang port owner's process to the echo server > and ping client. Sounds great. > We would be happy to share this work back to the community (I say > this before I've spoken with our lawyer :-) as open source. We'd have > to research the license issues, etc, but first things first - let's > get it working. Any advice is welcomed. Or just any cheers of support > are welcomed too :-) You might find it a lot easier to get (free) help if you could clarify that the end product *will* in fact be Open Source. ;-) If you intend to keep the result proprietary and use it for commercial gain, it's only fair that the people who offer qualified assistance should get paid too. I think this sort of thing would benefit greatly from being Open Source. Good luck convincing your lawyer of this. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From joelr1@REDACTED Wed Jul 15 23:01:00 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 15 Jul 2009 22:01:00 +0100 Subject: Unicast 20k messages, $500-$1000 bounty In-Reply-To: References: Message-ID: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> The Ultimate Erlang Challenge is on: http://github.com/tinycode/janus/tree/master Be the first to improve Janus to unicast to 20K clients with a consistent maximum latency of less than 1s and grab a $1000 bounty. Be the first to get the maximum latency consistently under 2s and claim $500. Develop on any hardware but test on Amazon EC2 for final numbers. Here's the current set for 20k clients with the bots running on one small instance and the server on another: setup: 47791.69ms, n: 20000, run: 55429.86ms -21.7180ms | min 500.0000ms | 1064 - 05.32% 1000.0000ms | 828 - 04.14% 1500.0000ms | 1596 - 07.98% 2000.0000ms | 2786 - 13.93% 2500.0000ms | 2272 - 11.36% 3000.0000ms | 2574 - 12.87% 3500.0000ms | 2717 - 13.58% 4000.0000ms | 2418 - 12.09% 4500.0000ms | 2594 - 12.97% 5000.0000ms | 1151 - 05.75% 4679.1540ms | max setup: 50666.82ms, n: 20000, run: 58361.03ms -25.4550ms | min 500.0000ms | 1330 - 06.65% 1000.0000ms | 811 - 04.06% 1500.0000ms | 792 - 03.96% 2000.0000ms | 2800 - 14.00% 2500.0000ms | 2484 - 12.42% 3000.0000ms | 2902 - 14.51% 3500.0000ms | 2526 - 12.63% 4000.0000ms | 2861 - 14.31% 4500.0000ms | 2777 - 13.88% 5000.0000ms | 717 - 03.58% 4716.3200ms | max setup: 42524.39ms, n: 20000, run: 50148.11ms 6.6660ms | min 500.0000ms | 1168 - 05.84% 1000.0000ms | 1016 - 05.08% 1500.0000ms | 1546 - 07.73% 2000.0000ms | 2442 - 12.21% 2500.0000ms | 2713 - 13.56% 3000.0000ms | 2684 - 13.42% 3500.0000ms | 2636 - 13.18% 4000.0000ms | 2363 - 11.82% 4500.0000ms | 2248 - 11.24% 5000.0000ms | 1184 - 05.92% 4680.1510ms | max Here are the numbers from my unibody MacBook Pro 2.93Ghz: setup: 15040.77ms, n: 20000, run: 23101.30ms 1.5010ms | min 500.0000ms | 579 1000.0000ms | 3693 1500.0000ms | 509 2000.0000ms | 1758 2500.0000ms | 3922 3000.0000ms | 2577 3500.0000ms | 2809 4000.0000ms | 1657 4500.0000ms | 2496 4185.8580ms | max setup: 14538.21ms, n: 20000, run: 22750.28ms 2.8300ms | min 500.0000ms | 400 1000.0000ms | 1541 1500.0000ms | 7107 2000.0000ms | 5402 2500.0000ms | 4670 3000.0000ms | 880 2595.8890ms | max setup: 15929.55ms, n: 20000, run: 23789.73ms 1.7730ms | min 500.0000ms | 1330 1000.0000ms | 668 1500.0000ms | 1878 2000.0000ms | 4109 2500.0000ms | 4954 3000.0000ms | 2546 3500.0000ms | 1259 4000.0000ms | 3256 3902.5510ms | max ok --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Wed Jul 15 23:20:58 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 15 Jul 2009 22:20:58 +0100 Subject: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> Message-ID: One last thing... There must be no errors in the console and all bots must run through successfully! On Jul 15, 2009, at 10:01 PM, Joel Reymont wrote: > The Ultimate Erlang Challenge is on: > > http://github.com/tinycode/janus/tree/master > > Be the first to improve Janus to unicast to 20K clients with a > consistent maximum latency of less than 1s and grab a $1000 bounty. > > Be the first to get the maximum latency consistently under 2s and > claim $500. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From wolfmanjm@REDACTED Thu Jul 16 01:11:11 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Wed, 15 Jul 2009 16:11:11 -0700 (PDT) Subject: Unicast 20k messages, $500-$1000 bounty In-Reply-To: References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> Message-ID: <94cc77df-ad9e-4211-875c-1f6843127441@o9g2000prg.googlegroups.com> Well as I have said my server already does this easily, but I have to ask did you try the one process per client that everyone was suggesting? On Jul 15, 2:20?pm, Joel Reymont wrote: > One last thing... There must be no errors in the console and all bots ? > must run through successfully! > > On Jul 15, 2009, at 10:01 PM, Joel Reymont wrote: > > > The Ultimate Erlang Challenge is on: > > >http://github.com/tinycode/janus/tree/master > > > Be the first to improve Janus to unicast to 20K clients with a ? > > consistent maximum latency of less than 1s and grab a $1000 bounty. > > > Be the first to get the maximum latency consistently under 2s and ? > > claim $500. > > --- > Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From joelr1@REDACTED Thu Jul 16 01:15:15 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 00:15:15 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <94cc77df-ad9e-4211-875c-1f6843127441@o9g2000prg.googlegroups.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <94cc77df-ad9e-4211-875c-1f6843127441@o9g2000prg.googlegroups.com> Message-ID: <6F4BAF82-5178-4665-AEF4-226128B55821@gmail.com> Jim, On Jul 16, 2009, at 12:11 AM, Jim Morris wrote: > Well as I have said my server already does this easily, but I have to > ask did you try the one process per client that everyone was > suggesting? Take a look at the code, I have 2 processes per client. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From wolfmanjm@REDACTED Thu Jul 16 02:11:17 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Wed, 15 Jul 2009 17:11:17 -0700 (PDT) Subject: ets traversal time In-Reply-To: References: Message-ID: <6243f848-8040-479e-a0f2-6858c513a9dc@y4g2000prf.googlegroups.com> Have you tried ets:foldl which is tail recursive (presumably as it does not specify in the docs, but it is in lists:foldl) or is there a reason to use foldr? If foldr is not tail recursive then I suspect that with 20K entries the stack will get used up and or eat a lot of memory which needs to be allocated and released, maybe even causing swapping to the disk. On Jul 15, 7:53?am, Joel Reymont wrote: > I have a table with 20k entries that I'm traversing like this: > > ? ? ?Msg1 = {message, iolist_to_binary(mochijson2:encode(JSON))}, > ? ? ?F = fun({Pid, _}, _) -> Pid ! Msg1 end, > ? ? ?F1 = fun() -> > ? ? ? ? ? ? ? ? ? A = now(), > ? ? ? ? ? ? ? ? ? ets:foldr(F, ignore, State#state.subs), > ? ? ? ? ? ? ? ? ? io:format("time: ~p~n", [timer:now_diff(now(), A) / ? > 1000]) > ? ? ? ? ? end, > ? ? ?spawn_link(F1), > > It takes me twice as long to traverse the table the second time around. > > Any suggestions as to why and how to fix this? > > ? ? ? ? Thanks, Joel > > -- > Case #1 > > info: [{memory,283357}, > ? ? ? ? {owner,<0.763.0>}, > ? ? ? ? {heir,none}, > ? ? ? ? {name,subs}, > ? ? ? ? {size,20000}, > ? ? ? ? {node,'ja...@REDACTED'}, > ? ? ? ? {named_table,false}, > ? ? ? ? {type,set}, > ? ? ? ? {keypos,1}, > ? ? ? ? {protection,protected}] > time: 1050.958ms > > Case #2 > info: [{memory,283357}, > ? ? ? ? {owner,<0.763.0>}, > ? ? ? ? {heir,none}, > ? ? ? ? {name,subs}, > ? ? ? ? {size,20000}, > ? ? ? ? {node,'ja...@REDACTED'}, > ? ? ? ? {named_table,false}, > ? ? ? ? {type,set}, > ? ? ? ? {keypos,1}, > ? ? ? ? {protection,protected}] > time: 2076.814ms > > --- > Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From rtrlists@REDACTED Thu Jul 16 10:03:24 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 16 Jul 2009 09:03:24 +0100 Subject: [erlang-questions] Re: XML-binary Optimized Packaging (XOP) ? In-Reply-To: <14f0e3620907151038v6a1c3037j7e05d14040040759@mail.gmail.com> References: <6a3ae47e0907150225q7c496ea9ra8ae02891561792c@mail.gmail.com> <6a3ae47e0907151011l67d49109kd738d9fed92dbac3@mail.gmail.com> <14f0e3620907151038v6a1c3037j7e05d14040040759@mail.gmail.com> Message-ID: <6a3ae47e0907160103g3b8a0d25x2854f5cd7fd0e7ef@mail.gmail.com> I did attach the file on the email, weird that it didn't show up for you. It's now also available here: http://www.tombob.com/unxop.erl Robby On Wed, Jul 15, 2009 at 6:38 PM, Gleb Peregud wrote: > Hi Robert. > > Looks like you've forgot to attach a file / paste a link :) > > Best, > Gleb > > On Wed, Jul 15, 2009 at 19:11, Robert Raschke > wrote: > > > > On Wed, Jul 15, 2009 at 10:25 AM, Robert Raschke < > rtrlists@REDACTED> > > wrote: > >> > >> Hi, > >> > >> before I embark on cobbling something together that makes sense of XML > >> munged into this XOP format (http://www.w3.org/TR/xop10/) I thought I'd > ask > >> if anyone out there has already done this in Erlang? > >> > >> Thanks for any info, > >> Robby > >> > > > > Right, I made myself a hacky un-XOP-er. It's a bit rough and ready, but > > works for my needs (I think). > > > > I'm probably not doing the setting of the #xmlText.type quite right (ie. > I > > don't). This works for me because I don't care what's inside the > xop:Include > > elements. But better would probably be a peek into the content-type of > the > > referenced MIME part and set the type to cdata if needed. Dunno. > > > > This thing depends on xmerl and the mimemail module from gen_smtp (a big > > thanks goes out to Andrew Thompson for writing that). > > > > Anyway, if anyone has a need for something similar, here it is. Beware > the > > bugs. > > > > Robby > > > > > From joelr1@REDACTED Thu Jul 16 12:09:06 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 11:09:06 +0100 Subject: [erlang-questions] Re: ets traversal time In-Reply-To: <6243f848-8040-479e-a0f2-6858c513a9dc@y4g2000prf.googlegroups.com> References: <6243f848-8040-479e-a0f2-6858c513a9dc@y4g2000prf.googlegroups.com> Message-ID: <29A91B70-E907-476E-B785-783E3555C9DB@gmail.com> On Jul 16, 2009, at 1:11 AM, Jim Morris wrote: > Have you tried ets:foldl which is tail recursive (presumably as it > does not specify in the docs, but it is in lists:foldl) or is there a > reason to use foldr? I'm not seeing much of a different with ets:foldl info: [{memory,283357}, {owner,<0.2316.0>}, {heir,none}, {name,subs}, {size,20000}, {node,'janus@REDACTED'}, {named_table,false}, {type,set}, {keypos,1}, {protection,protected}] time: 957.305 info: [{memory,283357}, {owner,<0.2316.0>}, {heir,none}, {name,subs}, {size,20000}, {node,'janus@REDACTED'}, {named_table,false}, {type,set}, {keypos,1}, {protection,protected}] time: 1014.135 Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Thu Jul 16 12:29:22 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 11:29:22 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <94cc77df-ad9e-4211-875c-1f6843127441@o9g2000prg.googlegroups.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <94cc77df-ad9e-4211-875c-1f6843127441@o9g2000prg.googlegroups.com> Message-ID: <1475114C-CAB8-4A4B-B259-5AC1F7764F0E@gmail.com> Some notes on the architecture... janus_acceptor spawns transport which initializes janus_flash. janus_flash:start sends a token to the client. janus_flash will also initialize a client_proxy. Think of client_proxy as the session process. Upon receipt of the token, client will try to subscribe and wait for subscription confirmation. Client will start a 5s timer once subscription is confirmed by the server. Client expects to receive its packet before the timer expires. topman maps subscription topics to pubsub instances keeping track of subscribers. pubsub keeps client_proxy (subscriber) pids in ETS and monitors subscribers to automatically unsubscribe if they die. pubsub will traverse the client_proxy pids in ETS once a publish request is received and gen_server:cast the message to each client_proxy. Message will be sent by client_proxy to the transport instance connected to the socket and that will send the message to the socket itself. bot will launch as many instances of the flashbot client as required. It then waits for bots to finish and calculates statistics. launcher takes care of starting bots on the node where it's running. The client load can actually be spread across multiple instances. Simply connect the nodes and invoke launcher:start(true) on each one. Try pg2:get_members(launcher) on the master bot node to make sure you have multiple bot launcher nodes. flashbot connects, subscribes and waits for its message. Note that it does not matter how long the setup time takes, the critical path is in pubsub:publish and, perhaps, in the parsing and handling of messages by flashbot. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From ttmrichter@REDACTED Thu Jul 16 12:45:55 2009 From: ttmrichter@REDACTED (ttmrichter@REDACTED) Date: Thu, 16 Jul 2009 10:45:55 +0000 Subject: [erlang-questions] Re: ets traversal time In-Reply-To: <29A91B70-E907-476E-B785-783E3555C9DB@gmail.com> Message-ID: <0016364582cc576a71046ed06001@google.com> > I'm not seeing much of a different with ets:foldl > time: 957.305 > time: 1014.135 >>> time: 1050.958ms >>> time: 2076.814ms I am. The first one is taking a little under 10% less time and the second one over 50% less time over the original foldr version. This isn't much of a difference? From joelr1@REDACTED Thu Jul 16 13:03:08 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 12:03:08 +0100 Subject: [erlang-questions] Re: ets traversal time In-Reply-To: <0016364582cc576a71046ed06001@google.com> References: <0016364582cc576a71046ed06001@google.com> Message-ID: On Jul 16, 2009, at 11:45 AM, ttmrichter@REDACTED wrote: > I am. The first one is taking a little under 10% less time and the > second one over 50% less time over the original foldr version. This > isn't much of a difference? You are right. I was hoping to see a drastic difference, though. That original 2s is more of an outlier. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Thu Jul 16 13:48:39 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 12:48:39 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> Message-ID: <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> I switched to ets:foldl as suggested by Jim Morris and switched to calculating timestamps for each message, as per the patch sent by Will Glozer. Here are the latest timings on my MBP 2.93Ghz. The maximum latency seems to be 2.1-2.2s, not sure what to make of the 1.5s. On to testing on EC2... Thanks, Joel --- (debug@REDACTED)1> bot:test(flashbot, 20000). =INFO REPORT==== 16-Jul-2009::12:24:08 === setup: 161030.13ms, good: 20000, bad: 0, run: 168624.54ms 0.9540ms | min 500.0000ms | 4858 - 24.29% 1000.0000ms | 1547 - 7.74% 1500.0000ms | 2551 - 12.75% 2000.0000ms | 8044 - 40.22% 2500.0000ms | 3000 - 15.00% 2181.8090ms | max ok (debug@REDACTED)3> bot:test(flashbot, 20000). =INFO REPORT==== 16-Jul-2009::12:36:02 === setup: 14496.73ms, good: 20000, bad: 0, run: 22286.76ms 0.5900ms | min 500.0000ms | 2810 - 14.05% 1000.0000ms | 1228 - 6.14% 1500.0000ms | 2470 - 12.35% 2000.0000ms | 10811 - 54.05% 2500.0000ms | 2681 - 13.41% 2258.7140ms | max ok (debug@REDACTED)4> bot:test(flashbot, 20000). =INFO REPORT==== 16-Jul-2009::12:43:19 === setup: 14338.41ms, good: 20000, bad: 0, run: 22433.91ms 0.5340ms | min 500.0000ms | 9485 - 47.42% 1000.0000ms | 6033 - 30.16% 1500.0000ms | 4482 - 22.41% 1483.6170ms | max ok (debug@REDACTED)5> bot:test(flashbot, 20000). =INFO REPORT==== 16-Jul-2009::12:45:08 === setup: 14256.80ms, good: 20000, bad: 0, run: 22089.76ms 3.6720ms | min 500.0000ms | 5368 - 26.84% 1000.0000ms | 874 - 4.37% 1500.0000ms | 2688 - 13.44% 2000.0000ms | 9781 - 48.91% 2500.0000ms | 1289 - 6.44% 2210.5790ms | max ok --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Thu Jul 16 14:10:31 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 13:10:31 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> Message-ID: <09D52C59-DBD6-4616-AA86-530D327211A0@gmail.com> EC2 run, one instance for the server and another one for bots. ~3s mostly. (debug@REDACTED)2> bot:test(flashbot, 20000, 'ip-10-244-47-97', 8081). =INFO REPORT==== 16-Jul-2009::12:05:07 === setup: 30900.04ms, good: 20000, bad: 0, run: 38579.24ms -38.6400ms | min 500.0000ms | 2650 - 13.25% 1000.0000ms | 1972 - 9.86% 1500.0000ms | 2234 - 11.17% 2000.0000ms | 4611 - 23.05% 2500.0000ms | 4518 - 22.59% 3000.0000ms | 4015 - 20.08% 2963.5520ms | max ok (debug@REDACTED)3> bot:test(flashbot, 20000, 'ip-10-244-47-97', 8081). =INFO REPORT==== 16-Jul-2009::12:06:29 === setup: 55302.13ms, good: 20000, bad: 0, run: 63033.07ms -41.0070ms | min 500.0000ms | 2836 - 14.18% 1000.0000ms | 1733 - 8.67% 1500.0000ms | 2275 - 11.38% 2000.0000ms | 4142 - 20.71% 2500.0000ms | 4555 - 22.78% 3000.0000ms | 4239 - 21.20% 3500.0000ms | 220 - 1.10% 3075.1930ms | max ok (debug@REDACTED)4> bot:test(flashbot, 20000, 'ip-10-244-47-97', 8081). =INFO REPORT==== 16-Jul-2009::12:07:51 === setup: 52532.12ms, good: 20000, bad: 0, run: 60196.70ms -43.1910ms | min 500.0000ms | 2330 - 11.65% 1000.0000ms | 2205 - 11.03% 1500.0000ms | 2158 - 10.79% 2000.0000ms | 3742 - 18.71% 2500.0000ms | 4371 - 21.86% 3000.0000ms | 4630 - 23.15% 3500.0000ms | 564 - 2.82% 3059.0860ms | max ok --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Thu Jul 16 15:13:47 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 14:13:47 +0100 Subject: easier tracing Message-ID: <23E40DCF-75CD-4174-AB7B-89415530CFAC@gmail.com> I have some nifty tracing shortcuts as part of Janus [1]. I hope someone will find them of help. t:t(lists), t:t([lists, gb_trees]), t:t({lists,reverse}), t:t([{lists,reverse}, gb_trees]) or any combination thereof will enable tracing for you. t:ut() will disable it. Thanks, Joel [1] http://is.gd/1AZGA --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From psa@REDACTED Thu Jul 16 15:57:26 2009 From: psa@REDACTED (=?ISO-8859-1?Q?Paulo_S=E9rgio_Almeida?=) Date: Thu, 16 Jul 2009 14:57:26 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> Message-ID: <4A5F31C6.7010903@di.uminho.pt> Hi Joel, Joel Reymont wrote: > I switched to ets:foldl as suggested by Jim Morris and switched to > calculating timestamps for each message, as per the patch sent by Will > Glozer. ets:foldr or foldl is slow for what you need. You don't need to accumulate or to look at the Ref, and the function passed is trivial. For what you need you should: - obtain the list of pids with ets:select, as in: ets:select(State#state.subs, [{{'$1', '_'}, [], ['$1']}]). - iterate the list in a tight loop as in: send([], _M) -> ok; send([H|T], M) -> H ! M, send(T, M). I haven't measured in the case, but on a similar problem using select and traversing the list made things faster. But another source of inefficiency is using 2 middleman procs (as opposed to just 1) between the pubsub and the socket. Regards, Paulo From joelr1@REDACTED Thu Jul 16 16:01:49 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 15:01:49 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F31C6.7010903@di.uminho.pt> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> Message-ID: <2FF749A5-D63A-4327-8021-AE5FD4C9CFD8@gmail.com> On Jul 16, 2009, at 2:57 PM, Paulo S?rgio Almeida wrote: > But another source of inefficiency is using 2 middleman procs (as > opposed to just 1) between the pubsub and the socket. We had a discussion about this in an earlier thread. I used to store a closure that captured a socket and invoke that in a loop. The overwhelming consensus was to use a middleman process :-). I guess I'll try the closure approach again. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From baryluk@REDACTED Thu Jul 16 16:02:25 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Thu, 16 Jul 2009 16:02:25 +0200 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F31C6.7010903@di.uminho.pt> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> Message-ID: <1247752945.5568.16.camel@sredniczarny.smp.if.uj.edu.pl> Dnia 2009-07-16, czw o godzinie 14:57 +0100, Paulo S?rgio Almeida pisze: > Hi Joel, > - iterate the list in a tight loop as in: > > send([], _M) -> ok; > send([H|T], M) -> H ! M, send(T, M). List comprehension can be faster: send(L, M) -> [ H ! M || H <- L], ok. -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From joelr1@REDACTED Thu Jul 16 16:08:21 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 15:08:21 +0100 Subject: trace patterns with fprof Message-ID: Is there a way to use trace patterns with fprof to specify a list of {module,function,arity} that I want to include in or exclude from profiling? I do this now to exclude gen_server, gen_fsm, proc_lib, etc. fprof:trace([start, {procs, all}, file]), erlang:trace_pattern(...) It doesn't work, I think, since I see warnings when profiling: fprof:profile([{dump, []}]). Reading trace data... Warning: {erlang,trace_pattern,3} called in "<0.80.0>" - trace may become corrupt! Warning: {erlang,trace_pattern,3} called in "<0.80.0>" - trace may become corrupt! Warning: {erlang,trace_pattern,3} called in "<0.80.0>" - trace may become corrupt! .................................................. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Thu Jul 16 16:23:57 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 15:23:57 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F31C6.7010903@di.uminho.pt> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> Message-ID: On Jul 16, 2009, at 2:57 PM, Paulo S?rgio Almeida wrote: > - obtain the list of pids with ets:select, as in: > > ets:select(State#state.subs, [{{'$1', '_'}, [], ['$1']}]). Would it not be faster to call ets:tab2list? I would think so, intuitively, since on select work needs to be done. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From psa@REDACTED Thu Jul 16 16:44:32 2009 From: psa@REDACTED (=?UTF-8?B?UGF1bG8gU8OpcmdpbyBBbG1laWRh?=) Date: Thu, 16 Jul 2009 15:44:32 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <1247752945.5568.16.camel@sredniczarny.smp.if.uj.edu.pl> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <1247752945.5568.16.camel@sredniczarny.smp.if.uj.edu.pl> Message-ID: <4A5F3CD0.1020803@di.uminho.pt> Witold Baryluk wrote: > Dnia 2009-07-16, czw o godzinie 14:57 +0100, Paulo S?rgio Almeida pisze: >> Hi Joel, > >> - iterate the list in a tight loop as in: >> >> send([], _M) -> ok; >> send([H|T], M) -> H ! M, send(T, M). > > List comprehension can be faster: > > send(L, M) -> > [ H ! M || H <- L], > ok. > Or it can be slower. It varies. :) They are basically the same if the comprehension optimizes away building a result list, as when using this "ok" here after it. Paulo From joelr1@REDACTED Thu Jul 16 16:49:01 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 15:49:01 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F3CD0.1020803@di.uminho.pt> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <1247752945.5568.16.camel@sredniczarny.smp.if.uj.edu.pl> <4A5F3CD0.1020803@di.uminho.pt> Message-ID: <1F878827-98CF-479D-900E-B607CB227E68@gmail.com> On Jul 16, 2009, at 3:44 PM, Paulo S?rgio Almeida wrote: > Or it can be slower. It varies. :) > They are basically the same if the comprehension optimizes away > building a result list, as when using this "ok" here after it. Also, instead of duplicating the table data by building a list, why not use ets:first and ets:next? --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From psa@REDACTED Thu Jul 16 16:56:03 2009 From: psa@REDACTED (=?ISO-8859-1?Q?Paulo_S=E9rgio_Almeida?=) Date: Thu, 16 Jul 2009 15:56:03 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> Message-ID: <4A5F3F83.3090906@di.uminho.pt> Joel Reymont wrote: > > On Jul 16, 2009, at 2:57 PM, Paulo S?rgio Almeida wrote: > >> - obtain the list of pids with ets:select, as in: >> >> ets:select(State#state.subs, [{{'$1', '_'}, [], ['$1']}]). > > Would it not be faster to call ets:tab2list? I have a weak memory, but I have the feeling select is faster in this case when you don't need all the content in the table. It is a built-in, it is producing less results (just a list of Pids, not of tuples), and then you don't have to extract the Pid from the tuples when traversing the list. Paulo From ulf.wiger@REDACTED Thu Jul 16 17:01:08 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 16 Jul 2009 17:01:08 +0200 Subject: [erlang-questions] Re: ets traversal time In-Reply-To: <6243f848-8040-479e-a0f2-6858c513a9dc@y4g2000prf.googlegroups.com> References: <6243f848-8040-479e-a0f2-6858c513a9dc@y4g2000prf.googlegroups.com> Message-ID: <4A5F40B4.3080106@erlang-consulting.com> Jim Morris wrote: > Have you tried ets:foldl which is tail recursive (presumably as it > does not specify in the docs, but it is in lists:foldl) or is there a > reason to use foldr? > > If foldr is not tail recursive then I suspect that with 20K entries > the stack will get used up and or eat a lot of memory which needs to > be allocated and released, maybe even causing swapping to the disk. The only difference between ets:foldl and ets:foldr is that foldl uses first/next and foldr uses last/prev. Apart from that, they are exactly the same. See ets.erl If the ets table is a 'set', last() -> first() and prev() -> next(), since there is no defined ordering in hash tables. See erl_db_hash.c. If there is a difference in the timings, I would first try to rule out measurement error or see if it falls within the margin of error for the measurement. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From psa@REDACTED Thu Jul 16 17:35:02 2009 From: psa@REDACTED (=?ISO-8859-1?Q?Paulo_S=E9rgio_Almeida?=) Date: Thu, 16 Jul 2009 16:35:02 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <1F878827-98CF-479D-900E-B607CB227E68@gmail.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <1247752945.5568.16.camel@sredniczarny.smp.if.uj.edu.pl> <4A5F3CD0.1020803@di.uminho.pt> <1F878827-98CF-479D-900E-B607CB227E68@gmail.com> Message-ID: <4A5F48A6.1060109@di.uminho.pt> Joel Reymont wrote: > > Also, instead of duplicating the table data by building a list, why not > use ets:first and ets:next? To be sure one has to measure. But my intuition is that select followed by list traversal avoids a ping-pong between built-in and user code. It will use a built-in function (C) to operate in bulk over the table, extracting only what is needed. Then you just traverse the list; and Erlang is very good operating on lists. Also the list produced should be quite compact, with good memory locality. <- Is this true? Also, the negative impact due to GC should be later, after the messages have been sent. But in this case not even GC is a problem as the process exits after sending the msgs. Btw, now I noticed a pontential problem. I don't know what deliveery semantics you aim for, but now you don't have total order in message delivery even for the same topic. As the sending is in a spawned proc, two clients can receive two msgs A and B from the same topic one in the order A -> B and the other B -> A. As you already have middleman, why not do the sending without spawning a proc? Regards, Paulo From joelr1@REDACTED Thu Jul 16 17:35:55 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 16:35:55 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F3F83.3090906@di.uminho.pt> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> Message-ID: <67B20F34-372B-42FC-954D-7DCDBD5B77F1@gmail.com> I'd like to introduce another useful constraint into the equation. You must grab the timestamp _before_ you start broadcasting since what we are trying to measure is delivery time from the moment the message is published. The time it takes for the message to travel through the server and into pubsub:publish is rather small and can be ignored but resetting the timestamp before sending the message to each client both skews the timings and does not measure "time from publishing". With that in mind, here's my latest set of Mac timings: 2s, 2.2s and 2.3s for 20k messages, with the starting time captured before broadcasting [1]. I traded the time taken to broadcast to a list over the time and effort taken to subscribe and unsubscribe. Setup time is much longer now since inserting, deleting and searching a tuple list of 20k items is garbage collection galore. Traversal time of 617.352ms, 645.084ms, 871.228ms and 697.16ms beats anything we have seen from ETS so far and I'm only looking at a maximum of 20k subscribers per server. I think it's better to build the list over time than to build it from ets at broadcasting time. Thanks, Joel [1] http://github.com/tinycode/janus/commit/19c3cc954fdbc5ecb7c950510a80f56423ab9ec9 --- (debug@REDACTED)3> bot:test(flashbot, 20000). =INFO REPORT==== 16-Jul-2009::16:10:26 === setup: 154206.92ms, good: 20000, bad: 0, run: 157926.75ms 2.4990ms | min 500.0000ms | 2359 - 11.79% 1000.0000ms | 6323 - 31.61% 1500.0000ms | 4934 - 24.67% 2000.0000ms | 5769 - 28.84% 2500.0000ms | 615 - 3.08% 2078.7770ms | max ok (debug@REDACTED)4> bot:test(flashbot, 20000). =INFO REPORT==== 16-Jul-2009::16:16:36 === setup: 150305.92ms, good: 20000, bad: 0, run: 154186.14ms 2.1070ms | min 500.0000ms | 2235 - 11.18% 1000.0000ms | 4840 - 24.20% 1500.0000ms | 5497 - 27.48% 2000.0000ms | 5283 - 26.41% 2500.0000ms | 2145 - 10.72% 2185.3810ms | max ok (debug@REDACTED)5> bot:test(flashbot, 20000). =INFO REPORT==== 16-Jul-2009::16:22:07 === setup: 201656.95ms, good: 20000, bad: 0, run: 205959.23ms 2.5690ms | min 500.0000ms | 2573 - 12.86% 1000.0000ms | 2704 - 13.52% 1500.0000ms | 5835 - 29.18% 2000.0000ms | 5527 - 27.63% 2500.0000ms | 3361 - 16.80% 2371.7610ms | max ok (debug@REDACTED)6> bot:test(flashbot, 20000). =INFO REPORT==== 16-Jul-2009::16:29:57 === setup: 153129.67ms, good: 20000, bad: 0, run: 157174.14ms 3.4600ms | min 500.0000ms | 2682 - 13.41% 1000.0000ms | 3922 - 19.61% 1500.0000ms | 5523 - 27.62% 2000.0000ms | 4728 - 23.64% 2500.0000ms | 3145 - 15.72% 2295.7770ms | max ok --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Thu Jul 16 17:47:23 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 16:47:23 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F48A6.1060109@di.uminho.pt> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <1247752945.5568.16.camel@sredniczarny.smp.if.uj.edu.pl> <4A5F3CD0.1020803@di.uminho.pt> <1F878827-98CF-479D-900E-B607CB227E68@gmail.com> <4A5F48A6.1060109@di.uminho.pt> Message-ID: On Jul 16, 2009, at 4:35 PM, Paulo S?rgio Almeida wrote: > I don't know what deliveery semantics you aim for, but now you don't > have total order in message delivery even for the same topic. As the > sending is in a spawned proc, two clients can receive two msgs A and > B from the same topic one in the order A -> B and the other B -> A. > As you already have middleman, why not do the sending without > spawning a proc? You are right, I'll fix this! --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From masse@REDACTED Thu Jul 16 17:54:58 2009 From: masse@REDACTED (mats cronqvist) Date: Thu, 16 Jul 2009 17:54:58 +0200 Subject: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F3F83.3090906@di.uminho.pt> ("Paulo =?iso-8859-1?Q?S=E9rg?= =?iso-8859-1?Q?io?= Almeida"'s message of "Thu\, 16 Jul 2009 15\:56\:03 +0100") References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> Message-ID: <87skgwfwrx.fsf@sterlett.hq.kred> Paulo S?rgio Almeida writes: > Joel Reymont wrote: >> >> On Jul 16, 2009, at 2:57 PM, Paulo S?rgio Almeida wrote: >> >>> - obtain the list of pids with ets:select, as in: >>> >>> ets:select(State#state.subs, [{{'$1', '_'}, [], ['$1']}]). >> >> Would it not be faster to call ets:tab2list? > > I have a weak memory, but I have the feeling select is faster in this > case when you don't need all the content in the table. It is a > built-in, it is producing less results (just a list of Pids, not of > tuples), and then you don't have to extract the Pid from the tuples > when traversing the list. +1 mats From joelr1@REDACTED Thu Jul 16 17:56:13 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 16:56:13 +0100 Subject: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <87skgwfwrx.fsf@sterlett.hq.kred> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <87skgwfwrx.fsf@sterlett.hq.kred> Message-ID: <16B0F282-1A92-464E-A4D5-7520B8475B5F@gmail.com> On Jul 16, 2009, at 4:54 PM, mats cronqvist wrote: >> I have a weak memory, but I have the feeling select is faster in this >> case when you don't need all the content in the table. It is a >> built-in, it is producing less results (just a list of Pids, not of >> tuples), and then you don't have to extract the Pid from the tuples >> when traversing the list. > > +1 How it would compare to building up a list over time? --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Thu Jul 16 18:09:04 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 17:09:04 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F3F83.3090906@di.uminho.pt> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> Message-ID: <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> On Jul 16, 2009, at 3:56 PM, Paulo S?rgio Almeida wrote: > I have a weak memory, but I have the feeling select is faster in > this case when you don't need all the content in the table. It is a > built-in, it is producing less results (just a list of Pids, not of > tuples), and then you don't have to extract the Pid from the tuples > when traversing the list. There's nothing like empirical proof. Traversal time with ets:select is 1046.224ms and 1009.256ms respectively. Compare to my previous timings. Trashing memory while building the list of subscribers over time beats other techniques thus far, by a large margin since we are looking at 2.9s and 3s here. =INFO REPORT==== 16-Jul-2009::17:05:17 === setup: 15100.95ms, good: 20000, bad: 0, run: 22863.07ms 5.3150ms | min 500.0000ms | 432 - 2.16% 1000.0000ms | 1230 - 6.15% 1500.0000ms | 6055 - 30.28% 2000.0000ms | 4970 - 24.85% 2500.0000ms | 4417 - 22.09% 3000.0000ms | 2896 - 14.48% 2923.3990ms | max ok (debug@REDACTED)3> bot:test(flashbot, 20000). =INFO REPORT==== 16-Jul-2009::17:05:47 === setup: 14442.21ms, good: 20000, bad: 0, run: 21925.20ms 6.3430ms | min 500.0000ms | 723 - 3.62% 1000.0000ms | 3308 - 16.54% 1500.0000ms | 3442 - 17.21% 2000.0000ms | 4217 - 21.09% 2500.0000ms | 1943 - 9.71% 3000.0000ms | 3084 - 15.42% 3500.0000ms | 1130 - 5.65% 4000.0000ms | 2153 - 10.76% 3711.4690ms | max ok --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From psa@REDACTED Thu Jul 16 18:27:34 2009 From: psa@REDACTED (=?ISO-8859-1?Q?Paulo_S=E9rgio_Almeida?=) Date: Thu, 16 Jul 2009 17:27:34 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <67B20F34-372B-42FC-954D-7DCDBD5B77F1@gmail.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <67B20F34-372B-42FC-954D-7DCDBD5B77F1@gmail.com> Message-ID: <4A5F54F6.8090900@di.uminho.pt> Joel Reymont wrote: > I traded the time taken to broadcast to a list over the time and effort > taken to subscribe and unsubscribe. Setup time is much longer now since > inserting, deleting and searching a tuple list of 20k items is garbage > collection galore. Traversal time of 617.352ms, 645.084ms, 871.228ms and > 697.16ms beats anything we have seen from ETS so far and I'm only > looking at a maximum of 20k subscribers per server. > > I think it's better to build the list over time than to build it from > ets at broadcasting time. Have you already tried using select? In my very slow very old Dell X1 with a 1.0GHz ULV processor, extracting the list of keys from a 20k entry ets table takes about 5 - 8 ms. I tried extracting using ets:foldl and was shocked. It was even much slower than I expected: it varies between 30 - 80 ms. I think you don't need to compromise and build the list before hand. If 8ms is not good enough. The solution to this case is ... the process dictionary (I don't have anymore pacience to hear how a sacrilege it is to use it; some other day I could digress about Erlang philosophy, encapsulation and mutable state, ...) For now I will just say that: - If I have 100000 processes not each of them is a gen_*. I use gen_* sparingly, for the main services. - Sometimes, for processes that provide a simple service, with little code, the process dictionary can be the right solution to store exactly this kind of information that would be otherwise stored in slower global ets tables, or even slower immutable containers (like dict). In this case, in my slow laptop, extracting the entries of a 20k process dictionary takes about 2 - 3 ms. Regards, Paulo From joelr1@REDACTED Thu Jul 16 18:35:33 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 17:35:33 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F54F6.8090900@di.uminho.pt> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <67B20F34-372B-42FC-954D-7DCDBD5B77F1@gmail.com> <4A5F54F6.8090900@di.uminho.pt> Message-ID: <010EF703-6F46-402A-8CFD-CE19306FDB38@gmail.com> On Jul 16, 2009, at 5:27 PM, Paulo S?rgio Almeida wrote: > Have you already tried using select? In my very slow very old Dell > X1 with a 1.0GHz ULV processor, extracting the list of keys from a > 20k entry ets table takes about 5 - 8 ms. I tried. See my previous email. 2.9-3s whereas I can consistently get 2.2s with a list. > the process dictionary How? > In this case, in my slow laptop, extracting the entries of a 20k > process dictionary takes about 2 - 3 ms. Are you suggesting get() to get all the key/value pairs? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From psa@REDACTED Thu Jul 16 18:36:26 2009 From: psa@REDACTED (=?ISO-8859-1?Q?Paulo_S=E9rgio_Almeida?=) Date: Thu, 16 Jul 2009 17:36:26 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> Message-ID: <4A5F570A.9070302@di.uminho.pt> Joel Reymont wrote: > There's nothing like empirical proof. Traversal time with ets:select is > 1046.224ms and 1009.256ms respectively. Compare to my previous timings. What??? For 20k entries? Just traversal time (without sending msgs)? There is something wrong. It cannot be this slow. This would be 50 us per entry. Paulo From joelr1@REDACTED Thu Jul 16 18:37:24 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 17:37:24 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F570A.9070302@di.uminho.pt> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> Message-ID: On Jul 16, 2009, at 5:36 PM, Paulo S?rgio Almeida wrote: > What??? For 20k entries? Just traversal time (without sending msgs)? > There is something wrong. It cannot be this slow. This would be 50 > us per entry. Apologies, I meant time taken to extract the list with ets:select and traverse the list while broadcasting. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From coa@REDACTED Thu Jul 16 19:15:42 2009 From: coa@REDACTED (=?ISO-8859-1?Q?Cl=E1udio_Amaral?=) Date: Thu, 16 Jul 2009 18:15:42 +0100 Subject: Compiler description Message-ID: <4A5F603E.8070707@dcc.fc.up.pt> Hi! I am starting to work with erlang and my objective is to develop a tool for formal verification of programs. In order to make it easy to integrate in the erlang distribution (if it matures enough) I will try to define my application embedded in the compilation process as an optional step involving the abstract representation. My question is: can anyone point me some document where I can easily learn how things work (with some detail) or the only way is to open the sources and hope find the right place to code? Regards and thanks for the time, Cl?udio Amaral. From joelr1@REDACTED Thu Jul 16 19:17:59 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 18:17:59 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F5B8A.3050004@di.uminho.pt> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <67B20F34-372B-42FC-954D-7DCDBD5B77F1@gmail.com> <4A5F54F6.8090900@di.uminho.pt> <010EF703-6F46-402A-8CFD-CE19306FDB38@gmail.com> <4A5F5B8A.3050004@di.uminho.pt> Message-ID: An interesting bit of info... This is how long it takes to broadcast (traverse + send) to bots running on 1 ec2 instance, in addition to the server instance info: 20000 time: 1168.112 info: 20000 time: 1230.546 info: 20000 time: 1095.887 And this is how long it takes to do the same for 20k bots spread over 3 instances: info: 20000 time: 5387.765 info: 20000 time: 5380.398 Could there be a Linux networking optimization of sorts for when all packets are going to a single server? This is Ubuntu 9.04. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From cliff@REDACTED Thu Jul 16 19:18:35 2009 From: cliff@REDACTED (Cliff Moon) Date: Thu, 16 Jul 2009 10:18:35 -0700 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> Message-ID: <4A5F60EB.3060609@moonpolysoft.com> The problem you're seeing there is process scheduling overhead. In the fun it sends a message and it's doing an ungodly amount of context switching. The way to get around this is to bump the process priority level of the process which iterates the ets table. On that node, I'd like to share my latest findings: =INFO REPORT==== 16-Jul-2009::12:44:30 === setup: 557892.25ms, good: 20000, bad: 0, run: 11261.47ms -382.1810ms | min 500.0000ms | 13035 - 65.18% 1000.0000ms | 6904 - 34.52% 1500.0000ms | 8 - 0.04% 2000.0000ms | 6 - 0.03% 2500.0000ms | 6 - 0.03% 3000.0000ms | 4 - 0.02% 3500.0000ms | 7 - 0.03% 4000.0000ms | 3 - 0.01% 4500.0000ms | 1 - 0.01% 5000.0000ms | 6 - 0.03% 10843.4820ms | 20 - 0.10% 10843.4820ms | max ok Overall latency is improved drastically in my version, however there are some stragglers which are taking way too long. I'm going to try and figure out what's going on with them. Joel Reymont wrote: > > On Jul 16, 2009, at 5:36 PM, Paulo S?rgio Almeida wrote: > >> What??? For 20k entries? Just traversal time (without sending msgs)? >> There is something wrong. It cannot be this slow. This would be 50 us >> per entry. > > > Apologies, I meant time taken to extract the list with ets:select and > traverse the list while broadcasting. > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From joelr1@REDACTED Thu Jul 16 19:21:25 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 18:21:25 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F60EB.3060609@moonpolysoft.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> Message-ID: <0FE006F0-DB60-4591-8065-D604BA8C2CA8@gmail.com> On Jul 16, 2009, at 6:18 PM, Cliff Moon wrote: > The problem you're seeing there is process scheduling overhead. In > the fun it sends a message and it's doing an ungodly amount of > context switching. The way to get around this is to bump the > process priority level of the process which iterates the ets table. My latest version on GitHub is drastically faster and does not use ETS. What kind of numbers do you get with that when you bump the process priority level? Can you show us the code that does the bumping of process priority? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Thu Jul 16 19:25:20 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 18:25:20 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F60EB.3060609@moonpolysoft.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> Message-ID: <4CF7952F-F3FF-4EA2-A70F-D31488A8A894@gmail.com> Indeed, as soon as I spread bot load over more than 1 server, broadcasting all of a sudden takes 5x longer. It's 1s vs 5s. Why? --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From fritchie@REDACTED Thu Jul 16 20:15:05 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 16 Jul 2009 13:15:05 -0500 Subject: [erlang-questions] XOP - final proof that UBF is the correct approach In-Reply-To: Message of "Wed, 15 Jul 2009 02:44:14 PDT." <44c39686-6fc8-41cf-9b5f-5d9ec9ec2ca3@g31g2000yqc.googlegroups.com> Message-ID: <96891.1247768105@snookles.snookles.com> Steve Davis wrote: sd> I'm trying to get a foothold into what kind of application would sd> encourage developers to favor UBF over XML. Steve, When I was at the Erlang Factory in Palo Alto back in April, whenever I'd mentioned that I'd been working on new UBF code, the universal reaction was, "I thought that code was dead." Nope, it's baaaaack.... :-) Joe Norton, one of my colleagues in Tokyo has done a lot of work to patch some bugs and add a bunch of type extensions and extra documentation to Joe Armstrong's original UBF code and support for other transport protocols (including JSON over a plain socket and JSON-RPC). I don't recall when our latest push to Github was, but it was probably long enough ago that at least one bugfix is missing. We're working on integrating UBF contracts with automatic QuickCheck generator creation. Together with Anders Nygren's ABNF compiler (also at Github), we've been starting to put together a nifty toolchain. The QC stuff isn't available at Github yet, but it will be, as soon as The Day Job gives us enough time to breathe. http://github.com/norton/ubf/tree/master Wow, Github's search doodad says that there's another set of UBF code hosted there. ... Oh, that's Steve's repository! It would be good to combine forces to thwart the forces of XML. -Scott From rtrlists@REDACTED Thu Jul 16 20:21:29 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 16 Jul 2009 19:21:29 +0100 Subject: W2003 vs. inet vs. inets ? Message-ID: <6a3ae47e0907161121q4a40dcfcp424f04f6779022d8@mail.gmail.com> Hi, I've come across a weird issue with the last R12B release (erts 5.6.5) running on Windows 2003 Server. When I start my node (long or short name makes no difference) I can find out about my IP address. But when I start inets and fire off an http request (which for a currently unknown reason times out), then I am no longer able to find my IP address: (erl@REDACTED)1> inet:getaddr("localhost", inet). {ok,{127,0,0,1}} (erl@REDACTED)2> inet:getaddr("vm-153013-2", inet). {ok,{172,16,3,32}} (erl@REDACTED)3> inets:start(). ok (erl@REDACTED)4> inet:getaddr("localhost", inet). {ok,{127,0,0,1}} (erl@REDACTED)5> inet:getaddr("vm-153013-2", inet). {ok,{172,16,3,32}} (erl@REDACTED)6> http:request("http://www.google.com"). {error,timeout} (erl@REDACTED)7> inet:getaddr("localhost", inet). {error,timeout} (erl@REDACTED)8> inet:getaddr("vm-153013-2", inet). {error,timeout} (erl@REDACTED)9> Has anyone seen something like this? And where would I start hunting? (I'll try R13 tomorrow.) Thanks for any pointers, Robby From thatsnotright@REDACTED Thu Jul 16 20:39:58 2009 From: thatsnotright@REDACTED (Rob Elsner) Date: Thu, 16 Jul 2009 12:39:58 -0600 Subject: File writing and odd behavior when unrelated IO activity occurs Message-ID: All, I'm recording a multicast stream using gen_udp in active mode (so my process receives a mailbox message every time a packet arrives). This is for a 35 megabit/second stream. I have a simple worker process doing the recording: loop(FileHandle, Socket, MulticastAddress, MulticastPort, SourceAddress, FilePrefix) -> receive {udp, _, _, _, Bin} -> case file:write(FileHandle, Bin) of ok -> recorder:loop(FileHandle, Socket, MulticastAddress, MulticastPort, SourceAddress, FilePrefix); {error, Reason} -> close_file(FileHandle), % close our current file close_socket(Socket, MulticastAddress, SourceAddress), log4erl:error("file:write returned error ~p", [Reason]) end; {mark} -> NewName = get_file_name(MulticastAddress, MulticastPort, SourceAddress, FilePrefix), % figure out the new file name based on address and time log4erl:info("Recording mark ~p ~p ~p New name ~p", [MulticastAddress, MulticastPort, SourceAddress, NewName]), {ok,NewHandle} = open_file(NewName), % open it close_file(FileHandle), % close our current file recorder:loop(NewHandle, Socket, MulticastAddress, MulticastPort, SourceAddress, FilePrefix); % keep writing! {stop} -> close_file(FileHandle), % close our current file close_socket(Socket, MulticastAddress, SourceAddress) after 10000 -> log4erl:warn("Timed out ~p ~p ~p ~p~n",[MulticastAddress,MulticastPort,SourceAddress,FilePrefix]), close_file(FileHandle), % close our current file close_socket(Socket, MulticastAddress, SourceAddress) end. These files get segmented at around 13GB for an hour long recording. The issue I am seeing is that using the above process, if I delete a file which has already been recorded, it causes the file:write process to "hang" which causes my mailbox to fill up and consume all 8GB of RAM on this box, before crashing the VM with an OOM error. I've tried the file open options with raw set and not set, with delayed_write. Does anyone have any thoughts about why a file delete operation on an unrelated file would cause file:write to block or take an excessive amount of time? I've tried ionice'ing the erlang VM to realtime highest priority, to no avail. Thanks, Rob Elsner From joelr1@REDACTED Thu Jul 16 21:11:15 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 20:11:15 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F60EB.3060609@moonpolysoft.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> Message-ID: <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> On Jul 16, 2009, at 6:18 PM, Cliff Moon wrote: > The problem you're seeing there is process scheduling overhead. In > the fun it sends a message and it's doing an ungodly amount of > context switching. The way to get around this is to bump the > process priority level of the process which iterates the ets table. bumping priority to high before the broadcasting list comprehension and lowering it after results in ~10ms broadcasting time and shaves ~300ms off max latency. Loving it! Mac is at ~2s (one box for everything) and ec2 is ~2.7s when bots are spread over 3 instances. It looks like process priority is a very useful tool in one's toolbox. (debug@REDACTED)4> bot:test(flashbot, 20000, 'ip-10-244-47-97', 8081). =INFO REPORT==== 16-Jul-2009::19:05:18 === setup: 69312.72ms, good: 20000, bad: 0, run: 72269.01ms 315.0020ms | min 500.0000ms | 2019 - 10.10% 1000.0000ms | 3651 - 18.25% 1500.0000ms | 4433 - 22.17% 2000.0000ms | 5259 - 26.30% 2500.0000ms | 4057 - 20.29% 3000.0000ms | 581 - 2.90% 2713.3960ms | max ok (debug@REDACTED)5> bot:test(flashbot, 20000, 'ip-10-244-47-97', 8081). =INFO REPORT==== 16-Jul-2009::19:08:37 === setup: 65606.03ms, good: 20000, bad: 0, run: 68362.39ms 328.4420ms | min 500.0000ms | 1361 - 6.80% 1000.0000ms | 4231 - 21.15% 1500.0000ms | 4477 - 22.38% 2000.0000ms | 4490 - 22.45% 2500.0000ms | 4051 - 20.26% 3000.0000ms | 1390 - 6.95% 2716.1440ms | max ok (debug@REDACTED)6> bot:test(flashbot, 20000, 'ip-10-244-47-97', 8081). =INFO REPORT==== 16-Jul-2009::19:10:45 === setup: 99778.01ms, good: 19953, bad: 47, run: 104700.73ms 313.9300ms | min 500.0000ms | 1420 - 7.12% 1000.0000ms | 4553 - 22.82% 1500.0000ms | 3987 - 19.98% 2000.0000ms | 4777 - 23.94% 2500.0000ms | 4241 - 21.25% 3000.0000ms | 968 - 4.85% 3500.0000ms | 0 - 0.00% 4000.0000ms | 0 - 0.00% 4500.0000ms | 7 - 0.04% 4143.9490ms | max ok --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From cliff@REDACTED Thu Jul 16 21:29:53 2009 From: cliff@REDACTED (Cliff Moon) Date: Thu, 16 Jul 2009 12:29:53 -0700 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> Message-ID: <4A5F7FB1.40109@moonpolysoft.com> Word dogg. On my ec2 cluster (2 machines, small instance) I'm pretty consistently getting under 2 seconds: (debug@REDACTED)1> bot:test(flashbot, 20000, "domU-12-31-39-02-B5-46.compute-1.internal", 8081). publishing =INFO REPORT==== 16-Jul-2009::14:34:42 === setup: 77401.66ms, good: 20000, bad: 0, run: 13670.42ms 590.9040ms | min 500.0000ms | 0 - 0.00% 1000.0000ms | 3553 - 17.77% 1500.0000ms | 8093 - 40.47% 2000.0000ms | 8354 - 41.77% 1861.2000ms | max ok (debug@REDACTED)2> bot:test(flashbot, 20000, "domU-12-31-39-02-B5-46.compute-1.internal", 8081). publishing =INFO REPORT==== 16-Jul-2009::14:36:32 === setup: 79961.47ms, good: 20000, bad: 0, run: 13641.45ms 459.8880ms | min 500.0000ms | 424 - 2.12% 1000.0000ms | 4116 - 20.58% 1500.0000ms | 7236 - 36.18% 2000.0000ms | 8224 - 41.12% 1760.0160ms | max ok The code is here: http://github.com/cliffmoon/janus I'll have a write up a little later that has the optimizations which seem to matter most. Joel Reymont wrote: > > On Jul 16, 2009, at 6:18 PM, Cliff Moon wrote: > >> The problem you're seeing there is process scheduling overhead. In >> the fun it sends a message and it's doing an ungodly amount of >> context switching. The way to get around this is to bump the process >> priority level of the process which iterates the ets table. > > > bumping priority to high before the broadcasting list comprehension > and lowering it after results in ~10ms broadcasting time and shaves > ~300ms off max latency. Loving it! > > Mac is at ~2s (one box for everything) and ec2 is ~2.7s when bots are > spread over 3 instances. It looks like process priority is a very > useful tool in one's toolbox. > > (debug@REDACTED)4> bot:test(flashbot, 20000, > 'ip-10-244-47-97', 8081). > > =INFO REPORT==== 16-Jul-2009::19:05:18 === > setup: 69312.72ms, good: 20000, bad: 0, run: 72269.01ms > 315.0020ms | min > 500.0000ms | 2019 - 10.10% > 1000.0000ms | 3651 - 18.25% > 1500.0000ms | 4433 - 22.17% > 2000.0000ms | 5259 - 26.30% > 2500.0000ms | 4057 - 20.29% > 3000.0000ms | 581 - 2.90% > 2713.3960ms | max > ok > (debug@REDACTED)5> bot:test(flashbot, 20000, > 'ip-10-244-47-97', 8081). > > =INFO REPORT==== 16-Jul-2009::19:08:37 === > setup: 65606.03ms, good: 20000, bad: 0, run: 68362.39ms > 328.4420ms | min > 500.0000ms | 1361 - 6.80% > 1000.0000ms | 4231 - 21.15% > 1500.0000ms | 4477 - 22.38% > 2000.0000ms | 4490 - 22.45% > 2500.0000ms | 4051 - 20.26% > 3000.0000ms | 1390 - 6.95% > 2716.1440ms | max > ok > (debug@REDACTED)6> bot:test(flashbot, 20000, > 'ip-10-244-47-97', 8081). > > =INFO REPORT==== 16-Jul-2009::19:10:45 === > setup: 99778.01ms, good: 19953, bad: 47, run: 104700.73ms > 313.9300ms | min > 500.0000ms | 1420 - 7.12% > 1000.0000ms | 4553 - 22.82% > 1500.0000ms | 3987 - 19.98% > 2000.0000ms | 4777 - 23.94% > 2500.0000ms | 4241 - 21.25% > 3000.0000ms | 968 - 4.85% > 3500.0000ms | 0 - 0.00% > 4000.0000ms | 0 - 0.00% > 4500.0000ms | 7 - 0.04% > 4143.9490ms | max > ok > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > From baryluk@REDACTED Thu Jul 16 22:21:12 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Thu, 16 Jul 2009 22:21:12 +0200 Subject: [erlang-questions] File writing and odd behavior when unrelated IO activity occurs In-Reply-To: References: Message-ID: <1247775672.5433.3.camel@sredniczarny.smp.if.uj.edu.pl> Dnia 2009-07-16, czw o godzinie 12:39 -0600, Rob Elsner pisze: > All, > > I'm recording a multicast stream using gen_udp in active mode (so my > process receives a mailbox message every time a packet arrives). This > is for a 35 megabit/second stream. I have a simple worker process > doing the recording: > > loop(FileHandle, Socket, MulticastAddress, MulticastPort, > SourceAddress, FilePrefix) -> > receive > {udp, _, _, _, Bin} -> > case file:write(FileHandle, Bin) of > ok -> > recorder:loop(FileHandle, Socket, MulticastAddress, > MulticastPort, SourceAddress, FilePrefix); > {error, Reason} -> > close_file(FileHandle), % close our current file > close_socket(Socket, MulticastAddress, SourceAddress), > log4erl:error("file:write returned error ~p", [Reason]) > end; > {mark} -> > NewName = get_file_name(MulticastAddress, MulticastPort, > SourceAddress, FilePrefix), % figure out the new file name based on > address and time > log4erl:info("Recording mark ~p ~p ~p New name ~p", > [MulticastAddress, MulticastPort, SourceAddress, NewName]), > {ok,NewHandle} = open_file(NewName), % open it > close_file(FileHandle), % close our current file > recorder:loop(NewHandle, Socket, MulticastAddress, MulticastPort, > SourceAddress, FilePrefix); % keep writing! > {stop} -> > close_file(FileHandle), % close our current file > close_socket(Socket, MulticastAddress, SourceAddress) > after 10000 -> > log4erl:warn("Timed out ~p ~p ~p > ~p~n",[MulticastAddress,MulticastPort,SourceAddress,FilePrefix]), > close_file(FileHandle), % close our current file > close_socket(Socket, MulticastAddress, SourceAddress) > end. > > These files get segmented at around 13GB for an hour long recording. > The issue I am seeing is that using the above process, if I delete a > file which has already been recorded, it causes the file:write process > to "hang" which causes my mailbox to fill up and consume all 8GB of > RAM on this box, before crashing the VM with an OOM error. > > I've tried the file open options with raw set and not set, with > delayed_write. Does anyone have any thoughts about why a file delete > operation on an unrelated file would cause file:write to block or take > an excessive amount of time? I've tried ionice'ing the erlang VM to > realtime highest priority, to no avail. > > Thanks, > Rob Elsner > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > Are you using ext3? There is problem on unlinking big files, data blocks need to be deallocated, and on ext3 this process locks other operations. I would recomend XFS on Linux for big files, especially in concurant write/read scenario. -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From cyril.romain@REDACTED Thu Jul 16 22:23:03 2009 From: cyril.romain@REDACTED (cyril romain) Date: Thu, 16 Jul 2009 22:23:03 +0200 Subject: [erlang-questions] Calling into Lua from Erlang Message-ID: Hi, Robert Raschke wrote: I will need to call Lua scripts from within Erlang soon. I was wondering if anyone has already looked into a slightly closer binding than calling an external program to run the Lua script and marshalling via stdin/out? I am currently undecided whether to make the Lua API available to be invoked from Erlang (via a linked-in driver) or if it might be more useful to write a Lua library that allows a Lua script to function as an Erlang node (similar to the Erlang Jinterface lib for Java). I would gratefully receive opinions and wisdom on this kind of stuff I'm please to announce the creation of erlua, a project I started some month ago to enable seamless interoperability between Erlang and the Lua programming language. Since no such binding is available in the FOSS world, I decided to make it free software. Erlua actually provides a C node (using EI interface only) that supports: - almost all conversions between Erlang types and lua types - syntax checking of lua file or code chunk - execution of lua file or lua code chunk from Erlang - getting/setting lua table and fields from Erlang - lua function calls from Erlang - remote Erlang procedure call from lua Erlua can be use for various purposes: to read configuration files written in lua, to embed a lightweight scripting language (lua) into an Erlang application, to use lua libraries from Erlang ... and many other ideas that probably comes to your mind :-) Future work: improve Erlang binding from Lua (e.g. to spawn processes and send Erlang messages from Lua), so that even more crazy things can be achieved ! Code is freely available under GPL at http://gitorious.org/erlua Happy hacking, Cyril From joelr1@REDACTED Thu Jul 16 22:23:11 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 21:23:11 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F7FB1.40109@moonpolysoft.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> <4A5F7FB1.40109@moonpolysoft.com> Message-ID: <8C6454A3-E55D-44A6-95F5-72D944C06092@gmail.com> Cliff, On Jul 16, 2009, at 8:29 PM, Cliff Moon wrote: > Word dogg. On my ec2 cluster (2 machines, small instance) I'm > pretty consistently getting under 2 seconds: Some comments on your code... Enabling SMP will not improve things on EC2 since it's single-core, try disabling it. Did you really see an improvement from +A 30? I thought it made sense in linked-in drivers mostly. I'm guilty of leaving +A 8 in my make file but that's because I forgot to remove it. Does {keepalive, true} really help? What did you change in gen_fsm.erl? I will remove all the messaging from bots back to the mothership since it's not needed, i.e. everything but success and failure. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From wglozer@REDACTED Thu Jul 16 22:23:24 2009 From: wglozer@REDACTED (Will) Date: Thu, 16 Jul 2009 13:23:24 -0700 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> References: <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> Message-ID: 2009/7/16 Joel Reymont : > > bumping priority to high before the broadcasting list comprehension and > lowering it after results in ~10ms broadcasting time and shaves ~300ms off > max latency. Loving it! > > Mac is at ~2s (one box for everything) and ec2 is ~2.7s when bots are spread > over 3 instances. It looks like process priority is a very useful tool in > one's toolbox. > I suspect that the bots are the bottleneck now. Here is the average max time reported out of 3 runs with 1-5 independent VMs running a total of 20,000 bots. The benchmark machine is a 3.0GHz dual, quad-core, Intel Xeon running 64-bit CentOS 5.2 and R13B01. 1 VM: 1723.75ms 2 VMs: 1429.36ms 3 VMs: 1030.18ms 4 VMs: 845.32ms 5 VMs: 784.75ms 5 VMs on one box is probably pushing it, even with 8 cores... -Will From thatsnotright@REDACTED Thu Jul 16 22:29:20 2009 From: thatsnotright@REDACTED (Rob Elsner) Date: Thu, 16 Jul 2009 14:29:20 -0600 Subject: [erlang-questions] File writing and odd behavior when unrelated IO activity occurs In-Reply-To: <1247775672.5433.3.camel@sredniczarny.smp.if.uj.edu.pl> References: <1247775672.5433.3.camel@sredniczarny.smp.if.uj.edu.pl> Message-ID: > Are you using ext3? There is problem on unlinking big files, data blocks > need to be deallocated, and on ext3 this process locks other operations. > I would recomend XFS on Linux for big files, especially in concurant > write/read scenario. I will investigate this. I wasn't aware of such a problem and just assumed it was something stupid I was doing in Erlang, or some unknown behavior (to me) with the file lib. Thank you for the suggestion, Rob From cliff@REDACTED Thu Jul 16 22:31:13 2009 From: cliff@REDACTED (Cliff Moon) Date: Thu, 16 Jul 2009 13:31:13 -0700 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> Message-ID: <4A5F8E11.6040106@moonpolysoft.com> So it turns out that the optimizations which mattered most were: compile everything with HiPE, including mochiweb bump the process priority for publishing have pubsub write directly to the socket during its loop The last two send it over the top. The reason that the wall clock time was so high during the ets fold was because it was getting hung up context switching processes. Each message sent out ends up putting a process in the run queue and erlang was not allowing the fold to complete before context switching off of the pubsub process. So bumping the priority level helps a lot with fold time, however it simply delays the pain of all of that context switch. The message has to wend it's way through two processes before it finally gets written to a socket. This means that at least 3 context switches, two for erlang procs and one for the linked in tcp driver, must happen per publish for 20k messages. So making pubsub write directly to the tcp socket is a big win in terms of absolute latency. I think there are some more inefficiencies to tease out, but I'm unsure whether it can be gotten below 1s on a small instance. Joel Reymont wrote: > > On Jul 16, 2009, at 6:18 PM, Cliff Moon wrote: > >> The problem you're seeing there is process scheduling overhead. In >> the fun it sends a message and it's doing an ungodly amount of >> context switching. The way to get around this is to bump the process >> priority level of the process which iterates the ets table. > > > bumping priority to high before the broadcasting list comprehension > and lowering it after results in ~10ms broadcasting time and shaves > ~300ms off max latency. Loving it! > > Mac is at ~2s (one box for everything) and ec2 is ~2.7s when bots are > spread over 3 instances. It looks like process priority is a very > useful tool in one's toolbox. > > (debug@REDACTED)4> bot:test(flashbot, 20000, > 'ip-10-244-47-97', 8081). > > =INFO REPORT==== 16-Jul-2009::19:05:18 === > setup: 69312.72ms, good: 20000, bad: 0, run: 72269.01ms > 315.0020ms | min > 500.0000ms | 2019 - 10.10% > 1000.0000ms | 3651 - 18.25% > 1500.0000ms | 4433 - 22.17% > 2000.0000ms | 5259 - 26.30% > 2500.0000ms | 4057 - 20.29% > 3000.0000ms | 581 - 2.90% > 2713.3960ms | max > ok > (debug@REDACTED)5> bot:test(flashbot, 20000, > 'ip-10-244-47-97', 8081). > > =INFO REPORT==== 16-Jul-2009::19:08:37 === > setup: 65606.03ms, good: 20000, bad: 0, run: 68362.39ms > 328.4420ms | min > 500.0000ms | 1361 - 6.80% > 1000.0000ms | 4231 - 21.15% > 1500.0000ms | 4477 - 22.38% > 2000.0000ms | 4490 - 22.45% > 2500.0000ms | 4051 - 20.26% > 3000.0000ms | 1390 - 6.95% > 2716.1440ms | max > ok > (debug@REDACTED)6> bot:test(flashbot, 20000, > 'ip-10-244-47-97', 8081). > > =INFO REPORT==== 16-Jul-2009::19:10:45 === > setup: 99778.01ms, good: 19953, bad: 47, run: 104700.73ms > 313.9300ms | min > 500.0000ms | 1420 - 7.12% > 1000.0000ms | 4553 - 22.82% > 1500.0000ms | 3987 - 19.98% > 2000.0000ms | 4777 - 23.94% > 2500.0000ms | 4241 - 21.25% > 3000.0000ms | 968 - 4.85% > 3500.0000ms | 0 - 0.00% > 4000.0000ms | 0 - 0.00% > 4500.0000ms | 7 - 0.04% > 4143.9490ms | max > ok > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > From joelr1@REDACTED Thu Jul 16 22:41:27 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 21:41:27 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F8E11.6040106@moonpolysoft.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> <4A5F8E11.6040106@moonpolysoft.com> Message-ID: <7F87F6A0-C5F3-4B47-B7C1-7F5720CF36EF@gmail.com> Cliff, Awesome work! On Jul 16, 2009, at 9:31 PM, Cliff Moon wrote: > I think there are some more inefficiencies to tease out, but I'm > unsure whether it can be gotten below 1s on a small instance. Have you tried splitting the bots over a couple more small ec2 instances? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From psa@REDACTED Thu Jul 16 22:47:09 2009 From: psa@REDACTED (=?ISO-8859-1?Q?Paulo_S=E9rgio_Almeida?=) Date: Thu, 16 Jul 2009 21:47:09 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> Message-ID: <4A5F91CD.7050608@di.uminho.pt> Joel Reymont wrote: > > On Jul 16, 2009, at 6:18 PM, Cliff Moon wrote: > >> The problem you're seeing there is process scheduling overhead. In >> the fun it sends a message and it's doing an ungodly amount of context >> switching. The way to get around this is to bump the process priority >> level of the process which iterates the ets table. I don't think the pubsub causes a huge amount of context switching. Sending a message in itself should not cause context switching. But there is in fact a huge amount of context switching due to the 2 * 20k middlemen. This means that even if the pubsub is preempted a single time, it will have to wait for a lot of switching before resuming the sending. Making the pubsub priority high makes it complete the sending to everyone before being preempted. However, I still think that performance should improve noticeably if there is one less hop of middlemen: it will be at least 20k less context switches and 20k less messages. Regards, Paulo From joelr1@REDACTED Thu Jul 16 22:49:44 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 21:49:44 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F91CD.7050608@di.uminho.pt> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> <4A5F91CD.7050608@di.uminho.pt> Message-ID: <6D7203E9-A9F1-4F44-8992-9DD5CD408CBF@gmail.com> On Jul 16, 2009, at 9:47 PM, Paulo S?rgio Almeida wrote: > But there is in fact a huge amount of context switching due to the 2 > * 20k middlemen. I used to have a version that used a closure with a socket to send directly. Consensus then was that a middleman process should be used :-). > However, I still think that performance should improve noticeably if > there is one less hop of middlemen: it will be at least 20k less > context switches and 20k less messages. There are no middlemen now, Cliff's version sends directly to the socket. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From fritchie@REDACTED Thu Jul 16 22:53:41 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 16 Jul 2009 15:53:41 -0500 Subject: [erlang-questions] File writing and odd behavior when unrelated IO activity occurs In-Reply-To: Message of "Thu, 16 Jul 2009 12:39:58 MDT." Message-ID: <7453.1247777621@snookles.snookles.com> Rob Elsner wrote: re> The issue I am seeing is that using the above process, if I delete a re> file which has already been recorded, it causes the file:write re> process to "hang" which causes my mailbox to fill up [...] If you aren't using "erl +A n" where "n" is some number of I/O worker threads, adding it may help if your bad behavior is that the entire VM is blocked waiting for a single unlink. If it's just a single Erlang process that's waiting for file:delete() and receiving active socket data, then I'd try: * rename the file to something harmless * spawn a proc to unlink the harmless name -Scott From wolfmanjm@REDACTED Thu Jul 16 23:01:23 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Thu, 16 Jul 2009 14:01:23 -0700 (PDT) Subject: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F8E11.6040106@moonpolysoft.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> <4A5F8E11.6040106@moonpolysoft.com> Message-ID: Seeing as I have a similar server that can do 20,000 subscribers easily under 1s (more like 600ms), I am trying to see where the major differences are. First with the change to a list of pids rather than an ets table makes the broadcast loop almost the same as mine. I store all the subscribers in a list, and add/delete them as needed (an uncommon occurrence). I also go through two gen_server procs before the tcp:send, so I don't think that is a problem. I do not use HIPE, I found it makes very little difference on my setup (maybe 5% at best). Doing a tcp send in the loop I think will make it much slower, for reasons I explain in the previous thread. I do not bump the priority around my broadcast list comprehension, although I may try that to see if it speeds my server up even more. Other than the JSON munging there is no other major difference anymore. other than the way the stats are gathered, which may be the source of discrepancy, I'll need to look more closely at how you do that, I do like the histogram approach so I may incorporate that in mine rather than min/max/ave which is what I do now. Also the way the test clients are setup is very different to the way I do it, although I doubt that is significant. I basically have a latency tester running in its own VM, and it spawns the given number of TCP clients as regular processes, and they do very little other than connect, and wait for messages, extract the timestamp from the message, and store the results. When the latency tester stops it sends a message to each client process to end, and they send their results to the tester. Although I'll never have 20,000 clients in a single topic, I need to have sub 10ms latency on every message with 200 subscribers, which I currently achieve handily. This has been an interesting exercise I look forward to seeing the end results. From joelr1@REDACTED Thu Jul 16 23:25:04 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 22:25:04 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> <4A5F8E11.6040106@moonpolysoft.com> Message-ID: Mac, HiPE-compiled everything, including mochiweb, consistently ~1s. This is the list version (no ets) that sends via 2 middlemen processes. =INFO REPORT==== 16-Jul-2009::22:20:59 === setup: 156250.41ms, good: 20000, bad: 0, run: 162741.96ms 1000.0000ms | 10868 - 54.34% 1500.0000ms | 8125 - 40.63% 1075.4410ms | max ok (debug@REDACTED)2> bot:test(flashbot, 20000). =INFO REPORT==== 16-Jul-2009::22:21:42 === setup: 17522.91ms, good: 20000, bad: 0, run: 23429.93ms 327.8270ms | min 500.0000ms | 2205 - 11.03% 1000.0000ms | 15385 - 76.92% 1500.0000ms | 2410 - 12.05% 1022.2170ms | max ok (debug@REDACTED)3> bot:test(flashbot, 20000). =INFO REPORT==== 16-Jul-2009::22:22:10 === setup: 17570.93ms, good: 20000, bad: 0, run: 23337.27ms 320.6370ms | min 500.0000ms | 849 - 4.25% 1000.0000ms | 18066 - 90.33% 1500.0000ms | 1085 - 5.42% 1009.3740ms | max ok --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From wolfmanjm@REDACTED Thu Jul 16 23:25:38 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Thu, 16 Jul 2009 14:25:38 -0700 (PDT) Subject: Unicast 20k messages, $500-$1000 bounty In-Reply-To: References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> <4A5F8E11.6040106@moonpolysoft.com> Message-ID: <0a84f244-3567-4b44-9060-3d277e9fe14b@y10g2000prf.googlegroups.com> Hmmm. setting the priority higher around my broadcast loop actually makes the average latency worse not better by about 8%, actually rather interesting, as my theory as to why broadcasting to process rather than directly to tcp hinges on parallel execution of system commands would predict that outcome. > > I do not bump the priority around my broadcast list comprehension, > although I may try that to see if it speeds my server up even more. > From joelr1@REDACTED Fri Jul 17 00:16:34 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jul 2009 23:16:34 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5F91CD.7050608@di.uminho.pt> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> <4A5F91CD.7050608@di.uminho.pt> Message-ID: <9DF5DF93-E9BA-4C5C-A15D-FF9AE036D659@gmail.com> More fun with Erlang optimization... bot:wait used to take 3 arguments and I thought I'd remove the last two since they are not needed, as well as remove the subscribing, connected and disconnected handling code. I then removed the sending of these messages from flashbot. The above trimming consistently bumped my latency from 1s to 2s on the Mac with HiPE. What gives? --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From psa@REDACTED Fri Jul 17 01:24:26 2009 From: psa@REDACTED (=?ISO-8859-1?Q?Paulo_S=E9rgio_Almeida?=) Date: Fri, 17 Jul 2009 00:24:26 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <6D7203E9-A9F1-4F44-8992-9DD5CD408CBF@gmail.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> <4A5F91CD.7050608@di.uminho.pt> <6D7203E9-A9F1-4F44-8992-9DD5CD408CBF@gmail.com> Message-ID: <4A5FB6AA.6090604@di.uminho.pt> Joel Reymont wrote: > There are no middlemen now, Cliff's version sends directly to the socket. I did not sugest sending directly to the socket in the loop because it is a bad idea: if there is a single slow client and gen_tcp:send blocks the remaining clients will suffer ... It is important to have one middleman to make the service tolerant of slow (or even malicious) clients. This is specially true for large messages. The middleman should also use send_timeout, and should keep the inbox small, retrieving all messages to data structure before doing a gen_tcp:send and possibly discarding messages if the client cannot keep up with the message rate. Regards, Paulo From cliff@REDACTED Fri Jul 17 01:30:27 2009 From: cliff@REDACTED (Cliff Moon) Date: Thu, 16 Jul 2009 16:30:27 -0700 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <9DF5DF93-E9BA-4C5C-A15D-FF9AE036D659@gmail.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> <4A5F91CD.7050608@di.uminho.pt> <9DF5DF93-E9BA-4C5C-A15D-FF9AE036D659@gmail.com> Message-ID: <4A5FB813.1040708@moonpolysoft.com> The latest timings from ec2, this time with 4 servers distributing the client load and a single server. The max latency hovers just under 1s most of the time. (debug@REDACTED)4> bot:test(flashbot, 20000, "domU-12-31-39-02-B5-46.compute-1.internal", 8081). publishing =INFO REPORT==== 16-Jul-2009::19:13:32 === setup: 39034.92ms, good: 20000, bad: 0, run: 5097.52ms -29.2580ms | min 500.0000ms | 10224 - 51.12% 1000.0000ms | 9776 - 48.88% 965.6640ms | max ok (debug@REDACTED)5> bot:test(flashbot, 20000, "domU-12-31-39-02-B5-46.compute-1.internal", 8081). publishing =INFO REPORT==== 16-Jul-2009::19:14:30 === setup: 36596.29ms, good: 20000, bad: 0, run: 5070.07ms -29.4500ms | min 500.0000ms | 10260 - 51.30% 1000.0000ms | 9740 - 48.70% 999.8180ms | max ok (debug@REDACTED)9> bot:test(flashbot, 20000, "domU-12-31-39-02-B5-46.compute-1.internal", 8081). publishing =INFO REPORT==== 16-Jul-2009::19:24:34 === setup: 36501.23ms, good: 20000, bad: 0, run: 5119.36ms -19.9900ms | min 500.0000ms | 10796 - 53.98% 1000.0000ms | 9204 - 46.02% 979.9250ms | max ok (debug@REDACTED)4> bot:test(flashbot, 20000, "domU-12-31-39-02-B5-46.compute-1.internal", 8081). publishing =INFO REPORT==== 16-Jul-2009::19:27:16 === setup: 37879.43ms, good: 20000, bad: 0, run: 5192.95ms -37.1400ms | min 500.0000ms | 10062 - 50.31% 1000.0000ms | 9938 - 49.69% 971.6150ms | max ok Joel Reymont wrote: > More fun with Erlang optimization... > > bot:wait used to take 3 arguments and I thought I'd remove the last > two since they are not needed, as well as remove the subscribing, > connected and disconnected handling code. I then removed the sending > of these messages from flashbot. > > The above trimming consistently bumped my latency from 1s to 2s on the > Mac with HiPE. > > What gives? > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > From wolfmanjm@REDACTED Fri Jul 17 01:37:40 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Thu, 16 Jul 2009 16:37:40 -0700 (PDT) Subject: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <9DF5DF93-E9BA-4C5C-A15D-FF9AE036D659@gmail.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> <4A5F91CD.7050608@di.uminho.pt> <9DF5DF93-E9BA-4C5C-A15D-FF9AE036D659@gmail.com> Message-ID: Sorry a quick look at the code didn't answer this question :) How many messages are you timing? I usually send 10 or more messages and average the time, or take the 95th percentile one, this may even out the results a bit. Also I run the server and do two runs of the clients, and time the second one to eliminate any one time dynamic loading glitches. This made run-to-run differences minimal. On Jul 16, 3:16?pm, Joel Reymont wrote: > More fun with Erlang optimization... > > bot:wait used to take 3 arguments and I thought I'd remove the last ? > two since they are not needed, as well as remove the subscribing, ? > connected and disconnected handling code. I then removed the sending ? > of these messages from flashbot. > > The above trimming consistently bumped my latency from 1s to 2s on the ? > Mac with HiPE. > > What gives? > > --- > Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From kunthar@REDACTED Fri Jul 17 01:45:31 2009 From: kunthar@REDACTED (Kunthar) Date: Fri, 17 Jul 2009 02:45:31 +0300 Subject: [erlang-questions] Calling into Lua from Erlang In-Reply-To: References: Message-ID: <9a09ca9a0907161645j485fee79h6eb9e9214859c16@mail.gmail.com> Great news. Thank you for your effort. On Thu, Jul 16, 2009 at 11:23 PM, cyril romain wrote: > Hi, > > Robert Raschke wrote: > > I will need to call Lua scripts from within Erlang soon. I was > wondering if anyone has already looked into a slightly closer binding > than calling an external program to run the Lua script and marshalling > via stdin/out? > > I am currently undecided whether to make the Lua API available to be > invoked from Erlang (via a linked-in driver) or if it might be more > useful to write a Lua library that allows a Lua script to function as > an Erlang node (similar to the Erlang Jinterface lib for Java). > > I would gratefully receive opinions and wisdom on this kind of stuff > > > I'm please to announce the creation of erlua, a project I started some month > ago to enable seamless interoperability between Erlang and the Lua > programming language. > Since no such binding is available in the FOSS world, I decided to make it > free software. > > Erlua actually provides a C node (using EI interface only) that supports: > - almost all conversions between Erlang types and lua types > - syntax checking of lua file or code chunk > - execution of lua file or lua code chunk from Erlang > - getting/setting lua table and fields from Erlang > - lua function calls from Erlang > - remote Erlang procedure call from lua > > Erlua can be use for various purposes: to read configuration files written > in lua, to embed a lightweight scripting language (lua) into an Erlang > application, to use lua libraries from Erlang ... and many other ideas that > probably comes to your mind :-) > Future work: improve Erlang binding from Lua (e.g. to spawn processes and > send Erlang messages from Lua), so that even more crazy things can be > achieved ! > > Code is freely available under GPL at http://gitorious.org/erlua > > Happy hacking, > > Cyril > From cliff@REDACTED Fri Jul 17 04:00:39 2009 From: cliff@REDACTED (Cliff Moon) Date: Thu, 16 Jul 2009 19:00:39 -0700 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5FB6AA.6090604@di.uminho.pt> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> <4A5F91CD.7050608@di.uminho.pt> <6D7203E9-A9F1-4F44-8992-9DD5CD408CBF@gmail.com> <4A5FB6AA.6090604@di.uminho.pt> Message-ID: <4A5FDB47.20603@moonpolysoft.com> Your assertion about gen_tcp:send blocking is incorrect in this particular case. In this test the async thread queue is enabled, which allows the io subsystem to queue blocking IO operations and execute them concurrently with the rest of the VM. Paulo S?rgio Almeida wrote: > Joel Reymont wrote: > >> There are no middlemen now, Cliff's version sends directly to the >> socket. > > I did not sugest sending directly to the socket in the loop because it > is a bad idea: if there is a single slow client and gen_tcp:send > blocks the remaining clients will suffer ... > > It is important to have one middleman to make the service tolerant of > slow (or even malicious) clients. This is specially true for large > messages. The middleman should also use send_timeout, and should keep > the inbox small, retrieving all messages to data structure before > doing a gen_tcp:send and possibly discarding messages if the client > cannot keep up with the message rate. > > Regards, > Paulo > > From fritchie@REDACTED Fri Jul 17 04:13:07 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 16 Jul 2009 21:13:07 -0500 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: Message of "Thu, 16 Jul 2009 19:00:39 PDT." <4A5FDB47.20603@moonpolysoft.com> Message-ID: <28004.1247796787@snookles.snookles.com> Cliff Moon wrote: cm> Your assertion about gen_tcp:send blocking is incorrect in this cm> particular case. In this test the async thread queue is enabled, cm> which allows the io subsystem to queue blocking IO operations and cm> execute them concurrently with the rest of the VM. Cliff, I haven't looked deeply into the guts of the R13B VM, but IIRC the R11 VM + inets driver didn't use the async thread pool. File I/O yes, TCP/UDP I/O I don't think so. Independent (and preferably authoritative :-) confirmation/refutation is welcome. -Scott From wglozer@REDACTED Fri Jul 17 06:40:49 2009 From: wglozer@REDACTED (Will) Date: Thu, 16 Jul 2009 21:40:49 -0700 Subject: announce: epgsql-1.2 Message-ID: Hello, I've released version 1.2 of epgsql, a pure Erlang database driver for PostgreSQL. The code can be found in mercurial repositories at: http://glozer.net/src/epgsql http://bitbucket.org/will/epgsql This release fixes a number of bugs, and also adds support for a {timeout, N} option to handle long-running queries. gen_tcp connections are now established with {nodelay, true} which dramatically improves the performance of equery and parse/bind/execute on Linux. -Will From joelr1@REDACTED Fri Jul 17 10:02:17 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jul 2009 09:02:17 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A5FB813.1040708@moonpolysoft.com> References: <8082FAD4-2DFE-4B28-93F3-06A012349DC6@gmail.com> <187DD93A-D076-4D1A-AB92-7D201F168522@gmail.com> <4A5F31C6.7010903@di.uminho.pt> <4A5F3F83.3090906@di.uminho.pt> <3DDE490C-F780-45FC-A9A8-B8A3728B511F@gmail.com> <4A5F570A.9070302@di.uminho.pt> <4A5F60EB.3060609@moonpolysoft.com> <911CF40A-4FC1-4623-B6E9-4C19D616C75A@gmail.com> <4A5F91CD.7050608@di.uminho.pt> <9DF5DF93-E9BA-4C5C-A15D-FF9AE036D659@gmail.com> <4A5FB813.1040708@moonpolysoft.com> Message-ID: <09916D9D-127B-4868-9C61-EECF9F79BD6D@gmail.com> On Jul 17, 2009, at 12:30 AM, Cliff Moon wrote: > The latest timings from ec2, this time with 4 servers distributing > the client load and a single server. The max latency hovers just > under 1s most of the time. Cliff takes the $1000 bounty and the contest is over :-). I am concerned about gen_tcp:send blocking on slow clients and having at least one middlemen process but that does not take away from Cliff's achievement. Woot! Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From psa@REDACTED Fri Jul 17 11:03:40 2009 From: psa@REDACTED (=?ISO-8859-1?Q?Paulo_S=E9rgio_Almeida?=) Date: Fri, 17 Jul 2009 10:03:40 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <28004.1247796787@snookles.snookles.com> References: <28004.1247796787@snookles.snookles.com> Message-ID: <4A603E6C.7050102@di.uminho.pt> Scott Lystig Fritchie wrote: > Cliff Moon wrote: > > cm> Your assertion about gen_tcp:send blocking is incorrect in this > cm> particular case. In this test the async thread queue is enabled, > cm> which allows the io subsystem to queue blocking IO operations and > cm> execute them concurrently with the rest of the VM. > > Cliff, I haven't looked deeply into the guts of the R13B VM, but IIRC > the R11 VM + inets driver didn't use the async thread pool. File I/O > yes, TCP/UDP I/O I don't think so. Independent (and preferably > authoritative :-) confirmation/refutation is welcome. I have a feeling async threads are for File I/O, but even if they are used for TCP, the problem still holds. They would tolerate a transient slowness. But if some client(s) are persistently slow and cannot keep up, after the TCP window has been exausted and all async threads have also been exausted, then it will block. You need to do some flow control. Possibly discarding messages and in the worst case even disconnecting the client. And as it should be done independently for each client, a per client middleman process is the best place to do it. Regards, Paulo From joelr1@REDACTED Fri Jul 17 11:10:39 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jul 2009 10:10:39 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A603E6C.7050102@di.uminho.pt> References: <28004.1247796787@snookles.snookles.com> <4A603E6C.7050102@di.uminho.pt> Message-ID: On Jul 17, 2009, at 10:03 AM, Paulo S?rgio Almeida wrote: > I have a feeling async threads are for File I/O, but even if they > are used for TCP, the problem still holds. They would tolerate a > transient slowness. But if some client(s) are persistently slow and > cannot keep up, after the TCP window has been exausted and all async > threads have also been exausted, then it will block. Do you have any code that simulates this scenario? This is what prim_inet:send looks like: %% This is a generic "port_command" interface used by TCP, UDP, SCTP, depending %% on the driver it is mapped to, and the "Data". It actually sends out data,-- %% NOT delegating this task to any back-end. For SCTP, this function MUST NOT %% be called directly -- use "sendmsg" instead: %% send(S, Data) when is_port(S) -> ?DBG_FORMAT("prim_inet:send(~p, ~p)~n", [S,Data]), try erlang:port_command(S, Data) of true -> receive {inet_reply,S,Status} -> ?DBG_FORMAT("prim_inet:send() -> ~p~n", [Status]), Status end catch error:_Error -> ?DBG_FORMAT("prim_inet:send() -> {error,einval}~n", []), {error,einval} end. It looks like it will block waiting for a port reply. Since the result of gen_tcp:send is ignored in the broadcasting loop, perhaps erlang:port_command can be used directly on the socket to make things completely asynchronous and avoid the need for a middleman process. I think solid proof is needed here rather than speciluation. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Fri Jul 17 11:43:20 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jul 2009 10:43:20 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: References: <28004.1247796787@snookles.snookles.com> <4A603E6C.7050102@di.uminho.pt> Message-ID: <87DEBDF2-99F8-4639-BB63-9A92564360F6@gmail.com> On Jul 17, 2009, at 10:10 AM, Joel Reymont wrote: > Since the result of gen_tcp:send is ignored in the broadcasting > loop, perhaps erlang:port_command can be used directly on the socket > to make things completely asynchronous and avoid the need for a > middleman process. The inet_reply messages would need to be ignored by pubsub:handle_info, of course. No middleman then? --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From 0x6e6562@REDACTED Fri Jul 17 11:54:50 2009 From: 0x6e6562@REDACTED (Ben Hood) Date: Fri, 17 Jul 2009 10:54:50 +0100 Subject: [erlang-questions] OTP-7548 code loading Message-ID: <269388e30907170254j33853446r8bb0234fd552ebff@mail.gmail.com> Hi, I've been looking into the experimental code loading feature introduced in OTP-7548. Albeit some apparent cosmetic issues (see http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:45215:200907:agakhlhaggkaecjnfbip), it seems to work. It's quite a convenient feature if you want/need to compose various different OTP apps. Is there anybody actually using this in a production scenario? Also, the feature has been been marked as experimental - is there any kind of general commitment going forwards to support some kind of code loading from archives, even if it's not in the current form? Thanks, Ben From wolfmanjm@REDACTED Fri Jul 17 12:12:33 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Fri, 17 Jul 2009 03:12:33 -0700 (PDT) Subject: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <87DEBDF2-99F8-4639-BB63-9A92564360F6@gmail.com> References: <28004.1247796787@snookles.snookles.com> <4A603E6C.7050102@di.uminho.pt> <87DEBDF2-99F8-4639-BB63-9A92564360F6@gmail.com> Message-ID: <7b1277d4-e1a8-4400-be33-21d7781363be@b14g2000yqd.googlegroups.com> > No middleman then? I think you are really asking for trouble, or at least a not very robust server, if you go that route. It is Socket programming 101 that writes can block when the TCP receiver does not read and the flow control causes the TCP buffers to backup. Additionally read the docs about gen_tcp:send, at the end in the example section where it states... "The fact that the send call does not accept a timeout option, is because timeouts on send is handled through the socket option send_timeout. The behavior of a send operation with no receiver is in a very high degree defined by the underlying TCP stack, as well as the network infrastructure. If one wants to write code that handles a hanging receiver that might eventually cause the sender to hang on a send call, one writes code like the following. " Although it does not explicitly state that gen_tcp:send can block I think it is is common knowledge that it does, and the explanation of how to handle timeouts would indicate that too. Again just my 0.02c worth. From joelr1@REDACTED Fri Jul 17 12:19:37 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jul 2009 11:19:37 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <7b1277d4-e1a8-4400-be33-21d7781363be@b14g2000yqd.googlegroups.com> References: <28004.1247796787@snookles.snookles.com> <4A603E6C.7050102@di.uminho.pt> <87DEBDF2-99F8-4639-BB63-9A92564360F6@gmail.com> <7b1277d4-e1a8-4400-be33-21d7781363be@b14g2000yqd.googlegroups.com> Message-ID: <035F4839-0510-4D09-A575-6ADC6B511C15@gmail.com> Jim, On Jul 17, 2009, at 11:12 AM, Jim Morris wrote: > It is Socket programming 101 that writes can block when the TCP > receiver does not read and the flow control causes the TCP buffers to > backup. This all makes sense. I will redo the code using just 1 middleman process then. Now, if we could just figure out why your server is so much faster ;-). Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From rtrlists@REDACTED Fri Jul 17 12:32:06 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 17 Jul 2009 11:32:06 +0100 Subject: W2003 vs. inet vs. inets ? In-Reply-To: <6a3ae47e0907161121q4a40dcfcp424f04f6779022d8@mail.gmail.com> References: <6a3ae47e0907161121q4a40dcfcp424f04f6779022d8@mail.gmail.com> Message-ID: <6a3ae47e0907170332w1dc9456dq7dbe0dc009fd1c10@mail.gmail.com> Hi, has anyone else come across inet_gethost.exe issues on Windows 2003 Server? I am seeing the following on 2003 (R13 has the exact same result, btw): Eshell V5.6.5 (abort with ^G) (erl@REDACTED)1> inet:getaddr("vm-153013-2", inet). {ok,{172,16,3,32}} (erl@REDACTED)2> inet:getaddr("www.google.com", inet). {error,timeout} And this on XP: Eshell V5.6.5 (abort with ^G) (erl@REDACTED)1> inet:getaddr("02533-Laptop", inet). {ok,{172,16,1,33}} (erl@REDACTED)2> inet:getaddr("www.google.com", inet). {ok,{209,85,227,147}} On the Windows 2003 Server, nslookup, wget, IE, etc. all have no difficulty in looking up www.google.com . If I turn on tracing of inet_gethost.exe I get this on the 2003 Server: ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):num_workers = 4, greedy_threshold = 3, debug_level = 7. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Created reader and writer threads. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Got data on index 0. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Got data from erlang. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):OPeration == 1. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Saved domainname . ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Created worker[2772] with fd 3485552 ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Saved domainname . ...\release\erts-5.6.5\bin\inet_gethost.exe[2772] (DEBUG):Worker got data on message que. ...\release\erts-5.6.5\bin\inet_gethost.exe[2772] (DEBUG):Worker got request, op = 1, proto = 2, data = localhost. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Got data on index 1. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Got data on index 0. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Got data from erlang. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):OPeration == 1. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Saved domainname . ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Saved domainname . ...\release\erts-5.6.5\bin\inet_gethost.exe[2772] (DEBUG):Worker got data on message que. ...\release\erts-5.6.5\bin\inet_gethost.exe[2772] (DEBUG):Worker got request, op = 1, proto = 1, data = vm-153013-2. ...\release\erts-5.6.5\bin\inet_gethost.exe[2772] (DEBUG):Starting gethostbyname(vm-153013-2) ...\release\erts-5.6.5\bin\inet_gethost.exe[2772] (DEBUG):gethostbyname OK ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Got data on index 1. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Got data on index 0. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Got data from erlang. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):OPeration == 1. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Saved domainname . google.com. ...\release\erts-5.6.5\bin\inet_gethost.exe[2772] (DEBUG):Worker got data on message que. ...\release\erts-5.6.5\bin\inet_gethost.exe[2772] (DEBUG):Worker got request, op = 1, proto = 1, data = www.google.com. ...\release\erts-5.6.5\bin\inet_gethost.exe[2772] (DEBUG):Starting gethostbyname(www.google.com) ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Saved domainname . google.com. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Got data on index 0. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Got data from erlang. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):OPeration == 3. ...\release\erts-5.6.5\bin\inet_gethost.exe[2824] (DEBUG):Stalled worker[2772] It looks like the call to gethostbyname() never returns! Where do I look next? Thanks for any hints and pointers, Robby From sverker@REDACTED Fri Jul 17 12:35:39 2009 From: sverker@REDACTED (Sverker Eriksson) Date: Fri, 17 Jul 2009 12:35:39 +0200 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: References: <28004.1247796787@snookles.snookles.com> <4A603E6C.7050102@di.uminho.pt> Message-ID: <4A6053FB.9080605@erix.ericsson.se> Joel Reymont wrote: > Since the result of gen_tcp:send is ignored in the broadcasting loop, > perhaps erlang:port_command can be used directly on the socket to make > things completely asynchronous and avoid the need for a middleman > process. > > I think solid proof is needed here rather than speciluation. > Calling erlang:port_command directly will not make things completely asyncronous. The inet driver will suspend the calling process when the send queue has reached its high watermark. You will block in erlang:port_command until some sending has succeeded and the send queue is below a low watermark. Or you can timeout using option send_timeout. /Sverker, Erlang/OTP From ulf.wiger@REDACTED Fri Jul 17 12:39:16 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 17 Jul 2009 12:39:16 +0200 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <035F4839-0510-4D09-A575-6ADC6B511C15@gmail.com> References: <28004.1247796787@snookles.snookles.com> <4A603E6C.7050102@di.uminho.pt> <87DEBDF2-99F8-4639-BB63-9A92564360F6@gmail.com> <7b1277d4-e1a8-4400-be33-21d7781363be@b14g2000yqd.googlegroups.com> <035F4839-0510-4D09-A575-6ADC6B511C15@gmail.com> Message-ID: <4A6054D4.3020605@erlang-consulting.com> Joel Reymont wrote: > Jim, > > On Jul 17, 2009, at 11:12 AM, Jim Morris wrote: > >> It is Socket programming 101 that writes can block when the TCP >> receiver does not read and the flow control causes the TCP buffers to >> backup. > > This all makes sense. I will redo the code using just 1 middleman > process then. You may want to consider the following passage from the inet man page on setopts/2: {delay_send, Boolean} Normally, when an Erlang process sends to a socket, the driver will try to immediately send the data. If that fails, the driver will use any means available to queue up the message to be sent whenever the operating system says it can handle it. Setting {delay_send, true} will make all messages queue up. This makes the messages actually sent onto the network be larger but fewer. The option actually affects the scheduling of send requests versus Erlang processes instead of changing any real property of the socket. Needless to say it is an implementation specific option. Default is false. I'm not sure if setting {delay_send, true} would have any beneficial effect on your particular problem, but the text does shed some light on the semantics of sending to a socket. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From joelr1@REDACTED Fri Jul 17 12:46:03 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jul 2009 11:46:03 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A6053FB.9080605@erix.ericsson.se> References: <28004.1247796787@snookles.snookles.com> <4A603E6C.7050102@di.uminho.pt> <4A6053FB.9080605@erix.ericsson.se> Message-ID: The challenge then would seem to be getting under 1s and 2s with a middleman process, to avoid blocking the send loop. Possible? On Jul 17, 2009, at 11:35 AM, Sverker Eriksson wrote: > Calling erlang:port_command directly will not make things completely > asyncronous. The inet driver will suspend the calling process when > the send queue has reached its high watermark. You will block in > erlang:port_command until some sending has succeeded and the send > queue is below a low watermark. Or you can timeout using option > send_timeout. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Fri Jul 17 13:03:26 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jul 2009 12:03:26 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A6054D4.3020605@erlang-consulting.com> References: <28004.1247796787@snookles.snookles.com> <4A603E6C.7050102@di.uminho.pt> <87DEBDF2-99F8-4639-BB63-9A92564360F6@gmail.com> <7b1277d4-e1a8-4400-be33-21d7781363be@b14g2000yqd.googlegroups.com> <035F4839-0510-4D09-A575-6ADC6B511C15@gmail.com> <4A6054D4.3020605@erlang-consulting.com> Message-ID: On Jul 17, 2009, at 11:39 AM, Ulf Wiger wrote: > {delay_send, Boolean} > Normally, when an Erlang process sends to a socket, the driver will > try to immediately send the data. If that fails, the driver will use > any means available to queue up the message to be sent whenever the > operating system says it can handle it. Setting {delay_send, true} > will make all messages queue up. On EC2 with 3 bot instances I seem to be doing ~200-300ms better with delay_send enabled and sending directly to the socket. Could be just a fluke. =INFO REPORT==== 17-Jul-2009::10:59:17 === setup: 32944.80ms, good: 20000, bad: 0, run: 37288.00ms -60.8130ms | min 500.0000ms | 4992 - 24.96% 1000.0000ms | 4652 - 23.26% 1500.0000ms | 5312 - 26.56% 2000.0000ms | 3914 - 19.57% 2500.0000ms | 1130 - 5.65% 2363.9200ms | max ok (debug@REDACTED)4> bot:test(flashbot, 20000, 'ip-10-244-47-97', 8081). =INFO REPORT==== 17-Jul-2009::11:00:01 === setup: 30712.31ms, good: 20000, bad: 0, run: 35419.80ms -61.9370ms | min 500.0000ms | 5376 - 26.88% 1000.0000ms | 4844 - 24.22% 1500.0000ms | 5187 - 25.93% 2000.0000ms | 3358 - 16.79% 2500.0000ms | 1235 - 6.17% 2472.4190ms | max ok (debug@REDACTED)5> bot:test(flashbot, 20000, 'ip-10-244-47-97', 8081). =INFO REPORT==== 17-Jul-2009::11:00:49 === setup: 30307.11ms, good: 20000, bad: 0, run: 35555.13ms -63.3260ms | min 500.0000ms | 6321 - 31.61% 1000.0000ms | 5827 - 29.14% 1500.0000ms | 5422 - 27.11% 2000.0000ms | 1438 - 7.19% 2500.0000ms | 992 - 4.96% 2318.4370ms | max ok --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Fri Jul 17 13:08:07 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jul 2009 12:08:07 +0100 Subject: 500k clients, 100 amazon ec2 instances Message-ID: <06DBB94A-BC50-4469-8E83-9A0B42031466@gmail.com> This is a bit off topic but is anyone from Amazon reading this list? I need 100 small EC2 instances as soon as possible, 20 server ones behind a load balancer and the rest for bots. I will only need this many instances for a short while. This is to expand my Ultimate Erlang Challenge to 500k clients, although I would be glad to try with up to 1 million and 200 instances respectively. I figure it will be faster to ask on the list than to go through official channels. Please pass the message if you know the right people! Thanks in advance, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Fri Jul 17 13:46:19 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jul 2009 12:46:19 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <7b1277d4-e1a8-4400-be33-21d7781363be@b14g2000yqd.googlegroups.com> References: <28004.1247796787@snookles.snookles.com> <4A603E6C.7050102@di.uminho.pt> <87DEBDF2-99F8-4639-BB63-9A92564360F6@gmail.com> <7b1277d4-e1a8-4400-be33-21d7781363be@b14g2000yqd.googlegroups.com> Message-ID: <8E7CE53A-EF2E-4A76-9558-4D902E602F90@gmail.com> On Jul 17, 2009, at 11:12 AM, Jim Morris wrote: >> No middleman then? > > I think you are really asking for trouble, or at least a not very > robust server, if you go that route. > > It is Socket programming 101 that writes can block when the TCP > receiver does not read and the flow control causes the TCP buffers to > backup. RabbitMQ 1.6 (bottom of rabbit_writer.erl) uses erlang:port_command directly. The premise is that selective receive of inet_reply messages in gen_tcp:send can be very slow when the sender's message queue is large. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Fri Jul 17 14:13:08 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jul 2009 13:13:08 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A6053FB.9080605@erix.ericsson.se> References: <28004.1247796787@snookles.snookles.com> <4A603E6C.7050102@di.uminho.pt> <4A6053FB.9080605@erix.ericsson.se> Message-ID: <6E2AE3F7-AECC-42A2-92D5-31B28502C37D@gmail.com> Can we have a final expert word on this from the OTP team? Must we use a middleman process or can we combine {delay_send, true} with directly sending to the socket when unicasting to 20k clients? Are there other tweaks to apply apart from raising priority to high around the broadcasting ets:foldr? This dispute must be settled as soon as possible for the benefit of future generations! On Jul 17, 2009, at 11:35 AM, Sverker Eriksson wrote: > Joel Reymont wrote: >> Since the result of gen_tcp:send is ignored in the broadcasting >> loop, perhaps erlang:port_command can be used directly on the >> socket to make things completely asynchronous and avoid the need >> for a middleman process. >> >> I think solid proof is needed here rather than speciluation. >> > Calling erlang:port_command directly will not make things completely > asyncronous. The inet driver will suspend the calling process when > the send queue has reached its high watermark. You will block in > erlang:port_command until some sending has succeeded and the send > queue is below a low watermark. Or you can timeout using option > send_timeout. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From psa@REDACTED Fri Jul 17 15:59:30 2009 From: psa@REDACTED (=?ISO-8859-1?Q?Paulo_S=E9rgio_Almeida?=) Date: Fri, 17 Jul 2009 14:59:30 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <6E2AE3F7-AECC-42A2-92D5-31B28502C37D@gmail.com> References: <28004.1247796787@snookles.snookles.com> <4A603E6C.7050102@di.uminho.pt> <4A6053FB.9080605@erix.ericsson.se> <6E2AE3F7-AECC-42A2-92D5-31B28502C37D@gmail.com> Message-ID: <4A6083C2.6090801@di.uminho.pt> Joel Reymont wrote: > Can we have a final expert word on this from the OTP team? > > Must we use a middleman process or can we combine {delay_send, true} > with directly sending to the socket when unicasting to 20k clients? I am no such expert, but there is a simple universal rule: there are no miracles. If a client stops receiving then you either; - discard messages to that client or disconnect the client or - the messages are going to be buffered somewhere (TCP stack, message queues, data structures), memory consumption will grow unboundedly, until the VM crashes out of memory. You cannot escape doing some flow control. I said a middleman was the best place to do it. As in convenient place. If you are desperate and want to do it with less convenience and more trouble, I guess the best way to obtain low latency will be: - use port:command to write to the socket; do the loop without waiting for the reply messages; - for each client keep a counter of outstanding replies from the port; update it as replies arrive; use ets to store this info and ets:update_counter; - if that counter reaches a given value start discarding messages (i.e. ignore that client in the send loop); for this you can use ets:select with a condition on the counter to obtain the list of ports to use in the loop. That's it. Have fun. Paulo From joelr1@REDACTED Fri Jul 17 16:01:05 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jul 2009 15:01:05 +0100 Subject: Unicast 40k messages, $1000 bounty Message-ID: The last bounty was claimed by Cliff Moon but I'm not entirely satisfied since getting under 1 second requires sending directly to the socket. As discussed in the 20k thread, this can cause the broadcasting loop to block on slow clients. The new challenge is to broadcast to 40k clients in under 1s on Amazon EC2 (small server instance) using a middleman. Same $1000 bounty. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Fri Jul 17 16:23:17 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jul 2009 15:23:17 +0100 Subject: Unicast 40k messages, $1000 bounty In-Reply-To: References: Message-ID: Just to clarify, this is a new 1K bounty, in addition to the one that goes to Cliff. On Jul 17, 2009, at 3:01 PM, Joel Reymont wrote: > The last bounty was claimed by Cliff Moon but I'm not entirely > satisfied since getting under 1 second requires sending directly to > the socket. As discussed in the 20k thread, this can cause the > broadcasting loop to block on slow clients. > > The new challenge is to broadcast to 40k clients in under 1s on > Amazon EC2 (small server instance) using a middleman. Same $1000 > bounty. --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Fri Jul 17 21:57:33 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jul 2009 20:57:33 +0100 Subject: hipe and smp Message-ID: <4025DE5B-AEBB-4348-8325-3A707E8B98D0@gmail.com> Do HiPE and SMP mix? I'm seeing segmentation faults on Ubuntu 9.04 x86_64. Is it worth troubleshooting and if so how? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From kostis@REDACTED Fri Jul 17 22:22:54 2009 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 17 Jul 2009 23:22:54 +0300 Subject: [erlang-questions] hipe and smp In-Reply-To: <4025DE5B-AEBB-4348-8325-3A707E8B98D0@gmail.com> References: <4025DE5B-AEBB-4348-8325-3A707E8B98D0@gmail.com> Message-ID: <4A60DD9E.6090708@cs.ntua.gr> Joel Reymont wrote: > Do HiPE and SMP mix? Yes, very much so. > I'm seeing segmentation faults on Ubuntu 9.04 x86_64. > > Is it worth troubleshooting and if so how? Yes, it's worth troubleshooting. You send us a (preferably small) program that shows the issue together with instructions on how to reproduce the problem. Kostis From joelr1@REDACTED Fri Jul 17 22:27:31 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jul 2009 21:27:31 +0100 Subject: [erlang-questions] hipe and smp In-Reply-To: <4A60DD9E.6090708@cs.ntua.gr> References: <4025DE5B-AEBB-4348-8325-3A707E8B98D0@gmail.com> <4A60DD9E.6090708@cs.ntua.gr> Message-ID: <25F7D68F-2B1A-4E67-BB17-6DBC3577FFC6@gmail.com> Kostis, On Jul 17, 2009, at 9:22 PM, Kostis Sagonas wrote: > Yes, it's worth troubleshooting. You send us a (preferably small) > program that shows the issue together with instructions on how to > reproduce the problem. You can just grab the Janus code from GitHub [1]. It crashes in burns upon startup when compiled with HiPE on x86-64. 1) get mochiweb and add +native to its list of compilation flags 2) compile mochiweb 3) cd janus 4) ln -s /mochiweb . 5) make sh Then see it crash and burn. Thanks, Joel [1] http://github.com/tinycode/janus/tree/master --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From juanjo@REDACTED Fri Jul 17 22:39:45 2009 From: juanjo@REDACTED (Juan Jose Comellas) Date: Fri, 17 Jul 2009 17:39:45 -0300 Subject: [erlang-questions] hipe and smp In-Reply-To: <25F7D68F-2B1A-4E67-BB17-6DBC3577FFC6@gmail.com> References: <4025DE5B-AEBB-4348-8325-3A707E8B98D0@gmail.com> <4A60DD9E.6090708@cs.ntua.gr> <25F7D68F-2B1A-4E67-BB17-6DBC3577FFC6@gmail.com> Message-ID: <1c3be50f0907171339o41893c87h9c1faefec265c1e1@mail.gmail.com> I believe mochiweb does not work with HiPE, as it uses parameterized modules. AFAIK that was not supported by HiPE (at least the last time I checked). On Fri, Jul 17, 2009 at 5:27 PM, Joel Reymont wrote: > Kostis, > > On Jul 17, 2009, at 9:22 PM, Kostis Sagonas wrote: > > Yes, it's worth troubleshooting. You send us a (preferably small) program >> that shows the issue together with instructions on how to reproduce the >> problem. >> > > > You can just grab the Janus code from GitHub [1]. > > It crashes in burns upon startup when compiled with HiPE on x86-64. > > 1) get mochiweb and add +native to its list of compilation flags > 2) compile mochiweb > 3) cd janus > 4) ln -s /mochiweb . > 5) make sh > > Then see it crash and burn. > > Thanks, Joel > > [1] http://github.com/tinycode/janus/tree/master > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From wolfmanjm@REDACTED Fri Jul 17 22:50:43 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Fri, 17 Jul 2009 13:50:43 -0700 (PDT) Subject: hipe and smp In-Reply-To: <25F7D68F-2B1A-4E67-BB17-6DBC3577FFC6@gmail.com> References: <4025DE5B-AEBB-4348-8325-3A707E8B98D0@gmail.com> <4A60DD9E.6090708@cs.ntua.gr> <25F7D68F-2B1A-4E67-BB17-6DBC3577FFC6@gmail.com> Message-ID: <7a01f8fc-2a05-4244-86d7-abad395f0bd6@y4g2000prf.googlegroups.com> I didn't know you were using 64 bit instances, is your Mac 64 bit too? All my tests are on a regular x86, I wonder if that makes a difference to the speed difference? > > It crashes in burns upon startup when compiled with HiPE on x86-64. > From joelr1@REDACTED Fri Jul 17 22:56:02 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jul 2009 21:56:02 +0100 Subject: [erlang-questions] Re: hipe and smp In-Reply-To: <7a01f8fc-2a05-4244-86d7-abad395f0bd6@y4g2000prf.googlegroups.com> References: <4025DE5B-AEBB-4348-8325-3A707E8B98D0@gmail.com> <4A60DD9E.6090708@cs.ntua.gr> <25F7D68F-2B1A-4E67-BB17-6DBC3577FFC6@gmail.com> <7a01f8fc-2a05-4244-86d7-abad395f0bd6@y4g2000prf.googlegroups.com> Message-ID: <2454DBD1-9453-43A3-BE28-EC0C7B8416EE@gmail.com> > I didn't know you were using 64 bit instances, is your Mac 64 bit too? Mac is 32 bits and EC2 small is 32 bits too. I'm trying a XL high-computing instance (8 cores) to run the bots, though. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From cliff@REDACTED Fri Jul 17 23:19:56 2009 From: cliff@REDACTED (Cliff Moon) Date: Fri, 17 Jul 2009 14:19:56 -0700 Subject: [erlang-questions] hipe and smp In-Reply-To: <1c3be50f0907171339o41893c87h9c1faefec265c1e1@mail.gmail.com> References: <4025DE5B-AEBB-4348-8325-3A707E8B98D0@gmail.com> <4A60DD9E.6090708@cs.ntua.gr> <25F7D68F-2B1A-4E67-BB17-6DBC3577FFC6@gmail.com> <1c3be50f0907171339o41893c87h9c1faefec265c1e1@mail.gmail.com> Message-ID: <4A60EAFC.6060501@moonpolysoft.com> I believer parameterized modules were fixed in HiPE for R13B. Juan Jose Comellas wrote: > I believe mochiweb does not work with HiPE, as it uses parameterized > modules. AFAIK that was not supported by HiPE (at least the last time I > checked). > > > On Fri, Jul 17, 2009 at 5:27 PM, Joel Reymont wrote: > > >> Kostis, >> >> On Jul 17, 2009, at 9:22 PM, Kostis Sagonas wrote: >> >> Yes, it's worth troubleshooting. You send us a (preferably small) program >> >>> that shows the issue together with instructions on how to reproduce the >>> problem. >>> >>> >> You can just grab the Janus code from GitHub [1]. >> >> It crashes in burns upon startup when compiled with HiPE on x86-64. >> >> 1) get mochiweb and add +native to its list of compilation flags >> 2) compile mochiweb >> 3) cd janus >> 4) ln -s /mochiweb . >> 5) make sh >> >> Then see it crash and burn. >> >> Thanks, Joel >> >> [1] http://github.com/tinycode/janus/tree/master >> >> --- >> Mac hacker with a performance bent >> http://www.linkedin.com/in/joelreymont >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> > > From mihai@REDACTED Fri Jul 17 23:22:40 2009 From: mihai@REDACTED (Mihai Balea) Date: Fri, 17 Jul 2009 17:22:40 -0400 Subject: Weird problem when using tcp sockets in HTTP mode Message-ID: <5EE0B723-C21E-443C-9DF5-2A52CC918F17@hates.ms> Hi all, I'm hitting a weird problem and I'm wondering if it is a bug somewhere in the system libs. I'm running R13B01. I am opening a tcp socket in http mode and apparently the HttpUri returned by gen_tcp:recv contains only the first key-value pair of the query string. Here's the code I'm running: -module(h). -compile(export_all). run() -> {ok, LS} = gen_tcp:listen(6969, [{packet, http}, {active, false}, {reuseaddr, true}, list]), {ok, S} = gen_tcp:accept(LS), do_recv(S). do_recv(S) -> {ok, Packet} = gen_tcp:recv(S, 0), io:format("~p\n", [Packet]), do_recv(S). Here's the query: curl -v http://localhost:6969/foo/bar?a=1&b=2&c=3 And the output of the program: 3> h:run(). {http_request,'GET',{abs_path,"/foo/bar?a=1"},{1,1}} {http_header,24,'User-Agent',undefined, "curl/7.19.5 (i386-apple-darwin9.7.0) libcurl/7.19.5 zlib/1.2.3"} {http_header,14,'Host',undefined,"localhost:6969"} {http_header,8,'Accept',undefined,"*/*"} http_eoh Am I doing something wrong or is this indeed a bug? Thanks, Mihai From mikpe@REDACTED Fri Jul 17 23:24:38 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Fri, 17 Jul 2009 23:24:38 +0200 Subject: [erlang-questions] hipe and smp In-Reply-To: <1c3be50f0907171339o41893c87h9c1faefec265c1e1@mail.gmail.com> References: <4025DE5B-AEBB-4348-8325-3A707E8B98D0@gmail.com> <4A60DD9E.6090708@cs.ntua.gr> <25F7D68F-2B1A-4E67-BB17-6DBC3577FFC6@gmail.com> <1c3be50f0907171339o41893c87h9c1faefec265c1e1@mail.gmail.com> Message-ID: <19040.60438.923285.63808@pilspetsen.it.uu.se> Juan Jose Comellas writes: > I believe mochiweb does not work with HiPE, as it uses parameterized > modules. AFAIK that was not supported by HiPE (at least the last time I > checked). Parameterized modules should work with HiPE in R13. If they don't, report that to erlang-bugs together with a minimal standalone test case. From ruslan@REDACTED Sat Jul 18 00:26:18 2009 From: ruslan@REDACTED (Ruslan Babayev) Date: Fri, 17 Jul 2009 15:26:18 -0700 Subject: [erlang-questions] Weird problem when using tcp sockets in HTTP mode In-Reply-To: <5EE0B723-C21E-443C-9DF5-2A52CC918F17@hates.ms> References: <5EE0B723-C21E-443C-9DF5-2A52CC918F17@hates.ms> Message-ID: <05FABB1B-2C9E-42D6-AFEA-8A530BFC3C87@babayev.com> Try enclosing the URL in double quotes: curl -v "http://localhost:6969/foo/bar?a=1&b=2&c=3" On Jul 17, 2009, at 2:22 PM, Mihai Balea wrote: > Hi all, > > I'm hitting a weird problem and I'm wondering if it is a bug > somewhere in the system libs. > > I'm running R13B01. I am opening a tcp socket in http mode and > apparently the HttpUri returned by gen_tcp:recv contains only the > first key-value pair of the query string. > > Here's the code I'm running: > > -module(h). > -compile(export_all). > > run() -> > {ok, LS} = gen_tcp:listen(6969, [{packet, http}, {active, false}, > {reuseaddr, true}, list]), > {ok, S} = gen_tcp:accept(LS), > do_recv(S). > > do_recv(S) -> > {ok, Packet} = gen_tcp:recv(S, 0), > io:format("~p\n", [Packet]), > do_recv(S). > > Here's the query: > curl -v http://localhost:6969/foo/bar?a=1&b=2&c=3 > > And the output of the program: > 3> h:run(). > {http_request,'GET',{abs_path,"/foo/bar?a=1"},{1,1}} > {http_header,24,'User-Agent',undefined, > "curl/7.19.5 (i386-apple-darwin9.7.0) libcurl/7.19.5 > zlib/1.2.3"} > {http_header,14,'Host',undefined,"localhost:6969"} > {http_header,8,'Accept',undefined,"*/*"} > http_eoh > > Am I doing something wrong or is this indeed a bug? > > Thanks, > Mihai > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From chsu79@REDACTED Sat Jul 18 00:36:00 2009 From: chsu79@REDACTED (Christian) Date: Sat, 18 Jul 2009 00:36:00 +0200 Subject: [erlang-questions] Weird problem when using tcp sockets in HTTP mode In-Reply-To: <5EE0B723-C21E-443C-9DF5-2A52CC918F17@hates.ms> References: <5EE0B723-C21E-443C-9DF5-2A52CC918F17@hates.ms> Message-ID: Did you try to quote the url when you pass it to curl? The shell probably tries to interpret ampersand to make it a background job. Otherwise, use wireshark and get the facts about what requests are sent. On Fri, Jul 17, 2009 at 23:22, Mihai Balea wrote: > Hi all, > > I'm hitting a weird problem and I'm wondering if it is a bug somewhere in > the system libs. > > I'm running R13B01. I am opening a tcp socket in http mode and apparently > the HttpUri returned by gen_tcp:recv contains only the first key-value pair > of the query string. > > Here's the code I'm running: > > -module(h). > -compile(export_all). > > run() -> > ? ?{ok, LS} = gen_tcp:listen(6969, [{packet, http}, {active, false}, > {reuseaddr, true}, list]), > ? ?{ok, S} = gen_tcp:accept(LS), > ? ?do_recv(S). > > do_recv(S) -> > ? ?{ok, Packet} = gen_tcp:recv(S, 0), > ? ?io:format("~p\n", [Packet]), > ? ?do_recv(S). > > Here's the query: > curl -v http://localhost:6969/foo/bar?a=1&b=2&c=3 > > And the output of the program: > 3> h:run(). > {http_request,'GET',{abs_path,"/foo/bar?a=1"},{1,1}} > {http_header,24,'User-Agent',undefined, > ? ? ? ? ? ? "curl/7.19.5 (i386-apple-darwin9.7.0) libcurl/7.19.5 > zlib/1.2.3"} > {http_header,14,'Host',undefined,"localhost:6969"} > {http_header,8,'Accept',undefined,"*/*"} > http_eoh > > Am I doing something wrong or is this indeed a bug? > > Thanks, > Mihai > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From raould@REDACTED Sat Jul 18 01:51:38 2009 From: raould@REDACTED (Raoul Duke) Date: Fri, 17 Jul 2009 16:51:38 -0700 Subject: suggestion for somebody who has free time :-) Message-ID: <91a2ba3e0907171651o6c27e15crbf57643d6fe2ae9@mail.gmail.com> "super productive lfe in a box" (along the lines of "clojure in a box" et. al.) consisting of: - erlang (duh) - lfe - dialyzer - emacs as the ide probably - project (makefiles) set up such that dialyzer and unit tests get auto-run as you are live editing code - heck for an extra fee also quick check integrated? :) From pfisher@REDACTED Sat Jul 18 03:36:09 2009 From: pfisher@REDACTED (Paul Fisher) Date: Fri, 17 Jul 2009 20:36:09 -0500 Subject: gen_server2 merge into OTP? Message-ID: <4A612709.6060503@alertlogic.net> Was wondering if there was any possibility that the gen_server2 [1] would have any chance of being merged into the standard OTP distribution? A good summary of the benefits can be found at [2]. [1] http://hg.rabbitmq.com/rabbitmq-server/file/b95f2fd4e3f6/src/gen_server2.erl [2] http://www.joeandmotorboat.com/2009/02/06/new-stuff-for-merle/ -- paul From joelr1@REDACTED Sat Jul 18 08:12:26 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 18 Jul 2009 07:12:26 +0100 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <6E2AE3F7-AECC-42A2-92D5-31B28502C37D@gmail.com> References: <28004.1247796787@snookles.snookles.com> <4A603E6C.7050102@di.uminho.pt> <4A6053FB.9080605@erix.ericsson.se> <6E2AE3F7-AECC-42A2-92D5-31B28502C37D@gmail.com> Message-ID: <6538A257-C38C-4224-9A4C-85A80F286375@gmail.com> Folks, Janus has been renamed to Hot Wheels and now lives at http://github.com/wagerlabs/hotwheels/tree/master . The goal is still to get a broadcast to 40k users in < 1s. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Sat Jul 18 08:14:51 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 18 Jul 2009 07:14:51 +0100 Subject: [erlang-questions] Re: Unicast 40k messages, $1000 bounty (Hot Wheels!) In-Reply-To: References: Message-ID: Folks, Janus has been renamed to Hot Wheels and now lives at http://github.com/wagerlabs/hotwheels/tree/master . The goal is still to get a broadcast to 40k users in < 1s. I'm making several protocol changes, e.g. {packet, 2} and moving away from JSON to a simple protocol where every packet has a 12 byte timestamp to measure latency, 1 byte for message type (1 = subscribe, 2 = unsubscribe), and size + body for topic, event, etc. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Sat Jul 18 10:11:27 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 18 Jul 2009 09:11:27 +0100 Subject: binary optimization Message-ID: How do I improve this? src/flashbot.erl:120: Warning: NOT OPTIMIZED: compiler limitation: instruction {apply_last,2,6} prevents delayed sub binary optimization 120: handle_info({tcp, _Sock, <>}, Where, State) -> Delta = timer:now_diff(now(), {A, B, C}), ?MODULE:Where(Bin, State#state{latency = Delta}); Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From ulf.wiger@REDACTED Sat Jul 18 10:44:35 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 18 Jul 2009 10:44:35 +0200 Subject: [erlang-questions] gen_server2 merge into OTP? In-Reply-To: <4A612709.6060503@alertlogic.net> References: <4A612709.6060503@alertlogic.net> Message-ID: <4A618B73.9030104@erlang-consulting.com> Hmm, there was a question on the list just recently which touched on how erl_eval drains the messages and resends them to itself, and how this severely punishes code where selective receive is /not/ used. BR, Ulf W Paul Fisher wrote: > Was wondering if there was any possibility that the gen_server2 [1] > would have any chance of being merged into the standard OTP > distribution? A good summary of the benefits can be found at [2]. > > [1] > http://hg.rabbitmq.com/rabbitmq-server/file/b95f2fd4e3f6/src/gen_server2.erl > > [2] http://www.joeandmotorboat.com/2009/02/06/new-stuff-for-merle/ > > > -- > paul > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From mikpe@REDACTED Sat Jul 18 11:25:55 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Sat, 18 Jul 2009 11:25:55 +0200 Subject: [erlang-questions] binary optimization In-Reply-To: References: Message-ID: <19041.38179.901233.130939@pilspetsen.it.uu.se> Joel Reymont writes: > How do I improve this? > > src/flashbot.erl:120: Warning: NOT OPTIMIZED: compiler limitation: > instruction {apply_last,2,6} prevents delayed sub binary optimization > > 120: handle_info({tcp, _Sock, <>}, > Where, State) -> > Delta = timer:now_diff(now(), {A, B, C}), > ?MODULE:Where(Bin, State#state{latency = Delta}); Get rid of the apply, i.e. the ?MODULE:Where call. That would be good for other performance reasons too. I don't know the exact limitations for the delayed sub binary optimization, but presumably the callee in the tailcall must be known at compile-time, and must presumably only be called with a delayed sub binary. It _may_ even be as strict as only allowing self-referential tailcalls. From matthew@REDACTED Sat Jul 18 12:38:45 2009 From: matthew@REDACTED (Matthew Sackman) Date: Sat, 18 Jul 2009 11:38:45 +0100 Subject: [erlang-questions] gen_server2 merge into OTP? In-Reply-To: <4A618B73.9030104@erlang-consulting.com> References: <4A612709.6060503@alertlogic.net> <4A618B73.9030104@erlang-consulting.com> Message-ID: <20090718103844.GA25084@wellquite.org> On Sat, Jul 18, 2009 at 10:44:35AM +0200, Ulf Wiger wrote: > Hmm, there was a question on the list just recently which > touched on how erl_eval drains the messages and resends > them to itself, and how this severely punishes code where > selective receive is /not/ used. The purpose of gen_server2 is a) to allow message priorities to be used; b) to make sure that when you do a call to another process, you don't risk having to scan a huge mailbox when the reply comes back because the mailbox is always kept small as it's constantly drained into our own priority queue. If you want to see an even funkier version of gen_server2, try http://hg.rabbitmq.com/rabbitmq-server/file/dc753bc0c54e/src/gen_server2.erl ;) Matthew From joelr1@REDACTED Sat Jul 18 12:49:13 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 18 Jul 2009 11:49:13 +0100 Subject: [erlang-questions] binary optimization In-Reply-To: <19041.38179.901233.130939@pilspetsen.it.uu.se> References: <19041.38179.901233.130939@pilspetsen.it.uu.se> Message-ID: On Jul 18, 2009, at 10:25 AM, Mikael Pettersson wrote: > Get rid of the apply, i.e. the ?MODULE:Where call. That would be good > for other performance reasons too. This doesn't fly either... src/flashbot.erl:117: Warning: NOT OPTIMIZED: sub binary is used or returned handle_info({tcp, _Sock, <>}, Where, State) -> Delta = timer:now_diff(now(), {A, B, C}), Where(Bin, State#state{latency = Delta}); Also, what about this? src/flashbot.erl:81: Warning: NOT OPTIMIZED: sub binary used by erlang:setelement/3 no_token(<>, State) -> send([?SUBSCRIBE, size(?TOPIC), ?TOPIC], State#state{token = Token}, not_subscribed). Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From richardc@REDACTED Sat Jul 18 13:04:14 2009 From: richardc@REDACTED (Richard Carlsson) Date: Sat, 18 Jul 2009 13:04:14 +0200 Subject: [erlang-questions] binary optimization In-Reply-To: References: <19041.38179.901233.130939@pilspetsen.it.uu.se> Message-ID: <4A61AC2E.2050107@it.uu.se> Joel Reymont wrote: > This doesn't fly either... > > src/flashbot.erl:117: Warning: NOT OPTIMIZED: sub binary is used or > returned > > handle_info({tcp, _Sock, <>}, Where, > State) -> > Delta = timer:now_diff(now(), {A, B, C}), > Where(Bin, State#state{latency = Delta}); You missed the important point in Mikael's mail: "presumably the callee in the tailcall must be known at compile-time, and must presumably only be called with a delayed sub binary." In the above, Bin gets passed to Where, which could be anything as far as the compiler knows. Either you ignore the warning, if you think that restructuring the code is not worth it here, or you try to rewrite the whole thing so you get a single main loop with direct self-recursive tail calls (if that is at all doable). > src/flashbot.erl:81: Warning: NOT OPTIMIZED: sub binary used by > erlang:setelement/3 > > no_token(<>, State) -> > send([?SUBSCRIBE, size(?TOPIC), ?TOPIC], > State#state{token = Token}, > not_subscribed). Well, it is. Token gets inserted into State. You could probably just ignore this one. /Richard From joelr1@REDACTED Sat Jul 18 13:41:00 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 18 Jul 2009 12:41:00 +0100 Subject: [erlang-questions] binary optimization In-Reply-To: <4A61AC2E.2050107@it.uu.se> References: <19041.38179.901233.130939@pilspetsen.it.uu.se> <4A61AC2E.2050107@it.uu.se> Message-ID: <7F2503DB-9186-4E51-8D76-269111F27A9D@gmail.com> What is a "delayed sub-binary"? On Jul 18, 2009, at 12:04 PM, Richard Carlsson wrote: > or you try to rewrite the whole thing so you get a single main loop > with > direct self-recursive tail calls (if that is at all doable). Wouldn't the tail call of the loop need to be written as ?MODULE:loop to enable code reloading? This would result in the previous "no optimization with apply" warning. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From mikpe@REDACTED Sat Jul 18 13:41:22 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Sat, 18 Jul 2009 13:41:22 +0200 Subject: [erlang-questions] binary optimization In-Reply-To: <4A61AC2E.2050107@it.uu.se> References: <19041.38179.901233.130939@pilspetsen.it.uu.se> <4A61AC2E.2050107@it.uu.se> Message-ID: <19041.46306.345021.701328@pilspetsen.it.uu.se> Richard Carlsson writes: > Joel Reymont wrote: > > This doesn't fly either... > > > > src/flashbot.erl:117: Warning: NOT OPTIMIZED: sub binary is used or > > returned > > > > handle_info({tcp, _Sock, <>}, Where, > > State) -> > > Delta = timer:now_diff(now(), {A, B, C}), > > Where(Bin, State#state{latency = Delta}); > > You missed the important point in Mikael's mail: > "presumably the callee in the tailcall must be known at compile-time, > and must presumably only be called with a delayed sub binary." > > In the above, Bin gets passed to Where, which could be anything > as far as the compiler knows. Either you ignore the warning, if you > think that restructuring the code is not worth it here, or you > try to rewrite the whole thing so you get a single main loop with > direct self-recursive tail calls (if that is at all doable). Well, lowering this higher-order code to first-order code is always possible (cite Reynolds 1972 or thereabouts), but those transformations tend to destroy the structure of the code. In this code, either enumerate the possible values for Where and invoke them via an explicit case expression, or specialise handle_info/3 into a set of handle_info_.../2 functions, one for each possible Where. Or just live with the fact that this call cancels delayed sub binary optimisation. > > src/flashbot.erl:81: Warning: NOT OPTIMIZED: sub binary used by > > erlang:setelement/3 > > > > no_token(<>, State) -> > > send([?SUBSCRIBE, size(?TOPIC), ?TOPIC], > > State#state{token = Token}, > > not_subscribed). > > Well, it is. Token gets inserted into State. To elaborate, storing Token in an aggregate and passing that elsewhere forces the implementation to convert Token from the delayed sub binary representation to a proper Erlang term, which requires memory allocation and initialisation. A possible solution is to change send/3 to send/4 with Token as an additional parameter, and then hope that send/4 is nice enough so that the compiler can keep Token in its delayed sub binary representation. From joelr1@REDACTED Sat Jul 18 13:46:28 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 18 Jul 2009 12:46:28 +0100 Subject: [erlang-questions] binary optimization In-Reply-To: <19041.46306.345021.701328@pilspetsen.it.uu.se> References: <19041.38179.901233.130939@pilspetsen.it.uu.se> <4A61AC2E.2050107@it.uu.se> <19041.46306.345021.701328@pilspetsen.it.uu.se> Message-ID: <824B7879-0C0F-4236-A3FB-84FCFB995CB9@gmail.com> On Jul 18, 2009, at 12:41 PM, Mikael Pettersson wrote: > In this code, either enumerate the possible values for Where and > invoke them via an explicit case expression, or specialise > handle_info/3 > into a set of handle_info_.../2 functions, one for each possible > Where. What exactly (and how much) would I be buying with this restructuring? Would I be avoiding duplicating the binary, i.e. extra memory allocation? Is there anything else? > To elaborate, storing Token in an aggregate and passing that elsewhere > forces the implementation to convert Token from the delayed sub binary > representation to a proper Erlang term, What is a delayed sub-binary representation? Is there a paper on this? > A possible solution is to change send/3 to send/4 > with Token as an additional parameter, and then hope that send/4 is > nice > enough so that the compiler can keep Token in its delayed sub binary > representation. Well, I do want to keep the token around in the state of the gen_server. I guess converting to send/4 is not gonna buy me anything since I will need to stuff Token into State in send/4. Am I right? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Sat Jul 18 13:53:58 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 18 Jul 2009 12:53:58 +0100 Subject: [erlang-questions] binary optimization In-Reply-To: <4A61AC2E.2050107@it.uu.se> References: <19041.38179.901233.130939@pilspetsen.it.uu.se> <4A61AC2E.2050107@it.uu.se> Message-ID: On Jul 18, 2009, at 12:04 PM, Richard Carlsson wrote: > You missed the important point in Mikael's mail: > "presumably the callee in the tailcall must be known at compile-time, > and must presumably only be called with a delayed sub binary." What about this one? I'm calling session:subscribe/2 and that's known at compile time. --- src/transport.erl:98: Warning: NOT OPTIMIZED: sub binary used by session:subscribe/2 98: ?????handle_info({tcp, Sock, <<_:96, ?SUBSCRIBE, Topic/ binary>>}, State) -> inet:setopts(Sock, [{active, once}]), session:subscribe(State#state.session, Topic), {noreply, State}; --- Is the compiler telling me that the sub binary is "improperly" used by session:subscribe/2, e.g. subscribe(Ref, Topic) -> gen_server:cast(Ref, {subscribe, Topic}). What are the problems with my matching instructions that the following warning is referring to? --- src/transport.erl:110: Warning: NOT OPTIMIZED: the binary matching instruction that follows in the same function have problems that prevent delayed sub binary optimization (probably indicated by INFO warnings) src/transport.erl:113: Warning: NOT OPTIMIZED: sub binary used by topic:publish/2 110: handle_info({tcp, Sock, <<_:96, ?PUBLISH, Len:32, Bin/binary>>}, State) -> inet:setopts(Sock, [{active, once}]), Len1 = Len * 8, 113: <> = Bin, topic:publish(Topic, Msg), {noreply, State}; Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From mikpe@REDACTED Sat Jul 18 14:47:52 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Sat, 18 Jul 2009 14:47:52 +0200 Subject: [erlang-questions] binary optimization In-Reply-To: <7F2503DB-9186-4E51-8D76-269111F27A9D@gmail.com> References: <19041.38179.901233.130939@pilspetsen.it.uu.se> <4A61AC2E.2050107@it.uu.se> <7F2503DB-9186-4E51-8D76-269111F27A9D@gmail.com> Message-ID: <19041.50296.215405.24706@pilspetsen.it.uu.se> Joel Reymont writes: > What is a "delayed sub-binary"? > > On Jul 18, 2009, at 12:04 PM, Richard Carlsson wrote: > > > or you try to rewrite the whole thing so you get a single main loop > > with > > direct self-recursive tail calls (if that is at all doable). > > Wouldn't the tail call of the loop need to be written as ?MODULE:loop > to enable code reloading? Only if you need immediate code upgrading in _this_ loop, which seems unlikely. Typically there should be a semi-infinite top-level message receive/send loop, which calls into short-lived lower-level worker loops, like your binary parsing thing. Only the top-level loop should have to care about code upgrades. From richardc@REDACTED Sat Jul 18 14:54:28 2009 From: richardc@REDACTED (Richard Carlsson) Date: Sat, 18 Jul 2009 14:54:28 +0200 Subject: [erlang-questions] binary optimization In-Reply-To: References: <19041.38179.901233.130939@pilspetsen.it.uu.se> <4A61AC2E.2050107@it.uu.se> Message-ID: <4A61C604.4050609@it.uu.se> Joel Reymont wrote: > What about this one? I'm calling session:subscribe/2 and that's known at > compile time. > > src/transport.erl:98: Warning: NOT OPTIMIZED: sub binary used by > session:subscribe/2 > > 98: ?????handle_info({tcp, Sock, <<_:96, ?SUBSCRIBE, Topic/binary>>}, > State) -> > inet:setopts(Sock, [{active, once}]), > session:subscribe(State#state.session, Topic), > {noreply, State}; Well, no. Because you could dynamically replace the session module at any time, so the compiler can't really make any assumptions about what that function does (or will do after a future code upgrade). You should probably not dwell too much on these warnings. They are hints to allow you to optimize what you're doing with binaries, but those optimizations can only happen in particular (though important) circumstances. Your code does not seem to be an example of those circumstances. Though I haven't taken a closer look at these binary optimizations myself, this is my understanding of it: 1) When you match out a subsection of a binary, the system may use a small (a few words) representation of this subsection in the form of a "sub-binary", which is simply a reference that says "I'm this part of that guy over there". The sub-binary will be allocated on the heap, just like any other data, and to the user it looks like you made a copy of a part of the original binary. If the part is quite small, the system might choose to copy the data to a new proper binary instead of making a sub-binary. So far so good. 2) Depending on what you're going to do with that sub-binary, it might be a waste of time to create it on the heap, if it's not going to live for very long (and in particular if this is in the middle of a loop over a binary). The "delayed sub-binary optimization" is when the compiler decides to keep the sub-binary info in registers or on the stack instead. If the functions makes a tail-recursive call to itself with the sub-binary as one of the arguments, the compiler can detect that it still doesn't need to actually create a heap object, but just loop with the info still kept in registers and/or on the stack. 3) This means that if what you're doing is to put the sub-binary in a data structure, this optimization is off. The same goes if you are passing it to an unknown function (or perhaps even to _any_ other function except for tail calls to the current function; this is where Mikael and I aren't sure). 4) Unless you _are_ doing some kind of loop over the contents of a binary, these optimizations are rather negligible, but when you _do_ need to traverse a binary, they can give a good speedup. /Richard From mikpe@REDACTED Sat Jul 18 15:03:32 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Sat, 18 Jul 2009 15:03:32 +0200 Subject: [erlang-questions] binary optimization In-Reply-To: <824B7879-0C0F-4236-A3FB-84FCFB995CB9@gmail.com> References: <19041.38179.901233.130939@pilspetsen.it.uu.se> <4A61AC2E.2050107@it.uu.se> <19041.46306.345021.701328@pilspetsen.it.uu.se> <824B7879-0C0F-4236-A3FB-84FCFB995CB9@gmail.com> Message-ID: <19041.51236.864447.428026@pilspetsen.it.uu.se> Joel Reymont writes: > > On Jul 18, 2009, at 12:41 PM, Mikael Pettersson wrote: > > > In this code, either enumerate the possible values for Where and > > invoke them via an explicit case expression, or specialise > > handle_info/3 > > into a set of handle_info_.../2 functions, one for each possible > > Where. > > What exactly (and how much) would I be buying with this restructuring? > > Would I be avoiding duplicating the binary, i.e. extra memory > allocation? That depends entirely on the nature of the callees. If they require converting the delayed sub binary to a proper term, then you won't save anything for that piece of the state. On the other hand, if they don't then yes you'd save some memory allocation. > Is there anything else? Calls to code which isn't known at compile-time are always expensive, both because they limit what optimisations the compiler can do, and because they require additional type checks or lookups at runtime. > What is a delayed sub-binary representation? Is there a paper on this? None that I know of. Basically, a binary is a full-blown term. A sub-binary is a binary constructed as a sub-segment of a binary, which allows the sub-binary to be represented as a small wrapper term containing a reference to the original big binary. A delayed sub-binary, presumably, is a sub-binary represented by a set of registers in the generated code. That's possible as long as the compiler can see all accesses to that sub-binary. However, the instant that delayed sub-binary "escapes" (is returned, passed to unknown code, or stored in an aggregate), it must be converted to a full-blown proper term, i.e. be allocated on the heap. Bj?rn is the expert on this stuff. > > A possible solution is to change send/3 to send/4 > > with Token as an additional parameter, and then hope that send/4 is > > nice > > enough so that the compiler can keep Token in its delayed sub binary > > representation. > > > Well, I do want to keep the token around in the state of the gen_server. > I guess converting to send/4 is not gonna buy me anything since I will > need to stuff Token into State in send/4. Am I right? Yes, stuffing Token into State makes it escape, requiring it to be converted to a full-blown term. From mikpe@REDACTED Sat Jul 18 15:14:38 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Sat, 18 Jul 2009 15:14:38 +0200 Subject: [erlang-questions] binary optimization In-Reply-To: References: <19041.38179.901233.130939@pilspetsen.it.uu.se> <4A61AC2E.2050107@it.uu.se> Message-ID: <19041.51902.991418.44243@pilspetsen.it.uu.se> On Sat, 18 Jul 2009 12:53:58 +0100, Joel Reymont wrote: > On Jul 18, 2009, at 12:04 PM, Richard Carlsson wrote: > > > You missed the important point in Mikael's mail: > > "presumably the callee in the tailcall must be known at compile-time, > > and must presumably only be called with a delayed sub binary." > > > What about this one? I'm calling session:subscribe/2 and that's known > at compile time. > > --- > src/transport.erl:98: Warning: NOT OPTIMIZED: sub binary used by =20 > session:subscribe/2 > > 98: handle_info({tcp, Sock, <<_:96, ?SUBSCRIBE, Topic/binary>>}, State) -> > inet:setopts(Sock, [{active, once}]), > session:subscribe(State#state.session, Topic), > {noreply, State}; Only the name is known, not the actual code. > What are the problems with my matching instructions that the following > warning is referring to? > > --- > src/transport.erl:110: Warning: NOT OPTIMIZED: the binary matching > instruction that follows in the same function have problems that > prevent delayed sub binary optimization (probably indicated by INFO > warnings) > src/transport.erl:113: Warning: NOT OPTIMIZED: sub binary used by > topic:publish/2 > > 110: handle_info({tcp, Sock, <<_:96, ?PUBLISH, Len:32, Bin/binary>>}, State) -> > inet:setopts(Sock, [{active, once}]), > Len1 = Len * 8, > 113: <> = Bin, > topic:publish(Topic, Msg), > {noreply, State}; topic:publish/2 causes Topic and Msg to escape, so they'll need full-blown term representations. The warning may refer to that, or it may be that since they are sub-binaries of Bin the compiler gives up on having Bin as a delayed sub-binary. From mihai@REDACTED Sat Jul 18 15:20:56 2009 From: mihai@REDACTED (Mihai Balea) Date: Sat, 18 Jul 2009 09:20:56 -0400 Subject: [erlang-questions] Weird problem when using tcp sockets in HTTP mode In-Reply-To: <5EE0B723-C21E-443C-9DF5-2A52CC918F17@hates.ms> References: <5EE0B723-C21E-443C-9DF5-2A52CC918F17@hates.ms> Message-ID: <1A494C2E-1066-4DDA-9E46-47FD35F24552@hates.ms> Steve, Ruslan and Christian: Obviously, you guys are right. Quoting the URI solves the problem. Sorry for the false alarm and thanks for helping me out Mihai PS: I guess I shouldn't attempt to figure out bugs late on a friday evening :) On Jul 17, 2009, at 5:22 PM, Mihai Balea wrote: > Hi all, > > I'm hitting a weird problem and I'm wondering if it is a bug > somewhere in the system libs. > > I'm running R13B01. I am opening a tcp socket in http mode and > apparently the HttpUri returned by gen_tcp:recv contains only the > first key-value pair of the query string. > > Here's the code I'm running: > > -module(h). > -compile(export_all). > > run() -> > {ok, LS} = gen_tcp:listen(6969, [{packet, http}, {active, false}, > {reuseaddr, true}, list]), > {ok, S} = gen_tcp:accept(LS), > do_recv(S). > > do_recv(S) -> > {ok, Packet} = gen_tcp:recv(S, 0), > io:format("~p\n", [Packet]), > do_recv(S). > > Here's the query: > curl -v http://localhost:6969/foo/bar?a=1&b=2&c=3 > > And the output of the program: > 3> h:run(). > {http_request,'GET',{abs_path,"/foo/bar?a=1"},{1,1}} > {http_header,24,'User-Agent',undefined, > "curl/7.19.5 (i386-apple-darwin9.7.0) libcurl/7.19.5 > zlib/1.2.3"} > {http_header,14,'Host',undefined,"localhost:6969"} > {http_header,8,'Accept',undefined,"*/*"} > http_eoh > > Am I doing something wrong or is this indeed a bug? > > Thanks, > Mihai > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org From ulf.wiger@REDACTED Sat Jul 18 15:53:52 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 18 Jul 2009 15:53:52 +0200 Subject: [erlang-questions] gen_server2 merge into OTP? In-Reply-To: <20090718103844.GA25084@wellquite.org> References: <4A612709.6060503@alertlogic.net> <4A618B73.9030104@erlang-consulting.com> <20090718103844.GA25084@wellquite.org> Message-ID: <4A61D3F0.30209@erlang-consulting.com> Matthew Sackman wrote: > > The purpose of gen_server2 is a) to allow message priorities to be used; > b) to make sure that when you do a call to another process, you don't > risk having to scan a huge mailbox when the reply comes back because the > mailbox is always kept small as it's constantly drained into our own > priority queue. Perhaps this would be a good time for someone well versed in the implementation to expand on exactly what's behind this: "OTP-7979 gen_server:call/2,3 will be somewhat faster if the calling process has a many messages in its message queue." (From the OTP R13B01 release notes.) I've browsed the sources to some extent, but it's not obvious where this optimization is done or what it entails. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From matthew@REDACTED Sat Jul 18 16:13:44 2009 From: matthew@REDACTED (Matthew Sackman) Date: Sat, 18 Jul 2009 15:13:44 +0100 Subject: [erlang-questions] gen_server2 merge into OTP? In-Reply-To: <4A61D3F0.30209@erlang-consulting.com> References: <4A612709.6060503@alertlogic.net> <4A618B73.9030104@erlang-consulting.com> <20090718103844.GA25084@wellquite.org> <4A61D3F0.30209@erlang-consulting.com> Message-ID: <20090718141344.GC25084@wellquite.org> On Sat, Jul 18, 2009 at 03:53:52PM +0200, Ulf Wiger wrote: > "OTP-7979 gen_server:call/2,3 will be somewhat > faster if the calling process has a many > messages in its message queue." > > (From the OTP R13B01 release notes.) > > I've browsed the sources to some extent, but it's > not obvious where this optimization is done or what > it entails. Agreed. I've just diff'd gen_server.erl between R12B05 and R13B01 and can't see anything relating to this. Matthew From matthew@REDACTED Sat Jul 18 16:56:51 2009 From: matthew@REDACTED (Matthew Sackman) Date: Sat, 18 Jul 2009 15:56:51 +0100 Subject: [erlang-questions] gen_server2 merge into OTP? In-Reply-To: <20090718141344.GC25084@wellquite.org> References: <4A612709.6060503@alertlogic.net> <4A618B73.9030104@erlang-consulting.com> <20090718103844.GA25084@wellquite.org> <4A61D3F0.30209@erlang-consulting.com> <20090718141344.GC25084@wellquite.org> Message-ID: <20090718145651.GD25084@wellquite.org> On Sat, Jul 18, 2009 at 03:13:44PM +0100, Matthew Sackman wrote: > On Sat, Jul 18, 2009 at 03:53:52PM +0200, Ulf Wiger wrote: > > "OTP-7979 gen_server:call/2,3 will be somewhat > > faster if the calling process has a many > > messages in its message queue." It's in gen.erl, do_call/4 and wait_resp_mon/3, where if you diff, you'll see the older versions have additional selective receives. So previously, a call would result in 3 selective receives and now only 1. OTOH, if you have a million messages in your mailbox, a factor of 3 is pretty much irrelevant. Getting that mailbox empty is what's important, which is why gen_server2 helps, because you are not scanning the internal queue. Matthew From ulf.wiger@REDACTED Sat Jul 18 18:16:37 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 18 Jul 2009 18:16:37 +0200 Subject: [erlang-questions] gen_server2 merge into OTP? In-Reply-To: <20090718145651.GD25084@wellquite.org> References: <4A612709.6060503@alertlogic.net> <4A618B73.9030104@erlang-consulting.com> <20090718103844.GA25084@wellquite.org> <4A61D3F0.30209@erlang-consulting.com> <20090718141344.GC25084@wellquite.org> <20090718145651.GD25084@wellquite.org> Message-ID: <4A61F565.50604@erlang-consulting.com> Another thing that has been discussed (well, not much, perhaps) that the reply message cannot possibly be among the old messages, since it contains a unique reference: MRef = erlang:monitor(process, Pid), Pid ! {'$gen_call', {self(),MRef}, Req}, receive {MRef, Reply} -> Reply; {'DOWN', MRef, _, _, Reason} -> error(...) after Time -> error(timeout) end. (This is not exactly how it's implemented, but in principle.) If the compiler could be made to recognize this*, selective receive could perhaps be made efficient in the general case, even with huge message queues. It would obviously only work under certain circumstances, but so it is with many other optimizations too. BR, Ulf W * The 'this' that the compiler would have to recognize is that the receive above is the very first receive after checking out the unique reference, and there is no message in the receive clause that doesn't contain this reference. Calling a remote function before entering receive would render this impossible, for example. Matthew Sackman wrote: > On Sat, Jul 18, 2009 at 03:13:44PM +0100, Matthew Sackman wrote: >> On Sat, Jul 18, 2009 at 03:53:52PM +0200, Ulf Wiger wrote: >>> "OTP-7979 gen_server:call/2,3 will be somewhat >>> faster if the calling process has a many >>> messages in its message queue." > > It's in gen.erl, do_call/4 and wait_resp_mon/3, where if you diff, you'll > see the older versions have additional selective receives. So > previously, a call would result in 3 selective receives and now only 1. > OTOH, if you have a million messages in your mailbox, a factor of 3 is > pretty much irrelevant. Getting that mailbox empty is what's important, > which is why gen_server2 helps, because you are not scanning the > internal queue. > > Matthew -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ro4tub@REDACTED Sun Jul 19 06:01:10 2009 From: ro4tub@REDACTED (Oscar) Date: Sun, 19 Jul 2009 12:01:10 +0800 Subject: how to use timer:apply_interval/4 Message-ID: <9a77767b0907182101k3b3ad568o665fabf7b0081b8d@mail.gmail.com> 1 #!/usr/bin/env escript 2 3 do_task(A) -> 4 io:format("~w: now=~w~n", [A, now()]). 5 6 main(_) -> 7 io:format("starting \n"), 8 {ok, _} = timer:apply_interval(80, ?MODULE, do_task,["info"]), 9 timer:sleep(100), 10 io:format("ending \n"). hkf@REDACTED:~/work/erlang/flower$ ./timer_main.erl ./timer_main.erl:3: Warning: function do_task/1 is unused starting =ERROR REPORT==== 19-Jul-2009::11:57:29 === Error in process <0.33.0> with exit value: {undef,[{'escript__timer_main.erl__1247__975849__641609',do_task,["info"]}]} ending I have two questions: 1. how to explain the error log? 2. how to get rid of the warning ''function do_task/1 is unused'? From steven.charles.davis@REDACTED Sun Jul 19 09:12:15 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 19 Jul 2009 00:12:15 -0700 (PDT) Subject: The meaning of ERTS CXC 138 10 Message-ID: I have looked around the docs and source code and didn't find the answer (though I may have missed it). I have frequently wondered about the *exact* meanings of "CXC" and the versioning numbers that follow (as returned by application:loaded_applications()) If anybody has a moment to clarify for me I'd be grateful. br /sd From bgustavsson@REDACTED Sun Jul 19 12:03:31 2009 From: bgustavsson@REDACTED (Bjorn Gustavsson) Date: Sun, 19 Jul 2009 12:03:31 +0200 Subject: [erlang-questions] binary optimization In-Reply-To: <824B7879-0C0F-4236-A3FB-84FCFB995CB9@gmail.com> References: <19041.38179.901233.130939@pilspetsen.it.uu.se> <4A61AC2E.2050107@it.uu.se> <19041.46306.345021.701328@pilspetsen.it.uu.se> <824B7879-0C0F-4236-A3FB-84FCFB995CB9@gmail.com> Message-ID: <6672d0160907190303o13826ee4oc6e291ffa4ceb69@mail.gmail.com> On Sat, Jul 18, 2009 at 1:46 PM, Joel Reymont wrote: > What is a delayed sub-binary representation? Is there a paper on this? There is some information about it in the Efficiency Guide: http://www.erlang.org/doc/efficiency_guide/binaryhandling.html#4 /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From ulf.wiger@REDACTED Sun Jul 19 12:39:58 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 19 Jul 2009 12:39:58 +0200 Subject: SV: [erlang-questions] The meaning of ERTS CXC 138 10 Message-ID: Ericsson is big on 3-letter abbreviations. CXC is an internal Ericsson product type code, and the 5-digit number that follows is a reference in the product database. BR, Ulf W -- originalmedd. -- ?mne: [erlang-questions] The meaning of ERTS CXC 138 10 Fr?n: Steve Davis Datum: 2009.07.19 09.12 I have looked around the docs and source code and didn't find the answer (though I may have missed it). I have frequently wondered about the *exact* meanings of "CXC" and the versioning numbers that follow (as returned by application:loaded_applications()) If anybody has a moment to clarify for me I'd be grateful. br /sd ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From steven.charles.davis@REDACTED Sun Jul 19 12:55:21 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 19 Jul 2009 05:55:21 -0500 Subject: SV: [erlang-questions] The meaning of ERTS CXC 138 10 In-Reply-To: References: Message-ID: <4A62FB99.8080808@gmail.com> Hi Ulf, Thanks for satisfying my curiosity. It was one of those things that whilst I knowing that it was essentially irrelevant to whatever I was doing, it always made me wonder. Best regards, Steve Ulf Wiger wrote: > Ericsson is big on 3-letter abbreviations. CXC is an internal Ericsson product type code, and the 5-digit number that follows is a reference in the product database. > > BR, > Ulf W > > -- originalmedd. -- > ?mne: [erlang-questions] The meaning of ERTS CXC 138 10 > Fr?n: Steve Davis > Datum: 2009.07.19 09.12 > > I have looked around the docs and source code and didn't find the > answer (though I may have missed it). > > I have frequently wondered about the *exact* meanings of "CXC" and the > versioning numbers that follow (as returned by > application:loaded_applications()) > > If anybody has a moment to clarify for me I'd be grateful. > > br > /sd > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From bflatmaj7th@REDACTED Sun Jul 19 13:08:03 2009 From: bflatmaj7th@REDACTED (Richard Andrews) Date: Sun, 19 Jul 2009 21:08:03 +1000 Subject: [erlang-questions] how to use timer:apply_interval/4 In-Reply-To: <9a77767b0907182101k3b3ad568o665fabf7b0081b8d@mail.gmail.com> References: <9a77767b0907182101k3b3ad568o665fabf7b0081b8d@mail.gmail.com> Message-ID: <7702c0610907190408y599475bk17cef7adad36b6b4@mail.gmail.com> An escript has no module. do_task would need to be exported and it would need to be exported from a module. As a result there is no way to call it. Similarly the error comes from the fact that there is no module and as a result ?MODULE expands to (presumably) some internal implementation value inserted by escript. HTH -- Rich On Sun, Jul 19, 2009 at 2:01 PM, Oscar wrote: > ?1 #!/usr/bin/env escript > ?2 > ?3 do_task(A) -> > ?4 ? ? io:format("~w: now=~w~n", [A, now()]). > ?5 > ?6 main(_) -> > ?7 ? ? io:format("starting \n"), > ?8 ? ? {ok, _} = timer:apply_interval(80, ?MODULE, do_task,["info"]), > ?9 ? ? timer:sleep(100), > ?10 ? ? io:format("ending \n"). > > > > hkf@REDACTED:~/work/erlang/flower$ ./timer_main.erl > ./timer_main.erl:3: Warning: function do_task/1 is unused > starting > > =ERROR REPORT==== 19-Jul-2009::11:57:29 === > Error in process <0.33.0> with exit value: > {undef,[{'escript__timer_main.erl__1247__975849__641609',do_task,["info"]}]} > > ending > > > > > I have two questions: > 1. how to explain the error log? > 2. how to get rid of the warning ''function do_task/1 is unused'? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From dmitriid@REDACTED Sun Jul 19 13:59:53 2009 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Sun, 19 Jul 2009 14:59:53 +0300 Subject: Beepbeep is now fully ewgi-compliant Message-ID: I would like to announce that I've pushed the latest changes to Beepbeep which make it the first (?) publicly available web-framework to support ewgi. See this version's changes: http://github.com/dmitriid/beepbeep/commit/b27ac8a359968d1dbf67e7d17d3ea669211f9bb6 and docs: http://beepbeep.dmitriid.com/documentation/middleware/ What's ewgi you ask? ewgi is Erlang's answer to Python's wsgi, http://www.python.org/dev/peps/pep-0333/ , and Ruby's Rack, http://rack.rubyforge.org/ It is a minimal interface between the webserver and you app. It lets you applicaton to be completely server-agnostic. More importantly, it allows chainable middleware. See Django's middleware, http://docs.djangoproject.com/en/dev/topics/http/middleware/ and ewgi example middleware, http://github.com/skarab/ewgi_examples ewgi's authors would like to see ewgi spec, http://code.google.com/p/ewgi/wiki/EWGISpecification , to become EEP in the future. However, there are not too many people talking about ewgi for some odd reason. Even though it's already being used in production: http://smarkets.files.wordpress.com/2009/07/huntermorris-erlangfactory.pdf From prikrutil@REDACTED Sun Jul 19 21:11:36 2009 From: prikrutil@REDACTED (Sergey Samokhin) Date: Sun, 19 Jul 2009 12:11:36 -0700 Subject: Is it safe to use both 5.7.0 and 5.7.1 VMs in one cluster? Message-ID: Hello! Is it safe to build an erlang cluster from aplications running under different VMs: say, 5.7.1 and 5.7.0? Let's suppose that I don't make use any specific features introduced in 5.7.1. The only things I'm interested in are communication by using standard RPC functions, mnesia compatibility (e.g. will using transactions still be safe when working in in such a "heterogeneous" cluster?) and standard net_kernel features set. Will there be any problems if I decide to mix VMs with different versions? Programming Erlang book says that we should avoid doing that: "Make sure that both systems have the same version of the code And the same version of Erlang. If you don?t do this, you?ll get serious and mysterious errors." Buy I've heard that there is backward compatibility between the two latest major releases. Thanks. -- Sergey Samokhin From prikrutil@REDACTED Sun Jul 19 21:45:39 2009 From: prikrutil@REDACTED (Sergey Samokhin) Date: Sun, 19 Jul 2009 12:45:39 -0700 Subject: Is it safe to use both 5.7.0 and 5.7.1 VMs in one cluster? In-Reply-To: References: Message-ID: I can explain why I need to be sure that a cluster I've built on top of different VMs is working right and when I may have different versions of VM working together. The only way I see to move my cluster between defferent releases of Erlang/OTP without taking it out of service is by stopping one by one all the nodes involved with further upgrading emulators they are using. So to upgrade the whole cluster I should do something like this: 1) [5.7.0]---[5.7.0] 2) [5.7.1]---[5.7.0] 3) [5.7.1]---[5.7.1] Before stopping a node one should make sure that functionality it provides is available on another node (if necessary). -- Sergey Samokhin From dave.pawson@REDACTED Mon Jul 20 09:52:38 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Mon, 20 Jul 2009 08:52:38 +0100 Subject: Glossary Message-ID: <711a73df0907200052y3c463de5o62acacd3a8995d86@mail.gmail.com> New to Erlang. Just reading the new O'Reilly book (very good btw). I'm a little unsure on terminology. Node Process Resource Task Event Is there anywhere where these are made clear (assuming an accepted definition is available!). I have my own understanding, but I'd prefer a more widely accepted definition. TIA -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From bflatmaj7th@REDACTED Mon Jul 20 10:15:37 2009 From: bflatmaj7th@REDACTED (Richard Andrews) Date: Mon, 20 Jul 2009 18:15:37 +1000 Subject: [erlang-questions] Glossary In-Reply-To: <711a73df0907200052y3c463de5o62acacd3a8995d86@mail.gmail.com> References: <711a73df0907200052y3c463de5o62acacd3a8995d86@mail.gmail.com> Message-ID: <7702c0610907200115m6347ac61v362c74d917b69aba@mail.gmail.com> On Mon, Jul 20, 2009 at 5:52 PM, Dave Pawson wrote: > New to Erlang. Just reading the new O'Reilly book (very good btw). > I'm a little unsure on terminology. I'll have a go... > Node It's an operating system process. A virtual machine which contains processes. Large systems off consist of several nodes distributed across many hosts for redundancy, load balancing or some other reason. Processes can send messages between themselves across nodes and across hosts (network transparent message passing). > Process A light-weight state machine with a mailbox for receiving messages from other processes or system IO resources. In java you might know them as green threads. The node schedules processes to run when there is something for them to do (like a message in the mailbox). > Resource Memory, socket, etc. Internally might also refer to some erlang specific bits and pieces like atoms and binaries (which you'll get to). > Task Ambiguous. Need context. This isn't an erlang-specific term I know. > Event Usually this means a message that was delivered to a process mailbox. From dave.pawson@REDACTED Mon Jul 20 10:28:39 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Mon, 20 Jul 2009 09:28:39 +0100 Subject: [erlang-questions] Glossary In-Reply-To: <7702c0610907200115m6347ac61v362c74d917b69aba@mail.gmail.com> References: <711a73df0907200052y3c463de5o62acacd3a8995d86@mail.gmail.com> <7702c0610907200115m6347ac61v362c74d917b69aba@mail.gmail.com> Message-ID: <711a73df0907200128x17e78cber35ab6457d2cba57c@mail.gmail.com> 2009/7/20 Richard Andrews : > On Mon, Jul 20, 2009 at 5:52 PM, Dave Pawson wrote: >> New to Erlang. Just reading the new O'Reilly book (very good btw). >> I'm a little unsure on terminology. > > I'll have a go... erlang.org/doc/glossary.html does it. I'm sure Maxim could have been more sarcastic though. > > >> Node > > It's an operating system process. A virtual machine which contains > processes. Large systems off consist of several nodes distributed > across many hosts for redundancy, load balancing or some other reason. > Processes can send messages between themselves across nodes and across > hosts (network transparent message passing). Node = (approx) a single processor/computer/machine? That's how I had it. Then one of those is 'the erlang node', i.e. where whole programs are started from. > >> Process > > A light-weight state machine with a mailbox for receiving messages > from other processes or system IO resources. In java you might know > them as green threads. The node schedules processes to run when there > is something for them to do (like a message in the mailbox). This seems the key bit. Yet least natural to get hold of. Receiving messages is a key part. Is it true that a process is generally a single module ('a chunk of code')? Even if code from other modules is used? And what's the relationship between scheduling, the VM and processes/modules? > >> Resource > > Memory, socket, etc. > Internally might also refer to some erlang specific bits and pieces > like atoms and binaries (which you'll get to). Less understood here. I guess resource is a bit vague! >From the glossary: Resource The actual resource to be managed. A resource is represented by a Managed Object. Each resource is mapped to one or several resources. which isn't much better/clearer. > >> Task > > Ambiguous. Need context. This isn't an erlang-specific term I know. The book doesn't have it indexed, I'll put that one down to 'a computing task to be executed' i.e. book specific :-) I wasn't sure if it was Erlang based. > >> Event > > Usually this means a message that was delivered to a process mailbox. Thanks. That's how I had it. Tks Richard. regards -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From masse@REDACTED Mon Jul 20 11:10:29 2009 From: masse@REDACTED (mats cronqvist) Date: Mon, 20 Jul 2009 11:10:29 +0200 Subject: Is it safe to use both 5.7.0 and 5.7.1 VMs in one cluster? In-Reply-To: (Sergey Samokhin's message of "Sun\, 19 Jul 2009 12\:11\:36 -0700") References: Message-ID: <87prbven3u.fsf@sterlett.hq.kred> Sergey Samokhin writes: > Hello! > > Is it safe to build an erlang cluster from aplications running under > different VMs: say, 5.7.1 and 5.7.0? Historically, VM's have been backwards compatible with the previous major version. So R14's are pretty much guaranteed to be able to talk to R13's, but might not be able to talk to R12's. (Just an example, I have no idea what R14 will actually do.) mats From bengt.kleberg@REDACTED Mon Jul 20 13:19:02 2009 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 20 Jul 2009 13:19:02 +0200 Subject: link: XML co-author likes Erlang Message-ID: <1248088742.4874.4.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, Not about Erlang, but it gets a mentioning towards the end: http://www.theregister.co.uk/2009/06/25/tim_bray_web_today_tomorrow bengt From bflatmaj7th@REDACTED Mon Jul 20 15:18:29 2009 From: bflatmaj7th@REDACTED (Richard Andrews) Date: Mon, 20 Jul 2009 23:18:29 +1000 Subject: [erlang-questions] Glossary In-Reply-To: <711a73df0907200128x17e78cber35ab6457d2cba57c@mail.gmail.com> References: <711a73df0907200052y3c463de5o62acacd3a8995d86@mail.gmail.com> <7702c0610907200115m6347ac61v362c74d917b69aba@mail.gmail.com> <711a73df0907200128x17e78cber35ab6457d2cba57c@mail.gmail.com> Message-ID: <7702c0610907200618l5cdd0345m4a4fa11dd2ec912d@mail.gmail.com> Apologies. This got long. I think it is reasonably accurate. Others will correct my mistakes I'm sure. >>> Node >> >> It's an operating system process. A virtual machine which contains >> processes. Large systems off consist of several nodes distributed >> across many hosts for redundancy, load balancing or some other reason. >> Processes can send messages between themselves across nodes and across >> hosts (network transparent message passing). > > Node = (approx) a single processor/computer/machine? > That's how I had it. ?Then one of those is 'the erlang node', i.e. where > whole programs are started from. Hmmm. A node is a program/process started in the operating system on a machine. You would see it as a task under Windows task manager or a process in UNIX top. It is a virtual machine program for running erlang code in. When a node program starts it is configured to internally start various erlang processes based on what that node is designed to achieve. Erlang nodes can start other nodes, but systems don't always emanate from a nucleus. It can be detrimental to fault tolerance (which is a big reason for using erlang). >>> Process >> >> A light-weight state machine with a mailbox for receiving messages >> from other processes or system IO resources. In java you might know >> them as green threads. The node schedules processes to run when there >> is something for them to do (like a message in the mailbox). > > This seems the key bit. Yet least natural to get hold of. > Receiving messages is a key part. An erlang program as a whole is event driven. An external event (eg. IO or timer) will cause a message to arrive at a process (eg. data from a socket) which will cause other messages to flow between processes, aggregating data, checking what should be done. Some processes are connected to the outside world (eg. network socket, file handle) and data will flow out of the erlang node via those. Erlang was designed for highly asynchronous applications with lots of partially completed tasks running concurrently and safely (not interfering with each other). Because erlang processes CANNOT access the information stored in any other process, the only way to get to that info is to ask nicely and wait for the response message. This might seem inefficient to a C coder who would normally just dereference the memory location but it removes a litany of programming problems: * All data access is sequential (no race conditions, no mutex deadlocks). * All data access is via a uniform mechanism so extending access to another node or across the globe is very easy and reliable. * Because state cannot be shared, when a process suffers a terminal fault (eg. unhandled situation), only that small process is killed. The rest of the system can be guaranteed to be unaffected and keeps on ticking. So what would typically be a segfault or abort in a C program becomes an internal process restart. Errors are contained and the system keeps running. There are no(?) blocking calls in erlang. AFAIK those that look like blocking calls are artificial constructs which wait for a particular event to be delivered. > Is it true that a process is generally a single module ('a chunk of code')? > Even if code from other modules is used? > And what's the relationship between scheduling, the VM and processes/modules? A process is started by specifying to the erlang node the module+function+arguments to start the process running (called spawning). This is often abbreviated to MFA (Module/Function/Arguments). A process runs until the code calls the exit function (or gets killed). OTP (which you are probably using) provides some common boiler-plate process templates like gen_server. These use one module to drive core process behaviour. So in this respect you are correct. And yes from that core module the code can run code from other modules. The erlang VM like an OS kernel it receives events when there is work for a process to perform (eg. timer expires, file descriptor/handle gets data, etc) and manages swapping CPU time between the internal erlang processes to achieve this goal. Modules are just containers of code which processes call as they run. You can think of modules as shared libraries that all the erlang processes can use. Unsolicited advice: Erlang programming syntax can seem just plain wrong to a newcomer. If you hit one of these issues where you think erlang must be the dumbest most inefficient language on earth, ask about it. Erlang is different, but it works the way it does for very good reasons. -- Rich From thomasl_erlang@REDACTED Mon Jul 20 15:25:19 2009 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 20 Jul 2009 06:25:19 -0700 (PDT) Subject: [erlang-questions] Any HTTP 1.0 clients out there? In-Reply-To: <990c88ad-c0a5-44ff-a87e-3b5c1c17ce77@s6g2000vbp.googlegroups.com> References: <990c88ad-c0a5-44ff-a87e-3b5c1c17ce77@s6g2000vbp.googlegroups.com> Message-ID: <483238.79562.qm@web111405.mail.gq1.yahoo.com> ----- Original Message ---- > From: Steve Davis > To: erlang-questions@REDACTED > Sent: Sunday, July 12, 2009 6:46:04 AM > Subject: [erlang-questions] Any HTTP 1.0 clients out there? > > This is not explicitly erlang related - but the telecoms expertise of > many on the list is, I think, likely to yield a reliable answer. > > Are there any *commercially relevant* HTTP/1.0 clients left out there? > I've not seen any discussions on the web about 1.0 since the > mid-2000s. That being the case -- is it reasonable to say that HTTP/ > 1.0 is not just outdated but effectively dead? This is perhaps a bit late to the party, but nginx proxies use HTTP/1.0 to the backend. http://wiki.nginx.org/NginxHttpProxyModule Best, Thomas From rvirding@REDACTED Mon Jul 20 16:20:47 2009 From: rvirding@REDACTED (Robert Virding) Date: Mon, 20 Jul 2009 16:20:47 +0200 Subject: [erlang-questions] suggestion for somebody who has free time :-) In-Reply-To: <91a2ba3e0907171651o6c27e15crbf57643d6fe2ae9@mail.gmail.com> References: <91a2ba3e0907171651o6c27e15crbf57643d6fe2ae9@mail.gmail.com> Message-ID: <3dbc6d1c0907200720w5eb336e1gc12d8cc9e32eaf6a@mail.gmail.com> 2009/7/18 Raoul Duke > "super productive lfe in a box" (along the lines of "clojure in a box" > et. al.) consisting of: > > - erlang (duh) > - lfe > - dialyzer > - emacs as the ide probably > - project (makefiles) set up such that dialyzer and unit tests get > auto-run as you are live editing code > - heck for an extra fee also quick check integrated? > > :) Slowly getting there. :-) Been working on leex during the summer but when that is done I plan to do some work on lfe-mode so that it works for running lfe in emacs. Dialyzer has not yet seen the light and can't handle lfe. :-) Robert From dave.pawson@REDACTED Mon Jul 20 16:39:12 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Mon, 20 Jul 2009 15:39:12 +0100 Subject: [erlang-questions] Glossary In-Reply-To: <7702c0610907200618l5cdd0345m4a4fa11dd2ec912d@mail.gmail.com> References: <711a73df0907200052y3c463de5o62acacd3a8995d86@mail.gmail.com> <7702c0610907200115m6347ac61v362c74d917b69aba@mail.gmail.com> <711a73df0907200128x17e78cber35ab6457d2cba57c@mail.gmail.com> <7702c0610907200618l5cdd0345m4a4fa11dd2ec912d@mail.gmail.com> Message-ID: <711a73df0907200739ta43ce79re9f5c5619a5f11bd@mail.gmail.com> 2009/7/20 Richard Andrews : > Apologies. This got long. I think it is reasonably accurate. Since it's just that bit different from other system+language setup, I think it's worth getting the understanding right. Tks for the patience. >>>> Node >> Node = (approx) a single processor/computer/machine? >> That's how I had it. ?Then one of those is 'the erlang node', i.e. where >> whole programs are started from. > > Hmmm. A node is a program/process started in the operating system on a > machine. You would see it as a task under Windows task manager or a > process in UNIX top. So not necessarily 'one per CPU'? With your definition I can see I could start several 'programs' from one machine. OK. > It is a virtual machine program for running > erlang code in. That's clear! When a node program starts it is configured to > internally start various erlang processes based on what that node is > designed to achieve. Erlang nodes can start other nodes, but systems > don't always emanate from a nucleus. It can be detrimental to fault > tolerance (which is a big reason for using erlang). OK, I'm happy with that one. > > >>>> Process >>> >>> A light-weight state machine with a mailbox for receiving messages >>> from other processes or system IO resources. In java you might know >>> them as green threads. The node schedules processes to run when there >>> is something for them to do (like a message in the mailbox). >> >> This seems the key bit. Yet least natural to get hold of. >> Receiving messages is a key part. > > An erlang program as a whole is event driven. [A program being a number of processes?] An external event (eg. > IO or timer) will cause a message to arrive at a process (eg. data > from a socket) which will cause other messages to flow between > processes, aggregating data, checking what should be done. Some > processes are connected to the outside world (eg. network socket, file > handle) and data will flow out of the erlang node via those. I'm clear with the messaging idea as implementing events. > > Erlang was designed for highly asynchronous applications with lots of > partially completed tasks running concurrently and safely (not > interfering with each other). Because erlang processes CANNOT access > the information stored in any other process, the only way to get to > that info is to ask nicely and wait for the response message. I like that clean interface. One of the Erlang differences I guess. This > might seem inefficient to a C coder I've done similar things in Assembler, and suffered the failures ;-) > ?* Because state cannot be shared, when a process suffers a terminal > fault (eg. unhandled situation), only that small process is killed. > The rest of the system can be guaranteed to be unaffected and keeps on > ticking. So what would typically be a segfault or abort in a C program > becomes an internal process restart. Errors are contained and the > system keeps running. I read that in the O'Reilly book. I've yet to play with it. Propogating errors to an appropriate handler level makes sense. >> Is it true that a process is generally a single module ('a chunk of code')? >> Even if code from other modules is used? >> And what's the relationship between scheduling, the VM and processes/modules? > > A process is started by specifying to the erlang node the > module+function+arguments to start the process running (called > spawning). This is often abbreviated to MFA > (Module/Function/Arguments). A process runs until the code calls the > exit function (or gets killed). Or hangs waiting for a message? OTP (which you are probably using) > provides some common boiler-plate process templates like gen_server. > These use one module to drive core process behaviour. So in this > respect you are correct. And yes from that core module the code can > run code from other modules. Thanks. > > The erlang VM like an OS kernel it receives events when there is work > for a process to perform (eg. timer expires, file descriptor/handle > gets data, etc) and manages swapping CPU time between the internal > erlang processes to achieve this goal. Modules are just containers of > code which processes call as they run. You can think of modules as > shared libraries that all the erlang processes can use. I'm good with that. My background I've written schedulers for real time systems which implement that 'kernel' type functionality. I guess I can ignore the VM when thinking about design, just let it do it's thing. > > > Unsolicited advice: Erlang programming syntax can seem just plain > wrong to a newcomer. I'll settle for 'odd' :-) Having played with Scheme and Lisp, it isn't much worse! If you hit one of these issues where you think > erlang must be the dumbest most inefficient language on earth, ask > about it. Erlang is different, but it works the way it does for very > good reasons. It seems to have that quirkiness that comes from doing a specific job? The O'Reilly book contains quite a bit of 'experience' put forth as advice and some solid logic as to why some things are as they are. I'm still quite intrigued by it. Again. Tks Richard. regards -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From thomasl_erlang@REDACTED Mon Jul 20 17:42:29 2009 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 20 Jul 2009 08:42:29 -0700 (PDT) Subject: [erlang-questions] comma vs andalso In-Reply-To: <4A548BAD.1030401@it.uu.se> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> <20090707041622.GB28401@hijacked.us> <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> <769861.4108.qm@web111415.mail.gq1.yahoo.com> <4A548BAD.1030401@it.uu.se> Message-ID: <782181.62770.qm@web111416.mail.gq1.yahoo.com> Sorry for dropping my end of this; here we go. ----- Original Message ---- > From: Richard Carlsson > > Thomas Lindgren wrote: > > Guards are an awful mess in Erlang. I'd be quite surprised if the > > above difference in behaviour was intentional. If so, what possible > > purpose does it serve? > > It is intentional in the respect that 'andalso'/'orelse' behave just > like any other operator in a guard test. As for purpose, all Erlang's > operators are allowed in guard expressions, so why make an exception? First, note that guards as such already are a limited form of boolean formulae, where comma is interpreted as conjunction/and, and semicolon is interpreted as disjunction/"or". Evaluation of guards was, I would suppose, inspired by logic programming: if the formula could be satisfied, the guard succeeded; if not, it simply failed. Thus, a guard like 1/X > Z/Y would not throw an exception when X or Y was zero, but simply fail (because the test is not true). For reasons unknown (except to some of the very senior members of this mailing list), the initial form of guards only permitted conjunctions of expressions G = E1,E2,E3 ... Some time later, a limited form of disjunction was added, permitting us to write (G1 ; G2 ; ... ; Gn) where the Gi were conjunctions, but still not permitting arbitrary boolean formulae as guards: no disjunctions inside conjunctions, in particular. As I recall, this was because of parser problems. When and/or was being added to guards, I do recall suggesting that general boolean guards should be permitted, to be written using nested and/or. (This was discussed sometime in the 1997-1999 timeframe, right?) But they weren't. > > At one point, you could also use "and"/"or", the boolean operators > > manque, in guards. (I haven't checked recently whether this is still > > the case.) So we then have three very similar sets of guard > > operators. > > 'and'/'or' have always (as far back as I can remember, anyway) been > allowed in guards, again, probably simply by virtue of being general > operators in the language. And they don't behave like ','/';' either. Well, to be precise they have only been allowed since some time after and/or were introduced into the language. (At the time when lots of stuff was added, such as funs, records and so on.) I mention this because I think the mess arises from stuff having been added incrementally with different intents by various people over the years. Moving on, no, indeed they don't behave the same, and I do consider that a problem. As a consequence, we have three subtly different ways to write boolean formulae in guards. (Sorry about the jargon, dear readers, I'm trying to stay away from "boolean expression" here since we already are using "expression" in another sense) And unfortunately, at this point, _none_ of the three ways actually permit writing full boolean formulae. I consider this a failure in how guards are designed, which is why I'm always complaining about it. > > Not to mention the twin set of type tests, introduced, as far as I > > know, to get rid of the lone double entendre of float/1 (as a test or > > conversion not the most frequent of operations). > > That was one details, yes, but the main reason was to make it possible > to refactor a piece of code without being forced to change the names > of the function tests if you moved an expression from within a guard > to an ordinary expression and vice versa. Recall that before the is_... > versions, there was no way of saying e.g., "Bool = (is_integer(X) and > is_atom(Y))" without writing a case/if or a separate predicate function. I seem to recall boolean BIFs for old-style type tests (atom/1, integer/1, ...) in older Erlangs, which would permit one to write (atom(X) and atom(Y)) in clause bodies. Perhaps I misremember; still, there is no technical reason to avoid them in favour of the longer new names, _except_ possibly the clash with float/1. In that regard, I would argue that it would have been far easier to rename the lone float conversion operation rather than all type tests. > The old type tests didn't correspond to any built-in functions, so you > had to treat them as "primops" inside the compiler, you couldn't refer > to them and pass them around, etc. But the old type test names "atom(X)" > and so forth could not simply be made to work outside guards because > there would be name clashes (with the float(X) bif and with any existing > code that defined a function such as list(X) or integer(X)), hence the > is_... prefix for the generally usable versions that are proper built-in > functions (defined in the 'erlang' module along with all the others). But this doesn't solve the problem -- it merely shifts name clashes to another part of the name space. Nor is there anything inherently impossible about defining and providing the BIF erlang:atom/1 instead of erlang:is_atom/1. So, to conclude: it seems to me as if keeping the short names would have been just about the same as the current approach, except saving three characters per type test. > > And now, for our convenience, the shorter form of these tests is being > > deprecated. > > Hold your horses - nobody is deprecating the use of ',' and ';'. (I was talking about the short type tests here.) > This fail-to-false > behaviour was in my opinion a mistake, because it occasionally hides > a bug in the guard and turns what should have been an observable runtime > crash into a silent "well, we take the next clause then". Some people > like to use this as a trick to write more compact guards, but that > makes it hard for someone reading the code to tell whether the > crash-jumps-to-the-next-case is intentional or not. See above for what I would think is the reasoning behind the classic semantics. The main drawback of explicit crashes in guards is that as a guard-writer you don't have a lot of opportunities to catch them. To catch and hide explicit crashes, you may then have to turn clause guards into case-expressions. (In this context, I'm tempted to instead make an argument for fullblown expressions in guards, but let's leave that little hairball for another day.) > ... I'm not sure I have any real arguments against nesting of ','/';', > but I fear the grammar could get rather messy, and that such nested > guards could be quite difficult to read in practice. Well, let me then register my strong vote for actually, finally implementing full boolean formulae in guards. Rather than making the code harder to read, it will become easier: there is no need to code around the issue when you actually need to compose disjunctions, and the tests themselves will be hidden in well-formed macros. Macros which as a bonus can be composed fairly nicely without obscure parsing errors. And, in contrast with using and/or/andalso/orelse, the composed guards will still behave like classic guards. Finally, a question regarding the grammar issue: this seems superficially like adding two more operators to the expression operator precedence grammar. Is there more and worse than that? Best, Thomas From thomasl_erlang@REDACTED Mon Jul 20 17:51:38 2009 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 20 Jul 2009 08:51:38 -0700 (PDT) Subject: [erlang-questions] comma vs andalso In-Reply-To: <3dbc6d1c0907081628j587cb1dfm4f398120e822e126@mail.gmail.com> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> <20090707041622.GB28401@hijacked.us> <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> <769861.4108.qm@web111415.mail.gq1.yahoo.com> <3dbc6d1c0907081628j587cb1dfm4f398120e822e126@mail.gmail.com> Message-ID: <620916.74733.qm@web111405.mail.gq1.yahoo.com> ----- Original Message ---- > From: Robert Virding > To: Thomas Lindgren .... > This functionality was added for just this case. There is logic in the > madness. It was simple before with only simple guard tests, it only became > confusing when logical expressions were allowed in guards. They are > practical but were only added later, THEY are the odd man out. This is also > why originally there were no problems with type tests without is_ in guards. > > Guards were always meant to be part of pattern matching and no more. (This fits my view of the history of guards, fwiw.) I've left a longer version of the argument in a reply to Richard, but basically, I think you would have saved some effort and gained some clarity and expressiveness by first of all implementing nested ","/";" as the connectives. Best, Thomas From thomasl_erlang@REDACTED Mon Jul 20 18:15:43 2009 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 20 Jul 2009 09:15:43 -0700 (PDT) Subject: [erlang-questions] comma vs andalso In-Reply-To: <4A55A3FC.8070109@it.uu.se> References: <4F81D77D-1EED-4C9F-B684-1ABC56DB1880@widetag.com> <482353F1-0191-428F-AB46-BA48809B428B@cs.otago.ac.nz> <20090707041622.GB28401@hijacked.us> <1E14DD28-EF1C-499A-83F0-DC2B540D2D0A@cs.otago.ac.nz> <769861.4108.qm@web111415.mail.gq1.yahoo.com> <4A548BAD.1030401@it.uu.se> <461A85DE-BF04-44FC-B0CF-FFE63979FAB2@cs.otago.ac.nz> <4A55A3FC.8070109@it.uu.se> Message-ID: <311028.24771.qm@web111409.mail.gq1.yahoo.com> ----- Original Message ---- > From: Richard Carlsson >... > > It's _still_ the case that almost no non-trivial expressions > > can be moved into a guard. > > True. But every bit helps. Ive never found "this does not solve all > problems" to be an argument for not making a partial improvement > (as long as it does not create an obstacle for future development). In my opinion a very dangerous attitude in a language like Erlang, where mistakes in design get set in stone. Off the top of my head: 1. records eternally forced to be tuples because of the chosen API; 2. and/or being strict, meaning we had to introduce andalso/orelse; 3. packages vs module names as atoms leading to unsoundness; 4. (the whole guard mess with all its needless duplication, as discussed elsewhere) or the following: .... > Yes, however, there was already a similar rule in place since > ancient times, and it stated that in the case you describe, the > built-in function takes precedence. Bummer. (We are now slowly > trying to phase out this old rule, though, taking baby steps.) I do think that rule has been criticized for being wrong basically since day one. (At least I have considered it an awful bug since I first encountered it; it makes a hollow mockery out of scoping) So it's good to hear it's getting fixed. > Neither necessary nor sufficient, but likely. It's a game of > probabilities. I _had_ seen several existing modules that used the > name list(Xs), and float(X) was already in use as a BIF for casting. > In comparison, the is_ convention was much less likely to cause > clashes (indeed, I recall no reports of any such when we introduced > the new names). And the convention has kept working for those type > tests that were added later, e.g., is_boolean(X), is_bitstring(X). Um, you do realize that the current deprecation of short tests is breaking miles and miles of code? We're hardly taking the path of least resistance here. Best, Thomas From thomasl_erlang@REDACTED Mon Jul 20 18:17:30 2009 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 20 Jul 2009 09:17:30 -0700 (PDT) Subject: [erlang-questions] Re: HTTP:set_option() equal function in iBrowse In-Reply-To: <60fcf412-28e9-491e-a883-404d57961adf@s6g2000vbp.googlegroups.com> References: <90b4299d0906182347v4ca3b100o5cca67028a3026f6@mail.gmail.com> <60fcf412-28e9-491e-a883-404d57961adf@s6g2000vbp.googlegroups.com> Message-ID: <824858.48574.qm@web111401.mail.gq1.yahoo.com> ----- Original Message ---- > From: Chandru > > Cookie management is not supported in ibrowse. I firmly believe that > it is a function of the application using HTTP, not the HTTP client > itself. Clearly someone enterprising should write a wrapper! Best, Thomas From thomasl_erlang@REDACTED Mon Jul 20 18:22:30 2009 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 20 Jul 2009 09:22:30 -0700 (PDT) Subject: [erlang-questions] Any interest in an RDS distribution? In-Reply-To: References: Message-ID: <327534.86014.qm@web111411.mail.gq1.yahoo.com> ----- Original Message ---- > From: Chris Tilt > > Hi. I have just written my first Erlang program, which is a port driver for > Infiniband (using our com layer). It is a prototype and not written as a > distribution replacement. It's just a regular port driver. But, the performance > was great. An echo/ping application that uses the port was measured at 20 usec > per 1K buffer (one hop - round trip was 40 usec to echo back the data packet), > which includes the time in and out of the Erlang port owner's process to the > echo server and ping client. .... > We would be happy to share this work back to the community (I say this before > I've spoken with our lawyer :-) as open source. We'd have to research the > license issues, etc, but first things first - let's get it working. Any advice > is welcomed. Or just any cheers of support are welcomed too :-) Cheers! :-) Best, Thomas From thomasl_erlang@REDACTED Mon Jul 20 18:25:06 2009 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 20 Jul 2009 09:25:06 -0700 (PDT) Subject: [erlang-questions] binary optimization In-Reply-To: <7F2503DB-9186-4E51-8D76-269111F27A9D@gmail.com> References: <19041.38179.901233.130939@pilspetsen.it.uu.se> <4A61AC2E.2050107@it.uu.se> <7F2503DB-9186-4E51-8D76-269111F27A9D@gmail.com> Message-ID: <636894.89231.qm@web111412.mail.gq1.yahoo.com> ----- Original Message ---- > From: Joel Reymont > > On Jul 18, 2009, at 12:04 PM, Richard Carlsson wrote: > > > or you try to rewrite the whole thing so you get a single main loop with > > direct self-recursive tail calls (if that is at all doable). > > Wouldn't the tail call of the loop need to be written as ?MODULE:loop to enable > code reloading? > > This would result in the previous "no optimization with apply" warning. I'm not sure this was answered, but: ?MODULE macroexpands to the current module, so the example would be a remote call foo:loop(...), not an apply. Best, Thomas From vances@REDACTED Mon Jul 20 19:30:26 2009 From: vances@REDACTED (Vance Shipley) Date: Mon, 20 Jul 2009 13:30:26 -0400 Subject: Solaris: Can't set long node name! Message-ID: <20090720173026.GD79703@h216-235-12-174.host.egate.net> On my default installs of Solaris 10 simply running 'erl -name foo' doesn't work. I know that I can tune things with settings in an inetrc file(*) and I'm looking at that now but I wonder what others have found works best. This is what I get: $ erl -name foo {error_logger,{{2009,7,20},{18,52,13}},"Can't set long node name!\nPlease check your configuration\n",[]} I wonder whether reporting with error_logger is the right thing to do here too. -- Vance (*) http://erlang.org/doc/apps/erts/inet_cfg.html#7 From thomasl_erlang@REDACTED Mon Jul 20 18:37:10 2009 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 20 Jul 2009 09:37:10 -0700 (PDT) Subject: [erlang-questions] compile transforms which change the module name In-Reply-To: <9b08084c0907100650s22940ed9ob767b5a7a838ff9@mail.gmail.com> References: <9b08084c0907092355t2f649bf6tfea654f971ad991e@mail.gmail.com> <4A5703BD.6060705@erix.ericsson.se> <4A571156.3060208@erlang-consulting.com> <9b08084c0907100336j3b222881y30e16afbab9542c1@mail.gmail.com> <95be1d3b0907100352s7cb2ecd8ka2b9c7e6793f32ab@mail.gmail.com> <9b08084c0907100650s22940ed9ob767b5a7a838ff9@mail.gmail.com> Message-ID: <685742.69532.qm@web111407.mail.gq1.yahoo.com> ----- Original Message ---- > From: Joe Armstrong > > > - what about M:F() calls and M:F/A references sent as data? > > Will break existing code - we need a new bif Mod1 = theModName(Mod) > which converts a module name like "lists" into the correct version "424acfe..." I'd suggest you generate that BIF/function when you are building your big module. > > > - updating a comment will invalidate the map... > > Yes - I'm not sure about this. I could make a check sum from > epp:parse_file then remove the > line numbers. You mean hashing after normalizing line numbers (e.g., setting them all to zero)? That worked well for a similar application, when applied to record definitions. (The goal at that time was to transparently permit many clashing record declarations, included from all sorts of places not known at compile-time, to coexist.) > > - it would be nice to have tools to do the reverse transformation for > > logs and error messages; also, to be able to find the right version of > > my_origonal_name() -> > xxxx. Note that if you are merging modules, this approach _will_ encounter name clashes. Best, Thomas From attila.r.nohl@REDACTED Mon Jul 20 19:48:09 2009 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 20 Jul 2009 19:48:09 +0200 Subject: [erlang-questions] Solaris: Can't set long node name! In-Reply-To: <20090720173026.GD79703@h216-235-12-174.host.egate.net> References: <20090720173026.GD79703@h216-235-12-174.host.egate.net> Message-ID: <401d3ba30907201048v1a6cbbck5d33f48d82ffcf2a@mail.gmail.com> 2009/7/20, Vance Shipley : > On my default installs of Solaris 10 simply running 'erl -name foo' > doesn't work. I know that I can tune things with settings in an > inetrc file(*) and I'm looking at that now but I wonder what others > have found works best. > > This is what I get: > > $ erl -name foo > {error_logger,{{2009,7,20},{18,52,13}},"Can't set long node > name!\nPlease check your configuration\n",[]} Is name resolving properly setup on that Solaris? From paul-trapexit@REDACTED Mon Jul 20 20:02:59 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Mon, 20 Jul 2009 11:02:59 -0700 (PDT) Subject: [erlang-questions] how to use timer:apply_interval/4 In-Reply-To: <7702c0610907190408y599475bk17cef7adad36b6b4@mail.gmail.com> References: <9a77767b0907182101k3b3ad568o665fabf7b0081b8d@mail.gmail.com> <7702c0610907190408y599475bk17cef7adad36b6b4@mail.gmail.com> Message-ID: You can cheese it by bouncing your apply off erlang:apply/2 which is exported etc. but which can take a fun reference. ------ #! /usr/bin/env escript -mode (compile). do_task (A) -> io:format ("~w: now=~w~n", [ A, now () ]). main (_) -> io:format ("starting \n"), { ok, _ } = timer:apply_interval (80, erlang, apply, [ fun do_task/1, [ "info" ] ]), timer:sleep (100), io:format ("ending \n"). ------ -- p On Sun, 19 Jul 2009, Richard Andrews wrote: > An escript has no module. > > do_task would need to be exported and it would need to be exported > from a module. As a result there is no way to call it. > Similarly the error comes from the fact that there is no module and as > a result ?MODULE expands to (presumably) some internal implementation > value inserted by escript. > > HTH > -- > Rich > > > On Sun, Jul 19, 2009 at 2:01 PM, Oscar wrote: > > ?1 #!/usr/bin/env escript > > ?2 > > ?3 do_task(A) -> > > ?4 ? ? io:format("~w: now=~w~n", [A, now()]). > > ?5 > > ?6 main(_) -> > > ?7 ? ? io:format("starting \n"), > > ?8 ? ? {ok, _} = timer:apply_interval(80, ?MODULE, do_task,["info"]), > > ?9 ? ? timer:sleep(100), > > ?10 ? ? io:format("ending \n"). > > > > > > > > hkf@REDACTED:~/work/erlang/flower$ ./timer_main.erl > > ./timer_main.erl:3: Warning: function do_task/1 is unused > > starting > > > > =ERROR REPORT==== 19-Jul-2009::11:57:29 === > > Error in process <0.33.0> with exit value: > > {undef,[{'escript__timer_main.erl__1247__975849__641609',do_task,["info"]}]} > > > > ending > > > > > > > > > > I have two questions: > > 1. how to explain the error log? > > 2. how to get rid of the warning ''function do_task/1 is unused'? > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From vances@REDACTED Mon Jul 20 20:08:15 2009 From: vances@REDACTED (Vance Shipley) Date: Mon, 20 Jul 2009 14:08:15 -0400 Subject: [erlang-questions] Solaris: Can't set long node name! In-Reply-To: <401d3ba30907201048v1a6cbbck5d33f48d82ffcf2a@mail.gmail.com> References: <20090720173026.GD79703@h216-235-12-174.host.egate.net> <401d3ba30907201048v1a6cbbck5d33f48d82ffcf2a@mail.gmail.com> Message-ID: <20090720180814.GE79703@h216-235-12-174.host.egate.net> Attila, Yes, yes it is. $ /usr/sbin/ping erlang.org erlang.org is alive -- -Vance On Mon, Jul 20, 2009 at 07:48:09PM +0200, Attila Rajmund Nohl wrote: } Is name resolving properly setup on that Solaris? From joelr1@REDACTED Mon Jul 20 20:21:44 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 20 Jul 2009 19:21:44 +0100 Subject: [erlang-questions] binary optimization In-Reply-To: <636894.89231.qm@web111412.mail.gq1.yahoo.com> References: <19041.38179.901233.130939@pilspetsen.it.uu.se> <4A61AC2E.2050107@it.uu.se> <7F2503DB-9186-4E51-8D76-269111F27A9D@gmail.com> <636894.89231.qm@web111412.mail.gq1.yahoo.com> Message-ID: <1D630196-EF94-40C0-90ED-7BB7DA4052F8@gmail.com> On Jul 20, 2009, at 5:25 PM, Thomas Lindgren wrote: > I'm not sure this was answered, but: ?MODULE macroexpands to the > current module, so the example would be a remote call foo:loop(...), > not an apply. I thought a remote call was rpc:call. What am I missing? If foo:loop is not an apply then why does the compiler complain about it? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From praveen.ray@REDACTED Mon Jul 20 20:43:17 2009 From: praveen.ray@REDACTED (Praveen Ray) Date: Mon, 20 Jul 2009 14:43:17 -0400 Subject: CPU not reaching 100% Message-ID: Hi All I've a simple process which in turn spawns 10 more processes - each hitting mnesia ram_tables continuously. As soon as all 10 processes are started, the CPU usage goes up (2 CPUs, 2 cores each). However, none of of the CPUs touches 50% - they all seem to churn at around 43-45%..How do I increase CPU utilization? I'm starting 'erl' without any additional flags and this is what it prints: *$elr* *Erlang R13B (erts-5.7.1) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]* ... Is there a flag I should be passing? Is 'kernel-poll:false' is sometthing I need to worry about? thanks -Praveen -- Yellowfish Technologies Inc http://www.yellowfish.biz praveen.ray@REDACTED (888) 817 2969 x 233 gtalk/skype: praveenray From prikrutil@REDACTED Mon Jul 20 20:53:54 2009 From: prikrutil@REDACTED (Sergey Samokhin) Date: Mon, 20 Jul 2009 11:53:54 -0700 Subject: Does mnesia create temporary ETS tables during transactions? Message-ID: Hi! Is this right that every time I do mnesia:transaction/1 a temprorary ETS table is created? -- Sergey Samokhin From joelr1@REDACTED Mon Jul 20 21:04:30 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 20 Jul 2009 20:04:30 +0100 Subject: httpd and detecting tcp_close Message-ID: <67BCB6CD-24BE-4182-B6DC-70F3F20FD7D8@gmail.com> Is there a way to hook into the tcp_close notification received by httpd? It's absolutely necessary that I get notified when the client end of a long-poll connection is broken but it doesn't seem like httpd provides an API for this. Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From kenneth.lundin@REDACTED Mon Jul 20 21:06:59 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Mon, 20 Jul 2009 21:06:59 +0200 Subject: [erlang-questions] Re: Is it safe to use both 5.7.0 and 5.7.1 VMs in one cluster? In-Reply-To: <87prbven3u.fsf@sterlett.hq.kred> References: <87prbven3u.fsf@sterlett.hq.kred> Message-ID: Hi, On Mon, Jul 20, 2009 at 11:10 AM, mats cronqvist wrote: > Sergey Samokhin writes: > >> Hello! >> >> Is it safe to build an erlang cluster from aplications running under >> different VMs: say, 5.7.1 and 5.7.0? Yes it is safe to do that during an upgrade phase but we don't really recommend running continuously with different versions of the Erlang VM in a cluster using distributed Erlang. > > ?Historically, VM's have been backwards compatible with the previous > ?major version. So R14's are pretty much guaranteed to be able to talk > ?to R13's, but might not be able to talk to R12's. (Just an example, I > ?have no idea what R14 will actually do.) Our strategy has been an still is to be backward compatible with 2 older major versions of Erlang/OTP when it comes to distributed Erlang. This means that Erlang nodes based on R13B (01, 02, ...) will be able to work together with R11B and R12B based nodes and of course R13B works well together with R13B01, 02, ... as well. When R14B is released it will be able to work together with nodes based on R12B and R13B. The interoperability between nodes based on different major versions of the Erlang VM is a necessity for smooth upgrade of a cluster (one node at a time) but the intention is to use this just for the purpose of upgrade and not for normal service of a cluster. Regards Kenneth Erlang/OTP, Ericsson If there are any exceptions to this strategy we will clearly say so in the release notes. > > ?mats > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From wolfmanjm@REDACTED Mon Jul 20 21:17:28 2009 From: wolfmanjm@REDACTED (Jim Morris) Date: Mon, 20 Jul 2009 12:17:28 -0700 (PDT) Subject: httpd and detecting tcp_close In-Reply-To: <67BCB6CD-24BE-4182-B6DC-70F3F20FD7D8@gmail.com> References: <67BCB6CD-24BE-4182-B6DC-70F3F20FD7D8@gmail.com> Message-ID: <7c76fd60-8a93-4446-9e6c-bfcbbef8307d@12g2000pri.googlegroups.com> I'm pretty sure you could hack it to do so, but I don't use httpd. I can tell you how I handled the situation last year when I wrote a Java/Jetty based long poll server that had the same problem. There was no way to hack Jetty to detect the close, so I had to add a timeout for every client on the server. Basically the long poll returned every 30 seconds, and the client would immediately re issue the poll. If the server did not get a long poll from the client within 60 seconds it would presume it was closed. This worked although sub optimal because it could take up to 60 seconds to detect a lost client. However it was the only option I could find. On Jul 20, 12:04?pm, Joel Reymont wrote: > Is there a way to hook into the tcp_close notification received by ? > httpd? > > It's absolutely necessary that I get notified when the client end of a ? > long-poll connection is broken but it doesn't seem like httpd provides ? > an API for this. > > ? ? ? ? Thanks, Joel > > --- > Mac hacker with a performance benthttp://www.linkedin.com/in/joelreymont > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From prikrutil@REDACTED Mon Jul 20 21:38:35 2009 From: prikrutil@REDACTED (Sergey Samokhin) Date: Mon, 20 Jul 2009 12:38:35 -0700 Subject: [erlang-questions] CPU not reaching 100% In-Reply-To: References: Message-ID: Hello. > I've a simple process which in turn spawns 10 more processes - each hitting > mnesia ram_tables continuously. As soon as all 10 processes are started, the > CPU usage goes up (2 CPUs, 2 cores each). However, none of of the CPUs > touches 50% - they all seem to churn at around 43-45%.. I have got some ideas to examine. Some of processes you spawn may blocks in receive statement waiting for a message (note that receive statement may be hidden inside common library functions you're using, say in gen_tcp:recv/2). To know what a process you are interestend in is doing try to call: process_info(Pid, current_function) It may throw light on the place where the problem is. Are your ram_tables replicated across several nodes? If so, you should take into account that having to send some TCP messages per each transaction through a network blocks mnesia:transaction/1 until all the replicas become synchronized. > How do I increase CPU utilization? At first you should find out where your processes spend their time and then try to get rid of those bottlenecks. Posting source code with small example illustrating your problem will be very helpful for those who can say "Aha, I know what actually is wrong!" > Is there a flag I should be passing? Is 'kernel-poll:false' is sometthing I > need to worry about? I don't think that turning kernel polling on (by passing +K to erl) will make your CPU as busy as you want. It's often used when dealing with sousands of sockets. -- Sergey Samokhin From prikrutil@REDACTED Mon Jul 20 21:43:14 2009 From: prikrutil@REDACTED (Sergey Samokhin) Date: Mon, 20 Jul 2009 12:43:14 -0700 Subject: [erlang-questions] Re: Is it safe to use both 5.7.0 and 5.7.1 VMs in one cluster? In-Reply-To: References: <87prbven3u.fsf@sterlett.hq.kred> Message-ID: Hello, Kenneth. Thanks for the detailed answer. -- Sergey Samokhin From thomasl_erlang@REDACTED Mon Jul 20 23:25:29 2009 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 20 Jul 2009 14:25:29 -0700 (PDT) Subject: [erlang-questions] binary optimization In-Reply-To: <1D630196-EF94-40C0-90ED-7BB7DA4052F8@gmail.com> References: <19041.38179.901233.130939@pilspetsen.it.uu.se> <4A61AC2E.2050107@it.uu.se> <7F2503DB-9186-4E51-8D76-269111F27A9D@gmail.com> <636894.89231.qm@web111412.mail.gq1.yahoo.com> <1D630196-EF94-40C0-90ED-7BB7DA4052F8@gmail.com> Message-ID: <675471.29415.qm@web111401.mail.gq1.yahoo.com> ----- Original Message ---- > From: Joel Reymont > > On Jul 20, 2009, at 5:25 PM, Thomas Lindgren wrote: > > > I'm not sure this was answered, but: ?MODULE macroexpands to the current > module, so the example would be a remote call foo:loop(...), not an apply. > > > I thought a remote call was rpc:call. What am I missing? Sorry, it's a bit of jargon again. mod:func(X1,...,Xn), sometimes known as a fully qualified call: you know the module and function invoked, though the precise call target depends on the currently loaded module. (Local calls by contrast are inside the current module, "do not check for code change", etc.) > If foo:loop is not an apply then why does the compiler complain about it? Higher-order calls include M:F(...), F(...), M:f(...) and so on, where the call target is for some reason dynamic. If there is no higher-order call there, I'm stumped. What does the beam code look like? Best, Thomas From ok@REDACTED Tue Jul 21 00:11:16 2009 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 21 Jul 2009 10:11:16 +1200 Subject: [erlang-questions] Glossary In-Reply-To: <711a73df0907200128x17e78cber35ab6457d2cba57c@mail.gmail.com> References: <711a73df0907200052y3c463de5o62acacd3a8995d86@mail.gmail.com> <7702c0610907200115m6347ac61v362c74d917b69aba@mail.gmail.com> <711a73df0907200128x17e78cber35ab6457d2cba57c@mail.gmail.com> Message-ID: <6039C765-5BC9-4D16-9522-4C747630F3C0@cs.otago.ac.nz> On Jul 20, 2009, at 8:28 PM, Dave Pawson wrote: >>> Node >> Node = (approx) a single processor/computer/machine? > That's how I had it. Then one of those is 'the erlang node', i.e. > where > whole programs are started from. There is no "THE" Erlang node. You can have any number of communicating nodes, and any number of programs running on them, and any of the programs might be started on any of the nodes. With virtualisation these days, a single board might be hosting several operating systems, each of which might count as a "node" in the conventional internet sense. So perhaps the best analogy these days is "an Erlang node is like a loaded and running guest operating system on a virtualised host". >>> Process >> >> A light-weight state machine with a mailbox for receiving messages >> from other processes or system IO resources. In java you might know >> them as green threads. The node schedules processes to run when there >> is something for them to do (like a message in the mailbox). > > This seems the key bit. Yet least natural to get hold of. Erlang processes are precisely the same as operating system tasks in the B6700 MCP (logically separate stacks sharing an address space) or threads in Microsoft's Singularity operating system. They are lightweight because the compiler guarantees that they *cannot* interfere with each other's data structures, so hardware protection schemes (and the heavy costs of switching them) are not needed. (This is one reason why a Foreign Function Interface for Erlang is a Bad Idea: load FFI code and you lose the guarantees, completely.) There is little or no *conceptual* difference between Erlang processes and Unix or Windows processes (without using intra-OS-process threads). They are isolated from each other, they run when they are ready to run and the system has capacity to run them, they communicate via channels (or possibly through file system objects). The differences are *practical*: Erlang processes take much less space, are much cheaper to create, and don't have heavy switching costs, so they are much more "thinkable". other, > > Receiving messages is a key part. > Is it true that a process is generally a single module ('a chunk of > code')? No. > > Even if code from other modules is used? I thought I understood the question, now I'm not so sure. Since a process does not come into existence until a spawn() of some kind is executed, and since such a spawn() must occur *somewhere*, that is in some module, you can associate a module with each process. Since each process starts out running some function, and that function must be in some module, you can associate a module with each process. There is no reason for those two modules to be the same, and they very often aren't. The code that the process spends most of its time executing may come from other modules entirely; the process may not even care if neither of the two associated modules exists any more. (By the way, I wrote that "you" can associate modules with processes. Erlang associates the starting function's module with the code, but not the spawning function's module.) More importantly, OTP provides a number of "behaviours". These are commonly useful patterns of concurrency that are more robustly engineered than anything you might initially have constructed yourself. Processes implemented using behaviours are an intimate mix of *two* modules: the behaviour module and the callback module. > > And what's the relationship between scheduling, the VM and processes/ > modules? Erlang UNIX VM architecture node computer process process module .o or .so file scheduling scheduling > Less understood here. I guess resource is a bit vague! No, "resource" just means in Erlang what it means in other languages. (Apple's use of the term "resource" in Mac OS is related but distinct because the notion of sharing is unimportant.) A resource is something that you get, use for a while, and release. It's like checking out a book from a library, reading it for a while, and returning it. The thing about resources is - other people/processes may want the same things - they have to be given back or others will never get them Think of files, databases, devices: You have to get a connection to the file or database or device. You get to use the resource _through_ the connection. You HAVE to close the connection. This concept is extremely important in most programming languages, with the possible exception of Limbo. The reason is that even in a garbage collected language, manual return of resources is important so that resources are returned for others to use *promptly*. C++ programmers tend to rely on destructors, via the "Resource Acquisition Is Initialisation" design pattern, but the nearest analogue of destructors in Java (finalizers) don't work the same way, and so manual release of resources is still needed. (Necula has a very nice paper about this. The point is that a lot of Java gets this wrong, thanks to exception (mis-)handling.) The concept is important for Erlang for the same reason. Once you have gained a resource, you MUST arrange for it to be given back, even if your process gets an exception, or is killed by a process at the other end of a link exiting. So don't worry about what a resource is. Worry about "what MUST I arrange to give back?" >>> Task Look that one up in an English dictionary, not in an Erlang book. A task might be something that a human has to do, not just a computer. From blueflycn@REDACTED Tue Jul 21 05:26:09 2009 From: blueflycn@REDACTED (Zhuguo Shi) Date: Mon, 20 Jul 2009 23:26:09 -0400 Subject: Is that possilbe to make a substitute of Hadoop in Erlang? Message-ID: <50db8f50907202026n502d0620n948be091d0ccaa60@mail.gmail.com> Hi there, I am new to Erlang and I wonder if I can make a substitute of Hadoop in Erlang? And what will be the advantages and disadvantages of doing so? And if I am building a huge computing cluster like Google's, is it much easier if I choose Erlang as the base layer? From vik@REDACTED Tue Jul 21 05:45:15 2009 From: vik@REDACTED (Vik Olliver) Date: Tue, 21 Jul 2009 15:45:15 +1200 Subject: Stopping ibrowse:send_req/4 GET requests using keep_alive Message-ID: <4A6539CB.8090306@catalyst.net.nz> Is there a way to force send_req to not use keep_alive? i.e. (and I'll spell this out because I'm not an HTTP wizard) so that only one GET request is issued per connection? Vik :v) From arun.suresh@REDACTED Tue Jul 21 06:27:25 2009 From: arun.suresh@REDACTED (Arun Suresh) Date: Tue, 21 Jul 2009 09:57:25 +0530 Subject: [erlang-questions] Is that possilbe to make a substitute of Hadoop in Erlang? In-Reply-To: <50db8f50907202026n502d0620n948be091d0ccaa60@mail.gmail.com> References: <50db8f50907202026n502d0620n948be091d0ccaa60@mail.gmail.com> Message-ID: <9f4a617f0907202127k347aa33ft80a7398b15d01491@mail.gmail.com> i guess this is what u r looking for.. http://discoproject.org/ On Tue, Jul 21, 2009 at 8:56 AM, Zhuguo Shi wrote: > Hi there, > > I am new to Erlang and I wonder if I can make a substitute of Hadoop in > Erlang? And what will be the advantages and disadvantages of doing so? > > And if I am building a huge computing cluster like Google's, is it much > easier if I choose Erlang as the base layer? > From blueflycn@REDACTED Tue Jul 21 10:12:42 2009 From: blueflycn@REDACTED (Zhuguo Shi) Date: Tue, 21 Jul 2009 04:12:42 -0400 Subject: Any message queue model in Erlang when doing Comet chat? Message-ID: <50db8f50907210112p3841331co699f735fa354ce60@mail.gmail.com> Hi there, I am doing Comet chat with Erlang. But when I send message using "Pid ! Msg", some of messages will lost, especially in a high sending frequency. So I wonder if there is any message queue model or data structure, or any way to solve this problem? Any suggestion is of great appreciation~ Thanks From peter.mechlenborg@REDACTED Tue Jul 21 11:11:49 2009 From: peter.mechlenborg@REDACTED (Peter Mechlenborg) Date: Tue, 21 Jul 2009 11:11:49 +0200 Subject: [erlang-questions] Any message queue model in Erlang when doing Comet chat? In-Reply-To: <50db8f50907210112p3841331co699f735fa354ce60@mail.gmail.com> References: <50db8f50907210112p3841331co699f735fa354ce60@mail.gmail.com> Message-ID: Hi I'm doing some comet stuff myself. Started from http://yoan.dosimple.ch/blog/2008/05/15/, it works great, thanks Yoan. I haven't seen any reliable communication protocol specific for comet, but if you google for: 'amqp javascript', it seems there is a javascript client for AMQP. I haven't tried it so I can't say how it works, but maybe it solves your needs. If you do find a good way to do reliable communication over comet, I would be interested in hearing about your experiences... Have fun, -- Peter Mechlenborg On Tue, Jul 21, 2009 at 10:12 AM, Zhuguo Shi wrote: > Hi there, > > I am doing Comet chat with Erlang. But when I send message using "Pid ! > Msg", some of messages will lost, especially in a high sending frequency. > So > I wonder if there is any message queue model or data structure, or any way > to solve this problem? > > Any suggestion is of great appreciation~ > > Thanks > From masse@REDACTED Tue Jul 21 11:12:32 2009 From: masse@REDACTED (mats cronqvist) Date: Tue, 21 Jul 2009 11:12:32 +0200 Subject: Does mnesia create temporary ETS tables during transactions? In-Reply-To: (Sergey Samokhin's message of "Mon\, 20 Jul 2009 11\:53\:54 -0700") References: Message-ID: <87hbx6e6wv.fsf@sterlett.hq.kred> Sergey Samokhin writes: > Hi! > > Is this right that every time I do mnesia:transaction/1 a temprorary > ETS table is created? Yes. Except, I don't know about nested transactions. mats From ulf.wiger@REDACTED Tue Jul 21 12:14:04 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 21 Jul 2009 11:14:04 +0100 (BST) Subject: [erlang-questions] Glossary In-Reply-To: <6039C765-5BC9-4D16-9522-4C747630F3C0@cs.otago.ac.nz> Message-ID: <31669990.19511248171244302.JavaMail.root@zimbra> Richard O'Keefe wrote: > > (This is one reason why a > Foreign Function Interface for Erlang is a Bad Idea: load > FFI code and you lose the guarantees, completely.) I'd phrase this differently. This is the main reason why the use of linked-in drivers has always been discouraged: run user-defined C code inside the virtual machine memory space and you lose the guarantees, completely. There is no difference in this regard between the current linked-in driver interface(s) and the proposed FFI - just the 'small' matter of making it more convenient to rid yourself of said guarantee (which, strictly speaking, states that only the virtual machine and compiler have the power to mess with the memory of your process, and they are much more likely to be correct than your own code.) Linked-in C code is sometimes necessary for acceptable performance, but it comes at a considerable cost in terms of handling and robustness. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd. http://www.erlang-consulting.com From blueflycn@REDACTED Tue Jul 21 12:23:07 2009 From: blueflycn@REDACTED (Zhuguo Shi) Date: Tue, 21 Jul 2009 06:23:07 -0400 Subject: [erlang-questions] Any message queue model in Erlang when doing Comet chat? In-Reply-To: References: <50db8f50907210112p3841331co699f735fa354ce60@mail.gmail.com> Message-ID: <50db8f50907210323mbf0900k64336d1108af8d61@mail.gmail.com> Hi Perter, For the moment I store all the messages in database with a timestamp, also I store the timestamp that every polling connection last received messages. Before every polling connection established I check the timestamp between message and the polling connection so that I can know what messages that haven't been sent yet. It works well in most conditions but I want to find a reliable way to solve this problem, like a message queue. On Tue, Jul 21, 2009 at 5:11 AM, Peter Mechlenborg < peter.mechlenborg@REDACTED> wrote: > Hi > I'm doing some comet stuff myself. Started from > http://yoan.dosimple.ch/blog/2008/05/15/, it works great, thanks Yoan. > I haven't seen any reliable communication protocol specific for comet, but > if you google for: 'amqp javascript', it seems there is a javascript client > for AMQP. I haven't tried it so I can't say how it works, but maybe it > solves your needs. > > If you do find a good way to do reliable communication over comet, I would > be interested in hearing about your experiences... > > Have fun, > > -- Peter Mechlenborg > > On Tue, Jul 21, 2009 at 10:12 AM, Zhuguo Shi wrote: > > > Hi there, > > > > I am doing Comet chat with Erlang. But when I send message using "Pid ! > > Msg", some of messages will lost, especially in a high sending frequency. > > So > > I wonder if there is any message queue model or data structure, or any > way > > to solve this problem? > > > > Any suggestion is of great appreciation~ > > > > Thanks > > > From joelr1@REDACTED Tue Jul 21 12:27:44 2009 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 21 Jul 2009 11:27:44 +0100 Subject: [erlang-questions] Any message queue model in Erlang when doing Comet chat? In-Reply-To: <50db8f50907210323mbf0900k64336d1108af8d61@mail.gmail.com> References: <50db8f50907210112p3841331co699f735fa354ce60@mail.gmail.com> <50db8f50907210323mbf0900k64336d1108af8d61@mail.gmail.com> Message-ID: <2D2E73FC-112F-4870-B1DC-851FEEF0C406@gmail.com> On Jul 21, 2009, at 11:23 AM, Zhuguo Shi wrote: > It works well in most conditions but I want to find a reliable way > to solve > this problem, like a message queue. Are you sure that messages are being lost with Pid ! Msg? How are you checking? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From chandrashekhar.mullaparthi@REDACTED Tue Jul 21 12:33:20 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 21 Jul 2009 11:33:20 +0100 Subject: [erlang-questions] Stopping ibrowse:send_req/4 GET requests using keep_alive In-Reply-To: <4A6539CB.8090306@catalyst.net.nz> References: <4A6539CB.8090306@catalyst.net.nz> Message-ID: 2009/7/21 Vik Olliver > Is there a way to force send_req to not use keep_alive? i.e. (and I'll > spell this out because I'm not an HTTP wizard) so that only one GET request > is issued per connection? > > Vik :v) > > You could try setting the http_vsn to 1.0. This way the server is forced to close the connection after serving one request. cheers Chandru From gleber.p@REDACTED Tue Jul 21 12:40:59 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Tue, 21 Jul 2009 12:40:59 +0200 Subject: [erlang-questions] Any message queue model in Erlang when doing Comet chat? In-Reply-To: <50db8f50907210112p3841331co699f735fa354ce60@mail.gmail.com> References: <50db8f50907210112p3841331co699f735fa354ce60@mail.gmail.com> Message-ID: <14f0e3620907210340k774d458bp97e17dcdff529a07@mail.gmail.com> On Tue, Jul 21, 2009 at 10:12, Zhuguo Shi wrote: > Hi there, > > I am doing Comet chat with Erlang. But when I send message using "Pid ! > Msg", some of messages will lost, especially in a high sending frequency. So > I wonder if there is any message queue model or data structure, or any way > to solve this problem? > > Any suggestion is of great appreciation~ > > Thanks According to my experience this should not happen. To back this up - Erlang Reference Manual [1] states that: "Message sending is asynchronous and safe, the message is guaranteed to eventually reach the recipient, provided that the recipient exists." Basically if process model in your application is properly constructed and maintained (i.e. you never send a message to non-existing process) you should not experience message loss. 1: http://erlang.org/doc/reference_manual/processes.html#10.5 Best, Gleb Peregud From zerthurd@REDACTED Tue Jul 21 12:44:59 2009 From: zerthurd@REDACTED (Maxim Treskin) Date: Tue, 21 Jul 2009 17:44:59 +0700 Subject: [erlang-questions] Any message queue model in Erlang when doing Comet chat? In-Reply-To: <50db8f50907210112p3841331co699f735fa354ce60@mail.gmail.com> References: <50db8f50907210112p3841331co699f735fa354ce60@mail.gmail.com> Message-ID: It is possible you have a wrong pattern-matching expression while receive call or you have another receive in call flow of this process. -- Maxim Treskin From public-mail@REDACTED Tue Jul 21 12:50:03 2009 From: public-mail@REDACTED (=?UTF-8?B?0JDQu9C10LrRgdC10Lkg0KHRg9C90LTRg9C60L7Qsg==?=) Date: Tue, 21 Jul 2009 14:50:03 +0400 Subject: [erlang-questions] Any message queue model in Erlang when doing Comet chat? In-Reply-To: <50db8f50907210323mbf0900k64336d1108af8d61@mail.gmail.com> References: <50db8f50907210112p3841331co699f735fa354ce60@mail.gmail.com> <50db8f50907210323mbf0900k64336d1108af8d61@mail.gmail.com> Message-ID: <8b6ed1540907210350nd16d0ddq3e5d64d86ce89e18@mail.gmail.com> What database is used? 2009/7/21 Zhuguo Shi : > Hi Perter, > > For the moment I store all the messages in database with a timestamp From kevin@REDACTED Tue Jul 21 12:25:00 2009 From: kevin@REDACTED (Kevin A. Smith) Date: Tue, 21 Jul 2009 06:25:00 -0400 Subject: [erlang-questions] Any message queue model in Erlang when doing Comet chat? In-Reply-To: References: <50db8f50907210112p3841331co699f735fa354ce60@mail.gmail.com> Message-ID: <4C9FC93C-497A-422A-B7A5-3E00C37DF77C@hypotheticalabs.com> And if you decide AMQP is the way to go, there's RabbitMQ: http://www.rabbitmq.com It has the added advantage of being written in Erlang, too. --Kevin On Jul 21, 2009, at 5:11 AM, Peter Mechlenborg wrote: > Hi > I'm doing some comet stuff myself. Started from > http://yoan.dosimple.ch/blog/2008/05/15/, it works great, thanks Yoan. > I haven't seen any reliable communication protocol specific for > comet, but > if you google for: 'amqp javascript', it seems there is a javascript > client > for AMQP. I haven't tried it so I can't say how it works, but maybe > it > solves your needs. > > If you do find a good way to do reliable communication over comet, I > would > be interested in hearing about your experiences... > > Have fun, > > -- Peter Mechlenborg > > On Tue, Jul 21, 2009 at 10:12 AM, Zhuguo Shi > wrote: > >> Hi there, >> >> I am doing Comet chat with Erlang. But when I send message using >> "Pid ! >> Msg", some of messages will lost, especially in a high sending >> frequency. >> So >> I wonder if there is any message queue model or data structure, or >> any way >> to solve this problem? >> >> Any suggestion is of great appreciation~ >> >> Thanks >> From attila.r.nohl@REDACTED Tue Jul 21 15:35:59 2009 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 21 Jul 2009 15:35:59 +0200 Subject: [erlang-questions] Glossary In-Reply-To: <7702c0610907210620p563e940cq75831d6547346a68@mail.gmail.com> References: <711a73df0907200052y3c463de5o62acacd3a8995d86@mail.gmail.com> <7702c0610907200115m6347ac61v362c74d917b69aba@mail.gmail.com> <711a73df0907200128x17e78cber35ab6457d2cba57c@mail.gmail.com> <7702c0610907200618l5cdd0345m4a4fa11dd2ec912d@mail.gmail.com> <401d3ba30907210517n2f2ed1a4u908e9f1c1648d534@mail.gmail.com> <7702c0610907210620p563e940cq75831d6547346a68@mail.gmail.com> Message-ID: <401d3ba30907210635p2d187a52kbaaf17c79c51d17f@mail.gmail.com> 2009/7/21, Richard Andrews : > On Tue, Jul 21, 2009 at 10:17 PM, Attila Rajmund > Nohl wrote: >> 2009/7/20, Richard Andrews : >> [...] >>> * All data access is sequential (no race conditions, no mutex >>> deadlocks). >> >> I think this is the number one myth about Erlang. Actually it's fairly >> easy to create race conditions and deadlocks: >> >> Consider a server application which accept connections from the >> network and executes commands received through this connection. One of >> these commands is the "exit" command which stops the application. >> Suppose this application is implemented with two processes, one >> handling the network protocol, and one executing the commands, etc. >> What happens, if the client is particulary stupid and sends the "exit" >> command twice really fast? When the first "exit" comes in, the command >> handling process exits and notifies the network handler process about >> this. However, the client could beat this notification with its second >> "exit" command: the network handling process is still alive and >> happily accepts the second "exit" command. Then is tries to send the >> "exit" command to the command handling process, but that's already >> gone, so one gets a huge crash report due to the race condition. This >> is a problem I'm currently working on. >> >> An other common error is two gen_server processes calling each other: >> a deadlock. Even though Erlang processes do not share memory, the >> application could share state among processes and there you've got the >> same problems of deadlocks, race conditions, etc. > > This is all very contrived. > Yes it is - strictly speaking - possible. I admit to hyperbole. But in > practice for me it just doesn't happen because I obey a few simple > rules of practice. Of course, I wanted to send it to the list the first time... The race condition I've described above very much exists in the OTP SSH server. There was also a thread here recently about trying to trace the erlang shell and that lead to a deadlock. In my experience these bugs do happen in Erlang programs and it's sometimes quite hard to find the cause - just like with race conditions and deadlocks in other languages. From kamiseq@REDACTED Tue Jul 21 16:06:58 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Tue, 21 Jul 2009 16:06:58 +0200 Subject: how to build erlang system that accesses external resources Message-ID: hi all, I have a working system written in delphi (and Im not a delphi expert and it wasnt my idea :)) that creates few threads (like 10-20) and each thread is able to load some dll and then through that dll is performing some action on local db or on other resource. the problem is that it is very hard now to controll threads, force them to kill etc and as I can see operating system is not helping at all so it is a p.i.t.a. so I was thinking to port the system into erlang. but we produced a lot of those dlls so it will be almost impossible to rewrite them into erlang once again. So (as Im quite new to erlang) I imagine that I create a erlang's system that starts and manages processes with ease but then I somehow need to load dll by some external system proccess and talk with it through ports. so it seems to me Im back where I started, doesnt it? again I cant have only one proccess for all dll loaded on the OS side, and I really cant figure out how I should plan everything. I havent read much about ports so maybe I should go through the chapter again, if so just tell me it is a good direction here. pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From gleber.p@REDACTED Tue Jul 21 16:32:08 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Tue, 21 Jul 2009 16:32:08 +0200 Subject: [erlang-questions] how to build erlang system that accesses external resources In-Reply-To: References: Message-ID: <14f0e3620907210732x2e313544u8c85d9ecafd41a74@mail.gmail.com> All options are described here http://erlang.org/doc/tutorial/part_frame.html Basically there are 3 (or four if FFI is counted) options (please correct me if I'm wrong): 1) Ports 2) Linked-in drivers 3) C-nodes 4) FFI Fourth one is not in official distribution, so I will skip it (you can read about it here [1]). Third one is quite complex to be written from scratch, but doable. I haven't written any C node in Delphi or C, so other would provide more information. Though writing Python C-nodes with twotp [3] is very easy :) Second one is strongly discouraged. It would mean that your Delphi/C code would be running inside Erlang VM and any single bug can bring done whole VM [2]. Moreover writing linked-in drivers is cumbersome and you have to obey few rules (e.g. timing), not to mention pain of managing threads inside the linked-in driver... Though if you need a fast and long-term solution you could use it, but cost of development would be much higher than using ports. If it is imperative to use Erlang I would go with ports. Ports are just simple external processes (it could C program, Delphi or even bash script), which communicates with Erlang using stdio/stdout. You have to define a communication protocol (it could be simple line text based protocol or some more complex binary protocol). Or you could use erl_interface / ei libraries to manage erlang term coding/decoding. It is really easy to manage ports from within Erlang (text line based example here [4]). This way you would be able to start multiple external processes, which will do the job (using existing DLLs) and you would communicate with them through stdio/stdout. When ports are used and everything's written according to OTP principles managing it is a breeze [5]. 1: http://muvara.org/crs4/erlang/ffi/ 2: http://erlang.org/doc/tutorial/c_portdriver.html 3: https://launchpad.net/twotp 4: http://github.com/gleber/sha1cruncher/blob/d2e8f0b924ed5d16ade815d9a1fd90cd129ebc40/src/main.erl#L160 (beware, it is very young and non-mature code) 5: http://www.trapexit.org/Writing_an_Erlang_Port_using_OTP_Principles 2009/7/21 pawe? kami?ski : > hi all, > I have a working system written in delphi (and Im not a delphi expert and it > wasnt my idea :)) that creates few threads (like 10-20) and each thread is > able to load some dll and then through that dll is performing some action on > local db or on other resource. the problem is that it is very hard now to > controll ?threads, force them to kill etc and as I can see operating system > is not helping at all so it is a p.i.t.a. so I was thinking to port the > system into erlang. but we produced a lot of those dlls so it will be almost > impossible to rewrite them into erlang once again. > > So (as Im quite new to erlang) I imagine that I create a erlang's system > that starts and manages processes with ease but then I somehow need to load > dll by some external system proccess and talk with it through ports. so it > seems to me Im back where I started, doesnt it? again I cant have only one > proccess for all dll loaded on the OS side, and I really cant figure out how > I should plan everything. > > I havent read much about ports so maybe I should go through the chapter > again, if so just tell me it is a good direction here. > > pozdrawiam > Pawe? Kami?ski > > kamiseq@REDACTED > pkaminski.prv@REDACTED > ______________________ > -- Gleb Peregud http://gleber.pl/ Every minute is to be grasped. Time waits for nobody. -- Inscription on a Zen Gong From kamiseq@REDACTED Tue Jul 21 16:40:26 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Tue, 21 Jul 2009 16:40:26 +0200 Subject: [erlang-questions] how to build erlang system that accesses external resources In-Reply-To: <14f0e3620907210732x2e313544u8c85d9ecafd41a74@mail.gmail.com> References: <14f0e3620907210732x2e313544u8c85d9ecafd41a74@mail.gmail.com> Message-ID: ok so for example I start proccess in erlang that creates port for me and all I need to do is to write a code that will handle dll. then I pass dll name to the port and the OS thread is transparent to me. ok I will read again about this. thanks pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From vances@REDACTED Tue Jul 21 17:01:47 2009 From: vances@REDACTED (Vance Shipley) Date: Tue, 21 Jul 2009 11:01:47 -0400 Subject: [erlang-questions] Glossary In-Reply-To: <31669990.19511248171244302.JavaMail.root@zimbra> References: <6039C765-5BC9-4D16-9522-4C747630F3C0@cs.otago.ac.nz> <31669990.19511248171244302.JavaMail.root@zimbra> Message-ID: <20090721150147.GG79703@h216-235-12-174.host.egate.net> If that is the a concern you may simply use distribution to isolate the node running the driver. On Tue, Jul 21, 2009 at 11:14:04AM +0100, Ulf Wiger wrote: } Linked-in C code is sometimes necessary for acceptable } performance, but it comes at a considerable cost in terms } of handling and robustness. -- -Vance From info@REDACTED Tue Jul 21 17:12:24 2009 From: info@REDACTED (info) Date: Tue, 21 Jul 2009 17:12:24 +0200 Subject: OTP application as service Message-ID: <200907211712238739319@its3.ch> Hi all, I finished the development of my OTP application. I run it in Eclipse with the command : application:start(my_application). I would like now to run it as one service of windows: How to do this or reference for learning how to do. Thank's John From rapsey@REDACTED Tue Jul 21 17:32:48 2009 From: rapsey@REDACTED (Rapsey) Date: Tue, 21 Jul 2009 17:32:48 +0200 Subject: [erlang-questions] OTP application as service In-Reply-To: <200907211712238739319@its3.ch> References: <200907211712238739319@its3.ch> Message-ID: <97619b170907210832k7ce4f2bcu6326ab6ae2f90a7e@mail.gmail.com> check out erlsrv http://erlang.org/doc/apps/erts/index.html Sergej On Tue, Jul 21, 2009 at 5:12 PM, info wrote: > Hi all, > I finished the development of my OTP application. > I run it in Eclipse with the command : application:start(my_application). > I would like now to run it as one service of windows: How to do this or > reference for learning how to do. > Thank's > > John > From ulf.wiger@REDACTED Tue Jul 21 18:02:16 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 21 Jul 2009 18:02:16 +0200 Subject: [erlang-questions] Re: Does mnesia create temporary ETS tables during transactions? In-Reply-To: <87hbx6e6wv.fsf@sterlett.hq.kred> References: <87hbx6e6wv.fsf@sterlett.hq.kred> Message-ID: <4A65E688.6050305@erlang-consulting.com> mats cronqvist wrote: > Sergey Samokhin writes: > >> Hi! >> >> Is this right that every time I do mnesia:transaction/1 a temprorary >> ETS table is created? > > Yes. > Except, I don't know about nested transactions. When you start a nested transaction, an ETS table is created, and all the data from the surrounding transaction are copied to it. Upon commit, all data is copied back; otherwise, the table is simply discarded. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From roberto@REDACTED Tue Jul 21 18:11:08 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 21 Jul 2009 18:11:08 +0200 Subject: message passing between nodes using custom TCP Message-ID: dear all, as some of you may already know i'm very interested in increasing message passing speed between erlang nodes. therefore, i started a simple test to see whether increasing the number of persistent TCP connections [i.e. pipes] between the nodes could speed things up. the idea is simple. - on the recipient node B, i start up a gen_server managing a listening TCP server and a series of recv processes which are spawned upon gen_tcp:accept/1; - on the sending node A, i start up a gen_server managing a series of clients processes which create a number of connections to the TCP server on B. this creates a certain number of TCP 'pipes' between A and B. processes sending messages from node A to processes on node B pass through the gen_server on A, by using a function, instead of the native erlang sending functionality, such as: send(Recipient, Message). this function converts the erlang terms Recipient and Message to binary, then sends this binary through TCP using one of the available clients on A. the binary data sent is received by the associated recv process on B, which converts the received binary data to the original erlang term and sends it to the destination process. the code i've developed works fine, and when i use just one TCP pipe the speed is only marginally slower to the native erlang sending functionality. however, when i use multiple TCP connections, the speed linearly degrades, and for instance when i set up something like 50 parallel TCP channels it gets down by 40%. i'm asking for a clue to have an understanding of the wrong assumptions i'm making, or what i could try more to achieve my objective. i do understand that this can be quite hard without seeing my code, but i have no problem in posting it if someone can help me out. i believe this can become quite interesting. cheers, r. From rtrlists@REDACTED Tue Jul 21 18:24:04 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 21 Jul 2009 17:24:04 +0100 Subject: [erlang-questions] OTP application as service In-Reply-To: <200907211712238739319@its3.ch> References: <200907211712238739319@its3.ch> Message-ID: <6a3ae47e0907210924x7ada47cm4e337e340fa1fde6@mail.gmail.com> On Tue, Jul 21, 2009 at 4:12 PM, info wrote: > Hi all, > I finished the development of my OTP application. > I run it in Eclipse with the command : application:start(my_application). > I would like now to run it as one service of windows: How to do this or > reference for learning how to do. > Thank's > > John > The Windows erlsrv.exe that comes with your Erlang distro can do this. I have found it to be quite difficult to get all the options right. I learnt a lot from looking at how RabbitMQ does it. Robby From kevin.hostelley@REDACTED Tue Jul 21 18:45:05 2009 From: kevin.hostelley@REDACTED (kevin.hostelley@REDACTED) Date: Tue, 21 Jul 2009 12:45:05 -0400 Subject: Fw: Newbie question Message-ID: I'm new to erlang and I was trying to write a simple message queue to try and learn the language. I'm getting a result I don't understand. Here's the code for the program: -module(queue). -export([start/0, listen/1]). listen([]) -> io:format("Listen with empty list.~n"), receive {get, Pid} -> Pid ! empty, listen([]); {put, Data} -> listen([Data]); stop -> io:format("Stopped.~n") end; listen(Elements) -> io:format("Listen with current list (~s).~n", [Elements]), receive {get, Pid} -> [Elem|NewList] = Elements, Pid ! {data, Elem}, listen(NewList); {put, Data} -> NewList2 = lists:append(Elements, [Data]), listen(NewList2); stop -> io:format("Stopping.~n") end. start() -> register(queue, spawn(queue, listen,[[]])). Then from the shell I run: queue ! {put, "ABC"}. queue ! {put, "DEF"}. queue ! {put, "GHI"}. queue ! {get, self()}. receive {data, Data} -> io:format("Got ~s~n", [Data]); empty -> io:format("Got empty~n"); AnyMessage -> io:format("Trap ~s~n", [AnyMessage]); _ -> io:format("Got something else~n") after 5000 -> io:format("timed out~n") end. And I get "Got ABC". I do another get and the same receive as above and I get: ** exception exit: {badarg,[{io,format,[<0.31.0>,"Trap ~s~n",[{data,"DEF"}]]}, {erl_eval,do_apply,5}, {shell,exprs,6}, {shell,eval_exprs,6}, {shell,eval_loop,3}]} in function io:o_request/3 I don't understand why the {data, Data} pattern isn't receiving the message. Can someone help me understand this? Thanks, Kevin Hostelley ***** PLEASE NOTE ***** This E-Mail/telefax message and any documents accompanying this transmission may contain privileged and/or confidential information and is intended solely for the addressee(s) named above. If you are not the intended addressee/recipient, you are hereby notified that any use of, disclosure, copying, distribution, or reliance on the contents of this E-Mail/telefax information is strictly prohibited and may result in legal action against you. Please reply to the sender advising of the error in transmission and immediately delete/destroy the message and any accompanying documents. Thank you.***** From zerthurd@REDACTED Tue Jul 21 18:59:13 2009 From: zerthurd@REDACTED (Maxim Treskin) Date: Tue, 21 Jul 2009 23:59:13 +0700 Subject: [erlang-questions] Fw: Newbie question In-Reply-To: References: Message-ID: Just use ~p instead of ~s in io:format/2 ~s prints strings, ~p prints any term -- Maxim Treskin From ajuttner.list@REDACTED Tue Jul 21 18:59:14 2009 From: ajuttner.list@REDACTED (Alpar Juttner) Date: Tue, 21 Jul 2009 17:59:14 +0100 Subject: [erlang-questions] Fw: Newbie question In-Reply-To: References: Message-ID: <1248195554.5048.358.camel@piko.site> Hi, > queue ! {put, "ABC"}. > queue ! {put, "DEF"}. > queue ! {put, "GHI"}. > queue ! {get, self()}. > receive > {data, Data} -> io:format("Got ~s~n", [Data]); > empty -> io:format("Got empty~n"); > AnyMessage -> io:format("Trap ~s~n", [AnyMessage]); > _ -> io:format("Got something else~n") > after 5000 -> io:format("timed out~n") > end. > > And I get "Got ABC". I do another get and the same receive as above and I > get: > > ** exception exit: {badarg,[{io,format,[<0.31.0>,"Trap > ~s~n",[{data,"DEF"}]]}, > {erl_eval,do_apply,5}, > {shell,exprs,6}, > {shell,eval_exprs,6}, > {shell,eval_loop,3}]} > in function io:o_request/3 > > > I don't understand why the {data, Data} pattern isn't receiving the > message. Can someone help me understand this? It is because Data is already bound to "ABC", so you try to match against {data, "ABC"}. Regards, Alpar From info@REDACTED Tue Jul 21 19:34:14 2009 From: info@REDACTED (info) Date: Tue, 21 Jul 2009 19:34:14 +0200 Subject: [erlang-questions] OTP application as service References: <200907211712238739319@its3.ch>, <6a3ae47e0907210924x7ada47cm4e337e340fa1fde6@mail.gmail.com> Message-ID: <200907211934142283511@its3.ch> You are right. Even RabbitMQ is difficult to understand ... I didn't see with which parameter we give the path to the application: erlsrv.exe add but after ? On Tue, Jul 21, 2009 at 4:12 PM, info wrote: Hi all, I finished the development of my OTP application. I run it in Eclipse with the command : application:start(my_application). I would like now to run it as one service of windows: How to do this or reference for learning how to do. Thank's John The Windows erlsrv.exe that comes with your Erlang distro can do this. I have found it to be quite difficult to get all the options right. I learnt a lot from looking at how RabbitMQ does it. Robby From kevin.hostelley@REDACTED Tue Jul 21 19:35:17 2009 From: kevin.hostelley@REDACTED (kevin.hostelley@REDACTED) Date: Tue, 21 Jul 2009 13:35:17 -0400 Subject: [erlang-questions] Fw: Newbie question In-Reply-To: <1248195554.5048.358.camel@piko.site> Message-ID: > > I don't understand why the {data, Data} pattern isn't receiving the > > message. Can someone help me understand this? > > It is because Data is already bound to "ABC", so you try to match > against {data, "ABC"}. > > Regards, > Alpar > > So "Data" is still in scope even though the receive ended? If the receive was inside a function, would the scope of "Data" be limited to that function? Thanks, Kevin Hostelley ***** PLEASE NOTE ***** This E-Mail/telefax message and any documents accompanying this transmission may contain privileged and/or confidential information and is intended solely for the addressee(s) named above. If you are not the intended addressee/recipient, you are hereby notified that any use of, disclosure, copying, distribution, or reliance on the contents of this E-Mail/telefax information is strictly prohibited and may result in legal action against you. Please reply to the sender advising of the error in transmission and immediately delete/destroy the message and any accompanying documents. Thank you.***** From zerthurd@REDACTED Tue Jul 21 20:10:39 2009 From: zerthurd@REDACTED (Maxim Treskin) Date: Wed, 22 Jul 2009 01:10:39 +0700 Subject: [erlang-questions] Fw: Newbie question In-Reply-To: References: <1248195554.5048.358.camel@piko.site> Message-ID: > > So "Data" is still in scope even though the receive ended? If the receive > was inside a function, would the scope of "Data" be limited to that > function? > > No. You have badarg error in io:format. There is no 'exception error: no match of right hand side value', only badarg. -- Maxim Treskin From kevin.hostelley@REDACTED Tue Jul 21 20:17:19 2009 From: kevin.hostelley@REDACTED (kevin.hostelley@REDACTED) Date: Tue, 21 Jul 2009 14:17:19 -0400 Subject: [erlang-questions] Fw: Newbie question In-Reply-To: Message-ID: Maxim Treskin wrote on 07/21/2009 02:10:39 PM: > So "Data" is still in scope even though the receive ended? ?If the receive > was inside a function, would the scope of "Data" be limited to that > function? > > > No. You have badarg error in io:format. There is no 'exception > error: no match of right hand side value', only badarg. > > -- > Maxim Treskin Thanks for the "~p" tip for io:format. But the problem I couldn't understand was why my first receive pattern was skipped. See Alpar's msg for the explanation. Thanks, Kevin Hostelley ***** PLEASE NOTE ***** This E-Mail/telefax message and any documents accompanying this transmission may contain privileged and/or confidential information and is intended solely for the addressee(s) named above. If you are not the intended addressee/recipient, you are hereby notified that any use of, disclosure, copying, distribution, or reliance on the contents of this E-Mail/telefax information is strictly prohibited and may result in legal action against you. Please reply to the sender advising of the error in transmission and immediately delete/destroy the message and any accompanying documents. Thank you.***** From zerthurd@REDACTED Tue Jul 21 20:23:51 2009 From: zerthurd@REDACTED (Maxim Treskin) Date: Wed, 22 Jul 2009 01:23:51 +0700 Subject: [erlang-questions] Fw: Newbie question In-Reply-To: References: Message-ID: Did you call queue ! {get, self()} before second receive? -- Maxim Treskin From vincent.dephily@REDACTED Tue Jul 21 20:37:45 2009 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Tue, 21 Jul 2009 20:37:45 +0200 Subject: edoc: commenting multiple patterns of a function Message-ID: <200907212037.46531.vincent.dephily@mobile-devices.fr> Hi list, not a big issue but it always bugged me: how do you add edoc comments to a function depending on its pattern match ? eg: -behavior(gen_server). ... % @doc Modify counter. handle_cast({inc, N}, Count) -> {noreply, Count + N}; % @doc Reset counter. handle_cast(reset, Count) -> {noreply, 0}; handle_cast(Other, Count) -> {stop, {api_violation, Other}, 0}. In this example, the edoc-generated file will just say "Modify counter", ignoring that the function can also "Reset counter". What I hoped to find in the doc, was the two @doc comments, ideally with the corresponding pattern (it would be great if that pattern could be specified in the docs). I could add all the documentation in the first @doc instance, but it'll be horribly unreadable (especially in a gen_server with numerous large handle_* functions). Or I could use a generic "handles gen_server casts, see source for possible casts", but it makes the edoc a bit useless (I allready know what the behavior callbacks do, no need to read project-specific doc). Any thoutghs ? -- Vincent de Phily Mobile Devices +33 (0) 666 301 306 +33 (0) 142 119 325 Warning This message (and any associated files) is intended only for the use of its intended recipient and may contain information that is confidential, subject to copyright or constitutes a trade secret. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this message, or files associated with this message, is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. Any views or opinions presented are solely those of the author vincent.dephily@REDACTED and do not necessarily represent those of the company. Although the company has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. From tom.mcnulty@REDACTED Tue Jul 21 21:14:15 2009 From: tom.mcnulty@REDACTED (Tom McNulty) Date: Tue, 21 Jul 2009 13:14:15 -0600 Subject: Mnesia Secondary Indexes Message-ID: Hi all, When adding an index to a table field, it appears to me (without looking at the source), that the indexing method is hash based. While this is optimal for fast secondary key lookups and join operations, it does not speed up range based queries. After going through all available manuals, I'm quite certain it's not possible to use tree- based indexing on a field, but please correct me if I'm wrong. My question finally, are there an plans to add tree-based indexing to mnesia? And in the meantime, are there any existing projects which hack on this support? - Tom From mevans@REDACTED Tue Jul 21 22:34:43 2009 From: mevans@REDACTED (Evans, Matthew) Date: Tue, 21 Jul 2009 16:34:43 -0400 Subject: [erlang-questions] Mnesia Secondary Indexes In-Reply-To: Message-ID: Hi, I recently faced a similar issue to this. I wanted to do a range-based query on a table (created as a bag) where each primary key could contain a large number of records (10K or so). I decided against using a secondary index as a key, but instead searched using a select, with a primary key and specifying a guard to do the range query. As expected, these queries took a long time. To overcome this performance issue I changed the table from a bag to a set and used a modified gb_tree module that returned a reference to the range. For example, the old key of: KeyAsInteger Became a tuple: {KeyAsInteger,RangeValue} The modified gb_tree accepted any value in the query, and would return the "closest" match. I could then use this as the RangeValue in the query to the tuple. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Tom McNulty Sent: Tuesday, July 21, 2009 3:14 PM To: erlang-questions@REDACTED Subject: [erlang-questions] Mnesia Secondary Indexes Hi all, When adding an index to a table field, it appears to me (without looking at the source), that the indexing method is hash based. While this is optimal for fast secondary key lookups and join operations, it does not speed up range based queries. After going through all available manuals, I'm quite certain it's not possible to use tree- based indexing on a field, but please correct me if I'm wrong. My question finally, are there an plans to add tree-based indexing to mnesia? And in the meantime, are there any existing projects which hack on this support? - Tom ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From tom.mcnulty@REDACTED Tue Jul 21 22:58:55 2009 From: tom.mcnulty@REDACTED (Tom McNulty) Date: Tue, 21 Jul 2009 14:58:55 -0600 Subject: [erlang-questions] Mnesia Secondary Indexes In-Reply-To: References: Message-ID: <0D8BD6B1-9198-4B49-B4C3-38EC0AFDB71E@cetiforge.com> Hi, I moved to something similar, except instead of a gb_tree, I'm using a secondary table as an ordered_set, which points into the first table using keys: {RangeVal, Key1}. As describe here: http://erlanganswers.com/web/mcedemo/mnesia/OrderedBy.html -- thanks. It should work (haven't finished testing, and need to read how ordered_set is implemented). Being 'lazy' though, I tend to want this to work to occur behind the scenes. - Tom On 21-Jul-09, at 2:34 PM, Evans, Matthew wrote: > Hi, > > I recently faced a similar issue to this. I wanted to do a range- > based query on a table (created as a bag) where each primary key > could contain a large number of records (10K or so). I decided > against using a secondary index as a key, but instead searched using > a select, with a primary key and specifying a guard to do the range > query. > > As expected, these queries took a long time. > > To overcome this performance issue I changed the table from a bag to > a set and used a modified gb_tree module that returned a reference > to the range. > > For example, the old key of: > > KeyAsInteger > > Became a tuple: > > {KeyAsInteger,RangeValue} > > The modified gb_tree accepted any value in the query, and would > return the "closest" match. I could then use this as the RangeValue > in the query to the tuple. > > > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang- > questions@REDACTED] On Behalf Of Tom McNulty > Sent: Tuesday, July 21, 2009 3:14 PM > To: erlang-questions@REDACTED > Subject: [erlang-questions] Mnesia Secondary Indexes > > Hi all, > > When adding an index to a table field, it appears to me (without > looking at the source), that the indexing method is hash based. While > this is optimal for fast secondary key lookups and join operations, it > does not speed up range based queries. After going through all > available manuals, I'm quite certain it's not possible to use tree- > based indexing on a field, but please correct me if I'm wrong. > > My question finally, are there an plans to add tree-based indexing to > mnesia? And in the meantime, are there any existing projects which > hack on this support? > > - Tom > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From clist@REDACTED Wed Jul 22 00:06:06 2009 From: clist@REDACTED (Angel Alvarez) Date: Wed, 22 Jul 2009 00:06:06 +0200 Subject: How to distribute code using code server to slave nodes Message-ID: <200907220006.07093.clist@uah.es> Hi Im learning erlang so i use to write toy programs just for fun now Im learning to use the pool module, so i cant spread workers over some computers. The problem is that prior to pspawn processes i have to manually distribute code among all members of the pool. I use to write something like this (i saw this somewhere on the net...) pool:start(pooltest, lists:concat(["-setcookie ", erlang:get_cookie()])), distribute_app([mymodule1,mymodule2, ... , mymodulen]), with distribute_app as: distribute_app(Modules) -> %% extraer todos los nodos excepto el Master RemoteNodes = [X || X <- pool:get_nodes(), X =/= node()], %% Transferir el c?digo lists:foreach(fun(Node) -> transfer_code(Node, Modules) end, RemoteNodes). transfer_code(Node, Modules) -> [transfer_module(Node, Module) || Module <- Modules]. transfer_module(Node, Module) -> {_Module, Binary, FileName} = code:get_object_code(Module), rpc:call(Node, code, load_binary, [Module, FileName, Binary]). Instead of doing this, can i instruct remote code servers to ask master code server for code when it need to locate new refences? Is seems to be related to starting erl with "--loader inet" ?Can anyone prove me with some pointers about this? Thanks -- Agua para todo? No, Agua para Todos. ->>----------------------------------------------- Clist UAH a.k.a Angel ---------------------------------[www.uah.es]-<<-- Vivimos en un sociedad exquisitamente dependiente de la ciencia y la tecnolog?a, cuando muy poco gente conoce algo de la ciencia ? de la tecnolog?a. From vik@REDACTED Wed Jul 22 00:26:27 2009 From: vik@REDACTED (Vik Olliver) Date: Wed, 22 Jul 2009 10:26:27 +1200 Subject: [erlang-questions] Stopping ibrowse:send_req/4 GET requests using keep_alive In-Reply-To: References: <4A6539CB.8090306@catalyst.net.nz> Message-ID: <4A664093.6020700@catalyst.net.nz> On 21/07/09 Chandru wrote: > You could try setting the http_vsn to 1.0. This way the server is > forced to close the connection after serving one request. Sounds good to me. I'll run some tests when the owners of the server are sufficiently caffeinated... Vik :v) From smiskovic@REDACTED Wed Jul 22 01:32:42 2009 From: smiskovic@REDACTED (Slobodan Miskovic) Date: Tue, 21 Jul 2009 16:32:42 -0700 (PDT) Subject: Starting a slave node in particular directory In-Reply-To: <20410557.4671248218817034.JavaMail.root@zimbra-vm.localdomain> Message-ID: <2538067.4691248219162600.JavaMail.root@zimbra-vm.localdomain> Hi all, I am writing a cluster master portion of my system but am having issues with the current working directory on slave nodes. For certain applications (in particular yaws:start_embedded) the files:set_cwd doesn't seem to affect directory in which they start; instead yaws starts in the cluster master startup folder. Is this expected, or an issue with yaws? Happens with erlang R12, R13; yaws 1.77 and 1.82 (1.84 doesn't want to start in embedded mode). I'm starting slave nodes with slave:start_link(Host, NodeName, "-rsh ssh -pa /path/to/ebins -run file set_cwd /path/to/rundir -s myapp start -setcookie " ++ Cookie) (Currently slave nodes are started on the same physical machine, but this may change later if it matters) Also, this seems to change the CWD for the master node as well!? Any pointers would be greatly appreciated. Slobo From blueflycn@REDACTED Wed Jul 22 03:12:21 2009 From: blueflycn@REDACTED (Zhuguo Shi) Date: Tue, 21 Jul 2009 21:12:21 -0400 Subject: [erlang-questions] Any message queue model in Erlang when doing Comet chat? In-Reply-To: <2D2E73FC-112F-4870-B1DC-851FEEF0C406@gmail.com> References: <50db8f50907210112p3841331co699f735fa354ce60@mail.gmail.com> <50db8f50907210323mbf0900k64336d1108af8d61@mail.gmail.com> <2D2E73FC-112F-4870-B1DC-851FEEF0C406@gmail.com> Message-ID: <50db8f50907211812h6e8bc248r42db39f8ff5e40bf@mail.gmail.com> Hi Peter, I did the checking by sending messages in a high frequency, like you enter some letters and press the "send" button quickly. My comet chat is like Facebook's and the polling connection is automatically disconnect and reconnect in 10 senconds (Facebook is about 50 seconds). So I doubt if you send a message to a user that just disconnected and have not connected, the message will lost. The test result is not good before I added the database sync mechanism. I am not sure if there is any problem in my code or test. I did this comet chat prototype just according to Richard's series ( http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1/ ). Any suggestions? On Tue, Jul 21, 2009 at 6:27 AM, Joel Reymont wrote: > > On Jul 21, 2009, at 11:23 AM, Zhuguo Shi wrote: > > It works well in most conditions but I want to find a reliable way to >> solve >> this problem, like a message queue. >> > > > Are you sure that messages are being lost with Pid ! Msg? > > How are you checking? > > Thanks, Joel > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > From blueflycn@REDACTED Wed Jul 22 03:24:33 2009 From: blueflycn@REDACTED (Zhuguo Shi) Date: Tue, 21 Jul 2009 21:24:33 -0400 Subject: [erlang-questions] Any message queue model in Erlang when doing Comet chat? In-Reply-To: <8b6ed1540907210350nd16d0ddq3e5d64d86ce89e18@mail.gmail.com> References: <50db8f50907210112p3841331co699f735fa354ce60@mail.gmail.com> <50db8f50907210323mbf0900k64336d1108af8d61@mail.gmail.com> <8b6ed1540907210350nd16d0ddq3e5d64d86ce89e18@mail.gmail.com> Message-ID: <50db8f50907211824p4d013bf3ube3c40fbee47c09f@mail.gmail.com> MySQL. I didn't choose mnesia because of the compatibility. But I am afraid when the concurrent connection reaches a number MySQL will be the bottle neck, so I am looking for a more reliable and efficient way. On Tue, Jul 21, 2009 at 6:50 AM, ??????? ???????? wrote: > What database is used? > > 2009/7/21 Zhuguo Shi : > > Hi Perter, > > > > For the moment I store all the messages in database with a timestamp > From yoursurrogategod@REDACTED Wed Jul 22 04:59:43 2009 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Tue, 21 Jul 2009 19:59:43 -0700 (PDT) Subject: Printing very large numbers without e Message-ID: <4e8554ff-4a8c-4528-b63e-aa0803e22272@c2g2000yqi.googlegroups.com> Hi, I'm not sure where to get an answer to this question. Say I want to see an integer printed out, but I don't want the e in the middle of my number. 1> io:format("~w~n", [1000000 / 2]). 5.0e5 ok How can I get 500000? I've looked at the documentation and couldn't find anything that would give me the whole number without e. Did I miss something? From steven.charles.davis@REDACTED Wed Jul 22 05:03:07 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 21 Jul 2009 20:03:07 -0700 (PDT) Subject: Stopping ibrowse:send_req/4 GET requests using keep_alive In-Reply-To: <4A664093.6020700@catalyst.net.nz> References: <4A6539CB.8090306@catalyst.net.nz> <4A664093.6020700@catalyst.net.nz> Message-ID: Alternatively, send "Connection: close" header in the request from the client. On Jul 21, 5:26?pm, Vik Olliver wrote: > On 21/07/09 Chandru wrote: > > ?You could try setting the http_vsn to 1.0. This way the server is > > ?forced to close the connection after serving one request. > > Sounds good to me. I'll run some tests when the owners of the server are > sufficiently caffeinated... > > Vik :v) > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From steven.charles.davis@REDACTED Wed Jul 22 05:28:17 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 21 Jul 2009 20:28:17 -0700 (PDT) Subject: Printing very large numbers without e In-Reply-To: <4e8554ff-4a8c-4528-b63e-aa0803e22272@c2g2000yqi.googlegroups.com> References: <4e8554ff-4a8c-4528-b63e-aa0803e22272@c2g2000yqi.googlegroups.com> Message-ID: Maybe (or maybe not) you are meaning... io:format("~w~n", [1000000 div 2]). ? /sd On Jul 21, 9:59?pm, "Yves S. Garret" wrote: > Hi, > > ? ?I'm not sure where to get an answer to this question. ?Say I want > to see an integer printed out, but I don't want the e in the middle of > my number. > > 1> io:format("~w~n", [1000000 / 2]). > 5.0e5 > ok > > How can I get 500000? ?I've looked at the documentation and couldn't > find anything that would give me the whole number without e. ?Did I > miss something? > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From Greg.Perry@REDACTED Wed Jul 22 05:52:02 2009 From: Greg.Perry@REDACTED (Greg Perry) Date: Tue, 21 Jul 2009 20:52:02 -0700 Subject: [erlang-questions] Re: Printing very large numbers without e Message-ID: Round it? Greg Perry Chief Executive Officer LiveAmmo Education, VMware Training Phone: 1+ (302) 223-3195 x111 Fax: (866) 541-9022 Email: Greg.Perry@REDACTED Web: http://www.LiveAmmo.com Twitter: http://twitter.com/liveammo ----- Original Message ----- From: erlang-questions@REDACTED To: erlang-questions@REDACTED Sent: Tue Jul 21 20:28:17 2009 Subject: [erlang-questions] Re: Printing very large numbers without e Maybe (or maybe not) you are meaning... io:format("~w~n", [1000000 div 2]). ? /sd On Jul 21, 9:59?pm, "Yves S. Garret" wrote: > Hi, > > ? ?I'm not sure where to get an answer to this question. ?Say I want > to see an integer printed out, but I don't want the e in the middle of > my number. > > 1> io:format("~w~n", [1000000 / 2]). > 5.0e5 > ok > > How can I get 500000? ?I've looked at the documentation and couldn't > find anything that would give me the whole number without e. ?Did I > miss something? > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From vik@REDACTED Wed Jul 22 06:24:02 2009 From: vik@REDACTED (Vik Olliver) Date: Wed, 22 Jul 2009 16:24:02 +1200 Subject: [erlang-questions] Re: Stopping ibrowse:send_req/4 GET requests using keep_alive In-Reply-To: References: <4A6539CB.8090306@catalyst.net.nz> <4A664093.6020700@catalyst.net.nz> Message-ID: <4A669462.3020405@catalyst.net.nz> Steve Davis wrote: > Alternatively, send "Connection: close" header in the request from the > client. > > Client already returns: HTTP/1.1 200 OK Date: Wed, 22 Jul 2009 03:34:32 GMT Server: Apache/1.3.34 (Debian) mod_perl/1.29 Connection: close Content-Type: text/plain; charset=utf-8 Done. Setting max_pipeline_size in the options for send_req/5 does not seem to actually set the MaxPSz (I have trace turned on). But setting it with set_dest/3 does work. I don't have set_max_pipeline_size/3 in this release and my URL includes a port number anyway, which I presumably would have to chop out etc. :( Vik :v) From steven.charles.davis@REDACTED Wed Jul 22 07:12:19 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 21 Jul 2009 22:12:19 -0700 (PDT) Subject: Stopping ibrowse:send_req/4 GET requests using keep_alive In-Reply-To: <4A669462.3020405@catalyst.net.nz> References: <4A6539CB.8090306@catalyst.net.nz> <4A664093.6020700@catalyst.net.nz> <4A669462.3020405@catalyst.net.nz> Message-ID: That looks like a server HTTP response... I was suggesting setting this in the client - also valid according to: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.10 On Jul 21, 11:24?pm, Vik Olliver wrote: > Steve Davis wrote: > > Alternatively, send "Connection: close" header in the request from the > > client. > > Client already returns: > > HTTP/1.1 200 OK > Date: Wed, 22 Jul 2009 03:34:32 GMT > Server: Apache/1.3.34 (Debian) mod_perl/1.29 > Connection: close > Content-Type: text/plain; charset=utf-8 > > Done. > > Setting max_pipeline_size in the options for send_req/5 does not seem to > actually set the MaxPSz (I have trace turned on). But setting it with > set_dest/3 does work. I don't have set_max_pipeline_size/3 in this > release and my URL includes a port number anyway, which I presumably > would have to chop out etc. :( > > Vik :v) > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From jim.mccoy@REDACTED Wed Jul 22 07:19:58 2009 From: jim.mccoy@REDACTED (Jim McCoy) Date: Tue, 21 Jul 2009 22:19:58 -0700 Subject: Mnesia local_content tables as transaction manager? Message-ID: I want to perform some transactions over processes (but will handle persistence via another channel) such that if process A and B are on the same node I can be certain that once a "transaction function" returns successfully both A and B have had the specified state transformation applied to them. Ideally this would also work if A and B were on different nodes, but I am content to move them both to the same node to perform the transaction. Is this something that mnesia local_content tables would be suitable for? Would mnesia/local_content be overkill for this task? Am I better off looking at another tool or example for this task? I guess what I am basically looking for is a distributed transaction manager and I am hoping that there might be some part of mnesia or other built-in facility that can be re-tasked for this purpose. Any thoughts? jim From vik@REDACTED Wed Jul 22 07:21:13 2009 From: vik@REDACTED (Vik Olliver) Date: Wed, 22 Jul 2009 17:21:13 +1200 Subject: [erlang-questions] Re: Stopping ibrowse:send_req/4 GET requests using keep_alive In-Reply-To: References: <4A6539CB.8090306@catalyst.net.nz> <4A664093.6020700@catalyst.net.nz> <4A669462.3020405@catalyst.net.nz> Message-ID: <4A66A1C9.4050006@catalyst.net.nz> On 22/07/09 Steve Davis wrote: > That looks like a server HTTP response... > > I was suggesting setting this in the client - also valid according to: > http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.10 I think I may have found the problem then: "Fixed bug in returning response when the server returns a Connection: Close header" http://github.com/cmullaparthi/ibrowse/commit/9b0a927e39e7c3145a6cac11409144d3089f17f9 Now to upgrade to the latest ibrowse, which will be fun as the old one is tangled in jungerl... Vik :v) From bflatmaj7th@REDACTED Wed Jul 22 08:03:16 2009 From: bflatmaj7th@REDACTED (Richard Andrews) Date: Wed, 22 Jul 2009 16:03:16 +1000 Subject: [erlang-questions] Printing very large numbers without e In-Reply-To: <4e8554ff-4a8c-4528-b63e-aa0803e22272@c2g2000yqi.googlegroups.com> References: <4e8554ff-4a8c-4528-b63e-aa0803e22272@c2g2000yqi.googlegroups.com> Message-ID: <7702c0610907212303s68b09fe4o80cfc934ea3adcd@mail.gmail.com> On Wed, Jul 22, 2009 at 12:59 PM, Yves S. Garret wrote: > Hi, > > ? I'm not sure where to get an answer to this question. ?Say I want > to see an integer printed out, but I don't want the e in the middle of > my number. > > 1> io:format("~w~n", [1000000 / 2]). > 5.0e5 > ok > > How can I get 500000? ?I've looked at the documentation and couldn't > find anything that would give me the whole number without e. ?Did I > miss something? 1> io:format("~B~n", [1000000 div 2]). 500000 ok From buricchio@REDACTED Wed Jul 22 09:29:56 2009 From: buricchio@REDACTED (Andrew) Date: Wed, 22 Jul 2009 10:29:56 +0300 Subject: OTP wx driver on OpenSolaris 2009.06 Message-ID: <4A66BFF4.2000807@gmail.com> Hi all! Have anybody managed to build latest OTP on latest OpenSolaris (2009.06) with latest WxWidgets (2.8.10 built from sources)? I have "..Can not link wx driver ..." message running OTP configure script. The wxWidgets built with --with-gtk=2 --with-opengl --disable-shared (it seems, OTP wx driver require that), but the problem seems somehow connected with '-shared' option of g++ at test of wx driver linking in configure script (for lib/wx). Possible some libs are not -fpic, I don't know. Any clue how build OTP with wx driver enabled? Andrew. From masse@REDACTED Wed Jul 22 10:10:38 2009 From: masse@REDACTED (mats cronqvist) Date: Wed, 22 Jul 2009 10:10:38 +0200 Subject: erlang-style procs in the browser Message-ID: <87my6xcf41.fsf@sterlett.hq.kred> John Resig writes about a next-gen browser feature that looks suspiciously like Erlang-style processes. http://ejohn.org/blog/web-workers/ From info@REDACTED Wed Jul 22 12:44:30 2009 From: info@REDACTED (info) Date: Wed, 22 Jul 2009 12:44:30 +0200 Subject: run an otp/application as a service Message-ID: <200907221244291597634@its3.ch> Hi All, Starting an OTP/Application with erlsrv is not evident for me even with the (poor) documentation ... I tried erlsrv.exe add ServiceName -args -s application:start(my_application) as I do it with Eclipse from the console. erlsrv.exe returns "unrecognized option application:start(my_application)" I also tried -args -Application start my_application Any ideas how to proceed ? John From baryluk@REDACTED Wed Jul 22 13:15:32 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Wed, 22 Jul 2009 13:15:32 +0200 Subject: [erlang-questions] Fw: Newbie question In-Reply-To: References: <1248195554.5048.358.camel@piko.site> Message-ID: <20090722111532.GA21205@smp.if.uj.edu.pl> On 07-21 13:35, kevin.hostelley@REDACTED wrote: > > > I don't understand why the {data, Data} pattern isn't receiving the > > > message. Can someone help me understand this? > > > > It is because Data is already bound to "ABC", so you try to match > > against {data, "ABC"}. > > > > So "Data" is still in scope even though the receive ended? If the receive > was inside a function, would the scope of "Data" be limited to that > function? Yes. Shell have somehowe different bounding and scope rules. Many things works very different in shell (it is generally emulation of real erlang code). Similary you can in shell do: case 1 of 2 -> Y = two; 1 -> X = one end. X. % this is bounded. In compiled code, you can only do this if all branches of case have "assigment" to X. Just remember to call f() or f(X) in shell. -- Witold Baryluk JID: witold.baryluk // jabster.pl -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: Digital signature URL: From clist@REDACTED Wed Jul 22 13:22:16 2009 From: clist@REDACTED (Angel) Date: Wed, 22 Jul 2009 13:22:16 +0200 Subject: [erlang-questions] How to distribute code using code server to slave nodes (II) In-Reply-To: <200907220006.07093.clist@uah.es> References: <200907220006.07093.clist@uah.es> Message-ID: <200907221322.16587.clist@uah.es> Hi again More on this issue ive managed to start a remote node using the code server start erlang in distribute mode.. on the Master Node. start the code server... code_server4:start( MyRemoteSlave). and finally slave:start(MyRemoteSlave,Nost,Name,SomeArgs). where SomeArgs contains "-setcookie mycookie and -hosts -loader inet" resulting in a timeout failure to start and setup the remote slave. But if i logon on the slave node and do... sinosuke@REDACTED:~> erl -hosts -loader inet -id one@ -name one@ -setcookie mycookie ??it success!! Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) (one@REDACTED)1> tcpdump shows network activity while the remote code server ask Master code server for the required stuff.. (adding +l to erl cmdline show remote code loading) So the aproach is correct but the remote node fails to load all the required code in time and the slave module on the master complains about timeout. ??Can i tweak this timeout value in order to let the remote node finish its code loading?? I whant to use "-loader inet" to avoid having to deploy my app code all over the remote nodes by hand. Regards Angel El Mi?rcoles, 22 de Julio de 2009 00:06:06 Angel Alvarez escribi?: > Hi > > Im learning erlang so i use to write toy programs just for fun > now Im learning to use the pool module, so i cant spread workers over some computers. > > The problem is that prior to pspawn processes i have to manually distribute code among all members of the pool. > I use to write something like this (i saw this somewhere on the net...) > > pool:start(pooltest, lists:concat(["-setcookie ", erlang:get_cookie()])), > distribute_app([mymodule1,mymodule2, ... , mymodulen]), > > > with distribute_app as: > > distribute_app(Modules) -> > %% extraer todos los nodos excepto el Master > RemoteNodes = [X || X <- pool:get_nodes(), X =/= node()], > %% Transferir el c?digo > lists:foreach(fun(Node) -> transfer_code(Node, Modules) end, RemoteNodes). > > transfer_code(Node, Modules) -> > [transfer_module(Node, Module) || Module <- Modules]. > > transfer_module(Node, Module) -> > {_Module, Binary, FileName} = code:get_object_code(Module), > rpc:call(Node, code, load_binary, [Module, FileName, Binary]). > > Instead of doing this, can i instruct remote code servers to ask master code server for code when > it need to locate new refences? > > Is seems to be related to starting erl with "--loader inet" ?Can anyone prove me with some pointers about this? > > Thanks > -- No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos. __________________________________________ Clist UAH a.k.a Angel __________________________________________ MySQL5: Vale, corromper los datos de forma silente no era una buena idea despues de todo. From roberto@REDACTED Wed Jul 22 13:25:35 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 22 Jul 2009 13:25:35 +0200 Subject: [erlang-questions] message passing between nodes using custom TCP In-Reply-To: References: Message-ID: <2DDD365E-CA17-4842-B03E-05CCC66BAFD2@widetag.com> On 21/lug/09, at 20:32, Tony Finch wrote: > > Multiple parallel TCP connections will only help across a WAN - you > should > see no benefit on a LAN. Your performance degradation probably comes > from > the multiplexing overhead. > > Tony. thank you tony. this test came out due to a twitter discussion with francesco, where he told me they were having such a mecanism: http://twitter.com/FrancescoC/status/2075669415 but i wonder if francesco was talking about a wan too, or if i a, missing a point somewhere. r. From clist@REDACTED Wed Jul 22 14:05:12 2009 From: clist@REDACTED (Angel Alvarez) Date: Wed, 22 Jul 2009 14:05:12 +0200 Subject: [erlang-questions] How to distribute code using code server to slave nodes (II) In-Reply-To: <200907221322.16587.clist@uah.es> References: <200907220006.07093.clist@uah.es> <200907221322.16587.clist@uah.es> Message-ID: <200907221405.12809.clist@uah.es> Hi stdlib/slave.erl function wait_for_slave has a timeout of 32 milliseconds that seem to be too short for a whole node startup sequence (on WLAN 54Mbits). Ill try to copy this module to private path and rename to myslave so altering the after clause dont mess with standard module and see wahts happens... wait_for_slave(Parent, Host, Name, Node, Args, LinkTo, Prog) -> Waiter = register_unique_name(0), case mk_cmd(Host, Name, Args, Waiter, Prog) of {ok, Cmd} -> %% io:format("Command: ~s~n", [Cmd]), open_port({spawn, Cmd}, [stream]), receive {SlavePid, slave_started} -> unregister(Waiter), slave_started(Parent, LinkTo, SlavePid) after 32000 -> %% If it seems that the node was partially started, %% try to kill it. Node = list_to_atom(lists:concat([Name, "@", Host])), case net_adm:ping(Node) of pong -> spawn(Node, erlang, halt, []), ok; _ -> ok end, Parent ! {result, {error, timeout}} end; Other -> Parent ! {result, Other} end. Regards /angel El Mi?rcoles, 22 de Julio de 2009 Angel escribi?: > Hi again > > More on this issue ive managed to start a remote node using the code server > > start erlang in distribute mode.. on the Master Node. > > start the code server... > code_server4:start( MyRemoteSlave). > > and finally > > slave:start(MyRemoteSlave,Nost,Name,SomeArgs). > > where SomeArgs contains "-setcookie mycookie and -hosts -loader inet" > > resulting in a timeout failure to start and setup the remote slave. > > But if i logon on the slave node and do... > > sinosuke@REDACTED:~> erl -hosts -loader inet -id one@ -name one@ -setcookie mycookie > > ??it success!! > > Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.7.2 (abort with ^G) > (one@REDACTED)1> > > > tcpdump shows network activity while the remote code server ask Master code server for the required stuff.. > (adding +l to erl cmdline show remote code loading) > > So the aproach is correct but the remote node fails to load all the required code in time and the slave module on the master > complains about timeout. > > ??Can i tweak this timeout value in order to let the remote node finish its code loading?? > > I whant to use "-loader inet" to avoid having to deploy my app code all over the remote nodes by hand. > > Regards Angel > > > El Mi?rcoles, 22 de Julio de 2009 00:06:06 Angel Alvarez escribi?: > > Hi > > > > Im learning erlang so i use to write toy programs just for fun > > now Im learning to use the pool module, so i cant spread workers over some computers. > > > > The problem is that prior to pspawn processes i have to manually distribute code among all members of the pool. > > I use to write something like this (i saw this somewhere on the net...) > > > > pool:start(pooltest, lists:concat(["-setcookie ", erlang:get_cookie()])), > > distribute_app([mymodule1,mymodule2, ... , mymodulen]), > > > > > > with distribute_app as: > > > > distribute_app(Modules) -> > > %% extraer todos los nodos excepto el Master > > RemoteNodes = [X || X <- pool:get_nodes(), X =/= node()], > > %% Transferir el c?digo > > lists:foreach(fun(Node) -> transfer_code(Node, Modules) end, RemoteNodes). > > > > transfer_code(Node, Modules) -> > > [transfer_module(Node, Module) || Module <- Modules]. > > > > transfer_module(Node, Module) -> > > {_Module, Binary, FileName} = code:get_object_code(Module), > > rpc:call(Node, code, load_binary, [Module, FileName, Binary]). > > > > Instead of doing this, can i instruct remote code servers to ask master code server for code when > > it need to locate new refences? > > > > Is seems to be related to starting erl with "--loader inet" ?Can anyone prove me with some pointers about this? > > > > Thanks > > > > > -- Agua para todo? No, Agua para Todos. ->>----------------------------------------------- Clist UAH a.k.a Angel ---------------------------------[www.uah.es]-<<-- G?indous 7 ser? peor y con m?s DRM, asi que usa Linux y disfruta de la vida From peter.mechlenborg@REDACTED Wed Jul 22 14:18:32 2009 From: peter.mechlenborg@REDACTED (Peter Mechlenborg) Date: Wed, 22 Jul 2009 14:18:32 +0200 Subject: [erlang-questions] Any message queue model in Erlang when doing Comet chat? In-Reply-To: <50db8f50907211812h6e8bc248r42db39f8ff5e40bf@mail.gmail.com> References: <50db8f50907210112p3841331co699f735fa354ce60@mail.gmail.com> <50db8f50907210323mbf0900k64336d1108af8d61@mail.gmail.com> <2D2E73FC-112F-4870-B1DC-851FEEF0C406@gmail.com> <50db8f50907211812h6e8bc248r42db39f8ff5e40bf@mail.gmail.com> Message-ID: Hi Do you have messages in a chat room persisted on the server (in MySQL) ? If so then I think what you are doing are fine. Every time a client connects it says: "give me all messages newer than time T" and if such messages exist the server delivers them to the client, otherwise it just holds the connection until the next message arrives or the 10 sec. time out occurs. What are your requirements for fault tolerance? Do you actually need to persist the chat rooms, or would it be ok to just hold them in RAM in the server? Maybe you can just cache the last few messages for each chat room in RAM, that way you would only need to touch the DB when a new client connects to a given room (maybe a DB can do that for you, I'm no DB expert). Maybe flat files would scale better than a RMDB? Have fun, -- Peter On Wed, Jul 22, 2009 at 3:12 AM, Zhuguo Shi wrote: > Hi Peter, > > I did the checking by sending messages in a high frequency, like you enter > some letters and press the "send" button quickly. My comet chat is like > Facebook's and the polling connection is automatically disconnect and > reconnect in 10 senconds (Facebook is about 50 seconds). So I doubt if you > send a message to a user that just disconnected and have not connected, the > message will lost. > > The test result is not good before I added the database sync mechanism. I > am not sure if there is any problem in my code or test. I did this comet > chat prototype just according to Richard's series ( > http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1/ > ). > > Any suggestions? > > From peter.mechlenborg@REDACTED Wed Jul 22 15:20:50 2009 From: peter.mechlenborg@REDACTED (Peter Mechlenborg) Date: Wed, 22 Jul 2009 15:20:50 +0200 Subject: include vs include_lib Message-ID: Hi I have found myself starting to always use '-include_lib()' instead of just '-include()', because it makes compilation easier - you don't need to specify include directories, and files become "self-contained" regarding compilation. Why do people use '-include()' instead of '-include_lib()'? Am I missing something obvious here? Have fun, -- Peter From baryluk@REDACTED Wed Jul 22 15:40:44 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Wed, 22 Jul 2009 15:40:44 +0200 Subject: [erlang-questions] How to distribute code using code server to slave nodes (II) In-Reply-To: <200907221405.12809.clist@uah.es> References: <200907220006.07093.clist@uah.es> <200907221322.16587.clist@uah.es> <200907221405.12809.clist@uah.es> Message-ID: <20090722134044.GA23173@smp.if.uj.edu.pl> On 07-22 14:05, Angel Alvarez wrote: > Hi > > stdlib/slave.erl > > function wait_for_slave has a timeout of 32 milliseconds that seem to be too short for a whole node startup sequence (on WLAN 54Mbits). > > after 32000 -> Sorry, but this is 32 seconds, not miliseconds. -- Witold Baryluk JID: witold.baryluk // jabster.pl -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: Digital signature URL: From masse@REDACTED Wed Jul 22 16:12:41 2009 From: masse@REDACTED (mats cronqvist) Date: Wed, 22 Jul 2009 16:12:41 +0200 Subject: include vs include_lib In-Reply-To: (Peter Mechlenborg's message of "Wed\, 22 Jul 2009 15\:20\:50 +0200") References: Message-ID: <87r5w8bycm.fsf@sterlett.hq.kred> Peter Mechlenborg writes: > Hi > I have found myself starting to always use '-include_lib()' instead of just > '-include()', because it makes compilation easier - you don't need to > specify include directories, and files become "self-contained" regarding > compilation. > > Why do people use '-include()' instead of '-include_lib()'? Am I missing > something obvious here? iirc, include_lib("blah/include/foo.hrl"). only works if "blah/ebin" is in the load path (i.e. what code:get_path() gives you). this might or might not be a problem. it's obviously not a problem for OTP supplied headers. mats From kevin@REDACTED Wed Jul 22 17:34:03 2009 From: kevin@REDACTED (Kevin A. Smith) Date: Wed, 22 Jul 2009 11:34:03 -0400 Subject: [erlang-questions] include vs include_lib In-Reply-To: References: Message-ID: For me, -include() and -include_lib() serve two different use cases. Use Case 1: Your project has -define'd constants, parse transforms, etc which are required for others to use your code. Examples include qlc or webmachine. Solution: Put your .hrl files in project_name/include. As long as your project is on the code path then clients of your code can use - include_lib() to load the .hrl files. Use Case 2: You've centralized common constants, etc in a .hrl file. These constants are intended to be used internally by your project code. This means at no time will any client need access to them. Solution: Put your .hrl files in project_name/src. Your code can use - include() to pull in the .hrl files. Keep in mind that both include directives are compile-time constructs. You can put all of your header files in src, compile your code using - include(), and then selectively "export" the header files you want to publish by copying them into include. I do this a lot since it makes my compile process easier -- no code path to futz with -- and it makes my client code cleaner -- they can use -include_lib to find my public header files. --Kevin On Jul 22, 2009, at 9:20 AM, Peter Mechlenborg wrote: > Hi > I have found myself starting to always use '-include_lib()' instead > of just > '-include()', because it makes compilation easier - you don't need to > specify include directories, and files become "self-contained" > regarding > compilation. > > Why do people use '-include()' instead of '-include_lib()'? Am I > missing > something obvious here? > > Have fun, > > -- Peter From ngreco@REDACTED Wed Jul 22 18:43:54 2009 From: ngreco@REDACTED (Nahuel Greco) Date: Wed, 22 Jul 2009 13:43:54 -0300 Subject: [erlang-questions] Solaris: Can't set long node name! In-Reply-To: <20090720180814.GE79703@h216-235-12-174.host.egate.net> References: <20090720173026.GD79703@h216-235-12-174.host.egate.net> <401d3ba30907201048v1a6cbbck5d33f48d82ffcf2a@mail.gmail.com> <20090720180814.GE79703@h216-235-12-174.host.egate.net> Message-ID: <47d93090907220943u4f324535i5d4b2944370b9230@mail.gmail.com> Try changing your hostname in /etc/hostname to a FQDN, e.g., myhost to myhost.mydomain Saludos, Nahuel Greco. On Mon, Jul 20, 2009 at 3:08 PM, Vance Shipley wrote: > Attila, > > Yes, yes it is. > > ? ? ? ?$ /usr/sbin/ping erlang.org > ? ? ? ?erlang.org is alive > > -- > ? ? ? ?-Vance > > On Mon, Jul 20, 2009 at 07:48:09PM +0200, Attila Rajmund Nohl wrote: > } ?Is name resolving properly setup on that Solaris? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From paul-trapexit@REDACTED Wed Jul 22 20:25:32 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Wed, 22 Jul 2009 11:25:32 -0700 (PDT) Subject: [erlang-questions] Mnesia local_content tables as transaction manager? In-Reply-To: References: Message-ID: global:trans/3 will let you lock a resource across multiple nodes. if your problem is simple this is sufficient. this won't help you with problems like "B dies while you are holding the global lock so you need to undo the operation on A", you'll have to roll stuff like that yourself. -- p On Tue, 21 Jul 2009, Jim McCoy wrote: > I want to perform some transactions over processes (but will handle > persistence via another channel) such that if process A and B are on > the same node I can be certain that once a "transaction function" > returns successfully both A and B have had the specified state > transformation applied to them. Ideally this would also work if A and > B were on different nodes, but I am content to move them both to the > same node to perform the transaction. Is this something that mnesia > local_content tables would be suitable for? Would > mnesia/local_content be overkill for this task? Am I better off > looking at another tool or example for this task? > > I guess what I am basically looking for is a distributed transaction > manager and I am hoping that there might be some part of mnesia or > other built-in facility that can be re-tasked for this purpose. Any > thoughts? > > jim > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From clist@REDACTED Wed Jul 22 21:03:39 2009 From: clist@REDACTED (Angel Alvarez) Date: Wed, 22 Jul 2009 21:03:39 +0200 Subject: [erlang-questions] How to distribute code using code server to slave nodes (II) (altering slave module for configurable timeout) In-Reply-To: <200907221405.12809.clist@uah.es> References: <200907220006.07093.clist@uah.es> <200907221322.16587.clist@uah.es> <200907221405.12809.clist@uah.es> Message-ID: <200907222103.39252.clist@uah.es> Tested and works!! Just in case someone finds it interesting scenario: Master Node "One" starts slaves by means of slave:start() for cluster operation Slaves bootup using remote code server provided by Master. Slaves timeout on LAN 100Mbits or less preveneting bootup (remote node start bur slave throws timeout error cancelling operation) stdlib/slave.erl has hardcoded timeout preventing proper operations due to full remote code loading overhead Solution (FOR TESTING) Instance stdlib/slave.erl into private myslave.erl adding minimal code to allow pass options down to inner receive construct where timeout needs to be changed on demand. minimal changes to stdlib/slave -> myslave to allow passing options to start/n Now you can set timeout as {timeout, Value} or defaul value 32000 TEST: Set timeout to 2 mins and wait for slave to finish bootup... myslave:start("192.168.9.240","one", [{timeout, 120000}], lists:concat(["-host 192.168.128.192 -loader inet ","-setcookie ", erlang:get_cookie()]) ). {error,timeout} Network isnt fast enough! Lets set timeout to 3 mins myslave:start("192.168.9.240","one", [{timeout, 180000}], lists:concat(["-hosts 192.168.128.192 -loader inet ","-setcookie ", erlang:get_cookie()]) ). {ok,'one@REDACTED'} Now my remote node can finish bootup sequence using the my local code server, and i dont have to distribute beam files for my application to all the nodes of my distributed cluster... :-) In addition to alter several start/1..5 wrapper "interesting" code lies in wait_for_slave() the code is clearly bad merey for testing porpouses... wait_for_slave(Parent, Host, Name, Node, Options, Args, LinkTo, Prog) -> Waiter = register_unique_name(0), %% case Options of [{timeout, Timeout }] -> SlaveTimeOut = Timeout; true -> SlaveTimeOut = 32000 end, %% case mk_cmd(Host, Name, Args, Waiter, Prog) of {ok, Cmd} -> %% io:format("Command: ~s~n", [Cmd]), open_port({spawn, Cmd}, [stream]), receive {SlavePid, slave_started} -> unregister(Waiter), slave_started(Parent, LinkTo, SlavePid) after SlaveTimeOut -> %% If it seems that the node was partially started, %% try to kill it. Node = list_to_atom(lists:concat([Name, "@", Host])), case net_adm:ping(Node) of pong -> spawn(Node, erlang, halt, []), ok; _ -> ok end, Parent ! {result, {error, timeout}} end; Other -> Parent ! {result, Other} end. Im not confident to provide industrial grade patches for this issue, but maybe the OTP team would provide proper upgrades on next releases if this feature is seen interesting. future directions include get list of "Master" loaded modules and compute total byte size. transfer over rsh mechanism equal amount of data to compute aproximate network bandwith Set apropiate timeout prior to bootup slaves... Regards /Angel El Mi?rcoles, 22 de Julio de 2009 Angel Alvarez escribi?: > Hi > > stdlib/slave.erl > > function wait_for_slave has a timeout of 32 milliseconds that seem to be too short for a whole node startup sequence (on WLAN 54Mbits). > > Ill try to copy this module to private path and rename to myslave so altering the after clause dont mess with standard > module and see wahts happens... > > > wait_for_slave(Parent, Host, Name, Node, Args, LinkTo, Prog) -> > Waiter = register_unique_name(0), > case mk_cmd(Host, Name, Args, Waiter, Prog) of > {ok, Cmd} -> > %% io:format("Command: ~s~n", [Cmd]), > open_port({spawn, Cmd}, [stream]), > receive > {SlavePid, slave_started} -> > unregister(Waiter), > slave_started(Parent, LinkTo, SlavePid) > after 32000 -> > %% If it seems that the node was partially started, > %% try to kill it. > Node = list_to_atom(lists:concat([Name, "@", Host])), > case net_adm:ping(Node) of > pong -> > spawn(Node, erlang, halt, []), > ok; > _ -> > ok > end, > Parent ! {result, {error, timeout}} > end; > Other -> > Parent ! {result, Other} > end. > > > Regards /angel > > El Mi?rcoles, 22 de Julio de 2009 Angel escribi?: > > Hi again > > > > More on this issue ive managed to start a remote node using the code server > > > > start erlang in distribute mode.. on the Master Node. > > > > start the code server... > > code_server4:start( MyRemoteSlave). > > > > and finally > > > > slave:start(MyRemoteSlave,Nost,Name,SomeArgs). > > > > where SomeArgs contains "-setcookie mycookie and -hosts -loader inet" > > > > resulting in a timeout failure to start and setup the remote slave. > > > > But if i logon on the slave node and do... > > > > sinosuke@REDACTED:~> erl -hosts -loader inet -id one@ -name one@ -setcookie mycookie > > > > ??it success!! > > > > Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] > > > > Eshell V5.7.2 (abort with ^G) > > (one@REDACTED)1> > > > > > > tcpdump shows network activity while the remote code server ask Master code server for the required stuff.. > > (adding +l to erl cmdline show remote code loading) > > > > So the aproach is correct but the remote node fails to load all the required code in time and the slave module on the master > > complains about timeout. > > > > ??Can i tweak this timeout value in order to let the remote node finish its code loading?? > > > > I whant to use "-loader inet" to avoid having to deploy my app code all over the remote nodes by hand. > > > > Regards Angel > > > > > > El Mi?rcoles, 22 de Julio de 2009 00:06:06 Angel Alvarez escribi?: > > > Hi > > > > > > Im learning erlang so i use to write toy programs just for fun > > > now Im learning to use the pool module, so i cant spread workers over some computers. > > > > > > The problem is that prior to pspawn processes i have to manually distribute code among all members of the pool. > > > I use to write something like this (i saw this somewhere on the net...) > > > > > > pool:start(pooltest, lists:concat(["-setcookie ", erlang:get_cookie()])), > > > distribute_app([mymodule1,mymodule2, ... , mymodulen]), > > > > > > > > > with distribute_app as: > > > > > > distribute_app(Modules) -> > > > %% extraer todos los nodos excepto el Master > > > RemoteNodes = [X || X <- pool:get_nodes(), X =/= node()], > > > %% Transferir el c?digo > > > lists:foreach(fun(Node) -> transfer_code(Node, Modules) end, RemoteNodes). > > > > > > transfer_code(Node, Modules) -> > > > [transfer_module(Node, Module) || Module <- Modules]. > > > > > > transfer_module(Node, Module) -> > > > {_Module, Binary, FileName} = code:get_object_code(Module), > > > rpc:call(Node, code, load_binary, [Module, FileName, Binary]). > > > > > > Instead of doing this, can i instruct remote code servers to ask master code server for code when > > > it need to locate new refences? > > > > > > Is seems to be related to starting erl with "--loader inet" ?Can anyone prove me with some pointers about this? > > > > > > Thanks > > > > > > > > > > > > -- No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos. ->>-------------------------------------------------- Angel J. Alvarez Miguel, Secci?n de Sistemas Area de Explotaci?n, Servicios Inform?ticos Edificio Torre de Control, Campus Externo UAH Alcal? de Henares 28806, Madrid ** ESPA?A ** RedIRIS Jabber: angel.uah.es@REDACTED ------------------------------------[www.uah.es]-<<-- MySQL5: Vale, corromper los datos de forma silente no era una buena idea despues de todo. -- No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos.->>----------------------------------------------- Clist UAH a.k.a Angel---------------------------------[www.uah.es]- From rapsey@REDACTED Thu Jul 23 07:59:45 2009 From: rapsey@REDACTED (Rapsey) Date: Thu, 23 Jul 2009 07:59:45 +0200 Subject: make on os x In-Reply-To: <20090723051703.GA31290@1wt.eu> References: <97619b170906110046u2e8bab45g4e9f53d345481702@mail.gmail.com> <97619b170906110051l4bc42045jc3e1a6b80eeb321c@mail.gmail.com> <20090723051703.GA31290@1wt.eu> Message-ID: <97619b170907222259r15c6b619ia3471c2f07570a0b@mail.gmail.com> Yes thank you. I figured it out eventually and used the same command as you wrote to build, but kqueue was still not getting enabled. This is the make command I eventually figured out works without issues (uses the default Makefile): make TARGET=osx CPU=i686 USE_KQUEUE=1 USE_POLL=1 USE_PCRE=1 Tested it on darwin and leopard and I haven't noticed any problems. Sergej On Thu, Jul 23, 2009 at 7:17 AM, Willy Tarreau wrote: > Hi, > > On Thu, Jun 11, 2009 at 09:51:00AM +0200, Rapsey wrote: > > Sorry error in -vv output, TARGET = darwin > > > > Sergej > > > > On Thu, Jun 11, 2009 at 9:46 AM, Rapsey wrote: > > > > > I'm trying to build haproxy with kqueue on osx leopard, but I don't > think > > > it's working. There is no mention of DENABLE_KQUEUE anywhere when it's > > > building it. > > > This is the make I use: > > > make Makefile.osx TARGET=darwin CPU=i686 USE_PCRE=1 all > > Ok you're not using the proper syntax, you need to use : > > make -f Makefile.osx TARGET=darwin CPU=i686 USE_PCRE=1 all > > Otherwise you tell make to build Makefile.osx, which already > exists so no error is reported. > > Also, please don't use 1.3.17 as it has a nasty bug which > can be triggered on 64-bit systems. > > Willy > > From rapsey@REDACTED Thu Jul 23 08:41:45 2009 From: rapsey@REDACTED (Rapsey) Date: Thu, 23 Jul 2009 08:41:45 +0200 Subject: make on os x In-Reply-To: <97619b170907222259r15c6b619ia3471c2f07570a0b@mail.gmail.com> References: <97619b170906110046u2e8bab45g4e9f53d345481702@mail.gmail.com> <97619b170906110051l4bc42045jc3e1a6b80eeb321c@mail.gmail.com> <20090723051703.GA31290@1wt.eu> <97619b170907222259r15c6b619ia3471c2f07570a0b@mail.gmail.com> Message-ID: <97619b170907222341s28744a5dt35d077c8842cbc15@mail.gmail.com> Whoops, replied to the wrong mailing list. Sergej On Thu, Jul 23, 2009 at 7:59 AM, Rapsey wrote: > Yes thank you. I figured it out eventually and used the same command as you > wrote to build, but kqueue was still not getting enabled. > This is the make command I eventually figured out works without issues > (uses the default Makefile): > > make TARGET=osx CPU=i686 USE_KQUEUE=1 USE_POLL=1 USE_PCRE=1 > > Tested it on darwin and leopard and I haven't noticed any problems. > > > Sergej > > > > On Thu, Jul 23, 2009 at 7:17 AM, Willy Tarreau wrote: > >> Hi, >> >> On Thu, Jun 11, 2009 at 09:51:00AM +0200, Rapsey wrote: >> > Sorry error in -vv output, TARGET = darwin >> > >> > Sergej >> > >> > On Thu, Jun 11, 2009 at 9:46 AM, Rapsey wrote: >> > >> > > I'm trying to build haproxy with kqueue on osx leopard, but I don't >> think >> > > it's working. There is no mention of DENABLE_KQUEUE anywhere when it's >> > > building it. >> > > This is the make I use: >> > > make Makefile.osx TARGET=darwin CPU=i686 USE_PCRE=1 all >> >> Ok you're not using the proper syntax, you need to use : >> >> make -f Makefile.osx TARGET=darwin CPU=i686 USE_PCRE=1 all >> >> Otherwise you tell make to build Makefile.osx, which already >> exists so no error is reported. >> >> Also, please don't use 1.3.17 as it has a nasty bug which >> can be triggered on 64-bit systems. >> >> Willy >> >> > From info@REDACTED Thu Jul 23 11:34:38 2009 From: info@REDACTED (info) Date: Thu, 23 Jul 2009 11:34:38 +0200 Subject: run an otp/application as a service Message-ID: <200907231134385053050@its3.ch> Hi All, Starting an OTP/Application with erlsrv is not evident for me even with the (poor) documentation ... I tried erlsrv.exe add ServiceName -args -s application:start(my_application) as I do it with Eclipse from the console. erlsrv.exe returns "unrecognized option application:start(my_application)" I also tried -args -Application start my_application Any ideas how to proceed ? John From vladdu55@REDACTED Thu Jul 23 12:08:49 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 23 Jul 2009 12:08:49 +0200 Subject: [erlang-questions] run an otp/application as a service In-Reply-To: <200907231134385053050@its3.ch> References: <200907231134385053050@its3.ch> Message-ID: <95be1d3b0907230308v78427c8du9893b21726a6d8dc@mail.gmail.com> Hi John, On Thu, Jul 23, 2009 at 11:34, info wrote: > > I tried erlsrv.exe add ServiceName -args -s application:start(my_application) > as I do it with Eclipse from the console. >From the docs: -s Mod [Func [Arg1, Arg2, ...]](init flag) Makes init call the specified function. Func defaults to start. If no arguments are provided, the function is assumed to be of arity 0. Otherwise it is assumed to be of arity 1, taking the list [Arg1,Arg2,...] as argument. so even if you call it like it's supposed to be -s application start my_application what will be called is application:start([my_application]) which won't work (notice the argument is a list). You should add your own startup function and let that start applications and whatever else you need. regards, Vlad From peter.mechlenborg@REDACTED Thu Jul 23 13:28:35 2009 From: peter.mechlenborg@REDACTED (Peter Mechlenborg) Date: Thu, 23 Jul 2009 13:28:35 +0200 Subject: [erlang-questions] include vs include_lib In-Reply-To: References: Message-ID: Hi Mats and Kevin Thanks for your input. Have fun, -- Peter From dave.pawson@REDACTED Thu Jul 23 14:06:36 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Thu, 23 Jul 2009 13:06:36 +0100 Subject: Mmmm. Erlang humour? Message-ID: <711a73df0907230506v257c3ae3pbf94b006e0cf57a@mail.gmail.com> http://browsertoolkit.com/fault-tolerance.png -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From gleber.p@REDACTED Thu Jul 23 14:11:45 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 23 Jul 2009 14:11:45 +0200 Subject: [erlang-questions] Mmmm. Erlang humour? In-Reply-To: <711a73df0907230506v257c3ae3pbf94b006e0cf57a@mail.gmail.com> References: <711a73df0907230506v257c3ae3pbf94b006e0cf57a@mail.gmail.com> Message-ID: <14f0e3620907230511m49f8fa6eie72911cfbf6232ed@mail.gmail.com> On Thu, Jul 23, 2009 at 14:06, Dave Pawson wrote: > http://browsertoolkit.com/fault-tolerance.png It's hilarious! From info@REDACTED Thu Jul 23 15:35:01 2009 From: info@REDACTED (info) Date: Thu, 23 Jul 2009 15:35:01 +0200 Subject: [erlang-questions] run an otp/application as a service References: <200907231134385053050@its3.ch>, <95be1d3b0907230308v78427c8du9893b21726a6d8dc@mail.gmail.com> Message-ID: <200907231535008706157@its3.ch> I don't understand what you mean ... I added one module starting.erl -module(starting). -export([start/0]. start()->application:start(my_application). erlsrv.exe add ServiceName -args -s starting(my_application) Hi John, On Thu, Jul 23, 2009 at 11:34, info wrote: > > I tried erlsrv.exe add ServiceName -args -s application:start(my_application) > as I do it with Eclipse from the console. >From the docs: -s Mod [Func [Arg1, Arg2, ...]](init flag) Makes init call the specified function. Func defaults to start. If no arguments are provided, the function is assumed to be of arity 0. Otherwise it is assumed to be of arity 1, taking the list [Arg1,Arg2,...] as argument. so even if you call it like it's supposed to be -s application start my_application what will be called is application:start([my_application]) which won't work (notice the argument is a list). You should add your own startup function and let that start applications and whatever else you need. regards, Vlad From vladdu55@REDACTED Thu Jul 23 15:45:21 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 23 Jul 2009 15:45:21 +0200 Subject: [erlang-questions] run an otp/application as a service In-Reply-To: <200907231535008706157@its3.ch> References: <200907231134385053050@its3.ch> <95be1d3b0907230308v78427c8du9893b21726a6d8dc@mail.gmail.com> <200907231535008706157@its3.ch> Message-ID: <95be1d3b0907230645m5635069aja65d4cbfac7a2764@mail.gmail.com> On Thu, Jul 23, 2009 at 15:35, info wrote: > I don't understand what you mean ... > I added one module starting.erl > > -module(starting). > -export([start/0]. > start()->application:start(my_application). > > erlsrv.exe?add?ServiceName?-args?-s?starting(my_application) I think it should be erlsrv.exe add ServiceName -args -s starting regards, Vlad From bengt.kleberg@REDACTED Thu Jul 23 15:48:42 2009 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 23 Jul 2009 15:48:42 +0200 Subject: [erlang-questions] run an otp/application as a service In-Reply-To: <200907231535008706157@its3.ch> References: <200907231134385053050@its3.ch> , <95be1d3b0907230308v78427c8du9893b21726a6d8dc@mail.gmail.com> <200907231535008706157@its3.ch> Message-ID: <1248356922.4817.37.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, Give then module definition below for starting you do the following with erl: erl -s starting I do not know how erlsrv.exe works, but if the rest of the command line is correct, the whole thing should then be: erlsrv.exe add ServiceName -args -s starting bengt On Thu, 2009-07-23 at 15:35 +0200, info wrote: > I don't understand what you mean ... > I added one module starting.erl > > -module(starting). > -export([start/0]. > start()->application:start(my_application). > > erlsrv.exe add ServiceName -args -s starting(my_application) > > > > Hi John, > > On Thu, Jul 23, 2009 at 11:34, info wrote: > > > > I tried erlsrv.exe add ServiceName -args -s application:start(my_application) > > as I do it with Eclipse from the console. > > From the docs: > > -s Mod [Func [Arg1, Arg2, ...]](init flag) > Makes init call the specified function. Func defaults to start. If > no arguments are provided, the function is assumed to be of arity 0. > Otherwise it is assumed to be of arity 1, taking the list > [Arg1,Arg2,...] as argument. > > so even if you call it like it's supposed to be > -s application start my_application > what will be called is application:start([my_application]) which won't > work (notice the argument is a list). > > You should add your own startup function and let that start > applications and whatever else you need. > > regards, > Vlad From chsu79@REDACTED Thu Jul 23 15:56:54 2009 From: chsu79@REDACTED (Christian) Date: Thu, 23 Jul 2009 15:56:54 +0200 Subject: If you dont spend enough time web surfing at work: stackoverflow.com Message-ID: Maybe I'm just very late to the game here, but stackoverflow.com is a great place for people to find answers to things related to software development, and there is a few people there answering erlang-related questions. I think if more people here would keep an eye on questions and vote up good answers, or leave comments when something is subtly wrong, this could help Erlang. Make it seem like you're not "alone programming in it". See http://stackoverflow.com/questions/tagged/erlang or subscribe RSS from http://stackoverflow.com/feeds/tag/erlang (try out creating an account with openid if you already have a gmail or yahoo account, or something else that is an openid publisher, I wish all sites did it this way). From dmercer@REDACTED Thu Jul 23 16:03:07 2009 From: dmercer@REDACTED (David Mercer) Date: Thu, 23 Jul 2009 09:03:07 -0500 Subject: [erlang-questions] Mmmm. Erlang humour? In-Reply-To: <711a73df0907230506v257c3ae3pbf94b006e0cf57a@mail.gmail.com> References: <711a73df0907230506v257c3ae3pbf94b006e0cf57a@mail.gmail.com> Message-ID: <4C4940B24D294213BAD7B845F23522C8@SSI.CORP> At the risk of being called a dullard, I don't get it. I couldn't figure out who was talking, since some of the speech balloons had multiple call-outs from them, and couldn't figure out if that was supposed to be the "chorus" or what? Or maybe the multiple call-outs was a crown on the speech balloons? Just couldn't quite figure it out. It made more sense if it was just those two guys talking... > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Dave Pawson > Sent: Thursday, July 23, 2009 7:07 AM > To: Erlang Questions > Subject: [erlang-questions] Mmmm. Erlang humour? > > http://browsertoolkit.com/fault-tolerance.png > > > > > -- > Dave Pawson > XSLT XSL-FO FAQ. > Docbook FAQ. > http://www.dpawson.co.uk > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org From info@REDACTED Thu Jul 23 16:27:48 2009 From: info@REDACTED (info) Date: Thu, 23 Jul 2009 16:27:48 +0200 Subject: [erlang-questions] run an otp/application as a service References: <200907231134385053050@its3.ch>, <95be1d3b0907230308v78427c8du9893b21726a6d8dc@mail.gmail.com>, <200907231535008706157@its3.ch>, <95be1d3b0907230645m5635069aja65d4cbfac7a2764@mail.gmail.com> Message-ID: <200907231627480776192@its3.ch> Unrecognized option starting On Thu, Jul 23, 2009 at 15:35, info wrote: > I don't understand what you mean ... > I added one module starting.erl > > -module(starting). > -export([start/0]. > start()- >application:start(my_application). > > erlsrv.exe add ServiceName -args -s starting(my_application) I think it should be erlsrv.exe add ServiceName -args -s starting regards, Vlad From rtrlists@REDACTED Thu Jul 23 18:05:11 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 23 Jul 2009 17:05:11 +0100 Subject: [erlang-questions] run an otp/application as a service In-Reply-To: <200907231627480776192@its3.ch> References: <200907231134385053050@its3.ch> <95be1d3b0907230308v78427c8du9893b21726a6d8dc@mail.gmail.com> <200907231535008706157@its3.ch> <95be1d3b0907230645m5635069aja65d4cbfac7a2764@mail.gmail.com> <200907231627480776192@its3.ch> Message-ID: <6a3ae47e0907230905v7f9e3ffdn51de707414386df0@mail.gmail.com> The -args option takes one (!) argument, so try: erlsrv.exe add ServiceName -args "-s starting" Robby From info@REDACTED Thu Jul 23 19:15:14 2009 From: info@REDACTED (info) Date: Thu, 23 Jul 2009 19:15:14 +0200 Subject: [erlang-questions] run an otp/application as a service References: <200907231134385053050@its3.ch>, <95be1d3b0907230308v78427c8du9893b21726a6d8dc@mail.gmail.com>, <200907231535008706157@its3.ch>, <95be1d3b0907230645m5635069aja65d4cbfac7a2764@mail.gmail.com>, <200907231627480776192@its3.ch>, <6a3ae47e0907230905v7f9e3ffdn51de707414386df0@mail.gmail.com> Message-ID: <200907231915136922978@its3.ch> Hi Robby, You, you know how to read the doc ! I progress ... The events observer gives this: Erlang machine stopped instantly (distribution name conflict?). The service is not restarted as OnFail is set to ignore. Any idea ? If I resume the situation: - my OTP application runs correctly in Eclipse; - my OTP application hangs in werl.exe - erlsrv.exe add ServiceName -args "-s startting" add the service to the system - erlsrv.exe start ServiceName start the service but the observer gives the above message and the service doesn't start. The -args option takes one (!) argument, so try: erlsrv.exe add ServiceName -args "-s starting" Robby From steven.charles.davis@REDACTED Thu Jul 23 19:20:57 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 23 Jul 2009 10:20:57 -0700 (PDT) Subject: The is_record() guard Message-ID: I have an application that uses records extensively. At one point in processing these records I have a match function: transform(R) when is_record(R, rec_a) -> ... ; transform(R) when is_record(R, rec_b) -> ... ; transform(R) when is_record(R, rec_c) -> ... ; ... A first run of fprof appears to show that this function match is bottlenecking (noticeably so, but not in dire fashion). I'm planning a comparative run using transform(element(1, R), R) as the call signature: transform(rec_a, R) -> ...; transform(rec_b, R) -> ...; transform(rec_c, R) -> ...; .... However - before I refactor, does anybody happen to know whether this performance hit would this be expected? Is the is_record guard known to be slow/heavy in some way? Regards, /sd From rtrlists@REDACTED Thu Jul 23 19:35:31 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 23 Jul 2009 18:35:31 +0100 Subject: [erlang-questions] run an otp/application as a service In-Reply-To: <200907231915136922978@its3.ch> References: <200907231134385053050@its3.ch> <95be1d3b0907230308v78427c8du9893b21726a6d8dc@mail.gmail.com> <200907231535008706157@its3.ch> <95be1d3b0907230645m5635069aja65d4cbfac7a2764@mail.gmail.com> <200907231627480776192@its3.ch> <6a3ae47e0907230905v7f9e3ffdn51de707414386df0@mail.gmail.com> <200907231915136922978@its3.ch> Message-ID: <6a3ae47e0907231035p73bfda4s28bb50efd4e7c2d4@mail.gmail.com> On Thu, Jul 23, 2009 at 6:15 PM, info wrote: > Hi Robby, > You, you know how to read the doc ! > I progress ... > The events observer gives this: > Erlang machine stopped instantly (distribution name conflict?). The service > is not restarted as OnFail is set to ignore. > Any idea ? > If I resume the situation: > - my OTP application runs correctly in Eclipse; > - my OTP application hangs in werl.exe > - erlsrv.exe add ServiceName -args "-s startting" add the service to the > system > - erlsrv.exe start ServiceName start the service but the observer gives the > above message and the service doesn't start. > > Do you require a given node name? You can add one using the -name option: erlsrv add ServiceName -name mynode -args "-s startting" And if you are dependent on finding stuff on your PATH and running in a particular folder, you may want to set those on the running service as well, e.g.: erlsrv add ServiceName -name mynode -workdir C:/TEMP -env PATH=%PATH% -args "-s startting" And if you have a proper OTP release structure, then start your service with the boot and config options. For example: erlsrv add ServiceName -name mynode -workdir C:/TEMP -env PATH=%PATH% -args "-boot myboot -config myconfig -setcookie mycookie" Oh, and it helps tremendously if everything is running in a folder that has no spaces! Windows is juuust grrreat! Robby From oscar@REDACTED Thu Jul 23 19:35:15 2009 From: oscar@REDACTED (=?ISO-8859-1?Q?Oscar_Hellstr=F6m?=) Date: Thu, 23 Jul 2009 18:35:15 +0100 Subject: [erlang-questions] The is_record() guard In-Reply-To: References: Message-ID: <4A689F53.4090204@erlang-consulting.com> Hi Steve, You could also do it this way: transform(#rec_a{} = R) -> ... ; transform(#rec_b{} = R) -> ... ; ... IIRC there will indeed be a performance hit when using guards instead of pattern matchings in the function head. Not sure if this is still the case, or if my benchmarks were sufficient though. The situation then was something like: normalize(Char) when Char < 10 -> ... normalize(Char) when Char > 13, Char < ... -> ... Putting the values explicitly in the function head gave more code, but better performance. Not that it mattered in the end though but useless optimizations are very common... Steve Davis wrote: > I have an application that uses records extensively. At one point in > processing these records I have a match function: > > transform(R) when is_record(R, rec_a) -> ... ; > transform(R) when is_record(R, rec_b) -> ... ; > transform(R) when is_record(R, rec_c) -> ... ; > ... > > A first run of fprof appears to show that this function match is > bottlenecking (noticeably so, but not in dire fashion). > > I'm planning a comparative run using transform(element(1, R), R) as > the call signature: > > transform(rec_a, R) -> ...; > transform(rec_b, R) -> ...; > transform(rec_c, R) -> ...; > .... > > However - before I refactor, does anybody happen to know whether this > performance hit would this be expected? > > Is the is_record guard known to be slow/heavy in some way? > > Regards, > /sd > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- Oscar Hellstr?m, oscar@REDACTED Office: +44 20 7655 0337 Mobile: +44 798 45 44 773 Erlang Training and Consulting Ltd http://www.erlang-consulting.com/ From info@REDACTED Thu Jul 23 19:52:22 2009 From: info@REDACTED (info) Date: Thu, 23 Jul 2009 19:52:22 +0200 Subject: [erlang-questions] run an otp/application as a service References: <200907231134385053050@its3.ch>, <95be1d3b0907230308v78427c8du9893b21726a6d8dc@mail.gmail.com>, <200907231535008706157@its3.ch>, <95be1d3b0907230645m5635069aja65d4cbfac7a2764@mail.gmail.com>, <200907231627480776192@its3.ch>, <6a3ae47e0907230905v7f9e3ffdn51de707414386df0@mail.gmail.com>, <200907231915136922978@its3.ch>, <6a3ae47e0907231035p73bfda4s28bb50efd4e7c2d4@mail.gmail.com> Message-ID: <200907231952216674182@its3.ch> No node name. I progress... I added the option DebugType console and the working directory. - erlsrv.exe add ServiceName -w ".../ebin" -DebugType console -args "-s starting" When I start the service the console gives me the following message: =INFO REPORT==== 22-Jul-2009::19:29:51 === > application: my_application > exited: {shutdown,{my_application,start,[normal,[]]}} > type: temporary > {error,{shutdown,{my_application,start,[normal,[]]}}} Note that, when I start my application in werl, I have the same message !!! ONLY with Eclipse can my application run ????? On Thu, Jul 23, 2009 at 6:15 PM, info wrote: > Hi Robby, > You, you know how to read the doc ! > I progress ... > The events observer gives this: > Erlang machine stopped instantly (distribution name conflict?). The service > is not restarted as OnFail is set to ignore. > Any idea ? > If I resume the situation: > - my OTP application runs correctly in Eclipse; > - my OTP application hangs in werl.exe > - erlsrv.exe add ServiceName -args "-s startting" add the service to the > system > - erlsrv.exe start ServiceName start the service but the observer gives the > above message and the service doesn't start. > > Do you require a given node name? You can add one using the -name option: erlsrv add ServiceName -name mynode -args "-s startting" And if you are dependent on finding stuff on your PATH and running in a particular folder, you may want to set those on the running service as well, e.g.: erlsrv add ServiceName -name mynode -workdir C:/TEMP -env PATH=%PATH% -args "-s startting" And if you have a proper OTP release structure, then start your service with the boot and config options. For example: erlsrv add ServiceName -name mynode -workdir C:/TEMP -env PATH=%PATH% -args "-boot myboot -config myconfig -setcookie mycookie" Oh, and it helps tremendously if everything is running in a folder that has no spaces! Windows is juuust grrreat! Robby From steven.charles.davis@REDACTED Thu Jul 23 20:02:24 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 23 Jul 2009 13:02:24 -0500 Subject: [erlang-questions] The is_record() guard In-Reply-To: <4A689F53.4090204@erlang-consulting.com> References: <4A689F53.4090204@erlang-consulting.com> Message-ID: <4A68A5B0.7020906@gmail.com> Hi Oscar, Oscar Hellstr?m wrote: > You could also do it this way: > > transform(#rec_a{} = R) -> ... ; > transform(#rec_b{} = R) -> ... ; > ... Ah yes. Hmmm... looks like I have bought myself two possible refactors to try/test now ;( > Not that it mattered in the end though but useless > optimizations are very common... > Yep, I have tried to write the code "correct" first - it was the fprof result that made me look at this - this match function is necessarily called at every invocation and is by far and away the slowest part of the code. Best regards, Steve From rvirding@REDACTED Thu Jul 23 22:16:19 2009 From: rvirding@REDACTED (Robert Virding) Date: Thu, 23 Jul 2009 22:16:19 +0200 Subject: [erlang-questions] The is_record() guard In-Reply-To: <4A68A5B0.7020906@gmail.com> References: <4A689F53.4090204@erlang-consulting.com> <4A68A5B0.7020906@gmail.com> Message-ID: <3dbc6d1c0907231316u506722b3w8c5348c055e5e060@mail.gmail.com> Hi Steve, 2009/7/23 Steve Davis > Hi Oscar, > Oscar Hellstr?m wrote: > >> You could also do it this way: >> >> transform(#rec_a{} = R) -> ... ; >> transform(#rec_b{} = R) -> ... ; >> ... >> > > Ah yes. Hmmm... looks like I have bought myself two possible refactors to > try/test now ;( This is is a very good way of doing the test, both clean and efficient. It will expand the records into tuples and do ONE tuple match for them all. It is probably even faster than extracting the record name from the tuple with element/1. > > > Not that it mattered in the end though but useless >> optimizations are very common... >> >> > Yep, I have tried to write the code "correct" first - it was the fprof > result that made me look at this - this match function is necessarily called > at every invocation and is by far and away the slowest part of the code. No, using the is_record guard test is not a good way of doing it if you have the record definition. It might also be useful if the record is in the middle of a large pattern and it would be messy to write it as a pattern, but it is slower. Robert From baryluk@REDACTED Thu Jul 23 22:30:42 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Thu, 23 Jul 2009 22:30:42 +0200 Subject: [erlang-questions] Re: Unicast 20k messages, $500-$1000 bounty In-Reply-To: <4A6054D4.3020605@erlang-consulting.com> References: <28004.1247796787@snookles.snookles.com> <4A603E6C.7050102@di.uminho.pt> <87DEBDF2-99F8-4639-BB63-9A92564360F6@gmail.com> <7b1277d4-e1a8-4400-be33-21d7781363be@b14g2000yqd.googlegroups.com> <035F4839-0510-4D09-A575-6ADC6B511C15@gmail.com> <4A6054D4.3020605@erlang-consulting.com> Message-ID: <1248381042.22533.8.camel@sredniczarny> Dnia 2009-07-17, pi? o godzinie 12:39 +0200, Ulf Wiger pisze: > Joel Reymont wrote: > > > Jim, > > > > On Jul 17, 2009, at 11:12 AM, Jim Morris wrote: > > > >> It is Socket programming 101 that writes can block when the TCP > >> receiver does not read and the flow control causes the TCP buffers to > >> backup. > > > > This all makes sense. I will redo the code using just 1 middleman > > process then. > > You may want to consider the following passage from the inet man page > on setopts/2: > > {delay_send, Boolean} > Normally, when an Erlang process sends to a socket, the driver will try > to immediately send the data. If that fails, the driver will use any > means available to queue up the message to be sent whenever the > operating system says it can handle it. Setting {delay_send, true} will > make all messages queue up. This makes the messages actually sent onto > the network be larger but fewer. The option actually affects the > scheduling of send requests versus Erlang processes instead of changing > any real property of the socket. Needless to say it is an implementation > specific option. Default is false. > > I'm not sure if setting {delay_send, true} would have any beneficial > effect on your particular problem, but the text does shed some light > on the semantics of sending to a socket. > > BR, > Ulf W There is also similar way of doing this under Linux. Linux' TCP sockets, have option called TCP_CORK. This is generally delay_send (but in kernel space). See man page tcp(7) for details, and other useful socket options. This is way I access this option in Erlang. cork(TcpSocket) -> inet:setopts(TcpSocket,[{raw,6,3,<<1:32/native>>}]). uncork(TcpSocket) -> inet:setopts(TcpSocket,[{raw,6,3,<<0:32/native>>}]). Witek -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From rvirding@REDACTED Fri Jul 24 00:15:23 2009 From: rvirding@REDACTED (Robert Virding) Date: Fri, 24 Jul 2009 00:15:23 +0200 Subject: [erlang-questions] If you dont spend enough time web surfing at work: stackoverflow.com In-Reply-To: References: Message-ID: <3dbc6d1c0907231515p7f060ae4pb9e5c5713ea500e@mail.gmail.com> I think they might be happy to get more people who answered questions and came with comments. Robert 2009/7/23 Christian > Maybe I'm just very late to the game here, but stackoverflow.com is a > great place for people to find answers to things related to software > development, and there is a few people there answering erlang-related > questions. > > I think if more people here would keep an eye on questions and vote up > good answers, or leave comments when something is subtly wrong, this > could help Erlang. Make it seem like you're not "alone programming in > it". > > See http://stackoverflow.com/questions/tagged/erlang or subscribe RSS > from http://stackoverflow.com/feeds/tag/erlang (try out creating an > account with openid if you already have a gmail or yahoo account, or > something else that is an openid publisher, I wish all sites did it > this way). > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From steven.charles.davis@REDACTED Fri Jul 24 01:38:21 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 23 Jul 2009 18:38:21 -0500 Subject: [erlang-questions] The is_record() guard In-Reply-To: <3dbc6d1c0907231316u506722b3w8c5348c055e5e060@mail.gmail.com> References: <4A689F53.4090204@erlang-consulting.com> <4A68A5B0.7020906@gmail.com> <3dbc6d1c0907231316u506722b3w8c5348c055e5e060@mail.gmail.com> Message-ID: <4A68F46D.6060800@gmail.com> Vary useful info -- thank you both Oscar and Robert. It definitely sounds like the 'direct record match' is the most promising answer. If the results don't meet this expectation then I'll post comparatives when I have some reliable numbers. /sd Robert Virding wrote: > Hi Steve, > > 2009/7/23 Steve Davis > > > Hi Oscar, > > Oscar Hellstr?m wrote: > > You could also do it this way: > > transform(#rec_a{} = R) -> ... ; > transform(#rec_b{} = R) -> ... ; > ... > > > Ah yes. Hmmm... looks like I have bought myself two possible > refactors to try/test now ;( > > > This is is a very good way of doing the test, both clean and efficient. > It will expand the records into tuples and do ONE tuple match for them > all. It is probably even faster than extracting the record name from the > tuple with element/1. > > > > Not that it mattered in the end though but useless > optimizations are very common... > > > Yep, I have tried to write the code "correct" first - it was the > fprof result that made me look at this - this match function is > necessarily called at every invocation and is by far and away the > slowest part of the code. > > > No, using the is_record guard test is not a good way of doing it if you > have the record definition. It might also be useful if the record is in > the middle of a large pattern and it would be messy to write it as a > pattern, but it is slower. > > Robert > From rtrlists@REDACTED Fri Jul 24 10:11:08 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 24 Jul 2009 09:11:08 +0100 Subject: [erlang-questions] run an otp/application as a service In-Reply-To: <200907231952216674182@its3.ch> References: <200907231134385053050@its3.ch> <95be1d3b0907230308v78427c8du9893b21726a6d8dc@mail.gmail.com> <200907231535008706157@its3.ch> <95be1d3b0907230645m5635069aja65d4cbfac7a2764@mail.gmail.com> <200907231627480776192@its3.ch> <6a3ae47e0907230905v7f9e3ffdn51de707414386df0@mail.gmail.com> <200907231915136922978@its3.ch> <6a3ae47e0907231035p73bfda4s28bb50efd4e7c2d4@mail.gmail.com> <200907231952216674182@its3.ch> Message-ID: <6a3ae47e0907240111m59a9b4c8xef765dfa8d0f2370@mail.gmail.com> On Thu, Jul 23, 2009 at 6:52 PM, info wrote: > No node name. > I progress... > I added the option DebugType console and the working directory. > - erlsrv.exe add ServiceName -w ".../ebin" -DebugType console > -args "-s starting" > When I start the service the console gives me the following message: > > =INFO REPORT==== 22-Jul-2009::19:29:51 === > > application: my_application > > exited: {shutdown,{my_application,start,[normal,[]]}} > > type: temporary > > {error,{shutdown,{my_application,start,[normal,[]]}}} > Note that, when I start my application in werl, I have the same message !!! > ONLY with Eclipse can my application run ????? > Sounds like your Eclipse environment isn't the same as the normal erl one. Never used Eclipse, so I wouldn't know where to look. It is sometimes easier to figure out what is wrong by starting an erl shell and then start your app from there. You might get a better insight that way. Did you get a crash dump? If yes, then you could see if there's anything interesing in there via the CrashDumpViewer (available as part of the webtool, try webtool:start() and go to the website it serves)). Robby From info@REDACTED Fri Jul 24 11:03:44 2009 From: info@REDACTED (info) Date: Fri, 24 Jul 2009 11:03:44 +0200 Subject: [erlang-questions] run an otp/application as a service References: <200907231134385053050@its3.ch>, <95be1d3b0907230308v78427c8du9893b21726a6d8dc@mail.gmail.com>, <200907231535008706157@its3.ch>, <95be1d3b0907230645m5635069aja65d4cbfac7a2764@mail.gmail.com>, <200907231627480776192@its3.ch>, <6a3ae47e0907230905v7f9e3ffdn51de707414386df0@mail.gmail.com>, <200907231915136922978@its3.ch>, <6a3ae47e0907231035p73bfda4s28bb50efd4e7c2d4@mail.gmail.com>, <200907231952216674182@its3.ch>, <8268eea30907240039gf1c73d6m3e306488fb00551b@mail.gmail.com> Message-ID: <200907241103443475081@its3.ch> Hi, The path is defined with -w, is it wrong ? Hi. What is your path to your application? My guess is that it can not find your application. I do not know if adding your path to your application with -pa : erlsrv.exe add ServiceName -w ".../ebin" -args "-pa C:\...\YourApp\ebin -s starting" Kind regards Andreas Hillqvist 2009/7/23 info : > No node name. > I progress... > I added the option DebugType console and the working directory. > - erlsrv.exe add ServiceName -w ".../ebin" -DebugType console -args "-s starting" > When I start the service the console gives me the following message: > > =INFO REPORT==== 22-Jul-2009::19:29:51 === > > application: my_application > > exited: {shutdown,{my_application,start,[normal,[]]}} > > type: temporary > > {error,{shutdown,{my_application,start,[normal,[]]}}} > Note that, when I start my application in werl, I have the same message !!! > ONLY with Eclipse can my application run ????? > > On Thu, Jul 23, 2009 at 6:15 PM, info wrote: > > > Hi Robby, > > You, you know how to read the doc ! > > I progress ... > > The events observer gives this: > > Erlang machine stopped instantly (distribution name conflict?). The service > > is not restarted as OnFail is set to ignore. > > Any idea ? > > If I resume the situation: > > - my OTP application runs correctly in Eclipse; > > - my OTP application hangs in werl.exe > > - erlsrv.exe add ServiceName -args "-s startting" add the service to the > > system > > - erlsrv.exe start ServiceName start the service but the observer gives the > > above message and the service doesn't start. > > > > > > Do you require a given node name? You can add one using the -name option: > > erlsrv add ServiceName -name mynode -args "-s startting" > > And if you are dependent on finding stuff on your PATH and running in a > particular folder, you may want to set those on the running service as well, > e.g.: > > erlsrv add ServiceName -name mynode -workdir C:/TEMP -env PATH=%PATH% -args > "-s startting" > > And if you have a proper OTP release structure, then start your service with > the boot and config options. For example: > > erlsrv add ServiceName -name mynode -workdir C:/TEMP -env PATH=%PATH% -args > "-boot myboot -config myconfig -setcookie mycookie" > > Oh, and it helps tremendously if everything is running in a folder that has > no spaces! Windows is juuust grrreat! > > Robby > From andreasmk2@REDACTED Fri Jul 24 11:52:59 2009 From: andreasmk2@REDACTED (Andreas Bakurov) Date: Fri, 24 Jul 2009 12:52:59 +0300 Subject: [erlang-questions] Mmmm. Erlang humour? In-Reply-To: <711a73df0907230506v257c3ae3pbf94b006e0cf57a@mail.gmail.com> References: <711a73df0907230506v257c3ae3pbf94b006e0cf57a@mail.gmail.com> Message-ID: > > http://browsertoolkit.com/fault-tolerance.png Key-Value store is also a database. (since the storage is logically organised, see wikipedia) Key-Value store is completely independent of Map-Reduce and vice versa. It's like deducing that all vegetables are green and anything is green is a vegetable. From rapsey@REDACTED Fri Jul 24 11:59:48 2009 From: rapsey@REDACTED (Rapsey) Date: Fri, 24 Jul 2009 11:59:48 +0200 Subject: [erlang-questions] Mmmm. Erlang humour? In-Reply-To: References: <711a73df0907230506v257c3ae3pbf94b006e0cf57a@mail.gmail.com> Message-ID: <97619b170907240259i624dbd4ve1edde097883db3e@mail.gmail.com> You must be a riot at dinner parties. Sergej On Fri, Jul 24, 2009 at 11:52 AM, Andreas Bakurov wrote: > > > > http://browsertoolkit.com/fault-tolerance.png > > > > Key-Value store is also a database. (since the storage is logically > organised, see wikipedia) > Key-Value store is completely independent of Map-Reduce and vice versa. > It's like deducing that all vegetables are green and anything is green is a > vegetable. > From v@REDACTED Fri Jul 24 14:20:32 2009 From: v@REDACTED (Valentin Micic) Date: Fri, 24 Jul 2009 14:20:32 +0200 Subject: lists:append performance Message-ID: <200907241219.n6OCJ9lc015863@mail.pharos-avantgard.com> Hi all, I've always been under an impression that inserting element into the list and then reversing the list to convert it from LIFO to FIFO would generally be faster then using lists:append/2 to create FIFO immediately. I've never tested this assertion for a relatively small lists, thus, I've performed some tests using a list of 240 elements, having each element as a string like: "The quick brown fox jumps over a lazy dog". I've written two different functions: 1) insert (Count) that would always execute an insert like this: ["1"|OldList] 2) append( Count ), which would always execute a call to lists:append( OldList, ["1"] ). I cannot say that I haven't been surprised by a result: 50> timer:tc( benchmarks, insert, [1000000000] ). {72152000,ok} )51> timer:tc( benchmarks, append, [1000000000] ). {72137000,ok} It seems that it doesn't make much of a difference if one uses one or another. Would this hold for a larger lists as well, and if not - how large a list should be in order to see a notable deterioration in lists:append/2 performance? Thanks in advance. V. From vladdu55@REDACTED Fri Jul 24 14:32:31 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 24 Jul 2009 14:32:31 +0200 Subject: [erlang-questions] lists:append performance In-Reply-To: <200907241219.n6OCJ9lc015863@mail.pharos-avantgard.com> References: <200907241219.n6OCJ9lc015863@mail.pharos-avantgard.com> Message-ID: <95be1d3b0907240532r119fa5d4me72f021bb41da2cc@mail.gmail.com> Hi, On Fri, Jul 24, 2009 at 14:20, Valentin Micic wrote: > I've always been under an impression that inserting element into the list > and then reversing the list to convert it from LIFO to FIFO would generally > be faster then using lists:append/2 to create FIFO immediately. I > 1) insert (Count) that would always execute an insert like this: > ["1"|OldList] Do you mean that insert does the above and then reverts the list, every time? If yes, that's the explanation (reversing takes a little longer than traversing the list) If no, then it would be interesting to see the benchmark code to find out what you did measure :-) regards, Vlad From v@REDACTED Fri Jul 24 14:47:04 2009 From: v@REDACTED (Valentin Micic) Date: Fri, 24 Jul 2009 14:47:04 +0200 Subject: [erlang-questions] lists:append performance In-Reply-To: <95be1d3b0907240532r119fa5d4me72f021bb41da2cc@mail.gmail.com> Message-ID: <200907241245.n6OCjelc016028@mail.pharos-avantgard.com> No, lists:reverse/1 is never executed. The goal was to see how slower lists:append/2 is. The code looks like indicated below: -module( benchmarks ). % -------------------- % List of 240 elements % -------------------- -define ( L240, [ "The quick brown fox jumps over a lazy dog", "The quick brown fox jumps over a lzay dog", ... ... "The quick brown fox jumps over a lzay dog"] ). export( [ split_test/1 % split_test( Count ), ,reverse_test/1 % reverse_test( Count ) ,insert/1 % insert( Count ) ,append/1 % append( Count ) ] ). ... ... insert( 0 ) -> ok; insert( Count ) -> _L = ["1"|?L240], isert( Count - 1 ) . append( 0 ) -> ok; append( Count ) -> _ = lists:append( ?L240, ["1"] ), append( Count - 1 ) . I've tried to measure how long it takes to execute a single operation. Did it using a substantial number of operations (Count=1000000000) so it can be measured. V. -----Original Message----- From: Vlad Dumitrescu [mailto:vladdu55@REDACTED] Sent: 24 July 2009 02:33 PM To: Valentin Micic Cc: Erlang Users' List Subject: Re: [erlang-questions] lists:append performance Hi, On Fri, Jul 24, 2009 at 14:20, Valentin Micic wrote: > I've always been under an impression that inserting element into the list > and then reversing the list to convert it from LIFO to FIFO would generally > be faster then using lists:append/2 to create FIFO immediately. I > 1) insert (Count) that would always execute an insert like this: > ["1"|OldList] Do you mean that insert does the above and then reverts the list, every time? If yes, that's the explanation (reversing takes a little longer than traversing the list) If no, then it would be interesting to see the benchmark code to find out what you did measure :-) regards, Vlad From vladdu55@REDACTED Fri Jul 24 15:00:56 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 24 Jul 2009 15:00:56 +0200 Subject: Fwd: [erlang-questions] lists:append performance In-Reply-To: <95be1d3b0907240600i3d7d990bo574119a79cee2795@mail.gmail.com> References: <95be1d3b0907240532r119fa5d4me72f021bb41da2cc@mail.gmail.com> <200907241245.n6OCjelc016028@mail.pharos-avantgard.com> <95be1d3b0907240600i3d7d990bo574119a79cee2795@mail.gmail.com> Message-ID: <95be1d3b0907240600r225cd564tcbd3776a1c94c7fe@mail.gmail.com> forgot to send to list... ---------- Forwarded message ---------- From: Vlad Dumitrescu Date: Fri, Jul 24, 2009 at 15:00 Subject: Re: [erlang-questions] lists:append performance To: Valentin Micic On Fri, Jul 24, 2009 at 14:47, Valentin Micic wrote: > No, lists:reverse/1 is never executed. The goal was to see how slower > lists:append/2 is. The code looks like indicated below: > > insert( 0 ) -> ok; > insert( Count ) > -> > ? _L = ["1"|?L240], > ? isert( Count - 1 ) > . > > append( 0 ) -> ok; > append( Count ) > -> > ? ?_ = lists:append( ?L240, ["1"] ), > ? ?append( Count - 1 ) Aha, okay, now I see. Compiling with the 'S' option gives this {function, insert, 1, 2}. ?{label,1}. ? ?{func_info,{atom,b},{atom,insert},1}. ?{label,2}. ? ?{test,is_eq_exact,{f,3},[{x,0},{integer,0}]}. ? ?{move,{atom,ok},{x,0}}. ? ?return. ?{label,3}. ? ?{gc_bif,'-',{f,0},1,[{x,0},{integer,1}],{x,0}}. ? ?{'%live',1}. ? ?{call_only,1,{f,2}}. {function, append, 1, 5}. ?{label,4}. ? ?{func_info,{atom,b},{atom,append},1}. ?{label,5}. ? ?{test,is_eq_exact,{f,6},[{x,0},{integer,0}]}. ? ?{move,{atom,ok},{x,0}}. ? ?return. ?{label,6}. ? ?{gc_bif,'-',{f,0},1,[{x,0},{integer,1}],{x,0}}. ? ?{'%live',1}. ? ?{call_only,1,{f,5}}. which means that the compiler has optimized away both list operations and the functions are in fact identical! regards, Vlad From v@REDACTED Fri Jul 24 15:24:46 2009 From: v@REDACTED (Valentin Micic) Date: Fri, 24 Jul 2009 15:24:46 +0200 Subject: [erlang-questions] lists:append performance In-Reply-To: <95be1d3b0907240600r225cd564tcbd3776a1c94c7fe@mail.gmail.com> Message-ID: <200907241323.n6ODNMlc016297@mail.pharos-avantgard.com> Thanks Vlad, I've adjusted my benchmark code to avoid optimization, by creating operating list dynamically: -export( [ insert/2 % insert( Count, ListSz ) ,append/2 % append( Count, ListSz ) ] ). insert( Count, ListSz ) -> do_insert( Count, do_make_list( [], ListSz ) ). append( Count, ListSz ) -> do_append( Count, do_make_list( [], ListSz ) ). do_insert( 0, _ ) -> ok; do_insert( Count, List ) -> _L = ["1"|List], do_insert( Count-1, List ) . do_append( 0, _ ) -> ok; do_append( Count, List ) -> _ = lists:append( List, ["1"] ), do_append( Count-1, List ) . do_make_list( L, 0 ) -> L; do_make_list( L, ListSz ) -> do_make_list( ["The quick brown fox jumps over a lzay dog"|L], ListSz-1 ). Now, I can see a drastic difference in performance: 90> timer:tc( benchmarks, insert, [1000000, 240] ). {15000,ok} 91> timer:tc( benchmarks, append, [1000000, 240] ). {2782000,ok} Thanks for your help. V. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Vlad Dumitrescu Sent: 24 July 2009 03:01 PM To: erlang-questions Subject: Fwd: [erlang-questions] lists:append performance forgot to send to list... ---------- Forwarded message ---------- From: Vlad Dumitrescu Date: Fri, Jul 24, 2009 at 15:00 Subject: Re: [erlang-questions] lists:append performance To: Valentin Micic On Fri, Jul 24, 2009 at 14:47, Valentin Micic wrote: > No, lists:reverse/1 is never executed. The goal was to see how slower > lists:append/2 is. The code looks like indicated below: > > insert( 0 ) -> ok; > insert( Count ) > -> > ? _L = ["1"|?L240], > ? isert( Count - 1 ) > . > > append( 0 ) -> ok; > append( Count ) > -> > ? ?_ = lists:append( ?L240, ["1"] ), > ? ?append( Count - 1 ) Aha, okay, now I see. Compiling with the 'S' option gives this {function, insert, 1, 2}. ?{label,1}. ? ?{func_info,{atom,b},{atom,insert},1}. ?{label,2}. ? ?{test,is_eq_exact,{f,3},[{x,0},{integer,0}]}. ? ?{move,{atom,ok},{x,0}}. ? ?return. ?{label,3}. ? ?{gc_bif,'-',{f,0},1,[{x,0},{integer,1}],{x,0}}. ? ?{'%live',1}. ? ?{call_only,1,{f,2}}. {function, append, 1, 5}. ?{label,4}. ? ?{func_info,{atom,b},{atom,append},1}. ?{label,5}. ? ?{test,is_eq_exact,{f,6},[{x,0},{integer,0}]}. ? ?{move,{atom,ok},{x,0}}. ? ?return. ?{label,6}. ? ?{gc_bif,'-',{f,0},1,[{x,0},{integer,1}],{x,0}}. ? ?{'%live',1}. ? ?{call_only,1,{f,5}}. which means that the compiler has optimized away both list operations and the functions are in fact identical! regards, Vlad ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From buricchio@REDACTED Fri Jul 24 15:53:38 2009 From: buricchio@REDACTED (Andrew) Date: Fri, 24 Jul 2009 16:53:38 +0300 Subject: 64bit, will wx driver use shared wxwidgets libs? Message-ID: <4A69BCE2.6020103@gmail.com> Will wx driver use shared libraries instead static one in future? Using static libs for shared library leads to some compilation problem on x86_64 *nix platforms, because wxWidgets have no -fPIC flag linking static libs. Andrew From camilo.cerchiari@REDACTED Fri Jul 24 18:22:51 2009 From: camilo.cerchiari@REDACTED (Camilo Cerchiari) Date: Fri, 24 Jul 2009 13:22:51 -0300 Subject: jinterface - type conversion Message-ID: <588b5a840907240922s11f011d2r996c13588e20db2f@mail.gmail.com> hello all, i'm starting to work with jinterface. of course, one of the first things i have to do is type conversion. as i see, every OtpErlangObject subclass has its own conversion method, returning different things. for example OtpErlangList has the method elements returning OtpErlangObject[], OtpErlangLong has longValue() returning a long, etc. however there is no abstract function in the abstract class OtpErlangObject, something like getValue(). my question is: is there any reason for not having such an OtpErlangObject.getValue() method? of course, every subclass should implement this method, and this IMHO is the best place to do it, OO speaking. that method would be very usefull, as it would do the conversions from erlang types to java classes, that otherwise should be done by the programmer, in a not so elegant way. even complex erlang variables (think of lists of tuples of lists...) would be transformed trivially into java similars. comments on this will be appreciated, salutes, Camilo From diginux@REDACTED Fri Jul 24 18:27:57 2009 From: diginux@REDACTED (Jordan Wilberding) Date: Fri, 24 Jul 2009 11:27:57 -0500 Subject: Chicago Erlang Meetup Message-ID: <5756037b0907240927lf09c52sead165a36e37c187@mail.gmail.com> Greetings, This is late notice, but tonight the Chicago Erlang Group[1] will be meeting tonight at Orbitz (500 W Madison)[2] at 6PM. The guest speaker today is Lennart Ohman[3] who will be presenting the history of OTP. As one of the people who worked on Erlang from it's early stages and responsible for having OTP be what it is today, he is sure to give an excellent talk. If you want to come, please RSVP Martin Logan ( martinjlogan@REDACTED) by 4PM today, so that you can get you on the list. Thanks! Jordan Wilberding [1] http://groups.google.com/group/ceug [2] http://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=&q=orbitz+500+w+madison,+chicago,+il&sll=37.0625,-95.677068&sspn=33.847644,44.296875&ie=UTF8&ll=41.883477,-87.640128&spn=0.007764,0.010815&z=16&iwloc=A [3] http://www.erlang-factory.com/conference/London2009/speakers/Lennartohman From coa@REDACTED Fri Jul 24 19:46:56 2009 From: coa@REDACTED (=?ISO-8859-1?Q?Cl=E1udio_Amaral?=) Date: Fri, 24 Jul 2009 18:46:56 +0100 Subject: erl_tidy options Message-ID: <4A69F390.9010404@dcc.fc.up.pt> Hi all! I am exploring the syntax tools provided by Erlang and I can't see the real purpose of erl_tidy and I can't understand the purpose of the options. For example, what is the gain of the transformation at the end of the compiling process? Ex - {auto_export_vars, true} option {X, Y} = case ... of ... -> {17, foo()}; ... -> {42, bar()} end to case ... of ... -> X = 17, Y = foo(), {X, Y}; ... -> X = 42, Y = bar(), {X, Y} end Or the option {auto_list_comp, true}, rewriting calls to lists:map/2 and lists:filter/2 to list comprehensions. Thanks, Cl?udio Amaral. From harveyd@REDACTED Fri Jul 24 21:27:00 2009 From: harveyd@REDACTED (Dale Harvey) Date: Fri, 24 Jul 2009 20:27:00 +0100 Subject: [erlang-questions] erl_tidy options In-Reply-To: <4A69F390.9010404@dcc.fc.up.pt> References: <4A69F390.9010404@dcc.fc.up.pt> Message-ID: 2009/7/24 Cl?udio Amaral > Hi all! > > I am exploring the syntax tools provided by Erlang and I can't see the real > purpose of erl_tidy and I can't understand the purpose of the options. > > For example, what is the gain of the transformation at the end of the > compiling process? > Ex - {auto_export_vars, true} option > {X, Y} = case ... of > ... -> {17, foo()}; > ... -> {42, bar()} > end > > to > case ... of > ... -> X = 17, Y = foo(), {X, Y}; > ... -> X = 42, Y = bar(), {X, Y} > end > That I dont like, in fact I almost wish erlang didnt allow exporting variables from case, its almost always less readable imo > > Or the option {auto_list_comp, true}, rewriting calls to lists:map/2 and > lists:filter/2 to list comprehensions. > This I do agree with though, list comprehensions are almost always shorter and more readable than map and filter lists:filter(fun(X) when X rem 2 =:= 0 -> true; (_) -> false end, lists:seq(0,10)). vs [ X || X <- lists:seq(0, 10), X rem 2 =:= 0 ]. I dont think I have seen any case for map / filter that are more readables than list comprehensions(once you get over the initial scare of the list comprehension syntax). > > Thanks, > Cl?udio Amaral. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From igorrs@REDACTED Sat Jul 25 00:31:31 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Fri, 24 Jul 2009 19:31:31 -0300 Subject: Log dumping not thread-safe? Message-ID: Hi. mnesia:dump_log() is not thread-safe. Am I right? In practice, that would mean you can never use this function, as Mnesia could be dumping the log at the same moment you called this function. Is that correct? Thanks. Igor. From dave.pawson@REDACTED Sat Jul 25 04:50:13 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Sat, 25 Jul 2009 03:50:13 +0100 Subject: guards Message-ID: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> filterHelper(Lst,Int,Result) when ((length(Lst) == 1) and (lists:nth(1,Lst) =< Int)) -> [lists:nth(1,Lst) | Result]; and I'm told ./math4.erl:132: illegal guard expression I believe I can't use 'user defined' references in a guard statement, but I'm unsure if lists: module is such a statement? Suggestions please TIA -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From james.hague@REDACTED Sat Jul 25 05:13:24 2009 From: james.hague@REDACTED (James Hague) Date: Fri, 24 Jul 2009 22:13:24 -0500 Subject: [erlang-questions] guards In-Reply-To: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> References: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> Message-ID: > I believe I can't use 'user defined' references in a guard statement, > but I'm unsure if ?lists: module is such a statement? lists:nth is not allowed in guards. It's a pretty short list, the functions that are allowed. From dave.pawson@REDACTED Sat Jul 25 05:25:04 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Sat, 25 Jul 2009 04:25:04 +0100 Subject: erlang.org 404 Message-ID: <711a73df0907242025s77d58859uf30b477ba06c4bda@mail.gmail.com> http://erlang.org/doc/man_index.html top of page. 'up' link to http://erlang.org/doc/apps/doc/index.html is 404? Perhaps should be http://www.erlang.se/doc/index.shtml regards -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From mjtruog@REDACTED Sat Jul 25 05:28:21 2009 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 24 Jul 2009 20:28:21 -0700 Subject: [erlang-questions] guards In-Reply-To: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> References: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> Message-ID: <4A6A7BD5.3010302@gmail.com> - Can't use parentheses in guards - All the guards are in the erlang module and are explicitly documented as allowed in guard expressions (usually is_something/1, erlang:length/1 works but supposedly is O(n), not O(1) like the others) - use "," for and, otherwise ";" for or... they evaluate in a sequential way - Guards help to catch errors early and give hints for optimizations, if they can occur - Specifying the type in the function argument is always faster than adding a guard function for the type (as seen in the "The is_record() guard" thread) Dave Pawson wrote: > filterHelper(Lst,Int,Result) when ((length(Lst) == 1) and > (lists:nth(1,Lst) =< Int)) -> > [lists:nth(1,Lst) | Result]; > > and I'm told > > ./math4.erl:132: illegal guard expression > > > I believe I can't use 'user defined' references in a guard statement, > but I'm unsure if lists: module is such a statement? > > Suggestions please > > TIA > > From dave.pawson@REDACTED Sat Jul 25 05:27:30 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Sat, 25 Jul 2009 04:27:30 +0100 Subject: [erlang-questions] guards In-Reply-To: References: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> Message-ID: <711a73df0907242027o535a423bxeae3a4ab369406a6@mail.gmail.com> Tks James 2009/7/25 James Hague : >> I believe I can't use 'user defined' references in a guard statement, >> but I'm unsure if ?lists: module is such a statement? > > lists:nth is not allowed in guards. ?It's a pretty short list, the > functions that are allowed. http://www.erlang.org/doc/reference_manual/expressions.html#6.24 In case anyone else is curious what the list is! regards -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From dave.pawson@REDACTED Sat Jul 25 05:31:33 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Sat, 25 Jul 2009 04:31:33 +0100 Subject: [erlang-questions] guards In-Reply-To: <4A6A7BD5.3010302@gmail.com> References: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> <4A6A7BD5.3010302@gmail.com> Message-ID: <711a73df0907242031r7bf072f6kc0c76e51cb86545c@mail.gmail.com> 2009/7/25 Michael Truog : > - Can't use parentheses in guards > - All the guards are in the erlang module and are explicitly documented > as allowed in guard expressions > ?(usually is_something/1, erlang:length/1 works but supposedly is O(n), > not O(1) like the others) > - use "," for and, otherwise ";" for or... they evaluate in a sequential way Thanks. Not clear in the book I'm using (uses guard(X,Y) when .... and ... or => etc. > - Specifying the type in the function argument is always faster than > adding a guard function for the type > ? (as seen in the "The is_record() guard" thread) I'll go hunting! regards -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From dave.pawson@REDACTED Sat Jul 25 10:46:55 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Sat, 25 Jul 2009 09:46:55 +0100 Subject: [erlang-questions] guards In-Reply-To: <4d08db370907250103m28d52caapde3ac0fcb207a189@mail.gmail.com> References: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> <4d08db370907250103m28d52caapde3ac0fcb207a189@mail.gmail.com> Message-ID: <711a73df0907250146x393c46e1n614f92a78c025e89@mail.gmail.com> 2009/7/25 Hynek Vychodil : > > > On Sat, Jul 25, 2009 at 4:50 AM, Dave Pawson wrote: >> >> filterHelper(Lst,Int,Result) when ((length(Lst) == 1) and >> (lists:nth(1,Lst) =< Int)) -> >> ? ?[lists:nth(1,Lst) | Result]; > > It is little bit overcomplicated version of > > filterHelper([X], Int, Result) when X =< Int -> [X|Result]; Mmm. Thanks... I think :-) (I'm trying to learn!) Does that check for a list length of 1? As well as the value being <= (who the heck chose =< that is a real kludge!!!) Int? > > and also lists:nth(1,L) is equivalent of hd(L) which can be used in guards > but in most of time use list decomposition in pattern match is more > effective way to do it. Ah! Didn't know about hd(L). Is there a quick reference chart for Erlang Bofs, perhaps the common module functions? Seems its something to fight going into any language! Thanks Kynek. regards -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From bqt@REDACTED Sat Jul 25 11:03:41 2009 From: bqt@REDACTED (Johnny Billquist) Date: Sat, 25 Jul 2009 11:03:41 +0200 Subject: [erlang-questions] guards In-Reply-To: <711a73df0907250146x393c46e1n614f92a78c025e89@mail.gmail.com> References: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> <4d08db370907250103m28d52caapde3ac0fcb207a189@mail.gmail.com> <711a73df0907250146x393c46e1n614f92a78c025e89@mail.gmail.com> Message-ID: <4A6ACA6D.4020406@softjar.se> Dave Pawson wrote: > 2009/7/25 Hynek Vychodil : >> >> On Sat, Jul 25, 2009 at 4:50 AM, Dave Pawson wrote: >>> filterHelper(Lst,Int,Result) when ((length(Lst) == 1) and >>> (lists:nth(1,Lst) =< Int)) -> >>> [lists:nth(1,Lst) | Result]; >> It is little bit overcomplicated version of >> >> filterHelper([X], Int, Result) when X =< Int -> [X|Result]; > > Mmm. Thanks... I think :-) > (I'm trying to learn!) > > Does that check for a list length of 1? A pattern of [X] can only match a list with one element. If the list have two elements, you'd have to write [X,Y] to match, and so on... If you want to match a list of unknown length, where you want to look at the first element, you should write [X|Y]. Y will then be the list with all the remaining elements after the first, which is bound to X. Check the first two elements by [X,Y|Z], and so on... And of course, to match a list with no elements, you write []. :-) Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: bqt@REDACTED || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol From dave.pawson@REDACTED Sat Jul 25 11:11:54 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Sat, 25 Jul 2009 10:11:54 +0100 Subject: [erlang-questions] guards In-Reply-To: <4A6ACA6D.4020406@softjar.se> References: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> <4d08db370907250103m28d52caapde3ac0fcb207a189@mail.gmail.com> <711a73df0907250146x393c46e1n614f92a78c025e89@mail.gmail.com> <4A6ACA6D.4020406@softjar.se> Message-ID: <711a73df0907250211ke55ab68lfa42fd14db8b230f@mail.gmail.com> 2009/7/25 Johnny Billquist : >>> It is little bit overcomplicated version of >>> >>> filterHelper([X], Int, Result) when X =< Int -> [X|Result]; >> >> Mmm. Thanks... I think :-) >> (I'm trying to learn!) >> >> Does that check for a list length of 1? > > A pattern of [X] can only match a list with one element. Couldn't X be a variable which might be a tuple or ... almost anything? Or is this pattern matching logic, that implies it is as you say? > If the list have two elements, you'd have to write [X,Y] to match, and so > on... Yes. Two element list would require the comma > If you want to match a list of unknown length, where you want to look at the > first element, you should write [X|Y]. I'm getting used to that pattern, very useful! Thanks. -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From whongo@REDACTED Sat Jul 25 11:23:20 2009 From: whongo@REDACTED (=?ISO-8859-1?Q?Martin_Engstr=F6m?=) Date: Sat, 25 Jul 2009 11:23:20 +0200 Subject: [erlang-questions] guards In-Reply-To: <711a73df0907250211ke55ab68lfa42fd14db8b230f@mail.gmail.com> References: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> <4d08db370907250103m28d52caapde3ac0fcb207a189@mail.gmail.com> <711a73df0907250146x393c46e1n614f92a78c025e89@mail.gmail.com> <4A6ACA6D.4020406@softjar.se> <711a73df0907250211ke55ab68lfa42fd14db8b230f@mail.gmail.com> Message-ID: <28423d490907250223s1fc2c644taadbd35fc45c2a13@mail.gmail.com> On Sat, Jul 25, 2009 at 11:11 AM, Dave Pawson wrote: > 2009/7/25 Johnny Billquist : > > >>> It is little bit overcomplicated version of > >>> > >>> filterHelper([X], Int, Result) when X =< Int -> [X|Result]; > >> > >> Mmm. Thanks... I think :-) > >> (I'm trying to learn!) > >> > >> Does that check for a list length of 1? > > > > A pattern of [X] can only match a list with one element. > > Couldn't X be a variable which might be a tuple or ... almost anything? > Or is this pattern matching logic, that implies it is as you say? > Yes, X would match anything. But [X] matches only a list with exactly one element. ;) /Martin From bqt@REDACTED Sat Jul 25 11:51:48 2009 From: bqt@REDACTED (Johnny Billquist) Date: Sat, 25 Jul 2009 11:51:48 +0200 Subject: [erlang-questions] guards In-Reply-To: <711a73df0907250211ke55ab68lfa42fd14db8b230f@mail.gmail.com> References: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> <4d08db370907250103m28d52caapde3ac0fcb207a189@mail.gmail.com> <711a73df0907250146x393c46e1n614f92a78c025e89@mail.gmail.com> <4A6ACA6D.4020406@softjar.se> <711a73df0907250211ke55ab68lfa42fd14db8b230f@mail.gmail.com> Message-ID: <4A6AD5B4.4090608@softjar.se> Dave Pawson wrote: > 2009/7/25 Johnny Billquist : > >>>> It is little bit overcomplicated version of >>>> >>>> filterHelper([X], Int, Result) when X =< Int -> [X|Result]; >>> Mmm. Thanks... I think :-) >>> (I'm trying to learn!) >>> >>> Does that check for a list length of 1? >> A pattern of [X] can only match a list with one element. > > Couldn't X be a variable which might be a tuple or ... almost anything? > Or is this pattern matching logic, that implies it is as you say? I'm making the assumption that X is unbound at the time, otherwise we have a different ballgame where much less will match (in fact, only a list, of which the only element is identical to whatever X is bound to). The *only* element in the list could, however, be a list, a tuple, or just about anything. There is no limits to what it could be. The point is, that that is the only element in the list. If you want to check further into what type of element you have in the list, then you could either expand the pattern more, or add guards. Guards are useful when you want to check some general property of the element, but for explicit details, pattern matching makes more sense. Assume that the list should hold just one element, and that one element should be a tuple with two elements. That would then match against the pattern [{X,Y}] If the first element of the tuple should be 'foo', then the pattern could be [{foo,X}] No need to use guards to check for that. However, if the list should hold one element, and that should be a tuple, but you don't know what kind of tuple, then you'd need a guard, as in: foo([X]) when is_tuple(X) -> because the pattern can't specify a tuple with a variable number of elements. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: bqt@REDACTED || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol From dave.pawson@REDACTED Sat Jul 25 12:00:12 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Sat, 25 Jul 2009 11:00:12 +0100 Subject: [erlang-questions] guards In-Reply-To: <4A6AD5B4.4090608@softjar.se> References: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> <4d08db370907250103m28d52caapde3ac0fcb207a189@mail.gmail.com> <711a73df0907250146x393c46e1n614f92a78c025e89@mail.gmail.com> <4A6ACA6D.4020406@softjar.se> <711a73df0907250211ke55ab68lfa42fd14db8b230f@mail.gmail.com> <4A6AD5B4.4090608@softjar.se> Message-ID: <711a73df0907250300g2cf65bdi796537a7816e8055@mail.gmail.com> 2009/7/25 Johnny Billquist : >>> A pattern of [X] can only match a list with one element. >> >> Couldn't X be a variable which might be a tuple or ... almost anything? >> Or is this pattern matching logic, that implies it is as you say? > > I'm making the assumption that X is unbound at the time, otherwise we have a > different ballgame where much less will match (in fact, only a list, of > which the only element is identical to whatever X is bound to). Yes. Thanks. I hadn't gone that far. No matter what ?type? it is, there is only one of them in the list! > > The *only* element in the list could, however, be a list, a tuple, or just > about anything. There is no limits to what it could be. The point is, that > that is the only element in the list. Two stage logic. It's a list length one, then deal with the list item. I'll get there eventually! > No need to use guards to check for that. However, if the list should hold > one element, and that should be a tuple, but you don't know what kind of > tuple, then you'd need a guard, as in: > > foo([X]) when is_tuple(X) -> > > because the pattern can't specify a tuple with a variable number of > elements. Thanks. -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From bgustavsson@REDACTED Sat Jul 25 12:45:29 2009 From: bgustavsson@REDACTED (Bjorn Gustavsson) Date: Sat, 25 Jul 2009 12:45:29 +0200 Subject: [erlang-questions] The is_record() guard In-Reply-To: <4A689F53.4090204@erlang-consulting.com> References: <4A689F53.4090204@erlang-consulting.com> Message-ID: <6672d0160907250345i7733bcf9h54e36214b95afb49@mail.gmail.com> 2009/7/23 Oscar Hellstr?m : > Hi Steve, > > You could also do it this way: > > transform(#rec_a{} = R) -> ... ; > transform(#rec_b{} = R) -> ... ; > ... Yes, that is the recommended way to do it. Not only will the matching in the function head be more efficient, but the compiler may also generate better code for further record operations in the body of the function. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From joelr1@REDACTED Sat Jul 25 14:49:52 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 25 Jul 2009 13:49:52 +0100 Subject: delivery of 'DOWN' to multiple nodes Message-ID: <69535B77-6E6F-4670-8CA4-3534FBF245D8@gmail.com> Suppose I have thousand of processes spread over several nodes, all monitoring a process on another node. How is 'DOWN' delivered when the monitored process exits? Does it get sent to every process or once to every node and then to the local processes? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From baryluk@REDACTED Sat Jul 25 18:01:38 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Sat, 25 Jul 2009 18:01:38 +0200 Subject: [erlang-questions] guards In-Reply-To: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> References: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> Message-ID: <1248537698.2853.0.camel@sredniczarny> Dnia 2009-07-25, sob o godzinie 03:50 +0100, Dave Pawson pisze: > filterHelper(Lst,Int,Result) when ((length(Lst) == 1) and > (lists:nth(1,Lst) =< Int)) -> > [lists:nth(1,Lst) | Result]; filterHelper([Elem],Int,Result) when Elem =< Int -> [Elem | Result]; simpler and a lot faster. -- Witold Baryluk From baryluk@REDACTED Sat Jul 25 19:31:22 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Sat, 25 Jul 2009 19:31:22 +0200 Subject: [erlang-questions] guards In-Reply-To: <711a73df0907251004j4bc54f32m55daafc3288bcb8f@mail.gmail.com> References: <711a73df0907241950j7ff9874au62f126542c10272e@mail.gmail.com> <1248537698.2853.0.camel@sredniczarny> <711a73df0907251004j4bc54f32m55daafc3288bcb8f@mail.gmail.com> Message-ID: <1248543082.11900.22.camel@sredniczarny> Dnia 2009-07-25, sob o godzinie 18:04 +0100, Dave Pawson pisze: > Thanks Witold, > > 2009/7/25 Witold Baryluk : > > Dnia 2009-07-25, sob o godzinie 03:50 +0100, Dave Pawson pisze: > >> filterHelper(Lst,Int,Result) when ((length(Lst) == 1) and > >> (lists:nth(1,Lst) =< Int)) -> > >> [lists:nth(1,Lst) | Result]; > > > > filterHelper([Elem],Int,Result) when Elem =< Int -> > > [Elem | Result]; > > > > simpler and a lot faster. > > > I'm looking for understanding today, > not speed! > > Speed will come, but later! > I did not understand the [A] idea, that it is a list of one element! Pattern matching is powerful and i hope you would like it :) so this "simpler" part. for example if you would like to only match one element lists with, value like second argument: filetrHelper([Int], Int, result) -> ... (eventually adding is_integer/1 test) "a lot faster" for two reasons. 1. compiler knows easier what is here, and extracts first (and only one) element of the list, and reuse it in the body of function clause. additionally if you have more clauses, compiler can more easily find common structure between them and do checks in best possible way. For example if two of them check if it have length == 1, check will be done only once, i hope, this is just a theory. In case of guards compiler probably isn't so smart. 2. I assumed this filterHelper is used in some kind of loop, so it is for example called N times. What if most times the first argument of filterHelper is actually big list (with K elements on avarage?). You will end traversing all this lists (there is no precomputed length of list, but thanks to this lists are smaller), only for testing it if it is of some small (1) size. This makes this code really slow, like N*K operations not just N. If K is large (10 or 100), this is can be problematic. This is the reasons why in my mind I see red alert blinking, always when I find code with length/1 calls in guards. I would be happy if it would be removed from allowed functions list in guards :) Eventually it would be nice to have guard of the form length(List, Min, Max), which would answer, if list List is of size between Min and Max (or infinity) inclusive. So being equivalence of length(List) >= Min ,length(List) =< Max, but faster. Only problem I see is with the "inproper" lists, like [1|2], length throws exceptions for them. So it is somehow unfortunete that there is no flag if lists is proper or not. Probably for the same reason that there is no length field. (or mayby this is because VM is doing some dirty tricks, and lists isn't really immutable). -- Witold Baryluk From paul-trapexit@REDACTED Sat Jul 25 20:10:12 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Sat, 25 Jul 2009 11:10:12 -0700 (PDT) Subject: ets select match specification analysis Message-ID: hi. i'm working on improving the query planning in tcerl and i realized i might be duplicating effort, so i was hoping to get some insight on how ets does it's thing. suppose i have an ordered_set ets table with key position 1. a match specification like [ { { '_', '_' }, [], [ '$_' ] } ] will do a full-table scan but [ { { { Value, '_' }, '_' }, [], [ '$_' ] } ] will not, instead it will only scan the range of keys which are tuples whose first element is Value.[1] currently tcerl will also recognize a partially bound prefix key and restrict the range of keys considered. however for an application i'm working on i need efficient queries corresponding to match specifications that look like [ { { { '$1', '_' }, '_' }, [ { '>', '$1', Lower }, { '<', '$1', Upper } ], [ '$_' ] } ] which ideally would only consider the range of keys defined by tuples whose first element is between Lower and Upper. i'm modified tcerl to detect this condition[2] but now i'm worried i'm slowly rewriting some ets function that is already available, has less defects, and/or is faster because it's a BIF. so: 1) is the match specification analysis in ets this sophisticated? 2) if yes, is it exposed reusably? thanks, -- p [1] http://erlang.org/documentation/doc-5.5.1/doc/efficiency_guide/tablesDatabases.html [2] http://code.google.com/p/tcerl/source/browse/trunk/tcerl/src/tcbdbmsutil.erl From joelr1@REDACTED Sat Jul 25 20:55:20 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 25 Jul 2009 19:55:20 +0100 Subject: {packet, 0} -> {packet, 2} on an existing socket Message-ID: <87BDE432-D558-40E4-813A-AFB6CB03713E@gmail.com> I'm having trouble switching from {packet, 0} to {packet, 2} on an existing socket that has been used to receive a packet and send a reply. The next packet comes in with the packet size prefix. Is this intentional? Thanks, Joel --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From joelr1@REDACTED Sat Jul 25 21:14:04 2009 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 25 Jul 2009 20:14:04 +0100 Subject: {packet, 0} -> {packet, 2} on an existing socket In-Reply-To: <87BDE432-D558-40E4-813A-AFB6CB03713E@gmail.com> References: <87BDE432-D558-40E4-813A-AFB6CB03713E@gmail.com> Message-ID: <17AFC93B-438E-4DF9-B1BD-BD7C139DC656@gmail.com> My bad, foot in mouth, I'm looking at two different sockets. On Jul 25, 2009, at 7:55 PM, Joel Reymont wrote: > I'm having trouble switching from {packet, 0} to {packet, 2} on an > existing socket that has been used to receive a packet and send a > reply. The next packet comes in with the packet size prefix. Is this > intentional? --- Mac hacker with a performance bent http://www.linkedin.com/in/joelreymont From ulf.wiger@REDACTED Sat Jul 25 22:03:42 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 25 Jul 2009 22:03:42 +0200 Subject: [erlang-questions] ets select match specification analysis In-Reply-To: References: Message-ID: <4A6B651E.3060704@erlang-consulting.com> Paul Mineiro wrote: > > currently tcerl will also recognize a partially bound prefix key and > restrict the range of keys considered. however for an application i'm > working on i need efficient queries corresponding to match specifications > that look like > > [ { { { '$1', '_' }, '_' }, > [ { '>', '$1', Lower }, { '<', '$1', Upper } ], > [ '$_' ] > } > ] > > which ideally would only consider the range of keys defined by tuples > whose first element is between Lower and Upper. Ideally, yes, but the way select() is implemented, it will still result in a full scan of the table, unless I've missed some key release note somewhere. I'd love to be proven wrong. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From tom.mcnulty@REDACTED Sat Jul 25 22:37:06 2009 From: tom.mcnulty@REDACTED (Tom McNulty) Date: Sat, 25 Jul 2009 14:37:06 -0600 Subject: [erlang-questions] ets select match specification analysis In-Reply-To: <4A6B651E.3060704@erlang-consulting.com> References: <4A6B651E.3060704@erlang-consulting.com> Message-ID: > > Paul Mineiro wrote: >> currently tcerl will also recognize a partially bound prefix key and >> restrict the range of keys considered. however for an application >> i'm >> working on i need efficient queries corresponding to match >> specifications >> that look like >> [ { { { '$1', '_' }, '_' }, >> [ { '>', '$1', Lower }, { '<', '$1', Upper } ], >> [ '$_' ] >> } >> ] >> which ideally would only consider the range of keys defined by tuples >> whose first element is between Lower and Upper. > > Ideally, yes, but the way select() is implemented, it will still > result in a full scan of the table, unless I've missed some > key release note somewhere. I'd love to be proven wrong. Should be able to avoid a full scan with ordered_set's Efficiency guide: "There are exceptions when the complete table is not scanned, for instance if part of the key is bound when searching an ordered_set table, or if it is a Mnesia table and there is a secondary index on the field that is selected/matched. If the key is fully bound there will, of course, be no point in doing a select/match, unless you have a bag table and you are only interested in a sub-set of the elements with the specific key." - Tom > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From steven.charles.davis@REDACTED Sat Jul 25 22:39:24 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sat, 25 Jul 2009 15:39:24 -0500 Subject: [erlang-questions] The is_record() guard In-Reply-To: <6672d0160907250345i7733bcf9h54e36214b95afb49@mail.gmail.com> References: <4A689F53.4090204@erlang-consulting.com> <6672d0160907250345i7733bcf9h54e36214b95afb49@mail.gmail.com> Message-ID: <4A6B6D7C.1000500@gmail.com> Hi Bjorn, Definitely the approach suggested by Oscar shows much better numbers in fprof... I also see other cases where functions that filter arguments using guards appear to be (quite a bit) less efficient than direct matches on the arguments. Would it be fair to say that, where possible. pattern matching function arguments is in general more efficient than using guards? Regards, Steve Bjorn Gustavsson wrote: > 2009/7/23 Oscar Hellstr?m : >> Hi Steve, >> >> You could also do it this way: >> >> transform(#rec_a{} = R) -> ... ; >> transform(#rec_b{} = R) -> ... ; >> ... > > Yes, that is the recommended way to do it. > > Not only will the matching in the function head be more efficient, but > the compiler > may also generate better code for further record operations in the > body of the function. > > /Bjorn > From vladdu55@REDACTED Sat Jul 25 23:25:14 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Sat, 25 Jul 2009 23:25:14 +0200 Subject: [erlang-questions] jinterface - type conversion In-Reply-To: <588b5a840907240922s11f011d2r996c13588e20db2f@mail.gmail.com> References: <588b5a840907240922s11f011d2r996c13588e20db2f@mail.gmail.com> Message-ID: <95be1d3b0907251425l4b28eefaw7be9d7b89eb172b9@mail.gmail.com> On Fri, Jul 24, 2009 at 18:22, Camilo Cerchiari wrote: > my question is: ?is there any reason for not having such an > OtpErlangObject.getValue() method? Hi, One reason is that some mappings between erlang and java types aren't one-to-one. For example, there are a lot of java types mapping to the erlang integer type. Also, the atom 'true' can be translated as a string or as a boolean. Another reason is IMHO that when jinterface was written, the author didn't have a lot of experience with OOP, so the result can be improved. Unfortunately, backwards compatibility is in the way. The solution would be to write another library (like it was done for the C mappings), but I guess that's not high priority. In the meantime, you can check the code in erlide, where we have implemented a lot of additions to jinterface. http://erlide.svn.sourceforge.net/viewvc/erlide/trunk/org.erlide.jinterface/ There are helpers for converting between erlang and java and back, using signature strings to let the programmer know what kind of data he wants/provides. We have "sprintf"-like methods for building erlang terms more easily in Java, and pattern matching methods to take those terms apart. There is even some incomplete support for generating bridging code to allow calling Java methods from Erlang, should the need arise. best regards, Vlad From kamiseq@REDACTED Sun Jul 26 13:40:18 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Sun, 26 Jul 2009 13:40:18 +0200 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <3E1B9BBB-A139-4D49-84EE-9DA2AAB9FE91@gmail.com> <1246534666.6215.23.camel@piko.site> <991018.15211.qm@web111414.mail.gq1.yahoo.com> Message-ID: recently I ve played with erlybird plugin for netbeans and it works fine but it is not project oriented plugin and it lacks templates to generate code like in erlang..anyway it is working :) pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From paul-trapexit@REDACTED Sun Jul 26 22:01:39 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Sun, 26 Jul 2009 13:01:39 -0700 (PDT) Subject: ets select match specification analysis In-Reply-To: References: Message-ID: Timing tests in R12B5 indicate the range-constrained prefix condition results in a full table scan for ets ordered_set. http://dukesoferl.blogspot.com/2009/07/ets-ordered-set-select-efficiency.html -- p On Sat, 25 Jul 2009, Paul Mineiro wrote: > hi. i'm working on improving the query planning in tcerl and i realized i > might be duplicating effort, so i was hoping to get some insight on how > ets does it's thing. > > suppose i have an ordered_set ets table with key position 1. a match > specification like > > [ { { '_', '_' }, [], [ '$_' ] } ] > > will do a full-table scan but > > [ { { { Value, '_' }, '_' }, [], [ '$_' ] } ] > > will not, instead it will only scan the range of keys which are tuples > whose first element is Value.[1] > > currently tcerl will also recognize a partially bound prefix key and > restrict the range of keys considered. however for an application i'm > working on i need efficient queries corresponding to match specifications > that look like > > [ { { { '$1', '_' }, '_' }, > [ { '>', '$1', Lower }, { '<', '$1', Upper } ], > [ '$_' ] > } > ] > > which ideally would only consider the range of keys defined by tuples > whose first element is between Lower and Upper. > > i'm modified tcerl to detect this condition[2] but now i'm worried i'm > slowly rewriting some ets function that is already available, has > less defects, and/or is faster because it's a BIF. > > so: > > 1) is the match specification analysis in ets this sophisticated? > 2) if yes, is it exposed reusably? > > thanks, > > -- p > > [1] http://erlang.org/documentation/doc-5.5.1/doc/efficiency_guide/tablesDatabases.html > [2] http://code.google.com/p/tcerl/source/browse/trunk/tcerl/src/tcbdbmsutil.erl > From kaiduanx@REDACTED Sun Jul 26 22:54:48 2009 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Sun, 26 Jul 2009 16:54:48 -0400 Subject: Distel stays on interpreting Message-ID: Hi, I followed the instruction on http://bc.tech.coop/blog/070528.html to give distel a try, but it looked like that Distel stayed on interpreting forever. If I toggle a breakpoint on the current line, it says "Module is not interpreted, can't set breakpoints". The following is from the message buffer, loading = (otp_doc "/home/kaiduanx/distel/ebin/otp_doc.beam") load: [badrpc [EXIT [undef ([distel rpc_entry (distel reload_module (modtest2 nil))] [rpc -handle_call/3-fun-0- 5])]]] loading = (fdoc "/home/kaiduanx/distel/ebin/fdoc.beam") loading = (distel_ie "/home/kaiduanx/distel/ebin/distel_ie.beam") loading = (distel "/home/kaiduanx/distel/ebin/distel.beam") loading = nil (Successfully uploaded backend modules into node) Interpreting: modtest2 if: Module is not interpreted, can't set breakpoints. Can someone point out what is wrong? Thanks, kaiduan From paul-trapexit@REDACTED Sun Jul 26 23:08:58 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Sun, 26 Jul 2009 14:08:58 -0700 (PDT) Subject: [erlang-questions] Re: ets select match specification analysis In-Reply-To: References: Message-ID: Well you can grab the etsselect module in the blog post and try in it the shell. I'll do this one for demonstration purposes. ---------- % erl Erlang (BEAM) emulator version 5.6.5 [source] [smp:2] [async-threads:0] [kernel-poll:false] Eshell V5.6.5 (abort with ^G) 1> etsselect:select_time ([ { { '$1', bar }, [ { '>=', '$1', { const, { foo, 1, 0 } } }, { '=<', '$1', { const, { foo, 1, 10 } } } ], [ '$_' ] } ]). [{5,5.44}, {25,9.56}, {125,29.01}, {625,144.73}, {3125,515.0666666666667}, {15625,2731.8333333333335}, {78125,15812.533333333333}, {390625,79042.6}, {1953125,409633.5}] -------- Verdict: full table scan. -- p On Sun, 26 Jul 2009, Tom McNulty wrote: > Hi Paul > > I'm curious to see how a purely range based query will stack up > against those results. That is, ranging over the entire key, as > opposed to a partially bound key. I would be quite surprised if this > also resulted in a full table scan. > > -Tom > > On 26-Jul-09, at 2:01 PM, Paul Mineiro wrote: > > > Timing tests in R12B5 indicate the range-constrained prefix condition > > results in a full table scan for ets ordered_set. > > > > http://dukesoferl.blogspot.com/2009/07/ets-ordered-set-select-efficiency.html > > > > -- p > > > > On Sat, 25 Jul 2009, Paul Mineiro wrote: > > > >> hi. i'm working on improving the query planning in tcerl and i > >> realized i > >> might be duplicating effort, so i was hoping to get some insight on > >> how > >> ets does it's thing. > >> > >> suppose i have an ordered_set ets table with key position 1. a match > >> specification like > >> > >> [ { { '_', '_' }, [], [ '$_' ] } ] > >> > >> will do a full-table scan but > >> > >> [ { { { Value, '_' }, '_' }, [], [ '$_' ] } ] > >> > >> will not, instead it will only scan the range of keys which are > >> tuples > >> whose first element is Value.[1] > >> > >> currently tcerl will also recognize a partially bound prefix key and > >> restrict the range of keys considered. however for an application > >> i'm > >> working on i need efficient queries corresponding to match > >> specifications > >> that look like > >> > >> [ { { { '$1', '_' }, '_' }, > >> [ { '>', '$1', Lower }, { '<', '$1', Upper } ], > >> [ '$_' ] > >> } > >> ] > >> > >> which ideally would only consider the range of keys defined by tuples > >> whose first element is between Lower and Upper. > >> > >> i'm modified tcerl to detect this condition[2] but now i'm worried > >> i'm > >> slowly rewriting some ets function that is already available, has > >> less defects, and/or is faster because it's a BIF. > >> > >> so: > >> > >> 1) is the match specification analysis in ets this sophisticated? > >> 2) if yes, is it exposed reusably? > >> > >> thanks, > >> > >> -- p > >> > >> [1] http://erlang.org/documentation/doc-5.5.1/doc/efficiency_guide/tablesDatabases.html > >> [2] http://code.google.com/p/tcerl/source/browse/trunk/tcerl/src/tcbdbmsutil.erl > >> > > > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > From harveyd@REDACTED Sun Jul 26 23:40:03 2009 From: harveyd@REDACTED (Dale Harvey) Date: Sun, 26 Jul 2009 22:40:03 +0100 Subject: [erlang-questions] Distel stays on interpreting In-Reply-To: References: Message-ID: I havent had the time to properly debug whats happening, however I have found that this only happens for me when distel is loaded first time specifically to do the interpretation. if you load up a file, press M-, to follow a function call to another module, then ensures distel is loaded, then when you interpret a module it (might) work. 2009/7/26 Kaiduan Xie > Hi, > > I followed the instruction on http://bc.tech.coop/blog/070528.html to > give distel a try, but it looked like that Distel stayed on > interpreting forever. If I toggle a breakpoint on the current line, it > says "Module is not interpreted, can't set breakpoints". > > The following is from the message buffer, > > loading = (otp_doc "/home/kaiduanx/distel/ebin/otp_doc.beam") > load: [badrpc [EXIT [undef ([distel rpc_entry (distel reload_module > (modtest2 nil))] [rpc -handle_call/3-fun-0- 5])]]] > loading = (fdoc "/home/kaiduanx/distel/ebin/fdoc.beam") > loading = (distel_ie "/home/kaiduanx/distel/ebin/distel_ie.beam") > loading = (distel "/home/kaiduanx/distel/ebin/distel.beam") > loading = nil > (Successfully uploaded backend modules into node) > Interpreting: modtest2 > if: Module is not interpreted, can't set breakpoints. > > Can someone point out what is wrong? > > Thanks, > > kaiduan > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From harveyd@REDACTED Sun Jul 26 23:42:39 2009 From: harveyd@REDACTED (Dale Harvey) Date: Sun, 26 Jul 2009 22:42:39 +0100 Subject: [erlang-questions] Distel stays on interpreting In-Reply-To: References: Message-ID: oh, also if you have flymake turned on, disable it then try, I believe the distel / flymake issue has been fixed, but again I havent tested. 2009/7/26 Dale Harvey > I havent had the time to properly debug whats happening, however I have > found that this only happens for me when distel is loaded first time > specifically to do the interpretation. > > if you load up a file, press M-, to follow a function call to another > module, then ensures distel is loaded, then when you interpret a module it > (might) work. > > 2009/7/26 Kaiduan Xie > > Hi, >> >> I followed the instruction on http://bc.tech.coop/blog/070528.html to >> give distel a try, but it looked like that Distel stayed on >> interpreting forever. If I toggle a breakpoint on the current line, it >> says "Module is not interpreted, can't set breakpoints". >> >> The following is from the message buffer, >> >> loading = (otp_doc "/home/kaiduanx/distel/ebin/otp_doc.beam") >> load: [badrpc [EXIT [undef ([distel rpc_entry (distel reload_module >> (modtest2 nil))] [rpc -handle_call/3-fun-0- 5])]]] >> loading = (fdoc "/home/kaiduanx/distel/ebin/fdoc.beam") >> loading = (distel_ie "/home/kaiduanx/distel/ebin/distel_ie.beam") >> loading = (distel "/home/kaiduanx/distel/ebin/distel.beam") >> loading = nil >> (Successfully uploaded backend modules into node) >> Interpreting: modtest2 >> if: Module is not interpreted, can't set breakpoints. >> >> Can someone point out what is wrong? >> >> Thanks, >> >> kaiduan >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > From kaiduanx@REDACTED Mon Jul 27 00:08:53 2009 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Sun, 26 Jul 2009 18:08:53 -0400 Subject: [erlang-questions] Distel stays on interpreting In-Reply-To: References: Message-ID: Dale, I believe distel is loaded because distel is shown under the Erlang menu, and I can list all processes. How to disable flymake? My .emacs is shown below, (custom-set-variables ;; custom-set-variables was added by Custom. ;; If you edit it by hand, you could mess it up, so be careful. ;; Your init file should contain only one such instance. ;; If there is more than one, they won't work right. '(inhibit-startup-screen t)) (custom-set-faces ;; custom-set-faces was added by Custom. ;; If you edit it by hand, you could mess it up, so be careful. ;; Your init file should contain only one such instance. ;; If there is more than one, they won't work right. ) ;; Erlang stuff (setq erlang-root-dir "/usr/local/lib/erlang") (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.6.2/emacs" load-path)) (setq exec-path (cons "/usr/local/lib/erlang/bin" exec-path)) (require 'erlang-start) ;; This is needed for Distel setup (let ((distel-dir "/home/kaiduanx/distel/elisp")) (unless (member distel-dir load-path) ;; Add distel-dir to the end of load-path (setq load-path (append load-path (list distel-dir))))) (require 'distel) (distel-setup) ;; Some Erlang customizations (add-hook 'erlang-mode-hook (lambda () ;; when starting an Erlang shell in Emacs, default in the node name (setq inferior-erlang-machine-options '("-sname" "emacs")) ;; add Erlang functions to an imenu menu (imenu-add-to-menubar "imenu"))) ;; A number of the erlang-extended-mode key bindings are useful in the shell too (defconst distel-shell-keys '(("\C-\M-i" erl-complete) ("\M-?" erl-complete) ("\M-." erl-find-source-under-point) ("\M-," erl-find-source-unwind) ("\M-*" erl-find-source-unwind) ) "Additional keys to bind when in Erlang shell.") (add-hook 'erlang-shell-mode-hook (lambda () ;; add some Distel bindings to the Erlang shell (dolist (spec distel-shell-keys) (define-key erlang-shell-mode-map (car spec) (cadr spec))))) Thanks, kaiduan On Sun, Jul 26, 2009 at 5:42 PM, Dale Harvey wrote: > oh, also if you have flymake turned on, disable it then try, I believe the > distel / flymake issue has been fixed, but again I havent tested. > > 2009/7/26 Dale Harvey >> >> I havent had the time to properly debug whats happening, however I have >> found that this only happens for me when distel is loaded first time >> specifically to do the interpretation. >> >> if you load up a file, press M-, to follow a function call to another >> module, then ensures distel is loaded, then when you interpret a module it >> (might) work. >> >> 2009/7/26 Kaiduan Xie >>> >>> Hi, >>> >>> I followed the instruction on http://bc.tech.coop/blog/070528.html to >>> give distel a try, ?but it looked like that Distel stayed on >>> interpreting forever. If I toggle a breakpoint on the current line, it >>> says "Module is not interpreted, can't set breakpoints". >>> >>> The following is from the message buffer, >>> >>> loading = (otp_doc "/home/kaiduanx/distel/ebin/otp_doc.beam") >>> load: [badrpc [EXIT [undef ([distel rpc_entry (distel reload_module >>> (modtest2 nil))] [rpc -handle_call/3-fun-0- 5])]]] >>> loading = (fdoc "/home/kaiduanx/distel/ebin/fdoc.beam") >>> loading = (distel_ie "/home/kaiduanx/distel/ebin/distel_ie.beam") >>> loading = (distel "/home/kaiduanx/distel/ebin/distel.beam") >>> loading = nil >>> (Successfully uploaded backend modules into node) >>> Interpreting: modtest2 >>> if: Module is not interpreted, can't set breakpoints. >>> >>> Can someone point out what is wrong? >>> >>> Thanks, >>> >>> kaiduan >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >> > > From cowboymathu@REDACTED Mon Jul 27 08:58:03 2009 From: cowboymathu@REDACTED (MAthuvathanan Mou.) Date: Mon, 27 Jul 2009 12:28:03 +0530 Subject: What \& means in regexp:sub/3 Message-ID: <90b4299d0907262358h6f67f26eg51a20da5e7ebd77e@mail.gmail.com> Dear All, In the manual for the regexp, *sub(String, RegExp, New) -> SubRes * *Types:* *String = RegExp = New = string() SubRes = {ok,NewString,RepCount} | {error,errordesc()} RepCount = integer() * *Substitutes the first occurrence of a substring matching RegExp in Stringwith the string New. A & in the string New is replaced by the matched substring of String. \& puts a literal & into the replacement string. * Can someone please explain the sentence "*\& puts a literal & into the replacement string. *" . If any example that will be helpful. Thanks, -- Mathuvathanan Mou. From cowboymathu@REDACTED Mon Jul 27 09:05:53 2009 From: cowboymathu@REDACTED (MAthuvathanan Mou.) Date: Mon, 27 Jul 2009 12:35:53 +0530 Subject: [erlang-questions] What is the best software development platform to use when working with Erlang? In-Reply-To: References: <2a67d3ff0906171534s668c825fib01113f3981afb14@mail.gmail.com> <1246534666.6215.23.camel@piko.site> <991018.15211.qm@web111414.mail.gq1.yahoo.com> Message-ID: <90b4299d0907270005u2405db3asfe93ea50caa545cf@mail.gmail.com> I am using UltraEdit with erlang syntax highlighting for years. have to give a try with emacs. Thanks, Mathu Mou. 2009/7/26 pawe? kami?ski > recently I ve played with erlybird plugin for netbeans and it works fine > but > it is not project oriented plugin and it lacks templates to generate code > like in erlang..anyway it is working :) > > pozdrawiam > Pawe? Kami?ski > > kamiseq@REDACTED > pkaminski.prv@REDACTED > ______________________ > -- Mathuvathanan Mou. From ttmrichter@REDACTED Mon Jul 27 09:22:19 2009 From: ttmrichter@REDACTED (ttmrichter@REDACTED) Date: Mon, 27 Jul 2009 07:22:19 +0000 Subject: [erlang-questions] What is the best software development platform to use when working with E In-Reply-To: <90b4299d0907270005u2405db3asfe93ea50caa545cf@mail.gmail.com> Message-ID: <0016364578587934cd046faad0d1@google.com> I use jEdit with some templating plug-ins that let me get a lot of boilerplate code out of the way quickly. When I get less lazy I'll add some scripts to generate a project from scratch. From kiszl@REDACTED Mon Jul 27 09:23:06 2009 From: kiszl@REDACTED (kiszl@REDACTED) Date: Mon, 27 Jul 2009 09:23:06 +0200 (CEST) Subject: [erlang-questions] What \& means in regexp:sub/3 In-Reply-To: <90b4299d0907262358h6f67f26eg51a20da5e7ebd77e@mail.gmail.com> References: <90b4299d0907262358h6f67f26eg51a20da5e7ebd77e@mail.gmail.com> Message-ID: <39481.194.88.55.211.1248679386.squirrel@localhost> Hi, The & character in the New string means: put here what RegExp matched from the string. > regexp:sub("Hard Day's Night", "^[A-Z][a-z]*", "The &est"). {ok, "The Hardest Day's Night", 1} Notice how & gets replaced by "Hard" in the result. Sometimes you might want to actually put the character & in the result. In this case you need to escape & in New string so it doesn't get replaced: > regexp:sub("Bonnie and Clyde", "and", "\\&"). {ok, "Bonnie & Clyde", 1} Regards, Zoltan. > Dear All, > > In the manual for the regexp, > > *sub(String, RegExp, New) -> SubRes > * > > *Types:* > > *String = RegExp = New = string() > SubRes = {ok,NewString,RepCount} | {error,errordesc()} > RepCount = integer() > * > *Substitutes the first occurrence of a substring matching RegExp in > Stringwith the string > New. A & in the string New is replaced by the matched substring of String. > \& puts a literal & into the replacement string. * > > Can someone please explain the sentence "*\& puts a literal & into the > replacement string. *" . If any example that will be helpful. > > Thanks, > -- > Mathuvathanan Mou. > From vladdu55@REDACTED Mon Jul 27 10:15:50 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 27 Jul 2009 10:15:50 +0200 Subject: file module doesn't support utf-8 file names Message-ID: <95be1d3b0907270115k5e0b6ba9uae476634bf27c04f@mail.gmail.com> Hi, with the recent addition of utf-8 support, there are high expectations about where utf-8 values are allowed. One of those is in file paths, where the file name is types as a string(), but the file:file_name/1 utility function only allows characters between 0 and 255. I think that just allowing integers up to 65535 should fix this. Similar issues seem to be present in epp.erl, erl_comment_scan.erl, erl_tidy.erl and igor.erl. best regards, Vlad From xuxiang1@REDACTED Mon Jul 27 12:24:57 2009 From: xuxiang1@REDACTED (George Xu) Date: Mon, 27 Jul 2009 18:24:57 +0800 Subject: Lost messages Message-ID: It is said that send message is safe. But I count the problem of lost message. The situation is like this: 1. I use a ets table to store all process to be communicated. 2. But those process will terminated with some cases. 3. In case of one process is terminated but still not reflected to the ets table, send message to this process happened sametimes. It seems the message is send, but it can not reach since the process is termiated and the message is lost. Anyone knows how to solve this problem? ?????????????????????????? ?? George Xu Mobile Internet Service Lab, Lonovo Corp. Research China Tel:86-10-58866181 E-mail:xuxiang1@REDACTED No.6 Shang Di West Road, Haidian District, Beijing, 100085 P.R.China From kamiseq@REDACTED Mon Jul 27 12:45:06 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Mon, 27 Jul 2009 12:45:06 +0200 Subject: [erlang-questions] Lost messages In-Reply-To: References: Message-ID: maybe event_manager could help here?? In other words to have some process that guards that all processes that should receive msg, are still there, so you can link them and do some logic when you receive 'EXIT' signal from killed receiver. then you store PIDs list in that guard process and it can raise exception if you try to send msg to non existing PID. anyway I would like to here some comments about that too:) cheers pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From kiszl@REDACTED Mon Jul 27 12:49:13 2009 From: kiszl@REDACTED (kiszl@REDACTED) Date: Mon, 27 Jul 2009 12:49:13 +0200 (CEST) Subject: [erlang-questions] Lost messages In-Reply-To: References: Message-ID: <4623.194.88.55.211.1248691753.squirrel@localhost> So you are basically sending a message to a non-existent process, and this message gets "lost". Given that you find this a problem, where would you expect the "lost" message to arrive at? Zoltan. > It is said that send message is safe. But I count the problem of lost > message. > > The situation is like this: > > 1. I use a ets table to store all process to be communicated. > 2. But those process will terminated with some cases. > 3. In case of one process is terminated but still not reflected to the ets > table, send message to this process happened sametimes. It seems the > message is send, but it can not reach since the process is termiated and > the message is lost. > > Anyone knows how to solve this problem? > > ???????????????????????????????????????????????????? > ???? George Xu > > Mobile Internet Service Lab, Lonovo Corp. Research China > Tel:86-10-58866181 > E-mail:xuxiang1@REDACTED > No.6 Shang Di West Road, Haidian District, > Beijing, 100085 > P.R.China > From kamiseq@REDACTED Mon Jul 27 13:21:29 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Mon, 27 Jul 2009 13:21:29 +0200 Subject: [erlang-questions] Lost messages In-Reply-To: References: Message-ID: just resend Attila Rajmund Nohl (attila.r.nohl@REDACTED) said : 2009/7/27, pawe? kami?ski : > maybe event_manager could help here?? > In other words to have some process that guards that all processes that > should receive msg, are still there, so you can link them and do some logic > when you receive 'EXIT' signal from killed receiver. then you store PIDs > list in that guard process and it can raise exception if you try to send msg > to non existing PID. > > anyway I would like to here some comments about that too:) This is still open to a race condition: what happens if the message had been already sent, but the recepient process gets killed before it reaches the receive statement? In this case the 'EXIT' signal would be received after the message had been already sent... pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From kamiseq@REDACTED Mon Jul 27 13:26:47 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Mon, 27 Jul 2009 13:26:47 +0200 Subject: [erlang-questions] Lost messages In-Reply-To: <401d3ba30907270412u7d9faa8bj1d999f2182c0c991@mail.gmail.com> References: <401d3ba30907270412u7d9faa8bj1d999f2182c0c991@mail.gmail.com> Message-ID: ok I see your point, so in general you need relay on receivers of your events that they always will be there. or implement your own send and confirm protocol within erl system pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ > > This is still open to a race condition: what happens if the message > had been already sent, but the recepient process gets killed before it > reaches the receive statement? In this case the 'EXIT' signal would be > received after the message had been already sent... > From mazen.harake@REDACTED Mon Jul 27 13:21:17 2009 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 27 Jul 2009 14:21:17 +0300 Subject: [erlang-questions] Lost messages In-Reply-To: References: Message-ID: <4A6D8DAD.9040304@erlang-consulting.com> Hi George, I'm curious in which situation you are considering this a problem. Can you please elaborate? Also, if you can, you should somehow link to the processes and clean them up from the ETS table to avoid this problem. Regards, /Mazen George Xu wrote: > It is said that send message is safe. But I count the problem of lost > message. > > The situation is like this: > > 1. I use a ets table to store all process to be communicated. > 2. But those process will terminated with some cases. > 3. In case of one process is terminated but still not reflected to the ets > table, send message to this process happened sametimes. It seems the > message is send, but it can not reach since the process is termiated and > the message is lost. > > Anyone knows how to solve this problem? > > ?????????????????????????? > ?? George Xu > > Mobile Internet Service Lab, Lonovo Corp. Research China > Tel:86-10-58866181 > E-mail:xuxiang1@REDACTED > No.6 Shang Di West Road, Haidian District, > Beijing, 100085 > P.R.China > From attila.r.nohl@REDACTED Mon Jul 27 14:39:56 2009 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 27 Jul 2009 14:39:56 +0200 Subject: [erlang-questions] Lost messages In-Reply-To: <4A6D8DAD.9040304@erlang-consulting.com> References: <4A6D8DAD.9040304@erlang-consulting.com> Message-ID: <401d3ba30907270539m601fafa5w96e0a8de31960227@mail.gmail.com> The linking itself does not avoid the problem, it just makes it occur less often (thus harder to reproduce in case it leads to a bug). If the message is sent after the process died, but before the 'EXIT' message is received (due to linking), it's still a problem. By the way, an interesting race condition I've seen a couple of months ago: process A links to process B, which in turn links to process C. Process A also monitors process C and none of the processes trap exits. What happens if process C gets (brutally) killed? The code I was fixing was based on the assumption that process A receives the {'DOWN', ...} message (because of the monitor), but in real life process A was killed before the {'DOWN',...} message was received, leading to a bug... 2009/7/27, Mazen Harake : > Hi George, > > I'm curious in which situation you are considering this a problem. Can > you please elaborate? > > Also, if you can, you should somehow link to the processes and clean > them up from the ETS table to avoid this problem. > > Regards, > > /Mazen > > George Xu wrote: >> It is said that send message is safe. But I count the problem of lost >> message. >> >> The situation is like this: >> >> 1. I use a ets table to store all process to be communicated. >> 2. But those process will terminated with some cases. >> 3. In case of one process is terminated but still not reflected to the ets >> >> table, send message to this process happened sametimes. It seems the >> message is send, but it can not reach since the process is termiated and >> the message is lost. >> >> Anyone knows how to solve this problem? >> >> ?????????????????????????? >> ?? George Xu >> >> Mobile Internet Service Lab, Lonovo Corp. Research China >> Tel:86-10-58866181 >> E-mail:xuxiang1@REDACTED >> No.6 Shang Di West Road, Haidian District, >> Beijing, 100085 >> P.R.China >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From mazen.harake@REDACTED Mon Jul 27 16:46:47 2009 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 27 Jul 2009 17:46:47 +0300 Subject: [erlang-questions] Lost messages In-Reply-To: <401d3ba30907270539m601fafa5w96e0a8de31960227@mail.gmail.com> References: <4A6D8DAD.9040304@erlang-consulting.com> <401d3ba30907270539m601fafa5w96e0a8de31960227@mail.gmail.com> Message-ID: <4A6DBDD7.7000304@erlang-consulting.com> You are absolutely right, I know this, but the point is not to solve the race condition in the ETS table, it is to clean it up. (There should be a way to clean it up I guess). In the end it is impossible to solve this problem since there is no way in Erlang to verify that the message was received without sending one back and if so, how long do you wait for a reply before you time out and does that mean that the process is dead if you time out? etc... The simple reasoning I use in this case is; If there are too many things I'm trying to control to make it work then perhaps my way of making it work is wrong from the beginning. Usually when things become too complex you are generally doing it wrong and too much complexity is an alarm clock to wake you up and rethink, that is what I feel everytime I encounter something like this anyway. This is why I was curious about what the actual problem is more then the solution to a perhaps imaginary problem. /M Attila Rajmund Nohl wrote: > The linking itself does not avoid the problem, it just makes it occur > less often (thus harder to reproduce in case it leads to a bug). If > the message is sent after the process died, but before the 'EXIT' > message is received (due to linking), it's still a problem. > > By the way, an interesting race condition I've seen a couple of months ago: > process A links to process B, which in turn links to process C. > Process A also monitors process C and none of the processes trap > exits. What happens if process C gets (brutally) killed? The code I > was fixing was based on the assumption that process A receives the > {'DOWN', ...} message (because of the monitor), but in real life > process A was killed before the {'DOWN',...} message was received, > leading to a bug... > > 2009/7/27, Mazen Harake : > >> Hi George, >> >> I'm curious in which situation you are considering this a problem. Can >> you please elaborate? >> >> Also, if you can, you should somehow link to the processes and clean >> them up from the ETS table to avoid this problem. >> >> Regards, >> >> /Mazen >> >> George Xu wrote: >> >>> It is said that send message is safe. But I count the problem of lost >>> message. >>> >>> The situation is like this: >>> >>> 1. I use a ets table to store all process to be communicated. >>> 2. But those process will terminated with some cases. >>> 3. In case of one process is terminated but still not reflected to the ets >>> >>> table, send message to this process happened sametimes. It seems the >>> message is send, but it can not reach since the process is termiated and >>> the message is lost. >>> >>> Anyone knows how to solve this problem? >>> >>> ?????????????????????????? >>> ?? George Xu >>> >>> Mobile Internet Service Lab, Lonovo Corp. Research China >>> Tel:86-10-58866181 >>> E-mail:xuxiang1@REDACTED >>> No.6 Shang Di West Road, Haidian District, >>> Beijing, 100085 >>> P.R.China >>> >>> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> From mazen.harake@REDACTED Mon Jul 27 16:50:42 2009 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 27 Jul 2009 17:50:42 +0300 Subject: [erlang-questions] Lost messages In-Reply-To: <401d3ba30907270539m601fafa5w96e0a8de31960227@mail.gmail.com> References: <4A6D8DAD.9040304@erlang-consulting.com> <401d3ba30907270539m601fafa5w96e0a8de31960227@mail.gmail.com> Message-ID: <4A6DBEC2.50404@erlang-consulting.com> Just noticed that I had written "..to avoid this problem.." which was a bit misleading... you are more then correct :) /M Attila Rajmund Nohl wrote: > The linking itself does not avoid the problem, it just makes it occur > less often (thus harder to reproduce in case it leads to a bug). If > the message is sent after the process died, but before the 'EXIT' > message is received (due to linking), it's still a problem. > > By the way, an interesting race condition I've seen a couple of months ago: > process A links to process B, which in turn links to process C. > Process A also monitors process C and none of the processes trap > exits. What happens if process C gets (brutally) killed? The code I > was fixing was based on the assumption that process A receives the > {'DOWN', ...} message (because of the monitor), but in real life > process A was killed before the {'DOWN',...} message was received, > leading to a bug... > > 2009/7/27, Mazen Harake : > >> Hi George, >> >> I'm curious in which situation you are considering this a problem. Can >> you please elaborate? >> >> Also, if you can, you should somehow link to the processes and clean >> them up from the ETS table to avoid this problem. >> >> Regards, >> >> /Mazen >> >> George Xu wrote: >> >>> It is said that send message is safe. But I count the problem of lost >>> message. >>> >>> The situation is like this: >>> >>> 1. I use a ets table to store all process to be communicated. >>> 2. But those process will terminated with some cases. >>> 3. In case of one process is terminated but still not reflected to the ets >>> >>> table, send message to this process happened sametimes. It seems the >>> message is send, but it can not reach since the process is termiated and >>> the message is lost. >>> >>> Anyone knows how to solve this problem? >>> >>> ?????????????????????????? >>> ?? George Xu >>> >>> Mobile Internet Service Lab, Lonovo Corp. Research China >>> Tel:86-10-58866181 >>> E-mail:xuxiang1@REDACTED >>> No.6 Shang Di West Road, Haidian District, >>> Beijing, 100085 >>> P.R.China >>> >>> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> From roberto@REDACTED Mon Jul 27 19:29:24 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 27 Jul 2009 19:29:24 +0200 Subject: Misultin v0.1 released Message-ID: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> Dear all, I've just released Misultin, an Erlang library for building fast lightweight HTTP servers. Why another lightweight HTTP library in Erlang, when there's such a popular and (may I add) wonderful Mochiweb library out there? The answer is simple. Misultin is targeted to be more lightweight, and faster. The first benchmarks show that Misultin is an average 25-30% faster than Mochiweb: http://code.google.com/p/misultin/wiki/Benchmarks examples of what can be done with Misultin are available here: http://code.google.com/p/misultin/wiki/ExamplesPage Cheers, >-|-|-(?> !r From rapsey@REDACTED Mon Jul 27 20:17:11 2009 From: rapsey@REDACTED (Rapsey) Date: Mon, 27 Jul 2009 20:17:11 +0200 Subject: [erlang-questions] Misultin v0.1 released In-Reply-To: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> References: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> Message-ID: <97619b170907271117y2fb29650wff0a5e65d5d387bd@mail.gmail.com> If performance is your priority, perhaps you should think about using binaries instead of lists for parsing requests (http_bin instead of http in setopts). Matching binaries is much faster. Also using file:read_file for sending files is not exactly the best design choice. Sergej On Mon, Jul 27, 2009 at 7:29 PM, Roberto Ostinelli wrote: > Dear all, > > I've just released Misultin, an Erlang library for building fast > lightweight HTTP servers. > > Why another lightweight HTTP library in Erlang, when there's such a popular > and (may I add) wonderful Mochiweb library out there? > > The answer is simple. Misultin is targeted to be more lightweight, and > faster. The first benchmarks show that Misultin is an average 25-30% faster > than Mochiweb: > > http://code.google.com/p/misultin/wiki/Benchmarks > > examples of what can be done with Misultin are available here: > > http://code.google.com/p/misultin/wiki/ExamplesPage > > Cheers, > > >-|-|-(?> > > !r > From prikrutil@REDACTED Mon Jul 27 20:47:11 2009 From: prikrutil@REDACTED (Sergey Samokhin) Date: Mon, 27 Jul 2009 11:47:11 -0700 Subject: Is mnesia:add_table_copy/3 completely synchronous? Message-ID: Hello! Do I need to call mnesia:wait_for_tables/2 right after mnesia:add_table_copy/3 to make sure that all tables I'm interested in get available before going further? In other words, is add_table_copy/3 completely synchronous? Thanks. -- Sergey Samokhin From roberto@REDACTED Mon Jul 27 21:19:25 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 27 Jul 2009 21:19:25 +0200 Subject: [erlang-questions] Misultin v0.1 released In-Reply-To: <97619b170907271117y2fb29650wff0a5e65d5d387bd@mail.gmail.com> References: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> <97619b170907271117y2fb29650wff0a5e65d5d387bd@mail.gmail.com> Message-ID: <884BCF01-E86F-4B30-A914-297E559DF373@widetag.com> hi sergej, i have avoided http_bin for this release, since it is a new packet type available only on R13. i'll probably integrate it and set a fallback to http for previous versions of erlang. thank you for this feedback. r. On 27/lug/09, at 20:17, Rapsey wrote: > If performance is your priority, perhaps you should think about using > binaries instead of lists for parsing requests (http_bin instead of > http in > setopts). Matching binaries is much faster. > Also using file:read_file for sending files is not exactly the best > design > choice. > > > Sergej > > On Mon, Jul 27, 2009 at 7:29 PM, Roberto Ostinelli >wrote: > >> Dear all, >> >> I've just released Misultin, an Erlang library for building fast >> lightweight HTTP servers. >> >> Why another lightweight HTTP library in Erlang, when there's such a >> popular >> and (may I add) wonderful Mochiweb library out there? >> >> The answer is simple. Misultin is targeted to be more lightweight, >> and >> faster. The first benchmarks show that Misultin is an average >> 25-30% faster >> than Mochiweb: >> >> http://code.google.com/p/misultin/wiki/Benchmarks >> >> examples of what can be done with Misultin are available here: >> >> http://code.google.com/p/misultin/wiki/ExamplesPage >> >> Cheers, >> >>> -|-|-(?> >> >> !r >> From dizzyd@REDACTED Mon Jul 27 21:49:21 2009 From: dizzyd@REDACTED (Dave Smith) Date: Mon, 27 Jul 2009 13:49:21 -0600 Subject: Missing HIPE modules in hipe.app Message-ID: Hi all... As a part of constructing an erlang embedded node, I found that the HIPE application is missing a number of modules from the .app file. Consequently, systools doesn't include those modules when the HIPE application is included in the release. I'm not sure if it's intentional or not, but thought I'd submit a patch against hipe.app (see attached). In the process, I wrote a verifier and found other problems as well: WARNING: snmp.app does not list snmpc.beam as a module WARNING: snmp.app does not list snmpc_lib.beam as a module WARNING: snmp.app does not list snmpc_mib_gram.beam as a module WARNING: snmp.app does not list snmpc_mib_to_hrl.beam as a module WARNING: snmp.app does not list snmpc_misc.beam as a module WARNING: snmp.app does not list snmpc_tok.beam as a module WARNING: asn1.app does not list asn1_db.beam as a module WARNING: asn1.app does not list asn1ct.beam as a module WARNING: asn1.app does not list asn1ct_check.beam as a module WARNING: asn1.app does not list asn1ct_constructed_ber.beam as a module WARNING: asn1.app does not list asn1ct_constructed_ber_bin_v2.beam as a module WARNING: asn1.app does not list asn1ct_constructed_per.beam as a module WARNING: asn1.app does not list asn1ct_gen.beam as a module WARNING: asn1.app does not list asn1ct_gen_ber.beam as a module WARNING: asn1.app does not list asn1ct_gen_ber_bin_v2.beam as a module WARNING: asn1.app does not list asn1ct_gen_per.beam as a module WARNING: asn1.app does not list asn1ct_gen_per_rt2ct.beam as a module WARNING: asn1.app does not list asn1ct_name.beam as a module WARNING: asn1.app does not list asn1ct_parser2.beam as a module WARNING: asn1.app does not list asn1ct_pretty_format.beam as a module WARNING: asn1.app does not list asn1ct_tok.beam as a module WARNING: asn1.app does not list asn1ct_value.beam as a module WARNING: sasl.app does not list si.beam as a module WARNING: sasl.app does not list si_sasl_supp.beam as a module Hopefully someone will find this useful. :) It would be great to see this included in the next release. D. From mikpe@REDACTED Mon Jul 27 23:55:51 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Mon, 27 Jul 2009 23:55:51 +0200 Subject: [erlang-questions] Missing HIPE modules in hipe.app In-Reply-To: References: Message-ID: <19054.8807.149918.977855@pilspetsen.it.uu.se> Dave Smith writes: > Hi all... > > As a part of constructing an erlang embedded node, I found that the > HIPE application is missing a number of modules from the .app file. > Consequently, systools doesn't include those modules when the HIPE > application is included in the release. I'm not sure if it's > intentional or not, but thought I'd submit a patch against hipe.app > (see attached). -ENOATTACHEDPATCH From dizzyd@REDACTED Tue Jul 28 00:11:06 2009 From: dizzyd@REDACTED (Dave Smith) Date: Mon, 27 Jul 2009 16:11:06 -0600 Subject: [erlang-questions] Missing HIPE modules in hipe.app In-Reply-To: <19054.8807.149918.977855@pilspetsen.it.uu.se> References: <19054.8807.149918.977855@pilspetsen.it.uu.se> Message-ID: Not sure what went wrong, here. Think I may not have waited long enough for gmail to upload the file (!!). Here is the patch again, this time against hipe.app.src, as suggested by my co-worker Jon Meredith. :) D. On Mon, Jul 27, 2009 at 3:55 PM, Mikael Pettersson wrote: > Dave Smith writes: > ?> Hi all... > ?> > ?> As a part of constructing an erlang embedded node, I found that the > ?> HIPE application is missing a number of modules from the .app file. > ?> Consequently, systools doesn't include those modules when the HIPE > ?> application is included in the release. I'm not sure if it's > ?> intentional or not, but thought I'd submit a patch against hipe.app > ?> (see attached). > > -ENOATTACHEDPATCH > From mikpe@REDACTED Tue Jul 28 00:31:17 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Tue, 28 Jul 2009 00:31:17 +0200 Subject: [erlang-questions] Missing HIPE modules in hipe.app In-Reply-To: References: <19054.8807.149918.977855@pilspetsen.it.uu.se> Message-ID: <19054.10933.736242.834672@pilspetsen.it.uu.se> Dave Smith writes: > Not sure what went wrong, here. Think I may not have waited long > enough for gmail to upload the file (!!). > > Here is the patch again, this time against hipe.app.src, as suggested > by my co-worker Jon Meredith. :) On first glance it looks correct but whitespace damaged (no tabs). I'll fix that up and apply it tomorrow. Thanks for reporting this. We obviously never use the app file so don't notice when it goes out of sync :-( /Mikael From roberto@REDACTED Tue Jul 28 00:49:18 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 28 Jul 2009 00:49:18 +0200 Subject: [erlang-questions] Misultin v0.1 released In-Reply-To: <97619b170907271117y2fb29650wff0a5e65d5d387bd@mail.gmail.com> References: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> <97619b170907271117y2fb29650wff0a5e65d5d387bd@mail.gmail.com> Message-ID: <91274D9A-DB6C-4D0D-8BD7-021486763DBC@widetag.com> > Also using file:read_file for sending files is not exactly the best > design > choice. added chunked file reading/sending using Misultin's stream support. everything is now packaged in a single zip for download. http://code.google.com/p/misultin/downloads/list r. From xuxiang1@REDACTED Tue Jul 28 02:07:22 2009 From: xuxiang1@REDACTED (George Xu) Date: Tue, 28 Jul 2009 08:07:22 +0800 Subject: [erlang-questions] Lost messages In-Reply-To: <401d3ba30907270539m601fafa5w96e0a8de31960227@mail.gmail.com> Message-ID: Yes?I already do the link and clean ETS table work. But no one can solve the race condition I think. So, in very rare case, I lost the message ('EXIT' is arrived after the message send out). I think this is not a "erLang way". It shall occur an error and let the sending process exit. Because that the message is important and the router which collect messages and send out can not be blocked by a dead / exit process, send a message back is not an acceptale solution. Does anyone have solutions for this? ?????????????????????????? ?? George Xu Mobile Internet Service Lab, Lonovo Corp. Research China Tel:86-10-58866181 E-mail:xuxiang1@REDACTED No.6 Shang Di West Road, Haidian District, Beijing, 100085 P.R.China Attila Rajmund Nohl Sent by: 2009-07-27 20:39 To Mazen Harake cc George Xu , erlang-questions@REDACTED Subject Re: [erlang-questions] Lost messages The linking itself does not avoid the problem, it just makes it occur less often (thus harder to reproduce in case it leads to a bug). If the message is sent after the process died, but before the 'EXIT' message is received (due to linking), it's still a problem. By the way, an interesting race condition I've seen a couple of months ago: process A links to process B, which in turn links to process C. Process A also monitors process C and none of the processes trap exits. What happens if process C gets (brutally) killed? The code I was fixing was based on the assumption that process A receives the {'DOWN', ...} message (because of the monitor), but in real life process A was killed before the {'DOWN',...} message was received, leading to a bug... 2009/7/27, Mazen Harake : > Hi George, > > I'm curious in which situation you are considering this a problem. Can > you please elaborate? > > Also, if you can, you should somehow link to the processes and clean > them up from the ETS table to avoid this problem. > > Regards, > > /Mazen > > George Xu wrote: >> It is said that send message is safe. But I count the problem of lost >> message. >> >> The situation is like this: >> >> 1. I use a ets table to store all process to be communicated. >> 2. But those process will terminated with some cases. >> 3. In case of one process is terminated but still not reflected to the ets >> >> table, send message to this process happened sametimes. It seems the >> message is send, but it can not reach since the process is termiated and >> the message is lost. >> >> Anyone knows how to solve this problem? >> >> ?????????????????????????? >> ?? George Xu >> >> Mobile Internet Service Lab, Lonovo Corp. Research China >> Tel:86-10-58866181 >> E-mail:xuxiang1@REDACTED >> No.6 Shang Di West Road, Haidian District, >> Beijing, 100085 >> P.R.China >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From ngocdaothanh@REDACTED Tue Jul 28 03:48:36 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Tue, 28 Jul 2009 10:48:36 +0900 Subject: [erlang-questions] Re: erlang-amf In-Reply-To: <3792a5cf-8649-489b-9d78-36800d12e102@g7g2000prg.googlegroups.com> References: <6211282E-2D62-4A6C-9758-E2A13D8D1D9D@babayev.com> <3792a5cf-8649-489b-9d78-36800d12e102@g7g2000prg.googlegroups.com> Message-ID: <5c493e530907271848g1b8a92b6n1f81756f98cfcef@mail.gmail.com> Hi. There are only .erl files, no .c files. Why do you need to use so complicated configure files to build them? I'm just curious. Ngoc. On Mon, Jul 6, 2009 at 4:03 PM, Ruslan wrote: > Forgot the actual link http://github.com/mujaheed/erlang-amf/tree/master > > On Jul 6, 12:01 am, Ruslan Babayev wrote: > > I have finally published the Erlang AMF library I have been working on > > to github. It has full support for Adobe AMF0 and AMF3 specs. > > > > Enjoy responsibly. > From romain.lenglet@REDACTED Tue Jul 28 04:11:00 2009 From: romain.lenglet@REDACTED (Romain Lenglet) Date: Tue, 28 Jul 2009 11:11:00 +0900 Subject: [erlang-questions] Re: erlang-amf In-Reply-To: <5c493e530907271848g1b8a92b6n1f81756f98cfcef@mail.gmail.com> References: <6211282E-2D62-4A6C-9758-E2A13D8D1D9D@babayev.com> <3792a5cf-8649-489b-9d78-36800d12e102@g7g2000prg.googlegroups.com> <5c493e530907271848g1b8a92b6n1f81756f98cfcef@mail.gmail.com> Message-ID: <3F08C615-ABB7-4AFD-B264-ADE2EA308635@berabera.info> Hi Ngoc, What files are you looking at? configure is generated automatically from configure.ac. I find this configure.ac exemplar. -- Romain Lenglet On Jul 28, 2009, at 10:48, Ngoc Dao wrote: > Hi. > > There are only .erl files, no .c files. Why do you need to use so > complicated configure files to build them? I'm just curious. > > Ngoc. > > > On Mon, Jul 6, 2009 at 4:03 PM, Ruslan wrote: > >> Forgot the actual link http://github.com/mujaheed/erlang-amf/tree/master >> >> On Jul 6, 12:01 am, Ruslan Babayev wrote: >>> I have finally published the Erlang AMF library I have been >>> working on >>> to github. It has full support for Adobe AMF0 and AMF3 specs. >>> >>> Enjoy responsibly. >> From ngocdaothanh@REDACTED Tue Jul 28 04:25:16 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Tue, 28 Jul 2009 11:25:16 +0900 Subject: [erlang-questions] Re: erlang-amf In-Reply-To: <3F08C615-ABB7-4AFD-B264-ADE2EA308635@berabera.info> References: <6211282E-2D62-4A6C-9758-E2A13D8D1D9D@babayev.com> <3792a5cf-8649-489b-9d78-36800d12e102@g7g2000prg.googlegroups.com> <5c493e530907271848g1b8a92b6n1f81756f98cfcef@mail.gmail.com> <3F08C615-ABB7-4AFD-B264-ADE2EA308635@berabera.info> Message-ID: <5c493e530907271925qc69dd7ay62f603d9f51e15a@mail.gmail.com> I mean the build process is so complicated. Why not use just a Emakefile and Makefile like this? http://github.com/ngocdaothanh/ale/blob/master/Emakefile http://github.com/ngocdaothanh/ale/blob/master/Makefile Ngoc. On Tue, Jul 28, 2009 at 11:11 AM, Romain Lenglet < romain.lenglet@REDACTED> wrote: > Hi Ngoc, > What files are you looking at? > configure is generated automatically from configure.ac. > I find this configure.ac exemplar. > -- > Romain Lenglet > > > On Jul 28, 2009, at 10:48, Ngoc Dao wrote: > > Hi. >> >> There are only .erl files, no .c files. Why do you need to use so >> complicated configure files to build them? I'm just curious. >> >> Ngoc. >> >> >> On Mon, Jul 6, 2009 at 4:03 PM, Ruslan wrote: >> >> Forgot the actual link http://github.com/mujaheed/erlang-amf/tree/master >>> >>> On Jul 6, 12:01 am, Ruslan Babayev wrote: >>> >>>> I have finally published the Erlang AMF library I have been working on >>>> to github. It has full support for Adobe AMF0 and AMF3 specs. >>>> >>>> Enjoy responsibly. >>>> >>> >>> > From mazen.harake@REDACTED Tue Jul 28 07:31:10 2009 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 28 Jul 2009 08:31:10 +0300 Subject: [erlang-questions] Misultin v0.1 released In-Reply-To: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> References: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> Message-ID: <4A6E8D1E.6060900@erlang-consulting.com> You seem to have the same flaw I found in Yaws a couple of months ago... Look at line 99 in misultin_socket.erl... it seems that you will endlessly collect headers... not a good idea. You should add a counter to allow perhaps only 30 headers or a 1000 or so otherwise your Erlang VM will come crashing down. oh and also... no documentation only a stupid online wiki :( Regards, /Mazen Roberto Ostinelli wrote: > Dear all, > > I've just released Misultin, an Erlang library for building fast > lightweight HTTP servers. > > Why another lightweight HTTP library in Erlang, when there's such a > popular and (may I add) wonderful Mochiweb library out there? > > The answer is simple. Misultin is targeted to be more lightweight, and > faster. The first benchmarks show that Misultin is an average 25-30% > faster than Mochiweb: > > http://code.google.com/p/misultin/wiki/Benchmarks > > examples of what can be done with Misultin are available here: > > http://code.google.com/p/misultin/wiki/ExamplesPage > > Cheers, > > >-|-|-(?> > > !r > From bgustavsson@REDACTED Tue Jul 28 09:07:26 2009 From: bgustavsson@REDACTED (Bjorn Gustavsson) Date: Tue, 28 Jul 2009 09:07:26 +0200 Subject: [erlang-questions] The is_record() guard In-Reply-To: <4A6B6D7C.1000500@gmail.com> References: <4A689F53.4090204@erlang-consulting.com> <6672d0160907250345i7733bcf9h54e36214b95afb49@mail.gmail.com> <4A6B6D7C.1000500@gmail.com> Message-ID: <6672d0160907280007x43a24492i94b88716614fcd19@mail.gmail.com> On Sat, Jul 25, 2009 at 10:39 PM, Steve Davis wrote: > Hi Bjorn, > > Definitely the approach suggested by Oscar shows much better numbers in > fprof... > > I also see other cases where functions that filter arguments using guards > appear to be (quite a bit) less efficient than direct matches on the > arguments. > > Would it be fair to say that, where possible. pattern matching function > arguments is in general more efficient than using guards? Yes. There is a section in the Efficiency Guide http://erlang.org/doc/efficiency_guide/functions.html#6.1 that gives some insight into why guards may be inefficient (and also shows an example where pattern matching without guards also can be inefficient). /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From roberto@REDACTED Tue Jul 28 09:50:18 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 28 Jul 2009 09:50:18 +0200 Subject: [erlang-questions] Misultin v0.1 released In-Reply-To: <4A6E8D1E.6060900@erlang-consulting.com> References: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> <4A6E8D1E.6060900@erlang-consulting.com> Message-ID: On 28/lug/09, at 07:31, Mazen Harake wrote: > You seem to have the same flaw I found in Yaws a couple of months > ago... > > Look at line 99 in misultin_socket.erl... it seems that you will > endlessly collect headers... not a good idea. You should add a > counter to allow perhaps only 30 headers or a 1000 or so otherwise > your Erlang VM will come crashing down. thank tou mazen, misultin_socket has already been patched to solve the issue you point out. http://code.google.com/p/misultin/source/detail?r=134 > oh and also... no documentation only a stupid online wiki :( it's a simple library, i believe Exports are pretty much inline with the rest of the erlang man pages: http://code.google.com/p/misultin/wiki/Exports this, coupled with examples, i really think should do. r. From roberto@REDACTED Tue Jul 28 10:11:14 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 28 Jul 2009 10:11:14 +0200 Subject: [erlang-questions] Misultin v0.1 released In-Reply-To: <50db8f50907272231m6c70a16pbffb2f7598b69c04@mail.gmail.com> References: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> <97619b170907271117y2fb29650wff0a5e65d5d387bd@mail.gmail.com> <91274D9A-DB6C-4D0D-8BD7-021486763DBC@widetag.com> <50db8f50907272231m6c70a16pbffb2f7598b69c04@mail.gmail.com> Message-ID: <61CB6F24-57DD-4CCE-9995-459DF78A2C4D@widetag.com> On 28/lug/09, at 07:31, Zhuguo Shi wrote: > Great news. But I still wonder if it can be a substitute of apache > or lighthttpd in realworld? Could you make a comparison of them? well, i believe it is a little like comparing apple to peers: apache is a fully featured web server while misultin is a library, and plus apache does not provide a dynamic content engine, you need to plug one in: php, python, ruby, ... and thus speed [not the number of handled connections] is dependant on the chosen engine. however, this is a common question and i will perform some testings on apache too. cheers, !r From antoine.koener@REDACTED Tue Jul 28 10:57:05 2009 From: antoine.koener@REDACTED (Antoine Koener) Date: Tue, 28 Jul 2009 10:57:05 +0200 Subject: [erlang-questions] Misultin v0.1 released In-Reply-To: <61CB6F24-57DD-4CCE-9995-459DF78A2C4D@widetag.com> References: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> <97619b170907271117y2fb29650wff0a5e65d5d387bd@mail.gmail.com> <91274D9A-DB6C-4D0D-8BD7-021486763DBC@widetag.com> <50db8f50907272231m6c70a16pbffb2f7598b69c04@mail.gmail.com> <61CB6F24-57DD-4CCE-9995-459DF78A2C4D@widetag.com> Message-ID: <1c89b3a10907280157u1e67ff97w12a5a7e192ae5058@mail.gmail.com> On Tue, Jul 28, 2009 at 10:11 AM, Roberto Ostinelli wrote: > On 28/lug/09, at 07:31, Zhuguo Shi wrote: > > Great news. But I still wonder if it can be a substitute of apache or >> lighthttpd in realworld? Could you make a comparison of them? >> > > well, i believe it is a little like comparing apple to peers: apache is a > fully featured web server while misultin is a library, and plus apache does > not provide a dynamic content engine, you need to plug one in: php, python, > ruby, ... and thus speed [not the number of handled connections] is > dependant on the chosen engine. > > however, this is a common question and i will perform some testings on > apache too. If you do benchmarks, please don't forget to test 'nginx'... I think that you won't beat any optimized httpd since the memory used by one client is rather low compared to what erlang does. (socket + buffers) See also: http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1/ From roberto@REDACTED Tue Jul 28 11:04:04 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 28 Jul 2009 11:04:04 +0200 Subject: [erlang-questions] Misultin v0.1 released In-Reply-To: <1c89b3a10907280157u1e67ff97w12a5a7e192ae5058@mail.gmail.com> References: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> <97619b170907271117y2fb29650wff0a5e65d5d387bd@mail.gmail.com> <91274D9A-DB6C-4D0D-8BD7-021486763DBC@widetag.com> <50db8f50907272231m6c70a16pbffb2f7598b69c04@mail.gmail.com> <61CB6F24-57DD-4CCE-9995-459DF78A2C4D@widetag.com> <1c89b3a10907280157u1e67ff97w12a5a7e192ae5058@mail.gmail.com> Message-ID: <4385922A-337F-40FB-B58E-70D7F47F116D@widetag.com> On 28/lug/09, at 10:57, Antoine Koener wrote: > > I think that you won't beat any optimized httpd since the memory > used by one client is rather low compared to what erlang does. > (socket + buffers) > > See also: http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1/ i'm pretty familiar with that article, and it relates to open connections which is quite a different matter. moreover, i tend to believe that erlang has been optimized on this since then. tests on cpu and memory will anyway have to be done. r. From hakan@REDACTED Tue Jul 28 11:12:37 2009 From: hakan@REDACTED (Hakan Mattsson) Date: Tue, 28 Jul 2009 11:12:37 +0200 (CEST) Subject: [erlang-questions] Is mnesia:add_table_copy/3 completely synchronous? In-Reply-To: References: Message-ID: On Mon, 27 Jul 2009, Sergey Samokhin wrote: > Do I need to call mnesia:wait_for_tables/2 right after > mnesia:add_table_copy/3 to make sure that all tables I'm > interested in get available before going further? No. > In other words, is add_table_copy/3 completely synchronous? Yes. The new replica is directly available when mnesia:add_table_copy/3 returns. /H?kan --- H?kan Mattsson (uabhams) Erlang/OTP, Ericsson AB From hakan@REDACTED Tue Jul 28 11:26:16 2009 From: hakan@REDACTED (Hakan Mattsson) Date: Tue, 28 Jul 2009 11:26:16 +0200 (CEST) Subject: [erlang-questions] Log dumping not thread-safe? In-Reply-To: References: Message-ID: On Fri, 24 Jul 2009, Igor Ribeiro Sucupira wrote: > mnesia:dump_log() is not thread-safe. Am I right? What do you mean with thread-safe in this context? > In practice, that would mean you can never use this function, as > Mnesia could be dumping the log at the same moment you called this > function. Is that correct? No. Mnesia will start dumping its transaction log when you invoke this function. But it does not interfere with ongoing transactions. You can still use Mnesia as usual while the dump is progressing as a background job. /H?kan --- H?kan Mattsson (uabhams) Erlang/OTP, Ericsson AB From steven.charles.davis@REDACTED Tue Jul 28 11:44:38 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 28 Jul 2009 02:44:38 -0700 (PDT) Subject: Misultin v0.1 released In-Reply-To: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> References: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> Message-ID: Hi Roberto, Always good to have more tools for building stuff with Erlang, but I am driven to comment on your benchmark comment: "Misultin is an average *25-30% faster* than Mochiweb" (your emphasis) ...well, this doesn't surprise me enormously as Misultin appears to do about 40-50% less than Mochiweb! I can see a use for Misultin for very simple serving apps, but as soon as you get into building any non-trivial application you'll need to add back that functionality -- and I suspect that will slow the server down a fair bit. Out of interest, you should take a look at the Yaws/Mochiweb comparison (link?). These are interesting because Yaws does in general 50-80% more for you than Mochiweb but yet compares very favorably on the benchmarks. Congrats on your release and thanks for sharing this with us. Best regards, Steve On Jul 27, 12:29?pm, Roberto Ostinelli wrote: > http://code.google.com/p/misultin/wiki/Benchmarks From mazen.harake@REDACTED Tue Jul 28 11:57:19 2009 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 28 Jul 2009 12:57:19 +0300 Subject: wx tutorial Message-ID: <4A6ECB7F.5060201@erlang-consulting.com> Hey, Just curious if there are (by now) experts using the wxwidget binding in Erlang... and if so, how about writing a small tutorial on how to use it (even if there are examples a tutorial is always nice). Perhaps on www.trapexit.org (or whatever)? Would be nice to have one, mapping the tutorials from C++ API to Erlang API was a bit more time consuming then I thought... Just a thought :) /Mazen From steven.charles.davis@REDACTED Tue Jul 28 12:18:08 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 28 Jul 2009 03:18:08 -0700 (PDT) Subject: gen_event:add_sup_handler... correct usage? Message-ID: <4c7edb5c-c29a-48e9-8c66-b274cec683af@w6g2000yqw.googlegroups.com> Hi, I have boot-time logging for a server that needs to start before anything really happens, i.e. **before the main supervisor is kicked off by application:start(my_server)**. Despite this requirement, I still need the error_logger process to supervise my gen_event handler. I have read and re-read the docs (and the error_logger/gen_event source code), but I am still not 100% clear or indeed happy that I'm doing the right thing wrt to OTP principles. I've implemented it by doing the following... ==== summarized for clarity ===== %% my_server_sup % note... this module manages both app and sup callbacks -behaviour(application). -behaviour(supervisor). .... start(_Type, Args) -> my_logger_h:start_link(), .... case lists:member(my_logger, gen_event:which_handlers (error_logger)) of true -> supervisor:start_link({local, ?MODULE}, ?MODULE, Args); _ -> io:format(user, "FATAL: Boot logger failed to initialize.", []), {error, boot_log_init} end. %% my_logger_h -behaviour(gen_event). .... start_link() -> ....various init stuff gen_event:add_sup_handler(error_logger, ?MODULE, {LogId, LogPath, Rollover}). ======end===== ...while all of this "works", I have concerns about the OTP-ness of doing this, and also a nagging feeling that the fact that application process is essentially the "supervisor" that's passed to error_logger, not the supervisor process itself. I was hoping for any advice/comment from those with more experience as I'm still learning decent OTP design. Regards and TIA, Steve From leap@REDACTED Tue Jul 28 16:41:45 2009 From: leap@REDACTED (Michael Turner) Date: Tue, 28 Jul 2009 14:41:45 +0000 Subject: R13B01: erts_cpu_info_update() "int mib[0];" + sysctl call trashes stack Message-ID: I'm trying to build R13B01. I was getting seg faults on erlexec. Bringing it up in gdb just gave me "No stack." Hours (and *hours*) later: I've traced it to the following in erl_misc_utils.c { int mib[0]; size_t len; .... Yes, that's legal (according to gcc, anyway). Even useful for some things. http://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Zero-Length.html In this context, however, it should be "int mib[2];" The problem is that there are assignments to mib[0] and mib[1] immediately following, clobbering some context. So, uh . . . this is the questions list, so I have a question after all: where do I report this as a bug? And suggest this fix? And all that? I'm pretty new to this. -michael turner From mononcqc@REDACTED Tue Jul 28 16:43:55 2009 From: mononcqc@REDACTED (Fred Hebert (MononcQc)) Date: Tue, 28 Jul 2009 10:43:55 -0400 Subject: [erlang-questions] wx tutorial In-Reply-To: <4A6ECB7F.5060201@erlang-consulting.com> References: <4A6ECB7F.5060201@erlang-consulting.com> Message-ID: <8b9ee55b0907280743t2a3b6666w5acaffb86156d0fc@mail.gmail.com> I'm not the guy who can help, but Cesarini & Thompson's Erlang Programming book ( http://www.amazon.com/Erlang-Programming-Francesco-Cesarini/dp/0596518188) provide a helpful chapter covering the basics of wxErlang with pointers on how to get more information. (In general, I'd recommend the book to anyone who wants to learn more about building OTP applications than what they've read in Joe Armstrong's book, so maybe this can make you wish to get it too). You've also got the api defined at http://www.erlang.org/~dgud/wxerlang/doc/index.html with the overview giving the very basics and the minimal -spec declarations included. On Tue, Jul 28, 2009 at 5:57 AM, Mazen Harake < mazen.harake@REDACTED> wrote: > Hey, > > Just curious if there are (by now) experts using the wxwidget binding in > Erlang... and if so, how about writing a small tutorial on how to use it > (even if there are examples a tutorial is always nice). Perhaps on > www.trapexit.org (or whatever)? > > > Would be nice to have one, mapping the tutorials from C++ API to Erlang API > was a bit more time consuming then I thought... > > Just a thought :) > > > > /Mazen > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From leap@REDACTED Tue Jul 28 16:48:28 2009 From: leap@REDACTED (Michael Turner) Date: Tue, 28 Jul 2009 14:48:28 +0000 Subject: opting out of jinterface build? In-Reply-To: <1248788394.32407.ezmlm@erlang.org> Message-ID: I'm building on a platform that doesn't have the required Java components. I won't need jinterface anyway. My config.log in .../erts says "WARNING: Could not find any useable java compiler, will skip: jinterface". But the Makefile produced still attempts to build something java-related anyway. My question: what's the clean way to leave java-related targets out of the build? -michael turner From dave.pawson@REDACTED Tue Jul 28 17:09:28 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Tue, 28 Jul 2009 16:09:28 +0100 Subject: [erlang-questions] wx tutorial In-Reply-To: <8b9ee55b0907280743t2a3b6666w5acaffb86156d0fc@mail.gmail.com> References: <4A6ECB7F.5060201@erlang-consulting.com> <8b9ee55b0907280743t2a3b6666w5acaffb86156d0fc@mail.gmail.com> Message-ID: <711a73df0907280809n1d4adc2t49ed9a98a4ebfb12@mail.gmail.com> 2009/7/28 Fred Hebert (MononcQc) : > I'm not the guy who can help, but Cesarini & Thompson's Erlang Programming > book ( > http://www.amazon.com/Erlang-Programming-Francesco-Cesarini/dp/0596518188) > provide a helpful chapter covering the basics of wxErlang with pointers on > how to get more information. > (In general, I'd recommend the book to anyone who wants to learn more about > building OTP applications than what they've read in Joe Armstrong's book, so > maybe this can make you wish to get it too). +1 to the recommendation. I'm new to Erlang and finding it a good read. It does need 'studying' though, I guess I've re-read chunks of it a number of times! More from it each time. I love the 'this is what our experience shows' kind of paragraphs. Clearly built from experience. Also provides some insights on where bits of Erlang came from, i.e. why they are like they are. Good book IMHO (just hope the binding holds up to the desktop abuse) regards -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From mikpe@REDACTED Tue Jul 28 18:00:04 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Tue, 28 Jul 2009 18:00:04 +0200 Subject: [erlang-questions] R13B01: erts_cpu_info_update() "int mib[0];" + sysctl call trashes stack In-Reply-To: References: Message-ID: <19055.8324.515261.713077@pilspetsen.it.uu.se> Michael Turner writes: > I'm trying to build R13B01. I was getting seg faults on erlexec. > Bringing it up in gdb just gave me "No stack." > > Hours (and *hours*) later: I've traced it to the following in > erl_misc_utils.c > > { > int mib[0]; > size_t len; > .... > > Yes, that's legal (according to gcc, anyway). Even useful for some > things. > > http://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Zero-Length.html > > In this context, however, it should be "int mib[2];" The problem is > that there are assignments to mib[0] and mib[1] immediately following, > clobbering some context. > > So, uh . . . this is the questions list, so I have a question after all: > where do I report this as a bug? And suggest this fix? And all that? > I'm pretty new to this. On what platform does this code get compiled? I would have expected recent GCCs to complain about the out-of-bounds indexing, but perhaps it doesn't get compiled on Linux. Anyway, you have indeed found a real bug. The code in question is utterly wrong. Proper bug reporting procedure is to send an email to the erlang-bugs mailing list, together with Erlang version information, platform information (in case the bug is platform specific), and whatever analysis you've done or test cases you've written. Suggested code changes should be in `patch -p1' form as produced by `diff -up` or its equivalent. For this particular case, it should look as follows: --- otp_src_R13B01/erts/lib_src/common/erl_misc_utils.c.~1~ 2009-06-05 14:53:41.000000000 +0200 +++ otp_src_R13B01/erts/lib_src/common/erl_misc_utils.c 2009-07-28 17:53:03.000000000 +0200 @@ -172,7 +172,7 @@ erts_cpu_info_update(erts_cpu_info_t *cp #elif defined(HAVE_SYS_SYSCTL_H) && defined(CTL_HW) && (defined(HW_NCPU) \ || defined(HW_AVAILCPU)) { - int mib[0]; + int mib[2]; size_t len; #ifdef HW_NCPU /Mikael From erlang-questions_efine@REDACTED Tue Jul 28 18:06:08 2009 From: erlang-questions_efine@REDACTED (Edwin Fine) Date: Tue, 28 Jul 2009 12:06:08 -0400 Subject: Mystery of good term gone bad In-Reply-To: <6c2563b20907280854t355b13b5i28eff78c8ea0b9d3@mail.gmail.com> References: <6c2563b20907280854t355b13b5i28eff78c8ea0b9d3@mail.gmail.com> Message-ID: <6c2563b20907280906y7b918203g9b65e18b16eb627b@mail.gmail.com> Can anyone tell me why the binary to term conversion shown below is "good" on systems 1, 2, and 3, and bad in system 4, when the binaries are identical across all of them? Surely R13B01 should be backward-compatible at the binary level? Systems 1 & 2: R13B System 3: R12B-4 System 4: R13B01 All 64-bit, all Linux. In the shell sessions below, I have included the binary data only the first example. They are identical in all examples. System 1: Erlang R13B (erts-5.7.1) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.7.1 (abort with ^G) (e7001@REDACTED)1> Bad = <<131,104,4,97,6,103,78,131,0,20,119,99,116,112,103,119, 95,114,101,108,64,108,111,99,97,108,104,111,115,116,0,0, 0,12,0,0,0,0,2,78,5,0,0,78,110,0,18,103,108,111,98,97, 108,95,110,97,109,101,95,115,101,114,118,101,114,131, 104,2,78,248,0,9,36,103,101,110,95,99,97,115,116,104,4, 78,65,0,12,105,110,105,116,95,99,111,110,110,101,99,116, 104,2,97,5,104,3,98,0,0,4,224,98,0,12,35,221,98,0,14,79, 143,67,131,104,4,78,76,0,6,108,111,99,107,101,114,78,77, 0,15,110,111,95,108,111,110,103,101,114,95,97,95,112, 105,100,106,103,67,131,0,0,0,13,0,0,0,0,2>>. (e7001@REDACTED)2> binary_to_term(Bad). {6,<6178.12.0>,'',global_name_server} System 2: Erlang R13B (erts-5.7.1) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] Eshell V5.7.1 (abort with ^G) ... (e10119@REDACTED)2> binary_to_term(Bad). {6,<6178.12.0>,'',global_name_server} System 3: Erlang (BEAM) emulator version 5.6.4 [source] [64-bit] [smp:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.6.4 (abort with ^G) ... (e8823@REDACTED)2> binary_to_term(Bad). {6,<5920.12.0>,'',global_name_server} System 4: Erlang R13B01 (erts-5.7.2) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) (e17785@REDACTED)2> binary_to_term(Bad). ** exception error: bad argument in function binary_to_term/1 called as binary_to_term(<<131,104,4,97,6,103,78,131,0,20,119,99,116,112,103,119, 95,114,101,108,64,108,111,99,97,108,104,111,...>>) From roberto@REDACTED Tue Jul 28 18:18:34 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 28 Jul 2009 18:18:34 +0200 Subject: [erlang-questions] Re: Misultin v0.1 released In-Reply-To: References: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> Message-ID: > > Hi Roberto, > > Always good to have more tools for building stuff with Erlang, but I > am driven to comment on your benchmark comment: > > "Misultin is an average *25-30% faster* than Mochiweb" (your emphasis) > > ...well, this doesn't surprise me enormously as Misultin appears to do > about 40-50% less than Mochiweb! > > I can see a use for Misultin for very simple serving apps, but as soon > as you get into building any non-trivial application you'll need to > add back that functionality -- and I suspect that will slow the server > down a fair bit. hi steve, i believe this to be incorrect. true, mochiweb comes loaded with many nice functionalities: json, cookies, multipart, ... fact is, these are not called on every request, but are *available* to a programmer upon need: an incoming request get does *not* get though all this code unless it is specifically asked by a programmer. misultin speed compared with mochiweb comes from the different approach to the socket handling, and that alone. as you can see, the benchmark used is very simple, it just echoes a variable passed in GET, and the code to do so is very similar for both libraries. > Out of interest, you should take a look at the Yaws/Mochiweb > comparison (link?). These are interesting because Yaws does in general > 50-80% more for you than Mochiweb but yet compares very favorably on > the benchmarks. if you are referring to joe's bench i'm pretty familiar with it :) > Congrats on your release and thanks for sharing this with us. > > Best regards, > Steve thank you for your feedback, steve. sincerely. r. From mononcqc@REDACTED Tue Jul 28 19:03:44 2009 From: mononcqc@REDACTED (Fred Hebert (MononcQc)) Date: Tue, 28 Jul 2009 13:03:44 -0400 Subject: Learn you some Erlang for great good! Message-ID: <8b9ee55b0907281003g311d71f9ie6ef1ddb141646ae@mail.gmail.com> Some of the readers of this list may be familiar with http://learnyouahaskell.com/, an online book/tutorial dedicated to teaching Haskell in a lighthearted way. I've always thought Erlang had a steep learning curve unless you decide to pay for books (the few of them are excellent, though). A good while ago, I took time to discuss with the author of Learn You a Haskell and I decided to start a version specifically for Erlang with his agreement. The objectives are pretty much the same (lighthearted tutorials for beginners, free of charge) and so is the concept. I've written a few chapters already (Introduction, how to install, basic shell commands, some of the basic data types) and although the text is still very short, I've got a website up and running. I'm looking for a few people with different levels of knowledge of Erlang in order to review the first chapters. I'm looking to validate: a. the accessibility of information; b. if the information is right; c. gather comments and criticism. I find it important to do a first review early in the writing in order to fix flaws or improve on parts of the writing before having to change too much stuff. If you feel like giving me a helping hand in making the first chapters as good as possible through your reviews, just send me an email back. I'll forward the URL and user/password needed to see the info (I'm protecting everything until reviewed, in order not to broadcast false or inappropriate information). Of course, once the first chapters are pushed in production, I'll keep writing and might ask for some more help, if you don't mind the occasional email. Thanks in advance. From steven.charles.davis@REDACTED Tue Jul 28 19:44:06 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 28 Jul 2009 12:44:06 -0500 Subject: [erlang-questions] Re: Misultin v0.1 released In-Reply-To: References: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> Message-ID: <4A6F38E6.30102@gmail.com> Hi Roberto, Yes, it's def true that many/most mochiweb modules are unrelated utility libraries. It would help perhaps (and I'm curious) if you were describe more explicitly what differences in the approach you use to the socket handling are relevant to the performance improvement. Thanks and regards, Steve Roberto Ostinelli wrote: > misultin speed compared with mochiweb comes from the different approach > to the socket handling, and that alone. as you can see, the benchmark > used is very simple, it just echoes a variable passed in GET, and the > code to do so is very similar for both libraries. From sverker@REDACTED Tue Jul 28 19:44:11 2009 From: sverker@REDACTED (Sverker Eriksson) Date: Tue, 28 Jul 2009 19:44:11 +0200 Subject: [erlang-questions] Mystery of good term gone bad In-Reply-To: <6c2563b20907280906y7b918203g9b65e18b16eb627b@mail.gmail.com> References: <6c2563b20907280854t355b13b5i28eff78c8ea0b9d3@mail.gmail.com> <6c2563b20907280906y7b918203g9b65e18b16eb627b@mail.gmail.com> Message-ID: <4A6F38EB.3020908@erix.ericsson.se> Edwin Fine wrote: > Can anyone tell me why the binary to term conversion shown below is "good" > on systems 1, 2, and 3, and bad in system 4, when the binaries are identical > across all of them? Surely R13B01 should be backward-compatible at the > binary level? > > Systems 1 & 2: R13B > System 3: R12B-4 > System 4: R13B01 > : > <<131,104,4,97,6,103,78,131,0,20,119,99,116,112,103,119, > 95,114,101,108,64,108,111,99,97,108,104,111,115,116,0,0, > 0,12,0,0,0,0,2,78,5,0,0,78,110,0,18,103,108,111,98,97, > 108,95,110,97,109,101,95,115,101,114,118,101,114,131, > 104,2,78,248,0,9,36,103,101,110,95,99,97,115,116,104,4, > 78,65,0,12,105,110,105,116,95,99,111,110,110,101,99,116, > 104,2,97,5,104,3,98,0,0,4,224,98,0,12,35,221,98,0,14,79, > 143,67,131,104,4,78,76,0,6,108,111,99,107,101,114,78,77, > 0,15,110,111,95,108,111,110,103,101,114,95,97,95,112, > 105,100,106,103,67,131,0,0,0,13,0,0,0,0,2>>. > Where did you get that binary from? It looks like the *internal* format used by the distribution when communicating terms between nodes. This format was changed in R13B01. You could maybe argue that the old versions should give badarg for this binary. /Sverker, Erlang/OTP From fritchie@REDACTED Tue Jul 28 20:14:35 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 28 Jul 2009 13:14:35 -0500 Subject: [erlang-questions] gen_event:add_sup_handler... correct usage? In-Reply-To: Message of "Tue, 28 Jul 2009 03:18:08 PDT." <4c7edb5c-c29a-48e9-8c66-b274cec683af@w6g2000yqw.googlegroups.com> Message-ID: <28996.1248804875@snookles.snookles.com> Steve Davis wrote: sd> I have boot-time logging for a server that needs to start before sd> anything really happens, i.e. **before the main supervisor is kicked sd> off by application:start(my_server)**. Looking at your code snippet, I think I agree that's what'll happen. Mostly. sd> Despite this requirement, I still need the error_logger process to sd> supervise my gen_event handler. Perhaps it'll be helpful to use the Erlang flavor of the journalist's rule, "Follow the money." Instead, it's "follow the process structure". In such pursuits, the "appmon" GUI app is an invaluable visualization tool. The WebAppMon tool within the "webtool" app is a close second.(*) Adding io:format() calls to my_server_app:start/2 and :stop/1 and all those other callback also helps figure out what's running where, e.g. start(Type, Args) -> io:format("~s: start() in pid ~p\n", [?MODULE, self()]), ... The error_logger process(**) doesn't appear (in R13B01) to be part of the kernel's supervisor hierarchy. Instead it claims to be spawned directly by the <0.1.0> process, very very early in the VM's startup phase. Try process_info(whereis(error_logger)). Any callback you register using gen_event:add_sup_handler(error_logger, ...) will be executed by the error_logger process. If one of your gen_event callbacks throws an exception of some sort, the gen_event framework will silently remove your callback module from its list ... so your use of gen_event:add_sup_handler/3 is a good idea. If your module has a bug, you'll be informed about it... ... except that the link that's created between gen_event:add_sup_handler/3's caller and the gen_event server will be the process that's executing my_server_sup:start(). You may not want that. Add an intentional bug to my_logger_h, trigger the bug, and see what other process(es) dies as a result.(***) sd> ..while all of this "works", I have concerns about the OTP-ness of sd> doing this, and also a nagging feeling that the fact that sd> application process is essentially the "supervisor" that's passed to sd> error_logger, not the supervisor process itself. You probably don't want to use the gen_event:which_handlers() method of deciding whether or not to call supervisor:start_link(). If I'm not mistaken, this sequence of commands: application:start(my_server_app). application:stop(my_server_app). application:start(my_server_app). ... won't start your app's supervisor the second time around. Stopping your app won't automagically remove the my_logger_h callback module from the error_logger server. (Unless you have removal code explicitly in my_server_app:stop/1.) As for "... 'supervisor' that's passed to error_logger...", hopefully it's clear that your nagging feeling doesn't make sense. Or, at least it doesn't to me, sorry. :-) -Scott (*) In OTP's HTML documentation, see the "Tool Applications" section, "webtool" app, "User's Guide" doc. (**) In OTP's HTML documentation, see "Basic Applications" section, "kernel" app, "error_logger" doc. (***) If you don't have the OTP SASL app running, you probably should. Otherwise, many errors inside the VM are invisible. Use application:start(sasl) or start the sasl app in your .boot scheme. HA HA HA! I followed my own advice and tried it: added a exit(bogus) call to my error handler's handle_event/2 callback. The supervisor silently consumes the {'EXIT', ...} message. Oops. Now *I* need to do some refactoring work! HA HA part 2. My custom error logger is critical. If there's a callback bug, I need to know about it ASAP. gen_event will silently drop errors, so I should use gen_event:add_sup_handler/3 to let someone in the world know about the error. Now I discover that the process death that I was expecting in such a doomsday situation ... doesn't happen. Cool! From diginux@REDACTED Tue Jul 28 20:59:28 2009 From: diginux@REDACTED (Jordan Wilberding) Date: Tue, 28 Jul 2009 13:59:28 -0500 Subject: [erlang-questions] wx tutorial In-Reply-To: <4A6ECB7F.5060201@erlang-consulting.com> References: <4A6ECB7F.5060201@erlang-consulting.com> Message-ID: <5756037b0907281159l57a53fe9tae9e4c9b68bdf59a@mail.gmail.com> I have used wx extensively in C++ before, and am now starting to use it with Erlang. I will be giving a talk on wx and and Erlang this weekend for our local Linux User Group. I will try to generate a tutorial from it. Thanks! Jordan Wilberding On Tue, Jul 28, 2009 at 4:57 AM, Mazen Harake < mazen.harake@REDACTED> wrote: > Hey, > > Just curious if there are (by now) experts using the wxwidget binding in > Erlang... and if so, how about writing a small tutorial on how to use it > (even if there are examples a tutorial is always nice). Perhaps on > www.trapexit.org (or whatever)? > > > Would be nice to have one, mapping the tutorials from C++ API to Erlang API > was a bit more time consuming then I thought... > > Just a thought :) > > > > /Mazen > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From erlang-questions_efine@REDACTED Tue Jul 28 21:14:32 2009 From: erlang-questions_efine@REDACTED (Edwin Fine) Date: Tue, 28 Jul 2009 15:14:32 -0400 Subject: [erlang-questions] Mystery of good term gone bad In-Reply-To: <4A6F38EB.3020908@erix.ericsson.se> References: <6c2563b20907280854t355b13b5i28eff78c8ea0b9d3@mail.gmail.com> <6c2563b20907280906y7b918203g9b65e18b16eb627b@mail.gmail.com> <4A6F38EB.3020908@erix.ericsson.se> Message-ID: <6c2563b20907281214p60f916e3wd9ee29d78c7c041a@mail.gmail.com> Sverker, It is from error log messages I have been getting from communications between two Erlang nodes on the same physical server. One node is an application of ours, the other is an ejabberd server. The communications between our node and the ejabberd server have gone bad. I can ping and remsh to ejabberd server no problem, but I cannot ping or remsh our application node from anywhere without getting the error messages below. Restarting everything doesn't help. I sent an earlier post with the details but it somehow didn't make it to the mailing list. They look like this: =ERROR REPORT==== 28-Jul-2009::11:40:13 === Got corrupted message on channel 4133 =ERROR REPORT==== 28-Jul-2009::11:40:13 === Got invalid data on distribution channel, offending packet is: <<112,131,104,4,97,6,103,78,131,0,20,119,99,116,112,103, 119,95,114,101,108,64,108,111,99,97,108,104,111,115,116, 0,0,0,12,0,0,0,0,2,78,5,0,0,78,110,0,18,103,108,111,98, 97,108,95,110,97,109,101,95,115,101,114,118,101,114,131, 104,2,78,248,0,9,36,103,101,110,95,99,97,115,116,104,4, 78,65,0,12,105,110,105,116,95,99,111,110,110,101,99,116, 104,2,97,5,104,3,98,0,0,4,224,98,0,12,35,221,98,0,14,69, 8,67,131,104,4,78,76,0,6,108,111,99,107,101,114,78,77,0, 15,110,111,95,108,111,110,103,101,114,95,97,95,112,105, 100,106,103,67,131,0,0,0,13,0,0,0,0,2>> Regards, Ed On Tue, Jul 28, 2009 at 1:44 PM, Sverker Eriksson wrote: > Edwin Fine wrote: > >> Can anyone tell me why the binary to term conversion shown below is "good" >> on systems 1, 2, and 3, and bad in system 4, when the binaries are >> identical >> across all of them? Surely R13B01 should be backward-compatible at the >> binary level? >> >> Systems 1 & 2: R13B >> System 3: R12B-4 >> System 4: R13B01 >> >> > : > >> <<131,104,4,97,6,103,78,131,0,20,119,99,116,112,103,119, >> 95,114,101,108,64,108,111,99,97,108,104,111,115,116,0,0, >> 0,12,0,0,0,0,2,78,5,0,0,78,110,0,18,103,108,111,98,97, >> 108,95,110,97,109,101,95,115,101,114,118,101,114,131, >> 104,2,78,248,0,9,36,103,101,110,95,99,97,115,116,104,4, >> 78,65,0,12,105,110,105,116,95,99,111,110,110,101,99,116, >> 104,2,97,5,104,3,98,0,0,4,224,98,0,12,35,221,98,0,14,79, >> 143,67,131,104,4,78,76,0,6,108,111,99,107,101,114,78,77, >> 0,15,110,111,95,108,111,110,103,101,114,95,97,95,112, >> 105,100,106,103,67,131,0,0,0,13,0,0,0,0,2>>. >> >> > Where did you get that binary from? It looks like the *internal* format > used by the distribution when communicating terms between nodes. This format > was changed in R13B01. > You could maybe argue that the old versions should give badarg for this > binary. > > /Sverker, Erlang/OTP > > > From roberto@REDACTED Tue Jul 28 21:15:58 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 28 Jul 2009 21:15:58 +0200 Subject: [erlang-questions] Re: Misultin v0.1 released In-Reply-To: <4A6F38E6.30102@gmail.com> References: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> <4A6F38E6.30102@gmail.com> Message-ID: > It would help perhaps (and I'm curious) if you were describe more > explicitly what differences in the approach you use to the socket handling > are relevant to the performance improvement. > i've seen that mochiweb uses too {packet, http}, which puts the socket into http mode. as per http://www.trapexit.org/A_fast_web_server_demonstrating_some_undocumented_Erlang_featuresthis is an undocumented erlang feature which is "not supported by Ericsson [but is] used in comercially shipping system". the socket waits for an http request line [request], then switches to 'header' mode until the headers are done {ok, http_eoh} [headers] before switching to reading raw body data [body]. therefore, i am unsure which parts of the code and socket handling are actually providing a better overall performance. to know, it would mean benchmarking the code on its main parts. cheers, r. From ruslan@REDACTED Wed Jul 29 01:00:16 2009 From: ruslan@REDACTED (Ruslan) Date: Tue, 28 Jul 2009 16:00:16 -0700 (PDT) Subject: erlang-amf In-Reply-To: <3F08C615-ABB7-4AFD-B264-ADE2EA308635@berabera.info> References: <6211282E-2D62-4A6C-9758-E2A13D8D1D9D@babayev.com> <3792a5cf-8649-489b-9d78-36800d12e102@g7g2000prg.googlegroups.com> <5c493e530907271848g1b8a92b6n1f81756f98cfcef@mail.gmail.com> <3F08C615-ABB7-4AFD-B264-ADE2EA308635@berabera.info> Message-ID: <4f501455-14fe-4110-bb5c-2f0cc730421f@y10g2000prg.googlegroups.com> Hi Ngoc, You are more than welcome to fork the project and package it with Emakefile and Makefile. Autoconf/automake allows you to do much more. For one you can specify the --prefix=LOCATION at configure time and have it installed there when you do make install, or do make dist etc, etc. Take a look at http://github.com/mujaheed/erlang-example/tree/master It has support for EDoc and Eunit. There's also framewerk template for Erlang http://code.google.com/p/fwtemplates/wiki/FwTemplateErlangWalkthrough Romain, thanks for comments. Cheers, Ruslan On Jul 27, 7:11?pm, Romain Lenglet wrote: > Hi Ngoc, > What files are you looking at? > configure is generated automatically from configure.ac. > I find this configure.ac exemplar. > -- > Romain Lenglet > > On Jul 28, 2009, at 10:48, Ngoc Dao wrote: > > > > > > > Hi. > > > There are only .erl files, no .c files. Why do you need to use so > > complicated configure files to build them? I'm just curious. > > > Ngoc. > > > On Mon, Jul 6, 2009 at 4:03 PM, Ruslan wrote: > > >> Forgot the actual linkhttp://github.com/mujaheed/erlang-amf/tree/master > > >> On Jul 6, 12:01 am, Ruslan Babayev wrote: > >>> I have finally published theErlang AMFlibrary I have been ? > >>> working on > >>> to github. It has full support for Adobe AMF0 and AMF3 specs. > > >>> Enjoy responsibly. > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From ruslan@REDACTED Wed Jul 29 01:12:04 2009 From: ruslan@REDACTED (Ruslan Babayev) Date: Tue, 28 Jul 2009 16:12:04 -0700 Subject: Flex Remoting with Erlang Message-ID: <57A5134B-94C4-4C75-8488-5AA762B48C30@babayev.com> Folks, I thought my work on Flex Remoting with Erlang could be useful to someone and I have put up my HTTP 1.1 server with Flex Remoting on github. I couldn't come up with a cool name for the project and just called it erlang-http. It is modular just like inets and has similar API. I almost the same functionality as inets. The module that handles Flex remoting is http_mod_amf. You can get it from http://github.com/mujaheed/erlang-http/tree/master erlang-http depends on erlang-amf which you can get from http://github.com/mujaheed/erlang-amf/tree/master Examples are at http://github.com/mujaheed/flex_examples/tree/master Drop me a line if you like what you see, or even if you don't :) your feedback is welcome. Cheers, Ruslan From igorrs@REDACTED Wed Jul 29 01:52:29 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Tue, 28 Jul 2009 20:52:29 -0300 Subject: [erlang-questions] Log dumping not thread-safe? In-Reply-To: References: Message-ID: By default, Mnesia dumps the transaction log automatically every 3 minutes (or after each hundred of log writes). What happens if I call mnesia:dump_log() at the same moment Mnesia is doing an automatic dump? I expected the calls to be serialized, so that one dump would only begin after the current dump finishes. But, it doesn't seem to be the case, as I had a Mnesia crash dump caused by one of my calls to mnesia:dump_log/0. Actually, concurrent calls to mnesia:dump_log/0 are not serialized, from what I understand from this: 1> mnesia:start(). ok 2> F = fun(F0) -> mnesia:dump_log(), F0(F0) end. #Fun 3> spawn(fun() -> F(F) end). <0.54.0> 4> spawn(fun() -> F(F) end). <0.1413.1> 5> =ERROR REPORT==== 28-Jul-2009::20:53:26 === Mnesia(nonode@REDACTED): ** ERROR ** (ignoring core) ** FATAL ** mnesia_controller crashed: {{case_clause, true}, [{mnesia_controller, add_worker, 2}, {mnesia_controller, handle_call, 3}, {gen_server, handle_msg, 5}, {proc_lib, init_p_do_apply, 3}]} state: {state, <0.40.0>, true, [], [], {0, nil}, [], [], {0, nil}, <0.1412.1>, [{dump_log, user, {<0.54.0>, #Ref<0.0.0.204553>}}], [], {interval, #Ref<0.0.0.53>}, false} =INFO REPORT==== 28-Jul-2009::20:53:29 === application: mnesia exited: shutdown type: temporary Thanks for replying. Igor. 2009/7/28 Hakan Mattsson : > On Fri, 24 Jul 2009, Igor Ribeiro Sucupira wrote: > >> mnesia:dump_log() is not thread-safe. Am I right? > > What do you mean with thread-safe in this context? > >> In practice, that would mean you can never use this function, as >> Mnesia could be dumping the log at the same moment you called this >> function. Is that correct? > > No. Mnesia will start dumping its transaction log when you invoke this > function. But it does not interfere with ongoing transactions. You can > still use Mnesia as usual while the dump is progressing as a background > job. > > /H?kan > --- > H?kan Mattsson (uabhams) > Erlang/OTP, Ericsson AB From bflatmaj7th@REDACTED Wed Jul 29 01:59:18 2009 From: bflatmaj7th@REDACTED (Richard Andrews) Date: Wed, 29 Jul 2009 09:59:18 +1000 Subject: [erlang-questions] Learn you some Erlang for great good! In-Reply-To: <8b9ee55b0907281003g311d71f9ie6ef1ddb141646ae@mail.gmail.com> References: <8b9ee55b0907281003g311d71f9ie6ef1ddb141646ae@mail.gmail.com> Message-ID: <7702c0610907281659p11f4ae83t30f59fa479f389f2@mail.gmail.com> Can you define the license under which the work will be available so others can make a more informed decision on whether to contribute their time. On Wed, Jul 29, 2009 at 3:03 AM, Fred Hebert (MononcQc) wrote: > Some of the readers of this list may be familiar with > http://learnyouahaskell.com/, an online book/tutorial dedicated to teaching > Haskell in a lighthearted way. > I've always thought Erlang had a steep learning curve unless you decide to > pay for books (the few of them are excellent, though). A good while ago, I > took time to discuss with the author of Learn You a Haskell and I decided to > start a version specifically for Erlang with his agreement. The objectives > are pretty much the same (lighthearted tutorials for beginners, free of > charge) and so is the concept. > I've written a few chapters already (Introduction, how to install, basic > shell commands, some of the basic data types) and although the text is still > very short, I've got a website up and running. > I'm looking for a few people with different levels of knowledge of Erlang in > order to review the first chapters. I'm looking to validate: ?a. the > accessibility of information; b. if the information is right; c. gather > comments and criticism. I find it important to do a first review early in > the writing in order to fix flaws or improve on parts of the writing before > having to change too much stuff. > If you feel like giving me a helping hand in making the first chapters as > good as possible through your reviews, just send me an email back. I'll > forward the URL and user/password needed to see the info (I'm protecting > everything until reviewed, in order not to broadcast false or inappropriate > information). > Of course, once the first chapters are pushed in production, I'll keep > writing and might ask for some more help, if you don't mind the occasional > email. > Thanks in advance. > From mononcqc@REDACTED Wed Jul 29 02:39:42 2009 From: mononcqc@REDACTED (Fred Hebert (MononcQc)) Date: Tue, 28 Jul 2009 20:39:42 -0400 Subject: [erlang-questions] Learn you some Erlang for great good! In-Reply-To: <7702c0610907281659p11f4ae83t30f59fa479f389f2@mail.gmail.com> References: <8b9ee55b0907281003g311d71f9ie6ef1ddb141646ae@mail.gmail.com> <7702c0610907281659p11f4ae83t30f59fa479f389f2@mail.gmail.com> Message-ID: <8b9ee55b0907281739o3fd1a500q31e6c2671d4c61d3@mail.gmail.com> For the text I'm going for the Creative Commons Attribution Non-Commercial No Derivative License (http://creativecommons.org/licenses/by-nc-nd/3.0/) If people bring good arguments in order to allow derivative work, I'm open to switching to the Attribution-Noncommercial-Share Alike Creative Commons license (http://creativecommons.org/licenses/by-nc-sa/3.0/). As for the code in the book itself, it's surely going to end up under the EPL (Erlang Public License). I thought of the MIT license but will probably use the EPL as it's what the whole Erlang distribution uses as far as I know. Does this answer the question? On Tue, Jul 28, 2009 at 7:59 PM, Richard Andrews wrote: > Can you define the license under which the work will be available so > others can make a more informed decision on whether to contribute > their time. > > On Wed, Jul 29, 2009 at 3:03 AM, Fred Hebert > (MononcQc) wrote: > > Some of the readers of this list may be familiar with > > http://learnyouahaskell.com/, an online book/tutorial dedicated to > teaching > > Haskell in a lighthearted way. > > I've always thought Erlang had a steep learning curve unless you decide > to > > pay for books (the few of them are excellent, though). A good while ago, > I > > took time to discuss with the author of Learn You a Haskell and I decided > to > > start a version specifically for Erlang with his agreement. The > objectives > > are pretty much the same (lighthearted tutorials for beginners, free of > > charge) and so is the concept. > > I've written a few chapters already (Introduction, how to install, basic > > shell commands, some of the basic data types) and although the text is > still > > very short, I've got a website up and running. > > I'm looking for a few people with different levels of knowledge of Erlang > in > > order to review the first chapters. I'm looking to validate: a. the > > accessibility of information; b. if the information is right; c. gather > > comments and criticism. I find it important to do a first review early in > > the writing in order to fix flaws or improve on parts of the writing > before > > having to change too much stuff. > > If you feel like giving me a helping hand in making the first chapters as > > good as possible through your reviews, just send me an email back. I'll > > forward the URL and user/password needed to see the info (I'm protecting > > everything until reviewed, in order not to broadcast false or > inappropriate > > information). > > Of course, once the first chapters are pushed in production, I'll keep > > writing and might ask for some more help, if you don't mind the > occasional > > email. > > Thanks in advance. > > > From ruslan@REDACTED Wed Jul 29 03:01:27 2009 From: ruslan@REDACTED (Ruslan Babayev) Date: Tue, 28 Jul 2009 18:01:27 -0700 Subject: Flex Remoting with Erlang In-Reply-To: <57A5134B-94C4-4C75-8488-5AA762B48C30@babayev.com> References: <57A5134B-94C4-4C75-8488-5AA762B48C30@babayev.com> Message-ID: <2838BF7B-B0ED-4254-AE35-4E1A033B7823@babayev.com> Examples have moved to http://github.com/mujaheed/erlang-flex-remoting/tree/master On Jul 28, 2009, at 4:12 PM, Ruslan Babayev wrote: > Folks, > > I thought my work on Flex Remoting with Erlang could be useful to > someone and > I have put up my HTTP 1.1 server with Flex Remoting on github. > > I couldn't come up with a cool name for the project and just called > it erlang-http. It is modular just like inets and has similar API. I > almost the same functionality as inets. The module that handles Flex > remoting is http_mod_amf. > > You can get it from http://github.com/mujaheed/erlang-http/tree/ > master > > erlang-http depends on erlang-amf which you can get from http://github.com/mujaheed/erlang-amf/tree/master > > Examples are at http://github.com/mujaheed/flex_examples/tree/master > > Drop me a line if you like what you see, or even if you don't :) > your feedback is welcome. > > Cheers, > Ruslan > From yoursurrogategod@REDACTED Wed Jul 29 04:55:50 2009 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Tue, 28 Jul 2009 19:55:50 -0700 (PDT) Subject: Nested for-loops, there has to be a better way to do this Message-ID: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> I was playing around with the idea of implementing a nested for-loop like construct (I will concede, they are evil, but at rare times, necessary.) In non-functional programming languages, they are trivial to do. However, this has proved to be one of those simple things that are a pain in the neck for me. The idea was to be able to pass in a list [5, 5, 10] (for example) and go through it all like 3 dimensional array. The below illustration is far from elegant (it's almost 23:00 where I live and need to go to work tomorrow early, so no time to come up with something more sane :) ), but is there as an inquisitive exercise. Now, in my opinion, this implementation sucks (first time I've ever gave such a problem a shot in Erlang.) Is there a better way to do this? Anyone would like to demonstrate it? Note: I've tested this code with the lists [2, 4] and [2, 4, 5]. Compiled with: Erlang (SMP,ASYNC_THREADS,HIPE) (BEAM) emulator version 5.6.5 My code: ====================================== -module(loopWrapper). -export([runLoop/1]). runLoop(List) -> [FirstVal | RestList] = List, NextVal = FirstVal - 1, if (length(RestList) > 0) and (FirstVal > 0) -> io:format("~B~n~n", [FirstVal]), runLoop(RestList), io:format("~n~n"), runLoop([NextVal | RestList]); FirstVal > 0 -> io:format("~B", [FirstVal]), runLoop([NextVal]); true -> null end. From yoursurrogategod@REDACTED Wed Jul 29 04:59:23 2009 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Tue, 28 Jul 2009 19:59:23 -0700 (PDT) Subject: Learn you some Erlang for great good! In-Reply-To: <8b9ee55b0907281003g311d71f9ie6ef1ddb141646ae@mail.gmail.com> References: <8b9ee55b0907281003g311d71f9ie6ef1ddb141646ae@mail.gmail.com> Message-ID: <97f48932-496e-4bdf-8c96-9eef37d588d0@j9g2000prh.googlegroups.com> Just a thought: As long as you clearly state on your website that the below info should not be considered reliable until said so otherwise, there shouldn't be a problem with having everyone on the net look at it and give you feedback. On Jul 28, 1:03?pm, "Fred Hebert (MononcQc)" wrote: > Some of the readers of this list may be familiar withhttp://learnyouahaskell.com/, an online book/tutorial dedicated to teaching > Haskell in a lighthearted way. > I've always thought Erlang had a steep learning curve unless you decide to > pay for books (the few of them are excellent, though). A good while ago, I > took time to discuss with the author of Learn You a Haskell and I decided to > start a version specifically for Erlang with his agreement. The objectives > are pretty much the same (lighthearted tutorials for beginners, free of > charge) and so is the concept. > I've written a few chapters already (Introduction, how to install, basic > shell commands, some of the basic data types) and although the text is still > very short, I've got a website up and running. > I'm looking for a few people with different levels of knowledge of Erlang in > order to review the first chapters. I'm looking to validate: ?a. the > accessibility of information; b. if the information is right; c. gather > comments and criticism. I find it important to do a first review early in > the writing in order to fix flaws or improve on parts of the writing before > having to change too much stuff. > If you feel like giving me a helping hand in making the first chapters as > good as possible through your reviews, just send me an email back. I'll > forward the URL and user/password needed to see the info (I'm protecting > everything until reviewed, in order not to broadcast false or inappropriate > information). > Of course, once the first chapters are pushed in production, I'll keep > writing and might ask for some more help, if you don't mind the occasional > email. > Thanks in advance. From yoursurrogategod@REDACTED Wed Jul 29 05:06:32 2009 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Tue, 28 Jul 2009 20:06:32 -0700 (PDT) Subject: Flex Remoting with Erlang In-Reply-To: <2838BF7B-B0ED-4254-AE35-4E1A033B7823@babayev.com> References: <57A5134B-94C4-4C75-8488-5AA762B48C30@babayev.com> <2838BF7B-B0ED-4254-AE35-4E1A033B7823@babayev.com> Message-ID: <2457db8e-5be6-404e-ad25-b93c1622dbbf@y4g2000prf.googlegroups.com> Just a thought, how does the name "Flexlang" sound for your project? On Jul 28, 9:01?pm, Ruslan Babayev wrote: > Examples have moved tohttp://github.com/mujaheed/erlang-flex-remoting/tree/master > > On Jul 28, 2009, at 4:12 PM, Ruslan Babayev wrote: > > > > > > > Folks, > > > I thought my work on Flex Remoting with Erlang could be useful to ? > > someone and > > I have put up my HTTP 1.1 server with Flex Remoting on github. > > > I couldn't come up with a cool name for the project and just called ? > > it erlang-http. It is modular just like inets and has similar API. I ? > > almost the same functionality as inets. The module that handles Flex ? > > remoting is http_mod_amf. > > > You can get it from ?http://github.com/mujaheed/erlang-http/tree/ > > master > > > erlang-http depends on erlang-amf which you can get fromhttp://github.com/mujaheed/erlang-amf/tree/master > > > Examples are athttp://github.com/mujaheed/flex_examples/tree/master > > > Drop me a line if you like what you see, or even if you don't :) ? > > your feedback is welcome. > > > Cheers, > > Ruslan > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From leap@REDACTED Wed Jul 29 05:13:52 2009 From: leap@REDACTED (Michael Turner) Date: Wed, 29 Jul 2009 03:13:52 +0000 Subject: Misultin v0.1 released (better wiki link + some OT questions) In-Reply-To: Message-ID: On 7/28/2009, "Roberto Ostinelli" wrote: >i've seen that mochiweb uses too {packet, http}, which puts the socket into http mode. as per http://www.trapexit.org/A_fast_web_server_demonstrating_some_undocumented_Erlang_featuresthis A better URL for that http://www.trapexit.org/A_fast_web_server_demonstrating_some_undocumented_Erlang_features A tiny one: http://tinyurl.com/cm9edz A couple questions, admittedly off-topic: (1) The above seems to be a wiki page, and I'd like to fix a few typos in it, and one apparently must sign up to edit (good idea), HOWEVER -- despite the promise here http://www.trapexit.org/index.php?title=Special:Userlogin i.e., "If you don't have an account create one here", I can't seem to create a user account for myself. (2) When I did a "reply" to this mailing list thread, I got no subject line, and no erlang-questions@REDACTED return address. I can see how the latter might be intentional, but what's up with the former? That seems like a bug. I don't want to copy-paste the subject line every time. Is this how it works for everybody, or is it something with my webmail interface? -michael turner From ok@REDACTED Wed Jul 29 05:32:50 2009 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 29 Jul 2009 15:32:50 +1200 Subject: [erlang-questions] Nested for-loops, there has to be a better way to do this In-Reply-To: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> Message-ID: On Jul 29, 2009, at 2:55 PM, Yves S. Garret wrote: > I was playing around with the idea of implementing a nested for- > loop... a > list [5, 5, 10] (for example) and go through it all like 3 dimensional > array. I don't understand what it is you want to do. Will something like D1 = lists:seq(1, 5), D2 = lists:seq(1, 5), D3 = lists:seq(1, 10), % do this for effect, not result [ effect(I1, I2, I3) || I1 <- D1, I2 <- D2, I3 <- D3 ] do what you want? I've been playing with a syntax that allows, amongst other special cases, ( effect(I1, I2, I3) || I1 <- D1, I2 <- D2, I3 <- D3 ) However, it's not really needed; all that's needed is for the Erlang compiler to notice that the result of a list comprehension is not being used, and just not bother constructing the result. From rvirding@REDACTED Wed Jul 29 05:37:49 2009 From: rvirding@REDACTED (Robert Virding) Date: Wed, 29 Jul 2009 05:37:49 +0200 Subject: [erlang-questions] Lost messages In-Reply-To: References: <401d3ba30907270539m601fafa5w96e0a8de31960227@mail.gmail.com> Message-ID: <3dbc6d1c0907282037q268b041ei6d2757b31035ab9b@mail.gmail.com> The problem of catching messages which go to non-existent processes is difficult and I don't believe there is a general solution which will handle all cases. Off-hand I can think the following cases: - The receiver process is on another node and the connection has gone down. - The receiver has died before the message is sent. - The receiver dies while the message is on it's way. - The receiver dies after the message has arrived, but before it can be received. - The receiver dies after the message has been received but before it can reply. - The receiver is nasty and just doesn't bother to reply. Etc... I believe that you have to look at your app and let it decide how best to detect and handle messages to which there is no reply. For example in the io module the calls use monitor_process to detect if a process has died before a reply message is sent, but this will not handle the case where the receiver refuses to reply, while in the old telephony protocols there were timeouts - if a reply didn't arrive within the timeout then it was an error whatever the reason. Both these methods work well for some cases but can be wrong for others. This is one reason we never added a general RPC function. The simple send/receive looks great but is not enough for serious applications, there you need to roll your own case which does what you need. Joe Armstrong wrote a good blog/posting (?) on that where he discussed the problem and showed some of all the options for writing a "correct" RPC. I don't have the link to it. All this was not to say that the problem in unsolvable, just that you have to work out what you need for your app. In your case maybe using monitor_process might work. Robert 2009/7/28 George Xu > Yes?I already do the link and clean ETS table work. But no one can solve > the race condition I think. > > So, in very rare case, I lost the message ('EXIT' is arrived after the > message send out). > > I think this is not a "erLang way". It shall occur an error and let the > sending process exit. > > Because that the message is important and the router which collect > messages and send out can not be blocked by a dead / exit process, send a > message back is not an acceptale solution. > > Does anyone have solutions for this? > > ?????????????????????????? > ?? George Xu > > Mobile Internet Service Lab, Lonovo Corp. Research China > Tel:86-10-58866181 > E-mail:xuxiang1@REDACTED > No.6 Shang Di West Road, Haidian District, > Beijing, 100085 > P.R.China > From mononcqc@REDACTED Wed Jul 29 05:44:11 2009 From: mononcqc@REDACTED (Fred Hebert (MononcQc)) Date: Tue, 28 Jul 2009 23:44:11 -0400 Subject: [erlang-questions] Re: Learn you some Erlang for great good! In-Reply-To: <97f48932-496e-4bdf-8c96-9eef37d588d0@j9g2000prh.googlegroups.com> References: <8b9ee55b0907281003g311d71f9ie6ef1ddb141646ae@mail.gmail.com> <97f48932-496e-4bdf-8c96-9eef37d588d0@j9g2000prh.googlegroups.com> Message-ID: <8b9ee55b0907282044w68c845b6k36acd87963e79101@mail.gmail.com> On Tue, Jul 28, 2009 at 10:59 PM, Yves S. Garret wrote: > Just a thought: > As long as you clearly state on your website that the below info > should not be considered reliable until said so otherwise, there > shouldn't be a problem with having everyone on the net look at it and > give you feedback. > > On Jul 28, 1:03 pm, "Fred Hebert (MononcQc)" > wrote: > > Some of the readers of this list may be familiar withhttp:// > learnyouahaskell.com/, an online book/tutorial dedicated to teaching > > Haskell in a lighthearted way. > > I've always thought Erlang had a steep learning curve unless you decide > to > > pay for books (the few of them are excellent, though). A good while ago, > I > > took time to discuss with the author of Learn You a Haskell and I decided > to > > start a version specifically for Erlang with his agreement. The > objectives > > are pretty much the same (lighthearted tutorials for beginners, free of > > charge) and so is the concept. > > I've written a few chapters already (Introduction, how to install, basic > > shell commands, some of the basic data types) and although the text is > still > > very short, I've got a website up and running. > > I'm looking for a few people with different levels of knowledge of Erlang > in > > order to review the first chapters. I'm looking to validate: a. the > > accessibility of information; b. if the information is right; c. gather > > comments and criticism. I find it important to do a first review early in > > the writing in order to fix flaws or improve on parts of the writing > before > > having to change too much stuff. > > If you feel like giving me a helping hand in making the first chapters as > > good as possible through your reviews, just send me an email back. I'll > > forward the URL and user/password needed to see the info (I'm protecting > > everything until reviewed, in order not to broadcast false or > inappropriate > > information). > > Of course, once the first chapters are pushed in production, I'll keep > > writing and might ask for some more help, if you don't mind the > occasional > > email. > > Thanks in advance. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > A similar notice is actually written in the dev version of the site (which I plan on making widely public instead of password protected after the first few chapters are published). The main page will acknowledge the possibility of having mistakes and asking people to tell me if they find any for the production version. However, I intend for the first chapters released to be spotless; If the tutorial is linked to other sites while it contains broken code, spelling or grammar mistakes and/or plainly wrong information, it is more than likely to gain a bad reputation that will never go away. I'm aiming for a good first impression to start on solid bases, then I'm going to relax and make things more open afterwards. From kaiduanx@REDACTED Wed Jul 29 05:46:58 2009 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Tue, 28 Jul 2009 23:46:58 -0400 Subject: Add micro-second to Error Logger Message-ID: Hi, Is there plan for OTP team to add micro-second to the error logger? =ERROR REPORT==== 28-Jul-2009::23:40:16 === =ERROR REPORT==== 28-Jul-2009::23:40:16.538 === In many applications, having the micro-second will help debugging if error logger is used to write debug information. Thanks, kaiduan From mononcqc@REDACTED Wed Jul 29 05:55:27 2009 From: mononcqc@REDACTED (Fred Hebert (MononcQc)) Date: Tue, 28 Jul 2009 23:55:27 -0400 Subject: [erlang-questions] Lost messages In-Reply-To: <3dbc6d1c0907282037q268b041ei6d2757b31035ab9b@mail.gmail.com> References: <401d3ba30907270539m601fafa5w96e0a8de31960227@mail.gmail.com> <3dbc6d1c0907282037q268b041ei6d2757b31035ab9b@mail.gmail.com> Message-ID: <8b9ee55b0907282055w11a8ad8g7f2fe3e5bb09f8a3@mail.gmail.com> On Tue, Jul 28, 2009 at 11:37 PM, Robert Virding wrote: > This is one reason we never added a general RPC function. The simple > send/receive looks great but is not enough for serious applications, there > you need to roll your own case which does what you need. Joe Armstrong > wrote > a good blog/posting (?) on that where he discussed the problem and showed > some of all the options for writing a "correct" RPC. I don't have the link > to it. > I believe the blog post is "The road we didn't go down": http://armstrongonsoftware.blogspot.com/2008/05/road-we-didnt-go-down.html From rvirding@REDACTED Wed Jul 29 06:01:20 2009 From: rvirding@REDACTED (Robert Virding) Date: Wed, 29 Jul 2009 06:01:20 +0200 Subject: [erlang-questions] Lost messages In-Reply-To: <8b9ee55b0907282055w11a8ad8g7f2fe3e5bb09f8a3@mail.gmail.com> References: <401d3ba30907270539m601fafa5w96e0a8de31960227@mail.gmail.com> <3dbc6d1c0907282037q268b041ei6d2757b31035ab9b@mail.gmail.com> <8b9ee55b0907282055w11a8ad8g7f2fe3e5bb09f8a3@mail.gmail.com> Message-ID: <3dbc6d1c0907282101i306691f3kbd46e38971eb8acc@mail.gmail.com> That's the one. 2009/7/29 Fred Hebert (MononcQc) > > On Tue, Jul 28, 2009 at 11:37 PM, Robert Virding wrote: > >> This is one reason we never added a general RPC function. The simple >> send/receive looks great but is not enough for serious applications, there >> you need to roll your own case which does what you need. Joe Armstrong >> wrote >> a good blog/posting (?) on that where he discussed the problem and showed >> some of all the options for writing a "correct" RPC. I don't have the link >> to it. >> > > I believe the blog post is "The road we didn't go down": > http://armstrongonsoftware.blogspot.com/2008/05/road-we-didnt-go-down.html > From carlmcdade@REDACTED Wed Jul 29 06:19:29 2009 From: carlmcdade@REDACTED (Carl McDade) Date: Wed, 29 Jul 2009 06:19:29 +0200 Subject: [Erlyaws-list] YAWS with PHP on Windows XP problem In-Reply-To: References: <4A61E732.8060905@pcinside.pl> <4A644A29.7070409@pcinside.pl> <65b2728e0907200636h4eceef53g5969a86fd08af8d9@mail.gmail.com> <4A6477BD.1030603@pcinside.pl> <65b2728e0907200747x36578e22q5bf1244946ff06eb@mail.gmail.com> <4A6491DC.3050407@pcinside.pl> <4A64F3E1.5000401@tail-f.com> <4A64F476.3010203@pcinside.pl> <4A64F7D3.80508@tail-f.com> Message-ID: Hi, Confirming: I just tried this on Vista,PHP-CGI.exe and got the same result, which is a blank page and no error outside of the path error in the interactive window. /Carl On Tue, Jul 21, 2009 at 3:14 AM, andrew mmc wrote: > Forgive me if I'm stating the obvious now, but I think the first problem was > because the file extension was cgi, it tried to execute the file, which it > couldn't do because it is on windows.? When you changed the extension to > php, yaws knew to pass through the file as a parameter to the php executable > - the correct behaviour. > > I think the problem is now with php.? PHP has some default behaviour set for > security reasons.? e.g. on *nix you have to explicitly compile it with the > option to run as cgi.? It also by default will not allow itself to execute > anything outside of its docroot, so perhaps the test file isn't in or below > the docroot set in php.ini? > > Otherwise, perhaps some of the other cgi specific options there could have > an effect? > > > 2009/7/21 Claes Wikstrom >> >> > Yes, this path exists. And it's correct - YAWS throws an error with >> > incorrect path. >> > >> > ("serwer" it's a Polish pronounceation of server, sorry for that - I've >> > ?to change it but later ;)) >> > >> >> Nice :-) >> >> > Still, problem is unresolved... >> >> I'll take a look at it - but not now. As I said before, this just >> needs to be debugged. I.e yaws_cgi.erl needs to be debug modified >> so that we >> >> 1. see what get's open_ported() ed >> 2. figure out why that doesn't work. >> >> >> >> /klacke >> >> >> ------------------------------------------------------------------------------ >> Enter the BlackBerry Developer Challenge >> This is your chance to win up to $100,000 in prizes! For a limited time, >> vendors submitting new applications to BlackBerry App World(TM) will have >> the opportunity to enter the BlackBerry Developer Challenge. See full >> prize >> details at: http://p.sf.net/sfu/Challenge >> _______________________________________________ >> Erlyaws-list mailing list >> Erlyaws-list@REDACTED >> https://lists.sourceforge.net/lists/listinfo/erlyaws-list > > > ------------------------------------------------------------------------------ > Enter the BlackBerry Developer Challenge > This is your chance to win up to $100,000 in prizes! For a limited time, > vendors submitting new applications to BlackBerry App World(TM) will have > the opportunity to enter the BlackBerry Developer Challenge. See full prize > details at: http://p.sf.net/sfu/Challenge > _______________________________________________ > Erlyaws-list mailing list > Erlyaws-list@REDACTED > https://lists.sourceforge.net/lists/listinfo/erlyaws-list > > -- Carl McDade Lead Developer ____________________ FireOrb CMS www.fireorb.info From carlmcdade@REDACTED Wed Jul 29 06:49:02 2009 From: carlmcdade@REDACTED (Carl McDade) Date: Wed, 29 Jul 2009 06:49:02 +0200 Subject: [Erlyaws-list] YAWS with PHP on Windows XP problem In-Reply-To: References: <4A61E732.8060905@pcinside.pl> <65b2728e0907200636h4eceef53g5969a86fd08af8d9@mail.gmail.com> <4A6477BD.1030603@pcinside.pl> <65b2728e0907200747x36578e22q5bf1244946ff06eb@mail.gmail.com> <4A6491DC.3050407@pcinside.pl> <4A64F3E1.5000401@tail-f.com> <4A64F476.3010203@pcinside.pl> <4A64F7D3.80508@tail-f.com> Message-ID: Just to make sure I rolled back from 1.84 to 1.82 and received the same result. On Wed, Jul 29, 2009 at 6:19 AM, Carl McDade wrote: > Hi, > > Confirming: I just tried this on Vista,PHP-CGI.exe and got the same > result, which is a blank page and no error outside of the path error > in the interactive window. > > /Carl > > > > On Tue, Jul 21, 2009 at 3:14 AM, andrew mmc wrote: >> Forgive me if I'm stating the obvious now, but I think the first problem was >> because the file extension was cgi, it tried to execute the file, which it >> couldn't do because it is on windows.? When you changed the extension to >> php, yaws knew to pass through the file as a parameter to the php executable >> - the correct behaviour. >> >> I think the problem is now with php.? PHP has some default behaviour set for >> security reasons.? e.g. on *nix you have to explicitly compile it with the >> option to run as cgi.? It also by default will not allow itself to execute >> anything outside of its docroot, so perhaps the test file isn't in or below >> the docroot set in php.ini? >> >> Otherwise, perhaps some of the other cgi specific options there could have >> an effect? >> >> >> 2009/7/21 Claes Wikstrom >>> >>> > Yes, this path exists. And it's correct - YAWS throws an error with >>> > incorrect path. >>> > >>> > ("serwer" it's a Polish pronounceation of server, sorry for that - I've >>> > ?to change it but later ;)) >>> > >>> >>> Nice :-) >>> >>> > Still, problem is unresolved... >>> >>> I'll take a look at it - but not now. As I said before, this just >>> needs to be debugged. I.e yaws_cgi.erl needs to be debug modified >>> so that we >>> >>> 1. see what get's open_ported() ed >>> 2. figure out why that doesn't work. >>> >>> >>> >>> /klacke >>> >>> >>> ------------------------------------------------------------------------------ >>> Enter the BlackBerry Developer Challenge >>> This is your chance to win up to $100,000 in prizes! For a limited time, >>> vendors submitting new applications to BlackBerry App World(TM) will have >>> the opportunity to enter the BlackBerry Developer Challenge. See full >>> prize >>> details at: http://p.sf.net/sfu/Challenge >>> _______________________________________________ >>> Erlyaws-list mailing list >>> Erlyaws-list@REDACTED >>> https://lists.sourceforge.net/lists/listinfo/erlyaws-list >> >> >> ------------------------------------------------------------------------------ >> Enter the BlackBerry Developer Challenge >> This is your chance to win up to $100,000 in prizes! For a limited time, >> vendors submitting new applications to BlackBerry App World(TM) will have >> the opportunity to enter the BlackBerry Developer Challenge. See full prize >> details at: http://p.sf.net/sfu/Challenge >> _______________________________________________ >> Erlyaws-list mailing list >> Erlyaws-list@REDACTED >> https://lists.sourceforge.net/lists/listinfo/erlyaws-list >> >> > > > > -- > Carl McDade > Lead Developer > ____________________ > > FireOrb CMS > www.fireorb.info > -- Carl McDade Lead Developer ____________________ FireOrb CMS www.fireorb.info From leap@REDACTED Wed Jul 29 07:51:01 2009 From: leap@REDACTED (Michael Turner) Date: Wed, 29 Jul 2009 05:51:01 +0000 Subject: [erlang-questions] R13B01: erts_cpu_info_update() "int mib[0];" + sysctl call trashes stack In-Reply-To: <19055.8324.515261.713077@pilspetsen.it.uu.se> Message-ID: <9ynB9qwW.1248846661.6157720.leap@gol.com> On 7/28/2009, "Mikael Pettersson" wrote: [snipped bit about "int mib[0];"] >On what platform does this code get compiled? i386_unknown_freebsd4.10, is what ./configure can figure. An old gcc as well. >I would have expected >recent GCCs to complain about the out-of-bounds indexing, I'm not so recent. Even with "-pedantic" it doesn't complain. I'm using a pretty old gcc, though: compiled in 1999. ("-pedantic" breaks other stuff, so I'll turn it off.) I don't understand those nested #ifdefs in erl_misc_utils.c, but for all I know this code hasn't been gotten past the C proprocessor in some years; maybe it's even part of some edit that had been (silently) abandoned. Like I say, I'm new to this. >Anyway, you have indeed found a real bug. The code in question is >utterly wrong. But it's surfacing now. Even though it might have been years since it was compiled. This raises a software engineering issue: why is there so much conditional compilation in the .c files? Shouldn't environment dependencies be isolated in platform-specific libraries or (failing that) in header files, to the extent possible? http://doc.cat-v.org/henry_spencer/ifdef_considered_harmful.pdf And what's the process for getting involved in cleaning up these sorts of messes? If it's "not really a bug", can I still submit it in patch format? >Proper bug reporting procedure is to send an email to the erlang-bugs >mailing list . . . Thanks, will do. Which brings up another software engineering / website / community issue: I didn't know what to do here, because I had not only found a bug, but wanted to submit a patch, and trapexit.org is misleading about this. I went to the obvious place: http://forum.trapexit.org/viewforum.php?f=3 [*] There, you'll see that (1) the most recent submitted patch is (implausibly) over a year ago; (2) the Site Admin's sticky posting says "For instruction [sic] how to register, please visit http://www.erlang.org/mailman/listinfo" -- which is a dead link. At that point, I gave up, came here, and asked. Sorry to be complaining so much here. I know it is an inappropriate list for these questions and comments. But I can't seem to get registered as a user. -michael turner [*] Not obvious from the URL? Meta-meta-issue: why aren't trapexit.org URLs more informative? Meta-meta-meta-issue: how does one get involved in improving www.trapexit.org? I'd sure like to put the Search box up near Navigation (search is, after all, navigation), or in the upper right hand corner, the most obvious place to look. http://www.useit.com/alertbox/20010513.html As it is, I have to hit my browser's right scroll bar twice to get to the www.trapexit.org search box. From kiszl@REDACTED Wed Jul 29 08:25:24 2009 From: kiszl@REDACTED (kiszl@REDACTED) Date: Wed, 29 Jul 2009 08:25:24 +0200 (CEST) Subject: [erlang-questions] Nested for-loops, there has to be a better way to do this In-Reply-To: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> Message-ID: <63418.194.88.55.211.1248848724.squirrel@localhost> Hi, Do you mean something like this? -------------------- -module(loop). -export([loop/2]). -export([callback/1]). callback(Numbers) -> io:format("~p~n", [Numbers]). loop(Callback, Limits) -> loop(Limits, lists:duplicate(length(Limits), 1), Callback). loop(Limits, Current, Callback) -> Callback(Current), case increment(Limits, Current) of finished -> ok; NewCurrent -> loop(Limits, NewCurrent, Callback) end. increment(Limits, Current) -> increment(Limits, Current, []). increment([], [], _) -> finished; increment([N|Limits], [N|Current], Done) -> increment(Limits, Current, [1|Done]); increment(_Limits, [N|Current], Done) -> Done ++ [N+1|Current]. -------------------- 1> loop:loop(fun loop:callback/1, [2,2,2,2,3]). [1,1,1,1,1] [2,1,1,1,1] [1,2,1,1,1] .... Regards, Z. > I was playing around with the idea of implementing a nested for-loop > like construct (I will concede, they are evil, but at rare times, > necessary.) In non-functional programming languages, they are trivial > to do. However, this has proved to be one of those simple things that > are a pain in the neck for me. The idea was to be able to pass in a > list [5, 5, 10] (for example) and go through it all like 3 dimensional > array. The below illustration is far from elegant (it's almost 23:00 > where I live and need to go to work tomorrow early, so no time to come > up with something more sane :) ), but is there as an inquisitive > exercise. Now, in my opinion, this implementation sucks (first time > I've ever gave such a problem a shot in Erlang.) > > Is there a better way to do this? Anyone would like to demonstrate > it? > > Note: I've tested this code with the lists [2, 4] and [2, 4, 5]. > > Compiled with: > Erlang (SMP,ASYNC_THREADS,HIPE) (BEAM) emulator version 5.6.5 > > My code: > ====================================== > -module(loopWrapper). > > -export([runLoop/1]). > > runLoop(List) -> > [FirstVal | RestList] = List, > NextVal = FirstVal - 1, > if (length(RestList) > 0) and (FirstVal > 0) -> > io:format("~B~n~n", [FirstVal]), > runLoop(RestList), > io:format("~n~n"), > runLoop([NextVal | RestList]); > FirstVal > 0 -> > io:format("~B", [FirstVal]), > runLoop([NextVal]); > true -> > null > end. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From erlang@REDACTED Wed Jul 29 08:35:31 2009 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 29 Jul 2009 08:35:31 +0200 Subject: [erlang-questions] Nested for-loops, there has to be a better way to do this In-Reply-To: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> Message-ID: <9b08084c0907282335s7fc42eb5r8e925e9597dae34c@mail.gmail.com> I don't understand what you want. If the input to your program is: [5,5,10] the output should be: X What is X? /Joe On Wed, Jul 29, 2009 at 4:55 AM, Yves S. Garret wrote: > I was playing around with the idea of implementing a nested for-loop > like construct (I will concede, they are evil, but at rare times, > necessary.) ?In non-functional programming languages, they are trivial > to do. ?However, this has proved to be one of those simple things that > are a pain in the neck for me. ?The idea was to be able to pass in a > list [5, 5, 10] (for example) and go through it all like 3 dimensional > array. ?The below illustration is far from elegant (it's almost 23:00 > where I live and need to go to work tomorrow early, so no time to come > up with something more sane :) ), but is there as an inquisitive > exercise. ?Now, in my opinion, this implementation sucks (first time > I've ever gave such a problem a shot in Erlang.) > > Is there a better way to do this? ?Anyone would like to demonstrate > it? > > Note: I've tested this code with the lists [2, 4] and [2, 4, 5]. > > Compiled with: > Erlang (SMP,ASYNC_THREADS,HIPE) (BEAM) emulator version 5.6.5 > > My code: > ====================================== > -module(loopWrapper). > > -export([runLoop/1]). > > runLoop(List) -> > ?[FirstVal | RestList] = List, > ?NextVal = FirstVal - 1, > ?if (length(RestList) > 0) and (FirstVal > 0) -> > ? ?io:format("~B~n~n", [FirstVal]), > ? ?runLoop(RestList), > ? ?io:format("~n~n"), > ? ?runLoop([NextVal | RestList]); > ?FirstVal > 0 -> > ? ?io:format("~B", [FirstVal]), > ? ?runLoop([NextVal]); > ?true -> > ? ?null > ?end. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From mazen.harake@REDACTED Wed Jul 29 10:01:17 2009 From: mazen.harake@REDACTED (Mazen Harake) Date: Wed, 29 Jul 2009 11:01:17 +0300 Subject: [erlang-questions] wx tutorial In-Reply-To: <5756037b0907281159l57a53fe9tae9e4c9b68bdf59a@mail.gmail.com> References: <4A6ECB7F.5060201@erlang-consulting.com> <5756037b0907281159l57a53fe9tae9e4c9b68bdf59a@mail.gmail.com> Message-ID: <4A7001CD.2040401@erlang-consulting.com> Ah that would be great! If you want, put it on trapexit.org, if you preffer another format then perhaps with your permission I can put it up there. I think that the wx binding is a really long needed feature in Erlang (despite some people hating it and saying Erlang shouldn't go near this kind of stuff). Thanks :) /Mazen Jordan Wilberding wrote: > I have used wx extensively in C++ before, and am now starting to use it with > Erlang. I will be giving a talk on wx and and Erlang this weekend for our > local Linux User Group. I will try to generate a tutorial from it. > > Thanks! > Jordan Wilberding > > On Tue, Jul 28, 2009 at 4:57 AM, Mazen Harake < > mazen.harake@REDACTED> wrote: > > >> Hey, >> >> Just curious if there are (by now) experts using the wxwidget binding in >> Erlang... and if so, how about writing a small tutorial on how to use it >> (even if there are examples a tutorial is always nice). Perhaps on >> www.trapexit.org (or whatever)? >> >> >> Would be nice to have one, mapping the tutorials from C++ API to Erlang API >> was a bit more time consuming then I thought... >> >> Just a thought :) >> >> >> >> /Mazen >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> > > From dmitriid@REDACTED Wed Jul 29 10:59:59 2009 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Wed, 29 Jul 2009 11:59:59 +0300 Subject: [erlang-questions] Re: Misultin v0.1 released In-Reply-To: References: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> Message-ID: <3E696E88-AB1E-4BB2-B97B-FFB95A771757@gmail.com> > >> Out of interest, you should take a look at the Yaws/Mochiweb >> comparison (link?). These are interesting because Yaws does in >> general >> 50-80% more for you than Mochiweb but yet compares very favorably on >> the benchmarks. > > if you are referring to joe's bench i'm pretty familiar with it :) There's also this one: http://www.joeandmotorboat.com/2009/01/03/nginx-vs-yaws-vs-mochiweb-web-server-performance-deathmatch-part-2/ From ttmrichter@REDACTED Wed Jul 29 13:14:56 2009 From: ttmrichter@REDACTED (ttmrichter@REDACTED) Date: Wed, 29 Jul 2009 11:14:56 +0000 Subject: [erlang-questions] Re: Learn you some Erlang for great good! In-Reply-To: <8b9ee55b0907282044w68c845b6k36acd87963e79101@mail.gmail.com> Message-ID: <0016364c5983130685046fd64c51@google.com> On Jul 29, 2009 11:44am, "Fred Hebert (MononcQc)" wrote: > However, I intend for the first chapters released to be spotless; If the > tutorial is linked to other sites while it contains broken code, spelling > or > grammar mistakes and/or plainly wrong information, it is more than likely > to gain a bad reputation that will never go away. > I'm aiming for a good first impression to start on solid bases, then I'm > going to relax and make things more open afterwards. I'm willing to help proof. (As an English teacher I proof-read stuff all the time.) On the subject of code, however, I recommend a simple pre-processing step that extracts the code in some way and actually executes it as part and parcel of building your web page. If the code fails (unexpectedly, that is -- use eunit or something similar as your test engine to help with expected vs. unexpected errors), your document simply doesn't get published. This is, apparently, how the code in the Real World Haskell book was done and, so far, I've not found any code errors in that book -- something positively shocking in my experience. From mikpe@REDACTED Wed Jul 29 13:18:03 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Wed, 29 Jul 2009 13:18:03 +0200 Subject: [erlang-questions] R13B01: erts_cpu_info_update() "int mib[0];" + sysctl call trashes stack In-Reply-To: <9ynB9qwW.1248846661.6157720.leap@gol.com> References: <19055.8324.515261.713077@pilspetsen.it.uu.se> <9ynB9qwW.1248846661.6157720.leap@gol.com> Message-ID: <19056.12267.263467.257126@pilspetsen.it.uu.se> Michael Turner writes: > And what's the process for getting involved in cleaning up these sorts > of messes? If it's "not really a bug", can I still submit it in > patch format? > > >Proper bug reporting procedure is to send an email to the erlang-bugs > >mailing list . . . > > Thanks, will do. > > Which brings up another software engineering / website / community issue: > I didn't know what to do here, because I had not only found a bug, but > wanted to submit a patch, and trapexit.org is misleading about this. > > I went to the obvious place: > > http://forum.trapexit.org/viewforum.php?f=3 [*] > > There, you'll see that > > (1) the most recent submitted patch is (implausibly) over a year ago; > > (2) the Site Admin's sticky posting says "For instruction [sic] how to > register, please visit http://www.erlang.org/mailman/listinfo" -- which > is a dead link. > > At that point, I gave up, came here, and asked. Sorry to be complaining > so much here. I know it is an inappropriate list for these questions > and comments. But I can't seem to get registered as a user. > > -michael turner > > [*] Not obvious from the URL? Meta-meta-issue: why aren't trapexit.org > URLs more informative? > > Meta-meta-meta-issue: how does one get involved in improving > www.trapexit.org? I'd sure like to put the Search box up near > Navigation (search is, after all, navigation), or in the upper right > hand corner, the most obvious place to look. > > http://www.useit.com/alertbox/20010513.html > > As it is, I have to hit my browser's right scroll bar twice to get to > the www.trapexit.org search box. www.erlang.org is the primary Erlang site, not www.trapexit.org. From harveyd@REDACTED Wed Jul 29 13:18:42 2009 From: harveyd@REDACTED (Dale Harvey) Date: Wed, 29 Jul 2009 12:18:42 +0100 Subject: [erlang-questions] Re: Learn you some Erlang for great good! In-Reply-To: <8b9ee55b0907282044w68c845b6k36acd87963e79101@mail.gmail.com> References: <8b9ee55b0907281003g311d71f9ie6ef1ddb141646ae@mail.gmail.com> <97f48932-496e-4bdf-8c96-9eef37d588d0@j9g2000prh.googlegroups.com> <8b9ee55b0907282044w68c845b6k36acd87963e79101@mail.gmail.com> Message-ID: I would be happy to help. I have also started on an erlang documentation site, which is basically putting a nicer interface on top of the module references etc, ill wait till its up then would probably be a good idea to talk about having them compliment each other. Cheers Dale 2009/7/29 Fred Hebert (MononcQc) > On Tue, Jul 28, 2009 at 10:59 PM, Yves S. Garret < > yoursurrogategod@REDACTED > > wrote: > > > Just a thought: > > As long as you clearly state on your website that the below info > > should not be considered reliable until said so otherwise, there > > shouldn't be a problem with having everyone on the net look at it and > > give you feedback. > > > > On Jul 28, 1:03 pm, "Fred Hebert (MononcQc)" > > wrote: > > > Some of the readers of this list may be familiar withhttp:// > > learnyouahaskell.com/, an online book/tutorial dedicated to teaching > > > Haskell in a lighthearted way. > > > I've always thought Erlang had a steep learning curve unless you decide > > to > > > pay for books (the few of them are excellent, though). A good while > ago, > > I > > > took time to discuss with the author of Learn You a Haskell and I > decided > > to > > > start a version specifically for Erlang with his agreement. The > > objectives > > > are pretty much the same (lighthearted tutorials for beginners, free of > > > charge) and so is the concept. > > > I've written a few chapters already (Introduction, how to install, > basic > > > shell commands, some of the basic data types) and although the text is > > still > > > very short, I've got a website up and running. > > > I'm looking for a few people with different levels of knowledge of > Erlang > > in > > > order to review the first chapters. I'm looking to validate: a. the > > > accessibility of information; b. if the information is right; c. gather > > > comments and criticism. I find it important to do a first review early > in > > > the writing in order to fix flaws or improve on parts of the writing > > before > > > having to change too much stuff. > > > If you feel like giving me a helping hand in making the first chapters > as > > > good as possible through your reviews, just send me an email back. I'll > > > forward the URL and user/password needed to see the info (I'm > protecting > > > everything until reviewed, in order not to broadcast false or > > inappropriate > > > information). > > > Of course, once the first chapters are pushed in production, I'll keep > > > writing and might ask for some more help, if you don't mind the > > occasional > > > email. > > > Thanks in advance. > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > A similar notice is actually written in the dev version of the site (which > I > plan on making widely public instead of password protected after the first > few chapters are published). The main page will acknowledge the possibility > of having mistakes and asking people to tell me if they find any for the > production version. > > However, I intend for the first chapters released to be spotless; If the > tutorial is linked to other sites while it contains broken code, spelling > or > grammar mistakes and/or plainly wrong information, it is more than likely > to gain a bad reputation that will never go away. > > I'm aiming for a good first impression to start on solid bases, then I'm > going to relax and make things more open afterwards. > From steven.charles.davis@REDACTED Wed Jul 29 13:26:31 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 29 Jul 2009 04:26:31 -0700 (PDT) Subject: How to get re:split to NOT return any empty binaries... Message-ID: <959fbd34-ffd0-4bf1-b3fa-b4daf6a2b4c2@o6g2000yqj.googlegroups.com> I've found myself writing... Result = [X || X = X1 <- re:split(Bin, Regex), X1 =/= <<>>], ..quite a lot. Is there a way to get split to NOT return empty binaries (trim options only removes the last one(s))? Best, Steve From mononcqc@REDACTED Wed Jul 29 13:52:59 2009 From: mononcqc@REDACTED (Fred Hebert (MononcQc)) Date: Wed, 29 Jul 2009 07:52:59 -0400 Subject: [erlang-questions] Re: Learn you some Erlang for great good! In-Reply-To: <0016364c5983130685046fd64c51@google.com> References: <8b9ee55b0907282044w68c845b6k36acd87963e79101@mail.gmail.com> <0016364c5983130685046fd64c51@google.com> Message-ID: <8b9ee55b0907290452hea317daqd1b374641a027bea@mail.gmail.com> On Wed, Jul 29, 2009 at 7:14 AM, wrote: > On Jul 29, 2009 11:44am, "Fred Hebert (MononcQc)" > wrote: > > However, I intend for the first chapters released to be spotless; If the > > tutorial is linked to other sites while it contains broken code, spelling > or > > grammar mistakes and/or plainly wrong information, it is more than > likely > > to gain a bad reputation that will never go away. > > > I'm aiming for a good first impression to start on solid bases, then I'm > > going to relax and make things more open afterwards. > > I'm willing to help proof. (As an English teacher I proof-read stuff all > the time.) > > On the subject of code, however, I recommend a simple pre-processing step > that extracts the code in some way and actually executes it as part and > parcel of building your web page. If the code fails (unexpectedly, that is > -- use eunit or something similar as your test engine to help with expected > vs. unexpected errors), your document simply doesn't get published. This is, > apparently, how the code in the Real World Haskell book was done and, so > far, I've not found any code errors in that book -- something positively > shocking in my experience. Having code extracted and executed is a really good idea. I'm going to see what I can do to build a system like that, if not just having tests on the side maintained manually. From jacek99@REDACTED Wed Jul 29 15:42:11 2009 From: jacek99@REDACTED (Jacek Furmankiewicz) Date: Wed, 29 Jul 2009 09:42:11 -0400 Subject: Ports used by Erlang OTP Message-ID: <269a5c30907290642k55e59c5dhf91fe4b3809f4a73@mail.gmail.com> Is there some documentation as to which ports are used internally by Erlang OTP? I need to document it for our operations team before they move the app into production. Our data centers are very hardened and all ports are closed by defaults. Only those specifically requested are opened. I guess I would need to know the ports used by EPMD and OTP for all the rpc:_ calls...or any other ports that are required for an OTP app to function. Thanks, Jacek From peter@REDACTED Wed Jul 29 15:57:12 2009 From: peter@REDACTED (Peter Sabaini) Date: Wed, 29 Jul 2009 15:57:12 +0200 Subject: [erlang-questions] Ports used by Erlang OTP In-Reply-To: <269a5c30907290642k55e59c5dhf91fe4b3809f4a73@mail.gmail.com> References: <269a5c30907290642k55e59c5dhf91fe4b3809f4a73@mail.gmail.com> Message-ID: <1248875832.5798.66.camel@gram.local> On Wed, 2009-07-29 at 09:42 -0400, Jacek Furmankiewicz wrote: > Is there some documentation as to which ports are used internally by Erlang > OTP? For OTP, this is configurable, see {inet_dist_listen_min, First} {inet_dist_listen_max, Last} in the kernel reference docs ( http://erlang.org/doc/man/kernel_app.html ) EPMD listens on port 4369 by default > I need to document it for our operations team before they move the app into > production. Our data centers are very hardened and all ports are closed by > defaults. > Only those specifically requested are opened. > > I guess I would need to know the ports used by EPMD and OTP for all the > rpc:_ calls...or any other ports that are required for an OTP app to > function. > > Thanks, > Jacek -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From jacek99@REDACTED Wed Jul 29 16:55:42 2009 From: jacek99@REDACTED (Jacek Furmankiewicz) Date: Wed, 29 Jul 2009 10:55:42 -0400 Subject: [erlang-questions] Ports used by Erlang OTP In-Reply-To: <1248875832.5798.66.camel@gram.local> References: <269a5c30907290642k55e59c5dhf91fe4b3809f4a73@mail.gmail.com> <1248875832.5798.66.camel@gram.local> Message-ID: <269a5c30907290755w6a6b5a22x90708a0ba33ee4e5@mail.gmail.com> Are there default values that we cal rely on to be consistent, or should we always specify it explicitly. Are there any port ranges that are recommended (i.e. I know below 1024 usually requires root access, etc.) Thanks On Wed, Jul 29, 2009 at 9:57 AM, Peter Sabaini wrote: > On Wed, 2009-07-29 at 09:42 -0400, Jacek Furmankiewicz wrote: > > Is there some documentation as to which ports are used internally by > Erlang > > OTP? > > For OTP, this is configurable, see > > {inet_dist_listen_min, First} > {inet_dist_listen_max, Last} > > in the kernel reference docs > ( http://erlang.org/doc/man/kernel_app.html ) > > EPMD listens on port 4369 by default > > > I need to document it for our operations team before they move the app > into > > production. Our data centers are very hardened and all ports are closed > by > > defaults. > > Only those specifically requested are opened. > > > > I guess I would need to know the ports used by EPMD and OTP for all the > > rpc:_ calls...or any other ports that are required for an OTP app to > > function. > > > > Thanks, > > Jacek > From erik@REDACTED Wed Jul 29 17:22:02 2009 From: erik@REDACTED (Erik Rigtorp) Date: Wed, 29 Jul 2009 17:22:02 +0200 Subject: Record efficiency Message-ID: Hi! I'm trying to figure out the performance characteristics of records. I've been looking at the code at it seems like named attribute access on records gets turned into some function evaluation by the preprocessor. I'm trying to read a bunch of attributes of a record and there are two ways to do this. Either: Rec#rec{attr1 = Attr1, attr2 = Attr2} Or: Attr1 = Rec#rec.attr1 Attr1 = Rec#rec.attr2 Will these two have the same performance or will the first one just be a match and the second one result in two function calls? Erik From baryluk@REDACTED Wed Jul 29 17:35:40 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Wed, 29 Jul 2009 17:35:40 +0200 Subject: [erlang-questions] Record efficiency In-Reply-To: References: Message-ID: <1248881740.9263.308.camel@sredniczarny> Dnia 2009-07-29, ?ro o godzinie 17:22 +0200, Erik Rigtorp pisze: > Hi! > > I'm trying to figure out the performance characteristics of records. > I've been looking at the code at it seems like named attribute access > on records gets turned into some function evaluation by the > preprocessor. I'm trying to read a bunch of attributes of a record and > there are two ways to do this. > > Either: > Rec#rec{attr1 = Attr1, attr2 = Attr2} this is record assigment, it is pattern only in function head right? > Or: > Attr1 = Rec#rec.attr1 > Attr1 = Rec#rec.attr2 > > Will these two have the same performance or will the first one just be > a match and the second one result in two function calls? They will be exactly the same (compile using erlc -S and compare assembly). It isn't two functions calls. Records are just syntactic sugar for tuples, and accessing "fields" of records are like element/2 call for tuple, but it isn't function exactly, it is BIF implemented directly in VM. One can argue that in second case compiler will test Rec variable two times, if it is really record rec, but actually it test it only once. -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From coa@REDACTED Wed Jul 29 18:25:26 2009 From: coa@REDACTED (=?ISO-8859-2?Q?Cl=E1udio_Amaral?=) Date: Wed, 29 Jul 2009 17:25:26 +0100 Subject: [erlang-questions] Nested for-loops, there has to be a better way to do this In-Reply-To: <63418.194.88.55.211.1248848724.squirrel@localhost> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <63418.194.88.55.211.1248848724.squirrel@localhost> Message-ID: <4A7077F6.10508@dcc.fc.up.pt> Used some of previous code... I tried to make a polymorphic function and coupled an example... this ended a little bit confusing, but as you can see, it makes all iterations (check the printed indexes, reversed). The body behaviour is the tricky stuff to code, but in exchange it is modular... Since I am an erlang newb, please correct anything you see as bad programming Cl?udio %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -module(for_loop). -export([for/4, show/0]). for(BodyFunc, BodyArgs, StepFunc, {First, Last}) -> for_(BodyFunc, BodyArgs, StepFunc, {First, Last}, First). for_(BodyFunc, BodyArgs, StepFunc, {_, LastTerm}, LastTerm) -> BodyFunc(BodyArgs, LastTerm); for_(BodyFunc, BodyArgs, StepFunc, LimitsTuple, IndexTerm) -> NextArgs = BodyFunc(BodyArgs, IndexTerm), for_(BodyFunc, NextArgs, StepFunc, LimitsTuple, StepFunc(IndexTerm)). show() -> for( fun(A1,A2)->body1(A1,A2) end, [{0,3}, [{1,0}], [{3,6},{1,2}]], fun(I) -> I+1 end, {0,3} ). body1([ LimitsTuple, [{NLoop,II}|Rest], []],I) -> print([ I | lists:map(fun({_,Snd}) -> Snd end, Rest) ]), IndexInfo = [{NLoop,II}|Rest], [ LimitsTuple, IndexInfo, []]; body1([ LimitsTuple, [{NLoop,II}|Rest], [{X1,Xn}|XS]],I) -> for( fun (A1,A2)->body1(A1,A2) end, [ LimitsTuple, [ {NLoop+1,II}, {NLoop,I} | Rest ], XS], fun(K) -> K+1 end, {X1,Xn} ), IndexInfo = [{NLoop,II}|Rest], [ LimitsTuple, IndexInfo, [{X1,Xn}|XS] ]. print(Numbers) -> io:format("indexes ~p~n", [Numbers]). kiszl@REDACTED escreveu: > Regards, > Z. > > >> I was playing around with the idea of implementing a nested for-loop >> like construct (I will concede, they are evil, but at rare times, >> necessary.) In non-functional programming languages, they are trivial >> to do. However, this has proved to be one of those simple things that >> are a pain in the neck for me. The idea was to be able to pass in a >> list [5, 5, 10] (for example) and go through it all like 3 dimensional >> array. The below illustration is far from elegant (it's almost 23:00 >> where I live and need to go to work tomorrow early, so no time to come >> up with something more sane :) ), but is there as an inquisitive >> exercise. Now, in my opinion, this implementation sucks (first time >> I've ever gave such a problem a shot in Erlang.) >> >> Is there a better way to do this? Anyone would like to demonstrate >> it? >> >> Note: I've tested this code with the lists [2, 4] and [2, 4, 5]. >> >> Compiled with: >> Erlang (SMP,ASYNC_THREADS,HIPE) (BEAM) emulator version 5.6.5 >> >> My code: >> ====================================== >> -module(loopWrapper). >> >> -export([runLoop/1]). >> >> runLoop(List) -> >> [FirstVal | RestList] = List, >> NextVal = FirstVal - 1, >> if (length(RestList) > 0) and (FirstVal > 0) -> >> io:format("~B~n~n", [FirstVal]), >> runLoop(RestList), >> io:format("~n~n"), >> runLoop([NextVal | RestList]); >> FirstVal > 0 -> >> io:format("~B", [FirstVal]), >> runLoop([NextVal]); >> true -> >> null >> end. >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From chandrashekhar.mullaparthi@REDACTED Wed Jul 29 20:04:08 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Wed, 29 Jul 2009 19:04:08 +0100 Subject: [erlang-questions] Stopping ibrowse:send_req/4 GET requests using keep_alive In-Reply-To: <4A67D69E.3050005@diamondage.co.nz> References: <4A6539CB.8090306@catalyst.net.nz> <4A67D69E.3050005@diamondage.co.nz> Message-ID: 2009/7/23 Vik Olliver > On 21/07/09 Chandru wrote: > >> You could try setting the http_vsn to 1.0. This way the server is >> forced to close the connection after serving one request. >> > > Looks like I either need to use set_max_pipeline/3 or the latest & greatest > fix for the Close in headers. Any idea when this will be pushed into Jungerl > or should I ask someone else about that? > > I've just pushed the latest code to jungerl. Sorry about the delay. cheers Chandru From the.ajarn@REDACTED Wed Jul 29 20:12:12 2009 From: the.ajarn@REDACTED (Brentley Jones) Date: Wed, 29 Jul 2009 13:12:12 -0500 Subject: [erlang-questions] Nested for-loops, there has to be a better way to do this In-Reply-To: <4A7077F6.10508@dcc.fc.up.pt> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <63418.194.88.55.211.1248848724.squirrel@localhost> <4A7077F6.10508@dcc.fc.up.pt> Message-ID: This is my take on it: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -module(for). -export([for/4, test_1/0, test_2/0]). for(BodyFun, Acc, ConditionFun, StepFun) -> Done = ConditionFun(Acc), if Done == false -> Acc; true -> NewAcc = BodyFun(Acc), LastAcc = StepFun(NewAcc), for(BodyFun, LastAcc, ConditionFun, StepFun) end. test_1() -> Body = fun(I) -> io:format("i=~p~n", [I]), I end, for(Body, 0, fun(I) -> I < 10 end, fun(I) -> I + 1 end), ok. test_2() -> CondFun = fun(X) -> X < 5 end, StepFun = fun(X) -> X + 1 end, BodyInner = fun(J) -> io:format("~p", [J]), J end, BodyOuter = fun(I) -> for(BodyInner, 0, CondFun, StepFun), io:format("~n"), I end, for(BodyOuter, 0, CondFun, StepFun), ok. %%%%% Output: 2> for:test_2(). 01234 01234 01234 01234 01234 ok 3> for:test_1(). i=0 i=1 i=2 i=3 i=4 i=5 i=6 i=7 i=8 i=9 ok On Jul 29, 2009, at 11:25 AM, Cl?udio Amaral wrote: > Used some of previous code... > > I tried to make a polymorphic function and coupled an example... > this ended a little bit confusing, but as you can see, it makes all > iterations (check the printed indexes, reversed). > > The body behaviour is the tricky stuff to code, but in exchange it > is modular... > > Since I am an erlang newb, please correct anything you see as bad > programming > > Cl?udio > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > -module(for_loop). > > -export([for/4, show/0]). > > > for(BodyFunc, BodyArgs, StepFunc, {First, Last}) -> > for_(BodyFunc, BodyArgs, StepFunc, {First, Last}, First). > > > for_(BodyFunc, BodyArgs, StepFunc, {_, LastTerm}, LastTerm) -> > BodyFunc(BodyArgs, LastTerm); > for_(BodyFunc, BodyArgs, StepFunc, LimitsTuple, IndexTerm) -> > NextArgs = BodyFunc(BodyArgs, IndexTerm), > for_(BodyFunc, NextArgs, StepFunc, LimitsTuple, > StepFunc(IndexTerm)). > > > show() -> > for( fun(A1,A2)->body1(A1,A2) end, > [{0,3}, [{1,0}], [{3,6},{1,2}]], > fun(I) -> I+1 end, > {0,3} > ). > > > body1([ LimitsTuple, [{NLoop,II}|Rest], []],I) -> > print([ I | lists:map(fun({_,Snd}) -> Snd end, Rest) ]), > IndexInfo = [{NLoop,II}|Rest], > [ LimitsTuple, IndexInfo, []]; > body1([ LimitsTuple, [{NLoop,II}|Rest], [{X1,Xn}|XS]],I) -> > for( fun (A1,A2)->body1(A1,A2) end, > [ LimitsTuple, [ {NLoop+1,II}, {NLoop,I} | Rest ], XS], > fun(K) -> K+1 end, > {X1,Xn} > ), > IndexInfo = [{NLoop,II}|Rest], > [ LimitsTuple, IndexInfo, [{X1,Xn}|XS] ]. > > print(Numbers) -> > io:format("indexes ~p~n", [Numbers]). > > > kiszl@REDACTED escreveu: >> Regards, >> Z. >> >> >>> I was playing around with the idea of implementing a nested for-loop >>> like construct (I will concede, they are evil, but at rare times, >>> necessary.) In non-functional programming languages, they are >>> trivial >>> to do. However, this has proved to be one of those simple things >>> that >>> are a pain in the neck for me. The idea was to be able to pass in a >>> list [5, 5, 10] (for example) and go through it all like 3 >>> dimensional >>> array. The below illustration is far from elegant (it's almost >>> 23:00 >>> where I live and need to go to work tomorrow early, so no time to >>> come >>> up with something more sane :) ), but is there as an inquisitive >>> exercise. Now, in my opinion, this implementation sucks (first time >>> I've ever gave such a problem a shot in Erlang.) >>> >>> Is there a better way to do this? Anyone would like to demonstrate >>> it? >>> >>> Note: I've tested this code with the lists [2, 4] and [2, 4, 5]. >>> >>> Compiled with: >>> Erlang (SMP,ASYNC_THREADS,HIPE) (BEAM) emulator version 5.6.5 >>> >>> My code: >>> ====================================== >>> -module(loopWrapper). >>> >>> -export([runLoop/1]). >>> >>> runLoop(List) -> >>> [FirstVal | RestList] = List, >>> NextVal = FirstVal - 1, >>> if (length(RestList) > 0) and (FirstVal > 0) -> >>> io:format("~B~n~n", [FirstVal]), >>> runLoop(RestList), >>> io:format("~n~n"), >>> runLoop([NextVal | RestList]); >>> FirstVal > 0 -> >>> io:format("~B", [FirstVal]), >>> runLoop([NextVal]); >>> true -> >>> null >>> end. >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >>> >> >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From kiszl@REDACTED Wed Jul 29 20:57:55 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Wed, 29 Jul 2009 20:57:55 +0200 Subject: [erlang-questions] Nested for-loops, there has to be a better way to do this In-Reply-To: References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <63418.194.88.55.211.1248848724.squirrel@localhost> <4A7077F6.10508@dcc.fc.up.pt> Message-ID: <4A709BB3.5010701@tmit.bme.hu> And what do I gain using this for function compared to simply writing an "ad-hoc" recursive function whenever needed ? for( fun(I) -> body(I) end, 0, fun(I) -> I < 20 end, fun(I) -> I + 2 end ) vs. my_fun(20) -> ok; my_fun(I) -> body(I), my_fun(I+2). Z. PS: seriously, is this what the original question was getting at? Brentley Jones wrote: > This is my take on it: > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > -module(for). > > -export([for/4, test_1/0, test_2/0]). > > for(BodyFun, Acc, ConditionFun, StepFun) -> > Done = ConditionFun(Acc), > if Done == false -> > Acc; > true -> > NewAcc = BodyFun(Acc), > LastAcc = StepFun(NewAcc), > for(BodyFun, LastAcc, ConditionFun, StepFun) > end. > > test_1() -> > Body = fun(I) -> > io:format("i=~p~n", [I]), > I > end, > for(Body, 0, fun(I) -> I < 10 end, fun(I) -> I + 1 end), > ok. > > test_2() -> > CondFun = fun(X) -> X < 5 end, > StepFun = fun(X) -> X + 1 end, > BodyInner = fun(J) -> > io:format("~p", [J]), > J > end, > BodyOuter = fun(I) -> > for(BodyInner, 0, CondFun, StepFun), > io:format("~n"), > I > end, > for(BodyOuter, 0, CondFun, StepFun), > ok. > > > %%%%% Output: > 2> for:test_2(). > 01234 > 01234 > 01234 > 01234 > 01234 > ok > 3> for:test_1(). > i=0 > i=1 > i=2 > i=3 > i=4 > i=5 > i=6 > i=7 > i=8 > i=9 > ok > > On Jul 29, 2009, at 11:25 AM, Cl?udio Amaral wrote: > >> Used some of previous code... >> >> I tried to make a polymorphic function and coupled an example... this >> ended a little bit confusing, but as you can see, it makes all >> iterations (check the printed indexes, reversed). >> >> The body behaviour is the tricky stuff to code, but in exchange it is >> modular... >> >> Since I am an erlang newb, please correct anything you see as bad >> programming >> >> Cl?udio >> >> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% >> -module(for_loop). >> >> -export([for/4, show/0]). >> >> >> for(BodyFunc, BodyArgs, StepFunc, {First, Last}) -> >> for_(BodyFunc, BodyArgs, StepFunc, {First, Last}, First). >> >> >> for_(BodyFunc, BodyArgs, StepFunc, {_, LastTerm}, LastTerm) -> >> BodyFunc(BodyArgs, LastTerm); >> for_(BodyFunc, BodyArgs, StepFunc, LimitsTuple, IndexTerm) -> >> NextArgs = BodyFunc(BodyArgs, IndexTerm), >> for_(BodyFunc, NextArgs, StepFunc, LimitsTuple, StepFunc(IndexTerm)). >> >> >> show() -> >> for( fun(A1,A2)->body1(A1,A2) end, >> [{0,3}, [{1,0}], [{3,6},{1,2}]], >> fun(I) -> I+1 end, >> {0,3} >> ). >> >> >> body1([ LimitsTuple, [{NLoop,II}|Rest], []],I) -> >> print([ I | lists:map(fun({_,Snd}) -> Snd end, Rest) ]), >> IndexInfo = [{NLoop,II}|Rest], >> [ LimitsTuple, IndexInfo, []]; >> body1([ LimitsTuple, [{NLoop,II}|Rest], [{X1,Xn}|XS]],I) -> >> for( fun (A1,A2)->body1(A1,A2) end, >> [ LimitsTuple, [ {NLoop+1,II}, {NLoop,I} | Rest ], XS], >> fun(K) -> K+1 end, >> {X1,Xn} >> ), >> IndexInfo = [{NLoop,II}|Rest], >> [ LimitsTuple, IndexInfo, [{X1,Xn}|XS] ]. >> >> print(Numbers) -> >> io:format("indexes ~p~n", [Numbers]). >> >> >> kiszl@REDACTED escreveu: >>> Regards, >>> Z. >>> >>> >>>> I was playing around with the idea of implementing a nested for-loop >>>> like construct (I will concede, they are evil, but at rare times, >>>> necessary.) In non-functional programming languages, they are trivial >>>> to do. However, this has proved to be one of those simple things that >>>> are a pain in the neck for me. The idea was to be able to pass in a >>>> list [5, 5, 10] (for example) and go through it all like 3 dimensional >>>> array. The below illustration is far from elegant (it's almost 23:00 >>>> where I live and need to go to work tomorrow early, so no time to come >>>> up with something more sane :) ), but is there as an inquisitive >>>> exercise. Now, in my opinion, this implementation sucks (first time >>>> I've ever gave such a problem a shot in Erlang.) >>>> >>>> Is there a better way to do this? Anyone would like to demonstrate >>>> it? >>>> >>>> Note: I've tested this code with the lists [2, 4] and [2, 4, 5]. >>>> >>>> Compiled with: >>>> Erlang (SMP,ASYNC_THREADS,HIPE) (BEAM) emulator version 5.6.5 >>>> >>>> My code: >>>> ====================================== >>>> -module(loopWrapper). >>>> >>>> -export([runLoop/1]). >>>> >>>> runLoop(List) -> >>>> [FirstVal | RestList] = List, >>>> NextVal = FirstVal - 1, >>>> if (length(RestList) > 0) and (FirstVal > 0) -> >>>> io:format("~B~n~n", [FirstVal]), >>>> runLoop(RestList), >>>> io:format("~n~n"), >>>> runLoop([NextVal | RestList]); >>>> FirstVal > 0 -> >>>> io:format("~B", [FirstVal]), >>>> runLoop([NextVal]); >>>> true -> >>>> null >>>> end. >>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>>> >>>> >>>> >>> >>> >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From the.ajarn@REDACTED Wed Jul 29 21:05:42 2009 From: the.ajarn@REDACTED (Brentley Jones) Date: Wed, 29 Jul 2009 14:05:42 -0500 Subject: [erlang-questions] Nested for-loops, there has to be a better way to do this In-Reply-To: <4A709BB3.5010701@tmit.bme.hu> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <63418.194.88.55.211.1248848724.squirrel@localhost> <4A7077F6.10508@dcc.fc.up.pt> <4A709BB3.5010701@tmit.bme.hu> Message-ID: <772561A5-A67A-4738-99DA-396CF2ED6C2B@gmail.com> On Jul 29, 2009, at 1:57 PM, Zoltan Lajos Kis wrote: > And what do I gain using this for function compared to simply > writing an "ad-hoc" recursive function whenever needed ? Nothing is gained by my bloated, yet modular function. I was just expanding on the modular function listed before mine. > PS: seriously, is this what the original question was getting at? I do think it shows how to do nested for loops in a way that looks very imperative, which is what I think the original question was getting at. On Jul 29, 2009, at 2:55 PM, Yves S. Garret wrote: > I was playing around with the idea of implementing a nested for-loop > like construct (I will concede, they are evil, but at rare times, > necessary.) In non-functional programming languages, they are trivial > to do. However, this has proved to be one of those simple things that > are a pain in the neck for me. From kiszl@REDACTED Wed Jul 29 21:13:33 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Wed, 29 Jul 2009 21:13:33 +0200 Subject: [erlang-questions] Nested for-loops, there has to be a better way to do this In-Reply-To: <772561A5-A67A-4738-99DA-396CF2ED6C2B@gmail.com> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <63418.194.88.55.211.1248848724.squirrel@localhost> <4A7077F6.10508@dcc.fc.up.pt> <4A709BB3.5010701@tmit.bme.hu> <772561A5-A67A-4738-99DA-396CF2ED6C2B@gmail.com> Message-ID: <4A709F5D.2060401@tmit.bme.hu> Brentley Jones wrote: > > On Jul 29, 2009, at 1:57 PM, Zoltan Lajos Kis wrote: > >> And what do I gain using this for function compared to simply writing >> an "ad-hoc" recursive function whenever needed ? > > Nothing is gained by my bloated, yet modular function. I was just > expanding on the modular function listed before mine. > >> PS: seriously, is this what the original question was getting at? > > I do think it shows how to do nested for loops in a way that looks > very imperative, which is what I think the original question was > getting at. > > On Jul 29, 2009, at 2:55 PM, Yves S. Garret wrote: > >> I was playing around with the idea of implementing a nested for-loop >> like construct (I will concede, they are evil, but at rare times, >> necessary.) In non-functional programming languages, they are trivial >> to do. However, this has proved to be one of those simple things that >> are a pain in the neck for me. But if you look just one sentence further... "The idea was to be able to pass in a list [5, 5, 10] (for example) and go through it all like 3 dimensional array." Anyway, we should get the answer by dawn (CET) :) Z. From the.ajarn@REDACTED Wed Jul 29 21:22:55 2009 From: the.ajarn@REDACTED (Brentley Jones) Date: Wed, 29 Jul 2009 14:22:55 -0500 Subject: [erlang-questions] Nested for-loops, there has to be a better way to do this In-Reply-To: <4A709F5D.2060401@tmit.bme.hu> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <63418.194.88.55.211.1248848724.squirrel@localhost> <4A7077F6.10508@dcc.fc.up.pt> <4A709BB3.5010701@tmit.bme.hu> <772561A5-A67A-4738-99DA-396CF2ED6C2B@gmail.com> <4A709F5D.2060401@tmit.bme.hu> Message-ID: On Jul 29, 2009, at 2:13 PM, Zoltan Lajos Kis wrote: > Brentley Jones wrote: >> >> On Jul 29, 2009, at 1:57 PM, Zoltan Lajos Kis wrote: >> >>> And what do I gain using this for function compared to simply >>> writing an "ad-hoc" recursive function whenever needed ? >> >> Nothing is gained by my bloated, yet modular function. I was just >> expanding on the modular function listed before mine. >> >>> PS: seriously, is this what the original question was getting at? >> >> I do think it shows how to do nested for loops in a way that looks >> very imperative, which is what I think the original question was >> getting at. >> >> On Jul 29, 2009, at 2:55 PM, Yves S. Garret wrote: >> >>> I was playing around with the idea of implementing a nested for-loop >>> like construct (I will concede, they are evil, but at rare times, >>> necessary.) In non-functional programming languages, they are >>> trivial >>> to do. However, this has proved to be one of those simple things >>> that >>> are a pain in the neck for me. > But if you look just one sentence further... > "The idea was to be able to pass in a list [5, 5, 10] (for example) > and go through it all like 3 dimensional array." The fact that he says "in non-functional programming languages, they are trivial to do." just lead me to try to think that way. In a non- functional programming language there isn't a construct that he can just pass a list which has lengths for each dimension, he would have to set up each for loop with those values as the conditionals. > Anyway, we should get the answer by dawn (CET) :) > > Z. I think with the resources that we have provided though that we have answered the question in one way or another (either with these modular for loop constructs or the nice list comprehensions , etc). And Joe asked a very good question, what does he want the output to be? Does he want a body that has access to i, j, k [5, 5, 10] and is called like three nested for-loops (which is what I thought), or something totally different? From s.merle@REDACTED Wed Jul 29 21:33:45 2009 From: s.merle@REDACTED (Sebastien Merle) Date: Wed, 29 Jul 2009 21:33:45 +0200 Subject: code path when using inet loader Message-ID: <102d3f890907291233g737e1d37p80f17c8f92f2c862@mail.gmail.com> Hi, I'm experimenting with remote code server using a master node with erl_boot_server that starts slave nodes with arguments "-loader inet". It seems that if the master is started with -pz arguments, the beam files from the added path cannot be loaded by the slave nodes. If I just start the master in the directory where the beam files are instead of adding the path with -pz, the slave can load the beam files. Any hints on why this is happening ? Regards, Sebastien Merle. From peter@REDACTED Wed Jul 29 22:02:39 2009 From: peter@REDACTED (Peter Sabaini) Date: Wed, 29 Jul 2009 22:02:39 +0200 Subject: [erlang-questions] Ports used by Erlang OTP In-Reply-To: <269a5c30907290755w6a6b5a22x90708a0ba33ee4e5@mail.gmail.com> References: <269a5c30907290642k55e59c5dhf91fe4b3809f4a73@mail.gmail.com> <1248875832.5798.66.camel@gram.local> <269a5c30907290755w6a6b5a22x90708a0ba33ee4e5@mail.gmail.com> Message-ID: <1248897759.5798.72.camel@gram.local> On Wed, 2009-07-29 at 10:55 -0400, Jacek Furmankiewicz wrote: > Are there default values that we cal rely on to be consistent, or should we > always specify it explicitly. AFAIK distributed Erlang VMs use random highports by default > Are there any port ranges that are recommended (i.e. I know below 1024 > usually requires root access, etc.) No recommendation I know of. The extent of the range also depends on how many distributed nodes you plan on connecting (every node will then hold a connection to every other node which could require quite a bit of ports; N*(N-1)/2 ports to be precise). > > Thanks > > On Wed, Jul 29, 2009 at 9:57 AM, Peter Sabaini wrote: > > > On Wed, 2009-07-29 at 09:42 -0400, Jacek Furmankiewicz wrote: > > > Is there some documentation as to which ports are used internally by > > Erlang > > > OTP? > > > > For OTP, this is configurable, see > > > > {inet_dist_listen_min, First} > > {inet_dist_listen_max, Last} > > > > in the kernel reference docs > > ( http://erlang.org/doc/man/kernel_app.html ) > > > > EPMD listens on port 4369 by default > > > > > I need to document it for our operations team before they move the app > > into > > > production. Our data centers are very hardened and all ports are closed > > by > > > defaults. > > > Only those specifically requested are opened. > > > > > > I guess I would need to know the ports used by EPMD and OTP for all the > > > rpc:_ calls...or any other ports that are required for an OTP app to > > > function. > > > > > > Thanks, > > > Jacek > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From coa@REDACTED Wed Jul 29 22:20:51 2009 From: coa@REDACTED (=?ISO-8859-1?Q?Cl=E1udio_Amaral?=) Date: Wed, 29 Jul 2009 21:20:51 +0100 Subject: [erlang-questions] Nested for-loops, there has to be a better way to do this In-Reply-To: References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <63418.194.88.55.211.1248848724.squirrel@localhost> <4A7077F6.10508@dcc.fc.up.pt> <4A709BB3.5010701@tmit.bme.hu> <772561A5-A67A-4738-99DA-396CF2ED6C2B@gmail.com> <4A709F5D.2060401@tmit.bme.hu> Message-ID: <4A70AF23.1090404@dcc.fc.up.pt> Brentley Jones escreveu: > On Jul 29, 2009, at 2:13 PM, Zoltan Lajos Kis wrote: > >> Brentley Jones wrote: >>> >>> On Jul 29, 2009, at 1:57 PM, Zoltan Lajos Kis wrote: >>> >>>> And what do I gain using this for function compared to simply >>>> writing an "ad-hoc" recursive function whenever needed ? >>> >>> Nothing is gained by my bloated, yet modular function. I was just >>> expanding on the modular function listed before mine. >>> >>>> PS: seriously, is this what the original question was getting at? >>> >>> I do think it shows how to do nested for loops in a way that looks >>> very imperative, which is what I think the original question was >>> getting at. >>> >>> On Jul 29, 2009, at 2:55 PM, Yves S. Garret wrote: >>> >>>> I was playing around with the idea of implementing a nested for-loop >>>> like construct (I will concede, they are evil, but at rare times, >>>> necessary.) In non-functional programming languages, they are trivial >>>> to do. However, this has proved to be one of those simple things that >>>> are a pain in the neck for me. >> But if you look just one sentence further... >> "The idea was to be able to pass in a list [5, 5, 10] (for example) >> and go through it all like 3 dimensional array." > > The fact that he says "in non-functional programming languages, they > are trivial to do." just lead me to try to think that way. In a > non-functional programming language there isn't a construct that he > can just pass a list which has lengths for each dimension, he would > have to set up each for loop with those values as the conditionals. > I'm with you in this... In the end, it should depend on the purpose of the program. So, if you are ok with rewriting your 'loops' when needed (making them more readable and/or efficient), the code can be much simpler (like the code from Richard O'Keefe). If you are interested in writing some function to use like an imperative 'for', you have to take into account the possibility of non-integer index term (and step function), a flexible body representation to allow any argument passing and simulate the state changes between iterations and the the hard job to the one calling the 'for' function >> Anyway, we should get the answer by dawn (CET) :) >> >> Z. > > > I think with the resources that we have provided though that we have > answered the question in one way or another (either with these modular > for loop constructs or the nice list comprehensions , etc). > > And Joe asked a very good question, what does he want the output to be? > Does he want a body that has access to i, j, k [5, 5, 10] and is > called like three nested for-loops (which is what I thought), or > something totally different? > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From mononcqc@REDACTED Wed Jul 29 23:08:37 2009 From: mononcqc@REDACTED (Fred Hebert (MononcQc)) Date: Wed, 29 Jul 2009 17:08:37 -0400 Subject: [erlang-questions] Nested for-loops, there has to be a better way to do this In-Reply-To: <4A70AF23.1090404@dcc.fc.up.pt> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <63418.194.88.55.211.1248848724.squirrel@localhost> <4A7077F6.10508@dcc.fc.up.pt> <4A709BB3.5010701@tmit.bme.hu> <772561A5-A67A-4738-99DA-396CF2ED6C2B@gmail.com> <4A709F5D.2060401@tmit.bme.hu> <4A70AF23.1090404@dcc.fc.up.pt> Message-ID: <8b9ee55b0907291408l44c57c3ep8c60e3d0db13f848@mail.gmail.com> I find nested lists comprehension to be the easiest to deal with when representing nested loops: [[ Fn(I,J) || J <- GenExp2 ] || I <- GenEXp1 ]. or indented in a different way: [[[ Fn(I, J, K) || K <- GenExp3 ] || J <- GenExp2 ] || I <- GenExp1 ]. This gives a nested list. If that's not what you want, you could use lists:flatten/1 to change that, but overall I find this form to be the cleanest (visually speaking). From kevin@REDACTED Wed Jul 29 23:24:33 2009 From: kevin@REDACTED (Kevin A. Smith) Date: Wed, 29 Jul 2009 17:24:33 -0400 Subject: New IRC Channel: #erlang-otp Message-ID: * Have an Erlang question you need answered quick? * Wishing you had a friendly place to hang out online with other Erlangers? * Tired of either the silence or trolling on #erlang? If you answered yes to any of these consider hanging out in #erlang- otp, the friendly Erlang IRC channel. Erlang newbies are especially welcome! --Kevin From tuncer.ayaz@REDACTED Wed Jul 29 23:34:35 2009 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 29 Jul 2009 23:34:35 +0200 Subject: [erlang-questions] New IRC Channel: #erlang-otp In-Reply-To: References: Message-ID: <4ac8254d0907291434l33127605nc4ec457373822cc4@mail.gmail.com> On Wed, Jul 29, 2009 at 11:24 PM, Kevin A. Smith wrote: > * Have an Erlang question you need answered quick? > > * Wishing you had a friendly place to hang out online with other Erlangers? > > * Tired of either the silence or trolling on #erlang? What's the motivation? Is #erlang owned by someone else and we can't control moderation? Or is that we wanted to have second and different channel? > If you answered yes to any of these consider hanging out in #erlang-otp, the > friendly Erlang IRC channel. Erlang newbies are especially welcome! #erlang-otp on freenode (irc.freenode.net) before anyone asks :) From peter@REDACTED Wed Jul 29 23:38:57 2009 From: peter@REDACTED (Peter Sabaini) Date: Wed, 29 Jul 2009 23:38:57 +0200 Subject: [erlang-questions] Ports used by Erlang OTP In-Reply-To: <4d08db370907291329i1881f318y8596c4fb0d0645ea@mail.gmail.com> References: <269a5c30907290642k55e59c5dhf91fe4b3809f4a73@mail.gmail.com> <1248875832.5798.66.camel@gram.local> <269a5c30907290755w6a6b5a22x90708a0ba33ee4e5@mail.gmail.com> <1248897759.5798.72.camel@gram.local> <4d08db370907291329i1881f318y8596c4fb0d0645ea@mail.gmail.com> Message-ID: <1248903537.5798.78.camel@gram.local> On Wed, 2009-07-29 at 22:29 +0200, Hynek Vychodil wrote: > > > On Wed, Jul 29, 2009 at 10:02 PM, Peter Sabaini > wrote: > On Wed, 2009-07-29 at 10:55 -0400, Jacek Furmankiewicz wrote: > > Are there default values that we cal rely on to be > consistent, or should we > > always specify it explicitly. > > > AFAIK distributed Erlang VMs use random highports by default > > > Are there any port ranges that are recommended (i.e. I know > below 1024 > > usually requires root access, etc.) > > > No recommendation I know of. The extent of the range also > depends on how > many distributed nodes you plan on connecting (every node will > then hold > a connection to every other node which could require quite a > bit of > ports; N*(N-1)/2 ports to be precise). > > No, it needs only N-1 ports on each of N node. Your formula is for > number of connections which is irrelevant here. Oops you are right, I mixed that up -- sorry. > > > > > > > > Thanks > > > > On Wed, Jul 29, 2009 at 9:57 AM, Peter Sabaini > wrote: > > > > > On Wed, 2009-07-29 at 09:42 -0400, Jacek Furmankiewicz > wrote: > > > > Is there some documentation as to which ports are used > internally by > > > Erlang > > > > OTP? > > > > > > For OTP, this is configurable, see > > > > > > {inet_dist_listen_min, First} > > > {inet_dist_listen_max, Last} > > > > > > in the kernel reference docs > > > ( http://erlang.org/doc/man/kernel_app.html ) > > > > > > EPMD listens on port 4369 by default > > > > > > > I need to document it for our operations team before > they move the app > > > into > > > > production. Our data centers are very hardened and all > ports are closed > > > by > > > > defaults. > > > > Only those specifically requested are opened. > > > > > > > > I guess I would need to know the ports used by EPMD and > OTP for all the > > > > rpc:_ calls...or any other ports that are required for > an OTP app to > > > > function. > > > > > > > > Thanks, > > > > Jacek > > > > > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try Good Data now for free: www.gooddata.com -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From kevin@REDACTED Wed Jul 29 23:44:38 2009 From: kevin@REDACTED (Kevin A. Smith) Date: Wed, 29 Jul 2009 17:44:38 -0400 Subject: [erlang-questions] New IRC Channel: #erlang-otp In-Reply-To: <4ac8254d0907291434l33127605nc4ec457373822cc4@mail.gmail.com> References: <4ac8254d0907291434l33127605nc4ec457373822cc4@mail.gmail.com> Message-ID: <1FD93F6A-7C6C-475C-B70A-D31EFE5C8EA6@hypotheticalabs.com> The problem is a social one, I think. There are a few people on #erlang who come across as pretty hostile, especially to newbies. I personal know of at least 8 experienced Erlangers who avoid the channel because of the environment there. There's no clear op on the channel so no real ability to fix the issue, at least as far as I can tell. Me and a few others decided that creating a new channel was the only way to route around the problem. Our hope is that the channel will provide a more friendly IRC face for the community and give people a place to hang out and exchange ideas and get help. --Kevin On Jul 29, 2009, at 5:34 PM, Tuncer Ayaz wrote: > On Wed, Jul 29, 2009 at 11:24 PM, Kevin A. > Smith wrote: >> * Have an Erlang question you need answered quick? >> >> * Wishing you had a friendly place to hang out online with other >> Erlangers? >> >> * Tired of either the silence or trolling on #erlang? > > What's the motivation? > Is #erlang owned by someone else and we can't control moderation? > Or is that we wanted to have second and different channel? > >> If you answered yes to any of these consider hanging out in #erlang- >> otp, the >> friendly Erlang IRC channel. Erlang newbies are especially welcome! > > #erlang-otp on freenode (irc.freenode.net) > > before anyone asks :) From tuncer.ayaz@REDACTED Thu Jul 30 00:16:05 2009 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Thu, 30 Jul 2009 00:16:05 +0200 Subject: [erlang-questions] New IRC Channel: #erlang-otp In-Reply-To: <1FD93F6A-7C6C-475C-B70A-D31EFE5C8EA6@hypotheticalabs.com> References: <4ac8254d0907291434l33127605nc4ec457373822cc4@mail.gmail.com> <1FD93F6A-7C6C-475C-B70A-D31EFE5C8EA6@hypotheticalabs.com> Message-ID: <4ac8254d0907291516u75bc36bck1c2ef1088757c9b@mail.gmail.com> On Wed, Jul 29, 2009 at 11:44 PM, Kevin A. Smith wrote: > The problem is a social one, I think. There are a few people on #erlang who > come across as pretty hostile, especially to newbies. I personal know of at > least 8 experienced Erlangers who avoid the channel because of the > environment there. There's no clear op on the channel so no real ability to > fix the issue, at least as far as I can tell. I don't know freenode's policies but I think we can fix the issues if someone with an @erlang.org/@erix.ericsson address contacts the freenode admins/ops. BTW, do you have any plans how to keep the climate in the new channel healthy besides banning/kicking? > Me and a few others decided that creating a new channel was the only way to > route around the problem. Our hope is that the channel will provide a more > friendly IRC face for the community and give people a place to hang out and > exchange ideas and get help. The pragmatic approach might be the right thing to do at the moment :). > --Kevin > On Jul 29, 2009, at 5:34 PM, Tuncer Ayaz wrote: > >> On Wed, Jul 29, 2009 at 11:24 PM, Kevin A. >> Smith wrote: >>> >>> * Have an Erlang question you need answered quick? >>> >>> * Wishing you had a friendly place to hang out online with other >>> Erlangers? >>> >>> * Tired of either the silence or trolling on #erlang? >> >> What's the motivation? >> Is #erlang owned by someone else and we can't control moderation? >> Or is that we wanted to have second and different channel? >> >>> If you answered yes to any of these consider hanging out in #erlang-otp, >>> the >>> friendly Erlang IRC channel. Erlang newbies are especially welcome! >> >> #erlang-otp on freenode (irc.freenode.net) >> >> before anyone asks :) > > From lemonkandy@REDACTED Thu Jul 30 02:25:18 2009 From: lemonkandy@REDACTED (Andy Kilroy) Date: Wed, 29 Jul 2009 17:25:18 -0700 (PDT) Subject: Nested for-loops, there has to be a better way to do this In-Reply-To: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> Message-ID: <7649d93f-45d4-4af7-9062-deeae218b825@o15g2000yqm.googlegroups.com> > Is there a better way to do this? Anyone would like to demonstrate > it? Recommend you learn about the lists:foldl and lists:foldr functions. Whenever I'm about to perform iterations over lists I always think of using these. It means that all you have to do is think 'What action do I want to perform in each iterative step?'; supply this as the function parameter to foldl (or foldr, whichever's appropriate). > The idea was to be able to pass in a > list [5, 5, 10] (for example) and go through it all like 3 dimensional > array. Hmm, I could only figure out what you wanted to do by running your program. I've coded a solution below, but bear in mind I've redefined your problem somewhat, to keep the solution fairly generic. Instead of printing to the terminal, I've stated that the loop must return a result. ---------------- -module(looper). -export([loop/3, nestedLooping/3]). loop(Acc, SideEffect, IntList) -> Exploded = explodeElements(IntList), nestedLooping(Acc, SideEffect, Exploded). nestedLooping(Acc, _SideEffect, []) -> Acc; nestedLooping(Acc, SideEffect, [InnerList | TheRest]) -> Func = fun(Elem, Accum) -> SideEffect(Elem, nestedLooping(Accum, SideEffect, TheRest)) end, lists:foldl(Func, Acc, InnerList). explodeElements([]) -> []; explodeElements([X | IntList]) -> [lists:seq(1, X) | explodeElements(IntList)]. ---------------- The clever work's done in the nestedLooping function. Tests are below, showing you how I use the loop definition above. I've tried to format the output list so that it's bit easier to spot how it corresponds with the output from your program. ---------------- empty_int_list_does_nothing(_Config) -> [] = looper:loop([], fun my_callback/2, []). two_dimen_iteration(_Config) -> [2, 4, 3, 2, 1, 1, 4, 3, 2, 1] = looper:loop([], fun my_callback/2, [2, 4]). three_dimen_iteration(_Config) -> [2, 4, 5, 4, 3, 2, 1, 3, 5, 4, 3, 2, 1, 2, 5, 4, 3, 2, 1, 1, 5, 4, 3, 2, 1, 1, 4, 5, 4, 3, 2, 1, 3, 5, 4, 3, 2, 1, 2, 5, 4, 3, 2, 1, 1, 5, 4, 3, 2, 1 ] = looper:loop([], fun my_callback/2, [2, 4, 5]). my_callback(Element, Accumulator) -> [Element | Accumulator]. ---------------- Hope that's helpful, Andy From lemonkandy@REDACTED Thu Jul 30 02:30:55 2009 From: lemonkandy@REDACTED (Andy Kilroy) Date: Wed, 29 Jul 2009 17:30:55 -0700 (PDT) Subject: Nested for-loops, there has to be a better way to do this In-Reply-To: <7649d93f-45d4-4af7-9062-deeae218b825@o15g2000yqm.googlegroups.com> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <7649d93f-45d4-4af7-9062-deeae218b825@o15g2000yqm.googlegroups.com> Message-ID: <706c3714-acf4-4391-a589-627921f0d278@i6g2000yqj.googlegroups.com> Oops, sorry for the badly judged textwidth :-S Andy From zerthurd@REDACTED Thu Jul 30 05:47:26 2009 From: zerthurd@REDACTED (Maxim Treskin) Date: Thu, 30 Jul 2009 10:47:26 +0700 Subject: [erlang-questions] New IRC Channel: #erlang-otp In-Reply-To: <4ac8254d0907291516u75bc36bck1c2ef1088757c9b@mail.gmail.com> References: <4ac8254d0907291434l33127605nc4ec457373822cc4@mail.gmail.com> <1FD93F6A-7C6C-475C-B70A-D31EFE5C8EA6@hypotheticalabs.com> <4ac8254d0907291516u75bc36bck1c2ef1088757c9b@mail.gmail.com> Message-ID: Why IRC? Why not Jabber? There is conference erlang@REDACTED BTW, jabber.org runs ejabberd :) -- Maxim Treskin From ttmrichter@REDACTED Thu Jul 30 07:11:30 2009 From: ttmrichter@REDACTED (ttmrichter@REDACTED) Date: Thu, 30 Jul 2009 05:11:30 +0000 Subject: [erlang-questions] New IRC Channel: #erlang-otp In-Reply-To: <4ac8254d0907291434l33127605nc4ec457373822cc4@mail.gmail.com> Message-ID: <0016e64bdff02741b0046fe556d4@google.com> On Jul 30, 2009 5:34am, Tuncer Ayaz wrote: > > * Have an Erlang question you need answered quick? > > * Wishing you had a friendly place to hang out online with other > Erlangers? > > * Tired of either the silence or trolling on #erlang? > What's the motivation? Well, looking at the last five hours of activity on #erlang, I saw three questions asked, no responses. (To be fair I did catch the tail end of what appeared to be a Q&A session upon logging in.) This dearth of activity is in spite of the way-too-many-people-for-me-to-count logged in. I also know that the one and only person I've ever felt the need to gag on Freenode was an Erlanger on #erlang. I'm on about 20 channels right now and have been so for years (with the specific channels varying, of course, with my interests). And despite all that I've only once felt the desire to gag someone, and that was on #erlang. (And the guy being gagged was being gagged because he was being a jerk to new users, no less. Not exactly the face any community wants to put forth, right?) Oh, the guy also cited Eric Naggum. As a good thing. From samb-bulk@REDACTED Thu Jul 30 07:37:38 2009 From: samb-bulk@REDACTED (Sam Bobroff) Date: Thu, 30 Jul 2009 15:37:38 +1000 Subject: Odd behavior with remote shells and command pipelines Message-ID: <4A7131A2.4030708@m5networks.com.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi everyone, I've noticed some odd behavior when using remote shells. Can anyone explain why they happen, and if they are bugs or not? The first thing is that it appears as if piping commands into erl causes Erlang to use a different shell process. Here's an example: In one terminal run: erl -sname bar Then in another terminal compare running: erl -sname foo -remsh bar@REDACTED And typing 'node().' into the shell (I get 'bar@REDACTED') to running this on the command line: echo "node()." | erl -sname foo -remsh bar@REDACTED (I get 'foo@REDACTED' and then "*** Terminating erlang ('foo@REDACTED')".) The second odd thing is that by using a pipe and rpc:call to start a remote shell it is possible to cause the remote node to shut down. For example: echo | erl -sname foo -noshell -eval "rpc:call('bar@REDACTED', shell, start, [])." And the node "bar@REDACTED" will shut down. This still happens even if bar is running restricted shells that do not allow q() or halt() to be called. It does not happen when using "-remsh". I'm using R13B01 on Ubuntu Linux. Sam. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKcTGim97/UHSa/AQRAunBAJ97VjEOcpqGeW2N4m7SrmRjEf5fUgCeJRP0 pQT1M0P270CQ1cWstwunaFY= =OYLB -----END PGP SIGNATURE----- From steven.charles.davis@REDACTED Thu Jul 30 09:06:36 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 30 Jul 2009 00:06:36 -0700 (PDT) Subject: Any HTTP 1.0 clients out there? In-Reply-To: <483238.79562.qm@web111405.mail.gq1.yahoo.com> References: <990c88ad-c0a5-44ff-a87e-3b5c1c17ce77@s6g2000vbp.googlegroups.com> <483238.79562.qm@web111405.mail.gq1.yahoo.com> Message-ID: <52415048-219d-499c-b397-3a3428ee1162@a13g2000yqc.googlegroups.com> On Jul 20, 8:25?am, Thomas Lindgren wrote: > This is perhaps a bit late to the party, but nginx proxies useHTTP/1.0to the backend. This is indeed truly bad news. At first I thought that I wasn't supporting OPTIONS correctly or that I had misconfigured Nginx. Over the last few days, I've been trying to get advice from those that know Nginx, but so far the message I am receiving from them is that Nginx upstream not only connects over 1.0, but cannot be configured to use 1.1. So... Ugh.I have not been able to find any reason *why* 1.0 is used by Nginx, and it seems like a purely arbitrary design decision. /sd From thijsterlouw@REDACTED Thu Jul 30 10:40:51 2009 From: thijsterlouw@REDACTED (Thijs Terlouw) Date: Thu, 30 Jul 2009 16:40:51 +0800 Subject: How do I create a specific ref() ? Message-ID: <4183f4b90907300140l116a737bmd46fa07b69bd8d15@mail.gmail.com> How do I create a new ref() for a specific value (based on an old #Ref)? I know I can use "erlang:make_ref() -> ref()" to get a new almost unique Reference. I can also use "erlang:ref_to_list(Ref) -> string()" to serialize a Ref to a String. It seems there is no "list_to_ref(String) -> ref()" function though, what's the reason for this? My use case: I am writing an async port program which is called from a gen_server() and I would like to prevent storing the a mapping {Seq -> From} in ETS. If I could just pass the serialized Pid + serialized Ref to the port program, my problems would be solved, because the port program could return the Pid + Ref and I deserialize them again. Cheers, Thijs From bengt.kleberg@REDACTED Thu Jul 30 10:57:58 2009 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 30 Jul 2009 10:57:58 +0200 Subject: [erlang-questions] How do I create a specific ref() ? In-Reply-To: <4183f4b90907300140l116a737bmd46fa07b69bd8d15@mail.gmail.com> References: <4183f4b90907300140l116a737bmd46fa07b69bd8d15@mail.gmail.com> Message-ID: <1248944278.4821.27.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, Would it be possible for you to always use the serialised ref() in your program, instead of the original, unserialised, ref()? Then you would never have to deserialize it. bengt On Thu, 2009-07-30 at 16:40 +0800, Thijs Terlouw wrote: > How do I create a new ref() for a specific value (based on an old #Ref)? > > I know I can use "erlang:make_ref() -> ref()" to get a new almost unique > Reference. > I can also use "erlang:ref_to_list(Ref) -> string()" to serialize a Ref to a > String. > It seems there is no "list_to_ref(String) -> ref()" function though, what's > the reason for this? > > My use case: > I am writing an async port program which is called from a gen_server() and I > would like to prevent storing the a mapping {Seq -> From} in ETS. If I could > just pass the serialized Pid + serialized Ref to the port program, my > problems would be solved, because the port program could return the Pid + > Ref and I deserialize them again. > > Cheers, > Thijs From baryluk@REDACTED Thu Jul 30 11:06:03 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Thu, 30 Jul 2009 11:06:03 +0200 Subject: [erlang-questions] How do I create a specific ref() ? In-Reply-To: <4183f4b90907300140l116a737bmd46fa07b69bd8d15@mail.gmail.com> References: <4183f4b90907300140l116a737bmd46fa07b69bd8d15@mail.gmail.com> Message-ID: <1248944763.7447.22.camel@sredniczarny> Dnia 2009-07-30, czw o godzinie 16:40 +0800, Thijs Terlouw pisze: > How do I create a new ref() for a specific value (based on an old #Ref)? > > I know I can use "erlang:make_ref() -> ref()" to get a new almost unique > Reference. > I can also use "erlang:ref_to_list(Ref) -> string()" to serialize a Ref to a > String. > It seems there is no "list_to_ref(String) -> ref()" function though, what's > the reason for this? > > My use case: > I am writing an async port program which is called from a gen_server() and I > would like to prevent storing the a mapping {Seq -> From} in ETS. If I could > just pass the serialized Pid + serialized Ref to the port program, my > problems would be solved, because the port program could return the Pid + > Ref and I deserialize them again. > > Cheers, > Thijs If think this is because after you serialize it to list, and original Ref isn't used anymore, make_ref() can return the same value. Kind of garbage collection (ref counting in VM?). If you would deserialize it back using list_to_ref, there can be logical conflict, you will have ref(), which is not unique anymore. So i think you need to remember refs directly in VM. -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From rtrlists@REDACTED Thu Jul 30 11:46:34 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 30 Jul 2009 10:46:34 +0100 Subject: [erlang-questions] Nested for-loops, there has to be a better way to do this In-Reply-To: <8b9ee55b0907291408l44c57c3ep8c60e3d0db13f848@mail.gmail.com> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <63418.194.88.55.211.1248848724.squirrel@localhost> <4A7077F6.10508@dcc.fc.up.pt> <4A709BB3.5010701@tmit.bme.hu> <772561A5-A67A-4738-99DA-396CF2ED6C2B@gmail.com> <4A709F5D.2060401@tmit.bme.hu> <4A70AF23.1090404@dcc.fc.up.pt> <8b9ee55b0907291408l44c57c3ep8c60e3d0db13f848@mail.gmail.com> Message-ID: <6a3ae47e0907300246j104f5973x854a004f87bea5a4@mail.gmail.com> On Wed, Jul 29, 2009 at 10:08 PM, Fred Hebert (MononcQc) wrote: > I find nested lists comprehension to be the easiest to deal with when > representing nested loops: > > [[ Fn(I,J) || J <- GenExp2 ] || I <- GenEXp1 ]. > > or indented in a different way: > > [[[ Fn(I, J, K) || K <- GenExp3 ] > || J <- GenExp2 ] > || I <- GenExp1 ]. > > This gives a nested list. If that's not what you want, you could use > lists:flatten/1 to change that, but overall I find this form to be the > cleanest (visually speaking). > I may be missing something obvious, but ... 1> [ {A,B,C,A+B+C} || A<-lists:seq(1,5), B<-lists:seq(1,5), C<-lists:seq(1,10) ]. [{1,1,1,3}, {1,1,2,4}, {1,1,3,5}, {1,1,4,6}, {1,1,5,7}, {1,1,6,8}, {1,1,7,9}, {1,1,8,10}, {1,1,9,11}, {1,1,10,12}, {1,2,1,4}, {1,2,2,5}, {1,2,3,6}, {1,2,4,7}, {1,2,5,8}, {1,2,6,9}, {1,2,7,10}, {1,2,8,11}, {1,2,9,12}, {1,2,10,13}, {1,3,1,5}, {1,3,2,6}, {1,3,3,7}, {1,3,4,8}, {1,3,5,9}, {1,3,6,...}, {1,3,...}, {1,...}, {...}|...] 2> Robby From s.merle@REDACTED Thu Jul 30 13:17:23 2009 From: s.merle@REDACTED (Sebastien Merle) Date: Thu, 30 Jul 2009 13:17:23 +0200 Subject: Efficient way to read data from a socket while calculating instant bitrate Message-ID: <102d3f890907300417j32ae145ahedfd9aa17f9160ab@mail.gmail.com> Hi, I am reading data from a socket, and I want to be able to calculate the instant bitrate over a period of time. I am currently doing something like: download(Sock, Monitor) -> erlang:send_after(?MONITORING_PERIOD, self(), {monitor}), download(Sock, Monitor, erlang:now(), 0). download(Sock, Monitor, LastTime, BytesDelta) -> ok = inet:setopts(Sock, [{active, once}]), receive {tcp, Sock, Data} -> download(Sock, Monitor, LastTime, BytesDelta + size(Data)); {tcp_closed, Sock} -> ok; {monitor} -> Now = erlang:now(), TimeDelta = timer:now_diff(Now, LastTime), Rate = (BytesDelta * 1000000) div TimeDelta, Monitor ! {rate, self(), Rate}, erlang:send_after(?MONITORING_PERIOD, self(), {monitor}), download(Sock, Monitor, Now, 0) end. This is working, but with very high bitrates it seems rather inefficient because it receives a lot of small messages. It request the packets one by one by calling inet:setopts because it does some client-side rate control at the same time (not included in the sample code). Any suggestions to make this more efficient ? Regards, Sebastien Merle. From mononcqc@REDACTED Thu Jul 30 13:57:07 2009 From: mononcqc@REDACTED (Fred Hebert (MononcQc)) Date: Thu, 30 Jul 2009 07:57:07 -0400 Subject: [erlang-questions] Nested for-loops, there has to be a better way to do this In-Reply-To: <6a3ae47e0907300246j104f5973x854a004f87bea5a4@mail.gmail.com> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <4A7077F6.10508@dcc.fc.up.pt> <4A709BB3.5010701@tmit.bme.hu> <772561A5-A67A-4738-99DA-396CF2ED6C2B@gmail.com> <4A709F5D.2060401@tmit.bme.hu> <4A70AF23.1090404@dcc.fc.up.pt> <8b9ee55b0907291408l44c57c3ep8c60e3d0db13f848@mail.gmail.com> <6a3ae47e0907300246j104f5973x854a004f87bea5a4@mail.gmail.com> Message-ID: <8b9ee55b0907300457v7a28a4dfpa2f1800e7bb2c4cb@mail.gmail.com> On Thu, Jul 30, 2009 at 5:46 AM, Robert Raschke wrote: > On Wed, Jul 29, 2009 at 10:08 PM, Fred Hebert (MononcQc) > wrote: > >> I find nested lists comprehension to be the easiest to deal with when >> representing nested loops: >> >> [[ Fn(I,J) || J <- GenExp2 ] || I <- GenEXp1 ]. >> >> or indented in a different way: >> >> ?[[[ Fn(I, J, K) || K <- GenExp3 ] >> ? ? ? ? ? ? ? ? || J <- GenExp2 ] >> ? ? ? ? ? ? ? ? || I <- GenExp1 ]. >> >> This gives a nested list. If that's not what you want, you could use >> lists:flatten/1 to change that, but overall I find this form to be the >> cleanest (visually speaking). >> > > I may be missing something obvious, but ... > > 1> [ {A,B,C,A+B+C} || A<-lists:seq(1,5), B<-lists:seq(1,5), > C<-lists:seq(1,10) ]. > [{1,1,1,3}, > ?{1,1,2,4}, > ?{1,1,3,5}, > ?{1,1,4,6}, > ?{1,1,5,7}, > ?{1,1,6,8}, > ?{1,1,7,9}, > ?{1,1,8,10}, > ?{1,1,9,11}, > ?{1,1,10,12}, > ?{1,2,1,4}, > ?{1,2,2,5}, > ?{1,2,3,6}, > ?{1,2,4,7}, > ?{1,2,5,8}, > ?{1,2,6,9}, > ?{1,2,7,10}, > ?{1,2,8,11}, > ?{1,2,9,12}, > ?{1,2,10,13}, > ?{1,3,1,5}, > ?{1,3,2,6}, > ?{1,3,3,7}, > ?{1,3,4,8}, > ?{1,3,5,9}, > ?{1,3,6,...}, > ?{1,3,...}, > ?{1,...}, > ?{...}|...] > 2> > > Robby > Haha, yes. Total brain fart on my end. That's what I should have posted instead. From ovidiudeac@REDACTED Thu Jul 30 14:29:43 2009 From: ovidiudeac@REDACTED (Ovidiu Deac) Date: Thu, 30 Jul 2009 15:29:43 +0300 Subject: Slow UTP packets Message-ID: <804cd4020907300529x262b052pb5013f31ad86087c@mail.gmail.com> Hi everybody, I want to do some speed tests for parsing packets and I wrote a small program which receives data on a multicast address and writes it to a file. While this application is running I measure the growing speed of the output file to get an idea about the speed. The problem is that the output speed is getting slower as the application runs. Initially the output file grows with about 1MB/s then it slows down to 700KB/s 500Kb/s... and so on until it reaches 1KB/s. Any idea why? I know that the data is sent to the multicast address with 1MB/s and also I have a python version which writes to the file with the constant speed of 1MB/s. So there has to be something wrong with my erlang program. Note that it's basically my first erlang application so I might done something really stupid. I'm using Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] I have this problem both with and without hipe compilation. See the code below. Thanks in advance, ovidiu -module(unpacker). -export([start/3]). start(OutputFile, Ip, Port) -> {ok, Socket} = gen_udp:open(Port, [binary, {active, true}, {reuseaddr, true},{ip, Ip}, {add_membership, {Ip, {0,0,0,0}}}]), {ok, File} = file:open(OutputFile, write), parseNextPacket(Socket, File). parseNextPacket(Socket, File) -> receive {udp, Socket, _Host, _Port, Data}-> io:format(File, "Packet: ~p", [Data]), parseNextPacket(Socket, File); Any -> io:format("Unknown message: ~p~n", [Any]), io:format(File, "Unknown message: ~p", [Any]), parseNextPacket(Socket, File) end. From rtrlists@REDACTED Thu Jul 30 14:48:18 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 30 Jul 2009 13:48:18 +0100 Subject: xmerl XSD validation vs xmerl_xs:select and namespaces Message-ID: <6a3ae47e0907300548k177f8178g41afa154b4e55136@mail.gmail.com> Hi, I'm a bit confused by the xmerl XSD validation. It seems that trying to validate internal elements sometimes fails for reasons that are not clear to me. When using the purchase order example from http://www.w3.org/TR/xmlschema-0/with the changes to make it use namespaces: And a simple XML file that goes with this: Alice Smith 123 Maple Street Mill Valley CA 90952 Robert Smith 8 Oak Avenue Old Town PA 95819 Hurry, my lawn is going wild Baby Monitor 1 39.98 1999-05-21 Everything validates just fine: Erlang (BEAM) emulator version 5.6.5 [smp:2] [async-threads:0] Eshell V5.6.5 (abort with ^G) 1> rr(xmerl). [xmerl_event,xmerl_fun_states,xmerl_scanner,xmlAttribute, xmlComment,xmlContext,xmlDecl,xmlDocument,xmlElement, xmlNamespace,xmlNode,xmlNsNode,xmlObj,xmlPI,xmlText] 2> {X01, _} = xmerl_scan:file("po.xml"). {#xmlElement{ name = purchaseOrder,expanded_name = purchaseOrder, nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1',nodes = []}, parents = [],pos = 1, attributes = [#xmlAttribute{ name = xmlns,expanded_name = [],nsinfo = [],namespace = [], parents = [],pos = 1,language = [], value = "http://www.example.com/PO1",normalized = false}, #xmlAttribute{ name = orderDate,expanded_name = [],nsinfo = [], namespace = [],parents = [],pos = 2,language = [], value = "1999-10-20",normalized = false}], content = [#xmlText{ parents = [{purchaseOrder,1}], pos = 1,language = [],value = "\n ",type = text}, #xmlElement{ name = shipTo,expanded_name = shipTo,nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1',nodes = []}, parents = [{purchaseOrder,1}], pos = 2, attributes = [#xmlAttribute{ name = country,expanded_name = [],nsinfo = [], namespace = [],parents = [],pos = 1,language = [],...}], content = [#xmlText{ parents = [{shipTo,2},{purchaseOrder,1}], pos = 1,language = [],value = "\n ",type = text}, #xmlElement{ name = name,expanded_name = name,nsinfo = [], namespace = #xmlNamespace{...}, parents = [...],...}, #xmlText{ parents = [{shipTo,2},{purchaseOrder,...}], pos = 3,language = [], value = [...],...}, #xmlElement{ name = street,expanded_name = street,nsinfo = [],...}, #xmlText{parents = [{...}|...],pos = 5,...}, #xmlElement{name = city,...}, #xmlText{...}, {...}|...], language = [],xmlbase = ".",elementdef = undeclared}, #xmlText{ parents = [{purchaseOrder,1}], pos = 3,language = [],value = "\n ",type = text}, #xmlElement{ name = billTo,expanded_name = billTo,nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1',nodes = []}, parents = [{purchaseOrder,1}], pos = 4, attributes = [#xmlAttribute{ name = country,expanded_name = [],nsinfo = [], namespace = [],parents = [],...}], content = [#xmlText{ parents = [{billTo,4},{purchaseOrder,...}], pos = 1,language = [], value = [...],...}, #xmlElement{ name = name,expanded_name = name,nsinfo = [],...}, #xmlText{parents = [{...}|...],pos = 3,...}, #xmlElement{name = street,...}, #xmlText{...}, {...}|...], language = [],xmlbase = ".",elementdef = undeclared}, #xmlText{ parents = [{purchaseOrder,1}], pos = 5,language = [],value = "\n ",type = text}, #xmlElement{ name = comment,expanded_name = comment,nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1',nodes = []}, parents = [{purchaseOrder,1}], pos = 6,attributes = [], content = [#xmlText{parents = [{...}|...],pos = 1,...}], language = [],xmlbase = ".",elementdef = undeclared}, #xmlText{ parents = [{purchaseOrder,1}], pos = 7,language = [],value = "\n ",type = text}, #xmlElement{ name = items,expanded_name = items,nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1',nodes = []}, parents = [{purchaseOrder,1}], pos = 8,attributes = [], content = [#xmlText{...},{...}|...], language = [], xmlbase = [...],...}, #xmlText{ parents = [{purchaseOrder,1}], pos = 9,language = [],value = "\n",type = text}], language = [],xmlbase = ".",elementdef = undeclared}, []} 3> xmerl_xsd:process_validate("po.xsd", X01). {#xmlElement{ name = purchaseOrder,expanded_name = purchaseOrder, nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1',nodes = []}, parents = [],pos = 1, attributes = [#xmlAttribute{ name = xmlns,expanded_name = [],nsinfo = [],namespace = [], parents = [],pos = 1,language = [], value = "http://www.example.com/PO1",normalized = false}, #xmlAttribute{ name = orderDate,expanded_name = [],nsinfo = [], namespace = [],parents = [],pos = 2,language = [], value = "1999-10-20",normalized = false}], content = [#xmlText{ parents = [{purchaseOrder,1}], pos = 1,language = [],value = "\n ",type = text}, #xmlElement{ name = shipTo,expanded_name = shipTo,nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1',nodes = []}, parents = [{purchaseOrder,1}], pos = 2, attributes = [#xmlAttribute{ name = country,expanded_name = [],nsinfo = [], namespace = [],parents = [],pos = 1,language = [],...}], content = [#xmlText{ parents = [{shipTo,2},{purchaseOrder,1}], pos = 1,language = [],value = "\n ",type = text}, #xmlElement{ name = name,expanded_name = name,nsinfo = [], namespace = #xmlNamespace{...}, parents = [...],...}, #xmlText{ parents = [{shipTo,2},{purchaseOrder,...}], pos = 3,language = [], value = [...],...}, #xmlElement{ name = street,expanded_name = street,nsinfo = [],...}, #xmlText{parents = [{...}|...],pos = 5,...}, #xmlElement{name = city,...}, #xmlText{...}, {...}|...], language = [],xmlbase = ".",elementdef = undeclared}, #xmlText{ parents = [{purchaseOrder,1}], pos = 3,language = [],value = "\n ",type = text}, #xmlElement{ name = billTo,expanded_name = billTo,nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1',nodes = []}, parents = [{purchaseOrder,1}], pos = 4, attributes = [#xmlAttribute{ name = country,expanded_name = [],nsinfo = [], namespace = [],parents = [],...}], content = [#xmlText{ parents = [{billTo,4},{purchaseOrder,...}], pos = 1,language = [], value = [...],...}, #xmlElement{ name = name,expanded_name = name,nsinfo = [],...}, #xmlText{parents = [{...}|...],pos = 3,...}, #xmlElement{name = street,...}, #xmlText{...}, {...}|...], language = [],xmlbase = ".",elementdef = undeclared}, #xmlText{ parents = [{purchaseOrder,1}], pos = 5,language = [],value = "\n ",type = text}, #xmlElement{ name = comment,expanded_name = comment,nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1',nodes = []}, parents = [{purchaseOrder,1}], pos = 6,attributes = [], content = [#xmlText{parents = [{...}|...],pos = 1,...}], language = [],xmlbase = ".",elementdef = undeclared}, #xmlText{ parents = [{purchaseOrder,1}], pos = 7,language = [],value = "\n ",type = text}, #xmlElement{ name = items,expanded_name = items,nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1',nodes = []}, parents = [{purchaseOrder,1}], pos = 8,attributes = [], content = [#xmlText{...},{...}|...], language = [], xmlbase = [...],...}, #xmlText{ parents = [{purchaseOrder,1}], pos = 9,language = [],value = "\n",type = text}], language = [],xmlbase = ".",elementdef = undeclared}, {xsd_state,"po.xsd", [138828883239791335015413488673396043202], false,".",[],[],[],qualified,unqualified,undefined, 'http://www.example.com/PO1', [{"po",'http://www.example.com/PO1'}, {"xsd",'http://www.w3.org/2001/XMLSchema'}, {"xml",'http://www.w3.org/XML/1998/namespace'}], [{'http://www.example.com/PO1', [{"xsd",'http://www.w3.org/2001/XMLSchema'}, {"po",'http://www.example.com/PO1'}]}], [{"#this#","po.xsd",'http://www.example.com/PO1'}, {"xml",[],'http://www.w3.org/XML/1998/namespace'}], 28685,false,false,[],[],#Fun,[],0,[], [],[],[],...}} But if I edit the XML to have some prefix namespaced elements around the outside: Alice Smith 123 Maple Street Mill Valley CA 90952 Robert Smith 8 Oak Avenue Old Town PA 95819 Hurry, my lawn is going wild Baby Monitor 1 39.98 1999-05-21 And then I retrieve the default namespaced purchaseOrder element using xmerl_xs:select/2, then the XSD validation fails: 4> {X02, _} = xmerl_scan:file("po.xml"). {#xmlElement{ name = 'p:foo',expanded_name = 'p:foo', nsinfo = {"p","foo"}, namespace = #xmlNamespace{default = [],nodes = [{"p",foobar}]}, parents = [],pos = 1, attributes = [#xmlAttribute{ name = 'xmlns:p',expanded_name = [],nsinfo = [], namespace = {"xmlns","p"}, parents = [],pos = 1,language = [],value = "foobar", normalized = false}], content = [#xmlElement{ name = 'p:bar',expanded_name = 'p:bar', nsinfo = {"p","bar"}, namespace = #xmlNamespace{default = [],nodes = [{"p",foobar}]}, parents = [{'p:foo',1}], pos = 1,attributes = [], content = [#xmlText{ parents = [{'p:bar',1},{'p:foo',1}], pos = 1,language = [],value = "\n",type = text}, #xmlElement{ name = purchaseOrder,expanded_name = purchaseOrder, nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1',...}, parents = [{...}|...], pos = 2,...}, #xmlText{ parents = [{'p:bar',1},{'p:foo',1}], pos = 3,language = [],value = "\n",type = text}], language = [],xmlbase = ".",elementdef = undeclared}], language = [],xmlbase = ".",elementdef = undeclared}, []} 5> [X03] = xmerl_xs:select("/p:foo/p:bar/*", X02). [#xmlElement{ name = purchaseOrder,expanded_name = purchaseOrder, nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1', nodes = [{"p",foobar}]}, parents = [{'p:bar',1},{'p:foo',1}], pos = 2, attributes = [#xmlAttribute{ name = xmlns,expanded_name = [],nsinfo = [],namespace = [], parents = [],pos = 1,language = [], value = "http://www.example.com/PO1",normalized = false}, #xmlAttribute{ name = orderDate,expanded_name = [],nsinfo = [], namespace = [],parents = [],pos = 2,language = [], value = "1999-10-20",normalized = false}], content = [#xmlText{ parents = [{purchaseOrder,2},{'p:bar',1},{'p:foo',1}], pos = 1,language = [],value = "\n ",type = text}, #xmlElement{ name = shipTo,expanded_name = shipTo,nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1', nodes = [{"p",foobar}]}, parents = [{purchaseOrder,2},{'p:bar',1},{'p:foo',1}], pos = 2, attributes = [#xmlAttribute{ name = country,expanded_name = [],nsinfo = [], namespace = [],parents = [],pos = 1,language = [],...}], content = [#xmlText{ parents = [{shipTo,2}, {purchaseOrder,2}, {'p:bar',1}, {'p:foo',...}], pos = 1,language = [],value = "\n ",type = text}, #xmlElement{ name = name,expanded_name = name,nsinfo = [], namespace = #xmlNamespace{...}, parents = [...],...}, #xmlText{ parents = [{shipTo,2},{purchaseOrder,...},{...}|...], pos = 3,language = [], value = [...],...}, #xmlElement{ name = street,expanded_name = street,nsinfo = [],...}, #xmlText{parents = [{...}|...],pos = 5,...}, #xmlElement{name = city,...}, #xmlText{...}, {...}|...], language = [],xmlbase = ".",elementdef = undeclared}, #xmlText{ parents = [{purchaseOrder,2},{'p:bar',1},{'p:foo',1}], pos = 3,language = [],value = "\n ",type = text}, #xmlElement{ name = billTo,expanded_name = billTo,nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1', nodes = [{"p",foobar}]}, parents = [{purchaseOrder,2},{'p:bar',1},{'p:foo',1}], pos = 4, attributes = [#xmlAttribute{ name = country,expanded_name = [],nsinfo = [], namespace = [],parents = [],...}], content = [#xmlText{ parents = [{billTo,4},{purchaseOrder,...},{...}|...], pos = 1,language = [], value = [...],...}, #xmlElement{ name = name,expanded_name = name,nsinfo = [],...}, #xmlText{parents = [{...}|...],pos = 3,...}, #xmlElement{name = street,...}, #xmlText{...}, {...}|...], language = [],xmlbase = ".",elementdef = undeclared}, #xmlText{ parents = [{purchaseOrder,2},{'p:bar',1},{'p:foo',1}], pos = 5,language = [],value = "\n ",type = text}, #xmlElement{ name = comment,expanded_name = comment,nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1', nodes = [{"p",foobar}]}, parents = [{purchaseOrder,2},{'p:bar',1},{'p:foo',1}], pos = 6,attributes = [], content = [#xmlText{parents = [{...}|...],pos = 1,...}], language = [],xmlbase = ".",elementdef = undeclared}, #xmlText{ parents = [{purchaseOrder,2},{'p:bar',1},{'p:foo',1}], pos = 7,language = [],value = "\n ",type = text}, #xmlElement{ name = items,expanded_name = items,nsinfo = [], namespace = #xmlNamespace{ default = 'http://www.example.com/PO1', nodes = [{"p",foobar}]}, parents = [{purchaseOrder,2},{'p:bar',1},{'p:foo',1}], pos = 8,attributes = [], content = [#xmlText{...},{...}|...], language = [], xmlbase = [...],...}, #xmlText{ parents = [{purchaseOrder,2},{'p:bar',1},{'p:foo',1}], pos = 9,language = [],value = "\n",type = text}], language = [],xmlbase = ".",elementdef = undeclared}] 6> xmerl_xsd:process_validate("po.xsd", X03). {error, [{"p:foo[1]/p:bar[1]",xmerl_xsd, {element_not_in_schema, [purchaseOrder, {purchaseOrder,[],foobar}, {schema,undefined,undefined,undefined,[],[],undefined, []}]}}]} Does anyone know why this might be the case? Thanks for any info, Robby From kevin@REDACTED Thu Jul 30 14:47:15 2009 From: kevin@REDACTED (Kevin A. Smith) Date: Thu, 30 Jul 2009 08:47:15 -0400 Subject: [erlang-questions] New IRC Channel: #erlang-otp In-Reply-To: References: <4ac8254d0907291434l33127605nc4ec457373822cc4@mail.gmail.com> <1FD93F6A-7C6C-475C-B70A-D31EFE5C8EA6@hypotheticalabs.com> <4ac8254d0907291516u75bc36bck1c2ef1088757c9b@mail.gmail.com> Message-ID: <8F2D6817-761C-4118-B825-B7F833A485C0@hypotheticalabs.com> We considered XMPP and I think there might be an effort to bridge the IRC channel into a Jabber chat room. We wanted to provide an alternative to the IRC channel so it seemed appropriate to continue to use IRC since that's where the people are. --Kevin On Jul 29, 2009, at 11:47 PM, Maxim Treskin wrote: > Why IRC? Why not Jabber? > There is conference erlang@REDACTED > BTW, jabber.org runs ejabberd :) > > -- > Maxim Treskin From kevin@REDACTED Thu Jul 30 14:52:44 2009 From: kevin@REDACTED (Kevin A. Smith) Date: Thu, 30 Jul 2009 08:52:44 -0400 Subject: [erlang-questions] New IRC Channel: #erlang-otp In-Reply-To: <4ac8254d0907291516u75bc36bck1c2ef1088757c9b@mail.gmail.com> References: <4ac8254d0907291434l33127605nc4ec457373822cc4@mail.gmail.com> <1FD93F6A-7C6C-475C-B70A-D31EFE5C8EA6@hypotheticalabs.com> <4ac8254d0907291516u75bc36bck1c2ef1088757c9b@mail.gmail.com> Message-ID: <8D25D895-EF5B-4707-9B98-07D2E55AF0F8@hypotheticalabs.com> On Jul 29, 2009, at 6:16 PM, Tuncer Ayaz wrote: > On Wed, Jul 29, 2009 at 11:44 PM, Kevin A. > Smith wrote: >> The problem is a social one, I think. There are a few people on >> #erlang who >> come across as pretty hostile, especially to newbies. I personal >> know of at >> least 8 experienced Erlangers who avoid the channel because of the >> environment there. There's no clear op on the channel so no real >> ability to >> fix the issue, at least as far as I can tell. > > I don't know freenode's policies but I think we can fix the issues > if someone with an @erlang.org/@erix.ericsson address contacts the > freenode admins/ops. > > BTW, do you have any plans how to keep the climate in the new > channel healthy besides banning/kicking? It still early days but we're working on drafting up some channel rules and a charter. We'll publicize those when they're further along but I expect fairly simple, common sense rules. For now, we hope that everyone can act like adults and just get along. My personal hope is by removing the fear of attacks for the simplest question people will begin to talk more. It's a social experiment so only time will tell if it works out. --Kevin From steven.charles.davis@REDACTED Thu Jul 30 14:54:11 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 30 Jul 2009 05:54:11 -0700 (PDT) Subject: Any HTTP 1.0 clients out there? In-Reply-To: <52415048-219d-499c-b397-3a3428ee1162@a13g2000yqc.googlegroups.com> References: <990c88ad-c0a5-44ff-a87e-3b5c1c17ce77@s6g2000vbp.googlegroups.com> <483238.79562.qm@web111405.mail.gq1.yahoo.com> <52415048-219d-499c-b397-3a3428ee1162@a13g2000yqc.googlegroups.com> Message-ID: A final followup and closure on this question for anyone following this thread. On Jul 30, 2:06 am, Steve Davis wrote: > So... Ugh.I have not been able to find any reason *why* 1.0 is used by > Nginx, and it seems like a purely arbitrary design decision. Talking to the NginX developers, I have clarified a couple of things: 1) HTTP/1.0 is in use by proxies, some mobile clients, and IE when configured to use a proxy. So 1.0 appears to still account for a non- trivial % of traffic on the web overall. 2) The NginX guys do not think that proxying via 1.0 will have a significant performance impact unless the backend is extremely quick. 3) HTTP/1.1 support for upstream is in the works at NginX in any case, with the motivation of coping with these superfast backends. So it sesms that the answer to my OP is "Yes, there are many 1.0 clients still out there", and it's not the dead protocol that I thought it was at all. Hence, I'm going to be fully correct and abide by the HTTP spec, and keep the support for HTTP/1.0. /sd From thatsnotright@REDACTED Thu Jul 30 14:58:13 2009 From: thatsnotright@REDACTED (Rob Elsner) Date: Thu, 30 Jul 2009 06:58:13 -0600 Subject: [erlang-questions] Slow UTP packets In-Reply-To: <804cd4020907300529x262b052pb5013f31ad86087c@mail.gmail.com> References: <804cd4020907300529x262b052pb5013f31ad86087c@mail.gmail.com> Message-ID: I am doing nearly an identical thing and record a 35Mbps multicast UDP stream (~4MB/s) just fine. The difference is I don't use io:write, I use file:write. I've had the file in raw mode and that works as well. I also use delayed_write, but in either case it works fine. I think you might be hitting an issue using io:write. Is there a chance you can use file:write and see how that performs? Rob On Thu, Jul 30, 2009 at 6:29 AM, Ovidiu Deac wrote: > Hi everybody, > > I want to do some speed tests for parsing packets and I wrote a small > program which receives data on a multicast address and writes it to a > file. While this application is running I measure the growing speed of > the output file to get an idea about the speed. From ovidiudeac@REDACTED Thu Jul 30 15:23:45 2009 From: ovidiudeac@REDACTED (Ovidiu Deac) Date: Thu, 30 Jul 2009 16:23:45 +0300 Subject: [erlang-questions] Slow UTP packets In-Reply-To: References: <804cd4020907300529x262b052pb5013f31ad86087c@mail.gmail.com> Message-ID: <804cd4020907300623l55bd2c11v61f14c0bebb3ce14@mail.gmail.com> Thanks! Indeed switching to file:write fixed the problem. But what is the explanation for the fact that the output slows during runtime? Is it a bug? Still I would like to write text files where the lines are formatted using ~p style. How can I do that fast? On Thu, Jul 30, 2009 at 3:58 PM, Rob Elsner wrote: > I am doing nearly an identical thing and record a 35Mbps multicast UDP > stream (~4MB/s) just fine. ?The difference is I don't use io:write, I > use file:write. ?I've had the file in raw mode and that works as well. > ?I also use delayed_write, but in either case it works fine. > > I think you might be hitting an issue using io:write. ?Is there a > chance you can use file:write and see how that performs? > > Rob > > On Thu, Jul 30, 2009 at 6:29 AM, Ovidiu Deac wrote: >> Hi everybody, >> >> I want to do some speed tests for parsing packets and I wrote a small >> program which receives data on a multicast address and writes it to a >> file. While this application is running I measure the growing speed of >> the output file to get an idea about the speed. > From bengt.kleberg@REDACTED Thu Jul 30 15:54:41 2009 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 30 Jul 2009 15:54:41 +0200 Subject: [erlang-questions] Slow UTP packets In-Reply-To: <804cd4020907300623l55bd2c11v61f14c0bebb3ce14@mail.gmail.com> References: <804cd4020907300529x262b052pb5013f31ad86087c@mail.gmail.com> <804cd4020907300623l55bd2c11v61f14c0bebb3ce14@mail.gmail.com> Message-ID: <1248962081.4821.50.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, To format beore writing taker a look at io_lib:format/2 (http://erlang.org/doc/man/io_lib.html). It handles formatting like io:fwrite/2, but returns an iolist instead of writing to an IO object. bengt On Thu, 2009-07-30 at 16:23 +0300, Ovidiu Deac wrote: > Thanks! > > Indeed switching to file:write fixed the problem. > > But what is the explanation for the fact that the output slows during > runtime? Is it a bug? > > Still I would like to write text files where the lines are formatted > using ~p style. How can I do that fast? > > On Thu, Jul 30, 2009 at 3:58 PM, Rob Elsner wrote: > > I am doing nearly an identical thing and record a 35Mbps multicast UDP > > stream (~4MB/s) just fine. The difference is I don't use io:write, I > > use file:write. I've had the file in raw mode and that works as well. > > I also use delayed_write, but in either case it works fine. > > > > I think you might be hitting an issue using io:write. Is there a > > chance you can use file:write and see how that performs? > > > > Rob > > > > On Thu, Jul 30, 2009 at 6:29 AM, Ovidiu Deac wrote: > >> Hi everybody, > >> > >> I want to do some speed tests for parsing packets and I wrote a small > >> program which receives data on a multicast address and writes it to a > >> file. While this application is running I measure the growing speed of > >> the output file to get an idea about the speed. > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From ovidiudeac@REDACTED Thu Jul 30 16:02:33 2009 From: ovidiudeac@REDACTED (Ovidiu Deac) Date: Thu, 30 Jul 2009 17:02:33 +0300 Subject: [erlang-questions] Slow UTP packets In-Reply-To: <1248962081.4821.50.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> References: <804cd4020907300529x262b052pb5013f31ad86087c@mail.gmail.com> <804cd4020907300623l55bd2c11v61f14c0bebb3ce14@mail.gmail.com> <1248962081.4821.50.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: <804cd4020907300702g30995093uc2b1d2f33c60fc2d@mail.gmail.com> Thanks. For the info. On Thu, Jul 30, 2009 at 4:54 PM, Bengt Kleberg wrote: > Greetings, > > To format beore writing taker a look at io_lib:format/2 > (http://erlang.org/doc/man/io_lib.html). It handles formatting like > io:fwrite/2, but returns an iolist instead of writing to an IO object. > > > bengt > > On Thu, 2009-07-30 at 16:23 +0300, Ovidiu Deac wrote: >> Thanks! >> >> Indeed switching to file:write fixed the problem. >> >> But what is the explanation for the fact that the output slows during >> runtime? Is it a bug? >> >> Still I would like to write text files where the lines are formatted >> using ~p style. How can I do that fast? >> >> On Thu, Jul 30, 2009 at 3:58 PM, Rob Elsner wrote: >> > I am doing nearly an identical thing and record a 35Mbps multicast UDP >> > stream (~4MB/s) just fine. ?The difference is I don't use io:write, I >> > use file:write. ?I've had the file in raw mode and that works as well. >> > ?I also use delayed_write, but in either case it works fine. >> > >> > I think you might be hitting an issue using io:write. ?Is there a >> > chance you can use file:write and see how that performs? >> > >> > Rob >> > >> > On Thu, Jul 30, 2009 at 6:29 AM, Ovidiu Deac wrote: >> >> Hi everybody, >> >> >> >> I want to do some speed tests for parsing packets and I wrote a small >> >> program which receives data on a multicast address and writes it to a >> >> file. While this application is running I measure the growing speed of >> >> the output file to get an idea about the speed. >> > >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From jacek99@REDACTED Thu Jul 30 16:08:13 2009 From: jacek99@REDACTED (Jacek Furmankiewicz) Date: Thu, 30 Jul 2009 10:08:13 -0400 Subject: [erlang-questions] Ports used by Erlang OTP In-Reply-To: <1248875832.5798.66.camel@gram.local> References: <269a5c30907290642k55e59c5dhf91fe4b3809f4a73@mail.gmail.com> <1248875832.5798.66.camel@gram.local> Message-ID: <269a5c30907300708y115c45acof0266b0b1074243a@mail.gmail.com> So basically I should create a kernel.app file and put it into ebin looking more or less like this? {application,kernel, [{inet_dist_listen_min,16000}, {inet_dist_listen_max,16050} ]}. Is there anything else that is required? Thanks, Jacek On Wed, Jul 29, 2009 at 9:57 AM, Peter Sabaini wrote: > On Wed, 2009-07-29 at 09:42 -0400, Jacek Furmankiewicz wrote: > > Is there some documentation as to which ports are used internally by > Erlang > > OTP? > > For OTP, this is configurable, see > > {inet_dist_listen_min, First} > {inet_dist_listen_max, Last} > > in the kernel reference docs > ( http://erlang.org/doc/man/kernel_app.html ) > > EPMD listens on port 4369 by default > > > I need to document it for our operations team before they move the app > into > > production. Our data centers are very hardened and all ports are closed > by > > defaults. > > Only those specifically requested are opened. > > > > I guess I would need to know the ports used by EPMD and OTP for all the > > rpc:_ calls...or any other ports that are required for an OTP app to > > function. > > > > Thanks, > > Jacek > From ro4tub@REDACTED Thu Jul 30 17:37:33 2009 From: ro4tub@REDACTED (Oscar) Date: Thu, 30 Jul 2009 23:37:33 +0800 Subject: can't start rb module Message-ID: <9a77767b0907300837x6b3d281aw136a8539f2167ca4@mail.gmail.com> 6> rb:start([{max, 20}]). ** exception exit: {noproc,{gen_server,call, [sasl_sup, {start_child,{rb_server,{rb,start_link,[[{max,20}]]}, temporary,brutal_kill,worker, [rb]}}, infinity]}} in function gen_server:call/3 what does the above exception mean? From dizzyd@REDACTED Thu Jul 30 17:45:04 2009 From: dizzyd@REDACTED (Dave Smith) Date: Thu, 30 Jul 2009 09:45:04 -0600 Subject: [erlang-questions] can't start rb module In-Reply-To: <9a77767b0907300837x6b3d281aw136a8539f2167ca4@mail.gmail.com> References: <9a77767b0907300837x6b3d281aw136a8539f2167ca4@mail.gmail.com> Message-ID: I believe it means you haven't started the SASL application. Try: application:start(sasl). and then use rb:start/1. D. On Thu, Jul 30, 2009 at 9:37 AM, Oscar wrote: > 6> rb:start([{max, 20}]). > ** exception exit: {noproc,{gen_server,call, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [sasl_sup, > > {start_child,{rb_server,{rb,start_link,[[{max,20}]]}, > > temporary,brutal_kill,worker, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[rb]}}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?infinity]}} > ? ? in function ?gen_server:call/3 > > > > what does the above exception mean? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From bengt.kleberg@REDACTED Thu Jul 30 17:49:12 2009 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 30 Jul 2009 17:49:12 +0200 Subject: [erlang-questions] can't start rb module In-Reply-To: <9a77767b0907300837x6b3d281aw136a8539f2167ca4@mail.gmail.com> References: <9a77767b0907300837x6b3d281aw136a8539f2167ca4@mail.gmail.com> Message-ID: <1248968952.4821.54.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, You might not have SASL running. Try application:which_applications(). and see if SASL is there. bengt n Thu, 2009-07-30 at 23:37 +0800, Oscar wrote: > 6> rb:start([{max, 20}]). > ** exception exit: {noproc,{gen_server,call, > [sasl_sup, > > {start_child,{rb_server,{rb,start_link,[[{max,20}]]}, > > temporary,brutal_kill,worker, > [rb]}}, > infinity]}} > in function gen_server:call/3 > > > > what does the above exception mean? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From baryluk@REDACTED Thu Jul 30 17:56:43 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Thu, 30 Jul 2009 17:56:43 +0200 Subject: [erlang-questions] can't start rb module In-Reply-To: <1248968952.4821.54.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> References: <9a77767b0907300837x6b3d281aw136a8539f2167ca4@mail.gmail.com> <1248968952.4821.54.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: <1248969403.6180.1.camel@sredniczarny> Dnia 2009-07-30, czw o godzinie 17:49 +0200, Bengt Kleberg pisze: > Greetings, > > You might not have SASL running. Try > application:which_applications(). > and see if SASL is there. Is rb an application? I remember in .app files, there are information which other applications we depend on, and need to be started before it. BTW. after starting sasl, rb:start/1,2 still doesn't work (with other error). -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From ro4tub@REDACTED Thu Jul 30 18:01:46 2009 From: ro4tub@REDACTED (Oscar) Date: Fri, 31 Jul 2009 00:01:46 +0800 Subject: [erlang-questions] can't start rb module In-Reply-To: <1248969403.6180.1.camel@sredniczarny> References: <9a77767b0907300837x6b3d281aw136a8539f2167ca4@mail.gmail.com> <1248968952.4821.54.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> <1248969403.6180.1.camel@sredniczarny> Message-ID: <9a77767b0907300901u70939bem926bf08a670361cf@mail.gmail.com> Anybody could tell me which document explain SASL configuration files? Thanks a lot. 2009/7/30 Witold Baryluk : > Dnia 2009-07-30, czw o godzinie 17:49 +0200, Bengt Kleberg pisze: >> Greetings, >> >> You might not have SASL running. Try >> application:which_applications(). >> and see if SASL is there. > > Is rb an application? I remember in .app files, there are information > which other applications we depend on, and need to be started before it. > > BTW. after starting sasl, rb:start/1,2 still doesn't work (with other > error). > > -- > Witold Baryluk > From ro4tub@REDACTED Thu Jul 30 18:12:57 2009 From: ro4tub@REDACTED (Oscar) Date: Fri, 31 Jul 2009 00:12:57 +0800 Subject: [erlang-questions] can't start rb module In-Reply-To: <9a77767b0907300901u70939bem926bf08a670361cf@mail.gmail.com> References: <9a77767b0907300837x6b3d281aw136a8539f2167ca4@mail.gmail.com> <1248968952.4821.54.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> <1248969403.6180.1.camel@sredniczarny> <9a77767b0907300901u70939bem926bf08a670361cf@mail.gmail.com> Message-ID: <9a77767b0907300912k479b261bo54e6c9e9cb3caa39@mail.gmail.com> resolved. there is no 'error_logs' directory. On Fri, Jul 31, 2009 at 12:01 AM, Oscar wrote: > Anybody could tell me which document explain SASL configuration files? > Thanks a lot. > > 2009/7/30 Witold Baryluk : >> Dnia 2009-07-30, czw o godzinie 17:49 +0200, Bengt Kleberg pisze: >>> Greetings, >>> >>> You might not have SASL running. Try >>> application:which_applications(). >>> and see if SASL is there. >> >> Is rb an application? I remember in .app files, there are information >> which other applications we depend on, and need to be started before it. >> >> BTW. after starting sasl, rb:start/1,2 still doesn't work (with other >> error). >> >> -- >> Witold Baryluk >> > From clist@REDACTED Thu Jul 30 18:46:46 2009 From: clist@REDACTED (Angel Alvarez) Date: Thu, 30 Jul 2009 18:46:46 +0200 Subject: [erlang-questions] code path when using inet loader In-Reply-To: <102d3f890907291233g737e1d37p80f17c8f92f2c862@mail.gmail.com> References: <102d3f890907291233g737e1d37p80f17c8f92f2c862@mail.gmail.com> Message-ID: <200907301846.46974.clist@uah.es> El Mi?rcoles, 29 de Julio de 2009 Sebastien Merle escribi?: > Hi, > > I'm experimenting with remote code server using a master node > with erl_boot_server that starts slave nodes with arguments "-loader inet". > > It seems that if the master is started with -pz arguments, the beam files > from the added path cannot be loaded by the slave nodes. > > If I just start the master in the directory where the beam files are instead > of adding the path with -pz, the slave can load the beam files. > > Any hints on why this is happening ? ?does code:get_path() return correct info on slaves? Maybe -pz arguments need to be passed to slave. i think slave code still controls loading but defers actual file loading to master server, so proper -pz must be passed on slave startup. /angel > > Regards, > Sebastien Merle. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- Agua para todo? No, Agua para Todos. ->>----------------------------------------------- Clist UAH a.k.a Angel ---------------------------------[www.uah.es]-<<-- No le dar?a Cocacola Zero, ni a mi peor enemigo. Para eso est? el gas Mostaza que es mas piadoso. From s.merle@REDACTED Thu Jul 30 19:14:05 2009 From: s.merle@REDACTED (Sebastien Merle) Date: Thu, 30 Jul 2009 19:14:05 +0200 Subject: [erlang-questions] code path when using inet loader In-Reply-To: <200907301846.46974.clist@uah.es> References: <102d3f890907291233g737e1d37p80f17c8f92f2c862@mail.gmail.com> <200907301846.46974.clist@uah.es> Message-ID: <102d3f890907301014m2b7064d0k29b461d6a0bce10b@mail.gmail.com> Indeed, the path added with -pz are not in the slave code:get_path() list. This is troublesome, because it mean the master node has to know how it has been setup and replicate the configuration to the slaves. It's not very flexible. Another option seems to create a .erlang file with code:add_pathz() in it, this is working but then you loose any custom configuration added to $HOME/.erlang and you have to run erl in this directory. So it's not a lot better in the end. On Thu, Jul 30, 2009 at 6:46 PM, Angel Alvarez wrote: > El Mi?rcoles, 29 de Julio de 2009 Sebastien Merle escribi?: >> Hi, >> >> I'm experimenting with remote code server using a master node >> with erl_boot_server that starts slave nodes with arguments "-loader inet". >> >> It seems that if the master is started with -pz arguments, the beam files >> from the added path cannot be loaded by the slave nodes. >> >> If I just start the master in the directory where the beam files are instead >> of adding the path with -pz, the slave can load the beam files. >> >> Any hints on why this is happening ? > > ?does code:get_path() return correct info on slaves? > > Maybe -pz arguments need to be passed to slave. i think slave code still controls loading but defers > actual file loading to master server, so proper -pz must be passed on slave startup. > > /angel > >> >> Regards, >> Sebastien Merle. >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > > > -- > Agua para todo? No, Agua para Todos. > ->>----------------------------------------------- > ? ?Clist UAH a.k.a Angel > ---------------------------------[www.uah.es]-<<-- > > No le dar?a Cocacola Zero, ni a mi peor enemigo. Para eso est? el gas Mostaza que es mas piadoso. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From peter@REDACTED Thu Jul 30 19:18:42 2009 From: peter@REDACTED (Peter Sabaini) Date: Thu, 30 Jul 2009 19:18:42 +0200 Subject: [erlang-questions] Ports used by Erlang OTP In-Reply-To: <269a5c30907300708y115c45acof0266b0b1074243a@mail.gmail.com> References: <269a5c30907290642k55e59c5dhf91fe4b3809f4a73@mail.gmail.com> <1248875832.5798.66.camel@gram.local> <269a5c30907300708y115c45acof0266b0b1074243a@mail.gmail.com> Message-ID: <1248974322.31249.3.camel@gram.local> On Thu, 2009-07-30 at 10:08 -0400, Jacek Furmankiewicz wrote: > So basically I should create a kernel.app file and put it into ebin > looking more or less like this? I had this in my global .erlang.config: [ {kernel, [ {inet_dist_listen_min,9100}, {inet_dist_listen_max,9200}]}, ... I'm sure there are other ways too. > {application,kernel, > [{inet_dist_listen_min,16000}, > {inet_dist_listen_max,16050} > ]}. > > Is there anything else that is required? > Thanks, Jacek > > On Wed, Jul 29, 2009 at 9:57 AM, Peter Sabaini > wrote: > On Wed, 2009-07-29 at 09:42 -0400, Jacek Furmankiewicz wrote: > > Is there some documentation as to which ports are used > internally by Erlang > > OTP? > > > For OTP, this is configurable, see > > {inet_dist_listen_min, First} > {inet_dist_listen_max, Last} > > in the kernel reference docs > ( http://erlang.org/doc/man/kernel_app.html ) > > EPMD listens on port 4369 by default > > > > I need to document it for our operations team before they > move the app into > > production. Our data centers are very hardened and all ports > are closed by > > defaults. > > Only those specifically requested are opened. > > > > I guess I would need to know the ports used by EPMD and OTP > for all the > > rpc:_ calls...or any other ports that are required for an > OTP app to > > function. > > > > Thanks, > > Jacek > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From clist@REDACTED Thu Jul 30 19:38:03 2009 From: clist@REDACTED (Angel Alvarez) Date: Thu, 30 Jul 2009 19:38:03 +0200 Subject: [erlang-questions] code path when using inet loader In-Reply-To: <102d3f890907301014m2b7064d0k29b461d6a0bce10b@mail.gmail.com> References: <102d3f890907291233g737e1d37p80f17c8f92f2c862@mail.gmail.com> <200907301846.46974.clist@uah.es> <102d3f890907301014m2b7064d0k29b461d6a0bce10b@mail.gmail.com> Message-ID: <200907301938.03882.clist@uah.es> El Jueves, 30 de Julio de 2009 Sebastien Merle escribi?: > Indeed, the path added with -pz are not in the slave code:get_path() list. > > This is troublesome, because it mean the master node has to know > how it has been setup and replicate the configuration to the slaves. > It's not very flexible. > > Another option seems to create a .erlang file with code:add_pathz() in it, > this is working but then you loose any custom configuration added to > $HOME/.erlang and you have to run erl in this directory. > So it's not a lot better in the end. well, you can use something like: cluster_add_pathz(NewPath) -> %% extract remote all nodes except master RemoteNodes = [X || X <- pool:get_nodes(), X =/= node()], %% foreach node do proper path loading lists:foreach(fun(Node) -> rpc:call(Node, code, add_pathz, [NewPath]). end, RemotesNodes). where pool:get_nodes can be changes to nodes() when no pool is used... *Warning* Untested code (written on the fly) /Angel > > On Thu, Jul 30, 2009 at 6:46 PM, Angel Alvarez wrote: > > El Mi?rcoles, 29 de Julio de 2009 Sebastien Merle escribi?: > >> Hi, > >> > >> I'm experimenting with remote code server using a master node > >> with erl_boot_server that starts slave nodes with arguments "-loader inet". > >> > >> It seems that if the master is started with -pz arguments, the beam files > >> from the added path cannot be loaded by the slave nodes. > >> > >> If I just start the master in the directory where the beam files are instead > >> of adding the path with -pz, the slave can load the beam files. > >> > >> Any hints on why this is happening ? > > > > ?does code:get_path() return correct info on slaves? > > > > Maybe -pz arguments need to be passed to slave. i think slave code still controls loading but defers > > actual file loading to master server, so proper -pz must be passed on slave startup. > > > > /angel > > > >> > >> Regards, > >> Sebastien Merle. > >> > >> ________________________________________________________________ > >> erlang-questions mailing list. See http://www.erlang.org/faq.html > >> erlang-questions (at) erlang.org > >> > >> > > > > > > > > -- > > Agua para todo? No, Agua para Todos. > > ->>----------------------------------------------- > > ? ?Clist UAH a.k.a Angel > > ---------------------------------[www.uah.es]-<<-- > > > > No le dar?a Cocacola Zero, ni a mi peor enemigo. Para eso est? el gas Mostaza que es mas piadoso. > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos. ->>----------------------------------------------- Clist UAH a.k.a Angel ---------------------------------[www.uah.es]-<<-- Warning: Microsoft_bribery.ISO contains OOXML code. From yoursurrogategod@REDACTED Fri Jul 31 03:59:38 2009 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Thu, 30 Jul 2009 18:59:38 -0700 (PDT) Subject: Nested for-loops, there has to be a better way to do this In-Reply-To: <9b08084c0907282335s7fc42eb5r8e925e9597dae34c@mail.gmail.com> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <9b08084c0907282335s7fc42eb5r8e925e9597dae34c@mail.gmail.com> Message-ID: Takes first int, processes it (in this case, prints it out), decrements it, takes the second int, processes it (prints it out), decrements it and on. When it gets to the very last item, it processes it until there are none left. Then goes to the one above it, processes it, decrements it... wash, rinse, repeat :) . In retrospect, I could have made this clearer. On Jul 29, 2:35?am, Joe Armstrong wrote: > I don't understand what you want. > > If the input to your program is: [5,5,10] > the output should be: X > > What is X? > > /Joe > > On Wed, Jul 29, 2009 at 4:55 AM, Yves S. > > > > > > Garret wrote: > > I was playing around with the idea of implementing a nested for-loop > > like construct (I will concede, they are evil, but at rare times, > > necessary.) ?In non-functional programming languages, they are trivial > > to do. ?However, this has proved to be one of those simple things that > > are a pain in the neck for me. ?The idea was to be able to pass in a > > list [5, 5, 10] (for example) and go through it all like 3 dimensional > > array. ?The below illustration is far from elegant (it's almost 23:00 > > where I live and need to go to work tomorrow early, so no time to come > > up with something more sane :) ), but is there as an inquisitive > > exercise. ?Now, in my opinion, this implementation sucks (first time > > I've ever gave such a problem a shot in Erlang.) > > > Is there a better way to do this? ?Anyone would like to demonstrate > > it? > > > Note: I've tested this code with the lists [2, 4] and [2, 4, 5]. > > > Compiled with: > > Erlang (SMP,ASYNC_THREADS,HIPE) (BEAM) emulator version 5.6.5 > > > My code: > > ====================================== > > -module(loopWrapper). > > > -export([runLoop/1]). > > > runLoop(List) -> > > ?[FirstVal | RestList] = List, > > ?NextVal = FirstVal - 1, > > ?if (length(RestList) > 0) and (FirstVal > 0) -> > > ? ?io:format("~B~n~n", [FirstVal]), > > ? ?runLoop(RestList), > > ? ?io:format("~n~n"), > > ? ?runLoop([NextVal | RestList]); > > ?FirstVal > 0 -> > > ? ?io:format("~B", [FirstVal]), > > ? ?runLoop([NextVal]); > > ?true -> > > ? ?null > > ?end. > > > ________________________________________________________________ > > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From yoursurrogategod@REDACTED Fri Jul 31 04:00:55 2009 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Thu, 30 Jul 2009 19:00:55 -0700 (PDT) Subject: Nested for-loops, there has to be a better way to do this In-Reply-To: <63418.194.88.55.211.1248848724.squirrel@localhost> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <63418.194.88.55.211.1248848724.squirrel@localhost> Message-ID: Yes, that looks like it could work, but I was hoping for something within one function. Don't take it the wrong way, but loops (to me) are supposed to be simple and when it involves more than one function, it just seems like over-kill :-) . On Jul 29, 2:25?am, ki...@REDACTED wrote: > Hi, > > Do you mean something like this? > > -------------------- > > -module(loop). > > -export([loop/2]). > -export([callback/1]). > > callback(Numbers) -> > ? ? io:format("~p~n", [Numbers]). > > loop(Callback, Limits) -> > ? ? loop(Limits, lists:duplicate(length(Limits), 1), Callback). > > loop(Limits, Current, Callback) -> > ? ? Callback(Current), > ? ? case increment(Limits, Current) of > ? ? ? ? finished -> > ? ? ? ? ? ? ok; > ? ? ? ? NewCurrent -> > ? ? ? ? loop(Limits, NewCurrent, Callback) > ? ? end. > > increment(Limits, Current) -> > ? ? increment(Limits, Current, []). > > increment([], [], _) -> > ? ? finished; > > increment([N|Limits], [N|Current], Done) -> > ? ? increment(Limits, Current, [1|Done]); > > increment(_Limits, [N|Current], Done) -> > ? ? Done ++ [N+1|Current]. > > -------------------- > > 1> loop:loop(fun loop:callback/1, [2,2,2,2,3]). > [1,1,1,1,1] > [2,1,1,1,1] > [1,2,1,1,1] > .... > > Regards, > Z. > > > > > > > I was playing around with the idea of implementing a nested for-loop > > like construct (I will concede, they are evil, but at rare times, > > necessary.) ?In non-functional programming languages, they are trivial > > to do. ?However, this has proved to be one of those simple things that > > are a pain in the neck for me. ?The idea was to be able to pass in a > > list [5, 5, 10] (for example) and go through it all like 3 dimensional > > array. ?The below illustration is far from elegant (it's almost 23:00 > > where I live and need to go to work tomorrow early, so no time to come > > up with something more sane :) ), but is there as an inquisitive > > exercise. ?Now, in my opinion, this implementation sucks (first time > > I've ever gave such a problem a shot in Erlang.) > > > Is there a better way to do this? ?Anyone would like to demonstrate > > it? > > > Note: I've tested this code with the lists [2, 4] and [2, 4, 5]. > > > Compiled with: > > Erlang (SMP,ASYNC_THREADS,HIPE) (BEAM) emulator version 5.6.5 > > > My code: > > ====================================== > > -module(loopWrapper). > > > -export([runLoop/1]). > > > runLoop(List) -> > > ? [FirstVal | RestList] = List, > > ? NextVal = FirstVal - 1, > > ? if (length(RestList) > 0) and (FirstVal > 0) -> > > ? ? io:format("~B~n~n", [FirstVal]), > > ? ? runLoop(RestList), > > ? ? io:format("~n~n"), > > ? ? runLoop([NextVal | RestList]); > > ? FirstVal > 0 -> > > ? ? io:format("~B", [FirstVal]), > > ? ? runLoop([NextVal]); > > ? true -> > > ? ? null > > ? end. > > > ________________________________________________________________ > > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From yoursurrogategod@REDACTED Fri Jul 31 04:02:05 2009 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Thu, 30 Jul 2009 19:02:05 -0700 (PDT) Subject: Nested for-loops, there has to be a better way to do this In-Reply-To: References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> Message-ID: <0358573f-12ab-4b67-b636-ef657b814cb5@18g2000yqa.googlegroups.com> Yes, as Joe Armstrong pointed out, I did not do a good job of explaining what I wanted *exactly*. But thanks for that piece of code, it looks like something I could use at one point. On Jul 28, 11:32?pm, "Richard O'Keefe" wrote: > On Jul 29, 2009, at 2:55 PM, Yves S. Garret wrote: > > > I was playing around with the idea of implementing a nested for- > > loop... a > > list [5, 5, 10] (for example) and go through it all like 3 dimensional > > array. > > I don't understand what it is you want to do. > Will something like > ? ? ? ? D1 = lists:seq(1, 5), > ? ? ? ? D2 = lists:seq(1, 5), > ? ? ? ? D3 = lists:seq(1, 10), > ? ? ? ? % do this for effect, not result > ? ? ? ? [ effect(I1, I2, I3) || I1 <- D1, I2 <- D2, I3 <- D3 ] > do what you want? > > I've been playing with a syntax that allows, amongst other > special cases, > ? ? ? ? ( effect(I1, I2, I3) || I1 <- D1, I2 <- D2, I3 <- D3 ) > However, it's not really needed; all that's needed is for the > Erlang compiler to notice that the result of a list comprehension > is not being used, and just not bother constructing the result. > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From yoursurrogategod@REDACTED Fri Jul 31 04:05:54 2009 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Thu, 30 Jul 2009 19:05:54 -0700 (PDT) Subject: Nested for-loops, there has to be a better way to do this In-Reply-To: <6a3ae47e0907300246j104f5973x854a004f87bea5a4@mail.gmail.com> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <63418.194.88.55.211.1248848724.squirrel@localhost> <4A7077F6.10508@dcc.fc.up.pt> <4A709BB3.5010701@tmit.bme.hu> <772561A5-A67A-4738-99DA-396CF2ED6C2B@gmail.com> <4A709F5D.2060401@tmit.bme.hu> <4A70AF23.1090404@dcc.fc.up.pt> <8b9ee55b0907291408l44c57c3ep8c60e3d0db13f848@mail.gmail.com> <6a3ae47e0907300246j104f5973x854a004f87bea5a4@mail.gmail.com> Message-ID: <30427adf-2eb2-4050-9616-988ffe5e5903@b14g2000yqd.googlegroups.com> Oh, that looks cool. Thanks for sharing. On Jul 30, 5:46?am, Robert Raschke wrote: > On Wed, Jul 29, 2009 at 10:08 PM, Fred Hebert (MononcQc) > > > > > > wrote: > > I find nested lists comprehension to be the easiest to deal with when > > representing nested loops: > > > [[ Fn(I,J) || J <- GenExp2 ] || I <- GenEXp1 ]. > > > or indented in a different way: > > > ?[[[ Fn(I, J, K) || K <- GenExp3 ] > > ? ? ? ? ? ? ? ? || J <- GenExp2 ] > > ? ? ? ? ? ? ? ? || I <- GenExp1 ]. > > > This gives a nested list. If that's not what you want, you could use > > lists:flatten/1 to change that, but overall I find this form to be the > > cleanest (visually speaking). > > I may be missing something obvious, but ... > > 1> [ {A,B,C,A+B+C} || A<-lists:seq(1,5), B<-lists:seq(1,5), > C<-lists:seq(1,10) ]. > [{1,1,1,3}, > ?{1,1,2,4}, > ?{1,1,3,5}, > ?{1,1,4,6}, > ?{1,1,5,7}, > ?{1,1,6,8}, > ?{1,1,7,9}, > ?{1,1,8,10}, > ?{1,1,9,11}, > ?{1,1,10,12}, > ?{1,2,1,4}, > ?{1,2,2,5}, > ?{1,2,3,6}, > ?{1,2,4,7}, > ?{1,2,5,8}, > ?{1,2,6,9}, > ?{1,2,7,10}, > ?{1,2,8,11}, > ?{1,2,9,12}, > ?{1,2,10,13}, > ?{1,3,1,5}, > ?{1,3,2,6}, > ?{1,3,3,7}, > ?{1,3,4,8}, > ?{1,3,5,9}, > ?{1,3,6,...}, > ?{1,3,...}, > ?{1,...}, > ?{...}|...] > 2> > > Robby From yoursurrogategod@REDACTED Fri Jul 31 04:07:05 2009 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Thu, 30 Jul 2009 19:07:05 -0700 (PDT) Subject: Nested for-loops, there has to be a better way to do this In-Reply-To: <4A709F5D.2060401@tmit.bme.hu> References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <63418.194.88.55.211.1248848724.squirrel@localhost> <4A7077F6.10508@dcc.fc.up.pt> <4A709BB3.5010701@tmit.bme.hu> <772561A5-A67A-4738-99DA-396CF2ED6C2B@gmail.com> <4A709F5D.2060401@tmit.bme.hu> Message-ID: Yeh... late at night... messing around with Erlang code... initially the idea was to make a bunch of nested loops, but it was for something completely different. On Jul 29, 3:13?pm, Zoltan Lajos Kis wrote: > Brentley Jones wrote: > > > On Jul 29, 2009, at 1:57 PM, Zoltan Lajos Kis wrote: > > >> And what do I gain using this for function compared to simply writing > >> an "ad-hoc" recursive function whenever needed ? > > > Nothing is gained by my bloated, yet modular function. I was just > > expanding on the modular function listed before mine. > > >> PS: seriously, is this what the original question was getting at? > > > I do think it shows how to do nested for loops in a way that looks > > very imperative, which is what I think the original question was > > getting at. > > > On Jul 29, 2009, at 2:55 PM, Yves S. Garret wrote: > > >> I was playing around with the idea of implementing a nested for-loop > >> like construct (I will concede, they are evil, but at rare times, > >> necessary.) ?In non-functional programming languages, they are trivial > >> to do. ?However, this has proved to be one of those simple things that > >> are a pain in the neck for me. > > But if you look just one sentence further... > "The idea was to be able to pass in a list [5, 5, 10] (for example) and > go through it all like 3 dimensional array." > > Anyway, we should get the answer by dawn (CET) :) > > Z. > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From mihai@REDACTED Fri Jul 31 05:02:42 2009 From: mihai@REDACTED (Mihai Balea) Date: Thu, 30 Jul 2009 23:02:42 -0400 Subject: [erlang-questions] Slow UTP packets In-Reply-To: <804cd4020907300623l55bd2c11v61f14c0bebb3ce14@mail.gmail.com> References: <804cd4020907300529x262b052pb5013f31ad86087c@mail.gmail.com> <804cd4020907300623l55bd2c11v61f14c0bebb3ce14@mail.gmail.com> Message-ID: <8F48DC24-3F10-44C5-9033-F11276942013@hates.ms> On Jul 30, 2009, at 9:23 AM, Ovidiu Deac wrote: > But what is the explanation for the fact that the output slows during > runtime? Is it a bug? One possible explanation would be accumulation of messages in your receiving process' mailbox. If you cannot consume messages as fast as they arrive, they will build up in your mailbox and slow down the process. Due to the way the mailbox is implemented, it cannot cope well with massive numbers of messages. I don't remember the details, but I believe it has to do with using O(n) algorithms. Maybe somebody in the know can provide more details. Mihai From corticalcomputer@REDACTED Fri Jul 31 06:19:56 2009 From: corticalcomputer@REDACTED (G.S.) Date: Thu, 30 Jul 2009 21:19:56 -0700 Subject: Mnesia Read/Write question. Message-ID: <2a67d3ff0907302119u6930a5ddpa8bd07d7215c0268@mail.gmail.com> Hello everyone, This is probably a very stupid question/problem, but I'm having trouble using the following commands: A) mnesia:dirty_write(Id#test{item = hello}) and mnesia:dirty_read(Id#test.item) I get: exception error: {badrecord,test} but this works on the same table/schema...: B) Thing = #test{id = 1, item = hello}, mnesia:dirty_write(test,Thing), mnesia:dirty_read(test,Id). Are "A)" valid commands, what am I doing wrong? Regards, -Gene From el03665@REDACTED Fri Jul 31 07:55:31 2009 From: el03665@REDACTED (Maria Christakis) Date: Fri, 31 Jul 2009 08:55:31 +0300 Subject: [erlang-questions] Mnesia Read/Write question. In-Reply-To: <2a67d3ff0907302119u6930a5ddpa8bd07d7215c0268@mail.gmail.com> References: <2a67d3ff0907302119u6930a5ddpa8bd07d7215c0268@mail.gmail.com> Message-ID: <4A728753.8040106@mail.ntua.gr> Hello, The problem probably is that your dirty_read/1 syntax is wrong. dirty_read({Tab, Key}) So if mnesia:dirty_read(test,Id) works, you should try mnesia:dirty_read({test,Id}) Maria G.S. wrote: > Hello everyone, > This is probably a very stupid question/problem, but I'm having trouble > using the following commands: > > A) mnesia:dirty_write(Id#test{item = hello}) and > mnesia:dirty_read(Id#test.item) I get: exception error: {badrecord,test} > but this works on the same table/schema...: > B) Thing = #test{id = 1, item = hello}, mnesia:dirty_write(test,Thing), > mnesia:dirty_read(test,Id). > > Are "A)" valid commands, what am I doing wrong? > > Regards, > -Gene > > From steven.charles.davis@REDACTED Fri Jul 31 08:04:03 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 30 Jul 2009 23:04:03 -0700 (PDT) Subject: Mnesia Read/Write question. In-Reply-To: <2a67d3ff0907302119u6930a5ddpa8bd07d7215c0268@mail.gmail.com> References: <2a67d3ff0907302119u6930a5ddpa8bd07d7215c0268@mail.gmail.com> Message-ID: <7488867d-a1ca-43c2-9181-5554b1884ab3@h21g2000yqa.googlegroups.com> Hello Gene, It looks like #test.id is the key, not #test.item. In the first example, I suspect id is undefined. In the second it is. To find by item, which appears to be what you are trying to do in (A), you would have to use matches. /sd On Jul 30, 11:19?pm, "G.S." wrote: > Hello everyone, > This is probably a very stupid question/problem, but I'm having trouble > using the following commands: > > A) mnesia:dirty_write(Id#test{item = hello}) and > mnesia:dirty_read(Id#test.item) ?I get: exception error: {badrecord,test} > but this works on the same table/schema...: > B) Thing = #test{id = 1, item = hello}, mnesia:dirty_write(test,Thing), > mnesia:dirty_read(test,Id). > > Are "A)" valid commands, what am I doing wrong? > > Regards, > -Gene From corticalcomputer@REDACTED Fri Jul 31 09:02:29 2009 From: corticalcomputer@REDACTED (G.S.) Date: Fri, 31 Jul 2009 00:02:29 -0700 Subject: [erlang-questions] Re: Mnesia Read/Write question. In-Reply-To: <7488867d-a1ca-43c2-9181-5554b1884ab3@h21g2000yqa.googlegroups.com> References: <2a67d3ff0907302119u6930a5ddpa8bd07d7215c0268@mail.gmail.com> <7488867d-a1ca-43c2-9181-5554b1884ab3@h21g2000yqa.googlegroups.com> Message-ID: <2a67d3ff0907310002p7845f5f9k933c56ac587ab8c1@mail.gmail.com> Hello, Maria, I'm not having problems with the B) version, only with A). (The error you noted was due to my misspelling it in the email). Could syntax in "A)" be deprecated? I got it from Mnesia A Distributed Robust DBMS for Telecommunications Applications, which I think was published in 98 or earlier. Regards, -Gene On Thu, Jul 30, 2009 at 11:04 PM, Steve Davis < steven.charles.davis@REDACTED> wrote: > Hello Gene, > > It looks like #test.id is the key, not #test.item. In the first > example, I suspect id is undefined. In the second it is. To find by > item, which appears to be what you are trying to do in (A), you would > have to use matches. > > /sd > > On Jul 30, 11:19 pm, "G.S." wrote: > > Hello everyone, > > This is probably a very stupid question/problem, but I'm having trouble > > using the following commands: > > > > A) mnesia:dirty_write(Id#test{item = hello}) and > > mnesia:dirty_read(Id#test.item) I get: exception error: {badrecord,test} > > but this works on the same table/schema...: > > B) Thing = #test{id = 1, item = hello}, mnesia:dirty_write(test,Thing), > > mnesia:dirty_read(test,Id). > > > > Are "A)" valid commands, what am I doing wrong? > > > > Regards, > > -Gene > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ola.a.andersson@REDACTED Fri Jul 31 11:23:10 2009 From: ola.a.andersson@REDACTED (Ola Andersson A) Date: Fri, 31 Jul 2009 11:23:10 +0200 Subject: [erlang-questions] Re: Mnesia Read/Write question. In-Reply-To: <2a67d3ff0907310002p7845f5f9k933c56ac587ab8c1@mail.gmail.com> References: <2a67d3ff0907302119u6930a5ddpa8bd07d7215c0268@mail.gmail.com> <7488867d-a1ca-43c2-9181-5554b1884ab3@h21g2000yqa.googlegroups.com> <2a67d3ff0907310002p7845f5f9k933c56ac587ab8c1@mail.gmail.com> Message-ID: <55F1DD5E09B0FE4FB35D919DAF2127D90108023C@esealmw105.eemea.ericsson.se> Hi Gene, Read the answers from Maria and Steve again. They are both right. Anyway, the first problem with A) is probably that Id do not contain a #test{} record. That is the most likely cause for the {badrecord,test} error. The second problem is that you're syntax for mnesia:dirty_read is wrong. Assuming that you have Id assigned to a #test{} record, the call translates to mnesia:dirty_read(undefined) (or what ever the value of the field item happens to be). This has obviously never worked. You have to specify the table that the key belong to. Like mnesia:dirty_read({test, Key}). /Ola. > -----Original Message----- > From: erlang-questions@REDACTED > [mailto:erlang-questions@REDACTED] On Behalf Of G.S. > Sent: den 31 juli 2009 09:02 > To: Steve Davis > Cc: Erlang > Subject: Re: [erlang-questions] Re: Mnesia Read/Write question. > > Hello, > > Maria, I'm not having problems with the B) version, only with > A). (The error you noted was due to my misspelling it in the email). > > Could syntax in "A)" be deprecated? I got it from Mnesia A > Distributed Robust DBMS for Telecommunications Applications, > which I think was published in 98 or earlier. > > Regards, > -Gene > > On Thu, Jul 30, 2009 at 11:04 PM, Steve Davis < > steven.charles.davis@REDACTED> wrote: > > > Hello Gene, > > > > It looks like #test.id is the key, not #test.item. In the first > > example, I suspect id is undefined. In the second it is. To find by > > item, which appears to be what you are trying to do in (A), > you would > > have to use matches. > > > > /sd > > > > On Jul 30, 11:19 pm, "G.S." wrote: > > > Hello everyone, > > > This is probably a very stupid question/problem, but I'm having > > > trouble using the following commands: > > > > > > A) mnesia:dirty_write(Id#test{item = hello}) and > > > mnesia:dirty_read(Id#test.item) I get: exception error: > > > {badrecord,test} but this works on the same table/schema...: > > > B) Thing = #test{id = 1, item = hello}, > > > mnesia:dirty_write(test,Thing), mnesia:dirty_read(test,Id). > > > > > > Are "A)" valid commands, what am I doing wrong? > > > > > > Regards, > > > -Gene > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > From corticalcomputer@REDACTED Fri Jul 31 11:35:34 2009 From: corticalcomputer@REDACTED (G.S.) Date: Fri, 31 Jul 2009 02:35:34 -0700 Subject: [erlang-questions] Re: Mnesia Read/Write question. In-Reply-To: <55F1DD5E09B0FE4FB35D919DAF2127D90108023C@esealmw105.eemea.ericsson.se> References: <2a67d3ff0907302119u6930a5ddpa8bd07d7215c0268@mail.gmail.com> <7488867d-a1ca-43c2-9181-5554b1884ab3@h21g2000yqa.googlegroups.com> <2a67d3ff0907310002p7845f5f9k933c56ac587ab8c1@mail.gmail.com> <55F1DD5E09B0FE4FB35D919DAF2127D90108023C@esealmw105.eemea.ericsson.se> Message-ID: <2a67d3ff0907310235l3fdd6927oa595c74e68092014@mail.gmail.com> Yeah, I finally realized what I was doing thanks. I don't know what I was thinking, but Steve pointed me in the right direction. Thanks. -Gene On Fri, Jul 31, 2009 at 2:23 AM, Ola Andersson A < ola.a.andersson@REDACTED> wrote: > Hi Gene, > > Read the answers from Maria and Steve again. They are both right. > Anyway, the first problem with A) is probably that Id do not contain a > #test{} record. > That is the most likely cause for the {badrecord,test} error. > The second problem is that you're syntax for mnesia:dirty_read is wrong. > Assuming that you have Id assigned to a #test{} record, the call > translates to mnesia:dirty_read(undefined) > (or what ever the value of the field item happens to be). > This has obviously never worked. You have to specify the table that the > key belong to. > Like mnesia:dirty_read({test, Key}). > /Ola. > > > -----Original Message----- > > From: erlang-questions@REDACTED > > [mailto:erlang-questions@REDACTED] On Behalf Of G.S. > > Sent: den 31 juli 2009 09:02 > > To: Steve Davis > > Cc: Erlang > > Subject: Re: [erlang-questions] Re: Mnesia Read/Write question. > > > > Hello, > > > > Maria, I'm not having problems with the B) version, only with > > A). (The error you noted was due to my misspelling it in the email). > > > > Could syntax in "A)" be deprecated? I got it from Mnesia A > > Distributed Robust DBMS for Telecommunications Applications, > > which I think was published in 98 or earlier. > > > > Regards, > > -Gene > > > > On Thu, Jul 30, 2009 at 11:04 PM, Steve Davis < > > steven.charles.davis@REDACTED> wrote: > > > > > Hello Gene, > > > > > > It looks like #test.id is the key, not #test.item. In the first > > > example, I suspect id is undefined. In the second it is. To find by > > > item, which appears to be what you are trying to do in (A), > > you would > > > have to use matches. > > > > > > /sd > > > > > > On Jul 30, 11:19 pm, "G.S." wrote: > > > > Hello everyone, > > > > This is probably a very stupid question/problem, but I'm having > > > > trouble using the following commands: > > > > > > > > A) mnesia:dirty_write(Id#test{item = hello}) and > > > > mnesia:dirty_read(Id#test.item) I get: exception error: > > > > {badrecord,test} but this works on the same table/schema...: > > > > B) Thing = #test{id = 1, item = hello}, > > > > mnesia:dirty_write(test,Thing), mnesia:dirty_read(test,Id). > > > > > > > > Are "A)" valid commands, what am I doing wrong? > > > > > > > > Regards, > > > > -Gene > > > > > > ________________________________________________________________ > > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > > erlang-questions (at) erlang.org > > > > > > > > > From Killy.Draw@REDACTED Fri Jul 31 14:34:42 2009 From: Killy.Draw@REDACTED (KDr2) Date: Fri, 31 Jul 2009 20:34:42 +0800 Subject: =?UTF-8?B?UmU6IFtlcmxhbmctY2hpbmE6MjU3M10gUmU6IFt6ZXV1eC1hbm5vdW5jZV0g5ZOy5oCd?= =?UTF-8?B?56S+5Yy677yIemV1dXguY29t77yJ5Y+R5biD6YCa5ZGK?= In-Reply-To: <82749060907310526j71c70d87ga300c124c9c9eb4@mail.gmail.com> References: <4A72BCE8.80205@zeuux.org> <9dad9f0a0907310255y30227a48n23022de833e42c15@mail.gmail.com> <82749060907310526j71c70d87ga300c124c9c9eb4@mail.gmail.com> Message-ID: ??????????????mail list?,??: erlang-bugs@REDACTED erlang-questions@REDACTED erlang-china@REDACTED lua@REDACTED perlchina@REDACTED python-list@REDACTED python-cn@REDACTED ruby-talk@REDACTED rubyonrails-talk@REDACTED pongba@REDACTED 2009/7/31 Rory Ye > ?K????group??? > > ?????? +1 > > 2009/7/31 KDr2 > > ?????? >> >> 2009/7/31 Zoom.Quiet >> >> 2009/7/31 Bill Xu : >>> > ???? >>> > >>> >>> zeuux.com ~ ????????????, ???????! >>> ???????? FLOSS ??/??/?? ????????; >>> ????,????????, ????FLOSS ??,??????? >>> >>> >>> > ?2006?12??????????????????????????????? >>> ???????????????????????????????????? >>> > ??????????????????????????????????? >>> > >>> > ??????????????zeuux.com?????????????????? >>> > ???????????????????????????????????? >>> ???????????????????????????????????? >>> > ???????????????????????????????????? >>> ???????????????????????????????????? >>> > ?????????????????????????????????? >>> > >>> > ?2009?6?12????????????????????????????? >>> > ????????zeuux.com?????????????????????? ???????bug?????????????????? >>> > >>> > ????????????????????????????????????? >>> ???????????????????????????????????? >>> > ?????????????????????????????????? http://www.zeuux.com?? >>> > >>> > ???????????????????????????????????? ?????? >>> > >>> > ??web2.0???????????beta?????????????? >>> > ?zeuux.com?????????????????????????????? >>> > ?bug????????????????????????????????? ?????????????????? >>> > >>> > ?????zeuux.com??? >>> > >>> > ????? >>> > >>> > >>> > ??? >>> > >>> > http://www.zeuux.com >>> > >>> > _______________________________________________ >>> > zeuux-announce mailing list >>> > zeuux-announce@REDACTED >>> > http://www.zeuux.org/mailman/listinfo/zeuux-announce >>> > >>> >>> >>> >>> -- >>> http://zoomquiet.org ????,Pythonic!-) >>> ?????(?????????)=??- ??- ??- ??- ?????? >>> >> >> >> >> -- >> Best Regards, >> -- KDr2, at x-macro.com. >> > > > > -- > My site:http://www.jdkcn.com > -- Best Regards, -- KDr2, at x-macro.com. From erlang@REDACTED Fri Jul 31 15:34:35 2009 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 31 Jul 2009 15:34:35 +0200 Subject: [erlang-questions] Re: Nested for-loops, there has to be a better way to do this In-Reply-To: References: <187b6a3f-498c-4281-bc30-0ad40a13c4c9@p36g2000prn.googlegroups.com> <9b08084c0907282335s7fc42eb5r8e925e9597dae34c@mail.gmail.com> Message-ID: <9b08084c0907310634q1d155fd2y3b34b79380ac7d9e@mail.gmail.com> I still don't understand. Erlang (or any functional programming languages) is made from lots of small functions. Each function has inputs and outputs and there is some relationship between the inputs and the output. To specify a function we must state what the inputs are, state what the outputs are and state what the relationship between the inputs and outputs are. The fact that you print something is irrelevant. You could just as well say that the output of a function is a list and that you then print the items in the list one at a time. I have tried to write a program that *exactly* follows your spec. Following the rules in your spec I conclude that if the input to your program is the list [4,2,1,1] then the output should be [4,2,1,5,4,3,2,1,1,3,2,1] Is this correct? Here's the program - read the comments to see how the code relates to the spec. -module(process). -compile(export_all). test() -> process([4,2,1,5]). process(In) -> Out = f(In), [io:format("~p~n", [I]) || I <- Out]. %% Now we have to write f %% The spec says this: %% Takes first int, processes it (in this case, prints it out), %% decrements it, takes the second int, processes it (prints it out), %% decrements it and on. When it gets to the very last item, it %% processes it until there are none left. Then goes to the one above %% it, processes it, decrements it... wash, rinse, repeat :) . %% What this mean ... lets' take it bit by bit. %% The first two lines say %% Takes first int, processes it (in this case, prints it out), %% decrements it, takes the second int, processes it (prints it out), %% What happens after the first int has been decremented? %% Now we have to "take the second int" wheres the first Int gone? %% It seems like it is pushed onto a stack, so we can deal with it later. %% The idea of a stack is reinforced by saying later "goes to the one above" %% So here's what I think the first two lines of the spec mean: f(L) -> process(L, [], []). process([Int|T], Stack, Print) -> %% Takes first int, processes it (in this case, prints it out), %% decrements it, takes the second int, processes it (prints it out), process(T, [Int-1|Stack], [Int|Print]); %% Now we get to the next major bit of the spec %% .. When it gets to the very last item, it %% .. processes it until there are none left. Then goes to the one above %% .. it, processes it, decrements it... wash, rinse, repeat :) . %% This is very confusing. The word "it" is referred to many times %% What is "it". %% "when it gets to the very last item, it process it until there are non left" %% But if it has got to the very last item there there is nothing to process %% because it is the last item. %% To resolve this we look at the next lines of the spec %% " When it gets to the very last item, it %% processes it until there are none left. Then goes to the one above %% it, processes it, decrements it... wash, rinse, repeat :) . %% I think this means this: process([], [H|Rest], Print) when H > 0-> process([], [H-1|Rest], [H|Print]); %% All this talk of decrementing things makes me wonder %% how the iteration will stop I assume it's when %% "it" reaches zero which is why I added the guard above. %% Let's add a base case process([], [0|Rest], Print) -> process([], Rest, Print); %% Finally when "it" is zero and "there is nothing above" process([], [], Print) -> lists:reverse(Print). %% end Incidentally if you had given a test case i.e that for the input [4,2,1,1] then the output should be [4,2,1,5,4,3,2,1,1,3,2,1] Then I would have immediately translated this into a unit test: test() -> [4,2,1,5,4,3,2,1,1,3,2,1] = f([4,2,1,1]). *before* writing the code Cheers /Joe On Fri, Jul 31, 2009 at 3:59 AM, Yves S. Garret wrote: > Takes first int, processes it (in this case, prints it out), > decrements it, takes the second int, processes it (prints it out), > decrements it and on. ?When it gets to the very last item, it > processes it until there are none left. ?Then goes to the one above > it, processes it, decrements it... wash, rinse, repeat :) . > > In retrospect, I could have made this clearer. > > > On Jul 29, 2:35?am, Joe Armstrong wrote: >> I don't understand what you want. >> >> If the input to your program is: [5,5,10] >> the output should be: X >> >> What is X? >> >> /Joe >> >> On Wed, Jul 29, 2009 at 4:55 AM, Yves S. >> >> >> >> >> >> Garret wrote: >> > I was playing around with the idea of implementing a nested for-loop >> > like construct (I will concede, they are evil, but at rare times, >> > necessary.) ?In non-functional programming languages, they are trivial >> > to do. ?However, this has proved to be one of those simple things that >> > are a pain in the neck for me. ?The idea was to be able to pass in a >> > list [5, 5, 10] (for example) and go through it all like 3 dimensional >> > array. ?The below illustration is far from elegant (it's almost 23:00 >> > where I live and need to go to work tomorrow early, so no time to come >> > up with something more sane :) ), but is there as an inquisitive >> > exercise. ?Now, in my opinion, this implementation sucks (first time >> > I've ever gave such a problem a shot in Erlang.) >> >> > Is there a better way to do this? ?Anyone would like to demonstrate >> > it? >> >> > Note: I've tested this code with the lists [2, 4] and [2, 4, 5]. >> >> > Compiled with: >> > Erlang (SMP,ASYNC_THREADS,HIPE) (BEAM) emulator version 5.6.5 >> >> > My code: >> > ====================================== >> > -module(loopWrapper). >> >> > -export([runLoop/1]). >> >> > runLoop(List) -> >> > ?[FirstVal | RestList] = List, >> > ?NextVal = FirstVal - 1, >> > ?if (length(RestList) > 0) and (FirstVal > 0) -> >> > ? ?io:format("~B~n~n", [FirstVal]), >> > ? ?runLoop(RestList), >> > ? ?io:format("~n~n"), >> > ? ?runLoop([NextVal | RestList]); >> > ?FirstVal > 0 -> >> > ? ?io:format("~B", [FirstVal]), >> > ? ?runLoop([NextVal]); >> > ?true -> >> > ? ?null >> > ?end. >> >> > ________________________________________________________________ >> > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html >> > erlang-questions (at) erlang.org >> >> ________________________________________________________________ >> erlang-questions mailing list. Seehttp://www.erlang.org/faq.html >> erlang-questions (at) erlang.org > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ovidiudeac@REDACTED Fri Jul 31 16:16:03 2009 From: ovidiudeac@REDACTED (Ovidiu Deac) Date: Fri, 31 Jul 2009 17:16:03 +0300 Subject: [erlang-questions] Slow UTP packets In-Reply-To: <8F48DC24-3F10-44C5-9033-F11276942013@hates.ms> References: <804cd4020907300529x262b052pb5013f31ad86087c@mail.gmail.com> <804cd4020907300623l55bd2c11v61f14c0bebb3ce14@mail.gmail.com> <8F48DC24-3F10-44C5-9033-F11276942013@hates.ms> Message-ID: <804cd4020907310716mbfaaa83g5b36c78f545742b0@mail.gmail.com> It makes sense but it doesn't look good. 1MB/s is not that much. ...and fast messaging between processes should be one of Erlang's strong points. Maybe I should use a passive socket instead. On Fri, Jul 31, 2009 at 6:02 AM, Mihai Balea wrote: > > On Jul 30, 2009, at 9:23 AM, Ovidiu Deac wrote: > >> But what is the explanation for the fact that the output slows during >> runtime? Is it a bug? > > One possible explanation would be accumulation of messages in your receiving > process' mailbox. > If you cannot consume messages as fast as they arrive, they will build up in > your mailbox and slow down the process. > > Due to the way the mailbox is implemented, it cannot cope well with massive > numbers of messages. I don't remember the details, but I believe it has to > do with using O(n) algorithms. Maybe somebody in the know can provide more > details. > > Mihai > > From mihai@REDACTED Fri Jul 31 16:33:49 2009 From: mihai@REDACTED (Mihai Balea) Date: Fri, 31 Jul 2009 10:33:49 -0400 Subject: [erlang-questions] Slow UTP packets In-Reply-To: <804cd4020907310716mbfaaa83g5b36c78f545742b0@mail.gmail.com> References: <804cd4020907300529x262b052pb5013f31ad86087c@mail.gmail.com> <804cd4020907300623l55bd2c11v61f14c0bebb3ce14@mail.gmail.com> <8F48DC24-3F10-44C5-9033-F11276942013@hates.ms> <804cd4020907310716mbfaaa83g5b36c78f545742b0@mail.gmail.com> Message-ID: <4E759EE4-C552-4DFA-820E-1E8CC8F5AAC3@hates.ms> On Jul 31, 2009, at 10:16 AM, Ovidiu Deac wrote: > It makes sense but it doesn't look good. 1MB/s is not that much. I think it's the number of messages that really matter, not the total size... > ...and fast messaging between processes should be one of Erlang's > strong points. > > Maybe I should use a passive socket instead. Using passive or {active, once} when dealing with sockets is always a good idea. However, while TCP will adjust the flow to accommodate your consumer's speed, UDP will simply just drop packets when you cannot process them fast enough. Just something to keep in mind. Mihai From ovidiudeac@REDACTED Fri Jul 31 16:47:42 2009 From: ovidiudeac@REDACTED (Ovidiu Deac) Date: Fri, 31 Jul 2009 17:47:42 +0300 Subject: how to break the problem. the erlang way? Message-ID: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> I'm doing some evaluation of erlang so I came with the following problem: "The application has to subscribe to a multicast address and receive binary packets. One packet per datagram. Each packet has a lenght, a sequence number and a number of messages inside. Packets have to be processed in their sequence number order. The messages have to be extracted from the packets and written in a file." I come with C++/Python experience and in OO approach I would have the following components: 1. A Receiver who connects to the multicast and receives the packets 2. An Orderer who's responsability is to order the packets by their sequence number and detect the missing ones. 3. An Unpacker who's responsability is to unpack the incomming packets and extract the messages. 4. A Decoder who does the deserialization of the messages 5. A Writer who puts the messages in the file. Now if I move all this to Erlang I would map the objects to processes. Instead of having objects with methods being called I have processes which receive messages. So the 5 components would run as separate processes. Each one does a little job and passes the result to the next one. Is this the Erlang way? Or is it just too much message passing overhead? From dave.pawson@REDACTED Fri Jul 31 16:56:09 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Fri, 31 Jul 2009 15:56:09 +0100 Subject: [erlang-questions] how to break the problem. the erlang way? In-Reply-To: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> References: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> Message-ID: <711a73df0907310756x1642bbbbjb4e921a1e745d9d9@mail.gmail.com> I'll await the answer to this one! The design (problem breakdown) is quit new to me. 2009/7/31 Ovidiu Deac : > I come with C++/Python experience and in OO approach I would have the > following components: > 1. A Receiver who connects to the multicast and receives the packets > 2. An Orderer who's responsability is to order the packets by their > sequence number and detect the missing ones. > 3. An Unpacker who's responsability is to unpack the incomming packets > and extract the messages. > 4. A Decoder who does the deserialization of the messages > 5. A Writer who puts the messages in the file. > > Now if I move all this to Erlang I would map the objects to processes. But still have the same breakdown? That's where I hesitated. Surely there's something different here? Perhaps 'get a packet' process, since it's easy to have lots of them? Extract messages sounds clean, dealing with finished messages. Less sure on the remainder. Anyone? regards -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From bengt.kleberg@REDACTED Fri Jul 31 16:57:42 2009 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 31 Jul 2009 16:57:42 +0200 Subject: [erlang-questions] how to break the problem. the erlang way? In-Reply-To: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> References: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> Message-ID: <1249052262.4681.20.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, A normal Erlang application can have thousands of processes, so 5 does not seem to be too many. If you write 5 processes, and time them, you can then start to move functionality (say 3, 4, 5) to a single process and measure if that makes things go faster. bengt On Fri, 2009-07-31 at 17:47 +0300, Ovidiu Deac wrote: > I'm doing some evaluation of erlang so I came with the following problem: > > "The application has to subscribe to a multicast address and receive > binary packets. One packet per datagram. Each packet has a lenght, a > sequence number and a number of messages inside. Packets have to be > processed in their sequence number order. The messages have to be > extracted from the packets and written in a file." > > I come with C++/Python experience and in OO approach I would have the > following components: > 1. A Receiver who connects to the multicast and receives the packets > 2. An Orderer who's responsability is to order the packets by their > sequence number and detect the missing ones. > 3. An Unpacker who's responsability is to unpack the incomming packets > and extract the messages. > 4. A Decoder who does the deserialization of the messages > 5. A Writer who puts the messages in the file. > > Now if I move all this to Erlang I would map the objects to processes. > Instead of having objects with methods being called I have processes > which receive messages. So the 5 components would run as separate > processes. Each one does a little job and passes the result to the > next one. > > Is this the Erlang way? Or is it just too much message passing overhead? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From rapsey@REDACTED Fri Jul 31 16:58:33 2009 From: rapsey@REDACTED (Rapsey) Date: Fri, 31 Jul 2009 16:58:33 +0200 Subject: [erlang-questions] how to break the problem. the erlang way? In-Reply-To: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> References: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> Message-ID: <97619b170907310758s6c1afd11v5048b86375832b29@mail.gmail.com> Yes something like that, though I'm sure you would not need 5 processes. Process tasks do not need to be so finely grained, but can be if you really need the flexibility. Sergej On Fri, Jul 31, 2009 at 4:47 PM, Ovidiu Deac wrote: > I'm doing some evaluation of erlang so I came with the following problem: > > "The application has to subscribe to a multicast address and receive > binary packets. One packet per datagram. Each packet has a lenght, a > sequence number and a number of messages inside. Packets have to be > processed in their sequence number order. The messages have to be > extracted from the packets and written in a file." > > I come with C++/Python experience and in OO approach I would have the > following components: > 1. A Receiver who connects to the multicast and receives the packets > 2. An Orderer who's responsability is to order the packets by their > sequence number and detect the missing ones. > 3. An Unpacker who's responsability is to unpack the incomming packets > and extract the messages. > 4. A Decoder who does the deserialization of the messages > 5. A Writer who puts the messages in the file. > > Now if I move all this to Erlang I would map the objects to processes. > Instead of having objects with methods being called I have processes > which receive messages. So the 5 components would run as separate > processes. Each one does a little job and passes the result to the > next one. > > Is this the Erlang way? Or is it just too much message passing overhead? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ovidiudeac@REDACTED Fri Jul 31 17:04:39 2009 From: ovidiudeac@REDACTED (Ovidiu Deac) Date: Fri, 31 Jul 2009 18:04:39 +0300 Subject: [erlang-questions] Slow UTP packets In-Reply-To: <4E759EE4-C552-4DFA-820E-1E8CC8F5AAC3@hates.ms> References: <804cd4020907300529x262b052pb5013f31ad86087c@mail.gmail.com> <804cd4020907300623l55bd2c11v61f14c0bebb3ce14@mail.gmail.com> <8F48DC24-3F10-44C5-9033-F11276942013@hates.ms> <804cd4020907310716mbfaaa83g5b36c78f545742b0@mail.gmail.com> <4E759EE4-C552-4DFA-820E-1E8CC8F5AAC3@hates.ms> Message-ID: <804cd4020907310804w3cee1424id85404d931705a56@mail.gmail.com> On Fri, Jul 31, 2009 at 5:33 PM, Mihai Balea wrote: > > On Jul 31, 2009, at 10:16 AM, Ovidiu Deac wrote: > >> It makes sense but it doesn't look good. 1MB/s is not that much. > > I think it's the number of messages that really matter, not the total > size... The average size of the packet is 150B. That means about 7000messages/s. It shouldn't be that much. > > >> ...and fast messaging between processes should be one of Erlang's strong >> points. >> >> Maybe I should use a passive socket instead. > > Using passive or {active, once} when dealing with sockets is always a good > idea. However, while TCP will adjust the flow to accommodate your consumer's > speed, UDP will simply just drop packets when you cannot process them fast > enough. > Just something to keep in mind. > > Mihai > Losing UDP packets I don't mind that much as having an application which slows down under load. :) From mihai@REDACTED Fri Jul 31 17:26:19 2009 From: mihai@REDACTED (Mihai Balea) Date: Fri, 31 Jul 2009 11:26:19 -0400 Subject: [erlang-questions] Slow UTP packets In-Reply-To: <804cd4020907310804w3cee1424id85404d931705a56@mail.gmail.com> References: <804cd4020907300529x262b052pb5013f31ad86087c@mail.gmail.com> <804cd4020907300623l55bd2c11v61f14c0bebb3ce14@mail.gmail.com> <8F48DC24-3F10-44C5-9033-F11276942013@hates.ms> <804cd4020907310716mbfaaa83g5b36c78f545742b0@mail.gmail.com> <4E759EE4-C552-4DFA-820E-1E8CC8F5AAC3@hates.ms> <804cd4020907310804w3cee1424id85404d931705a56@mail.gmail.com> Message-ID: On Jul 31, 2009, at 11:04 AM, Ovidiu Deac wrote: > The average size of the packet is 150B. That means about > 7000messages/s. It shouldn't be that much. I can see why that would be a problem. While passing 7000 messages per second should not be a problem, if you cannot consume them fast enough, they can accumulate _really_ fast and cause the symptoms you observed. From mihai@REDACTED Fri Jul 31 17:41:04 2009 From: mihai@REDACTED (Mihai Balea) Date: Fri, 31 Jul 2009 11:41:04 -0400 Subject: [erlang-questions] how to break the problem. the erlang way? In-Reply-To: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> References: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> Message-ID: Your approach would be subject to flow mismatches between producers and consumers, which could lead to message accumulation somewhere in your chain. You can use synchronous calls (like in gen_server:call), but then you serialize your processing and in that case you might as well use just one process. Depending on how heavy is your message unpacking and deserializing, you might want to do something like: Receiver -> spawn a process for each packet that unpacks/deserializes it -> Orderer/Writer That, of course, assumes packets can be processed independently. One advantage of having transient processes is that you can tune them to never need garbage collection, which is not the case for long lived processes. Anyways, you should always profile your code and see what works best for you Mihai On Jul 31, 2009, at 10:47 AM, Ovidiu Deac wrote: > I'm doing some evaluation of erlang so I came with the following > problem: > > "The application has to subscribe to a multicast address and receive > binary packets. One packet per datagram. Each packet has a lenght, a > sequence number and a number of messages inside. Packets have to be > processed in their sequence number order. The messages have to be > extracted from the packets and written in a file." > > I come with C++/Python experience and in OO approach I would have the > following components: > 1. A Receiver who connects to the multicast and receives the packets > 2. An Orderer who's responsability is to order the packets by their > sequence number and detect the missing ones. > 3. An Unpacker who's responsability is to unpack the incomming packets > and extract the messages. > 4. A Decoder who does the deserialization of the messages > 5. A Writer who puts the messages in the file. > > Now if I move all this to Erlang I would map the objects to processes. > Instead of having objects with methods being called I have processes > which receive messages. So the 5 components would run as separate > processes. Each one does a little job and passes the result to the > next one. > > Is this the Erlang way? Or is it just too much message passing > overhead? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org From essiene@REDACTED Fri Jul 31 20:40:25 2009 From: essiene@REDACTED (Essien Essien) Date: Fri, 31 Jul 2009 19:40:25 +0100 Subject: [erlang-questions] how to break the problem. the erlang way? In-Reply-To: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> References: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> Message-ID: <88b82c90907311140l5ee1e336o98ee6e4dd92d69ac@mail.gmail.com> Hi Ovidiu, On Fri, Jul 31, 2009 at 3:47 PM, Ovidiu Deac wrote: > I'm doing some evaluation of erlang so I came with the following problem: > > "The application has to subscribe to a multicast address and receive > binary packets. One packet per datagram. Each packet has a lenght, a > sequence number and a number of messages inside. Packets have to be > processed in their sequence number order. The messages have to be > extracted from the packets and written in a file." > > I come with C++/Python experience and in OO approach I would have the > following components: > 1. A Receiver who connects to the multicast and receives the packets > 2. An Orderer who's responsability is to order the packets by their > sequence number and detect the missing ones. > 3. An Unpacker who's responsability is to unpack the incomming packets > and extract the messages. > 4. A Decoder who does the deserialization of the messages > 5. A Writer who puts the messages in the file. > > Now if I move all this to Erlang I would map the objects to processes. > Instead of having objects with methods being called I have processes > which receive messages. So the 5 components would run as separate > processes. Each one does a little job and passes the result to the > next one. >From my own limited experience, the OO thought process can not be matched 1 to 1 to functional programming languages or even a process centric approach like erlang. My own personal rule of thumbs since after getting over the rush that is spawn(), and spawn_link() in erlang, is to remember that nothing has changed, we just have a "cheaper" way of creating "threads" (this is not 100% true, but its one way to look at it). A question to ask yourself above is, would any of the classes you would create in Python/C++ benefit from being threads, or seperate os level processes? if yes, then I think you've found a candidate component to be an erlang process, no... then... still no in erlang... if well... maybe but not so sure... i suggest you leave it till you realize it will work better as a process. There's nothing wrong with just making function calls in erlang... everything does not have to be a process for it to be the Erlang way. > > Is this the Erlang way? Or is it just too much message passing overhead? I'm definitely not qualified to be telling you of "The Erlang Way (tm)" ;) so this is my own personal reasoning after writing a decent bit of erlang in a space of nearly a year. Hey... but what do I know... I just started learning last year, so take it with a grain of salt... heck... swallow a bag of salt while at that :) > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ovidiudeac@REDACTED Fri Jul 31 22:02:58 2009 From: ovidiudeac@REDACTED (Ovidiu Deac) Date: Fri, 31 Jul 2009 23:02:58 +0300 Subject: [erlang-questions] how to break the problem. the erlang way? In-Reply-To: <4d08db370907310948p53f37cccrec892cff10d864a5@mail.gmail.com> References: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> <4d08db370907310948p53f37cccrec892cff10d864a5@mail.gmail.com> Message-ID: <804cd4020907311302w124a879m8a5a4838625b5017@mail.gmail.com> On Fri, Jul 31, 2009 at 7:48 PM, Hynek Vychodil wrote: > > > On Fri, Jul 31, 2009 at 4:47 PM, Ovidiu Deac wrote: >> >> I'm doing some evaluation of erlang so I came with the following problem: >> >> "The application has to subscribe to a multicast address and receive >> binary packets. One packet per datagram. Each packet has a lenght, a >> sequence number and a number of messages inside. Packets have to be >> processed in their sequence number order. The messages have to be >> extracted from the packets and written in a file." >> >> I come with C++/Python experience and in OO approach I would have the >> following components: >> 1. A Receiver who connects to the multicast and receives the packets >> 2. An Orderer who's responsability is to order the packets by their >> sequence number and detect the missing ones. >> 3. An Unpacker who's responsability is to unpack the incomming packets >> and extract the messages. >> 4. A Decoder who does the deserialization of the messages >> 5. A Writer who puts the messages in the file. >> >> Now if I move all this to Erlang I would map the objects to processes. >> Instead of having objects with methods being called I have processes >> which receive messages. So the 5 components would run as separate >> processes. Each one does a little job and passes the result to the >> next one. >> >> Is this the Erlang way? Or is it just too much message passing overhead? >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > First think what I usually do is determine what id data transformation and > what is process. They are different thinks. One process can do several data > transformations and also one data transformation can be done by several > processes in parallel. From your points only 1. na 5. seems as pure > processes for me. Point 2. can be achieved as process which can make thinks > easier but 3. and 4. are pure data transformations. In Erlang for data > transformations just write ordinal modules with pure functions. For > processes you also have in OTP tools how write most of their functionality > in call back modules. My initial design would be this: > > One process as receiver (1.) which receives packets and sends to another > processes which I would spawn for CPU intensive tasks for unpacking (3.) a > decoding (4.) of messages for workload of one package. Results will be send > to orderer/cache (2.) which serialise messages in right order for writting > to disk. File writer is process as usual in Erlang. Formating for file > format I would do in same process as unpacking and decoding. I can't see > reason why I should do it as last thing ;-) > > Receiver (1.) -> many proceses each do (3.), (4.), (5. formating) for one > incomming packet -> Order for write (2.) -> Write (5.) (But it can be just > "file" process or RAW file and then there will not be any other process.) > > (1.), (2.) gen_server > (3.), (4.), (5. formating) just plain modules with pure functions called > inside normal Erlang process. > There can be dispatcher/supervisor process for those but it is not necessary > and it will complicate things. > > Above is design form maximal thourgput. If you have some expectation of flow > control, reliability and other it will make it more complicate it too. > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill your > boss. ?Be a data hero! > Try Good Data now for free: www.gooddata.com > Thanks to everybody for the answers. I like this suggestion very much. Now that I think of it I can see it's basically a map-reduce approach i.e. brake the work in small units, spawn processes for each unit which do the processing and then collect the results. ...and if we consider scalability, this approach doesn't care about a fixed number of processing units so it should scale very well to any number of cores/processors available. Thanks again, ovidiu From ovidiudeac@REDACTED Fri Jul 31 22:09:54 2009 From: ovidiudeac@REDACTED (Ovidiu Deac) Date: Fri, 31 Jul 2009 23:09:54 +0300 Subject: [erlang-questions] how to break the problem. the erlang way? In-Reply-To: References: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> Message-ID: <804cd4020907311309g1bd41732v1bc3fc9f59ec5445@mail.gmail.com> On Fri, Jul 31, 2009 at 6:41 PM, Mihai Balea wrote: > Your approach would be subject to flow mismatches between producers and > consumers, which could lead to message accumulation somewhere in your chain. > You can use synchronous calls (like in gen_server:call), but then you > serialize your processing and in that case you might as well use just one > process. > > Depending on how heavy is your message unpacking and deserializing, you > might want to do something like: > > Receiver -> spawn a process for each packet that unpacks/deserializes it -> > Orderer/Writer > > That, of course, assumes packets can be processed independently. I totally agree that this approach is better then my initial one. > > One advantage of having transient processes is that you can tune them to > never need garbage collection, which is not the case for long lived > processes. Can you detail this part? I don't think I understand what you mean by "tune them to never need garbage collection" but it sounds good :) thanks, ovidiu From baryluk@REDACTED Fri Jul 31 22:27:51 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Fri, 31 Jul 2009 22:27:51 +0200 Subject: [erlang-questions] how to break the problem. the erlang way? In-Reply-To: <804cd4020907311309g1bd41732v1bc3fc9f59ec5445@mail.gmail.com> References: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> <804cd4020907311309g1bd41732v1bc3fc9f59ec5445@mail.gmail.com> Message-ID: <1249072071.29722.4.camel@sredniczarny> Dnia 2009-07-31, pi? o godzinie 23:09 +0300, Ovidiu Deac pisze: > On Fri, Jul 31, 2009 at 6:41 PM, Mihai Balea wrote: h > > > > One advantage of having transient processes is that you can tune them to > > never need garbage collection, which is not the case for long lived > > processes. > > Can you detail this part? I don't think I understand what you mean by > "tune them to never need garbage collection" but it sounds good :) I think He mean that if spawned process will do its job quickly (and it will not trigger garbage collection due to small usage of own heap and stack), all memory will be freed after its terminate. So it will still reclaim its memory, but it will be trivial and ultra fast. -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From mihai@REDACTED Fri Jul 31 22:39:21 2009 From: mihai@REDACTED (Mihai Balea) Date: Fri, 31 Jul 2009 16:39:21 -0400 Subject: [erlang-questions] how to break the problem. the erlang way? In-Reply-To: <1249072071.29722.4.camel@sredniczarny> References: <804cd4020907310747w21f4567djb3797798064d8520@mail.gmail.com> <804cd4020907311309g1bd41732v1bc3fc9f59ec5445@mail.gmail.com> <1249072071.29722.4.camel@sredniczarny> Message-ID: On Jul 31, 2009, at 4:27 PM, Witold Baryluk wrote: > Dnia 2009-07-31, pi? o godzinie 23:09 +0300, Ovidiu Deac pisze: >> On Fri, Jul 31, 2009 at 6:41 PM, Mihai Balea wrote: > h >>> >>> One advantage of having transient processes is that you can tune >>> them to >>> never need garbage collection, which is not the case for long lived >>> processes. >> >> Can you detail this part? I don't think I understand what you mean by >> "tune them to never need garbage collection" but it sounds good :) > > I think He mean that if spawned process will do its job quickly > (and it will not trigger garbage collection due to small usage of own > heap and stack), all memory will be freed after its terminate. > So it will still reclaim its memory, but it will be trivial and > ultra fast. Yes, that is gist of it. GC in Erlang is done on a per-process basis - each process has its own heap that is managed and collected independently. If the process is sufficiently short lived, it will terminate before GC is triggered and the entire process memory is reclaimed in one very fast step. By "fine-tuning" I was referring to the ability to set the initial process heap, when you spawn the process. Mihai